thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 19:02:04 +00:00
Code Issues Packages Projects Releases Wiki Activity
58,476 Commits 9,326 Branches 1,783 Tags 2.1 GiB
e7948819f9
Commit Graph

1367 Commits

Author SHA1 Message Date
alesapin
932ea0ef8b Merge branch 'master' into complete_zk_api 2020-11-10 13:56:11 +03:00
feng lv
9194985d48 Merge branch 'master' of github.com:ClickHouse/ClickHouse into union-distinct 2020-11-10 06:09:39 +00:00
Alexander Kuzmenkov
0530c40cd8 fixes 2020-11-10 08:50:32 +03:00
a.palagashvili
f999ea2a6f renamed files, added new library, changed error codes, added tests for file() function 2020-11-10 01:52:22 +03:00
Alexander Tokmakov
5cdfcfb307 remove other stringstreams 2020-11-09 22:12:44 +03:00
alesapin
09fac0da9d Better 2020-11-09 21:16:02 +03:00
alesapin
45b0894252 Some fixes 2020-11-09 20:50:39 +03:00
a.palagashvili
6286775031 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into issue-8828 2020-11-09 19:53:20 +03:00
Alexander Kuzmenkov
26229ed231 tmp spans for threads 
(doesn't compile because of json metadata changes)
 2020-11-09 18:07:38 +03:00
alesapin
4e846d2197 Merge branch 'master' into add_separate_pool_for_fetches 2020-11-09 10:00:21 +03:00
alesapin
456b0b94c1 Remove some debug 2020-11-09 09:54:35 +03:00
vdimir
a67f5b780f
Use sorted ip array instead of trie in TrieDictionary 2020-11-08 19:19:04 +03:00
alexey-milovidov
2d427eb1e8
Merge pull request #16496 from ClickHouse/register-formats 
Fix inconsistency in FormatFactory
 2020-11-08 14:27:33 +03:00
Alexey Milovidov
dba57529f8 Merge branch 'master' of github.com:yandex/ClickHouse into fix-stringstream 2020-11-07 22:03:38 +03:00
Alexey Milovidov
941ee1a214 Fix error 2020-11-07 22:02:22 +03:00
Alexey Milovidov
80fb30c06d Fix error 2020-11-07 19:09:15 +03:00
Alexey Milovidov
8b8383d375 Fix error 2020-11-07 19:08:19 +03:00
Alexey Milovidov
c3973123bd Fix error 2020-11-07 19:03:23 +03:00
a.palagashvili
268f289bd1 resolved conflict 2020-11-07 16:36:38 +03:00
feng lv
1ca7dcb0a8 fix conflict 2020-11-07 11:44:29 +00:00
alexey-milovidov
494c188559
Merge pull request #16757 from ClickHouse/tavplubix-patch-3 
Get rid of possible memory allocation in ThreadPool::worker(...)
 2020-11-07 13:40:31 +03:00
Alexey Milovidov
fd84d16387 Fix "server failed to start" error 2020-11-07 03:14:53 +03:00
Alexey Milovidov
57de99bcd9 More simple 2020-11-06 20:46:43 +03:00
Alexey Milovidov
ad52fceb83 Fix bad code in SmallObjectPool 2020-11-06 20:34:14 +03:00
tavplubix
044f143463
Update ThreadPool.cpp 2020-11-06 18:53:33 +03:00
hexiaoting
28e87da976 remove map literal and Merge remote-tracking branch 'origin/master' into dev_map2 2020-11-06 15:07:41 +08:00
alexey-milovidov
6d5e852356
Merge pull request #16677 from ClickHouse/field-infinite-convert-to-int 
Fix UBSan report when trying to convert infinite floating point number to integer
 2020-11-05 23:20:25 +03:00
Alexey Milovidov
b56486510f Improve Arcadia 2020-11-05 12:55:01 +03:00
Alexey Milovidov
65aa226bcd Fix test 2020-11-05 12:26:51 +03:00
Alexander Kuzmenkov
2a0d9da5e4 cleanup 2020-11-05 12:25:42 +03:00
Alexander Kuzmenkov
faf1f1e0cd Merge remote-tracking branch 'origin/master' into HEAD 2020-11-05 11:09:11 +03:00
alesapin
a9529e8d65 Bad code but all events 2020-11-04 21:54:55 +03:00
alesapin
e2bce14aed Merge branch 'master' into complete_zk_api 2020-11-04 20:37:30 +03:00
Alexey Milovidov
22954cc123 Fix UBSan report when trying to convert infinite floating point number to integer 2020-11-04 19:48:36 +03:00
a.palagashvili
ceda5cb9c5 fix codestyle, resolvec conflict 2020-11-04 19:47:04 +03:00
Alexey Milovidov
ff8f80b6b6 Remove old file 2020-11-04 13:34:47 +03:00
Alexey Milovidov
f7c77b4a25 Fix unit tests 2020-11-04 13:14:23 +03:00
alexey-milovidov
4fede5b008
Merge pull request #16438 from azat/errorCode-v2 
Add errorCodeToName() and system.errors
 2020-11-04 12:40:47 +03:00
alexey-milovidov
973c1d7983
Merge pull request #16529 from azat/log_queries_min_time 
Add log_queries_min_query_duration_ms
 2020-11-04 01:47:18 +03:00
alexey-milovidov
8a3f38f332
Merge branch 'master' into errorCode-v2 2020-11-04 01:40:41 +03:00
alesapin
598532b582 Remove strange method 2020-11-03 18:01:56 +03:00
alesapin
5700e5e46d Fix response serialization 2020-11-03 17:57:40 +03:00
alesapin
93c2ad6034 Something work 2020-11-03 17:49:30 +03:00
Alexander Tokmakov
d8ae9fcdb4 fixes, add shard name 2020-11-03 14:30:53 +03:00
alesapin
a02b4322b6 Merge branch 'master' into complete_zk_api 2020-11-03 13:13:07 +03:00
Alexander Kuzmenkov
f10a5207f4
Merge pull request #16601 from ClickHouse/ban_aggregate_function_in_primary_key 
Restrict to use non-comparable types in primary key
 2020-11-02 23:27:46 +03:00
Azat Khuzhin
3be8a56f5c Add log_queries_min_query_duration_ms 
Only queries slower then the value of this setting will go to system.query_log,
i.e. something like slow_query_log in mysql.

v2: log_queries_min_time renamed to log_queries_min_query_duration_ms
v3: add current_database into system.query_thread_log
v4: rewrite test using current_database
v5: fix query_duration_ms in system.query_thread_log
 2020-11-02 21:34:54 +03:00
alexey-milovidov
bfa0ddcdfe
Merge pull request #16596 from ClickHouse/strange-cpp 
Add strange C++ code that actually does not change anything
 2020-11-02 20:02:28 +03:00
feng lv
4d722bc52c fix conflict 2020-11-02 12:52:12 +00:00
a.palagashvili
805bfd286c resolved git modules 2020-11-02 15:16:49 +03:00
alesapin
f3b3025719 Ban non comparable types in primary key 2020-11-02 14:39:27 +03:00
Alexey Milovidov
67daa37a7f Add strange C++ code that actually does not change anything 2020-11-02 13:42:47 +03:00
hexiaoting
66bca89001 Style error fix 2020-11-02 17:23:02 +08:00
alesapin
8b0d9fbc07 Merge branch 'no_background_pool_no_more' into add_separate_pool_for_fetches 2020-11-02 10:54:09 +03:00
hexiaoting
3a70cf3773 Merge remote-tracking branch 'origin/master' into dev_map2 2020-11-02 14:46:56 +08:00
hexiaoting
2ba68d7494 Add map function 2020-11-02 14:05:53 +08:00
a.palagashvili
be2b002ff4 fixed cmake for building, added test for compression method, added basic code for encoding 2020-11-01 21:40:05 +03:00
Azat Khuzhin
5363c62fc5 Fix overflow check in ErrorCodes 2020-11-01 09:00:19 +03:00
Azat Khuzhin
c8f756707f Swap name and value in ErrorCodes definition 2020-10-31 11:32:44 +03:00
Azat Khuzhin
fff6b5549e Hide getCurrentTimeNanoseconds() to avoid possible errors 
To avoid further errors like in:
- c0e15ba348 ("Fix RealTimeMicroseconds ProfileEvents")
- c856d02e0f4a519cd2a29ffbe1135196de0bc65a ("Fix query_thread_log.query_duration_ms unit")
 2020-10-31 11:15:27 +03:00
alexey-milovidov
022648a154
Merge pull request #16558 from azat/query_thread_log_current_database 
Add current_database into query_thread_log
 2020-10-31 06:36:10 +03:00
alesapin
1fc7633289 Heartbeats working 2020-10-30 22:57:30 +03:00
Azat Khuzhin
5bab7a5bc7 Add current_database into query_thread_log 2020-10-30 21:16:10 +03:00
alesapin
c2525ef211 Server and client pinging each other 2020-10-30 17:16:47 +03:00
Azat Khuzhin
3661769e1b Remove some absolute changes for errorCodeToName/system.errors 2020-10-30 09:04:22 +03:00
Azat Khuzhin
0cd79de632 Eliminate extra strlen() in errorCodeToName() 2020-10-29 21:53:09 +03:00
Azat Khuzhin
cb50886ef1 Fix ErrorCodes increment on error 2020-10-29 21:24:52 +03:00
Azat Khuzhin
04bff595d3 Make check-style happy (by not using DB::ErrorCodes:: in comments) 2020-10-29 21:10:20 +03:00
Azat Khuzhin
7e79769ae4 Fix readability-inconsistent-declaration-parameter-name in ErrorCodes.h 2020-10-29 21:08:09 +03:00
Maxim Akhmedov
3627fabfb9 Remove -g0 form Arcadia build settings. 2020-10-29 17:37:23 +03:00
alesapin
5e2a3d12d7 Split requests and responses part from zookeeper 2020-10-29 17:25:44 +03:00
Azat Khuzhin
b42f77a791 Covert error codes list to lookup table and drop generation of them. 2020-10-29 10:55:41 +03:00
Azat Khuzhin
b2e2322895 Add system.errors table 
Contains error codes with number of times they have been triggered.

Columns:

-   `name` ([String](../../sql-reference/data-types/string.md)) — name of the error (`errorCodeToName`).
-   `code` ([Int32](../../sql-reference/data-types/int-uint.md)) — code number of the error.
-   `value` ([UInt64](../../sql-reference/data-types/int-uint.md)) - number of times this error has been happened.

**Example**

``` sql
SELECT *
FROM system.errors
WHERE value > 0
ORDER BY code ASC
LIMIT 1

┌─name─────────────┬─code─┬─value─┐
│ CANNOT_OPEN_FILE │   76 │     1 │
└──────────────────┴──────┴───────┘
 2020-10-29 10:55:38 +03:00
alesapin
0931658c2f Merge branch 'master' into add_separate_pool_for_fetches 2020-10-28 14:20:56 +03:00
alesapin
8a6be1602b
Merge pull request #16364 from ClickHouse/fix_uuid_mapping_exists 
Minor improvements in DatabaseCatalog
 2020-10-28 10:54:39 +03:00
Mikhail Filimonov
41971e073a
Fix typos reported by codespell 2020-10-27 12:04:03 +01:00
alexey-milovidov
a591d1f32a
Update ThreadStatus.cpp 2020-10-26 22:46:25 +03:00
feng lv
9b95ed9ead fix 2020-10-26 12:21:04 +00:00
Alexander Tokmakov
636da1318b Merge branch 'master' into database_replicated 2020-10-26 15:00:43 +03:00
alesapin
77d350aaeb Bigger setting value 2020-10-26 14:23:59 +03:00
Alexey Milovidov
0ba2fa8105 Fix "Arcadia" 2020-10-26 11:56:30 +03:00
Alexey Milovidov
88e846b36d Fix build on Mac OS 2020-10-26 07:42:22 +03:00
Alexey Milovidov
068235c2d7 Disable alt stack for sanitizers 2020-10-26 05:18:36 +03:00
Alexander Tokmakov
f8f2d62d70 minor improvements in DatabaseCatalog 2020-10-25 20:27:21 +03:00
Alexey Milovidov
4e2c0ba226 Provide diagnostics on stack overflow 2020-10-25 03:10:05 +03:00
alexey-milovidov
e00f6c4a0d
Merge branch 'master' into total_memory_tracker-by-default 2020-10-24 21:17:53 +03:00
feng lv
15e4e03469 add union distinct and setting union_default_mode 2020-10-24 13:18:04 +00:00
Azat Khuzhin
114ea9b1eb Fix accounting for new/delete from different threads for VariableContext::Thread 
MemoryTracker assumes that for VariableContext::Thread new/delete may be
called from different threads, hence the amount of memory can go
negative.

However the MemoryTracker is nested, so even if the negative amount is
allowed for VariableContext::Thread it does not allowed for anything
upper, and hence the MemoryTracking will not be decremented properly.

Fix this, by passing initial size to the parent free.

This should fix memory drift for HTTP queries.
 2020-10-23 21:07:52 +03:00
Azat Khuzhin
72d7b6117e Use total_memory_tracker when there is no other MemoryTracker object. 
This should significantly reduce the MemoryTracking drift, test shows
that there is 0 drift after query storm (100 queries, via http/tcp/tcp
in one session).

TL;DR;

To track memory, clickhouse creates memory tracker object for each
thread **explicitly**, but until it is not created the memory
allocations are not under account.
There should not be lot of allocations w/o memory tracker, since most of
the time it is created early enough, but even this maybe enough to
trigger some problems.

Plus sometimes it is not possible to create it, for example some 3d
party library does not allow to do this explicitly:
- for example before #15740 allocations from librdkafka threads,
- or even worse, poco threads, they don't have any routines to do this.
This won't be a problem for `MemoryTracking` metric if the deallocation
will be done from the same thread w/o memory tracker (or vise versa),
but this is not always true.

NOTE, that this will slow down per-thread allocations w/o memory
tracker, since before this patch there were no memory tracking for them
while now they will be accounted in total_memory_tracker, and for
total_memory_tracker max_untracked_memory is always reached.
But this should not be significant.
 2020-10-23 21:07:52 +03:00
hexiaoting
483be134b2 Support MergeTree Engine 2020-10-23 16:36:17 +08:00
Amos Bird
ccf59c6412
Fix definitely wrong bug 2020-10-22 22:56:45 +08:00
alexey-milovidov
2ae32e3d5c
Merge pull request #16101 from azat/http-memory-tracking-fixes 
Fix memory tracker settings (for HTTP queries)
 2020-10-22 00:00:47 +03:00
Alexey Milovidov
2619efadc8 Fix multiple issues with memory tracking 2020-10-21 03:31:34 +03:00
Alexander Tokmakov
72dbb43235 Merge branch 'master' into fork-valbaturin-replication-db-engine 2020-10-20 19:14:54 +03:00
Vladimir Chebotarev
aa5f207fd4
Added disable_merges option for volumes in multi-disk configuration (#13956) 
Co-authored-by: Alexander Kazakov <Akazz@users.noreply.github.com>
 2020-10-20 18:10:24 +03:00
Alexey Milovidov
448c87363c Style 2020-10-20 04:29:34 +03:00
Alexander Kuzmenkov
defcd29b31
Merge pull request #16036 from ClickHouse/stress_test_improvemets 
Minor improvements of stress test
 2020-10-19 16:12:30 +03:00
hexiaoting
fcd35b0619 Merge branch 'master' into dev_map2 2020-10-18 16:18:13 +08:00
hexiaoting
db8ba39c9c Implement Integer And Array for value type 2020-10-17 23:45:35 +08:00
Azat Khuzhin
082f30a3a1 Introduce ThreadStatus::applyQuerySettings() to reduce copy-paste 2020-10-17 15:01:53 +03:00
Alexander Tokmakov
026896835c fixes 2020-10-16 15:20:30 +03:00
Alexander Tokmakov
95ad45a3b0 minor improvements of stress test 2020-10-15 21:14:04 +03:00
Nikolai Kochetov
939d8edf33
Merge pull request #15999 from ClickHouse/fix-after-15437 
Continuation of 15437
 2020-10-15 18:20:48 +03:00
Vitaly Baranov
5f261cec3c
Merge pull request #15199 from filimonov/non_delimited_protobuf 
ProtobufSingle format
 2020-10-15 16:58:39 +03:00
Nikolai Kochetov
963ce315b2
Merge pull request #15454 from azat/dict-prealloc 
Speedup hashed/sparse_hashed dictionary loading by preallocating the hash table
 2020-10-15 15:03:53 +03:00
Nikolai Kochetov
40f62719ff Move event to stack. 2020-10-15 13:24:18 +03:00
Vitaly Baranov
997570780d
Merge branch 'master' into non_delimited_protobuf 2020-10-14 16:26:57 +03:00
Anton Popov
7470ac98bb
Merge pull request #15855 from hexiaoting/dev_fetch 
Add Support for OFFSET_FETCH_CLAUSE
 2020-10-14 15:21:28 +03:00
hexiaoting
cdb4dd57a6 Add new DataType Map(key,value) 2020-10-14 14:35:14 +08:00
hexiaoting
dea965832a Fix error for previous offset test 2020-10-13 11:32:43 +08:00
Azat Khuzhin
2947eff878 Fix query hang with connections_with_failover_max_tries = 0 (endless loop) 2020-10-12 22:07:26 +03:00
Mikhail Filimonov
87a8ba63f7
Few more CR fixes 2020-10-12 15:05:40 +02:00
alexey-milovidov
10e9d14466
Merge pull request #15818 from ClickHouse/style-pragma-once 
Check for #pragma once in headers
 2020-10-11 13:14:09 +03:00
alexey-milovidov
39e58128cd
Merge pull request #15812 from ClickHouse/bigint-hash 
Minor changes in BigInt hash
 2020-10-11 01:11:30 +03:00
Alexey Milovidov
269b6383f5 Check for #pragma once in headers 2020-10-10 21:37:02 +03:00
Alexey Milovidov
9f15daf45c Remove useless metrics to avoid confusion 2020-10-10 18:52:44 +03:00
Alexey Milovidov
a2ef0f6cba Minor changes in BigInt hash 2020-10-10 18:46:41 +03:00
Alexey Milovidov
4f2d4988c6 Fix build on Mac OS 2020-10-10 13:51:57 +03:00
Alexey Milovidov
2c71327ff5 Fix build on Mac OS 2020-10-10 11:02:02 +03:00
Azat Khuzhin
f4f79aa84a Add HashTable::reserve() (for preallocated hashed/sparse_hashed dictionaries) 2020-10-09 21:53:57 +03:00
Azat Khuzhin
553f6aa4cd Remove unused functions 2020-10-07 21:30:53 +03:00
alexey-milovidov
97d97f6b2e
Merge pull request #15513 from ClickHouse/aku/error-context 
Add some context to an error message
 2020-10-03 16:24:19 +03:00
alexey-milovidov
ae61f8f7a6
Update filesystemHelpers.cpp 2020-10-03 15:57:35 +03:00
alexey-milovidov
4d6402525a
Update filesystemHelpers.h 2020-10-03 15:57:12 +03:00
Alexey Milovidov
d426ba3074 Step 4: cleanup 2020-10-03 03:07:55 +03:00
Alexey Milovidov
b6ca8a0676 Step 3: check other usages, add a comment 2020-10-03 03:05:54 +03:00
Alexey Milovidov
2441be4b66 Step 2: move code to .cpp 2020-10-03 03:02:47 +03:00
Alexey Milovidov
6608de470d Step 1: handle EINTR in statvfs 2020-10-03 03:01:32 +03:00
Alexander Kuzmenkov
3be7c7745e Use a separate overload for formatting with fmt 2020-10-02 14:06:38 +03:00
Alexander Kuzmenkov
6e07bfe7bc return the comment 2020-10-01 17:29:22 +03:00
Alexander Kuzmenkov
c5d6e2e67c Add context to error messages 2020-10-01 14:02:53 +03:00
Nikolai Kochetov
45618c9d4c
Merge pull request #15437 from ClickHouse/azat-ThreadPool-UAF-fixes 
Destroy resources captured by lambda after `ThreadFromGlobalPool::join()`
 2020-09-30 16:50:30 +03:00
alesapin
01d8a36cd6
Merge pull request #15456 from ClickHouse/more-exception-safety-in-shellcommand 
Fix exception safety in ShellCommand
 2020-09-30 12:56:55 +03:00
Nikolai Kochetov
0e412686bc Better comments. 2020-09-30 11:25:22 +03:00
alexey-milovidov
412a54e356
Merge pull request #14693 from hustnn/zk_exist_fix2 
Fix potential memory leak caused by zookeeper exist calling
 2020-09-30 04:48:11 +03:00
Alexey Milovidov
78dbf10c7b Fix exception safety in ShellCommand 2020-09-30 03:00:45 +03:00
Nikolai Kochetov
727289cb09 Destroy resurces captured by lambda after ThreadFromGlobalPool::join(). 2020-09-29 23:43:02 +03:00
Nikolai Kochetov
a2c70c4611 Destroy resurces captured by lambda after ThreadFromGlobalPool::join(). 2020-09-29 22:11:40 +03:00
Nikolai Kochetov
a7e7fd084f Destroy resurces captured by lambda after ThreadFromGlobalPool::join(). 2020-09-29 13:48:11 +03:00
Azat Khuzhin
789434994a Reset job on failure too in ThreadPool::worker() 
job should be reseted before decrementing scheduled_jobs to
ensure that the Job destroyed before wait() returns.
 2020-09-29 10:56:36 +03:00
tavplubix
4578d89819
Merge pull request #15348 from ClickHouse/replicate_default_args 
Continuation of #14791
 2020-09-28 18:53:02 +03:00
tavplubix
421d3b0b86
Merge pull request #14760 from bharatnc/ncb/event_time_microsec_p2 
add event_time_microseconds field for system.{trace,text,query,query_thread}_log tables
 2020-09-28 18:34:21 +03:00
hustnn
a7024134dd Merge remote-tracking branch 'upstream/master' into zk_exist_fix2 2020-09-28 14:44:55 +08:00
Alexander Tokmakov
3bc8d9daa9 fix sync 2020-09-27 14:10:45 +03:00
Alexander Tokmakov
34addcf61f add review suggestions 2020-09-26 22:18:28 +03:00
alexey-milovidov
bbbe51033d
Merge pull request #15321 from ClickHouse/query-obfuscator 
Query obfuscator
 2020-09-26 19:43:36 +03:00
Alexey Milovidov
5e146eda04 Fix error 2020-09-26 06:02:05 +03:00
Alexey Milovidov
62c2c2000c Query obfuscator: development 2020-09-26 03:00:54 +03:00
bharatnc
24ea6b2380 TraceCollector - time s&us using same timespec 2020-09-25 10:39:37 -07:00
bharatnc
b0d4fe5114 TraceLog - add field event_time_microseconds 2020-09-25 10:39:37 -07:00
tavplubix
b96157e32b
Merge branch 'master' into enable_atomic_database_by_default 2020-09-25 13:42:39 +03:00
alexey-milovidov
47d150b116
Revert "Revert "Avoid deadlocks in Log/TinyLog"" 2020-09-25 02:29:16 +03:00
alexey-milovidov
13529265c4
Revert "Avoid deadlocks in Log/TinyLog" 2020-09-25 02:28:57 +03:00
Alexander Tokmakov
fb826557b7 Merge branch 'master' into enable_atomic_database_by_default 2020-09-25 02:04:01 +03:00
Alexey Milovidov
dce6a436f3 Fix strange code 2020-09-24 19:35:17 +03:00
Vitaly Baranov
0605808ed1
Merge pull request #14492 from vitlibar/use-previous-users-xml-if-failed-to-reload 
Use previous users.xml if failed to reload
 2020-09-24 16:25:19 +03:00
Alexander Kuzmenkov
813a876e93
Merge pull request #14962 from ClickHouse/log-avoid-deadlock 
Avoid deadlocks in Log/TinyLog
 2020-09-24 13:54:11 +03:00
Alexander Tokmakov
f0f91f84af Merge branch 'master' into enable_atomic_database_by_default 2020-09-23 21:50:15 +03:00
Alexander Tokmakov
a8266a3d3c fix DDDL with cross-replication and Atomic 2020-09-23 21:28:59 +03:00
Alexander Kuzmenkov
679c0988a5 Merge remote-tracking branch 'origin/master' into HEAD 2020-09-22 16:25:52 +03:00
Alexander Kuzmenkov
dde4cf70e1 Merge remote-tracking branch 'origin/master' into HEAD 2020-09-22 14:03:59 +03:00
Alexander Kuzmenkov
478c7309d4
Merge pull request #14843 from ClickHouse/aku/global-pool 
Exception on double init of global thread pool
 2020-09-22 13:44:39 +03:00
Alexander Kuzmenkov
c0241b617a Merge remote-tracking branch 'origin/master' into HEAD 2020-09-22 13:14:31 +03:00
hustnn
3a59a7d5b6 Remove root path before adding to watches 2020-09-22 09:28:13 +08:00
Niu Zhaojie
76e0147bf4 Use opt num to identify request type 2020-09-22 09:28:12 +08:00
hustnn
35f1db0043 Fix potential memory leak caused by zookeeper exist calling 2020-09-22 09:28:12 +08:00
Alexey Milovidov
1e65105b06 Merge branch 'master' into log-avoid-deadlock 2020-09-21 21:09:06 +03:00
alexey-milovidov
e29c4c3cc4
Merge pull request #15024 from ClickHouse/fix_renameat2_einval_on_macos 
Fix EINVAL in renameat2 on MacOS
 2020-09-21 21:08:17 +03:00
alexey-milovidov
10d3a1d730
Update ThreadPool.cpp 2020-09-21 20:49:15 +03:00
tavplubix
fd2cc343e6
Update renameat2.cpp 2020-09-21 15:37:34 +03:00
tavplubix
450c1a5709
Update renameat2.cpp 2020-09-21 13:19:02 +03:00
Alexander Tokmakov
ef85ce90c3 fix build 2020-09-21 13:17:33 +03:00
Azat Khuzhin
30f96dc525 Fix possible UAF from jobs in the ThreadPool on shutdown 
ThreadPoolImpl<>::worker signaling that job is done while still storing
std::function<> object, and this can lead to problems on shutdown, since
in in this cast std::function<> can refers to some global/static object
that had been already destroyed (typical example is Logger).

I believe that this is exactly what TSAN reports about (decoded
manually, since llvm-symbolizer does not work in the test env):

<details>

```

- 2020-09-20 17:44:43   Write of size 8 at 0x7b1000008f78 by main thread (mutexes: write M1432):
    operator delete(void*, unsigned long)
    ??:0:0

    Poco::Logger::~Logger()
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Logger.cpp:39:1
    non-virtual thunk to Poco::Logger::~Logger()
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Logger.cpp:0:0

    Poco::RefCountedObject::release() const
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/RefCountedObject.h:82:24
    Poco::Logger::shutdown()
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Logger.cpp:346:16

    Poco::AutoLoggerShutdown::~AutoLoggerShutdown()
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Logger.cpp:459:4

    cxa_at_exit_wrapper(void*)
    ??:0:0

```

```
- 2020-09-20 17:44:43   Previous atomic read of size 4 at 0x7b1000008f78 by thread T116:

    __tsan_atomic32_load
    ??:0:0

    int std::__1::__cxx_atomic_load<int>(std::__1::__cxx_atomic_base_impl<int> const*, std::__1::memory_order)
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/atomic:970:12
    std::__1::__atomic_base<int, false>::load(std::__1::memory_order) const
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/atomic:1487:17
    std::__1::__atomic_base<int, false>::operator int() const
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/atomic:1491:53
    Poco::Logger::is(int) const
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/Logger.h:1264:9
    MemoryTracker::logPeakMemoryUsage() const
    /build/obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:59:5

    MemoryTracker::~MemoryTracker()
    /build/obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:46:13

    DB::ThreadGroupStatus::~ThreadGroupStatus()
    /build/obj-x86_64-linux-gnu/../src/Common/ThreadStatus.h:51:7
    std::__1::__shared_ptr_emplace<DB::ThreadGroupStatus, std::__1::allocator<DB::ThreadGroupStatus> >::__on_zero_shared()
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3602:23

    std::__1::__shared_count::__release_shared()
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3440:9
    std::__1::__shared_weak_count::__release_shared()
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3482:27
    std::__1::shared_ptr<DB::ThreadGroupStatus>::~shared_ptr()
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:4207:19
    DB::PipelineExecutor::executeImpl(unsigned long)::$_4::~$_4()
    /build/obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:720:34
    ThreadFromGlobalPool::ThreadFromGlobalPool<DB::PipelineExecutor::executeImpl(unsigned long)::$_4>(DB::PipelineExecutor::executeImpl(unsigned long)::$_4&&)::'lambda'()::~()
    /build/obj-x86_64-linux-gnu/../src/Common/ThreadPool.h:161:54
    std::__1::__compressed_pair_elem<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::PipelineExecutor::executeImpl(unsigned long)::$_4>(DB::PipelineExecutor::executeImpl(unsigned long)::$_4&&)::'lambda'(), 0, false>::~__compressed_pair_elem()
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2188:8
    std::__1::__function::__alloc_func<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::PipelineExecutor::executeImpl(unsigned long)::$_4>(DB::PipelineExecutor::executeImpl(unsigned long)::$_4&&)::'lambda'(), std::__1::allocator<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::Pip>
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:1559:38
    std::__1::__function::__func<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::PipelineExecutor::executeImpl(unsigned long)::$_4>(DB::PipelineExecutor::executeImpl(unsigned long)::$_4&&)::'lambda'(), std::__1::allocator<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::PipelineE>
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:1706:10

    std::__1::__function::__value_func<void ()>::~__value_func()
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:1828:19
    std::__1::function<void ()>::~function()
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2460:43
    ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>)
    /build/obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:268:5

    void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()::operator()() const
    /build/obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:116:73
    decltype(std::__1::forward<void>(fp)(std::__1::forward<void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(fp0)...)) std::__1::__invoke<void ThreadPoolImpl<std::__1::thread>::scheduleI>
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3519:1
    void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'(>
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/thread:273:5
    void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned lon>
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/thread:284:5

    __tsan_thread_start_func
    ??:0:0
```

```
- 2020-09-20 17:44:43   Mutex M1432 (0x0000181213a8) created at:
    pthread_mutex_init
    ??:0:0

    Poco::MutexImpl::MutexImpl()
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Mutex_POSIX.cpp:64:6

    Poco::Mutex::Mutex()
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Mutex.cpp:34:8

    __cxx_global_var_init
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Logger.cpp:28:15
    _GLOBAL__sub_I_Logger.cpp
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Logger.cpp:0:0

    __libc_csu_init
    ??:0:0
```

</details>
 2020-09-20 23:46:28 +03:00
alexey-milovidov
7b35a6a59a
Merge pull request #15028 from ClickHouse/fix-bar 
Fix buffer overflow in "bar" function
 2020-09-20 19:04:15 +03:00
alexey-milovidov
1474b80ecc
Merge pull request #15008 from ClickHouse/clang-tidy-11-without-enabling 
Fix terribly wrong code
 2020-09-20 02:00:54 +03:00
Alexey Milovidov
dae3b40a55 Fix buffer overflow in "bar" function 2020-09-19 22:15:16 +03:00
Alexey Milovidov
059646743b Remove OpenCL 2020-09-19 17:20:00 +03:00
Alexander Tokmakov
46aa307d0a fix EINVAL in renameat2 on MacOS 2020-09-19 17:19:06 +03:00
Vitaly Baranov
33efb36714 Use previous users.xml if failed to reload. 2020-09-19 16:21:12 +03:00
Alexey Milovidov
4d06bc62b1 Allow concurrent reads while writing to TinyLog 2020-09-19 14:02:11 +03:00
alexey-milovidov
c461d782e4
Merge pull request #14929 from ClickHouse/mlock-code-segment 
Only mlock code segment
 2020-09-19 13:46:49 +03:00
Alexey Milovidov
ef5363b87f Fix terribly wrong code 2020-09-19 02:05:13 +03:00
Alexey Milovidov
7d0f3db639 Fix annoying "Arcadia" 2020-09-18 23:19:41 +03:00
Alexey Milovidov
8a8e4f8a41 Attempt to speed-up clang build 2020-09-18 21:58:57 +03:00
Alexey Milovidov
6e6909f47d Fix "Arcadia" 2020-09-18 21:54:54 +03:00
Alexey Milovidov
72c2201287 Fix "Arcadia" 2020-09-18 17:39:33 +03:00
Alexander Kuzmenkov
a9b2bff8ba Merge remote-tracking branch 'origin/master' into HEAD 2020-09-18 13:43:29 +03:00
Alexey Milovidov
1752bbc1ed Fix "Arcadia" build 2020-09-18 01:29:15 +03:00
Alexey Milovidov
a783fe110f Fix build 2020-09-17 21:15:19 +03:00
Alexey Milovidov
eb523f1a7c Merge branch 'master' into mlock-code-segment 2020-09-17 21:15:09 +03:00
Alexey Milovidov
20d07ed405 Update comment 2020-09-17 21:04:11 +03:00
Alexander Kuzmenkov
fb64cf210a straighten the protocol version 2020-09-17 17:37:29 +03:00
Vitaly Baranov
3356d75b23
Merge pull request #13156 from azat/cluster-secure 
Secure inter-cluster query execution (with initial_user as current query user) [v3]
 2020-09-17 17:11:00 +03:00
alexey-milovidov
496df5b3e9
Merge pull request #14678 from sundy-li/patch-2 
dynamic zookeeper config when session expired
 2020-09-17 17:05:22 +03:00
Alexey Milovidov
068e8576b7 Corrections 2020-09-17 15:53:52 +03:00
Alexey Milovidov
8bce20076c Only mlock code segment 2020-09-17 15:39:37 +03:00
Nikolai Kochetov
959d755a1b
Update Visitor.h 2020-09-17 15:08:55 +03:00
Nikolai Kochetov
225b85624e Merge branch 'master' into inline-gather-utils 2020-09-17 11:17:32 +03:00
sundy-li
544b2cb20d add configChanged method for zookeeper 
fix logic error && skip reload testkeeper
 2020-09-17 13:33:45 +08:00
Alexander Kuzmenkov
bcea99f2e5
Update ThreadPool.cpp 2020-09-16 11:59:58 +03:00
Alexander Kuzmenkov
5afb19faf1
Update ThreadPool.cpp 2020-09-15 19:58:09 +03:00
Alexander Kuzmenkov
a792850ecd
Update ThreadPool.cpp 2020-09-15 18:05:42 +03:00
alexey-milovidov
018f596d21
Merge pull request #14685 from ClickHouse/remap-executable 
Attempt to make performance test more reliable
 2020-09-15 15:09:47 +03:00
Alexander Kuzmenkov
d943bac1a4 Exception on double init of global thread pool 2020-09-15 13:30:16 +03:00
Artem Zuikov
51ba12c2c3
Try speedup build (#14809) 2020-09-15 12:55:57 +03:00
Nikolai Kochetov
95ae1105d9 Remove virtual destructors for Visitor. 2020-09-15 02:05:43 +03:00
Azat Khuzhin
785d1b2a75 OpenSSLHelpers cleanup 
Add few more specializations for encodeSHA256():
- std::string encodeSHA256(const std::string_view &);
- std::string encodeSHA256(const void *, size_t);
- void encodeSHA256(const void *, size_t, unsigned char *);
 2020-09-15 01:36:28 +03:00
Nikolai Kochetov
65a1057aae Revert "Try simplify Visitor." 
This reverts commit 9fcfa560ae.
 2020-09-15 00:40:41 +03:00
Nikolai Kochetov
9fcfa560ae Try simplify Visitor. 2020-09-14 23:35:45 +03:00
Nikita Mikhaylov
ddea62211d
Merge pull request #11948 from CurtizJ/fsync-parts 
Experiment. Allow to turn on fsync on inserts, merges and fetches.
 2020-09-14 22:37:14 +04:00
Alexey Milovidov
3530f83c01 Remove debug output 2020-09-14 21:12:09 +03:00
Alexey Milovidov
f9361acb88 Disable in shared libraries mode 2020-09-14 21:00:45 +03:00
Alexey Milovidov
0e73b8acf3 Merge branch 'master' into remap-executable 2020-09-14 20:26:16 +03:00
Alexey Milovidov
b966697939 Style 2020-09-14 16:31:30 +03:00
alexey-milovidov
cc3d1a3922
Merge pull request #14471 from ClickHouse/git-to-clickhouse 
Added a script to import git repository to ClickHouse
 2020-09-12 03:56:31 +03:00
Anton Popov
f2a5216e97 add waiting for fsync in WAL 2020-09-11 02:28:50 +03:00
Anton Popov
af4089c606 Merge remote-tracking branch 'upstream/master' into HEAD 2020-09-11 00:39:21 +03:00
Alexey Milovidov
5675efbd47 Fix build 2020-09-10 20:16:12 +03:00
Alexander Tokmakov
f19f02028b fix misleading exception message about uuid macro 2020-09-10 17:50:10 +03:00
Alexey Milovidov
65e9c678f8 Disable under certain conditions 2020-09-10 17:47:02 +03:00
Alexey Milovidov
532d121100 Fix typo 2020-09-10 12:14:31 +03:00
Alexey Milovidov
a644733139 Attempt to make performance test more reliable 2020-09-10 12:05:57 +03:00
alexey-milovidov
e65e29d537
Merge pull request #14647 from fastio/master 
Add QueryMemoryLimitExceeded event
 2020-09-10 06:12:39 +03:00
Nikita Mikhaylov
44726c37c3
Merge branch 'master' into fix-executable-dictionary-source-hangup 2020-09-09 23:01:41 +04:00
alexey-milovidov
453914b6b8
Update ProfileEvents.cpp 2020-09-09 21:02:53 +03:00
Peng Jian
9fa04cf48b Add QueryMemoryLimitExceeded event 2020-09-10 00:59:38 +08:00
Vasily Nemkov
3973a17530
MySql datatypes dateTime64 and decimal (#11512) 2020-09-09 15:18:02 +03:00
Artem Zuikov
36972d34a7
Fix logical error in GLOBAL JOIN with table function (#14545) 2020-09-09 13:03:13 +03:00
Ilya Yatsishin
a3d6732662
Merge pull request #14558 from bharatnc/ncb/improve-config-path-processing 
use std::filesystem::path in ConfigProcessor for filepath concatenation
 2020-09-08 14:39:23 +03:00
Nikita Mikhaylov
80f193f615
Merge pull request #14252 from bharatnc/ncb/time-query-log-in-ms 
add query_start_time_microseconds field to system.query_log & system.query_thread_log tables
 2020-09-08 15:11:50 +04:00
Ilya Yatsishin
11a247d2f4
Merge pull request #14556 from ClickHouse/fix-arcadia-2 
Lower binary size in "Arcadia" build
 2020-09-08 12:08:35 +03:00
alesapin
4364bff3bc
Merge pull request #12761 from kssenii/rabbitmq-improvements 
RabbitMQ improvements
 2020-09-08 09:38:25 +03:00
Alexey Milovidov
e3924b8057 Fix "Arcadia" 2020-09-08 01:14:13 +03:00
bharatnc
7b89adb10d use std::filesystem::path in ConfigProcessor 
This PR improves the way the ClickHouse config path,
preprocessed_configs path etc are being joined inside
`ConfigProcessor.cpp`.

Uses `std::filesystem::path` to join the paths instead of
standard string concatenation. This would improve path concatenation.
Current method of concatenation results in:

```bash
➜  clickhouse-server ls -latr
total 68
-rw-r--r--   1 root root  5587 Aug 31 05:00 users.xml
drwxr-xr-x 143 root root 12288 Sep  7 11:23 ..
lrwxrwxrwx   1 root root    41 Sep  7 11:23 preprocessed -> /var/lib/clickhouse//preprocessed_configs
drwxr-xr-x   2 root root  4096 Sep  7 11:23 users.d
-rw-r--r--   1 root root 34252 Sep  7 11:28 config.xml
drwxr-xr-x   4 root root  4096 Sep  7 11:28 .
drwxr-xr-x   2 root root  4096 Sep  7 11:28 config.d
```

The symlink `/var/lib/clickhouse//preprocessed_configs` will still work.
amd this PR is an improvement to the current method of path
concatenation.
 2020-09-07 12:45:52 -07:00
alexey-milovidov
e1104135c8
Merge pull request #13939 from fastio/master 
Multithreading implementation of Kafka engine
 2020-09-07 20:12:06 +03:00
kssenii
40c8290dde Tiny fixes, better tests 2020-09-07 14:48:50 +00:00
alesapin
17650e803f Tiny improvements 2020-09-07 14:08:53 +03:00
Alexey Milovidov
04a6965006 Fix error with executable dictionary source 2020-09-07 09:40:01 +03:00
Alexey Milovidov
1400bdbf83 Fix unit tests 2020-09-07 06:11:35 +03:00
Alexey Milovidov
83d2eafd12 Merge branch 'master' of github.com:yandex/ClickHouse into git-to-clickhouse 2020-09-07 03:19:44 +03:00
bharatnc
fd22450a2a Merge remote-tracking branch 'upstream/master' into ncb/time-query-log-in-ms 2020-09-06 13:19:27 -07:00
alexey-milovidov
a78b04b56a
Merge pull request #14455 from ClickHouse/prepare-clang-11 
Prepare for clang 11
 2020-09-06 11:02:58 +03:00
Peng Jian
e6bfd9d586 1. Add new setting for Kafka engine, named kafka_thread_per_consumer which default value is false. 2. Create separate thread pool for Kafka engine. 2020-09-06 11:17:19 +08:00
Anton Popov
3b9be70c8c Merge remote-tracking branch 'upstream/master' into HEAD 2020-09-05 18:53:55 +03:00
bharatnc
44e168bea7 query_start_time_microseconds field in system.query_log 
Add a new field called `query_start_time_microseconds`  that will add
`microseconds`. This new field will be of type `DateTime64`.
 2020-09-04 09:42:18 -07:00
Artem Zuikov
7c20aa2c62
Another 256-bit integers (strict 32 bytes) (#14229) 2020-09-04 16:33:02 +03:00
alexey-milovidov
54ede2e1bb
Merge pull request #14467 from ClickHouse/topk-check-overflow 
Check for array size overflow in topK
 2020-09-04 09:15:33 +03:00
Alexey Milovidov
d3fa22280a Merge branch 'prepare-clang-11' into git-to-clickhouse 2020-09-04 09:12:27 +03:00
Alexey Milovidov
23b9677879 Added a script to import git repository to ClickHouse 2020-09-04 08:46:58 +03:00
Alexey Milovidov
1cee6d5a31 Check for array size overflow in topK #14452 2020-09-04 04:05:57 +03:00
Alexey Milovidov
4b3220f79a Prepare for clang 11 2020-09-04 03:39:56 +03:00
Amos Bird
3817c0efa7
Remove redundant conditions 2020-09-04 02:13:57 +08:00
Anton Popov
4fe57fc169 Merge remote-tracking branch 'upstream/master' into HEAD 2020-09-02 01:29:44 +03:00
Artem Zuikov
d304d522cc
Refactoring: rename types to allow change bigint library (#14219) 2020-09-01 12:54:50 +03:00
alexey-milovidov
0f706c01ca
Merge pull request #13888 from vladimir-golovchenko/add-date-trunc-function 
Added date_trunc function
 2020-08-27 02:12:27 +03:00
alexey-milovidov
7271c9c983
Merge pull request #14059 from azat/gcc10-fixes 
gcc10 build fixes
 2020-08-26 03:31:03 +03:00
vladimir golovchenko
b0dd1d92b4 Added date_trunc function. 2020-08-25 16:41:08 -07:00
Azat Khuzhin
9071457d3f Fix C++20 comparison calls recursively with reversed arguments in UInt128.h (over.match.oper#3.4.4 in gcc10) 
Due to [1], gcc10 reports:

    ../src/Common/UInt128.h:92:81: error: in C++20 this comparison calls the current function recursively with reversed arguments [-Werror]
       92 | template <typename T> bool inline operator == (T a, const UInt128 b) { return b == a; }

  [1]: http://eel.is/c++draft/over.match.oper#3.4.4
 2020-08-25 21:19:57 +03:00
Alexander Tokmakov
59d879b1fe fix "no space left" extra info for Poco::Exception 2020-08-25 18:06:24 +03:00
Anton Popov
4df5c72878 Merge remote-tracking branch 'upstream/master' into HEAD 2020-08-24 12:57:27 +03:00
Vasily Nemkov
f94f786cc3 First attempt to fix data race in ConnectionPoolWithFailover::getStatus() 2020-08-20 23:25:38 +03:00
Artem Zuikov
becc186c91
Add support for extended precision integers and decimals (#13097) 2020-08-19 14:52:17 +03:00
Alexey Milovidov
5a00941303 Fix "Arcadia" 2020-08-19 12:31:28 +03:00
Alexey Milovidov
5c0a9648fb Remove even more useless code 2020-08-18 22:03:23 +03:00
Alexey Milovidov
eeb769d2d4 Remove useless code around zkutil 2020-08-18 22:02:07 +03:00
Artem Zuikov
bc6ddac52c one more unroll for arerage.cpp 2020-08-17 19:23:05 +03:00
Alexey Milovidov
09846a92cd Fix typos 2020-08-16 15:09:41 +03:00
alexey-milovidov
751cce6dfc
Update gtest_thread_pool_schedule_exception.cpp 2020-08-16 11:10:36 +03:00
Alexey Milovidov
1b929ec54a Fix flaky unit test 2020-08-16 11:07:36 +03:00
Alexey Milovidov
696f773267 Remove ZooKeeper from unit tests 2020-08-15 07:29:02 +03:00
Alexey Milovidov
52cf85eaee Fix clang build 2020-08-13 23:13:13 +03:00
tavplubix
4aff48dc3f
Merge pull request #10851 from zhang2014/feature/materialize_mysql_database 
ISSUES-4006 support MaterializeMySQL database engine
 2020-08-12 15:59:28 +03:00
Alexander Kuzmenkov
d42dbfce98 Do not overallocate when creating PODArray with given size 
Another part of #12278
 2020-08-10 23:36:52 +03:00
Alexey Milovidov
8054a08c55 Fix build after merge 2020-08-10 03:44:21 +03:00
Vitaly Baranov
2e6ba2a05d
Merge pull request #13496 from vitlibar/compound-identifiers-for-custom-settings 
Support compound identifiers for custom settings.
 2020-08-10 01:03:02 +03:00
alexey-milovidov
180ea39bf8
Merge pull request #13099 from ClickHouse/fixed-hash-map-implicit-zero-6 
Slightly improve performance of aggregation by UInt8/UInt16 keys, part 6
 2020-08-09 23:13:53 +03:00
Vitaly Baranov
2c5a2b2f43 Add function isValidIdentifier(). 2020-08-09 02:46:42 +03:00
Alexey Milovidov
182e2929f3 Merge branch 'master' into embed-configs 2020-08-09 02:41:44 +03:00
Alexey Milovidov
a2d9be1ed4 Merge branch 'fixed-hash-map-implicit-zero-6' of github.com:yandex/ClickHouse into fixed-hash-map-implicit-zero-6 2020-08-09 01:50:41 +03:00
Alexey Milovidov
d2351dc7f3 Fix clang static analyzer 2020-08-09 01:50:20 +03:00
Alexey Milovidov
350b8e87a7 Merge branch 'master' into fixed-hash-map-implicit-zero-6 2020-08-09 01:48:12 +03:00
alexey-milovidov
8e3f8aa0ab
Merge pull request #13491 from ClickHouse/aku/podarray 
Avoid overallocation in PODArray assignment
 2020-08-09 01:41:47 +03:00
alexey-milovidov
beed3c8244
Merge pull request #13500 from ClickHouse/hardening-and-better-error-messages 
Hardening and better error messages
 2020-08-08 17:13:43 +03:00
alexey-milovidov
c0fef71507
Merge pull request #13511 from ClickHouse/codespell 
Fix typos in code with codespell.
 2020-08-08 17:09:16 +03:00
Alexey Milovidov
0f79eb3cc5 Embed configs into binary 2020-08-08 06:42:42 +03:00
alexey-milovidov
321d4ed643
Merge pull request #13490 from zlobober/patch-4 
Demangle exception classname for std::exception
 2020-08-08 05:10:39 +03:00
Alexey Milovidov
12f66fa82c Fix 99% of typos 2020-08-08 04:01:47 +03:00
Alexey Milovidov
edd89a8610 Fix half of typos 2020-08-08 03:47:03 +03:00
alexey-milovidov
378b2b7e04
Update average.cpp 2020-08-08 02:06:55 +03:00
Alexander Kuzmenkov
ddf0fe30b3
Merge pull request #13336 from ianton-ru/select_insert_time 
Add QueryTimeMicroseconds, SelectQueryTimeMicroseconds and InsertQueryTimeMicroseconds
 2020-08-08 01:05:46 +03:00
Alexey Milovidov
b6625006dc Hardening and better error messages 2020-08-07 22:36:04 +03:00
Alexander Kuzmenkov
c06210a89a Avoid overallocation in PODArray assignment 
Part of #12438
 2020-08-07 18:18:32 +03:00
Max Akhmedov
026cb4ac1b
Demangle exception classname for std::exception 2020-08-07 17:52:32 +03:00
Alexander Kuzmenkov
5d60ab33a5
Merge pull request #13147 from ClickHouse/aku/podarray 
Assert that the source type is the same before memcpy in PODArray::insert
 2020-08-07 16:58:14 +03:00
alexey-milovidov
b5667e3b0f
Merge pull request #13441 from ClickHouse/fix-parse-date-time-best-effort 
Fix error in parseDateTimeBestEffort
 2020-08-07 14:02:59 +03:00
Alexey Milovidov
5c1703bd6c Merge branch 'master' into fixed-hash-map-implicit-zero-6 2020-08-07 02:58:57 +03:00
Alexey Milovidov
241017bc1f Fix error in parseDateTimeBestEffort 2020-08-07 02:52:18 +03:00
Alexander Kuzmenkov
da7bc8962d
Merge pull request #13417 from vladimir-golovchenko/fix-quota-statements-interval-formatting 
Fix a 'Week'-interval formatting for ATTACH/ALTER/CREATE QUOTA-statements
 2020-08-07 01:50:25 +03:00
Alexander Kuzmenkov
e7e71b8615 Use the memcpy predicate from 6f8a274ba0 2020-08-06 16:34:23 +03:00
Alexander Kuzmenkov
9409629511 Merge remote-tracking branch 'origin/master' into HEAD 2020-08-06 16:29:43 +03:00
vladimir golovchenko
3c307064b9 Fix a 'Week'-interval formatting for ATTACH/ALTER/CREATE QUOTA-statements. 2020-08-06 05:45:01 -07:00
Alexey Milovidov
8fd4038234 Fix UBSan error 2020-08-06 04:59:02 +03:00
Alexander Kuzmenkov
ead62bc9d7 Remove everything except static_assert for same type 2020-08-05 17:12:41 +03:00
Alexander Kuzmenkov
aa629a4c82 Merge remote-tracking branch 'origin/master' into HEAD 2020-08-05 17:09:56 +03:00
Alexander Kuzmenkov
0eaab3d095
Merge pull request #13191 from ClickHouse/aku/semicolon-cleanup 
Cleanup for #12999. NFC
 2020-08-05 14:53:27 +03:00
alexey-milovidov
85f28db714
Merge pull request #11524 from ClickHouse/stack-frams-size-limit 
Avoid too large stack frames
 2020-08-05 13:12:08 +03:00
Alexander Kuzmenkov
0d947b64bd fixup 2020-08-05 02:23:31 +03:00
Anton Ivashkin
327090b7ea Add QueryTimeMicroseconds, SelectQueryTimeMicroseconds and InsertQueryTimeMicroseconds to system.events, second attempt 2020-08-04 19:38:13 +03:00
Alexey Milovidov
d91471ea00 Merge branch 'master' into stack-frams-size-limit 2020-08-04 15:28:47 +03:00
zhang2014
c99da34039 ISSUES-4006 try fix unit test failure 2020-08-04 19:09:30 +08:00
Azat Khuzhin
d28c4c4a6c Merge remote-tracking branch 'upstream/master' into roundUpToPowerOfTwoOrZero-safety 
* upstream/master:
  fix left asof join with join_use_nulls (#13291)
  Revert "Add QueryTimeMicroseconds, SelectQueryTimeMicroseconds and InsertQuer…"
  Add test
  Added another check just in case
  Fixup
  Fix range checks in h3 functions
  Fix configs
  Update tests
  Check ALTERs; update test
  Sanity checks for MergeTreeSettings
 2020-08-04 09:29:46 +03:00
zhang2014
da2ad0799f ISSUES-4006 try fix build failure after merge master 2020-08-04 10:31:27 +08:00
zhang2014
7ea2eee98a ISSUES-4006 add some check & add comment 2020-08-04 10:31:27 +08:00
zhang2014
c173bd391f ISSUES-4006 try fix build & test failure 2020-08-04 10:31:27 +08:00
zhang2014
6b452cf83d ISSUES-4006 try fix build failure & fix float 32 parser 2020-08-04 10:31:27 +08:00
zhang2014
74f354d840 ISSUES-4006 add some check before create materialize mysql database 2020-08-04 10:31:27 +08:00
zhang2014
4e161bb42a ISSUES-4006 remove unused metric 2020-08-04 10:31:26 +08:00
zhang2014
4a99ca0d80 ISSUES-4006 refactor 2020-08-04 10:31:25 +08:00
Alexey Milovidov
05bfb70c7a Addition to Revert "Less indirection in FixedHashTable" 2020-08-04 01:40:55 +03:00
Alexey Milovidov
a7e5feb8b4 Added another method 2020-08-04 01:40:55 +03:00
Alexey Milovidov
d94717c699 Add one more method to the test 2020-08-04 01:40:55 +03:00
alexey-milovidov
4d5ba3571b Update average.cpp 2020-08-04 01:40:55 +03:00
Alexey Milovidov
f428c2da70 Add a test 2020-08-04 01:40:55 +03:00
alexey-milovidov
65c755190e
Revert "Add QueryTimeMicroseconds, SelectQueryTimeMicroseconds and InsertQuer…" 2020-08-04 01:18:59 +03:00
Azat Khuzhin
2fc1c859c7 Merge remote-tracking branch 'upstream/master' into roundUpToPowerOfTwoOrZero-safety 
* upstream/master:
  Update tips.md
  Wait with retries in fast test (#13284)
  update #syntax-expressions in syntax.md (#13269)
  fix test
  Set custom prefix in server-test.xml too.
  Update HashTable.h
  Attempt to fix "Arcadia" build
  More performance tests
  Fix error + one more optimization
  Attempt to fix "Arcadia" build
  Add test
  Continue
  Fix hash table with NaN keys
  Fix tests
  Fix errors
  Fix errors
  Revert some changes
  shellcheck --check-sourced --external-sources --severity style --exclude SC1071 --format diff *.sh ../1_stateful/*.sh | git apply
  Fix all info from Shellcheck
  fgrep -> grep -F
  shellcheck --check-sourced --external-sources --severity info --exclude SC1071 --format diff */*.sh | git apply
  Minor changes 2
  Minor changes
  Fix warnings found by Shellcheck
  Refresh ya.make
  Better code
  Fix assertion in KeyCondition
  Fix including Common/config.h in the Arcadia build.
  Fix bad code
  Add more threads for the 01323_too_many_threads_bug
  Exclude threads used for query_log processing 01323_too_many_threads_bug
  Use ILIKE over lower() LIKE in 01323_too_many_threads_bug
  Update test
  Fix style
  Simplify CMake for libunwind (suggestion from @azat)
  Normalize ya.make files, fix "Arcadia" build
  Fix 01323_too_many_threads_bug (use arrayUniq() over length())
  Fix logging in MergeTreeDataSelectExecutor for multiple threads (attach to thread group)
  Use "Not using primary index on part" over "Not using index on part" (add "primary")
  Exclude 01413_rows_events from the fasttest (TRUNCATE query_log)
  fasttest: use arrays for skip list (to make it more patch friendly)
  Update README.md
  Fix warning in libunwind
  Fix warning in libunwind
  Fix assert in geohashesInBox
  Fix assert in parseDateTimeBestEffort
  Add test
  Fix function if with nullable constexpr as cond that is not literal NULL
  Fix assert in *Map aggregate functions
  More tests
  Fix assert in arrayElement
  Fix assert in IN with tuple literals and functions
  Use stored size for UInt16 case
  Revert "Less indirection in FixedHashTable"
  Special case for aggregation by 8bit field
  Less indirection in FixedHashTable
  Update FixedHashTable.h
  Avoid increment of size in the inner loop of aggregation by UInt8/UInt16
  Trigger CI
  Slightly better
  Fix fuzz test
  Add test
  Function "materialize" should work for NULL as expected
  Minor changes in Debian scripts
  Fix Shellcheck errors
  Fix some spurious perf test failures
  Fix test
  +x for test
  Merge with master
  Added and updated tests
  More sane
  Remove ridiculous trash
  Use absolute paths to backup locations (shadow)
  Add test for macros usage in certain kafka setting.
  Add QueryTimeMicroseconds, SelectQueryTimeMicroseconds and InsertQueryTimeMicroseconds to system.events
  try fix sync
  Fix various code defects in groupArrayMoving functions #12568
  fix
  fixes
  fixes
  implement reneme dictionary query
  rename database with dictionaries
  fix tests
  use dictionary ids
  fix build
  add uuid to generated config
  add StorageID to IDictionaryBase 3
  add StorageID to IDictionaryBase 2
  add StorageID to IDictionaryBase 1
  write uuid for ddl dictionaries
  fixes
  add implicit uuid macro
  fix tests
  fix, add test
  implement rename database for atomic
  fix symlinks, add short syntax of attach database
  add uuid to atomic databases
 2020-08-03 22:51:40 +03:00
alexey-milovidov
ef0097d326
Merge pull request #12343 from ClickHouse/database_atomic_improvements 
DatabaseAtomic improvements
 2020-08-03 21:45:41 +03:00
Alexey Milovidov
1d091738dd Merge branch 'master' into stack-frams-size-limit 2020-08-03 20:41:47 +03:00
Alexander Kuzmenkov
0d4b8bc226 fixup 2020-08-03 20:22:59 +03:00
Alexander Kuzmenkov
59ed586b69 fixup 2020-08-03 20:15:08 +03:00
alexey-milovidov
3015503dc6
Merge pull request #13254 from ClickHouse/fix-hash-table-nan-key 
Fix hash table with NaN keys
 2020-08-03 19:37:17 +03:00
Alexander Kuzmenkov
004a393295 Merge remote-tracking branch 'origin/master' into HEAD 2020-08-03 19:20:05 +03:00
alexey-milovidov
f536836eee
Merge pull request #13091 from ClickHouse/fixed-hash-map-implicit-zero-4 
Slightly improve performance of aggregation by UInt8/UInt16 keys, part 4
 2020-08-03 19:07:35 +03:00
Alexander Kuzmenkov
c4a33430b1
Merge pull request #13028 from ianton-ru/select_insert_time 
Add QueryTimeMicroseconds, SelectQueryTimeMicroseconds and InsertQuer…
 2020-08-03 17:07:34 +03:00
Alexander Kuzmenkov
e7744cc15c Merge remote-tracking branch 'origin/master' into HEAD 2020-08-03 16:35:42 +03:00
Alexander Tokmakov
5b92f43f12 Merge branch 'master' into database_atomic_improvements 2020-08-03 14:35:40 +03:00
alexey-milovidov
79ba0a5be9
Update HashTable.h 2020-08-03 04:46:47 +03:00
Alexey Milovidov
486a4b3a17 Merge branch 'master' into fixed-hash-map-implicit-zero-4 2020-08-03 04:19:22 +03:00
Alexey Milovidov
23c268e5ef Continue 2020-08-03 03:01:23 +03:00
alexey-milovidov
1e91238466
Merge pull request #13225 from ClickHouse/map-skip-null 
Fix assert in *Map aggregate functions
 2020-08-03 02:12:48 +03:00
alexey-milovidov
6399a1df0f
Merge pull request #13250 from vitlibar/fix-including-common-config-header-arcadia 
Fix including Common/config.h in the Arcadia build.
 2020-08-03 01:39:22 +03:00
Alexey Milovidov
32c263aa9f Fix hash table with NaN keys 2020-08-03 01:20:35 +03:00
Alexey Milovidov
0fd7af8a63 Merge branch 'master' into stack-frams-size-limit 2020-08-03 00:06:00 +03:00
Alexey Milovidov
9611b2c7f4 Refresh ya.make 2020-08-03 00:04:48 +03:00
Vitaly Baranov
ac3596389d Fix including Common/config.h in the Arcadia build. 2020-08-02 23:44:59 +03:00
Alexey Milovidov
41111d1e57 Merge branch 'master' into stack-frams-size-limit 2020-08-02 19:18:33 +03:00
alexey-milovidov
b67e2cee35
Update BitHelpers.h 2020-08-02 19:12:46 +03:00
alexey-milovidov
ae572c3a7e
Update BitHelpers.h 2020-08-02 19:09:51 +03:00
Alexey Milovidov
38c4c727d4 Merge branch 'master' into map-skip-null 2020-08-02 17:22:21 +03:00
Alexey Milovidov
6f690b7c0d Normalize ya.make files, fix "Arcadia" build 2020-08-02 16:57:38 +03:00
Azat Khuzhin
c5e230eb53 Return passed number for numbers with MSB set in roundUpToPowerOfTwoOrZero() 
Otherwise roundUpToPowerOfTwoOrZero() will return 0 for then, and this
will cause 0 memory allocated and eventually SIGSEGV, like in [1].

  [1]: https://clickhouse-test-reports.s3.yandex.net/13201/5eaf39d3faf0abdd425cca9b7ee5778b06aad716/fuzzer/fuzzer.log#fail1
 2020-08-02 12:55:33 +03:00
Vitaly Baranov
18b21511a9
Merge pull request #13013 from vitlibar/implement-custom-settings 
Implement custom settings
 2020-08-02 05:01:14 +03:00
Alexey Milovidov
d3c0635fc0 Merge branch 'master' into stack-frams-size-limit 2020-08-02 04:35:09 +03:00
Alexey Milovidov
eb3422477e Fix assert in *Map aggregate functions 2020-08-02 04:29:52 +03:00
alexey-milovidov
4f0a9b6465
Merge pull request #12316 from ClickHouse/crash-log 
Add system.crash_log table.
 2020-08-02 01:28:14 +03:00
Alexey Milovidov
460bda7759 Use stored size for UInt16 case 2020-08-02 01:11:00 +03:00
Alexey Milovidov
8496c984c5 Revert "Less indirection in FixedHashTable" 
This reverts commit 743aee9940.
 2020-08-02 01:11:00 +03:00
Alexey Milovidov
1088bfffb1 Special case for aggregation by 8bit field 2020-08-02 01:11:00 +03:00
Alexey Milovidov
542ec9f777 Less indirection in FixedHashTable 2020-08-02 01:11:00 +03:00
alexey-milovidov
7d8eec5652 Update FixedHashTable.h 2020-08-02 01:11:00 +03:00
Alexey Milovidov
f1363403b7 Avoid increment of size in the inner loop of aggregation by UInt8/UInt16 2020-08-02 01:11:00 +03:00
alexey-milovidov
626b709085
Merge pull request #13055 from ClickHouse/fixed-hash-map-implicit-zero 
Slightly improve the inner loop of aggregation by UInt8/UInt16 keys
 2020-08-02 01:09:00 +03:00
Alexey Milovidov
64d91d0f90 Fix build; fix errors 2020-08-01 18:54:44 +03:00
Alexey Milovidov
9ef9d31f03 Crash log: development 2020-07-31 23:16:31 +03:00
Alexey Milovidov
59840c725c Merge remote-tracking branch 'origin/master' into crash-log 2020-07-31 22:52:37 +03:00
Vitaly Baranov
80a3caef1c Allow using any types for custom fields. 2020-07-31 20:57:49 +03:00
Vitaly Baranov
18e3f1f60d Split SettingsCollection.h into 3 files: SettingsFields.h, SettingsEnums.h, SettingsCollection.h 2020-07-31 19:11:27 +03:00
Vitaly Baranov
90602b869a Make SettingsChanges a class. 2020-07-31 19:11:27 +03:00
Vitaly Baranov
668653600c Use SettingMaxThreads only in Settings, call getNumberOfPhysicalCPUCores() instead of SettingMaxThreads::getAuto(). 2020-07-31 19:11:27 +03:00
Alexander Kuzmenkov
efb09d7ff5 Cleanup for #12999. NFC. 2020-07-31 17:53:41 +03:00
Alexey Milovidov
c3ad710b84 Merge branch 'master' into crash-log 2020-07-31 16:12:53 +03:00
alexey-milovidov
9c9ef0db9b
Merge branch 'master' into database_atomic_improvements 2020-07-31 14:56:49 +03:00
Alexey Milovidov
d63f82e36e Even more retries in zkutil gtest 2020-07-31 13:12:03 +03:00
alexey-milovidov
065db05665
Merge pull request #12638 from ianton-ru/selected_rows_event 
Add SelectedRows and SelectedBytes events
 2020-07-31 12:48:46 +03:00
alesapin
2eb01fd655
Merge pull request #13104 from ClickHouse/volumes-refactorings 
Merging #12670
 2020-07-31 11:21:30 +03:00
alexey-milovidov
74edcfb87e
Merge pull request #13073 from nikitamikhaylov/rewrite-tests 
BayesAB test
 2020-07-31 02:30:07 +03:00
alexey-milovidov
e8f5bbd05f
Update TraceCollector.cpp 2020-07-31 02:29:54 +03:00
Alexey Milovidov
c202691319 Merge branch 'master' into stack-frams-size-limit 2020-07-30 23:53:52 +03:00
Alexander Kuzmenkov
5214621830 [draft] only fix int16->int32 insert in PODArray 2020-07-30 18:20:47 +03:00
Alexander Kuzmenkov
14af140447 Merge remote-tracking branch 'origin/master' into HEAD 2020-07-30 18:17:55 +03:00
Nikita Mikhaylov
a948d20832 fix 2020-07-30 14:59:21 +03:00
Nikita Mikhaylov
4d49d2c671 another removes 2020-07-30 13:31:14 +03:00
alesapin
724b7be555 Merge branch 'volumes-related-refactorings' of https://github.com/excitoon-favorites/ClickHouse into volumes-refactorings 2020-07-30 12:20:48 +03:00
Alexey Milovidov
bc0d733b8e Merge remote-tracking branch 'origin/master' into fixed-hash-map-implicit-zero 2020-07-30 05:43:40 +03:00
alesapin
34e7b25d91
Merge pull request #12992 from ClickHouse/rerun_failed_in_fast_test 
Rerun tests in fast test
 2020-07-29 21:11:54 +03:00
Anton Ivashkin
ce25924871 Merge master 2020-07-29 19:32:30 +03:00
alesapin
d52f67d24e Fix abtesting build 2020-07-29 14:00:04 +03:00
Anton Ivashkin
f4efe3c2fd Add QueryTimeMicroseconds, SelectQueryTimeMicroseconds and InsertQueryTimeMicroseconds to system.events 2020-07-29 13:32:18 +03:00
Alexey Milovidov
490a8ba3d4 Slightly improve inner loop of aggregation by UInt8/UInt16 keys 2020-07-29 04:50:12 +03:00
Alexander Kuzmenkov
f6f2812f73 Block structure mismatch is a program error 2020-07-28 21:35:18 +03:00
Anton Ivashkin
89a59d43f2 Merge 2020-07-27 17:59:12 +03:00
Alexey Milovidov
916daf7ff0 Fix build of test under Mac OS X #12767 2020-07-27 14:34:52 +03:00
Gleb Novikov
3585b24f16 MultiDiskReservation, VolumeRAID1 and createVolumeFromConfig 
1. Implemented MultiDiskReservation to maintain multiple reservations under one
2. Implemented VolumeRAID1 as child of VolumeJBOD (since RAID1 is quite similar)
3. Implemented createVolumeFromConfig
 2020-07-26 21:27:36 +03:00
Alexander Tokmakov
237accd154 Merge branch 'master' into database_atomic_improvements 2020-07-23 06:33:20 +03:00
Nikolai Kochetov
76d784be9c
Merge pull request #12613 from ClickHouse/remove-sort-description-from-streams 
Remove sort description from streams
 2020-07-22 17:35:05 +03:00
Alexey Milovidov
c55834c40f Merge branch 'master' into stack-frams-size-limit 2020-07-22 08:49:00 +03:00
Alexander Tokmakov
9bcaaea3e0 Merge branch 'master' into database_atomic_improvements 2020-07-22 05:24:48 +03:00
Alexey Milovidov
7b81640e9b Add support for function if with Array(UUID) arguments #11066 2020-07-22 00:15:19 +03:00
Alexander Kuzmenkov
e45e2d174f Merge remote-tracking branch 'origin/master' into HEAD 2020-07-21 18:42:03 +03:00
Anton Ivashkin
742793be54 Add SelectedRows and SelectedBytes events 2020-07-21 17:50:02 +03:00
Alexey Milovidov
50bbfc848d Merge branch 'master' into stack-frams-size-limit 2020-07-21 14:45:15 +03:00
Nikolai Kochetov
a93a7d3a75 Remove unused error code. 2020-07-21 13:10:16 +03:00
alexey-milovidov
16e8c61438
Merge pull request #12566 from ClickHouse/race_condition_cache_dictionaries 
Fix race condition in clickhouse cache dictionary
 2020-07-21 11:27:08 +03:00
alesapin
6b7d1033e6 Comments for ugly fix 2020-07-20 16:06:44 +03:00
alexey-milovidov
7aee5abb83
Merge pull request #12569 from ClickHouse/aku/numa-metric 
Add perf metric for NUMA-local memory access
 2020-07-19 03:21:24 +03:00
alexey-milovidov
504783fe2b
Update ProfileEvents.cpp 2020-07-19 03:20:45 +03:00
Alexander Kuzmenkov
5a662ca48c Add perf metric for NUMA-local memory access 2020-07-17 20:10:59 +03:00
Alexander Tokmakov
a612080df1 Merge branch 'master' into database_atomic_improvements 2020-07-17 00:43:59 +03:00
alexey-milovidov
fde8c87a1f
Merge pull request #12426 from ClickHouse/log-engine-rollback-on-insert-error 
Rollback insertion error in Log engines
 2020-07-16 22:50:48 +03:00
Vitaly Baranov
000b197ad1
Merge pull request #11234 from traceon/ldap-per-user-authentication 
Add LDAP authentication support
 2020-07-16 13:17:21 +03:00
Alexey Milovidov
6df282e813 Fixup 2020-07-16 11:33:51 +03:00
Alexey Milovidov
3408b7e259 Merge branch 'master' into log-engine-rollback-on-insert-error 2020-07-16 05:34:02 +03:00
Alexey Milovidov
e1173bc3f7 Fixup 2020-07-16 05:00:21 +03:00
Alexey Milovidov
1bf679703d Merge branch 'master' into testkeeper-improve-performance 2020-07-16 04:58:35 +03:00
Alexey Milovidov
43990f734c Merge branch 'master' into avoid-overallocation3 2020-07-16 03:16:44 +03:00
Denis Glazachev
59cb758cf7 Merge branch 'master' into ldap-per-user-authentication 2020-07-16 02:29:24 +04:00
Alexander Kuzmenkov
f785837b1d
Merge pull request #12518 from ClickHouse/aku/android 
Some provisions for Android build
 2020-07-15 22:05:41 +03:00
Alexey Milovidov
d380ff83e2 Merge branch 'master' into testkeeper-improve-performance 2020-07-15 19:38:06 +03:00
Alexander Kuzmenkov
3e6b9f297f Some provisions for Android build 2020-07-15 14:16:00 +03:00
Alexey Milovidov
b665413869 Improve performance of TestKeeper 2020-07-15 01:53:34 +03:00
Alexey Milovidov
0c237de00d Fix build 2020-07-15 00:32:14 +03:00
Anton Ivashkin
52fea6a5a3 Merge master 2020-07-14 15:56:38 +03:00
Alexander Tokmakov
fdb02edc7c Merge branch 'master' into database_atomic_improvements 2020-07-13 17:28:48 +03:00
Alexander Tokmakov
04c3e7cab6 add implicit uuid macro 2020-07-13 16:18:38 +03:00
Anton Ivashkin
26fca949bd add profiling events from S3 request to system.events and system.query_log 2020-07-13 11:27:11 +03:00
Alexey Milovidov
befbae585b Revert the only change that should affect performance in gcc 2020-07-12 21:31:17 +03:00
Alexey Milovidov
8092f9f861 Merge branch 'master' into avoid-overallocation2 2020-07-12 06:06:50 +03:00
Alexey Milovidov
204a4af394 Rollback insertion error in Log engines #12402 2020-07-12 05:32:18 +03:00
Denis Glazachev
edb6ef8c09 Merge commit 'ceac649c01b0158090cd271776f3219f5e7ff57c' into ldap-per-user-authentication 
* commit 'ceac649c01b0158090cd271776f3219f5e7ff57c': (75 commits)
  [docs] split misc statements (#12403)
  Update 00405_pretty_formats.reference
  Update PrettyCompactBlockOutputFormat.cpp
  Update PrettyBlockOutputFormat.cpp
  Update DataTypeNullable.cpp
  Update 01383_remote_ambiguous_column_shard.sql
  add output_format_pretty_grid_charset setting in docs
  add setting output_format_pretty_grid_charset
  Added a test for #11135
  Update index.md
  RIGHT and FULL JOIN for MergeJoin (#12118)
  Update MergeTreeIndexFullText.cpp
  restart the tests
  [docs] add syntax highlight (#12398)
  query fuzzer
  Fix std::bad_typeid when JSON functions called with argument of wrong type.
  Allow typeid_cast() to cast nullptr to nullptr.
  fix another context-related segfault
  [security docs] actually, only admins can create advisories
  query fuzzer
  ...
 2020-07-11 21:32:36 +04:00
Alexander Kuzmenkov
10b0287fa4
Merge pull request #12341 from ClickHouse/aku/perfect-visitor 
More perfect forwarding in field visitors
 2020-07-11 13:36:37 +03:00
alexey-milovidov
c615ea658b
Merge pull request #12400 from vitlibar/fix-bad_typeid 
Fix std::bad_typeid when JSON functions called with argument of wrong type
 2020-07-11 05:40:44 +03:00
Vitaly Baranov
3a0d358694 Allow typeid_cast() to cast nullptr to nullptr. 2020-07-10 17:02:48 +03:00
alexey-milovidov
6479e2b406
Update FieldVisitors.h 2020-07-10 08:37:43 +03:00
Alexander Kuzmenkov
0001a08081 More perfect forwarding in field visitors 2020-07-09 19:16:33 +03:00
Denis Glazachev
5a4762bb8a
Merge branch 'master' into ldap-per-user-authentication 2020-07-09 14:50:02 +04:00
Alexey Milovidov
31cbdd1a56 system.crash_log: development 2020-07-09 07:15:45 +03:00
alexey-milovidov
072a8e0e40
Merge pull request #12182 from azat/fix-user-memory-tracking-drift 
Cap max_memory_usage* limits to the process resident memory
 2020-07-09 04:25:55 +03:00
Alexey Milovidov
edb68b7f6b Add test 2020-07-09 04:22:52 +03:00
Alexey Milovidov
70aae6eb44 Fix error 2020-07-09 04:21:25 +03:00
Alexey Milovidov
5ac167fcf0 Merge branch 'master' into avoid-overallocation2 2020-07-09 04:10:30 +03:00
alexey-milovidov
1062270580
Merge pull request #12296 from ClickHouse/fix_rare_hanging_in_dnsresolver 
Fix rare hanging in DNSResolver
 2020-07-09 00:35:07 +03:00
tavplubix
33c3545125
Merge pull request #12197 from nikitamikhaylov/csv-file-bugfix 
Headers for CSVWithNames in StorageFile
 2020-07-08 20:42:49 +03:00
Alexander Tokmakov
c48ed67760 get hostname without mutex 2020-07-08 17:39:48 +03:00
Alexey Milovidov
c4e925434a Fix build 2020-07-08 06:29:54 +03:00
Alexey Milovidov
a6dbe421ac Fix build 2020-07-08 05:31:29 +03:00
Alexey Milovidov
491bd44955 Improve performance of clang build 2020-07-08 04:41:09 +03:00
Alexey Milovidov
3471e526ff Improvements in PODArray 2020-07-08 04:26:44 +03:00
Alexey Milovidov
671c544b7d Improvements in PODArray 2020-07-08 04:24:38 +03:00
Alexey Milovidov
c34f3254f2 Improvements in PODArray 2020-07-08 04:21:39 +03:00
Alexey Milovidov
934d18b649 Added a test for PaddedPODArray just in case 2020-07-08 02:41:10 +03:00
Denis Glazachev
9effacfbc1 Merge branch 'master' into ldap-per-user-authentication 
* master: (1102 commits)
  Update README.md
  Update README.md
  Update README.md
  Update index.md
  [docs] add intrdocution for statements page (#12189)
  Revert "Run perf tests with memory sampling (for allocations >1M)"
  Sanitize LINK_LIBRARIES property for the directories (#12160)
  [docs] refactor Domains overview (#12186)
  DOCS-647: toStartOfSecond (#12190)
  [docs] add intrdocution for commercial page (#12187)
  DOCSUP-1348 Russian translation for new functions (#133) (#12194)
  changelog fixes
  Update index.md (#12191)
  Update zh kafka.md title (#12192)
  Added test for #3767
  style fix for #12152
  Tests for fixed issues #10846 and #7347
  changelog fixes
  [docs] introduction for special table engines (#12170)
  [docs] introduction for third-party interfaces (#12175)
  ...

# Conflicts:
#	src/Access/ya.make
#	src/Common/ErrorCodes.cpp
 2020-07-08 00:42:09 +04:00
Nikita Mikhaylov
270551e599 done 2020-07-07 14:45:20 +03:00
alesapin
be85f9f98a
Merge pull request #12165 from vitlibar/fix-split-dictionary-source-table-name 
Fix splitting table name of dictionary source
 2020-07-07 11:49:36 +03:00
Azat Khuzhin
68ca3b7aec Do not try to adjust memory tracker amount if it is not larger then in total 2020-07-07 11:26:40 +03:00
Azat Khuzhin
d68ecdc84c Cap max_memory_usage* limits to the process resident memory 
There are still some issues with memory tracking, but now with per-user
tracking:

    executeQuery: Code: 241, e.displayText() = DB::Exception: Memory limit (for user) exceeded: would use 437.72 GiB (attempt to allocate chunk of 4200926 bytes), maximum: 437.72 GiB (version 20.6.1.1) (from 10.7.140.7:31318)

Although the server is mostly idle:

    SELECT formatReadableSize(memory_usage)
    FROM system.processes

    ┌─formatReadableSize(memory_usage)─┐
    │ 289.28 MiB                       │
    │ 155.75 MiB                       │
    │ 0.00 B                           │
    └──────────────────────────────────┘

Refs: https://github.com/ClickHouse/ClickHouse/pull/10496/files#r450206865
Cc: @alexey-milovidov
 2020-07-07 01:51:21 +03:00
Vitaly Baranov
4733504b51 Don't split dictionary source's table name into schema and table name itself 
if ODBC driver doesn't support schema.
 2020-07-06 23:41:39 +03:00
Nikita Mikhaylov
5c6d6bdf54
Merge pull request #11995 from azat/load_balancing-priority 
Load balancing manual priority
 2020-07-06 13:32:20 +04:00
alexey-milovidov
f1151ed474
Merge pull request #12133 from ClickHouse/normalize-pid-file 
Normalize "pid" file handling #3501
 2020-07-04 21:36:27 +03:00
alexey-milovidov
d9ea8cfc42
Update StatusFile.cpp 2020-07-04 16:57:04 +03:00
Alexey Milovidov
176a7f2f72 Normalize "pid" file handling #3501 2020-07-04 16:54:24 +03:00
Val
147fa9fed9 fix type error in zookeeper 2020-07-04 16:48:44 +03:00
alesapin
64583ceb22 Merge branch 'master' into add-storage-rabbitmq-read-only 2020-07-02 16:08:29 +03:00
Alexander Kuzmenkov
aa8461e996
Update ThreadProfileEvents.cpp 2020-06-30 10:47:19 +03:00
Alexander Kuzmenkov
1d838b7b3a
Update ThreadProfileEvents.cpp 2020-06-30 09:56:11 +03:00
Azat Khuzhin
ebff4eae7d Add replica priority for load_balancing 
Make default 1, to match with <weight>
 2020-06-29 23:03:28 +03:00
Alexander Kuzmenkov
877c10f5fa
Update ThreadProfileEvents.cpp 2020-06-29 17:13:16 +03:00
Alexander Kuzmenkov
9b9030caad
Update ThreadProfileEvents.cpp 2020-06-29 15:57:36 +03:00
Alexander Kuzmenkov
af8d62bbcb Add TLB misses perf counters. 2020-06-29 15:48:18 +03:00
kssenii
649eb8e348 Move reading from RabbitMQ into background task 2020-06-28 18:20:18 +00:00
alexey-milovidov
7376f18467
Merge pull request #11923 from ucasFL/client-not-argue 
client exit with unrecognized arguments
 2020-06-28 17:38:25 +03:00
alexey-milovidov
86ef3e1d8c
Update ErrorCodes.cpp 2020-06-28 17:37:42 +03:00
Alexey Milovidov
1462a66d1e Fix typos 2020-06-27 22:05:00 +03:00
Anton Popov
b2aa565a37 allow to turn on fsync on inserts, merges and fetches 2020-06-27 17:52:16 +03:00
Val
cbcd1bea0e provide better comments and information 2020-06-27 17:00:44 +03:00
Val
1f03839830 add zookeeper tryRemoveChildren method 2020-06-27 16:59:09 +03:00
Val
5e076b464e add replicated db snapshot, integration test, repl alter queries, etc 
add an option to create replicated tables within replicated db without specifying zk path and replica id
add replicated sch pool
disable replication of alter queries for replicated tables in replicated dbs
snapshot prototype. amend of replicated db workflow
add prototype of integration tests for replicated db
 2020-06-27 16:58:15 +03:00
feng lv
a159f1be54 Merge branch 'master' of github.com:ClickHouse/ClickHouse into client-not-argue 2020-06-26 17:34:24 +08:00
Alexander Kuzmenkov
a8db0b40d0 add page fault perf events 2020-06-25 23:42:31 +03:00
alexey-milovidov
a34032cace
Merge pull request #11300 from blinkov/sentry 
Opt-in support for sending crash reports
 2020-06-25 17:20:50 +03:00
alexey-milovidov
40755de412
Merge pull request #11908 from ClickHouse/zookeeper-modernize 
Slightly modernize code around ZooKeeper
 2020-06-25 16:25:50 +03:00
feng lv
6f3b78c774 client exit with unrecognized arguments 
format code

fix
 2020-06-25 18:09:35 +08:00
Azat Khuzhin
236cac3d52 Fix jemalloc under OSX (by registering it as default zone explicitly) 
In case of OSX jemalloc register itself as a default zone allocator.

But when you link statically then zone_register() will not be called,
and even will be optimized out:

  $ nm clickhouse.patched  | grep -c zone_register
  0

Fix this, by manually calling it.

v2: extern C
 2020-06-24 21:05:35 +03:00
Alexander Kuzmenkov
2da7b356ab
Merge pull request #11668 from bharatnc/bnc/config-max-num-threads 
max_thread_pool_size setting for changing max Threads in Global Thread Pool
 2020-06-24 16:11:21 +03:00
Alexey Milovidov
5a7e921135 Slightly modernize code around ZooKeeper 2020-06-24 14:02:14 +03:00
Ivan Blinkov
3ed76a388e send build_id to sentry as well + fix some minor issues 2020-06-24 00:27:04 +03:00
Ivan Blinkov
45d1ca2567 merge master 2020-06-24 00:18:16 +03:00
Ivan Blinkov
87d56d700f restore old toStringImpl signature 2020-06-24 00:15:54 +03:00
bharatnc
9be7ab0341 fix style check - removed unused LOGICAL_ERROR 2020-06-23 13:06:22 -07:00
Alexey Milovidov
cda2687195 Merge branch 'master' into fix-fpe-datetime64 2020-06-23 22:45:45 +03:00
Alexander Kuzmenkov
f0fbba5522 allow implicit initialization 2020-06-23 20:31:46 +03:00
Alexander Kuzmenkov
1a63972a4b Merge remote-tracking branch 'origin/master' into bnc/config-max-num-threads 2020-06-23 20:25:07 +03:00
alexey-milovidov
1c1e12f06b
Merge pull request #11826 from ClickHouse/send-logs-on-fatal-errors 
Send logs to client on fatal errors if possible
 2020-06-23 09:49:48 +03:00
Alexey Milovidov
75357ab98d constexpr intExp10 2020-06-22 23:32:29 +03:00
Alexander Kuzmenkov
96d2e9c997 Initialize GlobalThreadPool explicitly 2020-06-22 22:04:12 +03:00
Alexander Kuzmenkov
bd5ab9c686 Merge remote-tracking branch 'origin/master' into bnc/config-max-num-threads 2020-06-22 15:55:39 +03:00
Alexey Milovidov
b50b4b14de Merge branch 'master' into send-logs-on-fatal-errors 2020-06-22 15:48:46 +03:00
Alexander Kuzmenkov
ffc5b21bad Merge remote-tracking branch 'origin/master' into HEAD 2020-06-22 15:28:03 +03:00
alexey-milovidov
d6abf45a2d
Merge pull request #11847 from ClickHouse/DimasKovas-arch-chooser 
Merging #10058
 2020-06-22 14:01:10 +03:00
alexey-milovidov
281a6d6e46
Merge pull request #11824 from ClickHouse/print-build-id 
Print build id in crash messages
 2020-06-22 13:59:48 +03:00
Ivan Blinkov
7b1ab482b0 restore immediate stacktrace output 2020-06-22 10:22:13 +03:00
Ivan Blinkov
2c0ff29c48 Merge branch 'master' of github.com:ClickHouse/ClickHouse into sentry 2020-06-22 10:06:36 +03:00