thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-17 20:02:05 +00:00
Code Issues Packages Projects Releases Wiki Activity
87,933 Commits 9,330 Branches 1,783 Tags 2.1 GiB
ef7d5b2a65
Commit Graph

25581 Commits

Author SHA1 Message Date
alesapin
bd7b3847c1 Some code 2022-04-19 01:09:09 +02:00
Robert Schulze
118e94523c
Activate clang-tidy warning "readability-container-contains" 
This check suggests replacing <Container>.count() by
<Container>.contains() which is more speaking and in case of
multimaps/multisets also faster.
 2022-04-18 23:53:11 +02:00
Larry Luo
a7ed0ef568 Fixed style check error 2022-04-18 17:20:58 -04:00
Larry Luo
5a5cc3dd13 Added keyword SALT to indicate random salt value in CREATE/ALTER USER results. 2022-04-18 17:02:22 -04:00
Alexey Milovidov
1e1ffbbfe2
Update SessionLog.cpp 2022-04-18 23:07:34 +03:00
Alexey Milovidov
8f8f798438
Merge pull request #36368 from ClickHouse/column-nodiscard 
Add nodiscard attribute to immutable methods of `IColumn` to avoid errors
 2022-04-18 23:03:02 +03:00
Anton Popov
56bdc654d4 Revert "Merge pull request #36337 from ClickHouse/revert-36126-fork_chmaster2" 
This reverts commit e04f80d4a7, reversing
changes made to 06e0afb67f.
 2022-04-18 17:59:46 +00:00
Larry Luo
e4d83f5627 Remove test code. 2022-04-18 13:48:50 -04:00
mergify[bot]
daca157111
Merge branch 'master' into fix_storage_distributed_ttl 2022-04-18 17:31:59 +00:00
Larry Luo
3b605b3fd7 Fixed check style error 2022-04-18 12:03:14 -04:00
larryluogit
393fd88d95
Merge branch 'ClickHouse:master' into feature-password 2022-04-18 11:39:40 -04:00
Larry Luo
6189b28489 Added unit tests for parsing sha256_password and sha256_hash with salt 2022-04-18 11:34:25 -04:00
lthaooo
1b533aa583 fix dictionary reload bug 2022-04-18 21:13:08 +08:00
kssenii
40f7ddb6cc Fix 2022-04-18 15:07:07 +02:00
Memo
a17f102fe6 fix style 2022-04-18 20:31:40 +08:00
Memo
335be4c807 hex support Int128/Int256/UInt128/UInt256 2022-04-18 20:13:43 +08:00
alesapin
a97754f462 Fix 2022-04-18 13:39:09 +02:00
alesapin
c4b70f3ed3 Merge branch 'master' into fix_trash 2022-04-18 12:20:09 +02:00
zhanglistar
5670dcf750 Fix crash in debug mode of ReadBufferFromHDFS. 2022-04-18 17:52:59 +08:00
alesapin
8e01b3a5d7
Merge pull request #36369 from ClickHouse/fix-strange-trash-in-keeper 
Fix strange trash in Keeper
 2022-04-18 11:44:02 +02:00
zhanglistar
355333b361
Merge branch 'ClickHouse:master' into fixhdfsreadbuffer 2022-04-18 16:42:03 +08:00
fenglv
735c38cdcc fix typo 2022-04-18 04:59:41 +00:00
Memo
b3adf150b5 add_query_level_settings 2022-04-18 12:15:41 +08:00
Alexey Milovidov
f5e270b2f8
Merge pull request #36353 from ucasfl/type-infer-null 
Implement type inference for INSERT INTO function null()
 2022-04-18 07:06:54 +03:00
Alexey Milovidov
0af183826d
Merge pull request #36358 from ClickHouse/don-t-randomize-priority 
Do not randomize "priority" setting
 2022-04-18 07:04:36 +03:00
Alexey Milovidov
1333b4cd89
Merge pull request #36345 from ucasfl/throw-no-data 
Add setting throw_if_no_data_to_insert
 2022-04-18 07:04:16 +03:00
Alexey Milovidov
f6ab2bd523
Merge pull request #36312 from ClickHouse/remove-arcadia 
Remove remaining parts of Arcadia
 2022-04-18 07:02:54 +03:00
Alexey Milovidov
36595e4206
Merge pull request #36320 from ClickHouse/fix-clang-tidy-14 
Fix clang-tidy-14 (part 1)
 2022-04-18 07:02:10 +03:00
Alexey Milovidov
48fd09c7db
Merge pull request #36310 from ClickHouse/cpuid-remove-trash 
Remove trash from CPUID
 2022-04-18 07:01:23 +03:00
Alexey Milovidov
f86808914c Fix strange trash in Keeper 2022-04-18 04:44:30 +02:00
Alexey Milovidov
c23e451b04 Add nodiscard attribute to immutable methods of IColumn to avoid errors 2022-04-18 04:22:40 +02:00
Alexey Milovidov
bd16102e53 Fix absolutely wrong code in Map data type 2022-04-18 04:05:49 +02:00
Alexey Milovidov
3817c6d710 Fix absolutely wrong code in Map data type 2022-04-18 03:58:32 +02:00
Alexey Milovidov
2855d987ce Merge branch 'master' into bool-sync-after-error 2022-04-18 02:44:40 +02:00
Alexey Milovidov
543e5d7242 Merge branch 'master' into musl-check 2022-04-18 02:40:13 +02:00
Alexey Milovidov
70d7ba5d61 Fix clang-tidy 2022-04-18 02:13:28 +02:00
Alexey Milovidov
6de5d8a554 Merge branch 'master' into fix-clang-tidy-14 2022-04-18 02:12:45 +02:00
Alexey Milovidov
0051e651c6
Update AsynchronousMetricLog.h 2022-04-18 03:01:57 +03:00
Alexey Milovidov
c7bca13a03 Remove abbreviation 2022-04-18 01:03:58 +02:00
Alexey Milovidov
242919eddd Remove abbreviation 2022-04-18 01:02:49 +02:00
Alexey Milovidov
8ea14d2664 Custom ORDER BY for system.asynchronous_metric_log 2022-04-17 23:59:05 +02:00
Alexey Milovidov
e36e22812a Custom column list for system.asynchronous_metric_log 2022-04-17 23:53:28 +02:00
Alexey Milovidov
300ef741cc Custom column list for system.asynchronous_metric_log 2022-04-17 23:51:56 +02:00
Alexey Milovidov
4c09e2d30a Custom column list for system.asynchronous_metric_log 2022-04-17 23:49:39 +02:00
Alexey Milovidov
37c61d9f0a Round values for the log 2022-04-17 22:07:50 +02:00
Alexey Milovidov
1f5df287c0 Remove useless code 2022-04-17 22:02:55 +02:00
Alexey Milovidov
6807f8428d Remove microseconds from system.asynchronous_metric_log 2022-04-17 22:01:58 +02:00
Alexey Milovidov
250ca83058 Allow custom default ORDER BY for system logs 2022-04-17 21:57:44 +02:00
Alexey Milovidov
a64765bc6f Allow custom default ORDER BY for system logs 2022-04-17 21:56:36 +02:00
Alexey Milovidov
aa93f4a074 Do not randomize "priority" setting 2022-04-17 20:48:02 +02:00
fenglv
0024a08ace fix 
fix typo
 2022-04-17 15:12:07 +00:00
fenglv
293d0a5d46 Implement type inference for INSERT INTO function null() 2022-04-17 15:07:56 +00:00
Alexey Milovidov
89deaf0af1
Merge pull request #36056 from azat/client-trailing-space 
Remove trailing whitespace in default client_name
 2022-04-17 17:37:43 +03:00
mergify[bot]
4fed033dca
Merge branch 'master' into alias 2022-04-17 14:37:04 +00:00
Alexey Milovidov
27afc08d8f Merge branch 'master' into cpuid-remove-trash 2022-04-17 16:30:30 +02:00
Alexey Milovidov
7de4f2b6c1 Merge branch 'master' into remove-arcadia 2022-04-17 16:30:09 +02:00
Alexey Milovidov
fb0a23881d Merge branch 'master' into fix-clang-tidy-14 2022-04-17 16:28:54 +02:00
Alexey Milovidov
576e897a78 Merge branch 'master' into bool-sync-after-error 2022-04-17 16:27:24 +02:00
awakeljw
017777793a Possible range issues in automatic assigned enums, also fix error message. 2022-04-17 21:40:48 +08:00
Robert Schulze
1e1df8e101
Replace remove-erase idiom by C++20 erase()/erase_if() 
- makes the code less verbose while being as efficient
 2022-04-17 12:04:47 +02:00
fenglv
b25850a665 Add setting throw_if_no_data_to_insert 2022-04-17 05:42:07 +00:00
Alexey Milovidov
42618239a4
Update ParserDataType.cpp 2022-04-17 01:53:11 +03:00
Alexey Milovidov
cb9675d0f2
Revert "support UNSIGNED modifier with unused parameters of INT" 2022-04-16 21:23:57 +03:00
fenglv
2392d4e2b5 fix 2022-04-16 16:08:28 +00:00
Alexey Milovidov
7206838c75 Fix tech debt for Bool and Map data types 2022-04-16 16:09:04 +02:00
fenglv
58111115c5 fix style 2022-04-16 06:21:09 +00:00
fenglv
74ef1b0198 Add aliases JSONLines and NDJSON for JSONEachRow 2022-04-16 06:01:07 +00:00
larryluogit
c98e25d36c
Merge branch 'ClickHouse:master' into feature-password 2022-04-15 22:56:34 -04:00
Alexey Milovidov
294efeccfe Fix clang-tidy-14 (part 1) 2022-04-16 04:54:04 +02:00
Alexey Milovidov
33ffdcaa4f Return two changes 2022-04-16 02:18:22 +02:00
Anton Popov
2de6668b3f show names of erroneous files 2022-04-16 00:10:47 +00:00
Alexey Milovidov
69b91c5410 Simplification #36313 2022-04-16 02:08:06 +02:00
Kseniia Sumarokova
0abc6290b1
Merge pull request #36253 from kssenii/fix-s3-tests 
Fix exception "File segment can be completed only by downloader" in tests
 2022-04-16 01:24:39 +02:00
Alexey Milovidov
40357637ec Remove Arcadia 2022-04-16 00:33:32 +02:00
Alexey Milovidov
e128d89957 Remove Arcadia 2022-04-16 00:24:58 +02:00
Alexey Milovidov
cbeeb7ec4f Remove Arcadia 2022-04-16 00:20:47 +02:00
Alexey Milovidov
a0fe29e243 Remove Arcadia 2022-04-15 23:59:49 +02:00
Alexey Milovidov
416fa95b87 Remove "Arcadia" build system 2022-04-15 23:57:09 +02:00
Alexey Milovidov
364f10aff5 Remove trash 2022-04-15 22:56:51 +02:00
Alexey Milovidov
58fc4e033a Remove trash 2022-04-15 22:54:08 +02:00
Larry Luo
01955fd221 Use upper case salt and address style check error 2022-04-15 16:46:47 -04:00
alesapin
1706ae9e15 Some trash implementation 2022-04-15 18:36:23 +02:00
alesapin
5a8419a48e Remove more trash 2022-04-15 17:05:17 +02:00
alesapin
eb7593f786 Remove more trash 2022-04-15 16:24:38 +02:00
alesapin
2fcc00380d Get rid of iheritance 2022-04-15 14:00:52 +02:00
alesapin
40e5ba21a4
Update src/Common/FileSegment.cpp 2022-04-15 11:55:05 +02:00
zhanglistar
fe93fd83f5
Merge branch 'ClickHouse:master' into fixhdfsreadbuffer 2022-04-15 16:27:20 +08:00
zhanglistar
e22a175afb Fix ReadBufferFromHDFS crash in debug mode 2022-04-15 14:04:30 +08:00
Yakov Olkhovskiy
51b4cef6f8
Merge pull request #36101 from awakeljw/fork_chmaster 
Auto assign numbers for Enum elements
 2022-04-14 23:53:03 -04:00
Rajkumar
883eea5318 code review comments addressing 2022-04-14 17:06:13 -07:00
avogar
ae01646136 Place new logic of schema inference in insert select from table function under setting 2022-04-14 19:05:20 +00:00
kssenii
d8e2d693e5 Allow missing columns for mongo storage 2022-04-14 19:31:07 +02:00
Kruglov Pavel
4bad2cd364
Merge pull request #36192 from Avogar/fix-parallel-read-buffer 
Use atomic instead of mutex + condvar in ParallelReadBuffer
 2022-04-14 18:36:16 +02:00
alesapin
c0317c818c Fix bug in keeper 2022-04-14 17:33:46 +02:00
Anton Popov
305dd57262
Merge branch 'master' into fix_storage_distributed_ttl 2022-04-14 14:51:15 +02:00
Alexander Tokmakov
819eac6297 fix 2022-04-14 14:48:24 +02:00
Nikolay Degterinsky
b36f810132
Merge pull request #36154 from bharatnc/ncb/h3Res0Indexes-fix 
remove unused array in h3Res0Indexes func
 2022-04-14 14:44:00 +02:00
tavplubix
3af2384066
Merge pull request #36190 from azat/revert-27002-mutation-stuck 
Revert "Fix possible mutation stuck due to race with DROP_RANGE"
 2022-04-14 15:03:35 +03:00
Anton Popov
c76b9cc9f5
Merge pull request #36215 from CurtizJ/fix-reading-empty-arrays 
Fix reading of empty arrays in reverse order
 2022-04-14 13:51:48 +02:00
kssenii
3e42a1711d Fix 2022-04-14 13:50:13 +02:00
Anton Popov
791454678b
Merge pull request #36126 from awakeljw/fork_chmaster2 
support UNSIGNED modifier with unused parameters of INT
 2022-04-14 13:43:51 +02:00
avogar
a4e3a5e05e Send both stdin data and data from query/data from infile in client 2022-04-14 11:30:52 +00:00
Kruglov Pavel
3545eba318
Merge pull request #36202 from ClickHouse/client-progress-indication 
Print progress bar on each ProfileEvents packet
 2022-04-14 12:32:07 +02:00
alesapin
bd629f4cd0
Merge pull request #36163 from ClickHouse/maybe_fix_36098 
Some fixes for ReplicatedMergeTree
 2022-04-14 12:12:55 +02:00
mergify[bot]
95901466e7
Merge branch 'master' into master2 2022-04-14 10:05:03 +00:00
Kruglov Pavel
392a178d3e
Merge pull request #36195 from Avogar/fix-json-schema-inference 
Fix possible segfault in schema inference for JSON formats
 2022-04-14 11:51:01 +02:00
mergify[bot]
7bad91e0f4
Merge branch 'master' into fix-parallel-read-buffer 2022-04-14 09:49:50 +00:00
Kruglov Pavel
0bae44abf0
Merge pull request #36209 from Avogar/check-magic-bytes 
Check ORC/Parquet/Arrow format magic bytes before loading file in memory
 2022-04-14 11:32:38 +02:00
Kruglov Pavel
4f9ee879d4
Merge pull request #36212 from Avogar/revert-fix 
Revert reverting "Fix crash in ParallelReadBuffer"
 2022-04-14 11:26:31 +02:00
Kseniia Sumarokova
04926d882a
Merge branch 'master' into change-hashed-path 2022-04-14 11:18:42 +02:00
Kseniia Sumarokova
4d667301cc
Merge pull request #36135 from kssenii/cache-fix 
Fix after #35475
 2022-04-14 11:11:13 +02:00
Antonio Andelic
4be7cc757d
Merge pull request #36125 from ClickHouse/memory-tracking-overloads 
Add new/delete overloads with align
 2022-04-14 08:16:51 +02:00
jewisliu
427c73e7a5 fix Enum overflow 2022-04-14 10:39:07 +08:00
Yakov Olkhovskiy
c1a06ac63a
Merge pull request #24896 from hexiaoting/dev_materialized_view 
Add type check when create materialized view with To clause
 2022-04-13 22:33:54 -04:00
Rajkumar
f5c224f9fe compilation error fix 2022-04-13 19:04:16 -07:00
Anton Popov
0595b5c22b fix reading of empty arrays in reverse order 2022-04-13 21:50:57 +00:00
avogar
ca14a29dbe Revert reverting "Fix crash in ParallelReadBuffer" 2022-04-13 20:39:12 +00:00
avogar
ae88549c4f Allow insert select for files with formats without schema inference 2022-04-13 20:02:52 +00:00
tavplubix
b2cd23658a
Revert "Fix crash in ParallelReadBuffer" 2022-04-13 22:45:34 +03:00
tavplubix
fd23c42ea7
Update StorageReplicatedMergeTree.cpp 2022-04-13 22:33:55 +03:00
avogar
42726639f3 Check ORC/Parquet/Arrow format magic bytes before loading file in memory 2022-04-13 19:27:38 +00:00
Rajkumar
ad71e93896 Merge branch 'master' into feature-password 2022-04-13 12:24:48 -07:00
avogar
f5f1db86d9 Remove commented code 2022-04-13 19:15:52 +00:00
avogar
8b60aeb7bc Improve schema inference for json objects 2022-04-13 19:13:40 +00:00
avogar
1c065f8c7a Some refactoring around schema inference with globs 2022-04-13 17:02:48 +00:00
tavplubix
6a165787a6
Merge pull request #36115 from ClickHouse/thread-pool-reset-thread-name 
Reset thread name in thread pool
 2022-04-13 19:10:04 +03:00
Alexander Tokmakov
66fdf35dfd remove outdated parts immediately on drop partition 2022-04-13 18:01:22 +02:00
Dmitry Novik
dc460f76d8 Print progress bar on each ProfileEvents packet 2022-04-13 15:58:15 +00:00
Alexander Gololobov
fcb83a12ff
Merge pull request #35953 from azat/ttl-mutations 
Require mutations for per-table TTL only when it had been changed
 2022-04-13 17:14:04 +02:00
Maksim Kita
ca89af89a1
Merge pull request #36088 from israelg99/fix_executable 
Fix CH silently failing when it cannot execute a file
 2022-04-13 16:53:00 +02:00
Alexander Tokmakov
40fea2b833 do not unfold {uuid} macro 2022-04-13 16:51:59 +02:00
Vladimir C
77dac8e490
Merge pull request #36179 from CurtizJ/dynamic-columns-10 2022-04-13 15:57:32 +02:00
mergify[bot]
2c5d6d132e
Merge branch 'master' into cache-fix 2022-04-13 13:32:56 +00:00
kssenii
0912b038fa Merge master 2022-04-13 15:30:01 +02:00
Azat Khuzhin
6dbc138e76 Remove duplicated parts removal from mutation. 
This commit reverts 701e2ffd79 ("Fix
possible mutation stuck due to race with DROP_RANGE"), and also adds
some clarification comments.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-13 16:09:17 +03:00
kssenii
05ceae428b Fix tests 2022-04-13 14:54:43 +02:00
Sergei Trifonov
9d1a2da685 merge master and fix conflicts 2022-04-13 14:53:36 +02:00
mergify[bot]
4fb66013fe
Merge branch 'master' into memory-tracking-overloads 2022-04-13 12:40:49 +00:00
Alexander Tokmakov
f2c4f88295 avoid moving replicated tables to Ordinary db 2022-04-13 14:38:20 +02:00
avogar
348cae0d16 Fix possible segfault in schema inference for JSON formats 2022-04-13 12:34:40 +00:00
tavplubix
43de4827d7
Merge pull request #36074 from ClickHouse/zookeeper_log_thread_query_id 
Add thread_id and query_id to zookeeper_log
 2022-04-13 15:34:04 +03:00
Kseniia Sumarokova
9ae047df1d
Update CachedReadBufferFromRemoteFS.cpp 2022-04-13 14:30:10 +02:00
mergify[bot]
ea3afd4c6c
Merge branch 'master' into musl-check 2022-04-13 12:28:34 +00:00
kssenii
a6dc437911 Merge master 2022-04-13 14:23:48 +02:00
Alexey Milovidov
ab80b94f45
Update ThreadPool.cpp 2022-04-13 15:18:49 +03:00
mergify[bot]
ba22cb76bb
Merge branch 'master' into master2 2022-04-13 11:51:43 +00:00
Kruglov Pavel
b2905c5aed
Update src/IO/ParallelReadBuffer.cpp 
Co-authored-by: tavplubix <tavplubix@gmail.com>
 2022-04-13 13:50:11 +02:00
Kruglov Pavel
03fc08cec3
Merge pull request #36137 from azat/client-profile-events 
ProfileEvents fixes
 2022-04-13 12:27:39 +02:00
avogar
a4d74f975e Use atomic instead of mutex + condvar in ParallelReadBuffer 2022-04-13 10:20:29 +00:00
Kseniia Sumarokova
07cfa980ac
Update FileCache.h 2022-04-13 12:08:25 +02:00
Kruglov Pavel
96a73a3392
Merge pull request #36169 from Avogar/fix-parallel-read-buffer 
Fix crash in ParallelReadBuffer
 2022-04-13 12:07:05 +02:00
Nikolai Kochetov
362fcfd2b8
Merge pull request #36075 from ClickHouse/fix-limit-push-down-over-window 
Disable LIMIT push down through WINDOW functions.
 2022-04-13 11:57:37 +02:00
Kseniia Sumarokova
44ecfd925b
Update FileSegment.cpp 2022-04-13 11:35:46 +02:00
Kseniia Sumarokova
053892777f
Update FileSegment.cpp 2022-04-13 11:35:12 +02:00
Kseniia Sumarokova
a611566266
Update FileCache.h 2022-04-13 11:33:15 +02:00
Kseniia Sumarokova
1b267462d8
Update CachedReadBufferFromRemoteFS.cpp 2022-04-13 11:27:47 +02:00
Kseniia Sumarokova
d9ce08915a
Merge pull request #36130 from yuuch/support_MySQL_style_FORMAT_Vertical_before_semicolon 
support  '\G;' at the end of query
 2022-04-13 10:39:40 +02:00
Kseniia Sumarokova
6b6671e89f
Merge pull request #36164 from azat/client-local-format-error 
clickhouse-client: properly cancel query in case of error during formatting data
 2022-04-13 10:37:35 +02:00
Antonio Andelic
835f752a1a Add maybe_unused for align arguments 2022-04-13 06:19:59 +00:00
Azat Khuzhin
891c601589 Fix ProfileEvents overlaps (for multiple queries) in clickhouse-local 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-12 22:27:12 +03:00
Azat Khuzhin
54a1cea7fd Send ProfileEvents in clickhouse-local on query finish (like server does) 
v2: send only for pulling pipelines
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-12 22:26:44 +03:00
Yakov Olkhovskiy
3d52afed71 only check columns sets intersection, some refactoring 2022-04-12 15:10:11 -04:00
Anton Popov
0679d4814d allow to convert empty string to objects 2022-04-12 18:58:40 +00:00
tavplubix
303ecc6f7d
Merge pull request #36136 from ClickHouse/transactions_fix_1 
Minor fix for transactions
 2022-04-12 21:38:47 +03:00
Azat Khuzhin
1a8b49b535 Remove redundant check from LocalConnection::finishQuery() 
It is called from only one place, and under state->is_finished, so state
is there.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-12 21:18:52 +03:00
Azat Khuzhin
097e2f626d Do not overlap profile events snapshots for queries 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-12 20:54:56 +03:00
Azat Khuzhin
e38057ddb0 Fix code alignment in ProfileEventsExt 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-12 20:54:56 +03:00
Azat Khuzhin
d9dd8f5f65 Fix filtering out snapshots from profile events 
This fixes POSITION_OUT_OF_BOUND error:

    $ clickhouse-client --print-profile-events --profile-events-delay-ms=-1 -n -q 'select sleep(1); select 1'
    0
    [p1.azat.localdomain] 2022.04.12 19:31:48 [ 0 ] ContextLock: 9 (increment)
    [p1.azat.localdomain] 2022.04.12 19:31:48 [ 0 ] FunctionExecute: 1 (increment)
    [p1.azat.localdomain] 2022.04.12 19:31:49 [ 0 ] MemoryTrackerUsage: 2132102 (gauge)
    [p1.azat.localdomain] 2022.04.12 19:31:49 [ 0 ] NetworkReceiveElapsedMicroseconds: 139 (increment)
    [p1.azat.localdomain] 2022.04.12 19:31:49 [ 0 ] NetworkSendBytes: 4850 (increment)
    [p1.azat.localdomain] 2022.04.12 19:31:49 [ 0 ] NetworkSendElapsedMicroseconds: 1844 (increment)
    [p1.azat.localdomain] 2022.04.12 19:31:48 [ 0 ] Query: 1 (increment)
    [p1.azat.localdomain] 2022.04.12 19:31:48 [ 0 ] RWLockAcquiredReadLocks: 1 (increment)
    [p1.azat.localdomain] 2022.04.12 19:31:48 [ 0 ] SelectQuery: 1 (increment)
    [p1.azat.localdomain] 2022.04.12 19:31:48 [ 0 ] SelectedBytes: 1 (increment)
    [p1.azat.localdomain] 2022.04.12 19:31:48 [ 0 ] SelectedRows: 1 (increment)
    [p1.azat.localdomain] 2022.04.12 19:31:49 [ 0 ] OSCPUVirtualTimeMicroseconds: 1842 (increment)
    [p1.azat.localdomain] 2022.04.12 19:31:49 [ 0 ] OSReadChars: 426 (increment)
    [p1.azat.localdomain] 2022.04.12 19:31:49 [ 0 ] OSWriteChars: 322 (increment)
    [p1.azat.localdomain] 2022.04.12 19:31:49 [ 0 ] RealTimeMicroseconds: 1002689 (increment)
    [p1.azat.localdomain] 2022.04.12 19:31:49 [ 0 ] SleepFunctionCalls: 1 (increment)
    [p1.azat.localdomain] 2022.04.12 19:31:49 [ 0 ] SleepFunctionMicroseconds: 1000000 (increment)
    [p1.azat.localdomain] 2022.04.12 19:31:49 [ 0 ] UserTimeMicroseconds: 1843 (increment)
    1
    Error on processing query: Code: 11. DB::Exception: Position out of bound in Block::erase(), max position = 5. (POSITION_OUT_OF_BOUND) (version 22.4.1.1)
    (query: select 1)

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-12 20:54:56 +03:00
Azat Khuzhin
1fa3e6a426 Fix printing ProfileEvents on client for multiple queries 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-12 20:54:56 +03:00
alesapin
f011864386 Build clickhouse-keeper with musl 2022-04-12 18:48:16 +02:00
Rajkumar
c711e42b62 argument changed to const ref 2022-04-12 09:31:32 -07:00
mergify[bot]
d5b2355944
Merge branch 'master' into memory-overcommit-free 2022-04-12 16:00:12 +00:00
Dmitry Novik
d9d2bf310d Fix test 2022-04-12 15:58:34 +00:00
Julian Gilyadov
a4f56f3330
Throw exception when file cant be executed instead of displaying success 2022-04-12 17:52:44 +02:00
Rajkumar
92c01c7a8c space removed 2022-04-12 07:37:48 -07:00
Rajkumar
14cb87e3ef password hash salt feature 2022-04-12 07:30:09 -07:00
Yakov Olkhovskiy
5868539605 check only if TO table exists 2022-04-12 10:17:39 -04:00
Antonio Andelic
1b4cebcfb6 Add new/delete overloads 2022-04-12 14:11:10 +00:00
avogar
de9c9462e5 Fix crash in ParallelReadBuffer 2022-04-12 13:54:53 +00:00
mergify[bot]
0b3c15c07a
Merge branch 'master' into projection-fix1 2022-04-12 13:49:28 +00:00
Yakov Olkhovskiy
25101de7e2 check only selected columns against destination table 2022-04-12 09:29:32 -04:00
Dmitry Novik
63de3c9746 Add comment with explanation 2022-04-12 13:00:06 +00:00
Dmitry Novik
188e1a87bd Check if enough memory is actually freed within timeout 2022-04-12 12:52:35 +00:00
alesapin
2f496c7945 Merge branch 'master' into musl-check 2022-04-12 14:40:47 +02:00
Azat Khuzhin
b91281455c clickhouse-client: properly cancel query in case of error during formatting data 
One example of such error is NO_ROW_DELIMITER [1]:

    $ clickhouse-client --stacktrace --multiquery <<<"SELECT * FROM no_length_delimiter_protobuf_00825 FORMAT ProtobufSingle SETTINGS format_schema = '$PWD/tests/queries/0_stateless/format_schemas/00825_protobuf_format_no_length_delimiter:Message'"
    Error on processing query: Code: 546. DB::Exception: The ProtobufSingle format can't be used to write multiple rows because this format doesn't have any row delimiter. (NO_ROW_DELIMITER), Stack trace (when copying this message, always include the lines below):

    ...
    3. /build/build_docker/../src/Common/Exception.cpp:56: DB::Exception::Exception()
    4. /build/build_docker/../src/Processors/Formats/Impl/ProtobufRowOutputFormat.cpp:43: DB::ProtobufRowOutputFormat::write()
    5. /build/build_docker/../src/Processors/Formats/IRowOutputFormat.cpp:34: DB::IRowOutputFormat::consume()
    6. /build/build_docker/../src/Processors/Formats/IOutputFormat.cpp:115: DB::IOutputFormat::write()
    7. /build/build_docker/../src/Client/ClientBase.cpp:398: DB::ClientBase::onData()

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/35865/b862fa7f3ede7d30b7c606c06e7fe1e273b49d32/stateless_tests__thread__actions__[3/3].html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-12 15:22:16 +03:00
Alexander Tokmakov
457a9e9691 fixes for ReplicatedMergeTree 2022-04-12 14:14:26 +02:00
alesapin
1b7b6af268
Merge pull request #36157 from bigo-sg/fixut_coo 
Fix UT error sometimes
 2022-04-12 14:12:24 +02:00
jewisliu
b2c10611d1 support UNSIGNED modifier with unused parameters of INT 2022-04-12 19:49:57 +08:00
tavplubix
2e95e0db3b
Merge pull request #36133 from Algunenano/osx_atomic_exchange 
Add support for atomic exchange in OSX
 2022-04-12 14:10:36 +03:00
Kseniia Sumarokova
abb498c9a5
Update ClientBase.cpp 2022-04-12 12:51:19 +02:00
Kseniia Sumarokova
50b5684d00
Update FileSegment.cpp 2022-04-12 11:59:58 +02:00
Dmitry Novik
9c20a492c8 Add unit tests 2022-04-12 09:46:26 +00:00
Alexander Tokmakov
6e4787560b fix another issue 2022-04-12 11:44:25 +02:00
Vladimir C
8873d7b0e7
Merge pull request #36082 from bigo-sg/cache_hive_files 2022-04-12 11:42:08 +02:00
Kseniia Sumarokova
b29202ac88
Merge pull request #36150 from kssenii/fix-cache-test 
Fix test 02241_remote_filesystem_cache_on_insert for database Ordinary
 2022-04-12 11:08:44 +02:00
Kseniia Sumarokova
74515a3523
Merge pull request #36057 from azat/client-cancel 
clickhouse-client: fix query cancellation if any result was not received yet
 2022-04-12 09:40:20 +02:00
Kseniia Sumarokova
a77a228ab8
Update FileSegment.cpp 2022-04-12 09:37:03 +02:00
zhanglistar
0ae820550d fix ut error sometimes 2022-04-12 14:30:53 +08:00
bharatnc
e5494de63c h3Res0Indexes - remove unused array 2022-04-11 22:18:55 -07:00
taiyang-li
9d31c44d39 fix bug of read buffer from hdfs 2022-04-12 12:01:51 +08:00
yuuch
588b582437 detail about the input 2022-04-12 11:01:20 +08:00
Yakov Olkhovskiy
dc314d53b3 do not create view before check, use MatchColumnsMode::Name mode 2022-04-11 20:28:11 -04:00
mergify[bot]
46b9a1ef80
Merge branch 'master' into client-trailing-space 2022-04-11 22:07:37 +00:00
Yakov Olkhovskiy
783dbb2e63
Merge branch 'master' into dev_materialized_view 2022-04-11 17:47:35 -04:00
mergify[bot]
8910869987
Merge branch 'master' into thread-pool-reset-thread-name 2022-04-11 21:19:05 +00:00
Kseniia Sumarokova
51dd39b42f
Update StorageSystemRemoteDataPaths.cpp 2022-04-11 22:46:51 +02:00
kssenii
fc62e95d76 Fix 2022-04-11 22:24:45 +02:00
kssenii
d2ccf876c6 Fix 2022-04-11 17:51:49 +02:00
Yakov Olkhovskiy
155a2a0d42
Merge pull request #35349 from yakov-olkhovskiy/interpolate-feature 
Interpolate feature
 2022-04-11 11:15:50 -04:00
Yakov Olkhovskiy
2588f8011f comment fix 2022-04-11 11:13:29 -04:00
Anton Popov
471e945efe
Merge pull request #35934 from ClickHouse/make_date 
Implementation of makeDateTime() and makeDateTime64() #30895
 2022-04-11 16:38:23 +02:00
Alexander Tokmakov
38c80940fd slightly beter fix 2022-04-11 16:25:59 +02:00
Kruglov Pavel
972357ebf6
Merge pull request #35748 from bigo-sg/s3_cluster_join 
Fixed : elimate AMBIGUOUS_COLUMN_NAME exception in hdfs/s3 cluster join query
 2022-04-11 16:22:53 +02:00
Yakov Olkhovskiy
7293e01647 some comments added 2022-04-11 09:47:27 -04:00
mergify[bot]
c117aa13c7
Merge branch 'master' into support_MySQL_style_FORMAT_Vertical_before_semicolon 2022-04-11 13:39:19 +00:00
Nikolai Kochetov
ac74757f92
Merge pull request #35973 from filimonov/kafka_read_consumers_in_parallel 
Make parallel reading from several kafka consumers work again
 2022-04-11 15:30:52 +02:00
Raúl Marín
83c814f658 OSX fixes 2022-04-11 15:25:20 +02:00
yuuch
36e2809fd7 support '\G;' at the end of query 2022-04-11 20:46:52 +08:00
Raúl Marín
6e346a7fc3 Support atomic replace under OSX 2022-04-11 14:32:55 +02:00
Vitaly Baranov
0ae63dfdbd
Fix code style in registerBackupEnginesFileAndDisk.cpp 2022-04-11 12:45:16 +02:00
alesapin
5357cad105
Merge pull request #36106 from ClickHouse/get_rid_of_fs_paths_keeper 
Get rid of fs paths keeper
 2022-04-11 12:24:46 +02:00
Vladimir C
1a604cd279
Merge pull request #36058 from xiedeyantu/master 2022-04-11 10:45:50 +02:00
Kseniia Sumarokova
beea4b103f
Merge pull request #36109 from ucasfl/window-view-fix 
Fix window view when is proc time and window kind larger than day
 2022-04-11 09:54:14 +02:00
Alexander Tokmakov
802f62a80e fix 2022-04-11 09:42:00 +02:00
mergify[bot]
aa8cefa847
Merge branch 'master' into client-cancel 2022-04-11 07:41:44 +00:00
taiyang-li
090fd72884 fix bug 2022-04-11 11:19:31 +08:00
Alexey Milovidov
f0fad98a73 Reset thread name in thread pool 2022-04-11 01:03:24 +02:00
Denny Crane
383076ca0e
fix typo 2022-04-10 16:42:43 -03:00
fenglv
cac2df914b fix 2022-04-10 17:28:17 +00:00
fenglv
24e86f81cb Fix window view when is proc time and window kind larger than day 
fix
 2022-04-10 17:26:09 +00:00
Vitaly Baranov
8dcb1358b7
Merge pull request #36072 from vitlibar/backups-improve-arguments-handling-and-file-removing 
Backups: improve arguments handling and file removing
 2022-04-10 16:51:04 +02:00
alesapin
324020a46f Restore class 2022-04-10 15:59:43 +02:00
alesapin
f61cad144c Get rid of strange abstraction 2022-04-10 15:55:05 +02:00
jewisliu
3521292fad Auto assign numbers for Enum elements 2022-04-10 11:57:25 +08:00
Kseniia Sumarokova
867b85dfad
Merge branch 'master' into change-hashed-path 2022-04-09 16:57:57 +02:00
alesapin
2c2b288b44
Merge pull request #36070 from ClickHouse/remove_unused_field 
Less dependencies from disks in buffers
 2022-04-09 15:01:11 +02:00
xiedeyantu
5efd8a4ec0 If data type has DEFAULT NULL in table definition, make data type Nullable 2022-04-09 15:00:41 +08:00
taiyang-li
7e89f760f3 remove useless code 2022-04-09 10:43:58 +08:00
Vitaly Baranov
758f8235c2 Collect backup entries before opening an archive for writing. 2022-04-09 01:44:46 +02:00
Vitaly Baranov
bcd727d535 Allow creating directories containing a new archive. 2022-04-09 01:32:17 +02:00
Vitaly Baranov
fcd28fc3f4 Make backup metadata clearer with new setting 'use_base'. 2022-04-09 01:07:14 +02:00
tavplubix
2894672ecc
Update DatabaseReplicatedWorker.cpp 2022-04-08 22:12:57 +03:00
tavplubix
faae540a85
Merge pull request #24258 from ClickHouse/mvcc_prototype 
MVCC prototype for MergeTree
 2022-04-08 22:02:13 +03:00
Anton Popov
ebcfe2fdc8 fix one more case with nested arrays 2022-04-08 18:29:26 +00:00
Kseniia Sumarokova
1e54c5c57b
Merge pull request #35885 from kssenii/fix-postgres-test-2 
fix postgres test
 2022-04-08 19:55:05 +02:00
Vitaly Baranov
71d54db215 Better handling arguments of backup engines. 2022-04-08 19:52:52 +02:00
taiyang-li
70f4503ba5 use global context for cache 2022-04-09 00:28:07 +08:00
taiyang-li
cd807da838 finish test 2022-04-09 00:15:33 +08:00
taiyang-li
e319df1799 finish dev 2022-04-08 23:58:56 +08:00
Kseniia Sumarokova
10d609fd0d
Update WriteBufferFromS3.cpp 2022-04-08 17:42:05 +02:00
Anton Popov
8abad2874e fix style-check 2022-04-08 15:36:15 +00:00
kssenii
138588be29 Change hashed path 2022-04-08 17:14:13 +02:00
Alexander Tokmakov
50a32a74b7 add thread_id and query_id to zookeeper_log 2022-04-08 17:10:08 +02:00
Anton Popov
6364b6f78f Merge remote-tracking branch 'upstream/master' into dynamic-columns-9 2022-04-08 15:05:51 +00:00
taiyang-li
2c99ef0ecc refactor HiveTableMetadata 2022-04-08 23:04:24 +08:00
Anton Popov
3fce23357c fix insertion of complex json with nested arrays 2022-04-08 14:58:02 +00:00
Yakov Olkhovskiy
87c2b3e2cf fixed Nullable, tests added 2022-04-08 10:52:10 -04:00
alesapin
4a03402da5
Update ReadBufferFromRemoteFSGather.h 2022-04-08 16:41:37 +02:00
alesapin
60e36abfe9 Get rid of metadata in buffers 2022-04-08 16:03:00 +02:00
Sergei Trifonov
fd001b5c80 fix param order 2022-04-08 15:51:53 +02:00
Nikolai Kochetov
2deec53162 Disable LIMIT push down through WINDOW functions. 2022-04-08 13:39:54 +00:00
alesapin
ad9fdc1d79 Don't use metadata path in write buffers 2022-04-08 15:30:46 +02:00
alesapin
0db8808f2e Remove unused field from buffers 2022-04-08 15:06:43 +02:00
Vitaly Baranov
46457c4d0c Improve removing of backup files after error. 2022-04-08 14:25:40 +02:00
Kruglov Pavel
d7b88d7683
Merge pull request #35582 from Avogar/improve-schema-inference 
Improve schema inference and add some fixes
 2022-04-08 13:44:52 +02:00
Alexander Tokmakov
141fbc6980 Merge branch 'master' into mvcc_prototype 2022-04-08 13:38:11 +02:00
Alexander Tokmakov
49c35f3261 Merge branch 'master' into mvcc_prototype 2022-04-08 13:34:40 +02:00
Anton Popov
a16ac67c0c
Merge pull request #36034 from amosbird/better-projection2 
Use storage_snapshot for projection analysis
 2022-04-08 13:22:53 +02:00
alesapin
becd4eda34
Merge pull request #36047 from ClickHouse/fix-typo 
Fix typo
 2022-04-08 12:38:24 +02:00
Vitaly Baranov
45dea9933b
Merge pull request #35883 from azat/format-settings 
Do not allow SETTINGS after FORMAT for INSERT queries
 2022-04-08 12:38:14 +02:00
kssenii
87855c4ae7 Merge master 2022-04-08 12:32:48 +02:00
kssenii
ef2b238444 Fix 2022-04-08 12:30:24 +02:00
alesapin
8ec802bc62
Merge pull request #35475 from kssenii/remote-fs-cache-improvements 
Allow to write remote fs cache on all write operations. Add `system.remote_filesystem_cache` table. Add `drop remote filesystem cache (<path>)` query. Add `system.remote_data_paths` table.
 2022-04-08 12:06:26 +02:00
Vladimir C
265fa90689
Merge pull request #36033 from vdimir/cross-join-empty-list 2022-04-08 11:59:21 +02:00
Azat Khuzhin
213b7bb222 clickhouse-client: fix query cancellation if any result was not received yet 
This should fix issues with queries left after tests like in:
- https://s3.amazonaws.com/clickhouse-test-reports/35865/10b9f38d8215cb57783125efe51a8c7aa48590a5/stateless_tests__debug__actions__[2/3].html
- https://s3.amazonaws.com/clickhouse-test-reports/35865/10b9f38d8215cb57783125efe51a8c7aa48590a5/stateless_tests__debug__actions__[3/3].html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-08 12:57:38 +03:00
taiyang-li
2e6f0db825 first commit 2022-04-08 15:12:24 +08:00
Azat Khuzhin
44bf8ba568 Remove trailing whitespace in default client_name 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-08 08:09:47 +03:00
taiyang-li
dc4c638af8 Merge branch 'master' into optimize_trivial_hive_query 2022-04-08 11:36:55 +08:00
lgbo-ustc
759e4a9a05 Merge remote-tracking branch 'ck/master' into s3_cluster_join 2022-04-08 09:25:30 +08:00
helifu
1196dc3dbd Don't materialize external tables if it's explain statement 
The explain statement of GLOBAL JOIN two distributed tables can speed up 100x:
  explain plan select ... from t1_dist global join t2_dist on ...
  explain pipeline select ... from t1_dist global join t2_dist on ...
 2022-04-08 09:00:55 +08:00
kssenii
0b7af1f26c Fix checks 2022-04-08 01:58:55 +02:00
Yakov Olkhovskiy
64dcddc6e3 fixed ASTInterpolateElement::clone, fixed QueryNormalizer to exclude ASTInterpolateElement::children 2022-04-07 17:41:05 -04:00
Alexander Tokmakov
6a46da93ae Merge branch 'master' into mvcc_prototype 2022-04-07 23:22:19 +02:00
Alexey Milovidov
88134c86a5 Fix typo 2022-04-07 20:32:47 +02:00
kssenii
5dce2f18b5 Better 2022-04-07 18:48:35 +02:00
Alexander Tokmakov
2e2e2b9190 fix a race condition 2022-04-07 18:17:43 +02:00
Alexey Milovidov
ffb40c4ff4
Merge pull request #35867 from azat/client-format-error-fix 
Properly cancel the query after client format error
 2022-04-07 17:42:16 +02:00
Kseniia Sumarokova
5f9bc1c68e
Merge pull request #35987 from awakeljw/fork_chmaster 
[Improvement] improvement in PARTITION ALL
 2022-04-07 17:36:35 +02:00
mergify[bot]
f84dfe5073
Merge branch 'master' into client-format-error-fix 2022-04-07 14:15:44 +00:00
Sergei Trifonov
01425df758 fix build 2022-04-07 16:07:12 +02:00
Azat Khuzhin
33d99c8ffb Introduce compatiblity mode for SETTINGS after FORMAT 
Add allow_settings_after_format_in_insert setting, OFF by default.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
v2: s/parser_settings_after_format_compact/allow_settings_after_format_in_insert/ (suggested by vitlibar)
v3: replace ParserSettings with a flag (requested by vitlibar)
 2022-04-07 16:36:34 +03:00
Azat Khuzhin
c5ffbc688e Do not allow SETTINGS after FORMAT for INSERT queries 
Parsing SETTINGS after FORMAT, that has been introduced in [1], can
interpret SETTING as some values, which is misleading.

  [1]: https://github.com/ClickHouse/ClickHouse/pull/4174/files#diff-ba7bd0657630b1cd94cf6ed364bd857338096f49f66dc82918438d6745753775R106

Note, that we are touching only INSERT queries, not SELECT, since this
is a backward incompatible change, and in case of modifying SELECT it
can break too much.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Fixes: #35100
Fixes: #20343
 2022-04-07 16:29:36 +03:00
Alexander Tokmakov
7f54e7b422 Merge branch 'master' into mvcc_prototype 2022-04-07 15:14:06 +02:00
taiyang-li
87507ec9e8 fix conflicts 2022-04-07 20:52:54 +08:00
taiyang-li
d7c79c3a54 merge master and solve conflicts 2022-04-07 20:48:16 +08:00
taiyang-li
e9de38c52b fix bug 2022-04-07 20:45:07 +08:00
avogar
1c783ed88a Resolve conflicts 2022-04-07 12:17:48 +00:00
Kruglov Pavel
73adbb4c15
Merge pull request #35986 from amosbird/better-scalar1 
Fix performance regression of scalar query
 2022-04-07 14:07:59 +02:00
Amos Bird
a071bde59e
Use storage_snapshot for projection analysis 2022-04-07 19:56:31 +08:00
Alexander Tokmakov
8290ffa88d Merge branch 'master' into mvcc_prototype 2022-04-07 13:50:42 +02:00
avogar
d2017a63b1 Merge branch 'master' of github.com:ClickHouse/ClickHouse into improve-schema-inference 2022-04-07 11:36:40 +00:00
vdimir
eb7ac4c3a6
Handle empty expression list in cross join 2022-04-07 11:32:32 +00:00
Kruglov Pavel
f3f8f27db5
Merge pull request #35735 from Avogar/allow-read-bools-as-numbers 
Allow to infer and parse bools as numbers in JSON input formats
 2022-04-07 13:20:49 +02:00
vdimir
69160fa3e3
remove dead code in needRewrite function 2022-04-07 10:46:39 +00:00
Nikolai Kochetov
a998d73ee2
Update StorageKafka.cpp 2022-04-07 12:37:11 +02:00
Alexander Gololobov
42d4a84a6f More tests for corner cases 2022-04-07 12:34:26 +02:00
Kseniia Sumarokova
5909b3c04d
Merge pull request #36015 from ucasfl/filelog-fix 
Fix data race in StorgeFileLog
 2022-04-07 12:23:12 +02:00
Kseniia Sumarokova
18a594a22e
Merge pull request #34631 from bigo-sg/use_minmax_index 
Use minmax index for orc/parquet file in Hive Engine
 2022-04-07 12:22:19 +02:00
tavplubix
6e9c028bbb
Merge pull request #35956 from Algunenano/limit_dns_retries 
DNS cache: Add option to drop elements after several consecutive failures
 2022-04-07 13:20:36 +03:00
Nikolai Kochetov
3e1b3f14c0
Merge pull request #34355 from azat/processors-profiling 
Profiling on Processors level
 2022-04-07 12:13:14 +02:00
alesapin
d21ed546a4
Merge pull request #35967 from ClickHouse/keeper-polishing 
Small code changes in ZooKeeper client/Keeper code
 2022-04-07 12:11:10 +02:00
Sergei Trifonov
03f76dda1d better 2022-04-07 12:03:54 +02:00
Sergei Trifonov
755d5c55f9 resolve conflict 2022-04-07 11:57:45 +02:00
Amos Bird
1238babb6f
Make SelectQueryInfo pseudo-copyable 2022-04-07 17:46:50 +08:00
Amos Bird
69022850b9
Better 2022-04-07 17:46:50 +08:00
Amos Bird
9cf5935604
Fix broken SET reuse during projection analysis. 2022-04-07 17:46:49 +08:00
jewisliu
77edd41b2e [Improvement] improvement in PARTITION ALL 
1. ASTPartition::formatImpl should output ALL while executing ALTER TABLE t DETACH PARTITION ALL
2. prohibit PARTITION ALL excepte DETACH PARTITION ALL
 2022-04-07 17:37:01 +08:00
Sergei Trifonov
9c2804b924 merge from master + resolve conflicts 2022-04-07 11:22:01 +02:00
Kseniia Sumarokova
99a55a2304
Merge pull request #35995 from vdimir/rename_join_ordinary_to_atomic 
Create parent directories in DiskLocal::replaceFile
 2022-04-07 10:23:40 +02:00
taiyang-li
2dc420c66b rename some symbols in hivefile 2022-04-07 15:48:42 +08:00
taiyang-li
4763a39802 merge bigo-sg/use_minmax_index and solve conflict 2022-04-07 15:45:28 +08:00
taiyang-li
046a2ba51c rename some symboles 2022-04-07 15:35:08 +08:00
taiyang-li
ad074fee91 merge use_minmax_index and solve conflict 2022-04-07 15:19:45 +08:00
taiyang-li
f02d769343 fix build error 2022-04-07 14:29:35 +08:00
Yakov Olkhovskiy
7dbe8bc2dc major bugs fixed, tests added, docs updated 2022-04-07 01:21:24 -04:00
taiyang-li
acc7046d54 remove some useless virtual and rename some functions in HiveFile 2022-04-07 11:46:57 +08:00
taiyang-li
df00bd214d merge bigo-sg/use_minmax_index and solve conflict 2022-04-07 11:18:24 +08:00
taiyang-li
2ef316801c Merge branch 'master' into use_minmax_index 2022-04-07 10:53:25 +08:00
fenglv
517c2ae8df Fix data race in StorgeFileLog 2022-04-07 02:20:20 +00:00
Alexander Gololobov
81d150ed43 Implementation of makeDateTime() and makeDateTime64() 2022-04-07 00:30:18 +02:00
Meena Renganathan
645e156af6 Updated the boringssl-cmake to match the latest broingssl module update 2022-04-06 14:52:33 -07:00
mergify[bot]
585cff1e95
Merge branch 'master' into limit_dns_retries 2022-04-06 18:23:17 +00:00
Anton Popov
13cb564d1e
Merge pull request #35716 from kssenii/fix-local-logs-level 
Fix send_logs_level for clickhouse local
 2022-04-06 18:52:47 +02:00
tavplubix
58db8cd9b3
Merge pull request #35946 from ClickHouse/faster_replicated_ddl 
Make some replicated DDL queries faster
 2022-04-06 19:25:25 +03:00
Kseniia Sumarokova
2aa3d32887
Merge pull request #35980 from ucasfl/thread_num 
parallel reading files for FileLog Engine
 2022-04-06 15:05:41 +02:00
Kruglov Pavel
ec2213493f
Merge branch 'master' into allow-read-bools-as-numbers 2022-04-06 14:53:02 +02:00
Antonio Andelic
cda8a18fe3
Merge pull request #35984 from ClickHouse/fix-url-data-race 
Fix data race in StorageURL
 2022-04-06 14:36:19 +02:00
kssenii
846faa51d8 Fix bug 2022-04-06 14:35:19 +02:00
Yakov Olkhovskiy
c73115ffb4
Merge pull request #35840 from zzsmdfj/issue/#34966_fix_dateTime_deserialize 
to #34966_fix_dateTime_deserialize
 2022-04-06 08:10:23 -04:00
Alexander Tokmakov
1309e781b6 apply suggestion 2022-04-06 13:56:26 +02:00
Kruglov Pavel
9141066de3
Merge branch 'master' into improve-schema-inference 2022-04-06 13:51:07 +02:00
vdimir
4cb7b7e49b
Create parent directories in DiskLocal::replaceFile 2022-04-06 11:15:37 +00:00
taiyang-li
0b0c8ef09e add integration tests 2022-04-06 18:47:34 +08:00
Kseniia Sumarokova
d3763c4a62 Update ClientBase.h 2022-04-06 12:13:34 +02:00
Kseniia Sumarokova
a383d9ee6a
Merge pull request #35916 from filimonov/kafka_metrics 
Add some metrics to engine Kafka
 2022-04-06 11:58:17 +02:00
Kseniia Sumarokova
bb26903675
Merge pull request #35771 from bigo-sg/fix-filebuffer-pos 
fix filebuffer pos in RemoteReadBuffer
 2022-04-06 11:57:05 +02:00
Amos Bird
df06f9f974
Fix performance regression of scalar query 2022-04-06 17:50:22 +08:00
Vladimir C
2ebae2d722
Merge pull request #35682 from CurtizJ/dynamic-columns-6 2022-04-06 11:48:07 +02:00
Raúl Marín
060f5118f0 Improvements based on PR review 2022-04-06 10:48:12 +02:00
taiyang-li
acb9f1632e suppoort skip splits in orc and parquet 2022-04-06 16:40:22 +08:00
Kseniia Sumarokova
0e1637dfaf
Merge pull request #35354 from kssenii/enable-testmode-by-default 
Remove testmode option, enable it unconditionally
 2022-04-06 10:06:08 +02:00
Antonio Andelic
bf1f34ddb3 Fix unit tests 2022-04-06 07:43:48 +00:00
Antonio Andelic
84eef61d17 Pull under reader mutex 2022-04-06 06:39:56 +00:00
fenglv
a2ce366c34 parallel reading files for FileLog Engine 2022-04-06 04:49:43 +00:00
Yakov Olkhovskiy
ac441b9dcf compiler suggestions 2022-04-06 00:05:06 -04:00
taiyang-li
43e8af697a fix code style 2022-04-06 11:41:16 +08:00
kssenii
d59d4eda4f Fix tests 2022-04-06 01:58:16 +02:00
Yakov Olkhovskiy
ea9ce3ea18 'T' is added as delimiter, tests added 2022-04-05 19:50:16 -04:00
Mikhail Filimonov
f9a4cd12d4
Make parallel reading of kafka consumers work again (issue #35153) 2022-04-05 21:46:56 +02:00
Yakov Olkhovskiy
6b9a349083
Update SortDescription.h 
wrong merge fix
 2022-04-05 15:02:34 -04:00
Yakov Olkhovskiy
90c4cd3de7
Merge branch 'master' into interpolate-feature 2022-04-05 14:39:07 -04:00
Yakov Olkhovskiy
e0d6033c39 all columns can participate in interpolate expression despite if they are selected or not, some optimization on expressionless INTERPOLATE 2022-04-05 14:26:49 -04:00
Mikhail Filimonov
3412be9d4d
Change KafkaDirectReads to KafkaConsumersInUse 2022-04-05 20:25:05 +02:00
Nikolai Kochetov
def33323a0
Merge pull request #35788 from amosbird/better-projection1 
Improve minmax_count_projection
 2022-04-05 19:59:45 +02:00
kssenii
cbd7f4acfb Fix 2022-04-05 19:24:10 +02:00
Alexander Tokmakov
37a06eec1a fixes 2022-04-05 17:36:53 +02:00
Raúl Marín
39cc3baf5f Keep the previous exception code 2022-04-05 16:37:53 +02:00
Antonio Andelic
d296eeee2d Small changes for Keeper 2022-04-05 13:56:28 +00:00
kssenii
83488b2d13 Better 2022-04-05 14:48:26 +02:00
Alexander Tokmakov
d9e3e9b69f Merge branch 'master' into faster_replicated_ddl 2022-04-05 14:38:38 +02:00
Alexander Tokmakov
1fe50ad201 Merge branch 'master' into mvcc_prototype 2022-04-05 14:38:02 +02:00
Maksim Kita
c3c284e6e6
Merge pull request #35917 from kitaisreal/executable-user-defined-function-fix 
Executable user defined functions prevent executing during analysis
 2022-04-05 14:05:52 +02:00
Raúl Marín
8a05cf3927 DNS cache: Add option to drop elements after several consecutive failures 2022-04-05 13:00:27 +02:00
Azat Khuzhin
8ebaf84989 Fix clang-tidy readability-simplify-boolean-expr warning in AlterCommands.cpp 
I want to make the code more readable before, but clang-tidy is too
smart...

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-05 13:39:39 +03:00
Nikolai Kochetov
4479b68980
Merge pull request #35623 from nickitat/function_calculation_after_sorting_and_limit 
Functions calculation after sorting
 2022-04-05 12:09:56 +02:00
alesapin
eaa72ef002
Merge pull request #35904 from ClickHouse/more_logs_on_unsuccessful_part_removal 
More logs on unsuccessful part removal
 2022-04-05 11:29:08 +02:00
Maksim Kita
6e1d837439 Fixed style check 2022-04-05 11:24:33 +02:00
Alexander Tokmakov
da00beaf7f Merge branch 'master' into mvcc_prototype 2022-04-05 11:14:42 +02:00
Maksim Kita
b160ffd726
Merge pull request #35723 from ClickHouse/array-has-all-sse-avx2-optimizations 
Merging #27653
 2022-04-05 11:09:14 +02:00
Kseniia Sumarokova
77c7545395
Merge pull request #35922 from kssenii/fix-race-in-cached-buffer 
Fix race in cached buffer
 2022-04-05 10:55:58 +02:00
Azat Khuzhin
c6f182163a Require mutations for per-table TTL only when it had been changed 
Before this patch only per-column TTL did not requires mutation if it
had not been changed, after per-table TTL will also check whether it had
been changed or not.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-05 11:08:35 +03:00
Amos Bird
5bc09550d8
Fix tests 2022-04-05 15:56:39 +08:00
Amos Bird
35a8bb2a9b
add comment 2022-04-05 15:56:38 +08:00
Amos Bird
163664fad7
Improve minmax_count_projection 2022-04-05 15:56:37 +08:00
Vladimir C
54b4049ea5
Merge pull request #35794 from awakeljw/fork_chmaster 2022-04-05 09:44:45 +02:00
Alexander Tokmakov
4e9ec5dc2f make some replicated DDL faster 2022-04-05 00:51:48 +02:00
kssenii
3fd462bc05 Merge master 2022-04-05 00:06:18 +02:00
Alexey Milovidov
9d5c13c71d
Merge pull request #35886 from azat/format-insert-infile-fix 
Fix formatting of INSERT INFILE queries (missing quotes)
 2022-04-05 00:47:54 +03:00
larryluogit
808d9afd0f
Fix optin.cplusplus.UninitializedObject issue (#35626) 
* Fix optin.cplusplus.UninitializedObject issue

* Enable optin.cplusplus.UninitializedObject
 2022-04-04 23:47:14 +03:00
Alexey Milovidov
1b562923f9
Merge pull request #35941 from ClickHouse/revert-35184-clang-tidy-issues 
Revert "clang-tidy report issues with Medium priority"
 2022-04-04 23:42:21 +03:00
alesapin
69fceb7e6d
Merge pull request #35926 from ClickHouse/better_consumers 
Better check for kafka_num_consumers
 2022-04-04 22:42:05 +02:00
Alexey Milovidov
4d6c030d23
Revert "clang-tidy report issues with Medium priority" 2022-04-04 23:41:42 +03:00
kssenii
79627798c4 Fix race 2022-04-04 20:44:39 +02:00
Mikhail Filimonov
53c7376e37
Add some metrics to engine Kafka 2022-04-04 20:25:53 +02:00
alesapin
d04c48e67a Fix build 2022-04-04 20:14:09 +02:00
alesapin
1e4fe5e3b5
Merge pull request #35882 from ClickHouse/more_metadata_alters 
Make more alters of nested types metadata-only
 2022-04-04 20:11:31 +02:00
Alexander Gololobov
f0de8eb625 Extracted argument handling into a separate class to reuse it for makeDateTime() and makeDateTime64() 2022-04-04 19:57:04 +02:00
alesapin
0477e74f42 Get rid of caps 2022-04-04 19:41:54 +02:00
alesapin
72331856eb fix message 2022-04-04 18:28:26 +02:00
alesapin
86f42e7a3a Better check for kafka_num_consumers 2022-04-04 18:07:36 +02:00
Kseniia Sumarokova
d697576967
Update ReadBufferFromS3.h 2022-04-04 17:53:01 +02:00
Vladimir C
b80f24957e
Merge pull request #35674 from vdimir/asof-key-null-issue-35565 2022-04-04 16:56:04 +02:00
Anton Popov
bd93be65aa
Merge pull request #35820 from azat/columns-ttl-fix 
Avoid processing per-column TTL multiple times
 2022-04-04 16:42:50 +02:00
kssenii
6c6fb5c3e8 Fix race in cached buffer 2022-04-04 16:32:37 +02:00
Anton Popov
d08d4a2437
Merge pull request #35806 from CurtizJ/dynamic-columns-8 
Fix inserts to columns of type `Object` in partitioned tables
 2022-04-04 16:16:54 +02:00
kssenii
4e50da722b Better s3 settings 2022-04-04 16:14:56 +02:00
Maksim Kita
2a8e479277 ExecutableUserDefinedFunction prevent function execution during query analysis 2022-04-04 15:56:01 +02:00
Antonio Andelic
db75bf6f5d
Merge pull request #35852 from ClickHouse/bigo-sg-add_hints 
Refactoring of hints for column descriptor
 2022-04-04 15:37:08 +02:00
alesapin
3ccf99c3d7
Merge pull request #35884 from ClickHouse/don_t_allow_to_reset_settings 
Forbid to reset non existing settings
 2022-04-04 15:27:34 +02:00
Nickita Taranov
0f94a58f3a use getName() 2022-04-04 14:59:38 +02:00
Nickita Taranov
698a984c07 throw if sorting column not found 2022-04-04 14:33:58 +02:00
Nickita Taranov
440e57769a more fizes 2022-04-04 14:33:58 +02:00
Nickita Taranov
ce40d84eef more fixes 2022-04-04 14:33:58 +02:00
Nickita Taranov
a39427f00b clean up 2022-04-04 14:33:57 +02:00
Nickita Taranov
eedcd61479 fix 2022-04-04 14:33:57 +02:00
Nickita Taranov
a08c035443 stash 2022-04-04 14:33:57 +02:00
Nickita Taranov
b095838444 stash 2022-04-04 14:33:57 +02:00
Nickita Taranov
4c51329ad6 stash 2022-04-04 14:33:57 +02:00
Alexander Tokmakov
a2167f12b8 Merge branch 'master' into mvcc_prototype 2022-04-04 14:24:23 +02:00
Nikita Taranov
bd89fcafdb
Make SortDescription::column_name always non-empty (#35805) 2022-04-04 14:17:15 +02:00
Maksim Kita
47528de78b Fix build 2022-04-04 14:07:05 +02:00
alesapin
9b75ef6ce9 Fix build 2022-04-04 14:03:16 +02:00
何李夫
09c04e4993
Improve the pipeline description for JOIN (#35612) 
Improve the pipeline description for JOIN
 2022-04-04 13:56:41 +02:00
Maksim Kita
af405d3ba6 Fixed style check 2022-04-04 13:34:27 +02:00
alesapin
885447342c More logs on unsuccessful part removal 2022-04-04 13:17:33 +02:00
Maksim Kita
e6c9a36ac7
Merge pull request #35733 from kitaisreal/ipv6-invalid-insert-test 
Added test for insert of invalid IPv6 value
 2022-04-04 12:28:16 +02:00
Azat Khuzhin
e5a837e574 Add id/parent_ids into system.processors_profile_log 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-04 13:19:43 +03:00
Azat Khuzhin
58ee917e94 Mesure processors profiles only if it was enabled 
Since it may use little extra CPU.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-04 13:19:43 +03:00
Azat Khuzhin
99528e296c Rename need_data_elapsed_us/port_full_elapsed_us to input_wait_us/output_wait_us 
$ gg -e need_data_ -e port_full_  | cut -d: -f1 | sort -u | xargs sed -i -e s/port_full_/output_wait_/g -e s/need_data_/input_wait_/g -e s/getPortFull/getOutputWait/g -e s/getNeedData/getInputWait/g

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-04 13:19:43 +03:00
Azat Khuzhin
765b4bc45a Implement log_processors_profiles (write to processors_profile_log, OFF by default) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-04 13:19:43 +03:00
Azat Khuzhin
5fd402eaba Measure time that Process spent in work/NeedData/PortFull 
Note, that right now it is done not in IProcessor, but in
ExecutingGraph/ExecutionThreadContext, to avoid lots of changes in the
IProcessor interface, to make review easier, but I'm not against of
change the IProcessor interface to incapsulate it there.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-04 13:19:43 +03:00
Azat Khuzhin
14538f6456 Add system.processors_profile_log 
This is the system table that will contain Processors level profiling.

v2: one entry per Processor, not 3 (PortFull/NeedData/work())
v3: us over ms
v4: Enable processors_profile_log table by default

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-04 13:19:41 +03:00
Nikolai Kochetov
19819c72f8
Merge pull request #35290 from bigo-sg/function_enumerate_streams 
Add function getTypeSerializationStreams
 2022-04-04 12:09:53 +02:00
alesapin
803a1a2a9c Fix tests and check 2022-04-04 11:19:02 +02:00
Anton Popov
11e18a16f3
Merge pull request #35724 from Avogar/fix-order 
Improve schema inference for JSONEachRow and TSKV formats
 2022-04-04 11:00:21 +02:00
kssenii
36c583d0de Better version of cache on insert 2022-04-04 10:58:59 +02:00
Anton Popov
ae53aae106 fix clang-tidy 2022-04-04 08:48:31 +00:00
Azat Khuzhin
93bbe9641a Fix formatting of INSERT ... COMPRESSION 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-04 10:33:12 +03:00
Azat Khuzhin
9a76efb850 Fix formatting of INSERT INFILE queries (missing quotes) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-04 10:33:12 +03:00
taiyang-li
38f149b533 optimize trivial count hive query 2022-04-04 15:28:26 +08:00
Antonio Andelic
a926bc19ea Address PR comments 2022-04-04 07:24:42 +00:00
Antonio Andelic
d96b682a55 Refactor 2022-04-04 07:24:42 +00:00
taiyang-li
6bc1786047 fix style 2022-04-04 07:24:42 +00:00
taiyang-li
d6247338de fix failed stateless tests 2022-04-04 07:24:42 +00:00
taiyang-li
eda299b48b fix building 2022-04-04 07:24:42 +00:00
taiyang-li
fd9a10ef53 add hints for projections 2022-04-04 07:24:42 +00:00
taiyang-li
4547ed370a add hints for column description 2022-04-04 07:24:42 +00:00
Alexey Milovidov
d9e5ca2119
Merge pull request #34394 from holadepo/last_day 
Add toLastDayOfMonth function
 2022-04-04 07:02:08 +03:00
taiyang-li
4e2d5f1841 Merge remote-tracking branch 'bigo-sg/use_minmax_index' into optimize_trivial_hive_query 2022-04-04 10:42:28 +08:00
taiyang-li
cbfc0f6bac fix typo 2022-04-04 10:42:22 +08:00
Yakov Olkhovskiy
ff4d295265 style fix 2022-04-03 22:19:35 -04:00
Yakov Olkhovskiy
95ad1bf6e1 use aliases if exist for original_select_set 2022-04-03 22:10:36 -04:00
alesapin
a8c1ccdb20 Forbit to reset non existing settings 2022-04-03 18:29:24 +02:00
Kseniia Sumarokova
30c1afe29d
Merge pull request #35646 from bigo-sg/reduce-mutex-scope 
narrow mutex scope when setenv LIBHDFS3_CONF
 2022-04-03 17:59:24 +02:00
alesapin
7b35920d4c Make more alters of nested types metadata-only 2022-04-03 14:03:34 +02:00
alesapin
91453fe4d6
Merge pull request #35759 from chen9t/fix_dirty_metacache 
Clean dirty meta cache when part is broken during part loading phase
 2022-04-03 13:33:46 +02:00
Azat Khuzhin
3ce4844d4b Properly cancel the query after client format error 
In case of format error (i.e. select 2 format Template settings
format_template_row='/dev/null') the client will reset the connection,
since it will not expect Data blocks.

To fix this, catch this client error and properly cancel the query. This
should also fix query hang checks (the one that executed after each test).

v2: use getCurrentExceptionMessage()/getCurrentExceptionCode()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-03 14:01:16 +03:00
Alexey Milovidov
5a47958744
Merge pull request #35736 from CurtizJ/quota-written-bytes 
Add quota for written bytes
 2022-04-03 05:26:49 +03:00
Azat Khuzhin
195b4c47ea Avoid processing per-column TTL multiple times 
Before this patch ttl.txt will not be written for per-column TTLs, and
hence it will be calculated every time after server restart of
DETACH/ATTACH cycle (note, that it will work w/o restart since in-memory
representation will avoid this).

v2: convert test to .sh to get correct current database over default for MV
v3: extract UUID to avoid error like in [1]:
    [ 490 ] {} <Error> void DB::SystemLog<DB::TextLogElement>::flushImpl(const std::vector<LogElement> &, uint64_t) []: Code: 349. DB::Exception: Cannot convert NULL value to non-Nullable type: While processing query_id LIKE concat('%', CAST(_CAST(NULL, 'Nullable(UUID)') AS uuid, 'String'), '%'): while pushing to view test_0hc2ro.this_text_log (c64e5af4-059e-4330-a728-354ecf83c031). (CANNOT_INSERT_NULL_IN_ORDINARY_COLUMN)
    [1]: https://s3.amazonaws.com/clickhouse-test-reports/35820/a512d322b024d37d2f1082c4833f59f86057555f/stateless_tests_flaky_check__address__actions_.html
v4: add no-parallel to avoid issues with disappeared underlying table while pushing to text_log
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-02 15:19:39 +03:00
mergify[bot]
1e43e26fa1
Merge branch 'master' into fix-order 2022-04-02 12:00:29 +00:00
Kruglov Pavel
687e9e7aeb
Merge pull request #35760 from Avogar/insert-from-input 
Improve schema inference for insert select from table functions
 2022-04-02 13:50:48 +02:00
Yakov Olkhovskiy
ec0ad8804a style fix 2022-04-01 21:45:58 -04:00
shuchaome
a565a93740 reduce mutex scope when setenv LIBHDFS3_CONF 2022-04-02 06:52:39 +08:00
Yakov Olkhovskiy
0116233d36 allow INTERPOLATE to reference optimized out columns 2022-04-01 16:18:19 -04:00
Anton Popov
a10bf550c4
Update AsynchronousInsertQueue.cpp 2022-04-01 20:57:41 +02:00
tavplubix
97f29ac0fe
Merge pull request #35817 from tcoyvwac/fix/exception-in-destructor-needs-noexcept-false 
Fix missing `noexcept(false)` flag on destructor which causes builds to fail
 2022-04-01 20:35:03 +03:00
Anton Popov
687942ce70 more strict quota for written bytes 2022-04-01 15:02:49 +00:00
Kseniia Sumarokova
fe8c91541d
Merge pull request #35833 from ucasfl/context 
Try remove unneed variable
 2022-04-01 16:58:42 +02:00
Yakov Olkhovskiy
38993f215f
Merge pull request #35706 from yakov-olkhovskiy/ephemeral-35641 
Allow EPHEMERAL without explicit default expression
 2022-04-01 10:49:29 -04:00
kssenii
6c8e073a61 Merge master 2022-04-01 16:48:29 +02:00
tcoyvwac
cb698c4cc1
Replace throw in debug mode in destructor 
FileSegmentsHolder:
- Replaced throw in destructor with assert for all #ifdef modes.
 2022-04-01 15:42:34 +02:00
Alexander Tokmakov
6591d1ceb7 Merge branch 'master' into mvcc_prototype 2022-04-01 15:38:46 +02:00
Antonio Andelic
5bcf772d7c
Merge pull request #35789 from amosbird/better-pipeline1 
Refactoring QueryPipeline
 2022-04-01 14:38:19 +02:00
zzsmdfj
3cae0c74d9 to #34966_fix_dateTime_deserialize 2022-04-01 20:16:07 +08:00
zzsmdfj
ececee3817 to #34966_fix_dateTime_deserialize 2022-04-01 20:13:34 +08:00
Anton Popov
075bcf3f5f
Merge pull request #35803 from Avogar/fix-formats-with-names 
Fix bug in indexes of not presented columns in -WithNames formats
 2022-04-01 13:33:35 +02:00
fenglv
97523a3329 Try remove unneed variable 2022-04-01 09:58:23 +00:00
Kseniia Sumarokova
cafff71d2f
Merge pull request #35799 from evillique/extract-parser-fix 
Fix extract function parser
 2022-04-01 10:59:39 +02:00
alesapin
f2c6387a8d
Merge pull request #35381 from kssenii/remote-filesystem-cache-fixes 
Cache fixes for high concurrency on corner cases
 2022-04-01 10:48:31 +02:00
Kseniia Sumarokova
d3b3294872
Merge pull request #35365 from bigo-sg/improve_access_type 
Improve check access in table functions
 2022-04-01 10:47:02 +02:00
Kseniia Sumarokova
9c39320e75
Merge pull request #35800 from ucasfl/mutex 
remove unused variable
 2022-04-01 10:46:23 +02:00
Kseniia Sumarokova
04729ecc70
Merge pull request #35798 from xiedeyantu/patch-1 
Delete duplicate code
 2022-04-01 10:43:31 +02:00
mergify[bot]
33115a55c3
Merge branch 'master' into better-pipeline1 2022-04-01 06:44:43 +00:00
Antonio Andelic
8cc18c851d
Merge pull request #35705 from ClickHouse/is-secure-client-info 
Added `is_secure` column to `query_log`
 2022-04-01 08:27:20 +02:00
jewisliu
9a4cc78dfb support ALTER TABLE t DETACH PARTITION ALL syntax 2022-04-01 13:13:07 +08:00
Alexey Milovidov
5c6b84c398
Merge pull request #35586 from kssenii/fix-virtual-columns-s3 
Fix s3 engine getting virtual columns
 2022-04-01 04:43:32 +03:00
Alexey Milovidov
475dec3809
Merge pull request #35814 from ClickHouse/merge_tree_arguments_error 
Better error message for incorrect *MergeTree arguments
 2022-04-01 03:04:43 +03:00
kssenii
603dcbb1b7 Extend test 2022-03-31 18:23:15 +02:00
Yakov Olkhovskiy
538373a79b style fix 2022-03-31 12:13:49 -04:00
Yakov Olkhovskiy
a15996315e bugfix - columns order tracking 2022-03-31 11:51:13 -04:00
filimonov
24d72a18d0
Update getNumberOfPhysicalCPUCores.cpp 
In Kubernetes shares are calculated from resources.requests.cpu, quota & period - from limits.

When you configure only requests without limits - the container still can use all the cores. So you see all the cores the system has.
(because of requests the pod just can't be scheduled on the node which have less cpus)
So the effective CPU count is all the available cores if you don't have limits, or calculated from quota & period if you have the limits.
 2022-03-31 17:21:54 +02:00
Alexander Tokmakov
63e8d73ba4 better message for incorrect *MergeTree arguments 2022-03-31 17:08:30 +02:00
Antonio Andelic
1d480178c4
Merge pull request #35785 from justnoise/remove-atomic-flag-init 
remove ATOMIC_FLAG_INIT: depreciated in C++20 and warns in clang-14
 2022-03-31 16:44:56 +02:00
avogar
ab2a963287 Merge branch 'master' of github.com:ClickHouse/ClickHouse into allow-read-bools-as-numbers 2022-03-31 14:09:43 +00:00
Kruglov Pavel
252d66e80d
Update src/Processors/Formats/ISchemaReader.cpp 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2022-03-31 16:08:37 +02:00
mergify[bot]
24ade25d61
Merge branch 'master' into improve-schema-inference 2022-03-31 13:42:47 +00:00
Yakov Olkhovskiy
0367cbad3c messed merge fixed 2022-03-31 09:35:19 -04:00
Anton Popov
42acb1dc29 fix inserts to columns of type Object in partitioned tables 2022-03-31 13:30:01 +00:00
Yakov Olkhovskiy
b5682c1f02 minor refactoring 2022-03-31 08:33:50 -04:00
Kruglov Pavel
a217e9e9d0
Remove unused header 2022-03-31 14:27:43 +02:00
mergify[bot]
3183b61c74
Merge branch 'master' into mvcc_prototype 2022-03-31 12:27:09 +00:00
avogar
836e7dae67 Fix bug in indexes of not presented columns in -WithNames formats 2022-03-31 12:24:40 +00:00
Nikita Mikhaylov
5f27f690f4 Revert "Added support for schema inference for hdfsCluster (#35602)" 
This reverts commit f6bfdcc0c9.
 2022-03-31 14:19:56 +02:00
fenglv
e2e4b02d13 remove unused variable 2022-03-31 12:01:10 +00:00
alesapin
bd2ab32e3f
Merge pull request #35772 from azat/keeper-eof 
clickhouse-keeper: correctly handle closed client connection
 2022-03-31 13:57:28 +02:00
Alexander Tokmakov
461ecd56a4 fix race when database is Ordinary 2022-03-31 13:47:38 +02:00
Nikolay Degterinsky
e022f8527b Fix extract parser 2022-03-31 11:42:35 +00:00
xiedeyantu
d141dbc388
Delete duplicate code 
Delete duplicate code
 2022-03-31 19:34:30 +08:00
Amos Bird
12449caf89
Refactoring QueryPipeline 2022-03-31 19:26:35 +08:00
avogar
d272356324 Minor code improvement 2022-03-31 10:55:09 +00:00
avogar
74275da7ee Make better 2022-03-31 10:52:34 +00:00
Kruglov Pavel
564a77c462
Fix build 2022-03-31 12:49:23 +02:00
Nikita Mikhaylov
f6bfdcc0c9
Added support for schema inference for hdfsCluster (#35602) 2022-03-31 12:47:36 +02:00
alesapin
e12860f646
Update src/Disks/IO/CachedReadBufferFromRemoteFS.cpp 2022-03-31 11:42:42 +02:00
alesapin
4c66c2e10f
Update src/Disks/IO/CachedReadBufferFromRemoteFS.cpp 2022-03-31 11:42:38 +02:00
Alexander Tokmakov
5a50ad9de3 Merge branch 'master' into mvcc_prototype 2022-03-31 11:35:04 +02:00
alesapin
2365bbf0d3
Merge pull request #34566 from ClickHouse/startup-sanity-checks 
add simple sanity checks for server startup
 2022-03-31 11:33:59 +02:00
Kseniia Sumarokova
a711fc0402
Merge pull request #35572 from bigo-sg/hdfs_config_prefix 
Deduce absolute hdfs config path
 2022-03-31 10:08:32 +02:00
Kseniia Sumarokova
0b20329156
Merge pull request #35727 from kssenii/fix-any-all 
Fix any/all(subquery)
 2022-03-31 10:05:37 +02:00
Antonio Andelic
e336302650 Merge branch 'master' into is-secure-client-info 2022-03-31 06:58:16 +00:00
Antonio Andelic
049559d510 Remove extra newline 2022-03-31 06:57:59 +00:00
Antonio Andelic
13b4fabee4
Merge pull request #35743 from bigo-sg/remote_filter_for_hive 
Respect remote_url_allow_hosts for hive
 2022-03-31 08:54:37 +02:00
Antonio Andelic
cdedcee0ce
Merge pull request #35746 from ClickHouse/use-common-io-thread-pool 
Use common IOThreadPool for S3 and URL
 2022-03-31 08:41:54 +02:00
Azat Khuzhin
ff0e46c394 clickhouse-keeper: finish session on client EOF (fixes heap-use-after-free) 
CI founds [1]:

    WARNING: ThreadSanitizer: heap-use-after-free (pid=1)
      Read of size 8 at 0x7b48001e0088 by thread T7 (mutexes: write M2588):
        ...
        1 DB::KeeperTCPHandler::runImpl()::$_0::operator()() const build_docker/../src/Server/KeeperTCPHandler.cpp:384:14 (clickhouse+0x194f974f)
        ...
        8 DB::KeeperDispatcher::setResponse() build_docker/../src/Coordination/KeeperDispatcher.cpp:222:9 (clickhouse+0x1999de82)
        9 DB::KeeperDispatcher::responseThread() build_docker/../src/Coordination/KeeperDispatcher.cpp:161:18 (clickhouse+0x1999d7a7)
        10 DB::KeeperDispatcher::initialize()::$_1::operator()() const build_docker/../src/Coordination/KeeperDispatcher.cpp:273:54 (clickhouse+0x199a55d1)

      Previous write of size 8 at 0x7b48001e0088 by thread T3:
        0 operator delete(void*, unsigned long) <null> (clickhouse+0xa0dad50)
        1 DB::KeeperTCPHandler::~KeeperTCPHandler() build_docker/../src/Server/KeeperTCPHandler.cpp:648:1 (clickhouse+0x194f8ee6)

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/35772/38873d726d22b15d4c5a284410956927b2e2f95f/integration_tests__thread__actions__[4/4].html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-31 09:31:34 +03:00
Brendan Cox
87fff8b2d3 remove ATOMIC_FLAG_INIT: depreciated in C++20 and warns in clang-14 2022-03-30 21:45:48 -07:00
taiyang-li
539ce5f1b8 Merge branch 'master' into use_minmax_index 2022-03-31 12:27:32 +08:00
taiyang-li
a5765dccb1 Merge branch 'master' into function_enumerate_streams 2022-03-31 12:21:00 +08:00
taiyang-li
70087bc959 Merge branch 'master' into improve_access_type 2022-03-31 12:19:17 +08:00
taiyang-li
f6ccda8097 Merge branch 'master' into hdfs_config_prefix 2022-03-31 12:16:05 +08:00
Yakov Olkhovskiy
3b40b56e56 refactored to remove parsers rependency on datatypes 2022-03-30 23:47:16 -04:00
taiyang-li
c11f563d47 Merge branch 'master' into remote_filter_for_hive 2022-03-31 10:54:55 +08:00
chen9t
b40a69f59a Code style 2022-03-31 10:10:05 +08:00
Nikolay Degterinsky
f055d7b692
Merge pull request #34055 from palegre-tiny/groupSortedArray 
Add groupSortedArray() function
 2022-03-31 01:20:15 +03:00
alesapin
81ce991494 Merge branch 'master' into startup-sanity-checks 2022-03-31 00:08:43 +02:00
Kruglov Pavel
4ec3c35e14
Merge pull request #35755 from Avogar/fix-custom-to-string 
Fix bug in conversion from custom types to string
 2022-03-31 00:06:48 +02:00
Nikita Taranov
30f2a942c5
Predict size of hash table for GROUP BY (#33439) 
* use AggregationMethod ctor with reserve

* add new settings

* add HashTablesStatistics

* support queries with limit

* support distributed and with external aggregation

* add new profile events

* add some tests

* add perf test

* export cache stats through AsynchronousMetrics

* rm redundant trace

* fix style

* fix 02122_parallel_formatting test

* review fixes

* fix 02122_parallel_formatting test

* apply also to two-level HTs

* try simpler strategy

* increase max_size_to_preallocate_for_aggregation for experiment

* fixes

* Revert "increase max_size_to_preallocate_for_aggregation for experiment"

This reverts commit 6cf6f75704.

* fix test

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2022-03-30 22:47:51 +02:00
Yakov Olkhovskiy
6a1e116c46 refactoring 2022-03-30 16:34:19 -04:00
Azat Khuzhin
38873d726d clickhouse-keeper: fix implementation of server with poll() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-30 21:30:12 +03:00
kssenii
98e85dd68e Fix build 2022-03-30 20:27:04 +02:00
kssenii
ac4897f4f9 Fix 2022-03-30 20:16:41 +02:00
Azat Khuzhin
9820230e95 clickhouse-keeper: correctly handle closed client connection 
This will avoid noisy message like:

    DB::Exception: Cannot read all data. Bytes read: 0. Bytes expected: 4.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-30 20:29:04 +03:00
Maksim Kita
e43fdcd7eb Function hasAll added dynamic dispatch for SSE4.2, AVX2 2022-03-30 18:41:34 +02:00
Maksim Kita
8d0a9689e4 Update gatherutils CMakeLists to use X86_INTRINSICS_FLAGS from cpu_features 2022-03-30 18:40:18 +02:00
Maksim Kita
91eec8962f Rename test 2022-03-30 18:39:28 +02:00
shuchaome
738966b6b6 fix filebuffer pos in RemoteReadBuffer 2022-03-31 00:03:45 +08:00
kssenii
c1ae5814a7 Merge master 2022-03-30 17:59:36 +02:00
Alexander Tokmakov
996d57b918 try fix build with gcc 2022-03-30 17:42:47 +02:00
kssenii
099fa0e668 Merge master 2022-03-30 17:33:32 +02:00
kssenii
0fc92fe2aa Get rid of all "remote" mentions 2022-03-30 17:15:29 +02:00
mergify[bot]
4813f0b305
Merge branch 'master' into is-secure-client-info 2022-03-30 15:12:56 +00:00
kssenii
b30948dd53 Relax test 2022-03-30 17:12:23 +02:00
Kruglov Pavel
771f34d793
Merge pull request #35691 from Avogar/fix-matview 
Fix creating materialized view with subquery after server restart
 2022-03-30 14:57:14 +02:00
avogar
849596c7a2 Improve schema inference for insert select queries 2022-03-30 12:44:53 +00:00
chen9t
5cb2301e39 Clean dirty meta cache when part is broken during part loading phase 2022-03-30 20:40:20 +08:00
Antonio Andelic
7bc5731025 Merge branch 'master' into is-secure-client-info 2022-03-30 11:26:05 +00:00
Antonio Andelic
a3ad99be03 Create CallbackRunner in Storage 2022-03-30 11:24:50 +00:00
avogar
af4bfec051 Fix bug in conversion from custom types to string 2022-03-30 11:19:03 +00:00
Maksim Kita
371cdc956a Added input format settings for parsing invalid IPv4, IPv6 addresses as default values 2022-03-30 12:54:19 +02:00
Vladimir C
31c367d3cd
Merge pull request #35651 from amosbird/columntransformerrename 2022-03-30 12:37:30 +02:00
kssenii
98ad3f4a91 Fix tests, rename some remote mentions 2022-03-30 12:21:59 +02:00
lgbo-ustc
08e1fdc6b8 Fixed :a elimate AMBIGUOUS_COLUMN_NAME exception 
elimate AMBIGUOUS_COLUMN_NAME exception in hdfs/s3 cluster when run a query with join
 2022-03-30 17:06:38 +08:00
Antonio Andelic
de2c6684a3 Remove setup and cleanup 2022-03-30 09:00:06 +00:00
Antonio Andelic
1173eab15e Use same IOThreadPool for URL and S3 2022-03-30 08:15:20 +00:00
taiyang-li
16bb4c4ad0 respect remote_url_allow_hosts for hive 2022-03-30 15:33:59 +08:00
Antonio Andelic
e15fccd31b Add is_secure info to HTTP 2022-03-30 06:39:40 +00:00
Antonio Andelic
d14ad227b1 Polish TCP is_secure flag 2022-03-30 06:39:40 +00:00
Antonio Andelic
b982959c4c Fix writting is_secure 2022-03-30 06:39:40 +00:00
Antonio Andelic
893b8a9513 Initial implementation of is_secure 2022-03-30 06:39:40 +00:00
Antonio Andelic
03a09dc914
Merge pull request #35571 from ClickHouse/parallel-s3-downloading 
Parallel S3 downloading
 2022-03-30 08:14:49 +02:00
Antonio Andelic
d85ed8f2a9
Merge pull request #35655 from ClickHouse/exception-compile-time-message-check 
Use compile-time check for `Exception` messages
 2022-03-30 08:11:32 +02:00
taiyang-li
0af6fdb576 fix building 2022-03-30 11:28:21 +08:00
taiyang-li
fcfabf4a0c Merge remote-tracking branch 'ck/master' into use_minmax_index 2022-03-30 11:23:24 +08:00
taiyang-li
47f3e9330e merge master and fix conflict 2022-03-30 11:06:51 +08:00
taiyang-li
e76ed3acde Merge branch 'master' into improve_access_type 2022-03-30 11:04:57 +08:00
taiyang-li
9ffb42d507 Merge branch 'master' into hdfs_config_prefix 2022-03-30 11:02:31 +08:00
Anton Popov
a842a81aba
Merge pull request #35690 from CurtizJ/flatten-tuple 
Add function `flattenTuple`
 2022-03-30 00:24:36 +02:00
Anton Popov
1cba31c305
Merge pull request #35687 from CurtizJ/dynamic-columns-7 
Fix reading from type object
 2022-03-30 00:21:32 +02:00
Anton Popov
caacc7d385 add quota for written bytes 2022-03-29 18:21:29 +00:00
kssenii
ce9131f905 Fix unit test 2022-03-29 19:57:55 +02:00
kssenii
f0e0d97752 Fix lask of query id in merges 2022-03-29 19:49:42 +02:00
avogar
000f3043e7 Make better 2022-03-29 17:40:07 +00:00
avogar
3fc36627b3 Allow to infer and parse bools as numbers in JSON input formats 2022-03-29 17:37:31 +00:00
kssenii
a2e950bd3b Fix 2022-03-29 17:34:50 +02:00
kssenii
421b1e5a81 Update 2022-03-29 17:33:02 +02:00
avogar
ce97ccbfb9 Improve schema inference for JSONEachRow and TSKV formats 2022-03-29 14:47:51 +00:00
Maksim Kita
2742b88e6c
Merge pull request #27653 from ContentSquare/hasAllAny_SIMD 
Implement HasAll specialization for SSE and AVX2
 2022-03-29 16:35:59 +02:00
Kruglov Pavel
a2fd09e031
Fix style 2022-03-29 16:34:07 +02:00
Alexander Tokmakov
287d858fda Merge branch 'master' into mvcc_prototype 2022-03-29 16:24:12 +02:00
Dmitry Novik
8f935a72d6
Merge pull request #33230 from CurtizJ/read_in_order_max_rows_to_read 
Proper handle of 'max_rows_to_read' in case of reading in order of sorting key
 2022-03-29 15:16:34 +02:00
Antonio Andelic
ef484547bf Fix GCC build 2022-03-29 13:16:11 +00:00
Antonio Andelic
9990abb76a Use compile-time check for Exception messages, fix wrong messages 2022-03-29 13:16:11 +00:00
Yakov Olkhovskiy
46fd69d2bf
Update ParserCreateQuery.h 
style fix
 2022-03-29 09:15:27 -04:00
avogar
97f5033ea9 Fix tests 2022-03-29 13:07:37 +00:00
mergify[bot]
343588de2c
Merge branch 'master' into improve-schema-inference 2022-03-29 13:06:00 +00:00
taiyang-li
67c3c0be3d Merge branch 'master' into improve_access_type 2022-03-29 20:42:57 +08:00
taiyang-li
631352ba4b Merge branch 'master' into hdfs_config_prefix 2022-03-29 20:40:32 +08:00
alesapin
842c0b8ff0 Merge remote-tracking branch 'origin/startup-sanity-checks' into startup-sanity-checks 2022-03-29 14:29:33 +02:00
kssenii
4374415bd7 Fix clang-tidy, style check 2022-03-29 14:20:21 +02:00
Anton Popov
9610139477
Merge pull request #35629 from CurtizJ/dynamic-columns-5 
Support schema inference for type `Object` in format `JSONEachRow`
 2022-03-29 14:17:09 +02:00
alesapin
ed2c461fe1 Merge branch 'master' into startup-sanity-checks 2022-03-29 14:09:20 +02:00