thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-14 02:12:21 +00:00
Code Issues Packages Projects Releases Wiki Activity
82,698 Commits 9,330 Branches 1,783 Tags 2.1 GiB
8076e1ef30
Commit Graph

22603 Commits

Author SHA1 Message Date
alexey-milovidov
798e0e8242
Merge pull request #34205 from ClickHouse/format-regexp-allow-cr-in-the-middle 
Format `Regexp`: allow `\r` in the middle
 2022-02-01 16:22:02 +03:00
Vitaly Baranov
1f81e43f1e Fix checking grants for SHOW GRANTS. 2022-02-01 19:55:24 +07:00
Anton Popov
c28255850a fix metric Query 2022-02-01 15:06:49 +03:00
Azat Khuzhin
d0d71180dd Add fmt::runtime in gtest_log 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-01 14:44:47 +03:00
Vitaly Baranov
2ef2479ddc
Add const to make clang-tidy happy 2022-02-01 18:38:52 +07:00
Azat Khuzhin
de80014eb5 Use fmt::format over Poco::format in PostgreSQLHandler 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-01 14:30:04 +03:00
Azat Khuzhin
5dfafd68a7 ReplicatedMergeTreeQueue: Fix fmt:: and reduce copy-paste of logging and out reason 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-01 14:30:04 +03:00
Azat Khuzhin
5be76bc969 Use proper fmt-like logging 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-01 14:30:04 +03:00
Azat Khuzhin
3b3635c6d5 Fix formatting error in logging messages 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-01 14:30:04 +03:00
Azat Khuzhin
743096a883 Use proper fmt:: like Exception ctor in DataPartsExchange 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-01 14:30:03 +03:00
Azat Khuzhin
cf93732de9 DatabaseSQLite: fix fmt error in DatabaseSQLite::checkSQLiteTable() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-01 14:30:03 +03:00
Azat Khuzhin
bedf208cbd Use fmt::runtime() for LOG_* for non constexpr 
Here is oneliner:

    $ gg 'LOG_\(DEBUG\|TRACE\|INFO\|TEST\|WARNING\|ERROR\|FATAL\)([^,]*, [a-zA-Z]' -- :*.cpp :*.h | cut -d: -f1 | sort -u | xargs -r sed -E -i 's#(LOG_[A-Z]*)\(([^,]*), ([A-Za-z][^,)]*)#\1(\2, fmt::runtime(\3)#'

Note, that I tried to do this with coccinelle (tool for semantic
patchin), but it cannot parse C++:

    $ cat fmt.cocci
    @@
    expression log;
    expression var;
    @@

    -LOG_DEBUG(log, var)
    +LOG_DEBUG(log, fmt::runtime(var))

I've also tried to use some macros/templates magic to do this implicitly
in logger_useful.h, but I failed to do so, and apparently it is not
possible for now.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

v2: manual fixes
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-01 14:30:03 +03:00
Azat Khuzhin
a09bc1d72e Add fmt::runtime() in Exception ctor 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-01 14:30:03 +03:00
Nikolai Kochetov
a6cc61bd14
Revert "Revert "Additionally check remote_fs_execute_merges_on_single_replica_time_threshold inside ReplicatedMergeTreeQueue"" 2022-02-01 14:17:46 +03:00
Amos Bird
e29c77f793 Fix link order issue of WriteBufferFromS3 
(cherry picked from commit f22b09f4fc)
 2022-02-01 10:42:06 +00:00
Nikolai Kochetov
2a6eb593be
Revert "Revert "Add pool to WriteBufferFromS3"" 2022-02-01 13:36:51 +03:00
alexey-milovidov
06477c2a7e Update ReplicatedMergeTreeSink.cpp 2022-02-01 09:22:40 +01:00
Igor Nikonov
f4c0b64420 Clean up: insert_deduplication_token setting for INSERT statement 
+ reduce number of allocations on replication merge tree path
+ bash test: move insert block settings into variable

Issue: ClickHouse#7461
 2022-02-01 09:22:33 +01:00
Vitaly Baranov
bc6802f815
Merge pull request #34064 from andremarianiello/propagate-grpc-tracing-context 
Propagate trace context from GRPC calls
 2022-02-01 14:11:43 +07:00
alexey-milovidov
99392b5ca7
Merge pull request #13544 from amosbird/mdha 
Multi-Disk auto-recovery.
 2022-02-01 06:13:26 +03:00
Alexey Milovidov
7dbf0dede5 Change severity of the "Cancelled merging parts" message in logs 2022-02-01 05:55:07 +03:00
alexey-milovidov
095d9bfa43
Revert "Add pool to WriteBufferFromS3" 2022-02-01 05:49:40 +03:00
alexey-milovidov
c9571c6086
Merge pull request #34202 from ClickHouse/speedup-format-regexp 
Improve performance of format Regexp
 2022-02-01 05:46:52 +03:00
alexey-milovidov
cbfcd45be3
Revert "Add func tests run with s3" 2022-02-01 05:46:13 +03:00
alexey-milovidov
a2aa147ce0
Merge pull request #34153 from ClickHouse/add_func_tests_over_s3 
Add func tests run with s3
 2022-02-01 05:46:09 +03:00
mergify[bot]
e229487817
Merge branch 'master' into mdha 2022-02-01 01:22:16 +00:00
Alexey Milovidov
989a5ad35e More cases for parseDateTimeBestEffort 2022-02-01 03:29:27 +03:00
Alexey Milovidov
e4e7169277 Remove some strange code 2022-02-01 02:52:36 +03:00
Alexey Milovidov
83136f3515 Allow \r in the middle of the line in format Regexp 2022-02-01 02:49:26 +03:00
Dmitry Novik
3a0ed0f8f5
Merge pull request #33324 from azat/buffer-different-structure 
Simplify different block structure (i.e. after ALTER) support for Buffer
 2022-02-01 02:19:49 +03:00
Alexey Milovidov
872d0a0fbe Improve performance of format Regexp 2022-02-01 02:07:48 +03:00
alexey-milovidov
15e4fe5c78
Revert "Additionally check remote_fs_execute_merges_on_single_replica_time_threshold inside ReplicatedMergeTreeQueue" 2022-02-01 01:51:39 +03:00
Amos Bird
ec7d367814
DiskLocal checker 
Add DiskLocal checker so that ReplicatedMergeTree can recover data when some of its disks are broken.
 2022-02-01 05:55:27 +08:00
alesapin
dd61d1c2de
Merge pull request #34172 from ClickHouse/fix_race_in_some_engines 
Fix benign race condition for storage HDFS, S3, URL
 2022-01-31 22:41:54 +03:00
alesapin
75d73d2785
Merge pull request #34139 from ClickHouse/fix_buf_s3_low_cardinality 
Fix bug with bounded S3 reads and LowCardinality
 2022-01-31 22:41:14 +03:00
Nikolai Kochetov
348d72266a
Merge pull request #34189 from ClickHouse/less-logging-for-remote_fs_execute_merges_on_single_replica_time_threshold 
Additionally check remote_fs_execute_merges_on_single_replica_time_threshold inside ReplicatedMergeTreeQueue
 2022-01-31 21:39:04 +03:00
Nikolai Kochetov
a207cdf28f Additionally check remote_fs_execute_merges_on_single_replica_time_threshold inside ReplicatedMergeTreeQueue. 2022-01-31 17:53:28 +00:00
avogar
058ef356ac Fix schema inference for table runction s3 2022-01-31 19:39:20 +03:00
Nikolai Kochetov
321fa4a9e8
Merge pull request #33291 from ClickHouse/add-pool-to-s3-write-buffer 
Add pool to WriteBufferFromS3
 2022-01-31 19:37:40 +03:00
tavplubix
d0ea72f878
Merge pull request #34005 from nicelulu/system_on_cluster 
Supports all types of SYSTEM query ON CLUSTER clause.
 2022-01-31 19:26:32 +03:00
alesapin
93c0700c4c Fix typo 2022-01-31 16:46:58 +03:00
alesapin
056b9e335f Fix comment 2022-01-31 16:39:42 +03:00
alesapin
31753afb7e Fix cancel logic in parallel parsing 2022-01-31 16:38:15 +03:00
Azat Khuzhin
9b8ee0fabe Cleanup cassandra contrib linking (linking with dictionraies is enough) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-31 16:01:11 +03:00
FArthur-cmd
3dc64f031a correct code style 2022-01-31 12:21:22 +00:00
FArthur-cmd
891411a2be add try catch to avoid errors on start 2022-01-31 12:13:28 +00:00
Maksim Kita
8513f20cfd
Merge pull request #34145 from kitaisreal/bitset-sort-performance-check 
pdqsort performance check
 2022-01-31 12:35:13 +01:00
alesapin
5230c59f05 Smaller scope 2022-01-31 14:03:06 +03:00
alesapin
cbf4790ea4 More clear error 2022-01-31 13:59:16 +03:00
alesapin
1adc7538fe Fix benign race condition for storage HDFS, S3, URL 2022-01-31 13:50:28 +03:00
alesapin
5be3968526 Fix bug in cache compressed read buffer 2022-01-31 13:40:26 +03:00
tavplubix
d19e24f530
Merge pull request #34096 from ClickHouse/fix_race_merge_selecting_task 
Fix race between mergeSelectingTask and queue reinitialization
 2022-01-31 12:16:29 +03:00
Maksim Kita
241c395329 Fixed style check 2022-01-31 09:12:08 +00:00
Kruglov Pavel
a9d0beb7ae
Fix data race in StorageFile (#34113) 
* Fix data race in StorageFile

* Update StorageFile.h

* Fix
 2022-01-31 11:58:40 +03:00
alesapin
55c7936257 Fix incorrect range for index 2022-01-31 11:11:32 +03:00
Maksim Kita
5ef83deaa6 Update sort to pdqsort 2022-01-30 19:49:48 +00:00
alesapin
4f1b902342 Fix compact parts as well 2022-01-30 22:36:19 +03:00
alesapin
ea8ec87b34 Fix stupid bug 2022-01-30 21:03:12 +03:00
alesapin
4bedcc19b5 Better invariants 2022-01-30 20:40:09 +03:00
achimbab
9822b7dfc9 Handle exceptional case where data are essentially constant. 2022-01-31 02:25:51 +09:00
alesapin
c237c03c50 Fix 2022-01-30 18:39:26 +03:00
alesapin
bf918892ac More clear code with less getMark calls 2022-01-30 18:21:05 +03:00
alesapin
cb45a348f1 Merge branch 'master' into fix_buf_s3_low_cardinality 2022-01-30 17:30:55 +03:00
achimbab
3f878b9cf6 Fix floating point comparison 2022-01-30 22:57:30 +09:00
Amos Bird
bb34435928
Better code 2022-01-30 19:48:35 +08:00
Amos Bird
54517753d7
Combining our lexer with replxx 2022-01-30 19:48:33 +08:00
zhangxiao871
7f2f7fa993 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into yandex-master 2022-01-30 13:46:52 +08:00
zhangxiao871
dd395c5721 Better test. 2022-01-30 13:46:11 +08:00
taiyang-li
e9c435a23f fix style 2022-01-30 13:23:11 +08:00
taiyang-li
c9d5251e12 finish dev 2022-01-30 09:10:27 +08:00
Maksim Kita
1eacd7a00c pdqsort performance check 2022-01-29 21:47:32 +00:00
alesapin
3f3e90c8ba Remove redundant code 2022-01-29 23:55:45 +03:00
Maksim Kita
a5fc8289ae
Merge pull request #34141 from kitaisreal/parser-dictionary-primary-key-fix 
Dictionary PRIMARY KEY parsing fix
 2022-01-29 19:55:32 +01:00
alexey-milovidov
2b43bad923
Update MergeTreeReaderStream.cpp 2022-01-29 19:28:39 +03:00
Maksim Kita
9c91a50050 Sort block refactoring 2022-01-29 15:50:55 +00:00
Maksim Kita
cfd3421e4f Dictionary PRIMARY KEY parsing fix 2022-01-29 15:50:10 +00:00
taiyang-li
5228a3e421 commit again 2022-01-29 23:42:04 +08:00
alesapin
7ada8227cf Fix bug with bounded S3 reads and LowCardinality 2022-01-29 18:28:40 +03:00
Maksim Kita
f3453024ff
Merge pull request #34060 from amosbird/optimizetupleorderby 
Make ORDER BY tuple almost as fast as ORDER BY columns
 2022-01-29 15:58:09 +01:00
Maksim Kita
8029d79280
Merge pull request #34126 from azat/field-vector-reserve 
Slightly optimize Array/Tuple/Map
 2022-01-29 14:25:39 +01:00
Amos Bird
04cff632db
Revise 2022-01-29 20:55:52 +08:00
Maksim Kita
76067efbef
Merge pull request #34076 from bigo-sg/poolbase 
Add expire() in PoolBase
 2022-01-29 13:51:55 +01:00
Azat Khuzhin
d37ebda2d2 Tiny cleanup in SerializationTuple 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-29 12:35:32 +03:00
Azat Khuzhin
9766619b55 Headers cleanup in ColumnMap 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-29 12:32:25 +03:00
Azat Khuzhin
f88ab143c4 Headers cleanup in ColumnArray 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-29 12:32:15 +03:00
Azat Khuzhin
3bc10871eb Minor cleanup in ColumnTuple 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-29 12:30:31 +03:00
Azat Khuzhin
b511abb72c Use reserve() over resize() for Map 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-29 12:29:13 +03:00
Azat Khuzhin
9e740eb9d0 Use reserve() over resize() for Tuple 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-29 12:29:13 +03:00
Azat Khuzhin
097bd6b7e8 Simplify ColumnArray::operator[] using ColumnArray::get() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-29 12:28:32 +03:00
Azat Khuzhin
06318ebed5 SerializationArray: Use reserve() over resize() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-29 12:28:32 +03:00
lgbo-ustc
5f1969c6c5 fixed code styles 2022-01-29 09:46:00 +08:00
Anton Popov
b950a12cb3
Merge pull request #34068 from CurtizJ/fix-async-insert-native 
Fix asynchronous inserts with `Native` format
 2022-01-29 01:24:53 +03:00
alexey-milovidov
6535b75322
Merge pull request #34001 from azat/memory-tracker-fix 
Fix memory accounting for queries that uses < max_untracker_memory
 2022-01-29 00:59:53 +03:00
Amos Bird
2b3ce910c6
Fix collation check 2022-01-29 04:49:58 +08:00
Federico Rodriguez
c0a5bd187a ExecuteScalarSubqueriesVisitor missing static const 
Added string_view
 2022-01-28 15:00:04 -05:00
Andre Marianiello
f5cbc6da43 Propagate trace context from GRPC calls 2022-01-28 14:27:04 -05:00
FArthur-cmd
cb4ad97d45 add test 2022-01-28 17:37:52 +00:00
Amos Bird
09a3ab79b7
Fix all const 2022-01-29 01:37:39 +08:00
Maksim Kita
b618febdec
Merge pull request #33970 from amosbird/fixwerror 
Fix missing -Werror flag
 2022-01-28 16:53:09 +01:00
Maksim Kita
e2de0dff8f
Merge pull request #34074 from azat/array-optimization 
Slightly optimize ColumnArray::get()/operator[] (by using reserve over resize)
 2022-01-28 16:09:44 +01:00
Maksim Kita
60129aaecb
Merge pull request #34052 from kitaisreal/remove-decimal-padded-pod-array 
Remove DecimalPaddedPODArray
 2022-01-28 15:54:41 +01:00
Azat Khuzhin
162f96f8e1 Get back detachQueryIfNotDetached() into ExternalLoader 
ExternalLoader from ThreadPool (async loading) is done from the server
context, not from the query context, and the context will already go
away, so we should detachQueryIfNotDetached() to avoid trigger assertion
in ThreadStatus.

CI: https://s3.amazonaws.com/clickhouse-test-reports/34001/8cace291d17fa9553a98b2a1e8bf15b30fe5a1bd/stateless_tests__debug__actions__[3/3].html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-28 16:25:33 +03:00
Azat Khuzhin
1519985c98 Fix possible "Can't attach query to the thread, it is already attached" 
After detachQueryIfNotDetached() had been removed it is not enough to
use attachTo() for ThreadPool (scheduleOrThrowOnError()) since the query
may be already attached, if the thread doing multiple jobs, so
CurrentThread::attachToIfDetached() should be used instead.

This should fix all the places from the failures on CI [1]:

    $ fgrep DB::CurrentThread::attachTo -A1 ~/Downloads/47.txt  | fgrep -v attachTo | cut -d' ' -f5,6 | sort | uniq -c
         92 --
          2 /fasttest-workspace/build/../../ClickHouse/contrib/libcxx/include/deque:1393: DB::ParallelParsingInputFormat::parserThreadFunction(std::__1::shared_ptr<DB::ThreadGroupStatus>,
          4 /fasttest-workspace/build/../../ClickHouse/src/Storages/MergeTree/MergeTreeData.cpp:1595: void
         87 /fasttest-workspace/build/../../ClickHouse/src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp:993: void

  [1]: https://github.com/ClickHouse/ClickHouse/runs/4954466034?check_suite_focus=true

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-28 16:25:33 +03:00
Azat Khuzhin
b0c862c297 Fix memory accounting for queries that uses < max_untracker_memory 
MemoryTracker starts accounting memory directly only after per-thread
allocation exceeded max_untracker_memory (or memory_profiler_step).

But even memory under this limit should be accounted too, and there is
code to do this in ThreadStatus dtor, however due to
PullingAsyncPipelineExecutor detached the query from thread group that
memory was not accounted.

So remove CurrentThread::detachQueryIfNotDetached() from threads that
uses ThreadFromGlobalPool since it has ThreadStatus, and the query will
be detached using CurrentThread::defaultThreadDeleter.

Note, that before this patch memory accounting works for HTTP queries
due to it had been accounted from ParallelFormattingOutputFormat, but
not for TCP.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-28 16:25:33 +03:00
Nikolai Kochetov
1c9f026178 Merge branch 'master' into add-pool-to-s3-write-buffer 2022-01-28 16:01:42 +03:00
Alexander Tokmakov
b3ddc601a5 fix race between mergeSelectingTask and queue reinitialization 2022-01-28 15:50:58 +03:00
mergify[bot]
c9a1c9a896
Merge branch 'master' into system_on_cluster 2022-01-28 10:42:26 +00:00
lgbo-ustc
c576393143 fixed code styles 2022-01-28 18:34:54 +08:00
alexey-milovidov
f6684dbc62
Merge pull request #32304 from devcrafter/deduplication_token_7461 
insert_deduplication_token setting for INSERT statement
 2022-01-28 13:03:55 +03:00
Vitaly Baranov
484743edff
Merge pull request #33201 from azat/client-interactive-suggest 
Client interactive suggest (extract info from CREATE queries)
 2022-01-28 16:30:16 +07:00
Kruglov Pavel
efa8775ea6
Merge pull request #33960 from Avogar/autodetect-format 
Detect format and schema from stdin in clickhouse-local
 2022-01-28 12:02:15 +03:00
lgbo-ustc
f5ffdaac21 fixed code styles 2022-01-28 16:55:17 +08:00
Azat Khuzhin
42ddf4d2b8 Fix negative StorageBufferBytes by proper accounting of empty blocks 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-28 11:10:00 +03:00
Azat Khuzhin
d77790182d Tiny cleanup in StorageBuffer::flushBuffer() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-28 11:10:00 +03:00
Azat Khuzhin
9948525816 Simplify different block sturcture (i.e. after ALTER) support for Buffer 
v2: fix empty block in case of flush

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-28 11:09:56 +03:00
Nikolai Kochetov
94999e86c2
Merge pull request #33392 from azat/_shard_num 
Replace old _shard_num implementation with shardNum() function
 2022-01-28 11:02:19 +03:00
zhangxiao871
78e2794ada Fix START MERGES and add test 2022-01-28 14:30:57 +08:00
lgbo-ustc
7a3324b29a add expire() in PoolBase 2022-01-28 14:03:34 +08:00
Azat Khuzhin
bfef9663ca Tiny cleanup of AggregateFunctionSimpleState/AggregateFunctionState 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-28 08:10:08 +03:00
Azat Khuzhin
4e07122079 Slightly optimize ColumnArray::get()/operator[] (by using reserve over resize) 
Note, that simple "SELECT range(100)" will execute
ColumnArray::operator[] 14 times (most of them from
DB::checkColumnStructure())

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-28 08:08:14 +03:00
Anton Popov
6c0959b907 fix asynchronous inserts with Native format 2022-01-28 03:25:15 +03:00
mergify[bot]
ebbfc0fd2f
Merge branch 'master' into remove-decimal-padded-pod-array 2022-01-27 20:12:41 +00:00
alexey-milovidov
63d8d75001
Merge pull request #33957 from azat/fix-getauxval 
Fix getauxval() in glibc-compatibility and fix some leaks (after LSan started to work)
 2022-01-27 23:05:23 +03:00
alexey-milovidov
4060290eb6
Merge pull request #33925 from benbiti/USE_DEBUG_HELPERS-OFF 
when not DEBUG set USE_DEBUG_HELPERS OFF
 2022-01-27 22:59:55 +03:00
alexey-milovidov
ea55c9a0ae
Merge pull request #33938 from CurtizJ/fix-mismatched-columns 
Fix 'Not found column' exception in distributed queries with arrays and tuples
 2022-01-27 22:58:48 +03:00
alexey-milovidov
cd2305eb57
Merge pull request #33956 from CurtizJ/fix-nested-tuples 
Fix reading of nested tuples
 2022-01-27 22:58:11 +03:00
alexey-milovidov
5a4ad04ae6
Merge pull request #33505 from zvonand/issue_14648 
Fix fromUnixTimestamp64 functions
 2022-01-27 22:54:53 +03:00
Raúl Marín
91878b97e4
Address AggregateFunctionIf.cpp warnings 
/mnt/ch/ClickHouse/src/AggregateFunctions/AggregateFunctionIf.cpp:75:79: warning: declaration shadows a field of 'AggregateFunctionIfNullUnary<result_is_nullable, serialize_flag>' [-Wshadow]
    inline bool singleFilter(const IColumn ** columns, size_t row_num, size_t num_arguments) const
                                                                              ^
/mnt/ch/ClickHouse/src/AggregateFunctions/AggregateFunctionIf.cpp:53:12: note: previous declaration is here
    size_t num_arguments;
           ^
 2022-01-28 02:07:42 +08:00
tavplubix
8c9266b24c
Merge pull request #33978 from amosbird/separatesystemlog 
Separate base parts out of SystemLog
 2022-01-27 20:07:55 +03:00
Maksim Kita
24267c8cfb Fixed tests 2022-01-27 16:02:31 +00:00
mergify[bot]
260942e0f0
Merge branch 'master' into hlee-issue-7334 2022-01-27 14:30:13 +00:00
Nikolay Degterinsky
c5ca5b608e
Merge pull request #33314 from evillique/classification 
Merge functions for text classification
 2022-01-27 17:15:08 +03:00
Amos Bird
faee95b897
Make ORDER BY tuple almost as fast as ORDER BY columns 
We have special optimizations for multiple column ORDER BY: https://github.com/ClickHouse/ClickHouse/pull/10831 . It's beneficial to also apply to tuple columns.

Before:

select * from numbers(300000000) order by (1 - number , number + 1 , number) limit 10;
2.613 sec.

After:

select * from numbers(300000000) order by (1 - number , number + 1 , number) limit 10;
0.755 sec

No tuple:

select * from numbers(300000000) order by 1 - number , number + 1 , number limit 10;
0.755 sec
 2022-01-27 21:42:08 +08:00
mergify[bot]
ad146a67ff
Merge branch 'master' into _shard_num 2022-01-27 13:11:42 +00:00
mergify[bot]
eb1bf824be
Merge branch 'master' into fix-mismatched-columns 2022-01-27 12:23:48 +00:00
Azat Khuzhin
4e4e70ec6e Fix leak in SimpleState function (due to recrusive reference) 
CI reports [1]:

    Indirect leak of 648 byte(s) in 9 object(s) allocated from:
    ...
        2 0x12b96503 in DB::AggregateFunctionSimpleState::getReturnType() const obj-x86_64-linux-gnu/../src/AggregateFunctions/AggregateFunctionSimpleState.h:47:15
    ...

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/33957/08f4f45fd9da923ae3e3fdd8a527c297d35247eb/stress_test__address__actions_.html

After we can get this query by using query_log artifact:

    $ wget https://s3.amazonaws.com/clickhouse-test-reports/33957/08f4f45fd9da923ae3e3fdd8a527c297d35247eb/stress_test__address__actions_/query_log_dump.tar
    $ tar -xf query_log_dump.tar
    $ clickhouse-local --path var/lib/clickhouse/
    SELECT query
    FROM system.query_log
    ARRAY JOIN used_aggregate_function_combinators AS func
    WHERE has(used_aggregate_functions, 'groupBitOr') AND has(used_aggregate_function_combinators, 'SimpleState') AND (type != 'QueryStart')

    Query id: 5b7722b3-f77e-4e7e-bd0b-586d6d32a899

    ┌─query────────────────────────────────────────────────────────────────────────────┐
    │ with groupBitOrSimpleState(number) as c select toTypeName(c), c from numbers(1); │
    └──────────────────────────────────────────────────────────────────────────────────┘

Fixes: 01570_aggregator_combinator_simple_state.sql
Fixes: #16853
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-27 15:23:04 +03:00
Azat Khuzhin
af9910b518 Fix leak of demangle result (for opentelementry) 
LSan found [1]:

    Direct leak of 5170176 byte(s) in 5049 object(s) allocated from:
        0 0xc598edd in malloc (/usr/bin/clickhouse+0xc598edd)
        1 0x39679739 in (anonymous namespace)::itanium_demangle::initializeOutputStream(char*, unsigned long*, (anonymous namespace)::itanium_demangle::OutputStream&, unsigned long) obj-x86_64-linux-gnu/../contrib/libcxxabi/src/demangle/Utility.h:178:31
        2 0x39679739 in __cxa_demangle obj-x86_64-linux-gnu/../contrib/libcxxabi/src/cxa_demangle.cpp:351:13
        3 0x28f6f3ed in DB::executeQueryImpl(char const*, char const*, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:662:44

  [1]:
https://s3.amazonaws.com/clickhouse-test-reports/33957/08f4f45fd9da923ae3e3fdd8a527c297d35247eb/stress_test__address__actions_.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-27 13:46:37 +03:00
Maksim Kita
af5ad2f370 Remove DecimalPaddedPODArray 2022-01-27 10:07:53 +00:00
Nikolai Kochetov
9b2998c639 Review fixes. 2022-01-26 18:08:01 +00:00
Nikolai Kochetov
a8171269a1 Review fixes. 2022-01-26 17:55:24 +00:00
Raúl Marín
5a59d976dd CurrentlyExecuting: Require mutex usage explicitly 2022-01-26 18:44:35 +01:00
zvonand
1384f9e89d moved common code out of template 2022-01-26 20:15:34 +03:00
alesapin
289a51b61d
Merge pull request #33905 from zhongyuankai/fix_same_disk_path 
Fix disk using the same path
 2022-01-26 17:38:08 +03:00
Kruglov Pavel
662ea9d024
Merge pull request #33927 from kitaisreal/range-hashed-dictionary-refactoring 
RangeHashedDictionary refactoring
 2022-01-26 15:26:57 +03:00
Nikolai Kochetov
efaed154d2
Update src/Disks/IDisk.h 
Co-authored-by: alesapin <alesapin@clickhouse.com>
 2022-01-26 14:58:38 +03:00
alexey-milovidov
0a112bcf61
Update StringSearcher.h 2022-01-26 13:45:26 +03:00
Azat Khuzhin
08f4f45fd9 Fix Context leak (recursive context reference) in StorageSQLite 
CI founds after LSan had been fixed [1]:

    01889_sqlite_read_write:                                                [ FAIL ] 8.32 sec. - return code: 1

    =================================================================
    ==20649==ERROR: LeakSanitizer: detected memory leaks

    Indirect leak of 1968 byte(s) in 1 object(s) allocated from:
        0 0xc5c1ffd in operator new(unsigned long) (/usr/bin/clickhouse+0xc5c1ffd)
        1 0x25e32d0d in std::__1::__unique_if<DB::StorageInMemoryMetadata>::__unique_single std::__1::make_unique<DB::StorageInMemoryMetadata, DB::StorageInMemoryMetadata const&>(DB::StorageInMemoryMetadata c>
        2 0x25e32d0d in DB::IStorage::setInMemoryMetadata(DB::StorageInMemoryMetadata const&) obj-x86_64-linux-gnu/../src/Storages/IStorage.h:194:22
        3 0x29bdee98 in DB::StorageSQLite::StorageSQLite(DB::StorageID const&, std::__1::shared_ptr<sqlite3>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std:>
        4 0x25ee61d6 in std::__1::shared_ptr<DB::StorageSQLite> shared_ptr_helper<DB::StorageSQLite>::create<DB::StorageID, std::__1::shared_ptr<sqlite3> const&, std::__1::basic_string<char, std::__1::char_tr>
        5 0x25ee61d6 in DB::TableFunctionSQLite::executeImpl(std::__1::shared_ptr<DB::IAST> const&, std::__1::shared_ptr<DB::Context const>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1:>

    SUMMARY: AddressSanitizer: 171256 byte(s) leaked in 130 allocation(s).

  [1]: https://github.com/ClickHouse/ClickHouse/runs/4929706698?check_suite_focus=true

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-26 12:56:27 +03:00
zhangxiao871
1bd74dda63 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into yandex-master 2022-01-26 17:40:32 +08:00
zhangxiao871
20b669f330 Supports all types of SYSTEM query ON CLUSTER clause. 2022-01-26 17:40:00 +08:00
zvonand
3879137994 Merge branch 'master' of github.com:ClickHouse/ClickHouse into issue_14648 2022-01-26 12:33:21 +03:00