thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-07 06:52:19 +00:00
Code Issues Packages Projects Releases Wiki Activity
131,626 Commits 9,200 Branches 1,782 Tags 2 GiB
ea138fe8c9
Commit Graph

131626 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
kssenii
041c5d4219 Make caches with the same path use the same cache objects 2023-12-27 18:37:43 +01:00
Alexey Milovidov
df6bec57aa
Merge branch 'master' into merge-tree-final-extract-non-intersecting-parts-ranges 2023-12-27 18:23:28 +01:00
Alexander Tokmakov
f924848347 partially revert #54368 (f28ad1e136) 2023-12-27 18:17:59 +01:00
Alexander Tokmakov
d7a6148828 partially revert #54706 (135df6870c) 2023-12-27 18:13:51 +01:00
Raúl Marín
dfe7b0e973 Keep message 2023-12-27 18:13:22 +01:00
Raúl Marín
5f183649b2 Avoid throwing ABORTED on normal situations 2023-12-27 17:44:46 +01:00
Alexey Milovidov
64b4e1a66f Reintroduce compatibility with is_deleted on a syntax level 2023-12-27 17:42:51 +01:00
avogar
fee2eadaf0 Fix parallel parsing for JSONCompactEachRow 2023-12-27 16:16:41 +00:00
Azat Khuzhin
217d24ab22 Add missing includes for memcpySmall 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-27 17:06:17 +01:00
Azat Khuzhin
058001eb31 Add missing includes into CompressionCodecDeflateQpl 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-27 17:06:17 +01:00
Azat Khuzhin
c6774cfe42 Add missing includes for ZipArchiveWriter 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-27 17:06:17 +01:00
Azat Khuzhin
f35f153719 Add missing includes into ThreadStatus 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-27 17:06:17 +01:00
Azat Khuzhin
cc296a1b8a Move some code of ThreadPool into module part 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-27 17:06:17 +01:00
Azat Khuzhin
cf1dae5321 Add missing includes into BitHelpers.h 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-27 17:06:17 +01:00
Azat Khuzhin
09599647e7 Move some code from PODArray into implementation part 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-27 17:06:17 +01:00
Raúl Marín
84727e986e Add test for #57086 2023-12-27 17:05:37 +01:00
Nikita Mikhaylov
a3fa378401
Update TTLAggregationAlgorithm.cpp 2023-12-27 17:02:52 +01:00
Nikita Mikhaylov
ac87daf8c3 Fix bug in TTL GROUP BY 
(cherry picked from commit d9f2a2935c1c6760ef2725f2eb6d13ed9f0815fe)
 2023-12-27 15:50:20 +00:00
Nikita Mikhaylov
3dbd3b3e61 Better 2023-12-27 15:50:20 +00:00
Nikita Mikhaylov
b60109d43e Better 2023-12-27 15:50:20 +00:00
Kruglov Pavel
745d9bb47f
Merge pull request #58214 from kevinmingtarja/optimize-long-and-notequals-chain 
Add optimization for AND notEquals chain in logical expression optimizer
 2023-12-27 15:51:59 +01:00
Alexey Milovidov
0e678fb6c1
Merge pull request #56996 from ClickHouse/vdimir/hash_join_max_block_size 
HashJoin respects max_joined_block_size_rows
 2023-12-27 15:46:46 +01:00
Alexey Milovidov
f00337e2ba
Merge pull request #57872 from CurtizJ/optimize-aggregation-consecutive-keys 
Better optimization of consecutive keys in aggregation
 2023-12-27 15:44:22 +01:00
Azat Khuzhin
964d074186 Remove extra excludes from PODArray 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-27 15:42:08 +01:00
Azat Khuzhin
a1ddfc909e Do not include PODArray.h into NamePrompter.h 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-27 15:42:08 +01:00
Azat Khuzhin
d4df5a2e5b Remove workaround for old enough MacOSX 
- this unit even not used (only in tests)
- MacOSX is cross compiled

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-27 15:42:08 +01:00
Azat Khuzhin
f1e67efa93 Define _DARWIN_C_SOURCE for darwin (for MAP_ANON/MAP_ANONYMOUS) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-27 15:42:08 +01:00
Azat Khuzhin
b9233f6d4f Move Allocator code into module part 
This should reduce amount of code that should be recompiled on
Exception.h changes (and everything else that had been included there).

This will actually not help a lot, because it is also included into
PODArray.h and ThreadPool.h at least... Sigh.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-27 15:42:08 +01:00
Alexey Milovidov
27bcdbe14a
Update src/Core/Settings.h 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2023-12-27 17:41:30 +03:00
Alexey Milovidov
1a93fd7f7d
Merge pull request #57829 from azat/kafka-fix-stat-leak 
Create consumers for Kafka tables on fly (but keep them for some period since last used)
 2023-12-27 15:35:05 +01:00
Alexey Milovidov
ad42115994
Merge pull request #58175 from Avogar/fix-array-subcolumns-read 
Fix possible PARAMETER_OUT_OF_BOUND error during subcolumns reading from wide part in MergeTree
 2023-12-27 15:30:02 +01:00
Alexander Tokmakov
01d042c490 Revert "Merge pull request #57932 from ClickHouse/remove-shit-cleanup" 
This reverts commit 2d58dc512c, reversing
changes made to 41873dc4a3.
 2023-12-27 13:46:06 +01:00
robot-clickhouse-ci-2
bd43660255
Merge pull request #58245 from Algunenano/fix_perf_readme 
Fix perf test README
 2023-12-27 13:33:50 +01:00
Alexey Milovidov
5f6318e51b
Merge pull request #58151 from ClickHouse/fix-legend 
Fix dashboard legend sorting and rows number
 2023-12-27 13:33:00 +01:00
Alexey Milovidov
d0bc4fafdb
Merge pull request #58220 from wangtZJU/fix_AddDefaultDatabaseVisitor_bad_performance 
fix CREATE VIEW hang
 2023-12-27 13:30:37 +01:00
Alexey Milovidov
77ec4a0422
Merge pull request #58218 from kitaisreal/merge-tree-automatically-derive-do-not-merge-across-partitions-select-final-setting 
MergeTree derive do_not_merge_across_partitions_select_final setting
 2023-12-27 13:26:17 +01:00
Raúl Marín
f8d9a850c7 Fix perf test README 2023-12-27 12:16:17 +00:00
Alexander Tokmakov
4c4623e3ab Revert "Merge pull request #58100 from ClickHouse/optimize-cleanup-as-noop" 
This reverts commit 3b1aa53460, reversing
changes made to b9466023ce.
 2023-12-27 13:03:53 +01:00
Alexander Tokmakov
eeadeaa89d Revert "Merge pull request #58104 from ClickHouse/cleanup-replication-compatibility" 
This reverts commit 34fd555ee6, reversing
changes made to cb53ee63be.
 2023-12-27 13:03:38 +01:00
Kevin Mingtarja
3d7596da4b Rename test, fix style 2023-12-27 17:28:01 +07:00
Raúl Marín
68787cedd6 Linter warnings 2023-12-27 08:51:17 +00:00
Azat Khuzhin
ebad1bf4f3 Move StorageKafka::createConsumer() into KafkaConsumer 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-27 09:49:07 +01:00
Azat Khuzhin
03218202d3 Fix data-race between StorageKafka::startup() and cleanConsumers() 
Actually now we can create consumer object in the ctor, no need to do
this in startup(), since consumer now do not connects to kafka.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-27 09:49:07 +01:00
Azat Khuzhin
1f03a21033 Update comment for statistics.interval.ms librdkafka option 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-27 09:49:07 +01:00
Azat Khuzhin
06a9e9a9ca Use separate thread for kafka consumers cleanup 
Since pool may exceed threads, while we need to run this thread always
to avoid memory leaking.

And this should not be a problem since librdkafka has multiple threads
for each consumer (5!) anyway.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-27 09:49:07 +01:00
Azat Khuzhin
a7453f7f14 Allow setThreadName() to truncate thread name instead of throw an error 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-27 09:49:07 +01:00
Azat Khuzhin
b19b70b8fc Add ability to configure TTL for kafka consumers 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-27 09:49:07 +01:00
Azat Khuzhin
2ff0bfb0a1 Preserve KafkaConsumer objects 
This will make system.kafka_consumers more useful, since after TTL
consumer object will be removed prio this patch, but after, all
information will be preserved.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-27 09:49:07 +01:00
Azat Khuzhin
db74549940 Enable stats for system.kafka_consumers back by default 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-27 09:49:07 +01:00
Azat Khuzhin
e7592c140e Create consumers for Kafka tables on fly (but keep them for 1min since last used) 
Pool of consumers created a problem for librdkafka internal statistics,
you need to read from the queue always, while in ClickHouse consumers
created regardless are there any readers or not (attached materialized
views or direct SELECTs).

Otherwise, this statistics messages got queued and never released,
which:
- creates live memory leak
- and also makes destroy very slow, due to librdkafka internals (it
  moves entries from this queue into another linked list, but in a
  with sorting, which is incredibly slow for linked lists)

So the idea is simple, let's create a pool of consumers only when they
are required, and destroy them after some timeout (right now it is 60
seconds) if nobody uses them, that way this problem should gone.

This should also reduce number of internal librdkafka threads, when
nobody reads from Kafka tables.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-27 09:49:07 +01:00