thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 19:32:07 +00:00
Code Issues Packages Projects Releases Wiki Activity
115,357 Commits 9,319 Branches 1,783 Tags 2.1 GiB
78c89da8bb
Commit Graph

5754 Commits

Author SHA1 Message Date
Sergei Trifonov
b6975d36e9
Fix bytesSize() of zk SetRequest 2023-04-06 19:05:32 +02:00
Alexey Milovidov
4ab8b35b00
Merge pull request #48444 from ClickHouse/better-exception-messages-from-keeper-client 
Better exception messages from Keeper client
 2023-04-06 18:36:39 +03:00
Azat Khuzhin
e91c8bfea3 Fix deadlock due to debug tracking of memory allocations 
Since memory tracker should be blocked globally (by default it is blocked only
for VariableContext::User).

<details>

<summary>stacktrace</summary>

    * thread 11, name = 'Collector', stop reason = trace
        frame 1: 0x00007ffff7e37ea2 libc.so.6`pthread_mutex_lock + 274
        frame 6: 0x000000001ee10298 clickhouse`toStringCached(pointers=0x00007fffedfef068, offset=0, size=45) + 88 at StackTrace.cpp:425
        frame 7: 0x000000001ee1128e clickhouse`StackTrace::toString(this=0x00007fffedfef058) const + 46 at StackTrace.cpp:444
        frame 8: 0x000000001ede58ff clickhouse`(anonymous namespace)::debugLogBigAllocationWithoutCheck(size=25187648) + 479 at MemoryTracker.cpp:98
        frame 9: 0x000000001ede5481 clickhouse`MemoryTracker::allocImpl(this=0x0000000030729e68, size=25187648, throw_if_memory_exceeded=false, query_tracker=0x00007ffff3c22440) + 3937 at MemoryTracker.cpp:352
        frame 10: 0x000000001ede477f clickhouse`MemoryTracker::allocImpl(this=0x00007ffff3c1e530, size=25187648, throw_if_memory_exceeded=false, query_tracker=0x00007ffff3c22440) + 607 at MemoryTracker.cpp:198
        frame 11: 0x000000001ede477f clickhouse`MemoryTracker::allocImpl(this=0x00007ffff3c22440, size=25187648, throw_if_memory_exceeded=false, query_tracker=0x0000000000000000) + 607 at MemoryTracker.cpp:198
        frame 12: 0x000000001ede477f clickhouse`MemoryTracker::allocImpl(this=0x00007fffedff3bf0, size=25187648, throw_if_memory_exceeded=false, query_tracker=0x0000000000000000) + 607 at MemoryTracker.cpp:198
        frame 13: 0x000000001ed6c4c9 clickhouse`CurrentMemoryTracker::allocImpl(size=25165824, throw_if_memory_exceeded=false) + 265 at CurrentMemoryTracker.cpp:58
        frame 14: 0x000000001ed6c6a1 clickhouse`CurrentMemoryTracker::allocNoThrow(size=25165824) + 33 at CurrentMemoryTracker.cpp:91
        frame 15: 0x000000001ed63079 clickhouse`operator new(unsigned long) [inlined] void Memory::trackMemory<>(size=25165824) + 89 at memory.h:177
        frame 16: 0x000000001ed63034 clickhouse`operator new(size=25165824) + 20 at new_delete.cpp:74
        ...
        frame 27: 0x000000001ee15895 clickhouse`DB::(anonymous namespace)::collectSymbols(info=0x00007ffff7062000, (null)=<unavailable>, data_ptr=0x00007fffec2b7000) + 53 at SymbolIndex.cpp:483
        frame 28: 0x000000002e1a5d57 clickhouse`::dl_iterate_phdr(callback=(clickhouse`DB::(anonymous namespace)::collectSymbols(dl_phdr_info*, unsigned long, void*) at SymbolIndex.cpp:480), data=0x00007fffec2b7000)(dl_phdr_info *, size_t, void *), void *) + 183 at phdr_cache.cpp:76
        frame 29: 0x000000001ee15549 clickhouse`DB::SymbolIndex::update(this=0x00007fffec2b7000) + 41 at SymbolIndex.cpp:515
        frame 30: 0x000000001ee1be35 clickhouse`DB::SymbolIndex::SymbolIndex(this=<unavailable>) + 53 at SymbolIndex.h:23
        frame 31: 0x000000001ee17a6c clickhouse`DB::SymbolIndex::instanceImpl() + 108 at SymbolIndex.cpp:555
        frame 32: 0x000000001ee17b4a clickhouse`DB::SymbolIndex::instance() + 10 at SymbolIndex.cpp:561
        frame 33: 0x000000001ee1070f clickhouse`void toStringEveryLineImpl<toStringCached(std::__1::array<void*, 45ul> const&, unsigned long, unsigned long)::$_0>(fatal=false, stack_trace=0x00007fffedff1ef8, callback=0x00007fffedff1ee0)::$_0&&) + 207 at StackTrace.cpp:349
        frame 34: 0x000000001ee1040b clickhouse`toStringCached(pointers=0x00007fffedff2928, offset=0, size=43) + 459 at StackTrace.cpp:436
        frame 35: 0x000000001ee1128e clickhouse`StackTrace::toString(this=0x00007fffedff2918) const + 46 at StackTrace.cpp:444
        frame 36: 0x000000001ede58ff clickhouse`(anonymous namespace)::debugLogBigAllocationWithoutCheck(size=25165824) + 479 at MemoryTracker.cpp:98
        frame 37: 0x000000001ede5481 clickhouse`MemoryTracker::allocImpl(this=0x00007fffedff3bf0, size=25165824, throw_if_memory_exceeded=false, query_tracker=0x0000000000000000) + 3937 at MemoryTracker.cpp:352
        frame 38: 0x000000001ed6c4c9 clickhouse`CurrentMemoryTracker::allocImpl(size=25165824, throw_if_memory_exceeded=false) + 265 at CurrentMemoryTracker.cpp:58
        frame 39: 0x000000001ed6c6a1 clickhouse`CurrentMemoryTracker::allocNoThrow(size=25165824) + 33 at CurrentMemoryTracker.cpp:91
        frame 40: 0x000000001ed63079 clickhouse`operator new(unsigned long) [inlined] void Memory::trackMemory<>(size=25165824) + 89 at memory.h:177
        frame 41: 0x000000001ed63034 clickhouse`operator new(size=25165824) + 20 at new_delete.cpp:74
        ...
        frame 55: 0x0000000025badace clickhouse`DB::WriteBufferFromS3::nextImpl(this=0x00007ffff3c58f00) + 366 at WriteBufferFromS3.cpp:110

</details>

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-06 16:03:46 +02:00
Sema Checherinda
b8729be97a
Merge pull request #47104 from CheSema/thead-group-for-merge 
merge and mutation make thread group for setting memory trackers right
 2023-04-06 15:46:56 +02:00
Robert Schulze
ce2692ba79
Merge pull request #47246 from ClickHouse/rs/m 
Change the behavior of formatter %M in function formatDateTime() from minutes to month name
 2023-04-06 09:19:50 +02:00
Alexey Milovidov
785ea8213b Better exception messages from Keeper client 2023-04-06 02:29:37 +02:00
Alexander Tokmakov
a154fa73f4
Merge pull request #48401 from ClickHouse/fix_00002_log_messages 
Find big allocations without memory limits checks
 2023-04-05 23:52:39 +03:00
Robert Schulze
809a7fbb7a
Incorporate more review feedback 2023-04-05 19:43:30 +00:00
Sema Checherinda
554c928534
Merge branch 'master' into thead-group-for-merge 2023-04-05 21:09:38 +02:00
robot-clickhouse
3d2c279859
Merge pull request #48242 from azat/better-throttling 
Per-query/per-server throttling for remote IO/local IO/BACKUPs
 2023-04-05 19:32:37 +02:00
Alexander Tokmakov
0e0e89d596 fix 2023-04-05 15:10:09 +02:00
Robert Schulze
598b050628
AST Fuzzer: Fix assertion in TopK serialization 
Problem:
  https://s3.amazonaws.com/clickhouse-test-reports/0/fa5b2bd4a5b02336bca8837c473a7124f8ecedf2/fuzzer_astfuzzerasan/report.html

The new assertion in the Varint code was introduced with (*). It rejects
values whose serialization cannot be deserialized (and this behavior
cannot be changed due to historical reasons). Such values should be
exceptionally rare in practice but AST fuzzer managers to trigger them.

The fix is similar to (**): Bypass the check by limiting the value to
the maximum allowed value.

(if AST fuzzer triggers finds more violations of the assertion, we might
consider throwing an exception instead)

(*) https://github.com/ClickHouse/ClickHouse/pull/48154
(**) https://github.com/ClickHouse/ClickHouse/pull/48154/files#diff-653c0a18dfdaa86262c78dc6b25550add0487f165b4ad053e86f530388f6203a
 2023-04-05 08:07:37 +00:00
Azat Khuzhin
a25dd1d348 Add ability to throttle local IO on per-query/server basis 
Server settings:
- max_local_read_bandwidth_for_server
- max_local_write_bandwidth_for_server

Query settings:
- max_local_read_bandwidth
- max_local_write_bandwidth

This is the preparation for adding ability to throttle BACKUPs

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:39:15 +02:00
Azat Khuzhin
b3406beeb7 Add per-query network throttling 
Controlled with:
- max_remote_read_network_bandwidth
- max_remote_write_network_bandwidth

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-05 09:39:15 +02:00
Arthur Passos
34be1893fd fix unit tests 2023-04-04 20:31:12 -03:00
Arthur Passos
474377c16f remove debug find functions and replace SearchSymbols assertion with an exception, check with upstream preferred way 2023-04-04 16:14:56 -03:00
Alexander Tokmakov
f5574d0d74 find big allocations without limits checks 2023-04-04 20:10:22 +02:00
Nikita Mikhaylov
fa5b2bd4a5
Added Keeper retries for backups operations (#47224) 2023-04-04 18:12:08 +02:00
Arthur Passos
fa1fdf7573 style 2023-04-03 15:28:17 -03:00
Arthur Passos
c80ba3d52e unit tests passing 2023-04-03 13:48:58 -03:00
Antonio Andelic
f5e97fbb66 Merge branch 'master' into alternative-keeper-configs 2023-04-03 15:06:52 +00:00
Azat Khuzhin
fbc99d7788 Use StrongTypedef for StatusInfo 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-01 16:01:33 +02:00
Azat Khuzhin
c64f9e6f07 Use StrongTypedef for ProfileEvents 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-01 16:01:33 +02:00
Azat Khuzhin
8d0e516310 Use StrongTypedef for CurrentMetrics to avoid possible incorrect usage 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-01 16:01:33 +02:00
Azat Khuzhin
f69441b633 Fix ThreadPool usage in gtest_thread_pool_limit 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-01 16:00:03 +02:00
Azat Khuzhin
ba6ecd2d4e Fix ThreadPool for DistributedSink 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-01 16:00:03 +02:00
Alexander Tokmakov
29e85bd9a4
Merge branch 'master' into fix-threadpools 2023-03-31 01:03:06 +03:00
Sergei Trifonov
01ed4ffca4
Merge pull request #47870 from ClickHouse/segmentator-fix 
Fix CPU usage counters update logic
 2023-03-30 22:55:58 +02:00
Azat Khuzhin
edeeac84cf Fix incorrect ThreadPool usage after ThreadPool introspection 
```
$ gg 'ThreadPool[^()]([A-Za-z_]\+,' src/
src/Interpreters/Context.cpp:        shared->load_marks_threadpool = std::make_unique<ThreadPool>(pool_size, pool_size, queue_size);
src/Interpreters/Context.cpp:        shared->prefetch_threadpool = std::make_unique<ThreadPool>(pool_size, pool_size, queue_size);
src/Interpreters/Context.cpp:        shared->threadpool_writer = std::make_unique<ThreadPool>(pool_size, pool_size, queue_size);
```

Fixes: #47880
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-03-30 20:08:38 +02:00
Alexey Milovidov
e982fb9f1c
Merge pull request #47880 from azat/threadpool-introspection 
ThreadPool metrics introspection
 2023-03-30 01:27:31 +03:00
Alexey Milovidov
13761fe891
Merge pull request #48133 from ClickHouse/allow-to-skip-errors-of-enums 
Allow skipping errors related to unknown enum values in row input formats
 2023-03-29 23:20:05 +03:00
Frank Chen
87d235e842
Some improvements about names of span logs (#47667) 2023-03-29 16:28:36 +02:00
Azat Khuzhin
f38a7aeabe ThreadPool metrics introspection 
There are lots of thread pools and simple local-vs-global is not enough
already, it is good to know which one in particular uses threads.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-03-29 10:46:59 +02:00
Sergei Trifonov
1ff6519c25
Merge branch 'master' into segmentator-fix 2023-03-29 02:29:43 +02:00
serxa
81af0b6deb avoid counters updates if not initialized 2023-03-29 00:29:21 +00:00
Alexey Milovidov
cfc4e40811 Allow skipping errors related to unknown enum values in row input formats 2023-03-29 00:31:53 +02:00
Alexander Tokmakov
614f8b313e
Merge pull request #47856 from Algunenano/zk_inject_timeout 
Allow injecting timeout errors on Keeper
 2023-03-28 23:40:51 +03:00
Yatsishin Ilya
26aacd3d2d Merge remote-tracking branch 'origin/master' into clickhouse-help 2023-03-28 16:57:58 +00:00
avogar
e63231700f Fix tests 2023-03-28 16:00:00 +00:00
Sema Checherinda
b774310dc9 keep query id for logging when optimize query 2023-03-28 16:44:47 +02:00
Sema Checherinda
10f1030a14
Merge pull request #47147 from aletik256/fix_JSON_searchField 
fix_JSON_searchField
 2023-03-28 14:45:09 +02:00
Antonio Andelic
a0b6cd63bb fix build 2023-03-28 07:23:29 +00:00
Antonio Andelic
4aa8c5fe49 Merge branch 'master' into alternative-keeper-configs 2023-03-28 07:23:24 +00:00
Raúl Marín
ff209db129
Merge branch 'master' into zk_inject_timeout 2023-03-27 19:46:10 +02:00
Sema Checherinda
a6ab33a906 no use query, but storage context 2023-03-27 16:30:32 +02:00
Han Fei
e3afa5090f
Merge pull request #47218 from hanfei1991/hanfei/optimize-regexp-tree-1 
Refine OptimizeRegularExpression Function and RegexpTreeDict
 2023-03-27 15:23:01 +02:00
Raúl Marín
714b54b322 Reimplement enable_fault_injections_during_startup 2023-03-27 15:06:46 +02:00
Sema Checherinda
26e76bb68c
Merge branch 'master' into fix_JSON_searchField 2023-03-27 10:47:57 +02:00
Sema Checherinda
db55fa864a fix build 2023-03-27 10:47:34 +02:00
Antonio Andelic
93f5920e02
Merge branch 'master' into alternative-keeper-configs 2023-03-27 10:00:21 +02:00
Vasily Nemkov
fa27ed9d41 Less noisy tests 2023-03-25 23:31:17 +01:00
Vasily Nemkov
ce608b135f Fixed implementation of find_first_symbols_sse42 run-time needle 
Added some tests
 2023-03-25 23:18:51 +01:00
Kruglov Pavel
713ecb2492
Merge branch 'master' into non-blocking-connect 2023-03-24 22:55:45 +01:00
Sema Checherinda
da4f2bd923 do not attach empty thread group 2023-03-24 19:53:16 +01:00
Sema Checherinda
bc107c70fa merge and mutation make thread group for setting memory trackers right 2023-03-24 19:53:16 +01:00
Maksim Kita
5cb2d30563
Merge pull request #47316 from ClickHouse/vdimir/transform_external_query_analyzer 
Support transformQueryForExternalDatabase for analyzer
 2023-03-24 17:53:00 +01:00
Raúl Marín
bf7b0c5cfe Different approach to inject timeouts 2023-03-24 17:12:14 +01:00
Sema Checherinda
1e462e02ed add ut 2023-03-24 17:00:04 +01:00
Sergei Trifonov
a05aa5c1c6
Merge branch 'master' into segmentator-fix 2023-03-24 16:19:57 +01:00
Kruglov Pavel
f3f93dd06c
Merge branch 'master' into non-blocking-connect 2023-03-24 15:59:40 +01:00
serxa
28ca2de86d fix 2023-03-24 14:58:50 +00:00
Vasily Nemkov
e22eee8055 Updated implementation of find_first_symbols for run-time needle 
Implemented compile-time and run-time dispatching between SSE4.2 and SSE2 implementations
Added find_first_symbols_sse2
Added tests
 2023-03-24 15:45:12 +01:00
Kruglov Pavel
3ee12e21fb
Merge branch 'master' into non-blocking-connect 2023-03-23 20:53:44 +01:00
Antonio Andelic
3ae09c6ce8
Merge branch 'master' into more-s3-events 2023-03-23 18:23:53 +01:00
Antonio Andelic
d6cbc5d05b Fix tests 2023-03-23 15:01:51 +00:00
Antonio Andelic
9ea8dc4e98 Merge branch 'master' into alternative-keeper-configs 2023-03-23 09:22:46 +00:00
Antonio Andelic
b57e82ded6 Add S3 write profile events 2023-03-23 09:05:41 +00:00
Antonio Andelic
a4d1cd514d Refactor 2023-03-23 08:58:56 +00:00
Han Fei
02de4ad6df address comments 2023-03-22 17:50:19 +01:00
Han Fei
575c4263a3 address comments 2023-03-22 17:47:25 +01:00
Yatsishin Ilya
5786aba5eb Merge remote-tracking branch 'origin/master' into clickhouse-help 2023-03-22 14:08:22 +00:00
serxa
4321569fc4 Merge branch 'segmentator-fix' of github.com:ClickHouse/ClickHouse into segmentator-fix 2023-03-22 13:51:23 +00:00
serxa
f3c12b8543 fix per-thread perf counters update 2023-03-22 13:51:07 +00:00
Antonio Andelic
65b99ec6a7 Merge branch 'master' into alternative-keeper-configs 2023-03-22 12:22:57 +00:00
vdimir
023d14a894 Support transformQueryForExternalDatabase for analyzer 2023-03-22 08:59:04 +01:00
Vitaly Baranov
4318212c73 Make a condition more clear. 2023-03-21 21:52:59 +01:00
Vitaly Baranov
096d0132af More corrections. 2023-03-21 21:52:59 +01:00
Vitaly Baranov
0a0e9d060f Corrections after review. 2023-03-21 21:52:59 +01:00
Vitaly Baranov
85d38493f6 Improve ThreadPool 
ThreadPool::setMaxThreads() can now start new threads (if there are scheduled jobs in the queue for them);
ThreadPool::setMaxFreeThreads() can now finish free threads.
 2023-03-21 21:52:59 +01:00
Nikolay Degterinsky
3a8314c13e
Merge pull request #47576 from HarryLeeIBM/hlee-s390x-storage-gen 
Fix SipHash hashing and byte order in GenerateRandom for s389x
 2023-03-21 21:06:08 +01:00
Arthur Passos
77419db791 add a few tests for runtime find_symbols, more are needed 2023-03-21 09:07:06 -03:00
Kseniia Sumarokova
3c550b4314
Merge pull request #46647 from kssenii/named-collections-finish 
Named collections: finish replacing old code for storages
 2023-03-21 12:36:46 +01:00
Kseniia Sumarokova
a9bcd022d5
Merge pull request #46241 from kssenii/named-collections-granular-access 
Allow separate grants for every named collection
 2023-03-21 12:27:53 +01:00
Vladimir C
5add1e1b2e
Merge pull request #47684 from ClickHouse/vdimir/improve_log_queries_cut 2023-03-21 11:17:54 +01:00
Robert Schulze
3b7f929001
Merge branch 'master' into rs/qc-cachebase 2023-03-20 17:08:47 +01:00
vdimir
8c48714236
wipeSensitiveDataAndCutToLength take into accound suffix length 2023-03-20 15:54:19 +00:00
kssenii
75624c52f7 Merge remote-tracking branch 'upstream/master' into named-collections-granular-access 2023-03-20 16:38:14 +01:00
vdimir
bb7a8ed862
Apply log_queries_cut_to_length in MergeTreeWhereOptimizer 2023-03-20 15:38:09 +00:00
Vitaly Baranov
198409e12a
Merge pull request #46085 from aalexfvk/alexfvk/store_udf_in_zookeeper 
Replication of user-defined SQL functions using ZooKeeper
 2023-03-20 13:41:29 +01:00
Kruglov Pavel
1d12e85d55
Merge branch 'master' into non-blocking-connect 2023-03-20 13:23:55 +01:00
kssenii
cae3b335d6 Merge remote-tracking branch 'upstream/master' into named-collections-finish 2023-03-20 11:23:22 +01:00
Alexey Milovidov
698fcba525
Merge branch 'master' into fix-mysql-leak 2023-03-20 07:37:01 +03:00
Alexey Milovidov
ddf29b69f9
Merge branch 'master' into fuzzer-of-data-formats 2023-03-20 03:26:51 +03:00
kssenii
ea375ef989 Fix memory leak 2023-03-20 00:40:06 +01:00
Robert Schulze
0e01e912fe
Preparation for libcxx(abi), llvm, clang-tidy 16 2023-03-19 17:25:41 +00:00
Robert Schulze
c4664c70a1
Merge branch 'master' into rs/qc-cachebase 2023-03-19 15:23:24 +01:00
Alexey Milovidov
6e1b45d1d6 Fix exotic trash 2023-03-19 06:48:18 +01:00
Alexey Milovidov
a99f10e16f Better error messages 2023-03-19 05:34:44 +01:00
Alexey Milovidov
ef01579726 Fix trash - the code written in C with pointers 2023-03-19 03:19:15 +01:00
Alexander Tokmakov
f1df91cd09
Merge pull request #47659 from ClickHouse/fix_993 
Fixes for 993 and friends
 2023-03-18 14:46:43 +03:00
Sema Checherinda
0b641fcead
Merge pull request #47564 from CheSema/remove-redundant 
remove counters for threads, fix negative counters
 2023-03-18 02:23:52 +01:00
Alexander Tokmakov
d11aee2b0c
Merge branch 'master' into fix_993 2023-03-17 21:28:23 +03:00
Sema Checherinda
e8295be9be if no group then do attachInternalTextLogsQueue to the thread only 2023-03-17 15:56:30 +01:00
Robert Schulze
f72a337074
Remove cruft from build 
No need to check compiler flags, clang >= 15 supports all of them.
 2023-03-17 13:44:04 +00:00
kssenii
bb0beb7449 Merge remote-tracking branch 'upstream/master' into named-collections-finish 2023-03-17 13:02:36 +01:00
Sema Checherinda
e2af8f6ed6 add a comment, rename a field 2023-03-17 11:59:44 +01:00
Aleksei Filatov
886b530963 [rev: 1] Fix review remarks 2023-03-17 13:56:05 +03:00
Aleksei Filatov
f84aa2f4ca Add classes for replication UDF to ZooKeeper 2023-03-17 13:56:04 +03:00
Sergei Trifonov
1a84203c7f
Merge pull request #47656 from ClickHouse/background_pool_limit_metric 
Add background pools size metrics
 2023-03-17 11:49:53 +01:00
Alexander Tokmakov
d621b2c4ad fix intersecting parts, better fault injections 2023-03-16 21:28:07 +01:00
Sema Checherinda
3c6deddd1d work with comments on PR 2023-03-16 19:55:58 +01:00
serxa
3ad7194e31 add pool size metrics 2023-03-16 17:31:41 +00:00
Robert Schulze
eb206bbe25
Fix clang-tidy build 2023-03-16 16:08:24 +00:00
Robert Schulze
7d7052350c
Merge branch 'master' into rs/qc-cachebase 2023-03-16 10:42:17 +01:00
sichenzhao
88de8d4cb3
Merge branch 'master' into cgroup_v2 2023-03-15 14:35:47 -07:00
alesapin
ef7508469c Fix wait for zero-copy replication move during manual move 2023-03-15 22:29:24 +01:00
Sema Checherinda
67ac858a52 remove counters for threads, fix negative counters 2023-03-15 22:22:22 +01:00
Dmitry Novik
be40218a9a
Revert "Revert #46622 (test_async_insert_memory)" 2023-03-15 22:14:54 +01:00
Han Fei
a532503466
Merge branch 'master' into hanfei/optimize-regexp-tree-1 2023-03-15 17:56:01 +01:00
Han Fei
d78a9e03ad refine 2023-03-15 15:38:11 +01:00
avogar
17f5356178 Improve async reading from socket 2023-03-15 12:18:08 +00:00
robot-ch-test-poll1
43db7c59d9
Merge pull request #47260 from ClickHouse/vdimir/storage_join_insert_select_deadlock 
Forbid insert select for the same StorageJoin
 2023-03-15 12:52:29 +01:00
Robert Schulze
2f75a6da61
Merge remote-tracking branch 'origin/master' into rs/qc-cachebase 2023-03-15 11:00:53 +00:00
Robert Schulze
75ca2dd0e2
Merge remote-tracking branch 'origin/master' into rs/qc-cachebase 2023-03-15 10:55:53 +00:00
Robert Schulze
5cee479f0c
Merge pull request #47424 from ClickHouse/rs/compile-with-cpp23 
Compile with C++23
 2023-03-15 10:55:42 +01:00
Yatsishin Ilya
45c5275bfb Merge remote-tracking branch 'origin/master' into clickhouse-help 2023-03-14 17:27:25 +00:00
Han Fei
076d33bb03 refine a little bit 2023-03-14 18:15:42 +01:00
Han Fei
01be209e43 fix test 2023-03-14 17:44:02 +01:00
HarryLeeIBM
a32d7a00bc Fix SipHash integer hashing and byte order issue in GenerateRandom for s390x 2023-03-14 11:54:17 -04:00
Sema Checherinda
95e994a9cd
Merge pull request #47304 from arthurpassos/fix_find_first_not_symbols 
Modify find_first_symbols so it works as expected for find_first_not_symbols
 2023-03-14 14:24:05 +01:00
Alexander Tokmakov
5a1ee881f8
Merge pull request #47547 from ClickHouse/filter_too_frequent_logs 
Hide too noisy log messages, fix some tests
 2023-03-14 13:53:21 +03:00
Robert Schulze
73afae2d3f
Introduce TTLCachePolicy 2023-03-14 09:45:20 +00:00
Robert Schulze
eed365c8c8
Cleanup: Slightly more naming consistency (probably rolls back earlier changes) 2023-03-14 09:11:36 +00:00
Robert Schulze
b985795593
Cleanup: Group ICachePolicy methods 2023-03-14 08:38:22 +00:00
Robert Schulze
e1fb25a004
Extend CacheBase with getWithKey() method 2023-03-14 08:38:22 +00:00
Robert Schulze
f1450923cb
Extend CacheBase with dump() method 2023-03-14 08:38:22 +00:00
Robert Schulze
e03618357e
Cleanup: Acquire lock in CacheBase::maxSize() 
The upcoming TTLPolicy will likely have a runtime-configurable cache
size.
 2023-03-14 08:38:22 +00:00
Michael Kolupaev
d3a514d221 Compress marks in memory 2023-03-13 16:29:00 -07:00
Alexander Tokmakov
ba6c6255a3 hide too noisy log messages, fix some tests 2023-03-14 00:23:36 +01:00
Alexey Milovidov
a411ae10b5
Merge pull request #47486 from ClickHouse/revert_25674 
Remove "asynchronous connection drain"
 2023-03-13 21:40:39 +03:00
Han Fei
de8d0040a4 refine code 2023-03-13 18:34:47 +01:00
Robert Schulze
1d8004756a
Cleanup: Reuse aliases instead of redeclaring it 2023-03-13 15:11:12 +00:00
Robert Schulze
657aa65446
Cleanup: Untangle CacheBase's constructors 
This prepares for the custom TTLCachePolicy which users should not be
able to configure explicitly.
 2023-03-13 15:11:12 +00:00
Vladimir C
9cb018545c
Merge pull request #47499 from FrankChen021/span_kind 2023-03-13 11:35:03 +01:00
Robert Schulze
475415e421
Cleanup: Move on_weight_loss_function into concrete cache policies 
Makes ICachePolicy a pure abstract interface
 2023-03-13 09:20:07 +00:00
Robert Schulze
d165a15c58
Cleanup: Simplify some typedefs 2023-03-13 09:20:02 +00:00
Robert Schulze
614810e471
Cleanup: Fix file name typo 2023-03-13 09:19:58 +00:00
Robert Schulze
9f6cb98c61
Cleanup: Remove default parameters from (S)LRUCachePolicy 
Allows to add new mandatory parameters without accidentally breaking
something.
 2023-03-13 09:19:55 +00:00
Robert Schulze
39898fbe64
Cleanup: max_elements_size --> max_entries 
To disambiguate the maximum number of entries from the maximum byte size
of an entry.
 2023-03-13 09:19:51 +00:00
Robert Schulze
7d5bc0d8c9
Cleanup: TrivialWeightFunction --> EqualWeightFunction 
A true "trival" weight function would return .size() of the cache entry.
 2023-03-13 09:19:45 +00:00
Robert Schulze
c300f5e68f
Cleanup: max_size --> max_size_in_bytes 
To disambiguate the threshold from the maximum number of cache entries.
 2023-03-13 09:19:39 +00:00
Frank Chen
868756ea76 Fix style 2023-03-13 09:40:55 +08:00
Alexey Milovidov
f445649845
Merge branch 'master' into rs/compile-with-cpp23 2023-03-12 21:33:20 +03:00
Frank Chen
dfcb9c9431 Add SpanKind 2023-03-12 21:30:29 +08:00
Alexander Tokmakov
7b1b238d0b Revert "Merge pull request #25674 from amosbird/distributedreturnconnection" 
This reverts commit 5ffd99dfd4, reversing
changes made to 2796aa333f.
 2023-03-11 19:09:47 +01:00
Alexander Tokmakov
0359b3e336 Revert "Merge pull request #46622 from ClickHouse/async-insert-memory-fix" 
This reverts commit 67469ad46b, reversing
changes made to 678e4250cd.
 2023-03-11 17:01:10 +01:00
Robert Schulze
348a40aa5d
Compile with C++23 2023-03-10 19:11:58 +00:00
Dmitry Novik
4f24a7bd11 Merge remote-tracking branch 'origin/master' into background-memory-tracker 2023-03-10 18:57:45 +00:00
Han Fei
39a1185486 fix test 2023-03-10 15:30:29 +01:00
Arthur Passos
dd77962048 refactor to use string instead of string_views 2023-03-10 10:42:22 -03:00
Arthur Passos
003bbb2685 initialize empty string_view with empty string so std::string_view::begin does not point to null 2023-03-09 13:19:51 -03:00
vdimir
c9e48f37cb
Forbid insert select for the same StorageJoin 2023-03-09 12:28:44 +00:00
Alexander Tokmakov
38f9f08e1a
Merge pull request #47268 from myrrc/improvement/more-to-base-3 
use std::lerp, constexpr hex.h
 2023-03-08 19:07:34 +03:00
Yatsishin Ilya
6cde37093a Merge remote-tracking branch 'origin/master' into clickhouse-help 2023-03-08 13:34:18 +00:00
Alexander Tokmakov
f5cf039190 check if a symlink exists more carefully 2023-03-08 00:04:55 +01:00
Mike Kot
9920a52c51 use std::lerp, constexpr hex.h 2023-03-07 22:50:17 +00:00
Arthur Passos
5ba1b5d43c add more descriptive tests 2023-03-07 14:04:48 -03:00
Arthur Passos
a42dee5bed
Merge branch 'ClickHouse:master' into fix_find_first_not_symbols 2023-03-07 12:19:23 -03:00
Arthur Passos
bf365cec8c Modify find_first_symbols so it works as expected for find_first_not_symbols 2023-03-07 10:08:04 -03:00
Antonio Andelic
915a04c0ec
Merge pull request #47261 from ClickHouse/join-threads-when-exception 
Join threads if exception happened in `ZooKeeperImpl` constructor
 2023-03-07 09:32:29 +01:00
Alexey Milovidov
93ba0f4bdf
Update StackTrace.cpp 2023-03-07 01:30:44 +03:00
sichenzhao
500f56430f
Merge branch 'master' into cgroup_v2 2023-03-06 13:38:33 -08:00
Han Fei
420108a7a0 support alternatives 2023-03-06 19:10:36 +01:00
Mike Kot
07a9e486e6 memset for frame pointers' 2023-03-06 19:35:00 +03:00
Mike Kot
4bc443f3c7 initial solution 2023-03-06 19:35:00 +03:00
Antonio Andelic
12525f768c
Add default constructor for MultiReadResponse (#47254) 
* Add default constructor for MultiReadResponse
* Remove optional
* Fix style
 2023-03-06 14:18:01 +01:00
Antonio Andelic
9117c7491d Join threads if exception happened in constructor 2023-03-06 12:55:02 +00:00
Han Fei
b7eef62458
Merge pull request #45491 from azat/dist/async-send-refactoring 
[RFC] Rewrite distributed sends to avoid using filesystem as a queue, use in-memory queue instead
 2023-03-06 12:32:33 +01:00
Nikolay Degterinsky
7f15575500
Merge pull request #47208 from ClibMouse/z_ip_coding_fix 
Fix ip coding on s390x
 2023-03-04 18:18:14 +01:00
Alexander Tokmakov
082194a755
Merge pull request #47175 from ClickHouse/correct_fatal_handling 
More correct handling of fatal errors
 2023-03-04 00:38:00 +03:00
Sema Checherinda
85178c9609
Merge pull request #47102 from CheSema/merge-mutate-do-thread-group 
do flushUntrackedMemory when context switches
 2023-03-03 20:23:25 +01:00
Han Fei
c1e80683c4 Refine OptimizeRegularExpression Function 2023-03-03 17:59:21 +01:00
Suzy Wang
e44e528118 remove unnecessary reverse memcpy 2023-03-03 06:20:18 -08:00
Alexander Tokmakov
cadaf06829
Merge pull request #46681 from ClickHouse/fix_insert_cancellation_in_native_protocol 
Fix queries cancellation when a client dies
 2023-03-03 16:27:20 +03:00
Alexander Tokmakov
b1a288d271
Merge branch 'master' into correct_fatal_handling 2023-03-03 14:50:07 +03:00
Kseniia Sumarokova
386663953c
Merge branch 'master' into named-collections-finish 2023-03-03 12:23:38 +01:00
Alexander Tokmakov
619b282060 more correct handling of fatal errors 2023-03-03 01:22:04 +01:00
LiuNeng
d4c5ab9dcd
Optimize one nullable key aggregate performance (#45772) 2023-03-02 21:01:52 +01:00
Sema Checherinda
7a9f4aab8e do not call adjustWithUntrackedMemory when untracked_memory is 0 2023-03-02 19:21:03 +01:00
Sema Checherinda
a6c2473518 do flushUntrackedMemory when context swith 2023-03-02 19:21:03 +01:00
kssenii
0c4bb3c80f Merge remote-tracking branch 'upstream/master' into named-collections-granular-access 2023-03-02 19:08:29 +01:00
Alexander Tokmakov
c518f3690e Merge branch 'master' into fix_insert_cancellation_in_native_protocol 2023-03-02 18:24:08 +01:00
Yatsishin Ilya
d4304d2028 Merge remote-tracking branch 'origin/master' into clickhouse-help 2023-03-02 16:16:18 +00:00
Robert Schulze
740aeaba1f
Apply some CTAD 2023-03-02 13:36:47 +00:00
Alexey Milovidov
97c0f8a0e0
Merge pull request #47027 from amosbird/clang16 
Prepare to build with clang 16
 2023-03-02 02:10:03 +03:00
Robert Schulze
15fec805fb
Merge pull request #46950 from ClickHouse/rs/uninline-likePatternToRegexp 
Un-inline likePatternToRegexp()
 2023-03-01 21:38:40 +01:00
Alexander Tokmakov
a97e15e36f Merge branch 'master' into fix_insert_cancellation_in_native_protocol 2023-03-01 14:26:41 +01:00
Amos Bird
2d8060d843
Rewrite VersionNumber in ClickHouse way 2023-03-01 11:07:31 +08:00
Azat Khuzhin
572dd76eae Extend ConcurrentBoundedQueue to deque 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Kruglov Pavel
66634cfd7c
Merge pull request #46618 from kitaisreal/map-field-to-string-fix 
Map field to string fix
 2023-02-28 20:29:40 +01:00
Robert Schulze
2c0395b346
Merge pull request #46814 from ClickHouse/rs/search-refactoring 
Cleanup string search code
 2023-02-28 15:04:19 +01:00
Maksim Kita
0764649c5d
Merge pull request #46632 from ClickHouse/analyzer-limit-settings-2 
Analyzer - apply limit and offset settings - second attempt
 2023-02-27 21:39:34 +03:00
kssenii
ad88251ee7 Fix tests 2023-02-27 17:42:04 +01:00
Robert Schulze
35f4b42608
Move likePatternToRegexp from Functions/ to Common/ 2023-02-27 16:41:15 +00:00
Robert Schulze
5c8edcc2e7
Merge pull request #46940 from ClickHouse/rs/sizes-of-arrays-dont-match 
Typo: SIZES_OF_ARRAYS_DOESNT_MATCH --> SIZES_OF_ARRAYS_DONT_MATCH
 2023-02-27 14:33:09 +01:00
kssenii
f398c5d471 Fix style 2023-02-27 12:22:27 +01:00
Robert Schulze
22d09caea2
SIZES_OF_ARRAYS_DOESN_MATCH --> SIZES_OF_ARRAYS_DONT_MATCH 2023-02-27 08:31:40 +00:00
Maksim Kita
8ad016dbe6 Map field to string fix 2023-02-26 13:38:45 +01:00
Azat Khuzhin
03e90cd7cf Add new metrics to system.asynchronous_metrics 
The following metrics can be useful to calculate various rates (i.e.
disk/network IO rates):
- AsynchronousHeavyMetricsUpdateInterval
- AsynchronousMetricsUpdateInterval

The following had been added by analogy with the
AsynchronousMetricsCalculationTimeSpent:
- AsynchronousHeavyMetricsCalculationTimeSpent

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-25 21:13:15 +01:00
Alexey Milovidov
f28ab14759 Better handling of fatal errors 2023-02-24 23:29:06 +01:00
kssenii
c2bcc4119f Better 2023-02-24 20:57:37 +01:00
Yakov Olkhovskiy
da326dda03 comments fix 2023-02-24 19:19:51 +00:00
kssenii
f2a9109fe1 Merge remote-tracking branch 'upstream/master' into named-collections-granular-access 2023-02-24 15:20:46 +01:00
Robert Schulze
627961d9db
More cosmetics 2023-02-24 10:14:58 +00:00
Robert Schulze
421afeeae0
Shuffle class order (and just that) 2023-02-24 10:13:35 +00:00
Robert Schulze
ef529de7db
Cosmetics 2023-02-24 10:12:47 +00:00
Alexander Tokmakov
fe92fd8a61 Merge branch 'master' into fix_insert_cancellation_in_native_protocol 2023-02-23 22:35:31 +01:00
Robert Schulze
6aa4e93284
Merge branch 'master' into StringSearcher 2023-02-23 11:54:05 +01:00
Jiebin Sun
d6710d9b34 Align all the SSE4.1 requirement and use needle_size 
Align all the SSE4.1 requirement from StringSearcher. Use needle_size
in while loop to make the code clean.

Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>
 2023-02-22 16:15:26 -05:00
Yakov Olkhovskiy
99095446af review suggestions 2023-02-22 17:22:13 +00:00
Jiebin Sun
1f62135ba7 Make the optimized SIMD StringSearcher clean 
This patch has revised the name of value and added comments to make
the SIMD StringSearcher clean and easy to understand based on pull
request 46289.

Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>
 2023-02-22 12:18:21 -05:00
Jiebin Sun
d220e7f4fc Optimize the SIMD StringSearcher if needle_size is large 
This patch offers an additional optimization when the needle_size is
large. If the needle_size is larger than the haystack_size, there is
no need to search any more.

The optimized SIMD StringSearcher has led at most 41.7% than Volnitsky
algorithm when the needle_size is less than 21, and fallen behind only
about 1% even when the needle_size is bigger than 50, which is not
considered as a common case.

Test platform: ICX server
Test query: SELECT COUNT(*) FROM hits WHERE URL LIKE '%{Needle}%';

Needle_size	opt/baseline
5		141.7%
6		129.4%
8		118.5%
9		112.3%
10		107.4%
14		103.4%
20		100.2%
21		100.7%
51		99.0%

Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>
 2023-02-22 11:58:17 -05:00
Jiebin Sun
f5a6a86dec Optimize the SIMD StringSearcher by searching first two chars 
This patch offers the optimized SIMD StringSearcher by searching the first
and second chars together rather than only the first char, which will result
in big performance gain. The patch also provides a quick path when the needle
size is 1.

With this patch, I have tested the 43 queries in clickbench on ICX server.
Query 20 has got 35% performance gain. Other StringSearcher related queries
have got around 10% performance improvement. And the overall geomean of all
the queries has got 4.1% performance gain.

Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>
 2023-02-22 11:55:30 -05:00
Alexander Tokmakov
e660c0838c fix multipart requests 2023-02-22 17:54:35 +01:00
Yatsishin Ilya
80857638c1 Merge remote-tracking branch 'origin/master' into clickhouse-help 2023-02-22 13:58:17 +00:00
kssenii
f0ca3ef207 Merge remote-tracking branch 'upstream/master' into named-collections-granular-access 2023-02-22 14:02:29 +01:00
kssenii
f518ddd96d Merge remote-tracking branch 'upstream/master' into named-collections-finish 2023-02-22 13:43:18 +01:00
Kruglov Pavel
3ba3fdbfa3
Merge pull request #46607 from kssenii/delay-loading-of-named-collections 
Do not load named collections on server startup (on first access instead)
 2023-02-22 13:22:34 +01:00
Dmitry Novik
67469ad46b
Merge pull request #46622 from ClickHouse/async-insert-memory-fix 
Fix MemoryTracker counters for async inserts
 2023-02-22 12:27:05 +01:00
Sichen Zhao
5772b9769e fix types 2023-02-21 22:49:40 -08:00
Sichen Zhao
d0de9d5887 updates for comments, treat special case for string value max memory, and extract shared code out 2023-02-21 22:47:26 -08:00
Alexander Tokmakov
592af6d652 fix incomplete interst through http 2023-02-22 02:34:03 +01:00
Yakov Olkhovskiy
cf493d1dfb replace settings limit and offset with corresponding expression nodes 2023-02-21 22:36:02 +00:00
Dmitry Novik
ad1e5f3918 Review fixes 2023-02-21 22:06:17 +00:00
Vladimir C
a791250a4a
Merge pull request #38614 from bharatnc/ncb/profile-events-partlog 2023-02-21 11:06:44 +01:00
Kseniia Sumarokova
24d0490ef3
Merge branch 'master' into delay-loading-of-named-collections 2023-02-21 00:24:18 +01:00
kssenii
a54b011670 Finish for mysql 2023-02-20 21:37:38 +01:00
Dmitry Novik
3c4c527bce Fix MemoryTracker counters for async inserts 2023-02-20 18:40:57 +00:00
kssenii
9d16205c8a Load named collections on first access 2023-02-20 16:33:45 +01:00
Alexey Milovidov
5ef159fa10
Merge pull request #46565 from ClickHouse/remove-pvs-studio 
Remove PVS-Studio
 2023-02-20 17:10:53 +03:00
Robert Schulze
135170b27d
Merge pull request #44973 from ClickHouse/remove-smp-limitation 
Slightly modify the detection of the number of CPU cores
 2023-02-20 11:22:39 +01:00
Alexey Milovidov
d8cda3dbb8 Remove PVS-Studio 2023-02-19 23:30:05 +01:00
Alexey Milovidov
a22f0eee2a
Merge branch 'master' into Aggregate_Sum_AVX512 2023-02-19 04:03:45 +03:00
Dmitry Novik
728434821a Merge remote-tracking branch 'origin/master' into background-memory-tracker 2023-02-16 12:35:07 +00:00
Alexey Milovidov
e8d1a2f099
Update AsynchronousMetrics.cpp 2023-02-15 23:52:53 +03:00
Dmitry Novik
e7a94a28ee
Merge branch 'master' into background-memory-tracker 2023-02-15 19:39:28 +01:00
kssenii
0c53f33e6f Make code more common: add GlobalWithParameter level 2023-02-15 16:59:43 +01:00
Nikolay Degterinsky
494fd472f0
Merge branch 'master' into ulid 2023-02-15 07:24:32 +01:00
vdimir
1b715a3388
Allow scope override in ThreadStatus::attachProfileCountersScope 2023-02-14 14:11:13 +00:00
vdimir
4c1aeb7972
Use overload of PartLog::addNewPart with profile_events 2023-02-14 14:11:12 +00:00
vdimir
fa08f72bba
Move subthread_profile_events to ThreadStatus 2023-02-14 14:11:11 +00:00
vdimir
994dcb93f9
Rename ScopedProfileEvents -> ProfileEventsScope 2023-02-14 14:11:10 +00:00
vdimir
a228f7f419
[wip] ProfileCounters for each part 2023-02-14 14:11:09 +00:00
Alexey Milovidov
f15623141e
Fix error 2023-02-14 14:11:08 +00:00
Alexey Milovidov
f554ff9d99
Remove trash 2023-02-14 14:11:08 +00:00
John Skopis
f7604cc686
[feat] Add ProfileEvents map to PartLog 
closes #10316
 2023-02-14 14:11:02 +00:00
Nikolay Degterinsky
09da5575e3 Merge remote-tracking branch 'upstream/master' into ulid 2023-02-14 13:41:17 +00:00
robot-clickhouse-ci-2
ea5b645af5
Merge pull request #46349 from ClickHouse/fix-crash-int-overflow-hash-table-deserialization-agg-state 
Check overflow in HashTable::alloc
 2023-02-14 10:02:13 +01:00
Alexander Gololobov
3f5887248e
Update HashTable.h 2023-02-13 17:30:57 +01:00
Nikolai Kochetov
ea19a0d46e
Update src/Common/HashTable/HashTable.h 
Co-authored-by: Alexander Gololobov <440544+davenger@users.noreply.github.com>
 2023-02-13 17:15:41 +01:00
Nikolai Kochetov
5d77407f39 Check overflow in HashTable::alloc 2023-02-13 14:50:23 +00:00
Suzy Wang
e140bed3ea
Merge branch 'master' into z-build-0120 2023-02-13 09:06:48 -05:00
Alexey Milovidov
2e57fab8e3 Instrumentation of callbacks for distributed queries 2023-02-12 10:04:05 +01:00
kssenii
3067c1d723 Merge remote-tracking branch 'upstream/master' into resubmit-prefetches 2023-02-11 11:36:23 +01:00
Kseniia Sumarokova
3be36c7979
Merge pull request #41976 from kssenii/allow-single-disk-instead-of-storage-policy 
Allow configuring storage as `SETTINGS disk='<disk_name>'` (instead of `storage_policy`) and explicit disk creation `SETTINGS disk=disk(<disk_configuration>)`
 2023-02-10 21:38:50 +01:00
Alexey Milovidov
d5d87bc6b7
Merge pull request #46152 from ClickHouse/with-fill-bigint 
More convenient usage of big integers and ORDER BY WITH FILL.
 2023-02-10 20:05:32 +03:00
Vladimir C
bd39e29cab
Merge pull request #46065 from aiven-sal/aiven-sal/siphash128 2023-02-10 10:41:45 +01:00
Suzy Wang
64313d7020
Merge branch 'master' into z-build-0120 2023-02-09 14:49:22 -05:00
Vitaly Baranov
39bacfb9dc
Merge branch 'master' into allow-single-disk-instead-of-storage-policy 2023-02-09 13:16:55 +01:00
Alexey Milovidov
d9fbf643bc
Merge pull request #45226 from hanfei1991/hanfei/gwp-asan 
Introduce llvm/gwp-asan
 2023-02-09 09:24:52 +03:00
pufit
7cf71d1f82
Merge pull request #46144 from ClickHouse/46131-fix-config-attributes-merge 
Fix config merge for XML attributes.
 2023-02-08 23:28:13 -05:00
Alexey Milovidov
292a5abbc9 Merge remote-tracking branch 'origin/master' into with-fill-bigint 2023-02-09 01:47:51 +01:00
Alexander Tokmakov
4fd4058259
Merge pull request #45629 from arthurpassos/fix_cares_crash 
Cancel c-ares failed requests and retry on system interrupts to prevent callbacks with dangling references and premature resolution failures
 2023-02-09 02:29:29 +03:00
Alexander Tokmakov
8101b044fa
Merge pull request #46091 from azat/sanity-assertions 
Sanity assertions for closing file descriptors
 2023-02-09 01:02:03 +03:00
Han Fei
64b84f8e31 fix style 2023-02-08 22:16:28 +01:00
kssenii
b0b865c32e Resubmit prefetches 2023-02-08 21:26:24 +01:00
Han Fei
8f665b5bbf fix 2023-02-08 18:28:06 +01:00
Han Fei
a4efb1fbda fix compile 2023-02-08 18:06:36 +01:00
Han Fei
d9aab84701 fix fork hang 2023-02-08 16:20:42 +01:00
pufit
e833decd88 try to fix memory leak 2023-02-08 15:17:46 +00:00
Antonio Andelic
45d4ba59dc
Merge pull request #46068 from bigo-sg/alloc_map_metrics 
Add new metric to record allocations times and bytes using mmap
 2023-02-08 15:49:20 +01:00
Alexey Milovidov
4f2a58ddbd Fix style 2023-02-08 08:44:31 +01:00
Alexey Milovidov
0791b85f57 Remove bits of trash 2023-02-08 08:18:31 +01:00
Alexey Milovidov
d66beb9ac4 Allow accurate comparison of Big Int with other integers 2023-02-08 07:56:00 +01:00
pufit
c066c0dfdb Fix config merge for an attributes 2023-02-07 23:46:13 +00:00
Dmitry Novik
d43d17f59a Address the review 2023-02-07 16:05:02 +00:00
Yatsishin Ilya
1baa15d603 Merge remote-tracking branch 'origin/master' into clickhouse-help 2023-02-07 15:46:59 +00:00
Han Fei
0f7defb87f
Merge branch 'master' into hanfei/gwp-asan 2023-02-07 14:55:55 +01:00
Robert Schulze
690d873126
Merge pull request #46088 from ClickHouse/rs/fix-liburing-build 
Fix build with `-DENABLE_LIBURING=0` (or `-DENABLE_LIBRARIES=0`)
 2023-02-07 12:16:14 +01:00
Azat Khuzhin
8cc41b7f41 Check return value of ::close() 
Note, that according close(2), EINTR should not be retriable for close()

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-07 11:28:22 +01:00
Salvatore Mesoraca
001b714d3e
Common: SipHash: add original 128-bit output implementation 2023-02-07 11:19:04 +01:00
Azat Khuzhin
e83163e25f Add sanity check of evtimer_del() in QueryProfiler 
Saw this one time on CI with EINVAL [1], while this was EINVAL this
time, next time it can be completelly different fd and break something
in a different place.

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/45654/1716af465d376f5335720b5045bcccdd9e1823aa/fast_test.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-06 20:13:05 +01:00
Dmitry Novik
1e065b32f3 Add MemoryTracker for the background tasks 2023-02-06 17:25:58 +00:00
Robert Schulze
8840aa9242
Fix build with -DENABLE_LIBURING=0 (or -DENABLE_LIBRARIES=0) 
Follow-up to #38456

The problem was that code which uses liburing was conditionally build
depending on define OS_LINUX and not based on whether liburing was
enabled/disabled at build time.
 2023-02-06 16:57:42 +00:00
Robert Schulze
4c89327c9f
Add TODO comment to MultiVersion.h 
Follow-up to #46057
 2023-02-06 11:04:58 +00:00
taiyang-li
7b10ac8d5b add new metric to record allocations times and bytes using mmap 2023-02-06 17:50:45 +08:00
Konstantin Morozov
ff05162904 change mutex to lock-free 2023-02-05 19:25:09 +03:00
kssenii
5abbc7cad6 Cleanup 2023-02-04 19:41:41 +01:00
kssenii
cce23cee51 Fix build 2023-02-04 13:49:32 +01:00
Yatsishin Ilya
f4cfd8a2d9 Merge remote-tracking branch 'origin' into clickhouse-help 2023-02-03 20:08:23 +00:00
kssenii
536c02ca8a Adding custom field continued 2023-02-03 21:05:18 +01:00
kssenii
5f124263a6 Revert changes with complex SettingsChanges 2023-02-03 21:04:45 +01:00
Sichen Zhao
aa5230b525 updates path name 2023-02-03 11:18:53 -08:00
Alexander Tokmakov
3f11948bb0
Merge branch 'master' into stack_trace_in_part_log 2023-02-03 20:05:00 +03:00
kssenii
12b62de374 Add field CustomType 2023-02-03 17:07:02 +01:00
Robert Schulze
85cbb9288c
Merge pull request #45456 from FrankChen021/uncaught_exception 
Fix uncaught exception in HTTPHandler
 2023-02-03 15:26:02 +01:00
Anton Popov
cdbe145bc1
Merge pull request #45796 from CurtizJ/fix-leak-in-azure-sdk 
Fix test `test_azure_blob_storage_zero_copy_replication ` (memory leak in azure sdk)
 2023-02-03 14:16:19 +01:00
Frank Chen
d38adfab30 Merge two overridden functions as one 2023-02-03 15:27:45 +08:00
Sichen Zhao
d4ca058643 updates to support cgroupv2 as well 2023-02-02 22:19:29 -08:00
Frank Chen
d3a05a11da Merge remote-tracking branch 'remotes/github/master' into stack_trace_in_part_log 2023-02-03 11:39:43 +08:00
Frank Chen
7ad4b3176a Clean code 2023-02-03 11:13:45 +08:00
Frank Chen
93ead69b7f Resolve review comments 2023-02-03 11:05:54 +08:00
kssenii
a391de18a1 Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2023-02-02 20:49:41 +01:00
Nikolai Kochetov
6cd0b51127
Merge pull request #45871 from ClickHouse/fix-ipv6-parser 
Fix ipv6 parser
 2023-02-01 14:28:54 +01:00
Robert Schulze
b512316586
Merge pull request #45682 from ClickHouse/rename-qrc-to-qc 
Rename "Query Result Cache" to "Query Cache"
 2023-02-01 11:23:29 +01:00
Alexey Milovidov
bf9f62dcbd
Merge pull request #38456 from sauliusvl/iouring 
Re-introduce io_uring read method
 2023-02-01 06:41:32 +03:00
Yakov Olkhovskiy
3d64c84571 fix ipv6 parser 2023-02-01 01:02:58 +00:00
Yatsishin Ilya
98edb9f06b Update help for clickhouse tools and add test 2023-01-31 12:19:37 +00:00
Robert Schulze
325c6bdf3d
Renaming: "Query Result Cache" --> "Query Cache" 
Reasons:

- The cache will at some point store intermediate results as opposed to
  only query results. We should change the terminology now without
  having to worry about backward compat.

- Equivalent caches in MySQL (1) and Starrocks (2) are called "query
  cache".

- The new name is ca. 13.8% more catchy.

(1) https://dev.mysql.com/doc/refman/5.6/en/query-cache.html
(2) https://docs.starrocks.io/en-us/2.5/using_starrocks/query_cache
 2023-01-31 09:54:34 +00:00
Alexey Milovidov
7aa10a430a
Merge branch 'master' into hanfei/gwp-asan 2023-01-31 04:13:59 +03:00
Anton Popov
31e8b692f4 fix typo 2023-01-31 01:13:23 +00:00
Anton Popov
839cd614fb fix memory leak in azure sdk 2023-01-31 01:01:10 +00:00
Vitaly Baranov
2e3a3cc4dc
Merge pull request #45701 from vitlibar/add-setting-allow-head-object-request 
Add new S3 setting allow_head_object_request
 2023-01-30 17:41:09 +01:00
Han Fei
4971358de5 fix init 2023-01-30 17:07:01 +01:00
Arthur Passos
7b1ceaa2c5
Merge branch 'master' into fix_cares_crash 2023-01-30 12:05:56 -03:00
Vladimir C
e7e8ae979f
Merge pull request #45271 from jh0x/feature-array-shuffle 2023-01-30 11:54:07 +01:00
Robert Schulze
66392b873c
Merge pull request #45513 from aiven-sal/aiven-sal/siphash_pr 
Keyed SipHash
 2023-01-30 11:10:00 +01:00
Alexey Milovidov
a7299746c7
Merge pull request #45743 from ClickHouse/oom-message 
Improve MEMERY_LIMIT_EXCEEDED exception message
 2023-01-30 00:46:04 +03:00
Robert Schulze
33a719b9be
Update getNumberOfPhysicalCPUCores.cpp 2023-01-29 21:37:19 +01:00
Robert Schulze
15ae2d1de5
Merge branch 'master' into aiven-sal/siphash_pr 2023-01-29 21:06:52 +01:00
Robert Schulze
1bf280cb50
Handle obscure case that /proc/cpuinfo is unmounted 2023-01-29 17:25:08 +00:00
alesapin
631c8fb155
Merge branch 'master' into add-setting-allow-head-object-request 2023-01-29 14:58:27 +01:00
Dmitry Novik
ec1f6bfd37 Improve MEMERY_LIMIT_EXCEEDED exception message 2023-01-29 01:52:37 +00:00
Saulius Valatka
8505b8c57a add profile events for failed/succeeded CQE completions 2023-01-28 21:54:44 +02:00
Saulius Valatka
8fa9a99ba1 implement pending request queue to prevent CQ overflows 2023-01-28 21:54:44 +02:00
Saulius Valatka
ac2c921bdf add initial io_uring support 2023-01-28 21:54:44 +02:00
Azat Khuzhin
6b42b66257 Simply filesystem helpers to check is-readable/writable/executable 
Use one system call - faccessat(), over multiple system calls.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-01-27 21:11:10 +01:00
Joanna Hulboj
9a559b5475 FIXUP: More comments about shuffle 2023-01-27 20:07:56 +00:00
Joanna Hulboj
31eb936457 Added Fisher-Yates shuffle and partial-shuffle 2023-01-27 20:07:55 +00:00
Arthur Passos
39cbf4d4dc make random number generator thread safe 2023-01-27 12:14:07 -03:00
Arthur Passos
e559a6b1d2 address some comments 2023-01-27 12:07:00 -03:00
Vitaly Baranov
d02b255b55 Add new setting "allow_head_object_request" to S3::RequestSettings. 2023-01-27 15:10:34 +01:00
Alexander Tokmakov
c366806c3e
Merge pull request #45527 from ClickHouse/exception_message_patterns4 
Better formatting for exception messages 2
 2023-01-27 15:31:52 +03:00
Salvatore Mesoraca
2e72e27206
common: siphash: add support for custom keys 2023-01-27 13:00:53 +01:00
Alexey Milovidov
5b257ab806
Merge pull request #45233 from ClickHouse/improve_week_day 
Revert "Revert "Improve week day""
 2023-01-27 02:44:17 +03:00
Arthur Passos
ea38df2c28 add comments to the code 2023-01-26 13:54:12 -03:00
Alexander Tokmakov
a584ad0eb1 forbid runtime strings 2023-01-26 10:52:47 +01:00
Alexander Tokmakov
9b670946db Merge branch 'master' into exception_message_patterns5 2023-01-26 00:41:32 +01:00
Arthur Passos
4a10f49ca8 spacing 2023-01-25 19:21:45 -03:00
Arthur Passos
912ac6fb05 remove maybe_unused 2023-01-25 19:09:35 -03:00