thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 19:32:07 +00:00
Code Issues Packages Projects Releases Wiki Activity
115,357 Commits 9,319 Branches 1,783 Tags 2.1 GiB
78c89da8bb
Commit Graph

5754 Commits

Author SHA1 Message Date
Sema Checherinda
0b641fcead
Merge pull request #47564 from CheSema/remove-redundant 
remove counters for threads, fix negative counters
 2023-03-18 02:23:52 +01:00
Alexander Tokmakov
d11aee2b0c
Merge branch 'master' into fix_993 2023-03-17 21:28:23 +03:00
Sema Checherinda
e8295be9be if no group then do attachInternalTextLogsQueue to the thread only 2023-03-17 15:56:30 +01:00
Robert Schulze
f72a337074
Remove cruft from build 
No need to check compiler flags, clang >= 15 supports all of them.
 2023-03-17 13:44:04 +00:00
kssenii
bb0beb7449 Merge remote-tracking branch 'upstream/master' into named-collections-finish 2023-03-17 13:02:36 +01:00
Sema Checherinda
e2af8f6ed6 add a comment, rename a field 2023-03-17 11:59:44 +01:00
Aleksei Filatov
886b530963 [rev: 1] Fix review remarks 2023-03-17 13:56:05 +03:00
Aleksei Filatov
f84aa2f4ca Add classes for replication UDF to ZooKeeper 2023-03-17 13:56:04 +03:00
Sergei Trifonov
1a84203c7f
Merge pull request #47656 from ClickHouse/background_pool_limit_metric 
Add background pools size metrics
 2023-03-17 11:49:53 +01:00
Alexander Tokmakov
d621b2c4ad fix intersecting parts, better fault injections 2023-03-16 21:28:07 +01:00
Sema Checherinda
3c6deddd1d work with comments on PR 2023-03-16 19:55:58 +01:00
serxa
3ad7194e31 add pool size metrics 2023-03-16 17:31:41 +00:00
Robert Schulze
eb206bbe25
Fix clang-tidy build 2023-03-16 16:08:24 +00:00
Robert Schulze
7d7052350c
Merge branch 'master' into rs/qc-cachebase 2023-03-16 10:42:17 +01:00
sichenzhao
88de8d4cb3
Merge branch 'master' into cgroup_v2 2023-03-15 14:35:47 -07:00
alesapin
ef7508469c Fix wait for zero-copy replication move during manual move 2023-03-15 22:29:24 +01:00
Sema Checherinda
67ac858a52 remove counters for threads, fix negative counters 2023-03-15 22:22:22 +01:00
Dmitry Novik
be40218a9a
Revert "Revert #46622 (test_async_insert_memory)" 2023-03-15 22:14:54 +01:00
Han Fei
a532503466
Merge branch 'master' into hanfei/optimize-regexp-tree-1 2023-03-15 17:56:01 +01:00
Han Fei
d78a9e03ad refine 2023-03-15 15:38:11 +01:00
avogar
17f5356178 Improve async reading from socket 2023-03-15 12:18:08 +00:00
robot-ch-test-poll1
43db7c59d9
Merge pull request #47260 from ClickHouse/vdimir/storage_join_insert_select_deadlock 
Forbid insert select for the same StorageJoin
 2023-03-15 12:52:29 +01:00
Robert Schulze
2f75a6da61
Merge remote-tracking branch 'origin/master' into rs/qc-cachebase 2023-03-15 11:00:53 +00:00
Robert Schulze
75ca2dd0e2
Merge remote-tracking branch 'origin/master' into rs/qc-cachebase 2023-03-15 10:55:53 +00:00
Robert Schulze
5cee479f0c
Merge pull request #47424 from ClickHouse/rs/compile-with-cpp23 
Compile with C++23
 2023-03-15 10:55:42 +01:00
Yatsishin Ilya
45c5275bfb Merge remote-tracking branch 'origin/master' into clickhouse-help 2023-03-14 17:27:25 +00:00
Han Fei
076d33bb03 refine a little bit 2023-03-14 18:15:42 +01:00
Han Fei
01be209e43 fix test 2023-03-14 17:44:02 +01:00
HarryLeeIBM
a32d7a00bc Fix SipHash integer hashing and byte order issue in GenerateRandom for s390x 2023-03-14 11:54:17 -04:00
Sema Checherinda
95e994a9cd
Merge pull request #47304 from arthurpassos/fix_find_first_not_symbols 
Modify find_first_symbols so it works as expected for find_first_not_symbols
 2023-03-14 14:24:05 +01:00
Alexander Tokmakov
5a1ee881f8
Merge pull request #47547 from ClickHouse/filter_too_frequent_logs 
Hide too noisy log messages, fix some tests
 2023-03-14 13:53:21 +03:00
Robert Schulze
73afae2d3f
Introduce TTLCachePolicy 2023-03-14 09:45:20 +00:00
Robert Schulze
eed365c8c8
Cleanup: Slightly more naming consistency (probably rolls back earlier changes) 2023-03-14 09:11:36 +00:00
Robert Schulze
b985795593
Cleanup: Group ICachePolicy methods 2023-03-14 08:38:22 +00:00
Robert Schulze
e1fb25a004
Extend CacheBase with getWithKey() method 2023-03-14 08:38:22 +00:00
Robert Schulze
f1450923cb
Extend CacheBase with dump() method 2023-03-14 08:38:22 +00:00
Robert Schulze
e03618357e
Cleanup: Acquire lock in CacheBase::maxSize() 
The upcoming TTLPolicy will likely have a runtime-configurable cache
size.
 2023-03-14 08:38:22 +00:00
Michael Kolupaev
d3a514d221 Compress marks in memory 2023-03-13 16:29:00 -07:00
Alexander Tokmakov
ba6c6255a3 hide too noisy log messages, fix some tests 2023-03-14 00:23:36 +01:00
Alexey Milovidov
a411ae10b5
Merge pull request #47486 from ClickHouse/revert_25674 
Remove "asynchronous connection drain"
 2023-03-13 21:40:39 +03:00
Han Fei
de8d0040a4 refine code 2023-03-13 18:34:47 +01:00
Robert Schulze
1d8004756a
Cleanup: Reuse aliases instead of redeclaring it 2023-03-13 15:11:12 +00:00
Robert Schulze
657aa65446
Cleanup: Untangle CacheBase's constructors 
This prepares for the custom TTLCachePolicy which users should not be
able to configure explicitly.
 2023-03-13 15:11:12 +00:00
Vladimir C
9cb018545c
Merge pull request #47499 from FrankChen021/span_kind 2023-03-13 11:35:03 +01:00
Robert Schulze
475415e421
Cleanup: Move on_weight_loss_function into concrete cache policies 
Makes ICachePolicy a pure abstract interface
 2023-03-13 09:20:07 +00:00
Robert Schulze
d165a15c58
Cleanup: Simplify some typedefs 2023-03-13 09:20:02 +00:00
Robert Schulze
614810e471
Cleanup: Fix file name typo 2023-03-13 09:19:58 +00:00
Robert Schulze
9f6cb98c61
Cleanup: Remove default parameters from (S)LRUCachePolicy 
Allows to add new mandatory parameters without accidentally breaking
something.
 2023-03-13 09:19:55 +00:00
Robert Schulze
39898fbe64
Cleanup: max_elements_size --> max_entries 
To disambiguate the maximum number of entries from the maximum byte size
of an entry.
 2023-03-13 09:19:51 +00:00
Robert Schulze
7d5bc0d8c9
Cleanup: TrivialWeightFunction --> EqualWeightFunction 
A true "trival" weight function would return .size() of the cache entry.
 2023-03-13 09:19:45 +00:00
Robert Schulze
c300f5e68f
Cleanup: max_size --> max_size_in_bytes 
To disambiguate the threshold from the maximum number of cache entries.
 2023-03-13 09:19:39 +00:00
Frank Chen
868756ea76 Fix style 2023-03-13 09:40:55 +08:00
Alexey Milovidov
f445649845
Merge branch 'master' into rs/compile-with-cpp23 2023-03-12 21:33:20 +03:00
Frank Chen
dfcb9c9431 Add SpanKind 2023-03-12 21:30:29 +08:00
Alexander Tokmakov
7b1b238d0b Revert "Merge pull request #25674 from amosbird/distributedreturnconnection" 
This reverts commit 5ffd99dfd4, reversing
changes made to 2796aa333f.
 2023-03-11 19:09:47 +01:00
Alexander Tokmakov
0359b3e336 Revert "Merge pull request #46622 from ClickHouse/async-insert-memory-fix" 
This reverts commit 67469ad46b, reversing
changes made to 678e4250cd.
 2023-03-11 17:01:10 +01:00
Robert Schulze
348a40aa5d
Compile with C++23 2023-03-10 19:11:58 +00:00
Dmitry Novik
4f24a7bd11 Merge remote-tracking branch 'origin/master' into background-memory-tracker 2023-03-10 18:57:45 +00:00
Han Fei
39a1185486 fix test 2023-03-10 15:30:29 +01:00
Arthur Passos
dd77962048 refactor to use string instead of string_views 2023-03-10 10:42:22 -03:00
Arthur Passos
003bbb2685 initialize empty string_view with empty string so std::string_view::begin does not point to null 2023-03-09 13:19:51 -03:00
vdimir
c9e48f37cb
Forbid insert select for the same StorageJoin 2023-03-09 12:28:44 +00:00
Alexander Tokmakov
38f9f08e1a
Merge pull request #47268 from myrrc/improvement/more-to-base-3 
use std::lerp, constexpr hex.h
 2023-03-08 19:07:34 +03:00
Yatsishin Ilya
6cde37093a Merge remote-tracking branch 'origin/master' into clickhouse-help 2023-03-08 13:34:18 +00:00
Alexander Tokmakov
f5cf039190 check if a symlink exists more carefully 2023-03-08 00:04:55 +01:00
Mike Kot
9920a52c51 use std::lerp, constexpr hex.h 2023-03-07 22:50:17 +00:00
Arthur Passos
5ba1b5d43c add more descriptive tests 2023-03-07 14:04:48 -03:00
Arthur Passos
a42dee5bed
Merge branch 'ClickHouse:master' into fix_find_first_not_symbols 2023-03-07 12:19:23 -03:00
Arthur Passos
bf365cec8c Modify find_first_symbols so it works as expected for find_first_not_symbols 2023-03-07 10:08:04 -03:00
Antonio Andelic
915a04c0ec
Merge pull request #47261 from ClickHouse/join-threads-when-exception 
Join threads if exception happened in `ZooKeeperImpl` constructor
 2023-03-07 09:32:29 +01:00
Alexey Milovidov
93ba0f4bdf
Update StackTrace.cpp 2023-03-07 01:30:44 +03:00
sichenzhao
500f56430f
Merge branch 'master' into cgroup_v2 2023-03-06 13:38:33 -08:00
Han Fei
420108a7a0 support alternatives 2023-03-06 19:10:36 +01:00
Mike Kot
07a9e486e6 memset for frame pointers' 2023-03-06 19:35:00 +03:00
Mike Kot
4bc443f3c7 initial solution 2023-03-06 19:35:00 +03:00
Antonio Andelic
12525f768c
Add default constructor for MultiReadResponse (#47254) 
* Add default constructor for MultiReadResponse
* Remove optional
* Fix style
 2023-03-06 14:18:01 +01:00
Antonio Andelic
9117c7491d Join threads if exception happened in constructor 2023-03-06 12:55:02 +00:00
Han Fei
b7eef62458
Merge pull request #45491 from azat/dist/async-send-refactoring 
[RFC] Rewrite distributed sends to avoid using filesystem as a queue, use in-memory queue instead
 2023-03-06 12:32:33 +01:00
Nikolay Degterinsky
7f15575500
Merge pull request #47208 from ClibMouse/z_ip_coding_fix 
Fix ip coding on s390x
 2023-03-04 18:18:14 +01:00
Alexander Tokmakov
082194a755
Merge pull request #47175 from ClickHouse/correct_fatal_handling 
More correct handling of fatal errors
 2023-03-04 00:38:00 +03:00
Sema Checherinda
85178c9609
Merge pull request #47102 from CheSema/merge-mutate-do-thread-group 
do flushUntrackedMemory when context switches
 2023-03-03 20:23:25 +01:00
Han Fei
c1e80683c4 Refine OptimizeRegularExpression Function 2023-03-03 17:59:21 +01:00
Suzy Wang
e44e528118 remove unnecessary reverse memcpy 2023-03-03 06:20:18 -08:00
Alexander Tokmakov
cadaf06829
Merge pull request #46681 from ClickHouse/fix_insert_cancellation_in_native_protocol 
Fix queries cancellation when a client dies
 2023-03-03 16:27:20 +03:00
Alexander Tokmakov
b1a288d271
Merge branch 'master' into correct_fatal_handling 2023-03-03 14:50:07 +03:00
Kseniia Sumarokova
386663953c
Merge branch 'master' into named-collections-finish 2023-03-03 12:23:38 +01:00
Alexander Tokmakov
619b282060 more correct handling of fatal errors 2023-03-03 01:22:04 +01:00
LiuNeng
d4c5ab9dcd
Optimize one nullable key aggregate performance (#45772) 2023-03-02 21:01:52 +01:00
Sema Checherinda
7a9f4aab8e do not call adjustWithUntrackedMemory when untracked_memory is 0 2023-03-02 19:21:03 +01:00
Sema Checherinda
a6c2473518 do flushUntrackedMemory when context swith 2023-03-02 19:21:03 +01:00
kssenii
0c4bb3c80f Merge remote-tracking branch 'upstream/master' into named-collections-granular-access 2023-03-02 19:08:29 +01:00
Alexander Tokmakov
c518f3690e Merge branch 'master' into fix_insert_cancellation_in_native_protocol 2023-03-02 18:24:08 +01:00
Yatsishin Ilya
d4304d2028 Merge remote-tracking branch 'origin/master' into clickhouse-help 2023-03-02 16:16:18 +00:00
Robert Schulze
740aeaba1f
Apply some CTAD 2023-03-02 13:36:47 +00:00
Alexey Milovidov
97c0f8a0e0
Merge pull request #47027 from amosbird/clang16 
Prepare to build with clang 16
 2023-03-02 02:10:03 +03:00
Robert Schulze
15fec805fb
Merge pull request #46950 from ClickHouse/rs/uninline-likePatternToRegexp 
Un-inline likePatternToRegexp()
 2023-03-01 21:38:40 +01:00
Alexander Tokmakov
a97e15e36f Merge branch 'master' into fix_insert_cancellation_in_native_protocol 2023-03-01 14:26:41 +01:00
Amos Bird
2d8060d843
Rewrite VersionNumber in ClickHouse way 2023-03-01 11:07:31 +08:00
Azat Khuzhin
572dd76eae Extend ConcurrentBoundedQueue to deque 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-28 22:33:36 +01:00
Kruglov Pavel
66634cfd7c
Merge pull request #46618 from kitaisreal/map-field-to-string-fix 
Map field to string fix
 2023-02-28 20:29:40 +01:00
Robert Schulze
2c0395b346
Merge pull request #46814 from ClickHouse/rs/search-refactoring 
Cleanup string search code
 2023-02-28 15:04:19 +01:00
Maksim Kita
0764649c5d
Merge pull request #46632 from ClickHouse/analyzer-limit-settings-2 
Analyzer - apply limit and offset settings - second attempt
 2023-02-27 21:39:34 +03:00
kssenii
ad88251ee7 Fix tests 2023-02-27 17:42:04 +01:00
Robert Schulze
35f4b42608
Move likePatternToRegexp from Functions/ to Common/ 2023-02-27 16:41:15 +00:00
Robert Schulze
5c8edcc2e7
Merge pull request #46940 from ClickHouse/rs/sizes-of-arrays-dont-match 
Typo: SIZES_OF_ARRAYS_DOESNT_MATCH --> SIZES_OF_ARRAYS_DONT_MATCH
 2023-02-27 14:33:09 +01:00
kssenii
f398c5d471 Fix style 2023-02-27 12:22:27 +01:00
Robert Schulze
22d09caea2
SIZES_OF_ARRAYS_DOESN_MATCH --> SIZES_OF_ARRAYS_DONT_MATCH 2023-02-27 08:31:40 +00:00
Maksim Kita
8ad016dbe6 Map field to string fix 2023-02-26 13:38:45 +01:00
Azat Khuzhin
03e90cd7cf Add new metrics to system.asynchronous_metrics 
The following metrics can be useful to calculate various rates (i.e.
disk/network IO rates):
- AsynchronousHeavyMetricsUpdateInterval
- AsynchronousMetricsUpdateInterval

The following had been added by analogy with the
AsynchronousMetricsCalculationTimeSpent:
- AsynchronousHeavyMetricsCalculationTimeSpent

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-25 21:13:15 +01:00
Alexey Milovidov
f28ab14759 Better handling of fatal errors 2023-02-24 23:29:06 +01:00
kssenii
c2bcc4119f Better 2023-02-24 20:57:37 +01:00
Yakov Olkhovskiy
da326dda03 comments fix 2023-02-24 19:19:51 +00:00
kssenii
f2a9109fe1 Merge remote-tracking branch 'upstream/master' into named-collections-granular-access 2023-02-24 15:20:46 +01:00
Robert Schulze
627961d9db
More cosmetics 2023-02-24 10:14:58 +00:00
Robert Schulze
421afeeae0
Shuffle class order (and just that) 2023-02-24 10:13:35 +00:00
Robert Schulze
ef529de7db
Cosmetics 2023-02-24 10:12:47 +00:00
Alexander Tokmakov
fe92fd8a61 Merge branch 'master' into fix_insert_cancellation_in_native_protocol 2023-02-23 22:35:31 +01:00
Robert Schulze
6aa4e93284
Merge branch 'master' into StringSearcher 2023-02-23 11:54:05 +01:00
Jiebin Sun
d6710d9b34 Align all the SSE4.1 requirement and use needle_size 
Align all the SSE4.1 requirement from StringSearcher. Use needle_size
in while loop to make the code clean.

Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>
 2023-02-22 16:15:26 -05:00
Yakov Olkhovskiy
99095446af review suggestions 2023-02-22 17:22:13 +00:00
Jiebin Sun
1f62135ba7 Make the optimized SIMD StringSearcher clean 
This patch has revised the name of value and added comments to make
the SIMD StringSearcher clean and easy to understand based on pull
request 46289.

Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>
 2023-02-22 12:18:21 -05:00
Jiebin Sun
d220e7f4fc Optimize the SIMD StringSearcher if needle_size is large 
This patch offers an additional optimization when the needle_size is
large. If the needle_size is larger than the haystack_size, there is
no need to search any more.

The optimized SIMD StringSearcher has led at most 41.7% than Volnitsky
algorithm when the needle_size is less than 21, and fallen behind only
about 1% even when the needle_size is bigger than 50, which is not
considered as a common case.

Test platform: ICX server
Test query: SELECT COUNT(*) FROM hits WHERE URL LIKE '%{Needle}%';

Needle_size	opt/baseline
5		141.7%
6		129.4%
8		118.5%
9		112.3%
10		107.4%
14		103.4%
20		100.2%
21		100.7%
51		99.0%

Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>
 2023-02-22 11:58:17 -05:00
Jiebin Sun
f5a6a86dec Optimize the SIMD StringSearcher by searching first two chars 
This patch offers the optimized SIMD StringSearcher by searching the first
and second chars together rather than only the first char, which will result
in big performance gain. The patch also provides a quick path when the needle
size is 1.

With this patch, I have tested the 43 queries in clickbench on ICX server.
Query 20 has got 35% performance gain. Other StringSearcher related queries
have got around 10% performance improvement. And the overall geomean of all
the queries has got 4.1% performance gain.

Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>
 2023-02-22 11:55:30 -05:00
Alexander Tokmakov
e660c0838c fix multipart requests 2023-02-22 17:54:35 +01:00
Yatsishin Ilya
80857638c1 Merge remote-tracking branch 'origin/master' into clickhouse-help 2023-02-22 13:58:17 +00:00
kssenii
f0ca3ef207 Merge remote-tracking branch 'upstream/master' into named-collections-granular-access 2023-02-22 14:02:29 +01:00
kssenii
f518ddd96d Merge remote-tracking branch 'upstream/master' into named-collections-finish 2023-02-22 13:43:18 +01:00
Kruglov Pavel
3ba3fdbfa3
Merge pull request #46607 from kssenii/delay-loading-of-named-collections 
Do not load named collections on server startup (on first access instead)
 2023-02-22 13:22:34 +01:00
Dmitry Novik
67469ad46b
Merge pull request #46622 from ClickHouse/async-insert-memory-fix 
Fix MemoryTracker counters for async inserts
 2023-02-22 12:27:05 +01:00
Sichen Zhao
5772b9769e fix types 2023-02-21 22:49:40 -08:00
Sichen Zhao
d0de9d5887 updates for comments, treat special case for string value max memory, and extract shared code out 2023-02-21 22:47:26 -08:00
Alexander Tokmakov
592af6d652 fix incomplete interst through http 2023-02-22 02:34:03 +01:00
Yakov Olkhovskiy
cf493d1dfb replace settings limit and offset with corresponding expression nodes 2023-02-21 22:36:02 +00:00
Dmitry Novik
ad1e5f3918 Review fixes 2023-02-21 22:06:17 +00:00
Vladimir C
a791250a4a
Merge pull request #38614 from bharatnc/ncb/profile-events-partlog 2023-02-21 11:06:44 +01:00
Kseniia Sumarokova
24d0490ef3
Merge branch 'master' into delay-loading-of-named-collections 2023-02-21 00:24:18 +01:00
kssenii
a54b011670 Finish for mysql 2023-02-20 21:37:38 +01:00
Dmitry Novik
3c4c527bce Fix MemoryTracker counters for async inserts 2023-02-20 18:40:57 +00:00
kssenii
9d16205c8a Load named collections on first access 2023-02-20 16:33:45 +01:00
Alexey Milovidov
5ef159fa10
Merge pull request #46565 from ClickHouse/remove-pvs-studio 
Remove PVS-Studio
 2023-02-20 17:10:53 +03:00
Robert Schulze
135170b27d
Merge pull request #44973 from ClickHouse/remove-smp-limitation 
Slightly modify the detection of the number of CPU cores
 2023-02-20 11:22:39 +01:00
Alexey Milovidov
d8cda3dbb8 Remove PVS-Studio 2023-02-19 23:30:05 +01:00
Alexey Milovidov
a22f0eee2a
Merge branch 'master' into Aggregate_Sum_AVX512 2023-02-19 04:03:45 +03:00
Dmitry Novik
728434821a Merge remote-tracking branch 'origin/master' into background-memory-tracker 2023-02-16 12:35:07 +00:00
Alexey Milovidov
e8d1a2f099
Update AsynchronousMetrics.cpp 2023-02-15 23:52:53 +03:00
Dmitry Novik
e7a94a28ee
Merge branch 'master' into background-memory-tracker 2023-02-15 19:39:28 +01:00
kssenii
0c53f33e6f Make code more common: add GlobalWithParameter level 2023-02-15 16:59:43 +01:00
Nikolay Degterinsky
494fd472f0
Merge branch 'master' into ulid 2023-02-15 07:24:32 +01:00
vdimir
1b715a3388
Allow scope override in ThreadStatus::attachProfileCountersScope 2023-02-14 14:11:13 +00:00
vdimir
4c1aeb7972
Use overload of PartLog::addNewPart with profile_events 2023-02-14 14:11:12 +00:00
vdimir
fa08f72bba
Move subthread_profile_events to ThreadStatus 2023-02-14 14:11:11 +00:00
vdimir
994dcb93f9
Rename ScopedProfileEvents -> ProfileEventsScope 2023-02-14 14:11:10 +00:00
vdimir
a228f7f419
[wip] ProfileCounters for each part 2023-02-14 14:11:09 +00:00
Alexey Milovidov
f15623141e
Fix error 2023-02-14 14:11:08 +00:00
Alexey Milovidov
f554ff9d99
Remove trash 2023-02-14 14:11:08 +00:00
John Skopis
f7604cc686
[feat] Add ProfileEvents map to PartLog 
closes #10316
 2023-02-14 14:11:02 +00:00
Nikolay Degterinsky
09da5575e3 Merge remote-tracking branch 'upstream/master' into ulid 2023-02-14 13:41:17 +00:00
robot-clickhouse-ci-2
ea5b645af5
Merge pull request #46349 from ClickHouse/fix-crash-int-overflow-hash-table-deserialization-agg-state 
Check overflow in HashTable::alloc
 2023-02-14 10:02:13 +01:00
Alexander Gololobov
3f5887248e
Update HashTable.h 2023-02-13 17:30:57 +01:00
Nikolai Kochetov
ea19a0d46e
Update src/Common/HashTable/HashTable.h 
Co-authored-by: Alexander Gololobov <440544+davenger@users.noreply.github.com>
 2023-02-13 17:15:41 +01:00
Nikolai Kochetov
5d77407f39 Check overflow in HashTable::alloc 2023-02-13 14:50:23 +00:00
Suzy Wang
e140bed3ea
Merge branch 'master' into z-build-0120 2023-02-13 09:06:48 -05:00
Alexey Milovidov
2e57fab8e3 Instrumentation of callbacks for distributed queries 2023-02-12 10:04:05 +01:00
kssenii
3067c1d723 Merge remote-tracking branch 'upstream/master' into resubmit-prefetches 2023-02-11 11:36:23 +01:00
Kseniia Sumarokova
3be36c7979
Merge pull request #41976 from kssenii/allow-single-disk-instead-of-storage-policy 
Allow configuring storage as `SETTINGS disk='<disk_name>'` (instead of `storage_policy`) and explicit disk creation `SETTINGS disk=disk(<disk_configuration>)`
 2023-02-10 21:38:50 +01:00
Alexey Milovidov
d5d87bc6b7
Merge pull request #46152 from ClickHouse/with-fill-bigint 
More convenient usage of big integers and ORDER BY WITH FILL.
 2023-02-10 20:05:32 +03:00
Vladimir C
bd39e29cab
Merge pull request #46065 from aiven-sal/aiven-sal/siphash128 2023-02-10 10:41:45 +01:00
Suzy Wang
64313d7020
Merge branch 'master' into z-build-0120 2023-02-09 14:49:22 -05:00
Vitaly Baranov
39bacfb9dc
Merge branch 'master' into allow-single-disk-instead-of-storage-policy 2023-02-09 13:16:55 +01:00
Alexey Milovidov
d9fbf643bc
Merge pull request #45226 from hanfei1991/hanfei/gwp-asan 
Introduce llvm/gwp-asan
 2023-02-09 09:24:52 +03:00
pufit
7cf71d1f82
Merge pull request #46144 from ClickHouse/46131-fix-config-attributes-merge 
Fix config merge for XML attributes.
 2023-02-08 23:28:13 -05:00
Alexey Milovidov
292a5abbc9 Merge remote-tracking branch 'origin/master' into with-fill-bigint 2023-02-09 01:47:51 +01:00
Alexander Tokmakov
4fd4058259
Merge pull request #45629 from arthurpassos/fix_cares_crash 
Cancel c-ares failed requests and retry on system interrupts to prevent callbacks with dangling references and premature resolution failures
 2023-02-09 02:29:29 +03:00
Alexander Tokmakov
8101b044fa
Merge pull request #46091 from azat/sanity-assertions 
Sanity assertions for closing file descriptors
 2023-02-09 01:02:03 +03:00
Han Fei
64b84f8e31 fix style 2023-02-08 22:16:28 +01:00
kssenii
b0b865c32e Resubmit prefetches 2023-02-08 21:26:24 +01:00
Han Fei
8f665b5bbf fix 2023-02-08 18:28:06 +01:00
Han Fei
a4efb1fbda fix compile 2023-02-08 18:06:36 +01:00
Han Fei
d9aab84701 fix fork hang 2023-02-08 16:20:42 +01:00
pufit
e833decd88 try to fix memory leak 2023-02-08 15:17:46 +00:00
Antonio Andelic
45d4ba59dc
Merge pull request #46068 from bigo-sg/alloc_map_metrics 
Add new metric to record allocations times and bytes using mmap
 2023-02-08 15:49:20 +01:00
Alexey Milovidov
4f2a58ddbd Fix style 2023-02-08 08:44:31 +01:00
Alexey Milovidov
0791b85f57 Remove bits of trash 2023-02-08 08:18:31 +01:00
Alexey Milovidov
d66beb9ac4 Allow accurate comparison of Big Int with other integers 2023-02-08 07:56:00 +01:00
pufit
c066c0dfdb Fix config merge for an attributes 2023-02-07 23:46:13 +00:00
Dmitry Novik
d43d17f59a Address the review 2023-02-07 16:05:02 +00:00
Yatsishin Ilya
1baa15d603 Merge remote-tracking branch 'origin/master' into clickhouse-help 2023-02-07 15:46:59 +00:00
Han Fei
0f7defb87f
Merge branch 'master' into hanfei/gwp-asan 2023-02-07 14:55:55 +01:00
Robert Schulze
690d873126
Merge pull request #46088 from ClickHouse/rs/fix-liburing-build 
Fix build with `-DENABLE_LIBURING=0` (or `-DENABLE_LIBRARIES=0`)
 2023-02-07 12:16:14 +01:00
Azat Khuzhin
8cc41b7f41 Check return value of ::close() 
Note, that according close(2), EINTR should not be retriable for close()

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-07 11:28:22 +01:00
Salvatore Mesoraca
001b714d3e
Common: SipHash: add original 128-bit output implementation 2023-02-07 11:19:04 +01:00
Azat Khuzhin
e83163e25f Add sanity check of evtimer_del() in QueryProfiler 
Saw this one time on CI with EINVAL [1], while this was EINVAL this
time, next time it can be completelly different fd and break something
in a different place.

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/45654/1716af465d376f5335720b5045bcccdd9e1823aa/fast_test.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-06 20:13:05 +01:00
Dmitry Novik
1e065b32f3 Add MemoryTracker for the background tasks 2023-02-06 17:25:58 +00:00
Robert Schulze
8840aa9242
Fix build with -DENABLE_LIBURING=0 (or -DENABLE_LIBRARIES=0) 
Follow-up to #38456

The problem was that code which uses liburing was conditionally build
depending on define OS_LINUX and not based on whether liburing was
enabled/disabled at build time.
 2023-02-06 16:57:42 +00:00
Robert Schulze
4c89327c9f
Add TODO comment to MultiVersion.h 
Follow-up to #46057
 2023-02-06 11:04:58 +00:00
taiyang-li
7b10ac8d5b add new metric to record allocations times and bytes using mmap 2023-02-06 17:50:45 +08:00
Konstantin Morozov
ff05162904 change mutex to lock-free 2023-02-05 19:25:09 +03:00
kssenii
5abbc7cad6 Cleanup 2023-02-04 19:41:41 +01:00
kssenii
cce23cee51 Fix build 2023-02-04 13:49:32 +01:00
Yatsishin Ilya
f4cfd8a2d9 Merge remote-tracking branch 'origin' into clickhouse-help 2023-02-03 20:08:23 +00:00
kssenii
536c02ca8a Adding custom field continued 2023-02-03 21:05:18 +01:00
kssenii
5f124263a6 Revert changes with complex SettingsChanges 2023-02-03 21:04:45 +01:00
Sichen Zhao
aa5230b525 updates path name 2023-02-03 11:18:53 -08:00
Alexander Tokmakov
3f11948bb0
Merge branch 'master' into stack_trace_in_part_log 2023-02-03 20:05:00 +03:00
kssenii
12b62de374 Add field CustomType 2023-02-03 17:07:02 +01:00
Robert Schulze
85cbb9288c
Merge pull request #45456 from FrankChen021/uncaught_exception 
Fix uncaught exception in HTTPHandler
 2023-02-03 15:26:02 +01:00
Anton Popov
cdbe145bc1
Merge pull request #45796 from CurtizJ/fix-leak-in-azure-sdk 
Fix test `test_azure_blob_storage_zero_copy_replication ` (memory leak in azure sdk)
 2023-02-03 14:16:19 +01:00
Frank Chen
d38adfab30 Merge two overridden functions as one 2023-02-03 15:27:45 +08:00
Sichen Zhao
d4ca058643 updates to support cgroupv2 as well 2023-02-02 22:19:29 -08:00
Frank Chen
d3a05a11da Merge remote-tracking branch 'remotes/github/master' into stack_trace_in_part_log 2023-02-03 11:39:43 +08:00
Frank Chen
7ad4b3176a Clean code 2023-02-03 11:13:45 +08:00
Frank Chen
93ead69b7f Resolve review comments 2023-02-03 11:05:54 +08:00
kssenii
a391de18a1 Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2023-02-02 20:49:41 +01:00
Nikolai Kochetov
6cd0b51127
Merge pull request #45871 from ClickHouse/fix-ipv6-parser 
Fix ipv6 parser
 2023-02-01 14:28:54 +01:00
Robert Schulze
b512316586
Merge pull request #45682 from ClickHouse/rename-qrc-to-qc 
Rename "Query Result Cache" to "Query Cache"
 2023-02-01 11:23:29 +01:00
Alexey Milovidov
bf9f62dcbd
Merge pull request #38456 from sauliusvl/iouring 
Re-introduce io_uring read method
 2023-02-01 06:41:32 +03:00
Yakov Olkhovskiy
3d64c84571 fix ipv6 parser 2023-02-01 01:02:58 +00:00
Yatsishin Ilya
98edb9f06b Update help for clickhouse tools and add test 2023-01-31 12:19:37 +00:00
Robert Schulze
325c6bdf3d
Renaming: "Query Result Cache" --> "Query Cache" 
Reasons:

- The cache will at some point store intermediate results as opposed to
  only query results. We should change the terminology now without
  having to worry about backward compat.

- Equivalent caches in MySQL (1) and Starrocks (2) are called "query
  cache".

- The new name is ca. 13.8% more catchy.

(1) https://dev.mysql.com/doc/refman/5.6/en/query-cache.html
(2) https://docs.starrocks.io/en-us/2.5/using_starrocks/query_cache
 2023-01-31 09:54:34 +00:00
Alexey Milovidov
7aa10a430a
Merge branch 'master' into hanfei/gwp-asan 2023-01-31 04:13:59 +03:00
Anton Popov
31e8b692f4 fix typo 2023-01-31 01:13:23 +00:00
Anton Popov
839cd614fb fix memory leak in azure sdk 2023-01-31 01:01:10 +00:00
Vitaly Baranov
2e3a3cc4dc
Merge pull request #45701 from vitlibar/add-setting-allow-head-object-request 
Add new S3 setting allow_head_object_request
 2023-01-30 17:41:09 +01:00
Han Fei
4971358de5 fix init 2023-01-30 17:07:01 +01:00
Arthur Passos
7b1ceaa2c5
Merge branch 'master' into fix_cares_crash 2023-01-30 12:05:56 -03:00
Vladimir C
e7e8ae979f
Merge pull request #45271 from jh0x/feature-array-shuffle 2023-01-30 11:54:07 +01:00
Robert Schulze
66392b873c
Merge pull request #45513 from aiven-sal/aiven-sal/siphash_pr 
Keyed SipHash
 2023-01-30 11:10:00 +01:00
Alexey Milovidov
a7299746c7
Merge pull request #45743 from ClickHouse/oom-message 
Improve MEMERY_LIMIT_EXCEEDED exception message
 2023-01-30 00:46:04 +03:00
Robert Schulze
33a719b9be
Update getNumberOfPhysicalCPUCores.cpp 2023-01-29 21:37:19 +01:00
Robert Schulze
15ae2d1de5
Merge branch 'master' into aiven-sal/siphash_pr 2023-01-29 21:06:52 +01:00
Robert Schulze
1bf280cb50
Handle obscure case that /proc/cpuinfo is unmounted 2023-01-29 17:25:08 +00:00
alesapin
631c8fb155
Merge branch 'master' into add-setting-allow-head-object-request 2023-01-29 14:58:27 +01:00
Dmitry Novik
ec1f6bfd37 Improve MEMERY_LIMIT_EXCEEDED exception message 2023-01-29 01:52:37 +00:00
Saulius Valatka
8505b8c57a add profile events for failed/succeeded CQE completions 2023-01-28 21:54:44 +02:00
Saulius Valatka
8fa9a99ba1 implement pending request queue to prevent CQ overflows 2023-01-28 21:54:44 +02:00
Saulius Valatka
ac2c921bdf add initial io_uring support 2023-01-28 21:54:44 +02:00
Azat Khuzhin
6b42b66257 Simply filesystem helpers to check is-readable/writable/executable 
Use one system call - faccessat(), over multiple system calls.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-01-27 21:11:10 +01:00
Joanna Hulboj
9a559b5475 FIXUP: More comments about shuffle 2023-01-27 20:07:56 +00:00
Joanna Hulboj
31eb936457 Added Fisher-Yates shuffle and partial-shuffle 2023-01-27 20:07:55 +00:00
Arthur Passos
39cbf4d4dc make random number generator thread safe 2023-01-27 12:14:07 -03:00
Arthur Passos
e559a6b1d2 address some comments 2023-01-27 12:07:00 -03:00
Vitaly Baranov
d02b255b55 Add new setting "allow_head_object_request" to S3::RequestSettings. 2023-01-27 15:10:34 +01:00
Alexander Tokmakov
c366806c3e
Merge pull request #45527 from ClickHouse/exception_message_patterns4 
Better formatting for exception messages 2
 2023-01-27 15:31:52 +03:00
Salvatore Mesoraca
2e72e27206
common: siphash: add support for custom keys 2023-01-27 13:00:53 +01:00
Alexey Milovidov
5b257ab806
Merge pull request #45233 from ClickHouse/improve_week_day 
Revert "Revert "Improve week day""
 2023-01-27 02:44:17 +03:00
Arthur Passos
ea38df2c28 add comments to the code 2023-01-26 13:54:12 -03:00
Alexander Tokmakov
a584ad0eb1 forbid runtime strings 2023-01-26 10:52:47 +01:00
Alexander Tokmakov
9b670946db Merge branch 'master' into exception_message_patterns5 2023-01-26 00:41:32 +01:00
Arthur Passos
4a10f49ca8 spacing 2023-01-25 19:21:45 -03:00
Arthur Passos
912ac6fb05 remove maybe_unused 2023-01-25 19:09:35 -03:00
Arthur Passos
9942fe7b5b cancel ares failed requests, listen to POLLRDNORM and retry on EINTR 2023-01-25 19:07:58 -03:00
Alexander Tokmakov
3744fa2c63 format more messages 2023-01-25 21:16:42 +01:00
sichenzhao
243ac52259
Added two metrics about memory usage in cgroup to asynchronous metrics (#45301) 2023-01-25 20:32:17 +01:00
Suzy Wang
d2d2e3f31e replace tab with spaces 2023-01-25 07:49:49 -08:00
Alexander Tokmakov
ae795d87b2 fix 2023-01-25 16:06:40 +01:00
Suzy Wang
fe47b43b6a replace tab with space 2023-01-25 05:59:10 -08:00
Alexander Tokmakov
6eb557b2ba Merge branch 'master' into exception_message_patterns4 2023-01-25 13:49:17 +01:00
Robert Schulze
59528cfca0
Merge pull request #45460 from ClickHouse/inv-index-cleanup 
Cleanup of inverted index
 2023-01-25 13:23:38 +01:00
Sergei Trifonov
0d1ea05ff6
Merge pull request #45007 from ClickHouse/cancellable-mutex-integration 
Fast shared mutex integration
 2023-01-25 11:15:46 +01:00
Alexander Tokmakov
d1baa7300c reformat ParsingException 2023-01-24 23:21:29 +01:00
Suzy Wang
3abb00b2e1 remove trailing whitespace 2023-01-24 07:59:56 -08:00
serxa
51da43d6cf fix try_shared_lock() in SharedMutex and CancelableSharedMutex 2023-01-24 14:36:07 +00:00
Robert Schulze
094e42d1dc
Fix style 2023-01-24 12:57:56 +00:00
Robert Schulze
9ff2bfcbf5
Merge remote-tracking branch 'origin/master' into inv-index-cleanup 
src/Interpreters/GinFilter.cpp
	src/Interpreters/InterpreterCreateQuery.cpp
	src/Storages/MergeTree/MergeTreeData.cpp
	src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp
	src/Storages/MergeTree/MergeTreeIndexInverted.cpp
 2023-01-24 10:09:42 +00:00
Robert Schulze
36d6af0e04
Yet another build fix 2023-01-24 09:57:12 +00:00
Suzy Wang
1176260b3c
Merge branch 'master' into z-build-0120 2023-01-24 01:14:42 -05:00
Alexander Tokmakov
c6910f39b9 fix 2023-01-24 01:11:58 +01:00
Alexander Tokmakov
414693feb2 fixes 2023-01-24 00:46:03 +01:00
Alexander Tokmakov
bb4c8e169f check number of parameters in format string 2023-01-23 23:16:16 +01:00
Alexander Tokmakov
3f6594f4c6 forbid old ctor of Exception 2023-01-23 22:18:05 +01:00
Alexander Tokmakov
70d1adfe4b
Better formatting for exception messages (#45449) 
* save format string for NetException

* format exceptions

* format exceptions 2

* format exceptions 3

* format exceptions 4

* format exceptions 5

* format exceptions 6

* fix

* format exceptions 7

* format exceptions 8

* Update MergeTreeIndexGin.cpp

* Update AggregateFunctionMap.cpp

* Update AggregateFunctionMap.cpp

* fix
 2023-01-24 00:13:58 +03:00
Alexey Milovidov
62a8de34cc
Merge pull request #44811 from azat/build/glibc2.36-fix 
Fix ASan builds for glibc 2.36+
 2023-01-23 23:57:20 +03:00
Robert Schulze
1a1fa4a40a
Fix ARM build 2023-01-23 19:48:09 +00:00
Robert Schulze
0ab7ae6c17
Merge pull request #43797 from ClickHouse/query-result-cache 
Query result cache [experimental]
 2023-01-23 19:54:35 +01:00
Kseniia Sumarokova
8ec39ab6f3
Merge pull request #45442 from kssenii/minor-changes 
Minor improvements around reading from remote
 2023-01-23 15:46:30 +01:00
Suzy Wang
fb46b5ed4d s390x support 2023-01-23 14:20:49 +00:00
Sergei Trifonov
0fbfa17863
Merge branch 'master' into cancellable-mutex-integration 2023-01-23 12:44:09 +01:00
Kseniia Sumarokova
c3d545b64a
Merge pull request #45489 from azat/build/queue-fwd-decl 
Forward declaration of ConcurrentBoundedQueue in ThreadStatus
 2023-01-22 20:23:05 +01:00
Robert Schulze
d07b7ce1ee
Some fixups 2023-01-22 17:25:59 +00:00
Robert Schulze
d7416f45ff
Merge remote-tracking branch 'origin/master' into remove-smp-limitation 2023-01-22 16:22:54 +00:00
Robert Schulze
340f406553
Merge branch 'master' into query-result-cache 2023-01-22 13:21:36 +01:00
Robert Schulze
531e64dc39
Merge branch 'master' into inv-index-cleanup 2023-01-22 13:20:38 +01:00
Alexander Gololobov
91fb1eab8c
Merge pull request #45373 from ClickHouse/throttler-metrics 
Add detailed profile events for throttling
 2023-01-22 11:20:45 +01:00
Robert Schulze
97d1bed114
Merge branch 'master' into improve_week_day 2023-01-21 20:40:33 +01:00
Azat Khuzhin
2a8f116c18 Forward declaration of ConcurrentBoundedQueue in ThreadStatus 
ThreadStatus is the header that recomplies almost all ClickHouse
modules.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-01-21 16:02:09 +01:00
Suzy Wang
19d26828a0 s390x build support 2023-01-20 21:16:55 +00:00
Alexander Tokmakov
910d6dc0ce
Merge pull request #45342 from ClickHouse/exception_message_patterns 
Save message format strings for DB::Exception
 2023-01-20 18:46:52 +03:00
Azat Khuzhin
bdeb5514c5 Fix ASan builds for glibc 2.36+ (use RTLD_NEXT for ThreadFuzzer interceptors) 
Recently I noticed that clickhouse compiled with ASan does not work with
newer glibc 2.36+, before I though that this was only about compiling
with old but using new, however that was not correct, ASan simply does
not work with glibc 2.36+.

Here is a simple reproducer [1]:

    $ cat > test-asan.cpp <<EOL
    #include <pthread.h>
    int main()
    {
        // something broken in ASan in interceptor for __pthread_mutex_lock
        // and only since glibc 2.36, and for pthread_mutex_lock everything is OK
        pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
        return __pthread_mutex_lock(&mutex);
    }
    EOL
    $ clang -g3 -o test-asan test-asan.cpp -fsanitize=address
    $ ./test-asan
    AddressSanitizer:DEADLYSIGNAL
    =================================================================
    ==15659==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x000000000000 bp 0x7fffffffccb0 sp 0x7fffffffcb98 T0)
    ==15659==Hint: pc points to the zero page.
    ==15659==The signal is caused by a READ memory access.
    ==15659==Hint: address points to the zero page.
        #0 0x0  (<unknown module>)
        #1 0x7ffff7cda28f  (/usr/lib/libc.so.6+0x2328f) (BuildId: 1e94beb079e278ac4f2c8bce1f53091548ea1584)

    AddressSanitizer can not provide additional info.
    SUMMARY: AddressSanitizer: SEGV (<unknown module>)
    ==15659==ABORTING

  [1]: https://gist.github.com/azat/af073e57a248e04488b21068643f079e

I've started observing glibc code, there was some changes in glibc, that
moves pthread functions out from libpthread.so.0 into libc.so.6
(somewhere between 2.31 and 2.35), but
the problem pops up only with 2.36, 2.35 works fine.

After this I've looked into changes between 2.35 and 2.36, and found
this patch [2] - "dlsym: Make RTLD_NEXT prefer default version
definition [BZ #14932]", that fixes this bug [3].

  [2]: https://sourceware.org/git/?p=glibc.git;a=commit;h=efa7936e4c91b1c260d03614bb26858fbb8a0204
  [3]: https://sourceware.org/bugzilla/show_bug.cgi?id=14932

The problem with using DL_LOOKUP_RETURN_NEWEST flag for RTLD_NEXT is
that it does not resolve hidden symbols (and __pthread_mutex_lock is
indeed hidden).

Here is a sample that will show the difference [4]:

    $ cat > test-dlsym.c <<EOL
    #define _GNU_SOURCE
    #include <dlfcn.h>
    #include <stdio.h>

    int main()
    {
        void *p = dlsym(RTLD_NEXT, "__pthread_mutex_lock");
        printf("__pthread_mutex_lock: %p (via RTLD_NEXT)\n", p);
        return 0;
    }
    EOL

    # glibc 2.35: __pthread_mutex_lock: 0x7ffff7e27f70 (via RTLD_NEXT)
    # glibc 2.36: __pthread_mutex_lock: (nil) (via RTLD_NEXT)

  [4]: https://gist.github.com/azat/3b5f2ae6011bef2ae86392cea7789eb7

But ThreadFuzzer uses internal symbols to wrap
pthread_mutex_lock/pthread_mutex_unlock, which are intercepted by ASan
and this leads to NULL dereference.

The fix was obvious - just use dlsym(RTLD_NEXT), however on older
glibc's this leads to endless recursion (see commits in the code). But
only for jemalloc [5], and even though sanitizers does not uses jemalloc
the code of ThreadFuzzer is generic and I don't want to guard it with
more preprocessors macros.

  [5]: https://gist.github.com/azat/588d9c72c1e70fc13ebe113197883aa2

So we have to use RTLD_NEXT only for ASan.

There is also one more interesting issue, if you will compile with clang
that itself had been compiled with newer libc (i.e. 2.36), you will get
the following error:

    $ podman run --privileged -v $PWD/.cmake-asan/programs:/root/bin -e PATH=/bin:/root/bin -e --rm -it ubuntu-dev-v3 clickhouse
    ==1==ERROR: AddressSanitizer failed to allocate 0x0 (0) bytes of SetAlternateSignalStack (error code: 22)
    ...
    ==1==End of process memory map.
    AddressSanitizer: CHECK failed: sanitizer_common.cpp:53 "((0 && "unable to mmap")) != (0)" (0x0, 0x0) (tid=1)
        <empty stack>

The problem is that since GLIBC_2.31, `SIGSTKSZ` is a call to
`getconf(_SC_MINSIGSTKSZ)`, but older glibc does not have it, so `-1`
will be returned and used as `SIGSTKSZ` instead.

The workaround to disable alternative stack:

    $ podman run --privileged -v $PWD/.cmake-asan/programs:/root/bin -e PATH=/bin:/root/bin -e ASAN_OPTIONS=use_sigaltstack=0 --rm -it ubuntu-dev-v3 clickhouse client --version
    ClickHouse client version 22.13.1.1.

Fixes: #43426
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-01-20 13:09:13 +01:00
Robert Schulze
f8b446f517
Move method implementations (no other changes) 2023-01-20 10:57:16 +00:00
Robert Schulze
0653f86de9
Various cosmetic cleanups 2023-01-20 10:45:35 +00:00
Han Fei
8b29f8406b
Merge branch 'master' into hanfei/gwp-asan 2023-01-20 11:03:12 +01:00
Frank Chen
f867c4c206 Add an overriden fromCurrentException to simply the code 2023-01-20 16:43:10 +08:00
Frank Chen
be8ceeb4bb Save stack trace in part_log 2023-01-20 16:22:40 +08:00
Frank Chen
75191fd868 Catch exception raised by function object 2023-01-20 15:10:05 +08:00
Frank Chen
2fb49503da Make addAttribute exception safe 2023-01-20 15:09:46 +08:00
kssenii
4ce8950712 Minor changes 2023-01-19 17:53:10 +01:00
larryluogit
52ae33dba7
Merge branch 'master' into ftsearch 2023-01-19 11:34:11 -05:00
Sergei Trifonov
c443c1ece0
Merge branch 'master' into hanfei/async-cache 2023-01-18 00:19:49 +01:00
Robert Schulze
4f90824347
Merge remote-tracking branch 'origin/master' into query-result-cache 2023-01-17 22:49:53 +00:00
Alexander Tokmakov
5cd90c1a3e Merge branch 'master' into exception_message_patterns 2023-01-17 20:04:04 +01:00
Alexander Tokmakov
72e8615bec formatting of some exception messages 2023-01-17 20:03:56 +01:00
serxa
ce7e22b87b add detailed profile events for throttling 2023-01-17 18:29:24 +00:00
Alexander Tokmakov
8b13b85ea0
Merge pull request #44543 from ClickHouse/text_log_add_pattern 
Add a column with a message pattern to text_log
 2023-01-17 20:19:32 +03:00
Han Fei
8a74238fe0 improve 2023-01-17 15:47:52 +01:00
HarryLeeIBM
e7add8218f Addressed more review comments and ClangTidy errors 2023-01-17 06:29:13 -08:00
Vitaly Baranov
b13498d9ba
Merge pull request #45288 from vitlibar/fix-s3-requests-without-region 
Fix s3 requests without region
 2023-01-17 09:24:59 +01:00
Alexander Tokmakov
522686f78b less empty patterns 2023-01-17 01:19:44 +01:00
Alexander Tokmakov
870cfcc36a less fmt::runtime usages 2023-01-17 00:11:59 +01:00
Alexander Tokmakov
e7899825e6 save format strings for DB::Exceptions 2023-01-16 23:20:33 +01:00
Nikolay Degterinsky
d414d4aa7a Merge remote-tracking branch 'upstream/master' into bcrypt 2023-01-16 22:14:03 +00:00
Kseniia Sumarokova
d859976fbd
Merge pull request #45250 from ClickHouse/43188_Record_startup_time_in_profileevents 
Record server startup time in ProfileEvents
 2023-01-16 12:20:37 +01:00
Alexander Tokmakov
9ad6e1b129
Update logger_useful.h 2023-01-16 14:09:55 +03:00
Vitaly Baranov
e435edb4ab Make checkObjectExists() easier. 2023-01-16 10:06:20 +01:00
Robert Schulze
099e30ef2a
Merge remote-tracking branch 'origin/master' into query-result-cache 2023-01-16 08:04:49 +00:00
Vitaly Baranov
21b8aaeb8b Stop using HeadObject requests in S3 
because they don't work well with endpoints without explicit region.
 2023-01-15 20:28:11 +01:00
Robert Schulze
bd41c74ddf
Various test, code and docs fixups 2023-01-15 13:47:34 +00:00
Alexander Tokmakov
d857d62a03 remove another set of macros 2023-01-13 20:34:31 +01:00
Alexander Tokmakov
2d7773fccc Merge branch 'master' into text_log_add_pattern 2023-01-13 20:33:46 +01:00
Han Fei
7feb9b7f6f Merge branch 'master' into hanfei/gwp-asan 2023-01-13 20:26:34 +01:00
Han Fei
214830c569 enable gwp_asan by option 2023-01-13 15:22:14 +01:00
Robert Schulze
4ea836b87e
Revert "Revert "update function DAYOFWEEK and add new function WEEKDAY for mysql/spark compatiability"" 
This reverts commit e37f572c34.
 2023-01-13 14:00:16 +00:00
Smita Kulkarni
a0fe26f506 Addressed review comments and updated name to ServerStartupMilliseconds - Record server startup time in ProfileEvents 2023-01-13 14:38:54 +01:00
Smita Kulkarni
cf5cb0da97 Record server startup time in ProfileEvents 
Implementation:
* Added ProfileEvents::ServerStartupTime.
* Recorded time from start of main till listening to sockets.
Testing:
* Added a test 02532_profileevents_server_startup_time.sql
 2023-01-13 13:47:54 +01:00
Robert Schulze
9779d034eb
Merge pull request #45144 from ClibMouse/crc-power-fix 
Changes to support the CRC32 in PowerPC.
 2023-01-13 11:24:18 +01:00
serxa
693489a8ad review fixes 2023-01-12 15:51:04 +00:00
Sergei Trifonov
12d8543578
Merge branch 'master' into cancellable-mutex-integration 2023-01-12 16:03:49 +01:00
Han Fei
f5e736e240 introduce llvm/gwp-asan 2023-01-12 15:00:26 +01:00
serxa
0841809a9c fix special builds 2023-01-12 01:30:42 +00:00
MeenaRenganathan22
28d5c3cf7f Addressed the review comments 2023-01-11 17:00:10 -08:00
HarryLeeIBM
6d576cf157 Added more documentation 2023-01-11 13:40:20 -08:00
MeenaRenganathan22
5fa5ae8099 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into crc-power-fix 2023-01-11 09:31:41 -08:00
Robert Schulze
9bb1e31369
Merge pull request #44860 from bigo-sg/improve_week_day 
update function DAYOFWEEK and add new function WEEKDAY for mysql/spark compatiability
 2023-01-11 13:58:45 +01:00
serxa
c0f529600d fix darwin build 2023-01-11 12:23:33 +00:00
MeenaRenganathan22
25c94dfa83 Changes to support the CRC32 in PowerPC to address the WeakHash collision issue. Update the reference to support the hash values based on the specific platform 2023-01-10 21:20:13 -08:00
serxa
da4e9c9430 fix SharedMutex build 2023-01-10 16:35:46 +00:00
HarryLeeIBM
bc48b17817 Addressed more review comments. 2023-01-10 08:26:27 -08:00
Robert Schulze
0c3b034887
Merge remote-tracking branch 'origin/master' into query-result-cache 2023-01-10 09:02:41 +00:00
serxa
6f31c5094d Merge branch 'cancellable-mutex' of github.com:ClickHouse/ClickHouse into cancellable-mutex 2023-01-10 01:27:11 +00:00
serxa
0f80ad6e06 make style-check happy 2023-01-10 01:26:59 +00:00
Sergei Trifonov
b4961a6253
Merge branch 'master' into cancellable-mutex 2023-01-10 02:20:42 +01:00
Robert Schulze
ad4884b394
Fix review comments, pt. I 2023-01-09 21:42:51 +00:00
serxa
ae56ac1b56 add TSA support 2023-01-09 16:46:07 +00:00
serxa
534db794c1 more review fixes 2023-01-09 15:05:41 +00:00
serxa
88c3c2946b review fixes + split into files 2023-01-09 14:58:44 +00:00
taiyang-li
a8da7b4c20 change as request 2023-01-09 20:39:46 +08:00
taiyang-li
4bc2c61406 change as request 2023-01-09 20:37:16 +08:00
Kseniia Sumarokova
b22b51213f
Merge pull request #44868 from kssenii/improve-async-buffer-from-remote 
Small perf improvement for async buffer from remote
 2023-01-09 13:31:15 +01:00
taiyang-li
20c7c0b1ef change as request 2023-01-09 18:21:31 +08:00
Sergei Trifonov
a4470dd1b2
Update src/Common/Threading.cpp 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2023-01-09 09:01:44 +01:00
Sergei Trifonov
81d2ea30ba
Merge branch 'master' into cancellable-mutex-integration 2023-01-07 19:37:46 +01:00
Sergei Trifonov
a3b81778cd
Merge branch 'master' into cancellable-mutex 2023-01-07 19:34:19 +01:00
Anton Popov
1f32ffedf8
Merge pull request #43221 from ClickHouse/refactoring-ip-types 
Replace domain IP types (IPv4, IPv6) with native
 2023-01-07 12:01:21 +01:00
serxa
9c51ed6865 fix tests 2023-01-07 00:32:49 +00:00
serxa
705c8f01af fix tests 2023-01-07 00:31:53 +00:00
serxa
15bb127b01 replace every std::shared_mutex with DB::FastSharedMutex 2023-01-06 23:35:38 +00:00
serxa
a6958fff45 git-apply PR #44924 up to commit c65573d 2023-01-06 23:04:10 +00:00
Sergei Trifonov
c65573dd7e
Merge branch 'master' into cancellable-mutex 2023-01-06 23:52:35 +01:00
serxa
00b2c96ce8 fix special builds 2023-01-06 22:48:19 +00:00
Robert Schulze
45dbcf88e5
Merge remote-tracking branch 'origin/master' into query-result-cache 2023-01-06 20:42:48 +00:00
Robert Schulze
a733303184
Disable HyperThreading on big machines 2023-01-06 19:59:54 +00:00
kssenii
81070907a2 Better time watch 2023-01-06 18:44:48 +01:00
kssenii
2a030c1dc0 Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy 2023-01-06 15:58:27 +01:00
Kseniia Sumarokova
573d3283b0
Merge pull request #44327 from kssenii/use-new-named-collections-code-2 
Replace old named collections code with new (from #43147) part 2
 2023-01-06 13:06:26 +01:00
Robert Schulze
11fb78191c
(draft) Remove SMP deactivation on big machines 
This might be a bit controversial but
1. AMD offers 48/67/64 (true) core systems already (Milan, (*)), up to
   96 cores/CPU soon (Genoa),
2. Intel sells a few Xeon CPUs with > cores (**),
and we shouldn't artificially limit core utilization on such sytems.

(*)  https://en.wikipedia.org/wiki/Epyc
(**) https://www.intel.com/content/www/us/en/products/details/processors/xeon/scalable/platinum/products.html
 2023-01-06 10:18:00 +00:00
serxa
d217136ede review fixes 2023-01-05 20:41:36 +00:00
Yakov Olkhovskiy
1be48f706b
Merge pull request #43706 from ClibMouse/s390x_crc32 
s390x weakhash32 support
 2023-01-05 14:45:20 -05:00
Sergei Trifonov
3f87e6cd77
Update src/Common/Threading.h 
Co-authored-by: Igor Nikonov <954088+devcrafter@users.noreply.github.com>
 2023-01-05 15:27:51 +01:00
Sergei Trifonov
985dff1dbf
Update src/Common/tests/gtest_threading.cpp 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2023-01-05 15:27:32 +01:00
Sergei Trifonov
c507d7ecef
Update src/Common/tests/gtest_threading.cpp 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2023-01-05 15:27:21 +01:00
Ilya Yatsishin
0a60fe6e60
Merge pull request #44895 from Algunenano/ignore_label_issues_on_hwmon 
Ignore hwmon sensors on label read issues
 2023-01-05 12:43:37 +01:00
serxa
abf63d0c33 cleanup 2023-01-05 00:26:28 +00:00
serxa
c4e896f73a cleanup 2023-01-05 00:23:10 +00:00
Sergei Trifonov
5d5cf7d5b0
Merge branch 'master' into cancellable-mutex 2023-01-05 00:50:38 +01:00
serxa
f706cf8903 fix style 2023-01-04 23:49:29 +00:00
serxa
e9e3414ae1 remove debug print 2023-01-04 23:04:04 +00:00
serxa
3ea04f0429 fix typos 2023-01-04 22:59:29 +00:00
serxa
baf6297f1d add fast and cancellable shared_mutex alternatives 2023-01-04 21:50:57 +00:00
Yakov Olkhovskiy
7a5a36cbed
Merge branch 'master' into refactoring-ip-types 2023-01-04 11:11:06 -05:00
Nikolay Degterinsky
aa52956412 Add bcrypt authentification type 2023-01-04 15:33:16 +00:00
Maksim Kita
2482acc600
Merge pull request #44881 from azat/concurrent-bounded-queue-fix 
Fix ConcurrentBoundedQueue::emplace() return value in case of finished queue
 2023-01-04 15:44:54 +03:00
Raúl Marín
05ce5eec06 Ignore hwmon sensors on label read issues 2023-01-04 12:05:13 +01:00
Alexey Milovidov
a804d9a643
Merge pull request #44856 from ClickHouse/fix-distinct-zero-size-key-44831 
Fix: sorted DISTINCT with empty string
 2023-01-04 05:16:00 +03:00
Alexey Milovidov
3a027d285f
Merge pull request #44828 from ClickHouse/remove-two-lines-of-code 
What if I will remove two lines of code?
 2023-01-04 04:50:52 +03:00
Nikolay Degterinsky
c6daf0a6eb Merge remote-tracking branch 'upstream/master' into ulid 2023-01-03 19:58:50 +00:00
Azat Khuzhin
a2f17d2d91 Fix ConcurrentBoundedQueue::emplace() return value in case of finished queue 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-01-03 20:36:12 +01:00
Robert Schulze
cfb6feffde
What happens if I remove these 139 lines of code? 2023-01-03 18:35:31 +00:00
Raúl Marín
0785bf6b29 Do not throw DB::Exception when folders do not exist 2023-01-03 17:05:54 +01:00
kssenii
67509aa2d5 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2023-01-03 16:41:30 +01:00
kssenii
bda8f0d410 Better async buffer from remote 2023-01-03 15:27:37 +01:00
Robert Schulze
421473e974
Merge pull request #44421 from ClickHouse/revert-of-revert-of-age-function 
Revert revert of age function
 2023-01-03 10:02:18 +01:00
Igor Nikonov
b7062c182a Fix: sorted DISTINCT with empty string 2023-01-03 00:23:05 +00:00
Alexey Milovidov
e855d3519a
Merge branch 'master' into refactoring-ip-types 2023-01-02 21:58:53 +03:00
Robert Schulze
e9e04166d9
Merge remote-tracking branch 'origin/master' into query-result-cache 2023-01-02 08:20:27 +00:00
Larry Luo
71da1b1379 Updated inverted index to use ActionsDAG 2022-12-31 18:05:06 -08:00
HarryLeeIBM
6a5fef632d Added more tests and experimental flag, etc. 2022-12-31 18:05:01 -08:00
Harry-Lee
b43ddde50a Refactor full text search according to reviews 2022-12-31 18:03:14 -08:00
Larry Luo
7d1c34316a Fixed a type in comments 2022-12-31 18:03:14 -08:00
Larry Luo
ca4893307b Reference FST paper title directly 2022-12-31 18:03:14 -08:00
Larry Luo
00e4411740 Created FST namespace to hide FST details 2022-12-31 18:03:14 -08:00
Larry Luo
514e17d037 Code clean up 2022-12-31 18:03:14 -08:00
Larry Luo
1d9bf9962c Code clean up 2022-12-31 18:03:14 -08:00
Larry Luo
7b850b6d93 Refactoring 2022-12-31 18:03:13 -08:00
Larry Luo
3652740535 Code clean up 2022-12-31 18:03:13 -08:00
Larry Luo
0b63d6b5be Resolve conflicts and style errors. 2022-12-31 18:03:13 -08:00
HarryLeeIBM
0839ff60ee Initial Checkin of full text search 2022-12-31 18:00:17 -08:00
Nikolay Degterinsky
dfe93b5d82
Merge pull request #42284 from Algunenano/perf_experiment 
Performance experiment
 2022-12-30 03:14:22 +01:00
Alexey Milovidov
9378194320
Merge pull request #44688 from ClickHouse/improve-odbc-test 
Improve odbc test
 2022-12-29 15:40:47 +03:00
Nikolay Degterinsky
945d50af87 Fix build 2022-12-29 02:17:52 +00:00
Raúl Marín
5de11979ce
Unify query elapsed time measurements (#43455) 
* Unify query elapsed time reporting

* add-test: Make shell tests executable

* Add some tests around query elapsed time

* Style and ubsan
 2022-12-28 21:01:41 +01:00
Alexey Milovidov
4fce10940e
Merge pull request #44629 from Algunenano/revert_flamegraph 
Revert "Merge pull request #38953 from ClickHouse/add-allocation-ptr-to-trace-log
 2022-12-28 22:56:50 +03:00
Raúl Marín
e915ce1e95 Merge remote-tracking branch 'blessed/master' into perf_experiment 2022-12-28 20:15:43 +01:00
Alexey Milovidov
c3eff78f20 Don't be kind with third-party code 2022-12-28 20:07:26 +01:00
taiyang-li
866c8d0fd0 change as request 2022-12-28 16:04:34 +08:00
taiyang-li
437c31eca3 merge master and fix conflicts 2022-12-28 15:02:37 +08:00
Alexey Milovidov
135a32c606
Merge branch 'master' into faster-startup-bc-check 2022-12-27 23:17:24 +03:00
Sergei Trifonov
56b3c6c68f
Merge pull request #44308 from ZhiguoZh/20221216-threadpool-notify 
Call std::condition_variable::notify_* without lock in ThreadPool
 2022-12-27 15:02:49 +01:00
Raúl Marín
948ce8db93 Revert "Merge pull request #38953 from ClickHouse/add-allocation-ptr-to-trace-log" 
This reverts commit e99849d031, reversing
changes made to 7ad3ff8a9e.
 2022-12-27 14:28:13 +01:00
Alexey Milovidov
42415d49ac
Merge pull request #44607 from ClickHouse/fix-log-messages-coordination 
Fix log messages in Coordination
 2022-12-27 15:46:14 +03:00
Alexey Milovidov
d68c5c24ca
Merge pull request #44610 from ClickHouse/better-log-meesage 
Better log message
 2022-12-27 15:45:38 +03:00
Raúl Marín
fc1fa82a39
Merge branch 'master' into perf_experiment 2022-12-27 10:51:58 +01:00
Alexey Milovidov
54c8b162c0 Better log message 2022-12-26 20:09:52 +01:00
Alexey Milovidov
f0fa0977ed Fix wrong log messages 2022-12-26 18:36:07 +01:00
Alexey Milovidov
4f3db6a63a Fix wrong log message 2022-12-26 18:35:43 +01:00
Alexey Milovidov
91c1f9de8f Faster server startup after stress test 2022-12-26 18:13:50 +01:00
Alexey Milovidov
f22fc35900 Minimal changes 2022-12-25 14:18:05 +01:00
Alexander Tokmakov
ca989e9212 less runtime format strings 2022-12-23 19:50:34 +01:00
Alexander Tokmakov
93509395fb add a column with a message pattern to text_log 2022-12-23 15:06:30 +01:00
alesapin
2fecf420cb
Merge pull request #44513 from ClickHouse/small_polishing 
Minor code polishing
 2022-12-23 13:47:26 +01:00
Alexander Gololobov
0788269c25
Merge pull request #44410 from ClickHouse/fix_unicode_bar 
Fix buffer overrun in UnicodeBar::render
 2022-12-23 13:17:24 +01:00
kssenii
8308177361 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2022-12-23 12:14:46 +01:00
kssenii
853f2ea123 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2022-12-23 11:49:02 +01:00
Alexander Gololobov
0ae7f03516
Merge branch 'master' into fix_unicode_bar 2022-12-22 14:53:33 +01:00
alesapin
fa467b3398 Minor code polishing 2022-12-22 14:31:42 +01:00
vdimir
88008726d8
Merge branch 'master' into vdimir/tmp-data-in-fs-cache-2 2022-12-22 10:58:39 +00:00
alesapin
380c6cebe4
Merge pull request #44393 from ClickHouse/small_improvement 
Lock table for share during startup for database ordinary
 2022-12-22 11:42:42 +01:00
vdimir
d30d25dbbe
Temporary files evict fs cache 2022-12-22 10:22:49 +00:00
Vladimir C
6a0210fb0f
Merge pull request #43785 from mo-avatar/binhexstringliterals 2022-12-22 10:56:18 +01:00
Yakov Olkhovskiy
a8cb29da4b
Merge branch 'master' into refactoring-ip-types 2022-12-21 23:56:24 -05:00
Alexander Gololobov
4b0cbdb4a4 Fixed unused value warning 2022-12-21 10:55:20 +01:00
Robert Schulze
f8378473c4
Fix call to wrong overload on Darwin 2022-12-21 09:00:44 +00:00
Zhiguo Zhou
823cb8a8f0 Call std::condition_variable::notify_one w/o lock in ThreadPool 
Calling notify_one function with lock held would let the notified
thread immediately block again, and increase the lock contention.
This commit moves the calls of notify_one function out of critical
sections of ThreadPool, whose lock (ThreadPoolImpl::mutex) is most
extensively contended in some workloads.
 2022-12-21 13:50:03 +08:00
kssenii
1d75f740d7 Fix tests 2022-12-20 22:33:54 +01:00
kssenii
6bd4f8c029 Merge remote-tracking branch 'upstream/master' into use-new-named-collections-code-2 2022-12-20 21:17:28 +01:00
alesapin
c902bca426 Remove races 2022-12-20 18:14:06 +01:00
Alexander Gololobov
424ae4686e Cleaups based on code review comments 2022-12-20 17:02:30 +01:00
Robert Schulze
3f03b0b53d
Revert "Revert "Add function 'age'"" 
This reverts commit 4549b14027.
 2022-12-20 10:18:43 +00:00
Raúl Marín
45d27f461b
Merge branch 'master' into perf_experiment 2022-12-20 09:07:48 +00:00
Alexander Gololobov
2ab6b7d033 Check remaining space to avoid buffer overruns; some cleanups 2022-12-20 02:39:44 +01:00
alesapin
c8d9ccf3cd Lock table for share during startup for database ordinary 2022-12-19 18:51:46 +01:00
Sergei Trifonov
6f7bba159f
Merge pull request #41840 from ClickHouse/io-scheduler 
IO scheduling subsystem
 2022-12-19 15:15:19 +01:00