Vasily Nemkov
fa27ed9d41
Less noisy tests
2023-03-25 23:31:17 +01:00
Vasily Nemkov
ce608b135f
Fixed implementation of find_first_symbols_sse42 run-time needle
...
Added some tests
2023-03-25 23:18:51 +01:00
Kruglov Pavel
713ecb2492
Merge branch 'master' into non-blocking-connect
2023-03-24 22:55:45 +01:00
Sema Checherinda
da4f2bd923
do not attach empty thread group
2023-03-24 19:53:16 +01:00
Sema Checherinda
bc107c70fa
merge and mutation make thread group for setting memory trackers right
2023-03-24 19:53:16 +01:00
Maksim Kita
5cb2d30563
Merge pull request #47316 from ClickHouse/vdimir/transform_external_query_analyzer
...
Support transformQueryForExternalDatabase for analyzer
2023-03-24 17:53:00 +01:00
Raúl Marín
bf7b0c5cfe
Different approach to inject timeouts
2023-03-24 17:12:14 +01:00
Sema Checherinda
1e462e02ed
add ut
2023-03-24 17:00:04 +01:00
Sergei Trifonov
a05aa5c1c6
Merge branch 'master' into segmentator-fix
2023-03-24 16:19:57 +01:00
Kruglov Pavel
f3f93dd06c
Merge branch 'master' into non-blocking-connect
2023-03-24 15:59:40 +01:00
serxa
28ca2de86d
fix
2023-03-24 14:58:50 +00:00
Vasily Nemkov
e22eee8055
Updated implementation of find_first_symbols for run-time needle
...
Implemented compile-time and run-time dispatching between SSE4.2 and SSE2 implementations
Added find_first_symbols_sse2
Added tests
2023-03-24 15:45:12 +01:00
Kruglov Pavel
3ee12e21fb
Merge branch 'master' into non-blocking-connect
2023-03-23 20:53:44 +01:00
Antonio Andelic
3ae09c6ce8
Merge branch 'master' into more-s3-events
2023-03-23 18:23:53 +01:00
Antonio Andelic
d6cbc5d05b
Fix tests
2023-03-23 15:01:51 +00:00
Antonio Andelic
9ea8dc4e98
Merge branch 'master' into alternative-keeper-configs
2023-03-23 09:22:46 +00:00
Antonio Andelic
b57e82ded6
Add S3 write profile events
2023-03-23 09:05:41 +00:00
Antonio Andelic
a4d1cd514d
Refactor
2023-03-23 08:58:56 +00:00
Han Fei
02de4ad6df
address comments
2023-03-22 17:50:19 +01:00
Han Fei
575c4263a3
address comments
2023-03-22 17:47:25 +01:00
Yatsishin Ilya
5786aba5eb
Merge remote-tracking branch 'origin/master' into clickhouse-help
2023-03-22 14:08:22 +00:00
serxa
4321569fc4
Merge branch 'segmentator-fix' of github.com:ClickHouse/ClickHouse into segmentator-fix
2023-03-22 13:51:23 +00:00
serxa
f3c12b8543
fix per-thread perf counters update
2023-03-22 13:51:07 +00:00
Antonio Andelic
65b99ec6a7
Merge branch 'master' into alternative-keeper-configs
2023-03-22 12:22:57 +00:00
vdimir
023d14a894
Support transformQueryForExternalDatabase for analyzer
2023-03-22 08:59:04 +01:00
Vitaly Baranov
4318212c73
Make a condition more clear.
2023-03-21 21:52:59 +01:00
Vitaly Baranov
096d0132af
More corrections.
2023-03-21 21:52:59 +01:00
Vitaly Baranov
0a0e9d060f
Corrections after review.
2023-03-21 21:52:59 +01:00
Vitaly Baranov
85d38493f6
Improve ThreadPool
...
ThreadPool::setMaxThreads() can now start new threads (if there are scheduled jobs in the queue for them);
ThreadPool::setMaxFreeThreads() can now finish free threads.
2023-03-21 21:52:59 +01:00
Nikolay Degterinsky
3a8314c13e
Merge pull request #47576 from HarryLeeIBM/hlee-s390x-storage-gen
...
Fix SipHash hashing and byte order in GenerateRandom for s389x
2023-03-21 21:06:08 +01:00
Arthur Passos
77419db791
add a few tests for runtime find_symbols, more are needed
2023-03-21 09:07:06 -03:00
Kseniia Sumarokova
3c550b4314
Merge pull request #46647 from kssenii/named-collections-finish
...
Named collections: finish replacing old code for storages
2023-03-21 12:36:46 +01:00
Kseniia Sumarokova
a9bcd022d5
Merge pull request #46241 from kssenii/named-collections-granular-access
...
Allow separate grants for every named collection
2023-03-21 12:27:53 +01:00
Vladimir C
5add1e1b2e
Merge pull request #47684 from ClickHouse/vdimir/improve_log_queries_cut
2023-03-21 11:17:54 +01:00
Robert Schulze
3b7f929001
Merge branch 'master' into rs/qc-cachebase
2023-03-20 17:08:47 +01:00
vdimir
8c48714236
wipeSensitiveDataAndCutToLength take into accound suffix length
2023-03-20 15:54:19 +00:00
kssenii
75624c52f7
Merge remote-tracking branch 'upstream/master' into named-collections-granular-access
2023-03-20 16:38:14 +01:00
vdimir
bb7a8ed862
Apply log_queries_cut_to_length in MergeTreeWhereOptimizer
2023-03-20 15:38:09 +00:00
Vitaly Baranov
198409e12a
Merge pull request #46085 from aalexfvk/alexfvk/store_udf_in_zookeeper
...
Replication of user-defined SQL functions using ZooKeeper
2023-03-20 13:41:29 +01:00
Kruglov Pavel
1d12e85d55
Merge branch 'master' into non-blocking-connect
2023-03-20 13:23:55 +01:00
kssenii
cae3b335d6
Merge remote-tracking branch 'upstream/master' into named-collections-finish
2023-03-20 11:23:22 +01:00
Alexey Milovidov
698fcba525
Merge branch 'master' into fix-mysql-leak
2023-03-20 07:37:01 +03:00
Alexey Milovidov
ddf29b69f9
Merge branch 'master' into fuzzer-of-data-formats
2023-03-20 03:26:51 +03:00
kssenii
ea375ef989
Fix memory leak
2023-03-20 00:40:06 +01:00
Robert Schulze
0e01e912fe
Preparation for libcxx(abi), llvm, clang-tidy 16
2023-03-19 17:25:41 +00:00
Robert Schulze
c4664c70a1
Merge branch 'master' into rs/qc-cachebase
2023-03-19 15:23:24 +01:00
Alexey Milovidov
6e1b45d1d6
Fix exotic trash
2023-03-19 06:48:18 +01:00
Alexey Milovidov
a99f10e16f
Better error messages
2023-03-19 05:34:44 +01:00
Alexey Milovidov
ef01579726
Fix trash - the code written in C with pointers
2023-03-19 03:19:15 +01:00
Alexander Tokmakov
f1df91cd09
Merge pull request #47659 from ClickHouse/fix_993
...
Fixes for 993 and friends
2023-03-18 14:46:43 +03:00
Sema Checherinda
0b641fcead
Merge pull request #47564 from CheSema/remove-redundant
...
remove counters for threads, fix negative counters
2023-03-18 02:23:52 +01:00
Alexander Tokmakov
d11aee2b0c
Merge branch 'master' into fix_993
2023-03-17 21:28:23 +03:00
Sema Checherinda
e8295be9be
if no group then do attachInternalTextLogsQueue to the thread only
2023-03-17 15:56:30 +01:00
Robert Schulze
f72a337074
Remove cruft from build
...
No need to check compiler flags, clang >= 15 supports all of them.
2023-03-17 13:44:04 +00:00
kssenii
bb0beb7449
Merge remote-tracking branch 'upstream/master' into named-collections-finish
2023-03-17 13:02:36 +01:00
Sema Checherinda
e2af8f6ed6
add a comment, rename a field
2023-03-17 11:59:44 +01:00
Aleksei Filatov
886b530963
[rev: 1] Fix review remarks
2023-03-17 13:56:05 +03:00
Aleksei Filatov
f84aa2f4ca
Add classes for replication UDF to ZooKeeper
2023-03-17 13:56:04 +03:00
Sergei Trifonov
1a84203c7f
Merge pull request #47656 from ClickHouse/background_pool_limit_metric
...
Add background pools size metrics
2023-03-17 11:49:53 +01:00
Alexander Tokmakov
d621b2c4ad
fix intersecting parts, better fault injections
2023-03-16 21:28:07 +01:00
Sema Checherinda
3c6deddd1d
work with comments on PR
2023-03-16 19:55:58 +01:00
serxa
3ad7194e31
add pool size metrics
2023-03-16 17:31:41 +00:00
Robert Schulze
eb206bbe25
Fix clang-tidy build
2023-03-16 16:08:24 +00:00
Robert Schulze
7d7052350c
Merge branch 'master' into rs/qc-cachebase
2023-03-16 10:42:17 +01:00
sichenzhao
88de8d4cb3
Merge branch 'master' into cgroup_v2
2023-03-15 14:35:47 -07:00
alesapin
ef7508469c
Fix wait for zero-copy replication move during manual move
2023-03-15 22:29:24 +01:00
Sema Checherinda
67ac858a52
remove counters for threads, fix negative counters
2023-03-15 22:22:22 +01:00
Dmitry Novik
be40218a9a
Revert "Revert #46622 (test_async_insert_memory)"
2023-03-15 22:14:54 +01:00
Han Fei
a532503466
Merge branch 'master' into hanfei/optimize-regexp-tree-1
2023-03-15 17:56:01 +01:00
Han Fei
d78a9e03ad
refine
2023-03-15 15:38:11 +01:00
avogar
17f5356178
Improve async reading from socket
2023-03-15 12:18:08 +00:00
robot-ch-test-poll1
43db7c59d9
Merge pull request #47260 from ClickHouse/vdimir/storage_join_insert_select_deadlock
...
Forbid insert select for the same StorageJoin
2023-03-15 12:52:29 +01:00
Robert Schulze
2f75a6da61
Merge remote-tracking branch 'origin/master' into rs/qc-cachebase
2023-03-15 11:00:53 +00:00
Robert Schulze
75ca2dd0e2
Merge remote-tracking branch 'origin/master' into rs/qc-cachebase
2023-03-15 10:55:53 +00:00
Robert Schulze
5cee479f0c
Merge pull request #47424 from ClickHouse/rs/compile-with-cpp23
...
Compile with C++23
2023-03-15 10:55:42 +01:00
Yatsishin Ilya
45c5275bfb
Merge remote-tracking branch 'origin/master' into clickhouse-help
2023-03-14 17:27:25 +00:00
Han Fei
076d33bb03
refine a little bit
2023-03-14 18:15:42 +01:00
Han Fei
01be209e43
fix test
2023-03-14 17:44:02 +01:00
HarryLeeIBM
a32d7a00bc
Fix SipHash integer hashing and byte order issue in GenerateRandom for s390x
2023-03-14 11:54:17 -04:00
Sema Checherinda
95e994a9cd
Merge pull request #47304 from arthurpassos/fix_find_first_not_symbols
...
Modify find_first_symbols so it works as expected for find_first_not_symbols
2023-03-14 14:24:05 +01:00
Alexander Tokmakov
5a1ee881f8
Merge pull request #47547 from ClickHouse/filter_too_frequent_logs
...
Hide too noisy log messages, fix some tests
2023-03-14 13:53:21 +03:00
Robert Schulze
73afae2d3f
Introduce TTLCachePolicy
2023-03-14 09:45:20 +00:00
Robert Schulze
eed365c8c8
Cleanup: Slightly more naming consistency (probably rolls back earlier changes)
2023-03-14 09:11:36 +00:00
Robert Schulze
b985795593
Cleanup: Group ICachePolicy methods
2023-03-14 08:38:22 +00:00
Robert Schulze
e1fb25a004
Extend CacheBase with getWithKey() method
2023-03-14 08:38:22 +00:00
Robert Schulze
f1450923cb
Extend CacheBase with dump() method
2023-03-14 08:38:22 +00:00
Robert Schulze
e03618357e
Cleanup: Acquire lock in CacheBase::maxSize()
...
The upcoming TTLPolicy will likely have a runtime-configurable cache
size.
2023-03-14 08:38:22 +00:00
Michael Kolupaev
d3a514d221
Compress marks in memory
2023-03-13 16:29:00 -07:00
Alexander Tokmakov
ba6c6255a3
hide too noisy log messages, fix some tests
2023-03-14 00:23:36 +01:00
Alexey Milovidov
a411ae10b5
Merge pull request #47486 from ClickHouse/revert_25674
...
Remove "asynchronous connection drain"
2023-03-13 21:40:39 +03:00
Han Fei
de8d0040a4
refine code
2023-03-13 18:34:47 +01:00
Robert Schulze
1d8004756a
Cleanup: Reuse aliases instead of redeclaring it
2023-03-13 15:11:12 +00:00
Robert Schulze
657aa65446
Cleanup: Untangle CacheBase's constructors
...
This prepares for the custom TTLCachePolicy which users should not be
able to configure explicitly.
2023-03-13 15:11:12 +00:00
Vladimir C
9cb018545c
Merge pull request #47499 from FrankChen021/span_kind
2023-03-13 11:35:03 +01:00
Robert Schulze
475415e421
Cleanup: Move on_weight_loss_function into concrete cache policies
...
Makes ICachePolicy a pure abstract interface
2023-03-13 09:20:07 +00:00
Robert Schulze
d165a15c58
Cleanup: Simplify some typedefs
2023-03-13 09:20:02 +00:00
Robert Schulze
614810e471
Cleanup: Fix file name typo
2023-03-13 09:19:58 +00:00
Robert Schulze
9f6cb98c61
Cleanup: Remove default parameters from (S)LRUCachePolicy
...
Allows to add new mandatory parameters without accidentally breaking
something.
2023-03-13 09:19:55 +00:00
Robert Schulze
39898fbe64
Cleanup: max_elements_size --> max_entries
...
To disambiguate the maximum number of entries from the maximum byte size
of an entry.
2023-03-13 09:19:51 +00:00
Robert Schulze
7d5bc0d8c9
Cleanup: TrivialWeightFunction --> EqualWeightFunction
...
A true "trival" weight function would return .size() of the cache entry.
2023-03-13 09:19:45 +00:00
Robert Schulze
c300f5e68f
Cleanup: max_size --> max_size_in_bytes
...
To disambiguate the threshold from the maximum number of cache entries.
2023-03-13 09:19:39 +00:00
Frank Chen
868756ea76
Fix style
2023-03-13 09:40:55 +08:00
Alexey Milovidov
f445649845
Merge branch 'master' into rs/compile-with-cpp23
2023-03-12 21:33:20 +03:00
Frank Chen
dfcb9c9431
Add SpanKind
2023-03-12 21:30:29 +08:00
Alexander Tokmakov
7b1b238d0b
Revert "Merge pull request #25674 from amosbird/distributedreturnconnection"
...
This reverts commit 5ffd99dfd42796aa333f
2023-03-11 19:09:47 +01:00
Alexander Tokmakov
0359b3e336
Revert "Merge pull request #46622 from ClickHouse/async-insert-memory-fix"
...
This reverts commit 67469ad46b678e4250cd
2023-03-11 17:01:10 +01:00
Robert Schulze
348a40aa5d
Compile with C++23
2023-03-10 19:11:58 +00:00
Dmitry Novik
4f24a7bd11
Merge remote-tracking branch 'origin/master' into background-memory-tracker
2023-03-10 18:57:45 +00:00
Han Fei
39a1185486
fix test
2023-03-10 15:30:29 +01:00
Arthur Passos
dd77962048
refactor to use string instead of string_views
2023-03-10 10:42:22 -03:00
Arthur Passos
003bbb2685
initialize empty string_view with empty string so std::string_view::begin does not point to null
2023-03-09 13:19:51 -03:00
vdimir
c9e48f37cb
Forbid insert select for the same StorageJoin
2023-03-09 12:28:44 +00:00
Alexander Tokmakov
38f9f08e1a
Merge pull request #47268 from myrrc/improvement/more-to-base-3
...
use std::lerp, constexpr hex.h
2023-03-08 19:07:34 +03:00
Yatsishin Ilya
6cde37093a
Merge remote-tracking branch 'origin/master' into clickhouse-help
2023-03-08 13:34:18 +00:00
Alexander Tokmakov
f5cf039190
check if a symlink exists more carefully
2023-03-08 00:04:55 +01:00
Mike Kot
9920a52c51
use std::lerp, constexpr hex.h
2023-03-07 22:50:17 +00:00
Arthur Passos
5ba1b5d43c
add more descriptive tests
2023-03-07 14:04:48 -03:00
Arthur Passos
a42dee5bed
Merge branch 'ClickHouse:master' into fix_find_first_not_symbols
2023-03-07 12:19:23 -03:00
Arthur Passos
bf365cec8c
Modify find_first_symbols so it works as expected for find_first_not_symbols
2023-03-07 10:08:04 -03:00
Antonio Andelic
915a04c0ec
Merge pull request #47261 from ClickHouse/join-threads-when-exception
...
Join threads if exception happened in `ZooKeeperImpl` constructor
2023-03-07 09:32:29 +01:00
Alexey Milovidov
93ba0f4bdf
Update StackTrace.cpp
2023-03-07 01:30:44 +03:00
sichenzhao
500f56430f
Merge branch 'master' into cgroup_v2
2023-03-06 13:38:33 -08:00
Han Fei
420108a7a0
support alternatives
2023-03-06 19:10:36 +01:00
Mike Kot
07a9e486e6
memset for frame pointers'
2023-03-06 19:35:00 +03:00
Mike Kot
4bc443f3c7
initial solution
2023-03-06 19:35:00 +03:00
Antonio Andelic
12525f768c
Add default constructor for MultiReadResponse ( #47254 )
...
* Add default constructor for MultiReadResponse
* Remove optional
* Fix style
2023-03-06 14:18:01 +01:00
Antonio Andelic
9117c7491d
Join threads if exception happened in constructor
2023-03-06 12:55:02 +00:00
Han Fei
b7eef62458
Merge pull request #45491 from azat/dist/async-send-refactoring
...
[RFC] Rewrite distributed sends to avoid using filesystem as a queue, use in-memory queue instead
2023-03-06 12:32:33 +01:00
Nikolay Degterinsky
7f15575500
Merge pull request #47208 from ClibMouse/z_ip_coding_fix
...
Fix ip coding on s390x
2023-03-04 18:18:14 +01:00
Alexander Tokmakov
082194a755
Merge pull request #47175 from ClickHouse/correct_fatal_handling
...
More correct handling of fatal errors
2023-03-04 00:38:00 +03:00
Sema Checherinda
85178c9609
Merge pull request #47102 from CheSema/merge-mutate-do-thread-group
...
do flushUntrackedMemory when context switches
2023-03-03 20:23:25 +01:00
Han Fei
c1e80683c4
Refine OptimizeRegularExpression Function
2023-03-03 17:59:21 +01:00
Suzy Wang
e44e528118
remove unnecessary reverse memcpy
2023-03-03 06:20:18 -08:00
Alexander Tokmakov
cadaf06829
Merge pull request #46681 from ClickHouse/fix_insert_cancellation_in_native_protocol
...
Fix queries cancellation when a client dies
2023-03-03 16:27:20 +03:00
Alexander Tokmakov
b1a288d271
Merge branch 'master' into correct_fatal_handling
2023-03-03 14:50:07 +03:00
Kseniia Sumarokova
386663953c
Merge branch 'master' into named-collections-finish
2023-03-03 12:23:38 +01:00
Alexander Tokmakov
619b282060
more correct handling of fatal errors
2023-03-03 01:22:04 +01:00
LiuNeng
d4c5ab9dcd
Optimize one nullable key aggregate performance ( #45772 )
2023-03-02 21:01:52 +01:00
Sema Checherinda
7a9f4aab8e
do not call adjustWithUntrackedMemory when untracked_memory is 0
2023-03-02 19:21:03 +01:00
Sema Checherinda
a6c2473518
do flushUntrackedMemory when context swith
2023-03-02 19:21:03 +01:00
kssenii
0c4bb3c80f
Merge remote-tracking branch 'upstream/master' into named-collections-granular-access
2023-03-02 19:08:29 +01:00
Alexander Tokmakov
c518f3690e
Merge branch 'master' into fix_insert_cancellation_in_native_protocol
2023-03-02 18:24:08 +01:00
Yatsishin Ilya
d4304d2028
Merge remote-tracking branch 'origin/master' into clickhouse-help
2023-03-02 16:16:18 +00:00
Robert Schulze
740aeaba1f
Apply some CTAD
2023-03-02 13:36:47 +00:00
Alexey Milovidov
97c0f8a0e0
Merge pull request #47027 from amosbird/clang16
...
Prepare to build with clang 16
2023-03-02 02:10:03 +03:00
Robert Schulze
15fec805fb
Merge pull request #46950 from ClickHouse/rs/uninline-likePatternToRegexp
...
Un-inline likePatternToRegexp()
2023-03-01 21:38:40 +01:00
Alexander Tokmakov
a97e15e36f
Merge branch 'master' into fix_insert_cancellation_in_native_protocol
2023-03-01 14:26:41 +01:00
Amos Bird
2d8060d843
Rewrite VersionNumber in ClickHouse way
2023-03-01 11:07:31 +08:00
Azat Khuzhin
572dd76eae
Extend ConcurrentBoundedQueue to deque
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-28 22:33:36 +01:00
Kruglov Pavel
66634cfd7c
Merge pull request #46618 from kitaisreal/map-field-to-string-fix
...
Map field to string fix
2023-02-28 20:29:40 +01:00
Robert Schulze
2c0395b346
Merge pull request #46814 from ClickHouse/rs/search-refactoring
...
Cleanup string search code
2023-02-28 15:04:19 +01:00
Maksim Kita
0764649c5d
Merge pull request #46632 from ClickHouse/analyzer-limit-settings-2
...
Analyzer - apply limit and offset settings - second attempt
2023-02-27 21:39:34 +03:00
kssenii
ad88251ee7
Fix tests
2023-02-27 17:42:04 +01:00
Robert Schulze
35f4b42608
Move likePatternToRegexp from Functions/ to Common/
2023-02-27 16:41:15 +00:00
Robert Schulze
5c8edcc2e7
Merge pull request #46940 from ClickHouse/rs/sizes-of-arrays-dont-match
...
Typo: SIZES_OF_ARRAYS_DOESNT_MATCH --> SIZES_OF_ARRAYS_DONT_MATCH
2023-02-27 14:33:09 +01:00
kssenii
f398c5d471
Fix style
2023-02-27 12:22:27 +01:00
Robert Schulze
22d09caea2
SIZES_OF_ARRAYS_DOESN_MATCH --> SIZES_OF_ARRAYS_DONT_MATCH
2023-02-27 08:31:40 +00:00
Maksim Kita
8ad016dbe6
Map field to string fix
2023-02-26 13:38:45 +01:00
Azat Khuzhin
03e90cd7cf
Add new metrics to system.asynchronous_metrics
...
The following metrics can be useful to calculate various rates (i.e.
disk/network IO rates):
- AsynchronousHeavyMetricsUpdateInterval
- AsynchronousMetricsUpdateInterval
The following had been added by analogy with the
AsynchronousMetricsCalculationTimeSpent:
- AsynchronousHeavyMetricsCalculationTimeSpent
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-25 21:13:15 +01:00
Alexey Milovidov
f28ab14759
Better handling of fatal errors
2023-02-24 23:29:06 +01:00
kssenii
c2bcc4119f
Better
2023-02-24 20:57:37 +01:00
Yakov Olkhovskiy
da326dda03
comments fix
2023-02-24 19:19:51 +00:00
kssenii
f2a9109fe1
Merge remote-tracking branch 'upstream/master' into named-collections-granular-access
2023-02-24 15:20:46 +01:00
Robert Schulze
627961d9db
More cosmetics
2023-02-24 10:14:58 +00:00
Robert Schulze
421afeeae0
Shuffle class order (and just that)
2023-02-24 10:13:35 +00:00
Robert Schulze
ef529de7db
Cosmetics
2023-02-24 10:12:47 +00:00
Alexander Tokmakov
fe92fd8a61
Merge branch 'master' into fix_insert_cancellation_in_native_protocol
2023-02-23 22:35:31 +01:00
Robert Schulze
6aa4e93284
Merge branch 'master' into StringSearcher
2023-02-23 11:54:05 +01:00
Jiebin Sun
d6710d9b34
Align all the SSE4.1 requirement and use needle_size
...
Align all the SSE4.1 requirement from StringSearcher. Use needle_size
in while loop to make the code clean.
Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>
2023-02-22 16:15:26 -05:00
Yakov Olkhovskiy
99095446af
review suggestions
2023-02-22 17:22:13 +00:00
Jiebin Sun
1f62135ba7
Make the optimized SIMD StringSearcher clean
...
This patch has revised the name of value and added comments to make
the SIMD StringSearcher clean and easy to understand based on pull
request 46289.
Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>
2023-02-22 12:18:21 -05:00
Jiebin Sun
d220e7f4fc
Optimize the SIMD StringSearcher if needle_size is large
...
This patch offers an additional optimization when the needle_size is
large. If the needle_size is larger than the haystack_size, there is
no need to search any more.
The optimized SIMD StringSearcher has led at most 41.7% than Volnitsky
algorithm when the needle_size is less than 21, and fallen behind only
about 1% even when the needle_size is bigger than 50, which is not
considered as a common case.
Test platform: ICX server
Test query: SELECT COUNT(*) FROM hits WHERE URL LIKE '%{Needle}%';
Needle_size opt/baseline
5 141.7%
6 129.4%
8 118.5%
9 112.3%
10 107.4%
14 103.4%
20 100.2%
21 100.7%
51 99.0%
Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>
2023-02-22 11:58:17 -05:00
Jiebin Sun
f5a6a86dec
Optimize the SIMD StringSearcher by searching first two chars
...
This patch offers the optimized SIMD StringSearcher by searching the first
and second chars together rather than only the first char, which will result
in big performance gain. The patch also provides a quick path when the needle
size is 1.
With this patch, I have tested the 43 queries in clickbench on ICX server.
Query 20 has got 35% performance gain. Other StringSearcher related queries
have got around 10% performance improvement. And the overall geomean of all
the queries has got 4.1% performance gain.
Signed-off-by: Jiebin Sun <jiebin.sun@intel.com>
2023-02-22 11:55:30 -05:00
Alexander Tokmakov
e660c0838c
fix multipart requests
2023-02-22 17:54:35 +01:00
Yatsishin Ilya
80857638c1
Merge remote-tracking branch 'origin/master' into clickhouse-help
2023-02-22 13:58:17 +00:00
kssenii
f0ca3ef207
Merge remote-tracking branch 'upstream/master' into named-collections-granular-access
2023-02-22 14:02:29 +01:00
kssenii
f518ddd96d
Merge remote-tracking branch 'upstream/master' into named-collections-finish
2023-02-22 13:43:18 +01:00
Kruglov Pavel
3ba3fdbfa3
Merge pull request #46607 from kssenii/delay-loading-of-named-collections
...
Do not load named collections on server startup (on first access instead)
2023-02-22 13:22:34 +01:00
Dmitry Novik
67469ad46b
Merge pull request #46622 from ClickHouse/async-insert-memory-fix
...
Fix MemoryTracker counters for async inserts
2023-02-22 12:27:05 +01:00
Sichen Zhao
5772b9769e
fix types
2023-02-21 22:49:40 -08:00
Sichen Zhao
d0de9d5887
updates for comments, treat special case for string value max memory, and extract shared code out
2023-02-21 22:47:26 -08:00
Alexander Tokmakov
592af6d652
fix incomplete interst through http
2023-02-22 02:34:03 +01:00
Yakov Olkhovskiy
cf493d1dfb
replace settings limit and offset with corresponding expression nodes
2023-02-21 22:36:02 +00:00
Dmitry Novik
ad1e5f3918
Review fixes
2023-02-21 22:06:17 +00:00
Vladimir C
a791250a4a
Merge pull request #38614 from bharatnc/ncb/profile-events-partlog
2023-02-21 11:06:44 +01:00
Kseniia Sumarokova
24d0490ef3
Merge branch 'master' into delay-loading-of-named-collections
2023-02-21 00:24:18 +01:00
kssenii
a54b011670
Finish for mysql
2023-02-20 21:37:38 +01:00
Dmitry Novik
3c4c527bce
Fix MemoryTracker counters for async inserts
2023-02-20 18:40:57 +00:00
kssenii
9d16205c8a
Load named collections on first access
2023-02-20 16:33:45 +01:00
Alexey Milovidov
5ef159fa10
Merge pull request #46565 from ClickHouse/remove-pvs-studio
...
Remove PVS-Studio
2023-02-20 17:10:53 +03:00
Robert Schulze
135170b27d
Merge pull request #44973 from ClickHouse/remove-smp-limitation
...
Slightly modify the detection of the number of CPU cores
2023-02-20 11:22:39 +01:00
Alexey Milovidov
d8cda3dbb8
Remove PVS-Studio
2023-02-19 23:30:05 +01:00
Alexey Milovidov
a22f0eee2a
Merge branch 'master' into Aggregate_Sum_AVX512
2023-02-19 04:03:45 +03:00
Dmitry Novik
728434821a
Merge remote-tracking branch 'origin/master' into background-memory-tracker
2023-02-16 12:35:07 +00:00
Alexey Milovidov
e8d1a2f099
Update AsynchronousMetrics.cpp
2023-02-15 23:52:53 +03:00
Dmitry Novik
e7a94a28ee
Merge branch 'master' into background-memory-tracker
2023-02-15 19:39:28 +01:00
kssenii
0c53f33e6f
Make code more common: add GlobalWithParameter level
2023-02-15 16:59:43 +01:00
Nikolay Degterinsky
494fd472f0
Merge branch 'master' into ulid
2023-02-15 07:24:32 +01:00
vdimir
1b715a3388
Allow scope override in ThreadStatus::attachProfileCountersScope
2023-02-14 14:11:13 +00:00
vdimir
4c1aeb7972
Use overload of PartLog::addNewPart with profile_events
2023-02-14 14:11:12 +00:00
vdimir
fa08f72bba
Move subthread_profile_events to ThreadStatus
2023-02-14 14:11:11 +00:00
vdimir
994dcb93f9
Rename ScopedProfileEvents -> ProfileEventsScope
2023-02-14 14:11:10 +00:00
vdimir
a228f7f419
[wip] ProfileCounters for each part
2023-02-14 14:11:09 +00:00
Alexey Milovidov
f15623141e
Fix error
2023-02-14 14:11:08 +00:00
Alexey Milovidov
f554ff9d99
Remove trash
2023-02-14 14:11:08 +00:00
John Skopis
f7604cc686
[feat] Add ProfileEvents map to PartLog
...
closes #10316
2023-02-14 14:11:02 +00:00
Nikolay Degterinsky
09da5575e3
Merge remote-tracking branch 'upstream/master' into ulid
2023-02-14 13:41:17 +00:00
robot-clickhouse-ci-2
ea5b645af5
Merge pull request #46349 from ClickHouse/fix-crash-int-overflow-hash-table-deserialization-agg-state
...
Check overflow in HashTable::alloc
2023-02-14 10:02:13 +01:00
Alexander Gololobov
3f5887248e
Update HashTable.h
2023-02-13 17:30:57 +01:00
Nikolai Kochetov
ea19a0d46e
Update src/Common/HashTable/HashTable.h
...
Co-authored-by: Alexander Gololobov <440544+davenger@users.noreply.github.com>
2023-02-13 17:15:41 +01:00
Nikolai Kochetov
5d77407f39
Check overflow in HashTable::alloc
2023-02-13 14:50:23 +00:00
Suzy Wang
e140bed3ea
Merge branch 'master' into z-build-0120
2023-02-13 09:06:48 -05:00
Alexey Milovidov
2e57fab8e3
Instrumentation of callbacks for distributed queries
2023-02-12 10:04:05 +01:00
kssenii
3067c1d723
Merge remote-tracking branch 'upstream/master' into resubmit-prefetches
2023-02-11 11:36:23 +01:00
Kseniia Sumarokova
3be36c7979
Merge pull request #41976 from kssenii/allow-single-disk-instead-of-storage-policy
...
Allow configuring storage as `SETTINGS disk='<disk_name>'` (instead of `storage_policy`) and explicit disk creation `SETTINGS disk=disk(<disk_configuration>)`
2023-02-10 21:38:50 +01:00
Alexey Milovidov
d5d87bc6b7
Merge pull request #46152 from ClickHouse/with-fill-bigint
...
More convenient usage of big integers and ORDER BY WITH FILL.
2023-02-10 20:05:32 +03:00
Vladimir C
bd39e29cab
Merge pull request #46065 from aiven-sal/aiven-sal/siphash128
2023-02-10 10:41:45 +01:00
Suzy Wang
64313d7020
Merge branch 'master' into z-build-0120
2023-02-09 14:49:22 -05:00
Vitaly Baranov
39bacfb9dc
Merge branch 'master' into allow-single-disk-instead-of-storage-policy
2023-02-09 13:16:55 +01:00
Alexey Milovidov
d9fbf643bc
Merge pull request #45226 from hanfei1991/hanfei/gwp-asan
...
Introduce llvm/gwp-asan
2023-02-09 09:24:52 +03:00
pufit
7cf71d1f82
Merge pull request #46144 from ClickHouse/46131-fix-config-attributes-merge
...
Fix config merge for XML attributes.
2023-02-08 23:28:13 -05:00
Alexey Milovidov
292a5abbc9
Merge remote-tracking branch 'origin/master' into with-fill-bigint
2023-02-09 01:47:51 +01:00
Alexander Tokmakov
4fd4058259
Merge pull request #45629 from arthurpassos/fix_cares_crash
...
Cancel c-ares failed requests and retry on system interrupts to prevent callbacks with dangling references and premature resolution failures
2023-02-09 02:29:29 +03:00
Alexander Tokmakov
8101b044fa
Merge pull request #46091 from azat/sanity-assertions
...
Sanity assertions for closing file descriptors
2023-02-09 01:02:03 +03:00
Han Fei
64b84f8e31
fix style
2023-02-08 22:16:28 +01:00
kssenii
b0b865c32e
Resubmit prefetches
2023-02-08 21:26:24 +01:00
Han Fei
8f665b5bbf
fix
2023-02-08 18:28:06 +01:00
Han Fei
a4efb1fbda
fix compile
2023-02-08 18:06:36 +01:00
Han Fei
d9aab84701
fix fork hang
2023-02-08 16:20:42 +01:00
pufit
e833decd88
try to fix memory leak
2023-02-08 15:17:46 +00:00
Antonio Andelic
45d4ba59dc
Merge pull request #46068 from bigo-sg/alloc_map_metrics
...
Add new metric to record allocations times and bytes using mmap
2023-02-08 15:49:20 +01:00
Alexey Milovidov
4f2a58ddbd
Fix style
2023-02-08 08:44:31 +01:00
Alexey Milovidov
0791b85f57
Remove bits of trash
2023-02-08 08:18:31 +01:00
Alexey Milovidov
d66beb9ac4
Allow accurate comparison of Big Int with other integers
2023-02-08 07:56:00 +01:00
pufit
c066c0dfdb
Fix config merge for an attributes
2023-02-07 23:46:13 +00:00
Dmitry Novik
d43d17f59a
Address the review
2023-02-07 16:05:02 +00:00
Yatsishin Ilya
1baa15d603
Merge remote-tracking branch 'origin/master' into clickhouse-help
2023-02-07 15:46:59 +00:00
Han Fei
0f7defb87f
Merge branch 'master' into hanfei/gwp-asan
2023-02-07 14:55:55 +01:00
Robert Schulze
690d873126
Merge pull request #46088 from ClickHouse/rs/fix-liburing-build
...
Fix build with `-DENABLE_LIBURING=0` (or `-DENABLE_LIBRARIES=0`)
2023-02-07 12:16:14 +01:00
Azat Khuzhin
8cc41b7f41
Check return value of ::close()
...
Note, that according close(2), EINTR should not be retriable for close()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-07 11:28:22 +01:00
Salvatore Mesoraca
001b714d3e
Common: SipHash: add original 128-bit output implementation
2023-02-07 11:19:04 +01:00
Azat Khuzhin
e83163e25f
Add sanity check of evtimer_del() in QueryProfiler
...
Saw this one time on CI with EINVAL [1], while this was EINVAL this
time, next time it can be completelly different fd and break something
in a different place.
[1]: https://s3.amazonaws.com/clickhouse-test-reports/45654/1716af465d376f5335720b5045bcccdd9e1823aa/fast_test.html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-02-06 20:13:05 +01:00
Dmitry Novik
1e065b32f3
Add MemoryTracker for the background tasks
2023-02-06 17:25:58 +00:00
Robert Schulze
8840aa9242
Fix build with -DENABLE_LIBURING=0 (or -DENABLE_LIBRARIES=0)
...
Follow-up to #38456
The problem was that code which uses liburing was conditionally build
depending on define OS_LINUX and not based on whether liburing was
enabled/disabled at build time.
2023-02-06 16:57:42 +00:00
Robert Schulze
4c89327c9f
Add TODO comment to MultiVersion.h
...
Follow-up to #46057
2023-02-06 11:04:58 +00:00
taiyang-li
7b10ac8d5b
add new metric to record allocations times and bytes using mmap
2023-02-06 17:50:45 +08:00
Konstantin Morozov
ff05162904
change mutex to lock-free
2023-02-05 19:25:09 +03:00
kssenii
5abbc7cad6
Cleanup
2023-02-04 19:41:41 +01:00
kssenii
cce23cee51
Fix build
2023-02-04 13:49:32 +01:00
Yatsishin Ilya
f4cfd8a2d9
Merge remote-tracking branch 'origin' into clickhouse-help
2023-02-03 20:08:23 +00:00
kssenii
536c02ca8a
Adding custom field continued
2023-02-03 21:05:18 +01:00
kssenii
5f124263a6
Revert changes with complex SettingsChanges
2023-02-03 21:04:45 +01:00
Sichen Zhao
aa5230b525
updates path name
2023-02-03 11:18:53 -08:00
Alexander Tokmakov
3f11948bb0
Merge branch 'master' into stack_trace_in_part_log
2023-02-03 20:05:00 +03:00
kssenii
12b62de374
Add field CustomType
2023-02-03 17:07:02 +01:00
Robert Schulze
85cbb9288c
Merge pull request #45456 from FrankChen021/uncaught_exception
...
Fix uncaught exception in HTTPHandler
2023-02-03 15:26:02 +01:00
Anton Popov
cdbe145bc1
Merge pull request #45796 from CurtizJ/fix-leak-in-azure-sdk
...
Fix test `test_azure_blob_storage_zero_copy_replication ` (memory leak in azure sdk)
2023-02-03 14:16:19 +01:00
Frank Chen
d38adfab30
Merge two overridden functions as one
2023-02-03 15:27:45 +08:00
Sichen Zhao
d4ca058643
updates to support cgroupv2 as well
2023-02-02 22:19:29 -08:00
Frank Chen
d3a05a11da
Merge remote-tracking branch 'remotes/github/master' into stack_trace_in_part_log
2023-02-03 11:39:43 +08:00
Frank Chen
7ad4b3176a
Clean code
2023-02-03 11:13:45 +08:00
Frank Chen
93ead69b7f
Resolve review comments
2023-02-03 11:05:54 +08:00
kssenii
a391de18a1
Merge remote-tracking branch 'upstream/master' into allow-single-disk-instead-of-storage-policy
2023-02-02 20:49:41 +01:00
Nikolai Kochetov
6cd0b51127
Merge pull request #45871 from ClickHouse/fix-ipv6-parser
...
Fix ipv6 parser
2023-02-01 14:28:54 +01:00
Robert Schulze
b512316586
Merge pull request #45682 from ClickHouse/rename-qrc-to-qc
...
Rename "Query Result Cache" to "Query Cache"
2023-02-01 11:23:29 +01:00
Alexey Milovidov
bf9f62dcbd
Merge pull request #38456 from sauliusvl/iouring
...
Re-introduce io_uring read method
2023-02-01 06:41:32 +03:00
Yakov Olkhovskiy
3d64c84571
fix ipv6 parser
2023-02-01 01:02:58 +00:00
Yatsishin Ilya
98edb9f06b
Update help for clickhouse tools and add test
2023-01-31 12:19:37 +00:00
Robert Schulze
325c6bdf3d
Renaming: "Query Result Cache" --> "Query Cache"
...
Reasons:
- The cache will at some point store intermediate results as opposed to
only query results. We should change the terminology now without
having to worry about backward compat.
- Equivalent caches in MySQL (1) and Starrocks (2) are called "query
cache".
- The new name is ca. 13.8% more catchy.
(1) https://dev.mysql.com/doc/refman/5.6/en/query-cache.html
(2) https://docs.starrocks.io/en-us/2.5/using_starrocks/query_cache
2023-01-31 09:54:34 +00:00
Alexey Milovidov
7aa10a430a
Merge branch 'master' into hanfei/gwp-asan
2023-01-31 04:13:59 +03:00
Anton Popov
31e8b692f4
fix typo
2023-01-31 01:13:23 +00:00
Anton Popov
839cd614fb
fix memory leak in azure sdk
2023-01-31 01:01:10 +00:00
Vitaly Baranov
2e3a3cc4dc
Merge pull request #45701 from vitlibar/add-setting-allow-head-object-request
...
Add new S3 setting allow_head_object_request
2023-01-30 17:41:09 +01:00
Han Fei
4971358de5
fix init
2023-01-30 17:07:01 +01:00
Arthur Passos
7b1ceaa2c5
Merge branch 'master' into fix_cares_crash
2023-01-30 12:05:56 -03:00
Vladimir C
e7e8ae979f
Merge pull request #45271 from jh0x/feature-array-shuffle
2023-01-30 11:54:07 +01:00
Robert Schulze
66392b873c
Merge pull request #45513 from aiven-sal/aiven-sal/siphash_pr
...
Keyed SipHash
2023-01-30 11:10:00 +01:00
Alexey Milovidov
a7299746c7
Merge pull request #45743 from ClickHouse/oom-message
...
Improve MEMERY_LIMIT_EXCEEDED exception message
2023-01-30 00:46:04 +03:00
Robert Schulze
33a719b9be
Update getNumberOfPhysicalCPUCores.cpp
2023-01-29 21:37:19 +01:00
Robert Schulze
15ae2d1de5
Merge branch 'master' into aiven-sal/siphash_pr
2023-01-29 21:06:52 +01:00
Robert Schulze
1bf280cb50
Handle obscure case that /proc/cpuinfo is unmounted
2023-01-29 17:25:08 +00:00
alesapin
631c8fb155
Merge branch 'master' into add-setting-allow-head-object-request
2023-01-29 14:58:27 +01:00
Dmitry Novik
ec1f6bfd37
Improve MEMERY_LIMIT_EXCEEDED exception message
2023-01-29 01:52:37 +00:00
Saulius Valatka
8505b8c57a
add profile events for failed/succeeded CQE completions
2023-01-28 21:54:44 +02:00
Saulius Valatka
8fa9a99ba1
implement pending request queue to prevent CQ overflows
2023-01-28 21:54:44 +02:00
Saulius Valatka
ac2c921bdf
add initial io_uring support
2023-01-28 21:54:44 +02:00
Azat Khuzhin
6b42b66257
Simply filesystem helpers to check is-readable/writable/executable
...
Use one system call - faccessat(), over multiple system calls.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-27 21:11:10 +01:00
Joanna Hulboj
9a559b5475
FIXUP: More comments about shuffle
2023-01-27 20:07:56 +00:00
Joanna Hulboj
31eb936457
Added Fisher-Yates shuffle and partial-shuffle
2023-01-27 20:07:55 +00:00
Arthur Passos
39cbf4d4dc
make random number generator thread safe
2023-01-27 12:14:07 -03:00
Arthur Passos
e559a6b1d2
address some comments
2023-01-27 12:07:00 -03:00
Vitaly Baranov
d02b255b55
Add new setting "allow_head_object_request" to S3::RequestSettings.
2023-01-27 15:10:34 +01:00
Alexander Tokmakov
c366806c3e
Merge pull request #45527 from ClickHouse/exception_message_patterns4
...
Better formatting for exception messages 2
2023-01-27 15:31:52 +03:00
Salvatore Mesoraca
2e72e27206
common: siphash: add support for custom keys
2023-01-27 13:00:53 +01:00
Alexey Milovidov
5b257ab806
Merge pull request #45233 from ClickHouse/improve_week_day
...
Revert "Revert "Improve week day""
2023-01-27 02:44:17 +03:00
Arthur Passos
ea38df2c28
add comments to the code
2023-01-26 13:54:12 -03:00
Alexander Tokmakov
a584ad0eb1
forbid runtime strings
2023-01-26 10:52:47 +01:00
Alexander Tokmakov
9b670946db
Merge branch 'master' into exception_message_patterns5
2023-01-26 00:41:32 +01:00
Arthur Passos
4a10f49ca8
spacing
2023-01-25 19:21:45 -03:00
Arthur Passos
912ac6fb05
remove maybe_unused
2023-01-25 19:09:35 -03:00
Arthur Passos
9942fe7b5b
cancel ares failed requests, listen to POLLRDNORM and retry on EINTR
2023-01-25 19:07:58 -03:00
Alexander Tokmakov
3744fa2c63
format more messages
2023-01-25 21:16:42 +01:00
sichenzhao
243ac52259
Added two metrics about memory usage in cgroup to asynchronous metrics ( #45301 )
2023-01-25 20:32:17 +01:00
Suzy Wang
d2d2e3f31e
replace tab with spaces
2023-01-25 07:49:49 -08:00
Alexander Tokmakov
ae795d87b2
fix
2023-01-25 16:06:40 +01:00
Suzy Wang
fe47b43b6a
replace tab with space
2023-01-25 05:59:10 -08:00
Alexander Tokmakov
6eb557b2ba
Merge branch 'master' into exception_message_patterns4
2023-01-25 13:49:17 +01:00
Robert Schulze
59528cfca0
Merge pull request #45460 from ClickHouse/inv-index-cleanup
...
Cleanup of inverted index
2023-01-25 13:23:38 +01:00
Sergei Trifonov
0d1ea05ff6
Merge pull request #45007 from ClickHouse/cancellable-mutex-integration
...
Fast shared mutex integration
2023-01-25 11:15:46 +01:00
Alexander Tokmakov
d1baa7300c
reformat ParsingException
2023-01-24 23:21:29 +01:00
Suzy Wang
3abb00b2e1
remove trailing whitespace
2023-01-24 07:59:56 -08:00
serxa
51da43d6cf
fix try_shared_lock() in SharedMutex and CancelableSharedMutex
2023-01-24 14:36:07 +00:00
Robert Schulze
094e42d1dc
Fix style
2023-01-24 12:57:56 +00:00
Robert Schulze
9ff2bfcbf5
Merge remote-tracking branch 'origin/master' into inv-index-cleanup
...
src/Interpreters/GinFilter.cpp
src/Interpreters/InterpreterCreateQuery.cpp
src/Storages/MergeTree/MergeTreeData.cpp
src/Storages/MergeTree/MergeTreeDataPartWriterOnDisk.cpp
src/Storages/MergeTree/MergeTreeIndexInverted.cpp
2023-01-24 10:09:42 +00:00
Robert Schulze
36d6af0e04
Yet another build fix
2023-01-24 09:57:12 +00:00
Suzy Wang
1176260b3c
Merge branch 'master' into z-build-0120
2023-01-24 01:14:42 -05:00
Alexander Tokmakov
c6910f39b9
fix
2023-01-24 01:11:58 +01:00
Alexander Tokmakov
414693feb2
fixes
2023-01-24 00:46:03 +01:00
Alexander Tokmakov
bb4c8e169f
check number of parameters in format string
2023-01-23 23:16:16 +01:00
Alexander Tokmakov
3f6594f4c6
forbid old ctor of Exception
2023-01-23 22:18:05 +01:00
Alexander Tokmakov
70d1adfe4b
Better formatting for exception messages ( #45449 )
...
* save format string for NetException
* format exceptions
* format exceptions 2
* format exceptions 3
* format exceptions 4
* format exceptions 5
* format exceptions 6
* fix
* format exceptions 7
* format exceptions 8
* Update MergeTreeIndexGin.cpp
* Update AggregateFunctionMap.cpp
* Update AggregateFunctionMap.cpp
* fix
2023-01-24 00:13:58 +03:00
Alexey Milovidov
62a8de34cc
Merge pull request #44811 from azat/build/glibc2.36-fix
...
Fix ASan builds for glibc 2.36+
2023-01-23 23:57:20 +03:00
Robert Schulze
1a1fa4a40a
Fix ARM build
2023-01-23 19:48:09 +00:00
Robert Schulze
0ab7ae6c17
Merge pull request #43797 from ClickHouse/query-result-cache
...
Query result cache [experimental]
2023-01-23 19:54:35 +01:00
Kseniia Sumarokova
8ec39ab6f3
Merge pull request #45442 from kssenii/minor-changes
...
Minor improvements around reading from remote
2023-01-23 15:46:30 +01:00
Suzy Wang
fb46b5ed4d
s390x support
2023-01-23 14:20:49 +00:00
Sergei Trifonov
0fbfa17863
Merge branch 'master' into cancellable-mutex-integration
2023-01-23 12:44:09 +01:00
Kseniia Sumarokova
c3d545b64a
Merge pull request #45489 from azat/build/queue-fwd-decl
...
Forward declaration of ConcurrentBoundedQueue in ThreadStatus
2023-01-22 20:23:05 +01:00
Robert Schulze
d07b7ce1ee
Some fixups
2023-01-22 17:25:59 +00:00
Robert Schulze
d7416f45ff
Merge remote-tracking branch 'origin/master' into remove-smp-limitation
2023-01-22 16:22:54 +00:00
Robert Schulze
340f406553
Merge branch 'master' into query-result-cache
2023-01-22 13:21:36 +01:00
Robert Schulze
531e64dc39
Merge branch 'master' into inv-index-cleanup
2023-01-22 13:20:38 +01:00
Alexander Gololobov
91fb1eab8c
Merge pull request #45373 from ClickHouse/throttler-metrics
...
Add detailed profile events for throttling
2023-01-22 11:20:45 +01:00
Robert Schulze
97d1bed114
Merge branch 'master' into improve_week_day
2023-01-21 20:40:33 +01:00
Azat Khuzhin
2a8f116c18
Forward declaration of ConcurrentBoundedQueue in ThreadStatus
...
ThreadStatus is the header that recomplies almost all ClickHouse
modules.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-21 16:02:09 +01:00
Suzy Wang
19d26828a0
s390x build support
2023-01-20 21:16:55 +00:00
Alexander Tokmakov
910d6dc0ce
Merge pull request #45342 from ClickHouse/exception_message_patterns
...
Save message format strings for DB::Exception
2023-01-20 18:46:52 +03:00
Azat Khuzhin
bdeb5514c5
Fix ASan builds for glibc 2.36+ (use RTLD_NEXT for ThreadFuzzer interceptors)
...
Recently I noticed that clickhouse compiled with ASan does not work with
newer glibc 2.36+, before I though that this was only about compiling
with old but using new, however that was not correct, ASan simply does
not work with glibc 2.36+.
Here is a simple reproducer [1]:
$ cat > test-asan.cpp <<EOL
#include <pthread.h>
int main()
{
// something broken in ASan in interceptor for __pthread_mutex_lock
// and only since glibc 2.36, and for pthread_mutex_lock everything is OK
pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
return __pthread_mutex_lock(&mutex);
}
EOL
$ clang -g3 -o test-asan test-asan.cpp -fsanitize=address
$ ./test-asan
AddressSanitizer:DEADLYSIGNAL
=================================================================
==15659==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x000000000000 bp 0x7fffffffccb0 sp 0x7fffffffcb98 T0)
==15659==Hint: pc points to the zero page.
==15659==The signal is caused by a READ memory access.
==15659==Hint: address points to the zero page.
#0 0x0 (<unknown module>)
#1 0x7ffff7cda28f (/usr/lib/libc.so.6+0x2328f) (BuildId: 1e94beb079e278ac4f2c8bce1f53091548ea1584)
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV (<unknown module>)
==15659==ABORTING
[1]: https://gist.github.com/azat/af073e57a248e04488b21068643f079e
I've started observing glibc code, there was some changes in glibc, that
moves pthread functions out from libpthread.so.0 into libc.so.6
(somewhere between 2.31 and 2.35), but
the problem pops up only with 2.36, 2.35 works fine.
After this I've looked into changes between 2.35 and 2.36, and found
this patch [2] - "dlsym: Make RTLD_NEXT prefer default version
definition [BZ #14932 ]", that fixes this bug [3].
[2]: https://sourceware.org/git/?p=glibc.git;a=commit;h=efa7936e4c91b1c260d03614bb26858fbb8a0204
[3]: https://sourceware.org/bugzilla/show_bug.cgi?id=14932
The problem with using DL_LOOKUP_RETURN_NEWEST flag for RTLD_NEXT is
that it does not resolve hidden symbols (and __pthread_mutex_lock is
indeed hidden).
Here is a sample that will show the difference [4]:
$ cat > test-dlsym.c <<EOL
#define _GNU_SOURCE
#include <dlfcn.h>
#include <stdio.h>
int main()
{
void *p = dlsym(RTLD_NEXT, "__pthread_mutex_lock");
printf("__pthread_mutex_lock: %p (via RTLD_NEXT)\n", p);
return 0;
}
EOL
# glibc 2.35: __pthread_mutex_lock: 0x7ffff7e27f70 (via RTLD_NEXT)
# glibc 2.36: __pthread_mutex_lock: (nil) (via RTLD_NEXT)
[4]: https://gist.github.com/azat/3b5f2ae6011bef2ae86392cea7789eb7
But ThreadFuzzer uses internal symbols to wrap
pthread_mutex_lock/pthread_mutex_unlock, which are intercepted by ASan
and this leads to NULL dereference.
The fix was obvious - just use dlsym(RTLD_NEXT), however on older
glibc's this leads to endless recursion (see commits in the code). But
only for jemalloc [5], and even though sanitizers does not uses jemalloc
the code of ThreadFuzzer is generic and I don't want to guard it with
more preprocessors macros.
[5]: https://gist.github.com/azat/588d9c72c1e70fc13ebe113197883aa2
So we have to use RTLD_NEXT only for ASan.
There is also one more interesting issue, if you will compile with clang
that itself had been compiled with newer libc (i.e. 2.36), you will get
the following error:
$ podman run --privileged -v $PWD/.cmake-asan/programs:/root/bin -e PATH=/bin:/root/bin -e --rm -it ubuntu-dev-v3 clickhouse
==1==ERROR: AddressSanitizer failed to allocate 0x0 (0) bytes of SetAlternateSignalStack (error code: 22)
...
==1==End of process memory map.
AddressSanitizer: CHECK failed: sanitizer_common.cpp:53 "((0 && "unable to mmap")) != (0)" (0x0, 0x0) (tid=1)
<empty stack>
The problem is that since GLIBC_2.31, `SIGSTKSZ` is a call to
`getconf(_SC_MINSIGSTKSZ)`, but older glibc does not have it, so `-1`
will be returned and used as `SIGSTKSZ` instead.
The workaround to disable alternative stack:
$ podman run --privileged -v $PWD/.cmake-asan/programs:/root/bin -e PATH=/bin:/root/bin -e ASAN_OPTIONS=use_sigaltstack=0 --rm -it ubuntu-dev-v3 clickhouse client --version
ClickHouse client version 22.13.1.1.
Fixes : #43426
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-01-20 13:09:13 +01:00
Robert Schulze
f8b446f517
Move method implementations (no other changes)
2023-01-20 10:57:16 +00:00
Robert Schulze
0653f86de9
Various cosmetic cleanups
2023-01-20 10:45:35 +00:00
Han Fei
8b29f8406b
Merge branch 'master' into hanfei/gwp-asan
2023-01-20 11:03:12 +01:00
Frank Chen
f867c4c206
Add an overriden fromCurrentException to simply the code
2023-01-20 16:43:10 +08:00
Frank Chen
be8ceeb4bb
Save stack trace in part_log
2023-01-20 16:22:40 +08:00
Frank Chen
75191fd868
Catch exception raised by function object
2023-01-20 15:10:05 +08:00
Frank Chen
2fb49503da
Make addAttribute exception safe
2023-01-20 15:09:46 +08:00
kssenii
4ce8950712
Minor changes
2023-01-19 17:53:10 +01:00
larryluogit
52ae33dba7
Merge branch 'master' into ftsearch
2023-01-19 11:34:11 -05:00
Sergei Trifonov
c443c1ece0
Merge branch 'master' into hanfei/async-cache
2023-01-18 00:19:49 +01:00
Robert Schulze
4f90824347
Merge remote-tracking branch 'origin/master' into query-result-cache
2023-01-17 22:49:53 +00:00
Alexander Tokmakov
5cd90c1a3e
Merge branch 'master' into exception_message_patterns
2023-01-17 20:04:04 +01:00
Alexander Tokmakov
72e8615bec
formatting of some exception messages
2023-01-17 20:03:56 +01:00
serxa
ce7e22b87b
add detailed profile events for throttling
2023-01-17 18:29:24 +00:00
Alexander Tokmakov
8b13b85ea0
Merge pull request #44543 from ClickHouse/text_log_add_pattern
...
Add a column with a message pattern to text_log
2023-01-17 20:19:32 +03:00
Han Fei
8a74238fe0
improve
2023-01-17 15:47:52 +01:00
HarryLeeIBM
e7add8218f
Addressed more review comments and ClangTidy errors
2023-01-17 06:29:13 -08:00
Vitaly Baranov
b13498d9ba
Merge pull request #45288 from vitlibar/fix-s3-requests-without-region
...
Fix s3 requests without region
2023-01-17 09:24:59 +01:00
Alexander Tokmakov
522686f78b
less empty patterns
2023-01-17 01:19:44 +01:00
Alexander Tokmakov
870cfcc36a
less fmt::runtime usages
2023-01-17 00:11:59 +01:00
Alexander Tokmakov
e7899825e6
save format strings for DB::Exceptions
2023-01-16 23:20:33 +01:00
Nikolay Degterinsky
d414d4aa7a
Merge remote-tracking branch 'upstream/master' into bcrypt
2023-01-16 22:14:03 +00:00
Kseniia Sumarokova
d859976fbd
Merge pull request #45250 from ClickHouse/43188_Record_startup_time_in_profileevents
...
Record server startup time in ProfileEvents
2023-01-16 12:20:37 +01:00
Alexander Tokmakov
9ad6e1b129
Update logger_useful.h
2023-01-16 14:09:55 +03:00
Vitaly Baranov
e435edb4ab
Make checkObjectExists() easier.
2023-01-16 10:06:20 +01:00
Robert Schulze
099e30ef2a
Merge remote-tracking branch 'origin/master' into query-result-cache
2023-01-16 08:04:49 +00:00
Vitaly Baranov
21b8aaeb8b
Stop using HeadObject requests in S3
...
because they don't work well with endpoints without explicit region.
2023-01-15 20:28:11 +01:00
Robert Schulze
bd41c74ddf
Various test, code and docs fixups
2023-01-15 13:47:34 +00:00
Alexander Tokmakov
d857d62a03
remove another set of macros
2023-01-13 20:34:31 +01:00
Alexander Tokmakov
2d7773fccc
Merge branch 'master' into text_log_add_pattern
2023-01-13 20:33:46 +01:00
Han Fei
7feb9b7f6f
Merge branch 'master' into hanfei/gwp-asan
2023-01-13 20:26:34 +01:00
Han Fei
214830c569
enable gwp_asan by option
2023-01-13 15:22:14 +01:00
Robert Schulze
4ea836b87e
Revert "Revert "update function DAYOFWEEK and add new function WEEKDAY for mysql/spark compatiability""
...
This reverts commit e37f572c34
2023-01-13 14:00:16 +00:00
Smita Kulkarni
a0fe26f506
Addressed review comments and updated name to ServerStartupMilliseconds - Record server startup time in ProfileEvents
2023-01-13 14:38:54 +01:00
Smita Kulkarni
cf5cb0da97
Record server startup time in ProfileEvents
...
Implementation:
* Added ProfileEvents::ServerStartupTime.
* Recorded time from start of main till listening to sockets.
Testing:
* Added a test 02532_profileevents_server_startup_time.sql
2023-01-13 13:47:54 +01:00
Robert Schulze
9779d034eb
Merge pull request #45144 from ClibMouse/crc-power-fix
...
Changes to support the CRC32 in PowerPC.
2023-01-13 11:24:18 +01:00
serxa
693489a8ad
review fixes
2023-01-12 15:51:04 +00:00
Sergei Trifonov
12d8543578
Merge branch 'master' into cancellable-mutex-integration
2023-01-12 16:03:49 +01:00
Han Fei
f5e736e240
introduce llvm/gwp-asan
2023-01-12 15:00:26 +01:00
serxa
0841809a9c
fix special builds
2023-01-12 01:30:42 +00:00
MeenaRenganathan22
28d5c3cf7f
Addressed the review comments
2023-01-11 17:00:10 -08:00
HarryLeeIBM
6d576cf157
Added more documentation
2023-01-11 13:40:20 -08:00
MeenaRenganathan22
5fa5ae8099
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into crc-power-fix
2023-01-11 09:31:41 -08:00
Robert Schulze
9bb1e31369
Merge pull request #44860 from bigo-sg/improve_week_day
...
update function DAYOFWEEK and add new function WEEKDAY for mysql/spark compatiability
2023-01-11 13:58:45 +01:00
serxa
c0f529600d
fix darwin build
2023-01-11 12:23:33 +00:00
MeenaRenganathan22
25c94dfa83
Changes to support the CRC32 in PowerPC to address the WeakHash collision issue. Update the reference to support the hash values based on the specific platform
2023-01-10 21:20:13 -08:00
serxa
da4e9c9430
fix SharedMutex build
2023-01-10 16:35:46 +00:00
HarryLeeIBM
bc48b17817
Addressed more review comments.
2023-01-10 08:26:27 -08:00
Robert Schulze
0c3b034887
Merge remote-tracking branch 'origin/master' into query-result-cache
2023-01-10 09:02:41 +00:00
serxa
6f31c5094d
Merge branch 'cancellable-mutex' of github.com:ClickHouse/ClickHouse into cancellable-mutex
2023-01-10 01:27:11 +00:00
serxa
0f80ad6e06
make style-check happy
2023-01-10 01:26:59 +00:00
Sergei Trifonov
b4961a6253
Merge branch 'master' into cancellable-mutex
2023-01-10 02:20:42 +01:00
Robert Schulze
ad4884b394
Fix review comments, pt. I
2023-01-09 21:42:51 +00:00
serxa
ae56ac1b56
add TSA support
2023-01-09 16:46:07 +00:00
serxa
534db794c1
more review fixes
2023-01-09 15:05:41 +00:00
serxa
88c3c2946b
review fixes + split into files
2023-01-09 14:58:44 +00:00
taiyang-li
a8da7b4c20
change as request
2023-01-09 20:39:46 +08:00
taiyang-li
4bc2c61406
change as request
2023-01-09 20:37:16 +08:00
Kseniia Sumarokova
b22b51213f
Merge pull request #44868 from kssenii/improve-async-buffer-from-remote
...
Small perf improvement for async buffer from remote
2023-01-09 13:31:15 +01:00
taiyang-li
20c7c0b1ef
change as request
2023-01-09 18:21:31 +08:00
Sergei Trifonov
a4470dd1b2
Update src/Common/Threading.cpp
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2023-01-09 09:01:44 +01:00
Sergei Trifonov
81d2ea30ba
Merge branch 'master' into cancellable-mutex-integration
2023-01-07 19:37:46 +01:00
