Maksim Kita
f9326b00c8
PODArray assign empty array fix
2022-08-11 16:26:41 +02:00
Raúl Marín
11a274e990
Clean up constinit usage and add a comment
2022-08-11 13:27:53 +02:00
Nikolai Kochetov
971d94e492
Merge pull request #38682 from ClickHouse/separate-counter-for-rss-in-memory-tracker
...
Use separate counter for RSS in global memory tracker.
2022-08-11 10:11:18 +02:00
Duc Canh Le
84cd867aa8
materialize column instead of handling column in hash method
2022-08-11 10:46:06 +08:00
Alexey Milovidov
8374f31306
Merge pull request #39425 from arenadata/ADQM-485
...
Add support of dates from year 1900 to 2299 for Date32 and DateTime64
2022-08-11 05:01:53 +03:00
Raúl Marín
d7a545e30d
Try to optimize CurrentMemoryTracker alloc and free
2022-08-10 22:05:09 +02:00
Nikolai Kochetov
c9353cbae8
Merge branch 'master' into separate-counter-for-rss-in-memory-tracker
2022-08-10 21:39:50 +02:00
Alexey Milovidov
0b24dbaec8
Merge
2022-08-10 19:41:44 +02:00
KinderRiven
1aa7bbcbbd
fix unique_ptr
2022-08-11 00:11:41 +08:00
KinderRiven
9b7f87677d
fix
2022-08-11 00:11:41 +08:00
KinderRiven
9ba94e64f9
fix
2022-08-11 00:11:41 +08:00
KinderRiven
2ae02a4921
fix style
2022-08-11 00:11:40 +08:00
KinderRiven
76e0aad69e
fix
2022-08-11 00:11:40 +08:00
KinderRiven
fbaa70b313
fix
2022-08-11 00:11:40 +08:00
KinderRiven
f6a58bff4c
fix build
2022-08-11 00:11:40 +08:00
KinderRiven
d2b5581632
fix
2022-08-11 00:11:40 +08:00
KinderRiven
1b01cc8ed9
fix
2022-08-11 00:11:40 +08:00
KinderRiven
164fa1ab0e
fix build and style
2022-08-11 00:11:40 +08:00
KinderRiven
61b580aba4
add note
2022-08-11 00:11:40 +08:00
KinderRiven
081cd4938a
fix style
2022-08-11 00:11:40 +08:00
KinderRiven
9d83b93e88
fix rebase
2022-08-11 00:11:40 +08:00
KinderRiven
50fd740ec3
fix
2022-08-11 00:11:40 +08:00
KinderRiven
c5f9022510
fix
2022-08-11 00:11:40 +08:00
KinderRiven
43cf771657
better
2022-08-11 00:11:40 +08:00
KinderRiven
ffaf44c1c1
fix style
2022-08-11 00:11:40 +08:00
KinderRiven
7f54fa726b
Decoupling cache functions and algorithms
2022-08-11 00:11:40 +08:00
Kseniia Sumarokova
a4d513657c
Merge pull request #40010 from ClickHouse/catboost-further-preparation
...
Further preparation for catboost integration into library-bridge
2022-08-10 17:35:12 +02:00
kssenii
0ff5609738
Finish support for persistent files
2022-08-10 16:54:59 +02:00
Nikita Mikhaylov
8e8e845a89
Merge branch 'master' into separate-counter-for-rss-in-memory-tracker
2022-08-10 16:44:24 +02:00
Duc Canh Le
bf5e9173ad
fix hashMethodOneNumber constructor
2022-08-10 21:01:09 +08:00
Duc Canh Le
ec49f7b3a3
fix style
2022-08-10 19:28:08 +08:00
Duc Canh Le
17fa8076e3
better implementation
2022-08-10 18:59:13 +08:00
Duc Canh Le
7910a93b02
fix other hash methods and add tests
2022-08-10 18:59:13 +08:00
Duc Canh Le
1a08161dbf
simplify solution
2022-08-10 18:59:13 +08:00
Duc Canh Le
cdd5b32d29
fix HashMethodOneNumber with const column
2022-08-10 18:59:13 +08:00
kssenii
7b66b96ec5
Minor change
2022-08-10 12:20:54 +02:00
kssenii
5c3227ba56
Merge master
2022-08-10 12:00:34 +02:00
Antonio Andelic
4023d4a37a
Add support for drop/truncate and tests
2022-08-10 07:24:56 +00:00
Alexey Milovidov
33e43cc6f9
Merge branch 'master' into server-side-time
2022-08-10 07:28:15 +03:00
Robert Schulze
05ea33c897
Revert "Revert "ColumnVector: optimize filter with AVX512VBMI2 compress store""
...
This reverts commit 5524706b78
.
2022-08-09 17:00:13 +00:00
Kruglov Pavel
5ebf230dbb
Merge pull request #39426 from kitaisreal/shell-command-wait-pid-refactoring
...
ShellCommand wait pid refactoring
2022-08-09 11:48:20 +02:00
Roman Vasin
96598e3574
Replace normalizeDayNum() by std::clamp()
2022-08-09 08:52:02 +00:00
alexX512
21624708aa
Fix tidy build
2022-08-09 08:36:49 +00:00
alexX512
c9c26d4898
Fix review
2022-08-08 21:34:24 +00:00
Robert Schulze
e0d5020a92
Add simple versioning to the *-bridge-to-server protocol
...
- In general, it is expected that clickhouse-*-bridges and
clickhouse-server were build from the same source version (e.g. are
upgraded "atomically"). If that is not the case, we should at least
be able to detect the mismatch and abort.
- This commit adds a URL parameter "version", defined in a header shared
by the server and bridges. The bridge returns an error in case of
mismatch.
- The version is *not* send and checked for "ping" requests (used for
handshake), only for regular requests send after handshake. This is
because the internally thrown server-side exception due to HTTP
failure does not propagate the exact HTTP error (it only stores the
error as text), and as a result, the server-side handshake code
simply retries in case of error with exponential backoff and finally
fails with a "timeout error". This is reasonable as pings typically
fail due to time out. However, without a rework of HTTP exceptions,
version mismatch during ping would also appear as "timeout" which is
too misleading. The behavior may be changed later if needed.
- Note that introducing a version parameter does not represent a
protocol upgrade itself. Bridges older than the server will simply
ignore the field. Only servers older than the bridges receive an error
but such a situation should never occur in practice.
2022-08-08 19:40:37 +00:00
Alexey Milovidov
db85ebfaa7
Add server-side time to Progress
2022-08-08 06:55:41 +02:00
Alexey Milovidov
5524706b78
Revert "ColumnVector: optimize filter with AVX512VBMI2 compress store"
2022-08-08 05:03:15 +03:00
alexX512
f16c668a0c
Delete Args from CacheBase
2022-08-07 20:44:26 +00:00
alexX512
1c9d40e4a8
Fix build error
2022-08-07 20:37:11 +00:00
alexX512
269631cfbe
Fix typo
2022-08-07 20:02:32 +00:00
alexX512
62f84b88eb
Review fixes
2022-08-07 20:02:21 +00:00
alexX512
8adf9f2448
fix compile error
2022-08-07 19:59:39 +00:00
alexX512
2be72560f4
Add loading of mark cache policy
2022-08-07 19:59:39 +00:00
alexX512
a057e1607a
Fix remove operation, add evict ion i from cachef number of elements greater than max_elements_size
2022-08-07 19:59:39 +00:00
alexX512
8be62770aa
Test with SLRU and debug logs
2022-08-07 19:59:39 +00:00
alexX512
c044fee7a0
Test with LRU
2022-08-07 19:59:39 +00:00
alexX512
a238e349cd
Unblocking get max_size
2022-08-07 19:59:39 +00:00
alexX512
daab74925c
Test with default cache policy = SLRU
2022-08-07 19:59:39 +00:00
alexX512
994a9fc005
Minor fixes
2022-08-07 19:59:38 +00:00
alexX512
50a8887ab9
Add [[maybe_unused]] locks and delete all objects from cache queue
2022-08-07 19:59:38 +00:00
alexX512
69cc89e36e
Check with default cache policy = LRU
2022-08-07 19:59:38 +00:00
alexX512
a94950d3ac
Test only with LRU
2022-08-07 19:59:38 +00:00
alexX512
9940b69bd7
Include fix in ICachePolicy.h
2022-08-07 19:59:38 +00:00
alexX512
910645a52a
Include fix in SLRUCachePolicy.h
2022-08-07 19:59:38 +00:00
alexX512
4f52efd4bb
Build fix
2022-08-07 19:59:38 +00:00
alexX512
6bf29cb610
Change class LRUCache to class CachBase. Check running CacheBase with default pcahce policy SLRU
2022-08-07 19:59:30 +00:00
alexX512
c606ccc75d
Fix formatting problems
2022-08-07 19:16:40 +00:00
alexX512
1133e42367
Add CacheBase clas instead of LRUCache and SLRUCache fo simpler configuration oache policiesf
2022-08-07 19:16:40 +00:00
alexX512
a0d2da7261
Add tests for SLRU cache
2022-08-07 19:16:40 +00:00
AlPerevyshin
d7bdf9e601
Add SLRU cache
2022-08-07 19:16:38 +00:00
avogar
9b1a267203
Refactor, remove TTL, add size limit, add system table and system query
2022-08-05 16:20:15 +00:00
kssenii
91f4143b6e
Merge master
2022-08-05 12:14:08 +02:00
Alexey Milovidov
3130aa7218
Merge pull request #39732 from HarryLeeIBM/hlee-s390x-unaligned
...
Fix Endian issue in SipHash for s390x
2022-08-05 07:24:57 +03:00
kssenii
7a9b0bc47f
Merge master
2022-08-05 01:48:52 +02:00
Alexey Milovidov
24ee944986
Update TaskStatsInfoGetter.cpp
2022-08-04 07:34:36 +03:00
Fangyuan Deng
0bf5c3d1a1
Merge branch 'master' into fix_netlink_leak
2022-08-04 11:42:17 +08:00
pzhdfy
848a349e42
fix potential netlink leak when init fail in TaskStatsInfoGetter
2022-08-04 11:37:27 +08:00
Alexey Milovidov
f474eb957d
Merge pull request #39758 from ClickHouse/tsan_clang_15
...
Try clang-15 for build with tsan
2022-08-04 01:56:52 +03:00
Nikita Mikhaylov
f144eae388
Fix typo and extra dots in exception messages from OverCommitTracker ( #39858 )
2022-08-03 15:23:07 +02:00
Robert Schulze
00a7c8733b
Merge pull request #39633 from guowangy/filter-vbmi2
...
ColumnVector: optimize filter with AVX512VBMI2 compress store
2022-08-03 09:23:24 +02:00
Frank Chen
035896fa16
Fix build check
2022-08-02 20:18:42 +08:00
Frank Chen
1c9b462c80
Merge remote-tracking branch 'origin/master' into tracing_context_propagation
2022-08-02 20:18:22 +08:00
Alexander Tokmakov
b4c4448f0f
fix
2022-08-02 12:07:38 +02:00
Alexander Tokmakov
82b50e79cf
Merge branch 'master' into tsan_clang_15
2022-08-02 13:00:55 +03:00
Frank Chen
57dde8c250
Clean up header inclusion
...
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2022-08-02 15:43:19 +08:00
Alexey Milovidov
316528817b
Merge pull request #39179 from azat/fsync-profile-events
...
Add profile events for fsync
2022-08-02 05:44:35 +03:00
Wangyang Guo
6a7213291b
Merge master and resolve conflict
2022-08-02 10:40:40 +08:00
Frank Chen
40c6e4c0d6
Merge remote-tracking branch 'origin/master' into tracing_context_propagation
2022-08-02 10:02:09 +08:00
Alexander Tokmakov
ecf7ce1f74
Merge branch 'master' into zookeeper_client_fault_injection
2022-08-01 20:49:01 +02:00
Alexander Tokmakov
0d68b1c67f
fix build with clang-15
2022-08-01 18:00:54 +02:00
Robert Schulze
bf574b9154
Merge pull request #39760 from ClickHouse/bit-fiddling
...
Use std::popcount, ::countl_zero, ::countr_zero functions
2022-08-01 17:04:51 +02:00
Alexander Tokmakov
80f9ba9186
Merge pull request #39690 from ClickHouse/show-addresses-in-stack-traces
...
Configuration parameter to hide addresses in stack traces
2022-08-01 14:20:37 +03:00
Wangyang Guo
b05be56eef
ColumnVector: naming style fix
2022-08-01 10:16:15 +08:00
Robert Schulze
d39259a4c0
More conversions
2022-07-31 22:06:02 +00:00
Azat Khuzhin
3e627e2861
Add profile events for fsync
...
The following new provile events had been added:
- FileSync - Number of times the F_FULLFSYNC/fsync/fdatasync function was called for files.
- DirectorySync - Number of times the F_FULLFSYNC/fsync/fdatasync function was called for directories.
- FileSyncElapsedMicroseconds - Total time spent waiting for F_FULLFSYNC/fsync/fdatasync syscall for files.
- DirectorySyncElapsedMicroseconds - Total time spent waiting for F_FULLFSYNC/fsync/fdatasync syscall for directories.
v2: rewrite test to sh with retries
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-31 23:19:30 +03:00
kssenii
e5f4a619ed
Merge master
2022-07-31 20:24:40 +03:00
Robert Schulze
a7734672b9
Use std::popcount, ::countl_zero, ::countr_zero functions
...
- Introduced with the C++20 <bit> header
- The problem with __builtin_c(l|t)z() is that 0 as input has an
undefined result (*) and the code did not always check. The std::
versions do not have this issue.
- In some cases, we continue to use buildin_c(l|t)z(), (e.g. in
src/Common/BitHelpers.h) because the std:: versions only accept
unsigned inputs (and they also check that) and the casting would be
ugly.
(*) https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html
2022-07-31 15:16:51 +00:00
Robert Schulze
52d08d9db4
Merge pull request #39520 from ClickHouse/no-split-binary
...
Remove SPLIT_BINARY
2022-07-31 14:23:31 +02:00
Alexey Milovidov
d285648a37
Merge branch 'master' into shell-command-wait-pid-refactoring
2022-07-31 02:57:46 +03:00
Robert Schulze
85773e0926
Merge pull request #38615 from liyinsg/simplified_function_registration_interface
...
Simplified function registration interface
2022-07-31 00:18:37 +02:00
Alexey Milovidov
c0d7b6efc3
Suggestions from @tavplubix
2022-07-30 01:45:06 +02:00
Alexey Milovidov
4aa96767d5
Merge branch 'master' of github.com:ClickHouse/ClickHouse into show-addresses-in-stack-traces
2022-07-30 01:44:04 +02:00
HarryLeeIBM
cc73b53116
Fix Endian issue in SipHash for s390x
2022-07-29 13:21:50 -07:00
Robert Schulze
3d1797f75f
Merge remote-tracking branch 'origin/master' into no-split-binary
2022-07-29 12:17:43 +00:00
Robert Schulze
a3c52f1140
Merge pull request #39679 from azat/fix-utils
...
Fix utils build on CI
2022-07-29 14:02:53 +02:00
Azat Khuzhin
498c8b3c52
Fix clang-tidy in utils/examples
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-29 11:36:51 +03:00
Li Yin
4088c0a7f3
Automated function registration
...
Automated register all functions with below naming convention by
iterating through the symbols:
void DB::registerXXX(DB::FunctionFactory &)
2022-07-29 15:39:50 +08:00
Alexey Milovidov
552b517e59
Allow to hide addresses in stack traces
2022-07-29 00:13:02 +02:00
Azat Khuzhin
e1f8cff3b4
Add duration_ms into system.zookeeper_log
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-28 22:34:51 +03:00
Robert Schulze
199e254777
Merge remote-tracking branch 'origin/master' into no-split-binary
2022-07-28 15:54:22 +00:00
Antonio Andelic
15e999f8bc
Merge pull request #39096 from ClickHouse/keeper-version-check
...
Add version for Keeper API
2022-07-28 10:10:49 +02:00
Sergei Trifonov
edf6b02723
Merge pull request #37558 from ClickHouse/concurrency-control
...
add concurrency control for pipeline executors
2022-07-28 08:56:24 +02:00
Yakov Olkhovskiy
f1bc767ccf
Merge pull request #39627 from HarryLeeIBM/hlee-s390x-wide_int
...
Fix wide integer unit tests on s390x platform
2022-07-27 21:30:12 -04:00
Alexander Tokmakov
8fc075a527
Merge pull request #39619 from ClickHouse/try-do-enable-global-allocations-for-attach
...
Replace MemoryTrackerBlockerInThread to LockMemoryExceptionInThread
2022-07-27 13:13:04 +03:00
Kruglov Pavel
381ea139c2
Merge branch 'master' into schema-inference-cache
2022-07-27 11:35:36 +02:00
Antonio Andelic
904a05ac21
Merge pull request #39496 from azat/custom-tld-exclamation-asterisk
...
Add support of !/* (exclamation/asterisk) in custom TLDs
2022-07-27 08:55:49 +02:00
Wangyang Guo
e6752d687f
TargetSpecific: add AVX512VBMI2 support
2022-07-27 13:30:43 +08:00
Wangyang Guo
50fdbcdc12
CpuId: add AVX512VBMI2 detection
2022-07-27 13:30:43 +08:00
HarryLeeIBM
d475086100
Fix wide integer unit test
2022-07-26 12:57:45 -07:00
Nikolai Kochetov
f1818a9103
Make only one possible public ctor in MemoryTrackerBlockerInThread
2022-07-26 19:04:20 +00:00
Nikolai Kochetov
dcb86eca0b
Make only one possible public ctor in MemoryTrackerBlockerInThread
2022-07-26 19:03:26 +00:00
Nikolai Kochetov
fa0707b4e7
Review fixes.
2022-07-26 17:51:09 +00:00
Nikolai Kochetov
33ce662d3e
Replace MemoryTrackerBlockerInThread to LockMemoryExceptionInThread in some places. Reduced MemoryTrackerBlockerInThread level to User.
2022-07-26 15:22:00 +00:00
Kruglov Pavel
c683cb252f
Merge pull request #39227 from amosbird/rename-log1
...
Rename log when rename merge tree tables
2022-07-26 17:12:44 +02:00
Sergei Trifonov
24ab5fbb86
fix finish() condition to account only active threads
2022-07-26 11:32:45 +02:00
Antonio Andelic
35b61cc94e
Merge branch 'master' into keeper-version-check
2022-07-26 07:47:57 +00:00
Azat Khuzhin
1d4a7c7290
Add support of !/* (exclamation/asterisk) in custom TLDs
...
Public suffix list may contain special characters (you may find format
here - [1]):
- asterisk (*)
- exclamation mark (!)
[1]: https://github.com/publicsuffix/list/wiki/Format
It is easier to describe how it should be interpreted with an examples.
Consider the following part of the list:
*.sch.uk
*.kawasaki.jp
!city.kawasaki.jp
And here are the results for `cutToFirstSignificantSubdomainCustom()`:
If you have only asterisk (*):
foo.something.sheffield.sch.uk -> something.sheffield.sch.uk
sheffield.sch.uk -> sheffield.sch.uk
If you have exclamation mark (!) too:
foo.kawasaki.jp -> foo.kawasaki.jp
foo.foo.kawasaki.jp -> foo.foo.kawasaki.jp
city.kawasaki.jp -> city.kawasaki.jp
some.city.kawasaki.jp -> city.kawasaki.jp
TLDs had been verified wit the following script [2], to match with
python publicsuffix2 module.
[2]: https://gist.github.com/azat/c1a7a9f1e3519793134ef4b1df5461a6
v2: fix StringHashTable padding requirements
Fixes : #39468
Follow-up for: #17748
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-26 08:34:30 +03:00
Roman Vasin
b462366415
Merge branch 'master' of github.com:ClickHouse/ClickHouse into ADQM-485
2022-07-25 17:55:47 +00:00
Roman Vasin
b412ea5f6d
Improve generateRandom() for Date32; fix tests 01087_table_function_generate, 01277_fromUnixTimestamp64, 01691_DateTime64_clamp and 01702_toDateTime_from_string_clamping
2022-07-25 17:06:11 +00:00
Sergei Trifonov
d9d1381c3f
Merge branch 'master' into concurrency-control
2022-07-25 14:16:46 +02:00
mergify[bot]
f667eff025
Merge branch 'master' into shell-command-wait-pid-refactoring
2022-07-25 11:42:05 +00:00
Robert Schulze
24c74da473
Merge pull request #39430 from azat/getauxval-fix
...
Fix LSan by fixing getauxval() (resubmit v4)
2022-07-25 10:43:09 +02:00
Alexey Milovidov
388d06fda1
Merge pull request #39535 from ClickHouse/stringref
...
Less usage of StringRef
2022-07-25 04:06:11 +03:00
Azat Khuzhin
93f876b58a
Add a test for LSan
...
(cherry picked from commit 51e7c41883
)
v2: fix type check
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-25 01:21:40 +03:00
Robert Schulze
4333750985
Less usage of StringRef
...
... replaced by std::string_view, see #39262
2022-07-24 18:33:52 +00:00
Robert Schulze
73c04b64e3
Merge pull request #39493 from nicelulu/fix_build_osx
...
Fix build on mac osx
2022-07-24 18:11:34 +02:00
Robert Schulze
c788e05c77
Merge pull request #39292 from zvonand/zvonand-b58-datatype
...
Simplify Base58 encoding/decoding
2022-07-24 18:09:40 +02:00
Andrey Zvonov
032fcab70e
set char to be explicitely signed
2022-07-24 14:23:22 +02:00
Alexey Milovidov
071374b152
Remove SPLIT_BINARY
2022-07-24 01:15:54 +02:00
Constantine Peresypkin
d645d03e89
fix -DENABLE_EXAMPLES=1
in master
2022-07-23 19:00:52 +02:00
Antonio Andelic
6aff87d4b5
Ignore system paths from snapshots and logstore
2022-07-23 13:46:59 +00:00
Antonio Andelic
46a6fbd7aa
Small polish
2022-07-22 11:03:07 +00:00
zhangxiao871
0d4d1e66b5
Fix build
2022-07-22 18:53:44 +08:00
Alexander Tokmakov
bed2206ae9
Merge pull request #39460 from ClickHouse/remove_some_dead_and_commented_code
...
Remove some dead and commented code
2022-07-22 13:24:34 +03:00
Roman Vasin
0b102c6d1f
Fix code style
2022-07-22 08:24:05 +00:00
Antonio Andelic
3040ff0959
Merge branch 'master' into keeper-version-check
2022-07-22 08:07:55 +00:00
Roman Vasin
74d2bf8d95
Fix gTest for DateLUTTest
2022-07-22 07:49:02 +00:00
Kruglov Pavel
9252f42b4c
Merge branch 'master' into schema-inference-cache
2022-07-21 18:59:14 +02:00
Robert Schulze
ea0a3bf600
Merge branch 'master' into stringref-to-string_view
2022-07-21 18:33:06 +02:00
Maksim Kita
0159fe1274
Merge branch 'master' into shell-command-wait-pid-refactoring
2022-07-21 16:38:28 +02:00
Andrey Zvonov
e473606cd1
Merge branch 'master' into zvonand-b58-datatype
2022-07-21 15:55:05 +02:00
Alexander Tokmakov
9e9969cea7
Merge pull request #37827 from arthurpassos/host_regexp_multiple_domains
...
Test host_regexp against all PTR records instead of only one
2022-07-21 16:43:05 +03:00
Alexander Tokmakov
a8da5d96fc
remove some dead and commented code
2022-07-21 15:05:48 +02:00
Maksim Kita
7f4a1b8bb8
ShellCommand wait pid refactoring
2022-07-21 11:45:09 +02:00
Nikolai Kochetov
e15967e9db
Merge pull request #38475 from ClickHouse/additional-filters
...
Additional filters for a table (from setting)
2022-07-21 07:52:04 +02:00
Alexey Milovidov
4305eacd92
Merge pull request #39434 from ClickHouse/revert-39299-fix-lsan-v3
...
Revert "[RFC] Fix LSan by fixing getauxval()"
2022-07-21 06:47:54 +03:00
Alexey Milovidov
f4f5a5e044
Merge pull request #39397 from azat/fix-extractAll
...
Fix UB (stack-use-after-scope) in extactAll()
2022-07-21 05:21:18 +03:00
Alexey Milovidov
72dda35108
Revert "[RFC] Fix LSan by fixing getauxval()"
2022-07-21 04:34:04 +03:00
Nikolai Kochetov
91043351aa
Fixing build.
2022-07-20 20:30:16 +00:00
Kseniia Sumarokova
3759ee76a7
Merge pull request #39386 from ClickHouse/kssenii-patch-4
...
Rethrow exception on cache initialisation on server startup, better error message
2022-07-20 21:26:25 +02:00
Roman Vasin
e3192cf753
Correct docs to reflect new range 1900..2299 for Date32 and DateTime64; Cleanup code
2022-07-20 15:19:02 +00:00
zvonand
11b8d788ca
small improvements
2022-07-20 13:36:47 +02:00
Kruglov Pavel
3046cd6d29
Merge branch 'master' into schema-inference-cache
2022-07-20 13:30:42 +02:00
Kseniia Sumarokova
0840dfe2e0
Update LRUFileCache.cpp
2022-07-20 11:42:24 +02:00
Maksim Kita
d3ce4ffc88
Merge pull request #38929 from pkit/pkit/wait_pid_fix
...
fix long wait for process exit in ShellCommand
2022-07-20 11:26:38 +02:00
Azat Khuzhin
18eb8b6d48
Fix UB (stack-use-after-scope) in extactAll()
...
After #37544 OptimizedRegularExpressionImpl started to be moved, but
StringSearcher is not copyable since it holds pointers that goes out of
scope after move (before Regexps::get() returns std::shared_ptr<Regexp>
but it had been replaced with Regexps::createRegexp() that returns
Regexp object).
<details>
<summary>ASan report</summary>
==48348==ERROR: AddressSanitizer: stack-use-after-scope on address 0x7fff577239a9 at pc 0x00001518209b bp 0x7fff57723820 sp 0x7fff57723818
READ of size 1 at 0x7fff577239a9 thread T0
0 0x1518209a in char8_t const* DB::StringSearcher<true, true>::search<char8_t>(char8_t const*, char8_t const*) const /bld/./src/Common/StringSearcher.h:730:41
1 0x1518dd3f in char8_t const* DB::StringSearcher<true, true>::search<char8_t>(char8_t const*, unsigned long) const /bld/./src/Common/StringSearcher.h:751:16
2 0x1518dd3f in OptimizedRegularExpressionImpl<false>::match(char const*, unsigned long, std::__1::vector<OptimizedRegularExpressionDetails::Match, std::__1::allocator<OptimizedRegularExpressionDetails::Match> >&, unsigned int) const /bld/./src/Common/OptimizedRegularExpression.cpp:463:54
3 0x1811cb42 in DB::ExtractAllImpl::get(char const*&, char const*&) /bld/./src/Functions/FunctionsStringArray.h:588:18
4 0x1811aa62 in DB::FunctionTokens<DB::ExtractAllImpl>::executeImpl(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long) const /bld/./src/Functions/FunctionsStringArray.h:704:30
5 0x14fe17b4 in DB::IFunction::executeImplDryRun(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long) const /bld/./src/Functions/IFunction.h:409:16
Address 0x7fff577239a9 is located in stack of thread T0 at offset 201 in frame
0 0x1518d98f in OptimizedRegularExpressionImpl<false>::match(char const*, unsigned long, std::__1::vector<OptimizedRegularExpressionDetails::Match, std::__1::allocator<OptimizedRegularExpressionDetails::Match> >&, unsigned int) const /bld/./src/Common/OptimizedRegularExpression.cpp:439
</details>
CI: https://s3.amazonaws.com/clickhouse-test-reports/39342/c6f7698f9ad6ae22199182ebf7c3b2dac77d69d8/fuzzer_astfuzzerasan,actions//report.html
Fixes : #37544 (cc @rschu1ze)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-20 10:18:44 +03:00
Roman Vasin
5f9c293963
Fix addDays() and addWeeks() in upper and lower limits of Date and Date32
2022-07-19 17:29:08 +00:00
Kseniia Sumarokova
891dbccdc7
Update LRUFileCache.cpp
2022-07-19 19:18:42 +02:00
Robert Schulze
02ed126533
Update src/Common/TraceSender.cpp
...
Co-authored-by: Nikolay Degterinsky <43110995+evillique@users.noreply.github.com>
2022-07-19 18:44:31 +02:00
Sergei Trifonov
499818751e
Merge pull request #39280 from ClickHouse/avg-cpu-progress
...
fix cpu usage metric in client
2022-07-19 15:17:44 +02:00
Kseniia Sumarokova
2e3ceb351e
Update LRUFileCache.cpp
2022-07-19 14:44:56 +02:00
Kseniia Sumarokova
f7bb69ef7a
Update LRUFileCache.cpp
2022-07-19 14:43:11 +02:00
Kseniia Sumarokova
af7acb1782
Update LRUFileCache.cpp
2022-07-19 14:41:31 +02:00
Kseniia Sumarokova
5ad4e9d28f
Update LRUFileCache.cpp
2022-07-19 14:33:50 +02:00
Robert Schulze
7146685f9d
Merge pull request #39299 from azat/fix-lsan-v3
...
[RFC] Fix LSan by fixing getauxval()
2022-07-19 14:27:41 +02:00
Constantine Peresypkin
246614229f
fix long wait for process exit in ShellCommand
...
fixes #38889
2022-07-19 14:07:27 +02:00
kssenii
21ebf8874e
Merge remote-tracking branch 'upstream/master' into make-cache-composable
2022-07-19 13:36:42 +02:00
Nikolai Kochetov
fc6982f9e2
Fixing test.
2022-07-19 10:10:02 +00:00
Antonio Andelic
ce570b6ee3
Add logs and 4LW for api version
2022-07-19 09:02:57 +00:00
Antonio Andelic
e6ded88ea3
Small refactoring
2022-07-19 08:51:12 +00:00
Robert Schulze
81ef1099cc
Even less usage of StringRef
...
--> see #39300
2022-07-19 07:01:06 +00:00
Nikolai Kochetov
5a3cb07711
Fix rss
2022-07-18 21:24:41 +00:00
Nikolai Kochetov
3ee3144708
Merge branch 'master' into separate-counter-for-rss-in-memory-tracker
2022-07-18 23:20:38 +02:00
Nikolai Kochetov
87e5b31598
Do not include memory buffered by allocator into drift.
2022-07-18 21:16:16 +00:00
Robert Schulze
6df3c9d799
Merge pull request #39300 from ClickHouse/stringref-to-stringview
...
First try at reducing the use of StringRef
2022-07-18 19:20:16 +02:00
Roman Vasin
28fd774df8
Correct 01821_to_date_time_ubsan and 01921_datatype_date32 tests
2022-07-18 16:44:14 +00:00
Antonio Andelic
cfc741030f
Revert to the version with path
2022-07-18 14:30:15 +00:00
kssenii
0aed5a602a
Merge master
2022-07-18 16:16:28 +02:00
Azat Khuzhin
51e7c41883
Add a test for LSan
...
v2: fix type check
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-18 16:49:49 +03:00
Sergei Trifonov
30cb4e168e
cleanup
2022-07-18 13:31:32 +02:00
Sergei Trifonov
0ffe5bd39c
fix typo and style
2022-07-18 13:26:33 +02:00
Sergei Trifonov
2593dbcaa6
reuse Common/ExponentiallySmoothedCounter.h
2022-07-18 12:21:44 +02:00
Antonio Andelic
758a61a36f
Close socket if ApiVersion wasn't processed
2022-07-18 09:43:21 +00:00
Antonio Andelic
5170ac0898
Merge branch 'master' into keeper-version-check
2022-07-18 08:57:50 +00:00
Antonio Andelic
b8ffb151dc
Some small polishing
2022-07-18 08:52:52 +00:00
Antonio Andelic
742e3d7643
Merge branch 'master' into keeper-version-check
2022-07-18 08:45:00 +00:00
Antonio Andelic
7d7c4ce4cd
Add ApiVersion request
2022-07-18 08:44:38 +00:00
Alexey Milovidov
f1256c3aef
Merge pull request #37891 from guowangy/lz4-decompress-vbmi
...
LZ4 decompress: add VBMI optimized copyOverlap32Shuffle
2022-07-18 00:37:59 +03:00
Robert Schulze
13482af4ee
First try at reducing the use of StringRef
...
- to be replaced by std::string_view
- suggested in #39262
2022-07-17 17:26:02 +00:00
Robert Schulze
fffeab2e86
Merge pull request #39262 from ClickHouse/stringref-by-value
...
Pass const StringRef by value, not by reference
2022-07-17 16:53:15 +02:00
kssenii
3ccbf5ced0
Review fixes
2022-07-17 15:13:28 +02:00
Wangyang Guo
c07c65615d
fix ENABLE_MULTITARGET_CODE not defined warning in some builds
2022-07-17 12:21:10 +08:00
kssenii
ca09c6c2c0
Fix some review comments, fix after merge
2022-07-16 23:30:54 +02:00
kssenii
8ad26b3111
Merge master
2022-07-16 22:50:03 +02:00
zvonand
3089da16be
fix identation
2022-07-16 19:07:42 +02:00
zvonand
d07a652883
merge
2022-07-16 19:05:07 +02:00
zvonand
4ab52b6873
added new DataType + fixes
2022-07-16 18:58:47 +02:00
Alexey Milovidov
e37caa5bd6
Merge pull request #39216 from Algunenano/rotational_error
...
Don't report system.errors when the disk is not rotational
2022-07-16 16:16:52 +03:00
Sergei Trifonov
82f98cfe6b
remove initial linear growth, increase avg period to 3 sec
2022-07-16 02:17:18 +02:00
Sergei Trifonov
8a5286ea4c
fix cpu usage metric in client
2022-07-16 00:27:26 +02:00
Arthur Passos
abb550d2ff
minor style fixes
2022-07-15 12:42:07 -03:00
Alexander Tokmakov
5ef7f5f494
Merge pull request #39229 from azat/fix-noexcept-scope
...
[RFC] Fix NOEXCEPT_SCOPE (before it calls std::terminate and looses the exception)
2022-07-15 18:14:56 +03:00
Robert Schulze
deda29b46b
Pass const StringRef by value, not by reference
...
See #39224
2022-07-15 11:34:56 +00:00
Kruglov Pavel
a944a92d4c
Merge pull request #39224 from Avogar/string-view-by-value
...
Pass const std::string_view by value, not by reference
2022-07-15 13:05:56 +02:00
Roman Vasin
1d0818d9cf
Set max year to 2299; Code cleanup; Make working 02245_make_datetime64 test
2022-07-15 10:33:52 +00:00
Antonio Andelic
73e0c35ab0
Use 4LW for api version
2022-07-15 10:13:39 +00:00
Robert Schulze
8e44c2c375
Merge pull request #39223 from ClickHouse/warnings
...
Turn some warnings on
2022-07-15 10:06:59 +02:00
Wangyang Guo
397a4ac888
TargetSpecific: add AVX512VBMI support
2022-07-15 15:07:09 +08:00
Sergei Trifonov
62f975da60
Merge pull request #39095 from ClickHouse/remote-io-bandwidth-limits
...
add max_remote_{read,write}_network_bandwidth_for_server settings
2022-07-14 23:14:53 +02:00
Roman Vasin
266039ea64
Correct gTests for DateLUT
2022-07-14 19:00:17 +00:00
Raúl Marín
3fd327b23a
Use std::filesystem::path::operator/
2022-07-14 19:25:40 +02:00
avogar
9291d33080
Pass const std::string_view & by value, not by reference
2022-07-14 16:11:57 +00:00
Azat Khuzhin
b20407fab9
Fix NOEXCEPT_SCOPE (before it calls std::terminate and looses the exception)
...
Current implementation of NOEXCEPT_SCOPE will not work, you cannot
rethrow exception outside the catch block, this will simply terminate
(via std::terminate) the program.
In other words NOEXCEPT_SCOPE macro will simply call std::terminate on
exception and will lost original exception.
But if NOEXCEPT_SCOPE will accept the code that should be runned w/o
exceptions, then it can catch exception and log it, rewrite it in this
way.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-14 17:16:18 +03:00
Amos Bird
96bb6e0cd2
Rename log when rename merge tree tables
2022-07-14 21:22:46 +08:00
Robert Schulze
3d734a0640
Enable -Wnested-anon-types
2022-07-14 12:36:55 +00:00
Robert Schulze
add83f7836
Merge pull request #39213 from ClickHouse/enable-wdeprecated-dynamic-exception-spec
...
Enable warning "-Wdeprecated-dynamic-exception-spec"
2022-07-14 13:09:18 +02:00
Sergei Trifonov
8a693469f0
Merge branch 'master' into remote-io-bandwidth-limits
2022-07-14 12:38:35 +02:00
Kruglov Pavel
b38241b08a
Merge branch 'master' into schema-inference-cache
2022-07-14 12:29:54 +02:00
Raúl Marín
35d3a0dd1f
Don't report system.errors when the disk is not rotational
2022-07-14 11:08:26 +02:00
Antonio Andelic
0e25dbbbeb
Fix stateless test
2022-07-14 08:01:20 +00:00
Robert Schulze
110862a11e
Merge pull request #38502 from ClibMouse/Issue_38498
...
Fixed bug in parsing {..} regex
2022-07-14 08:27:31 +02:00
Nikolai Kochetov
812143c76b
Merge pull request #39160 from azat/threadpool-fix
...
ThreadPool fixes
2022-07-13 22:54:57 +02:00
Arthur Passos
7a09aa478a
Add missing include header for osx builds
2022-07-13 17:45:30 -03:00
kssenii
615221f73b
Minor changes
2022-07-13 19:22:47 +02:00
Sergei Trifonov
f854507729
merge master and resolve conflicts
2022-07-13 19:07:28 +02:00
Heena Bansal
76cbfedd2a
Merge branch 'master' into Issue_38498
2022-07-13 11:50:26 -04:00
Antonio Andelic
f05f22aa13
Merge branch 'master' into keeper-version-check
2022-07-13 15:25:48 +00:00
Kseniia Sumarokova
71aba5b4c6
Merge pull request #38860 from kssenii/object-storages-refactor
...
Refactoring of code around object storages, added LocalObjectStorage (extracted this diff from PR #36171 )
2022-07-13 16:51:56 +02:00
Arthur Passos
7f4043a3ed
Force use of c-ares inet_net_pton and style adjustments
2022-07-13 11:51:43 -03:00
kssenii
0944183eec
Fix merge
2022-07-13 16:50:31 +02:00
Robert Schulze
87197f8f60
Enable -Wdeprecated-dynamic-exception-spec
2022-07-13 13:49:40 +00:00
Antonio Andelic
93d8b8bb61
small fixes
2022-07-13 13:01:13 +00:00
Azat Khuzhin
68789895a4
ThreadPool: incapsulate Poco::Event and std: 🧵 :id into State struct
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-13 15:51:49 +03:00
Azat Khuzhin
7ffa15b94b
ThreadPool: fix thread_id assignment
...
As found by @KochetovNicolai before this patch, lambda in
ThreadFromGlobalPool() ctor assigns value only to a copy of the
thread_id value, and so check in joinable() had been working
incorrectly, fix this by changing the value not the shared_ptr itself.
Also it is not safe to assign thread_id w/o atomics, since this can be
racy, so wrap id with std::atomic<>
Fixes : #28431
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-13 15:42:46 +03:00
Azat Khuzhin
25eb82f120
ThreadPool: do not use joinable() internally
...
joinable() should be used only outside, since internally it is enough to
know `state` to know that something is wrong.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-13 15:42:46 +03:00
Azat Khuzhin
3473b80077
ThreadPool: add some clarification comments
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-13 15:42:46 +03:00
Arthur Passos
395dada988
minor style adjustments
2022-07-13 09:40:56 -03:00
kssenii
0d2d3ac2b4
Merge master
2022-07-13 13:26:15 +02:00
Yakov Olkhovskiy
334b3f3c73
Merge pull request #38864 from azat/fix-rwlock
...
Fix waiting of shared lock after exclusive lock failure
2022-07-13 02:10:35 -04:00
Arthur Passos
d306c1b6f4
Use Strings alias instead of std::vec<std::string>>
2022-07-12 20:37:39 -03:00
Arthur Passos
cb349c05c5
minor adjustments
2022-07-12 19:27:44 -03:00
Arthur Passos
381d60c510
minor adjustments
2022-07-12 17:55:49 -03:00
Arthur Passos
397d05eb9c
Add DNSPTRResolverProvider docs
2022-07-12 14:31:33 -03:00
kssenii
6f29eada80
Merge master
2022-07-12 19:28:18 +02:00
Arthur Passos
d48690d455
Make CaresPTRResolver a singleton through DNSPTRResolverProvider, add comments and address minor comments
2022-07-12 14:21:10 -03:00
mergify[bot]
48688da835
Merge branch 'master' into concurrency-control
2022-07-12 10:57:13 +00:00
Yakov Olkhovskiy
5887deac99
Merge branch 'master' into fix-rwlock
2022-07-11 16:01:16 -04:00
Robert Schulze
1a7727a254
Prefix overridden add_executable() command with "clickhouse_"
...
A simple HelloWorld program with zero includes except iostream triggers
a build of ca. 2000 source files. The reason is that ClickHouse's
top-level CMakeLists.txt overrides "add_executable()" to link all
binaries against "clickhouse_new_delete". This links against
"clickhouse_common_io", which in turn has lots of 3rd party library
dependencies ... Without linking "clickhouse_new_delete", the number of
compiled files for "HelloWorld" goes down to ca. 70.
As an example, the self-extracting-executable needs none of its current
dependencies but other programs may also benefit.
In order to restore access to the original "add_executable()", the
overriding version is now prefixed. There is precedence for a
"clickhouse_" prefix (as opposed to "ch_"), for example
"clickhouse_split_debug_symbols". In general prefixing makes sense also
because overriding CMake commands relies on undocumented behavior and is
considered not-so-great practice (*).
(*) https://crascit.com/2018/09/14/do-not-redefine-cmake-commands/
2022-07-11 19:36:18 +02:00
Arthur Passos
dcc8c646fd
Throw proper DB::Exception in case c-ares initialization fails
2022-07-11 14:29:29 -03:00
Arthur Passos
0c6329bdd3
ares_channel unique_ptr instead of shared_ptr
2022-07-11 10:19:25 -03:00
Arthur Passos
5c221464ba
Forward declare ares_channel and apply -wno-reserved-identifier to CARESPTRResolver unit
2022-07-11 10:10:31 -03:00
Sergei Trifonov
43779ec280
add max_remote_{read,write}_network_bandwidth_for_server settings
2022-07-11 14:59:39 +02:00
Antonio Andelic
197388e29c
Fix list request
2022-07-11 12:13:50 +00:00
Antonio Andelic
41dd4e13f4
Merge branch 'master' into keeper-version-check
2022-07-11 09:38:20 +00:00
Antonio Andelic
37f799550b
Merge pull request #38072 from lingpeng0314/master
...
Add Keeper related monitoring data
2022-07-10 09:39:57 +02:00
kssenii
4c88527c8e
Review fixes
2022-07-09 19:35:07 +02:00
kssenii
bf1c99daab
Refactoring of object storages, add LocalObjectStorage
2022-07-09 12:39:31 +02:00
Frank Chen
da57a993e4
Fix CI
2022-07-09 13:43:10 +08:00
Arthur Passos
4f4acfabe5
c-ares draft wrapper and draft build rules
2022-07-08 17:52:14 -03:00
Dmitry Novik
99cef119e1
Remove logging from OvercommitTracker
2022-07-08 15:34:23 +00:00
Frank Chen
6ae2504e57
Fix code
2022-07-08 18:55:59 +08:00
Roman Vasin
12f4a48957
Extend LUT range to 1900..2300
2022-07-08 06:48:05 +00:00
Nikolay Degterinsky
5cd06ac488
Merge pull request #38947 from soyayaos/master
...
Rename NUMBER_OF_DIMENSIONS_MISMATHED const to NUMBER_OF_DIMENSIONS_MISMATCHED
2022-07-07 22:25:42 +02:00
Alexander Tokmakov
b0f8970d07
Merge pull request #38961 from Algunenano/reduce_zk_eexists
...
Stop reporting Zookeeper "Node exists" exceptions in system.errors when they are expected
2022-07-07 23:16:46 +03:00
Alexander Tokmakov
1d93dc1c9a
Merge pull request #38963 from ClickHouse/fix_38962
...
Avoid weird exception in Keeper
2022-07-07 22:01:35 +03:00
Robert Schulze
f15d9ca59c
Merge pull request #38774 from zvonand/zvonand-nnd
...
Reintroduce nonNegativeDerivative()
2022-07-07 20:39:13 +02:00
Alexander Tokmakov
7630fc76ae
avoid weird exception
2022-07-07 17:19:05 +02:00
Raúl Marín
c3935000a8
Avoid reporting ZK EEXIST errors on checkPartChecksumsAndCommit
2022-07-07 16:10:05 +02:00
Alexander Tokmakov
fdd2db4ace
Move check for denied allocations ( #38858 )
...
* move check for denied allocations
* remove include
* make hardening softer
2022-07-07 14:24:36 +02:00
Antonio Andelic
e96af48e28
Polish
2022-07-07 11:35:02 +00:00
Frank Chen
47da972277
Allows re-entering on thread context to make code more robust.
2022-07-07 17:45:08 +08:00
Frank Chen
4ef85164aa
Fix file name
2022-07-07 17:44:50 +08:00
Frank Chen
93dc109e36
Fix code
2022-07-07 17:44:19 +08:00
Frank Chen
c0a67dc6b9
Add API to simplify user code
2022-07-07 17:44:05 +08:00
Frank Chen
57a7e4a7c9
Remove old API reference
2022-07-07 17:42:35 +08:00
Frank Chen
32931857f0
Propagate tracing context across threads
2022-07-07 17:41:43 +08:00
Frank Chen
2e8c530bed
Move thread trace context out of ThreadStatus
2022-07-07 17:41:10 +08:00
Vladimir Galunshchikov
cacc833da6
Rename NUMBER_OF_DIMENSIONS_MISMATHED const to NUMBER_OF_DIMENSIONS_MISMATCHED
2022-07-07 11:15:59 +03:00
Alexey Milovidov
02adad3f27
Merge pull request #37860 from amosbird/remove-duplicated-log
...
Remove duplicate peak mem log
2022-07-07 08:16:06 +03:00
Andrey Zvonov
7de39d9b15
Merge branch 'master' of github.com:ClickHouse/ClickHouse into zvonand-nnd
2022-07-06 10:59:35 +03:00
HeenaBansal2009
d89ba2e5d9
Review Comments
2022-07-05 21:18:39 -07:00
Azat Khuzhin
98cd92017e
Fix waiting of shared lock after exclusive lock failure
...
When WRITE lock attemp fails (exclusive lock for ALTER/DELETE), and
there are multiple READ locks (shared lock for SELECT/INSERT), i.e. one
INSERT is in progress and one SELECT is queued after ALTER/DELETE
started but before it fails, this SELECT will wait until INSERT will
finishes.
This happens because in case of WRITE lock failure it does not notify
the next READ lock that can be acquired.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-05 19:12:47 +03:00
Kseniia Sumarokova
3e803006a2
Merge pull request #37171 from tchepavel/nats-integration
...
Nats Integration
2022-07-05 17:47:02 +02:00
kssenii
2281f16c15
Fixes
2022-07-05 13:17:44 +02:00
alesapin
bd8a3ee841
Merge pull request #38338 from ClickHouse/zookeeper-add-extra-list-argument
...
Extend ZooKeeper list request with support for filtering persistent or ephemeral nodes only
2022-07-05 12:53:51 +02:00
kssenii
b52084265c
Merge master
2022-07-04 21:37:43 +02:00
Kseniia Sumarokova
6c3d648f49
Merge branch 'master' into nats-integration
2022-07-04 21:06:07 +02:00
zvonand
8a270c01e9
fix floating point in intervals
2022-07-04 20:45:05 +03:00
Arthur Passos
5c00dcd884
Link cpp-dns/udns unconditionally and update submodules to clickhouse fork
2022-07-04 10:07:03 -03:00
Arthur Passos
dcda1c5eb9
Fix style case and DNS error message
2022-07-04 10:06:27 -03:00
Arthur Passos
d66154e697
Test host_regexp against all PTR records instead of only one
2022-07-04 10:05:28 -03:00
Kruglov Pavel
3d47416f8b
Merge pull request #38559 from ClickHouse/Avogar-patch-1
...
Add logging in Epoll and TimerDescriptor in case of EINTR
2022-07-04 15:05:08 +02:00
Alexey Milovidov
7adc87a37d
Merge pull request #38413 from nickitat/cache_values_inside_hash_table_grower
...
Precalculate values inside `HashTableGrower`
2022-07-02 23:30:24 +03:00
Kruglov Pavel
eaf8397c13
Fix style
2022-07-01 19:51:07 +02:00
Kruglov Pavel
fe0dad5429
Update src/Common/TimerDescriptor.cpp
2022-07-01 16:23:20 +02:00
Kruglov Pavel
4550acdc35
Update src/Common/Epoll.cpp
...
Co-authored-by: Azat Khuzhin <a3at.mail@gmail.com>
2022-07-01 16:23:03 +02:00
Robert Schulze
2a1ede0f5a
Merge pull request #38589 from ClickHouse/fix-zero-bytes-in-haystack
...
Fix countSubstrings() & position() on patterns with 0-bytes
2022-07-01 16:15:43 +02:00
zvonand
3b5332d15e
Revert "Revert "Non Negative Derivative window function""
...
This reverts commit dea3b5bfce
.
2022-07-01 18:59:07 +05:00
Nikolai Kochetov
b7529986c0
Use separate counter for RSS in global memory tracker.
2022-07-01 12:09:32 +00:00
Vitaly Baranov
cadb496894
Merge pull request #38537 from vitlibar/backup-improvements-8
...
Backup Improvements 8
2022-07-01 11:20:46 +02:00
Alexey Milovidov
20841f0e1e
Merge pull request #38551 from ClickHouse/revert-37628-non-neg-deriv
...
Revert "Non Negative Derivative window function"
2022-07-01 02:46:28 +03:00
Robert Schulze
1e3b5bfcb7
Fix test 00233_position_function_family
2022-06-30 11:43:25 +00:00
Vitaly Baranov
461a31f237
Improve gathering metadata for backup - part 2.
2022-06-30 08:37:17 +02:00
Robert Schulze
81bb2242fd
Fix countSubstrings() & position() on patterns with 0-bytes
...
SQL functions countSubstrings(), countSubstringsCaseInsensitive(),
countSubstringsUTF8(), position(), positionCaseInsensitive(),
positionUTF8() with non-const pattern argument use fallback sorters
LibCASCIICaseSensitiveStringSearcher and LibCASCIICaseInsensitiveStringSearcher
which call ::strstr(), resp. ::strcasestr(). These functions assume that
the haystack is 0-terminated and they even document that. However, the
callers did not check if the haystack contains 0-byte (perhaps because
its sort of expensive). As a consequence, if the haystack contained a
zero byte in it's payload, matches behind this zero byte were ignored.
create table t (id UInt32, pattern String) engine = MergeTree() order by id;
insert into t values (1, 'x');
select countSubstrings('aaaxxxaa\0xxx', pattern) from t;
We returned 3 before this commit, now we return 6
2022-06-29 21:41:18 +00:00
Nikita Taranov
7d2e8b0883
clean up
2022-06-29 22:15:12 +02:00
Kruglov Pavel
77873e75bf
Update TimerDescriptor.cpp
2022-06-29 13:01:14 +02:00
Kruglov Pavel
30fd086b5e
Add logging in Epoll and TimerDescriptor in case of EINTR
2022-06-29 12:59:18 +02:00
Antonio Andelic
7a8fbbf132
Merge branch 'master' into zookeeper-add-extra-list-argument
2022-06-29 08:31:03 +00:00
Antonio Andelic
1d26446f84
Define new list request type
2022-06-29 08:30:39 +00:00
Robert Schulze
bb441faa42
Merge pull request #38491 from ClickHouse/clang-tidy-in-branches
...
Fix some clang-tidy warnings in headers
2022-06-29 10:22:04 +02:00
Alexey Milovidov
dea3b5bfce
Revert "Non Negative Derivative window function"
2022-06-29 08:56:15 +03:00
HeenaBansal2009
608bfb8453
Fixed {..} regex parsing as shell does
2022-06-28 12:49:31 -07:00
Robert Schulze
f692ead6ad
Don't use std::unique_lock unless we have to
...
Replace where possible by std::lock_guard which is more light-weight.
2022-06-28 19:19:06 +00:00
Robert Schulze
c22038d48b
More clang-tidy fixes
2022-06-28 11:50:05 +00:00
HeenaBansal2009
6a5819a405
Fixed bug in parsing{..} regex
2022-06-27 22:51:31 -07:00
Robert Schulze
5e28580f11
Fix some clang-tidy warnings in headers
2022-06-27 23:49:07 +02:00
Dmitry Novik
b629d557ce
Merge pull request #38246 from ClickHouse/overcommit-deadlock-fix
...
Fix deadlock in OvercommitTracker logging
2022-06-27 21:57:31 +02:00
Nikita Taranov
986ead7358
use old HashTableGrower with *WithStackMemory data structures
2022-06-27 20:34:18 +02:00
Robert Schulze
53cd8820d3
Merge pull request #38434 from ClickHouse/multi-match-with-nonconst-patterns
...
Refactoring to enable multi-match functions with non-const needles
2022-06-27 19:35:48 +02:00
kssenii
5c26f99bc5
Merge master
2022-06-27 13:36:41 +02:00
Antonio Andelic
c06776ce0d
Merge branch 'master' into zookeeper-add-extra-list-argument
2022-06-27 06:53:29 +00:00
Antonio Andelic
0d906e311d
PR review fixes
2022-06-27 06:53:09 +00:00
Robert Schulze
bb7c627964
Cosmetics: Pass patterns around as std::string_view instead of StringRef
...
- The patterns are not used in hashing, there should not be a performance
impact when we use stuff from the standard library instead.
- added forgotten .reserve() in FunctionsMultiStringPosition.h
2022-06-26 15:32:19 +00:00
mergify[bot]
f63c959679
Merge branch 'master' into cleanup_garbage_in_store_dir
2022-06-26 13:35:10 +00:00
Nikita Taranov
95d413d7ec
put the thing into its own cache line
2022-06-26 01:38:45 +02:00
Alexey Milovidov
25cc406201
Update src/Common/ThreadStatus.cpp
...
Co-authored-by: Dmitry Novik <n0vik@clickhouse.com>
2022-06-26 01:29:06 +03:00
Nikita Taranov
abcefeeb41
cleanup
2022-06-25 23:15:59 +02:00
mergify[bot]
469f24a12c
Merge branch 'master' into zookeeper-add-extra-list-argument
2022-06-25 12:01:06 +00:00
Alexey Milovidov
e33f236d50
Slight improvement
2022-06-25 03:59:01 +02:00
Alexey Milovidov
b89f01438e
Lower mutex scope
2022-06-25 03:05:55 +02:00
Nikita Taranov
6639d94125
stash
2022-06-25 01:32:21 +02:00
Nikita Taranov
c476427422
stash
2022-06-25 01:27:24 +02:00
Nikita Taranov
f6ef78c0f1
stash
2022-06-25 01:26:46 +02:00
kssenii
708b93a03b
Merge master
2022-06-25 00:08:24 +02:00
kssenii
082c306ee8
Fix test, fix style
2022-06-25 00:04:21 +02:00
Kruglov Pavel
86e8f31ad4
Merge branch 'master' into schema-inference-cache
2022-06-24 16:10:25 +02:00
mergify[bot]
92eff431be
Merge branch 'master' into overcommit-deadlock-fix
2022-06-24 13:54:11 +00:00
Alexander Tokmakov
cfe0e434e5
Revert "Add support for io_uring read method"
2022-06-24 12:19:41 +03:00
mergify[bot]
a0eb831cd1
Merge branch 'master' into overcommit-deadlock-fix
2022-06-23 23:02:59 +00:00
Alexey Milovidov
812ab9bd6b
Merge pull request #36103 from sauliusvl/uring
...
Add support for io_uring read method
2022-06-24 00:34:29 +03:00
Alexander Tokmakov
74f38710a8
Merge branch 'master' into cleanup_garbage_in_store_dir
2022-06-23 21:43:28 +02:00
kssenii
6fbd49f554
Merge master
2022-06-23 21:40:01 +02:00
kssenii
468c98ff42
Better
2022-06-23 17:46:27 +02:00
kssenii
f7b329ee57
Merge master
2022-06-23 14:56:48 +02:00
Antonio Andelic
ae3d9fd962
Expose list request type in internal client
2022-06-23 10:28:12 +00:00
Antonio Andelic
568c957eb2
Fix TestKeeper
2022-06-23 10:28:12 +00:00
Antonio Andelic
3a71b63b5d
Add list request type
2022-06-23 10:28:12 +00:00
Antonio Andelic
a5ee918bd2
Merge pull request #38047 from ClickHouse/keeper-sequential-consistency-reads
...
Add `SYNC` command to internal ZooKeeper client
2022-06-23 08:43:24 +02:00
kssenii
e40d9bcf55
Merge master
2022-06-22 23:28:52 +02:00
mergify[bot]
9688388be2
Merge branch 'master' into concurrency-control
2022-06-22 15:07:56 +00:00
Dmitry Novik
3544fbe5c4
cleanup
2022-06-22 14:05:20 +00:00
kssenii
4178abc0a5
Merge master
2022-06-21 22:38:51 +02:00
kssenii
90ce0b44f0
Fixes
2022-06-21 21:03:18 +02:00
Robert Schulze
0d80874d40
Merge pull request #38068 from ClickHouse/clang-tsa
...
Support for Clang Thread Safety Analysis (TSA)
2022-06-21 20:19:33 +02:00
Sergei Trifonov
dc59d208de
unittest style/perf minor fixes
2022-06-21 18:38:51 +02:00
kssenii
481644939a
Add describe cache command
2022-06-21 16:07:39 +02:00
avogar
d37ad2e6de
Implement cache for schema inference for file/s3/hdfs/url
2022-06-21 13:02:48 +00:00
Dmitry Novik
099055c183
Explicitly forbid allocations in OvercommitTracker
2022-06-21 12:26:13 +00:00
Alexander Tokmakov
ba0fcec993
add background cleanup of store/ subdirs
2022-06-21 12:35:47 +02:00
Saulius Valatka
160c95a6b8
unpoisson memory read by io_uring, handle interrupted syscalls, do not throw from monitor thread
2022-06-21 13:20:21 +03:00
Saulius Valatka
7b5e13a9eb
implement pending request queue to prevent CQ overflows, more error checking, reduce size of io_uring
2022-06-21 13:17:56 +03:00
Dmitry Novik
e10f079bd3
Get rid of allocations in OvercommitTracker
2022-06-21 10:15:33 +00:00
Kseniia Sumarokova
71ee5dbbb2
Merge pull request #38227 from kssenii/diff-reduce
...
buffer's getFileSize small changes
2022-06-21 12:12:22 +02:00
mergify[bot]
bdd5b94192
Merge branch 'master' into keeper-sequential-consistency-reads
2022-06-21 08:13:04 +00:00
Saulius Valatka
0dc7854af1
add initial io_uring support
2022-06-21 09:59:46 +03:00
lingpeng0314
c18dd55400
Resolve comments
...
- refine metric description
2022-06-21 10:30:19 +08:00
Dmitry Novik
9ca368ac20
Use do while(false) in macro
2022-06-20 23:10:40 +00:00
mergify[bot]
9bdd9e14a6
Merge branch 'master' into fix_flaky_tests_with_transactions
2022-06-20 18:11:30 +00:00
Dmitry Novik
53d656d89f
Fix deadlock in OvercommitTracker logging
2022-06-20 17:35:24 +00:00
Robert Schulze
55b39e709d
Merge remote-tracking branch 'origin/master' into clang-tsa
2022-06-20 16:39:32 +02:00
Antonio Andelic
0c76fc1121
Fix style
2022-06-20 14:21:59 +00:00
Robert Schulze
5a4f21c50f
Support for Clang Thread Safety Analysis (TSA)
...
- TSA is a static analyzer build by Google which finds race conditions
and deadlocks at compile time.
- It works by associating a shared member variable with a
synchronization primitive that protects it. The compiler can then
check at each access if proper locking happened before. A good
introduction are [0] and [1].
- TSA requires some help by the programmer via annotations. Luckily,
LLVM's libcxx already has annotations for std::mutex, std::lock_guard,
std::shared_mutex and std::scoped_lock. This commit enables them
(--> contrib/libcxx-cmake/CMakeLists.txt).
- Further, this commit adds convenience macros for the low-level
annotations for use in ClickHouse (--> base/defines.h). For
demonstration, they are leveraged in a few places.
- As we compile with "-Wall -Wextra -Weverything", the required compiler
flag "-Wthread-safety-analysis" was already enabled. Negative checks
are an experimental feature of TSA and disabled
(--> cmake/warnings.cmake). Compile times did not increase noticeably.
- TSA is used in a few places with simple locking. I tried TSA also
where locking is more complex. The problem was usually that it is
unclear which data is protected by which lock :-(. But there was
definitely some weird code where locking looked broken. So there is
some potential to find bugs.
*** Limitations of TSA besides the ones listed in [1]:
- The programmer needs to know which lock protects which piece of shared
data. This is not always easy for large classes.
- Two synchronization primitives used in ClickHouse are not annotated in
libcxx:
(1) std::unique_lock: A releaseable lock handle often together with
std::condition_variable, e.g. in solve producer-consumer problems.
(2) std::recursive_mutex: A re-entrant mutex variant. Its usage can be
considered a design flaw + typically it is slower than a standard
mutex. In this commit, one std::recursive_mutex was converted to
std::mutex and annotated with TSA.
- For free-standing functions (e.g. helper functions) which are passed
shared data members, it can be tricky to specify the associated lock.
This is because the annotations use the normal C++ rules for symbol
resolution.
[0] https://clang.llvm.org/docs/ThreadSafetyAnalysis.html
[1] https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/42958.pdf
2022-06-20 16:13:25 +02:00
Antonio Andelic
777ae72259
Merge branch 'master' into keeper-sequential-consistency-reads
2022-06-20 14:06:45 +00:00
kssenii
5dd1bb2fd8
improvements for getFileSize
2022-06-20 15:22:56 +02:00
Antonio Andelic
6c225bb04d
Update src/Common/ZooKeeper/ZooKeeperImpl.cpp
...
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
2022-06-20 14:55:59 +02:00
kssenii
90be49faa4
Merge master
2022-06-20 14:26:56 +02:00
Vitaly Baranov
06c4082a7c
Merge pull request #37358 from vitlibar/backup-improvements-6
...
Backup Improvements 6
2022-06-20 14:13:30 +02:00
kssenii
84a2bb031c
Fix
2022-06-20 14:02:06 +02:00
mergify[bot]
4ed606a275
Merge branch 'master' into fix_flaky_tests_with_transactions
2022-06-20 10:17:51 +00:00
Kseniia Sumarokova
a756b4be27
Merge pull request #37391 from azat/insert-profile-events-fix
...
Send profile events for INSERT queries (previously only SELECT was supported)
2022-06-20 12:16:29 +02:00
lingpeng0314
f06b19bdbf
Add keeper monitoring data
2022-06-20 11:32:18 +08:00
Vitaly Baranov
a0c558a17e
Implement backup/restore for ACL system tables (system.users, system.roles, etc.)
2022-06-17 18:14:31 +02:00
Alexander Tokmakov
e9c2157f01
Merge branch 'master' into fix_flaky_tests_with_transactions
2022-06-17 15:57:30 +02:00
alesapin
1c7a7da487
Merge pull request #38088 from kssenii/diff-with-cache
...
Extract some diff from pr #36171
2022-06-17 12:07:31 +02:00
Alexander Tokmakov
39c0219c11
fixes
2022-06-16 19:41:32 +02:00
alesapin
ca33ff93cb
Merge pull request #37555 from ClickHouse/revert-37534-revert-37036-keeper-preprocess-operations
...
Add support for preprocessing ZooKeeper operations and real-time digest in `clickhouse-keeper`
2022-06-16 17:29:00 +02:00
kssenii
797482f132
Better
2022-06-16 16:40:29 +02:00
Azat Khuzhin
b3bf7589ef
Fix possible concurrent access in ProgressIndication
...
In case of all of the above:
- clickhouse-local
- input_format_parallel_parsing=true
- write_progress_on_update=true
It is possible concurrent access to the following:
- writeProgress() (class properties) (guarded with progress_mutex)
- thread_data/host_cpu_usage (guarded with profile_events_mutex)
v2: decrease number of rows for INSERT ProfileEvents test (10 times)
CI: https://s3.amazonaws.com/clickhouse-test-reports/37391/4bd5c335182279dcc5020aa081b13c3044135951/stateless_tests__debug__actions__[1/3].html
v3: decrease number of rows for INSERT ProfileEvents test (10 times) and add a comment
CI: https://s3.amazonaws.com/clickhouse-test-reports/37391/026d7f732cb166c90d6c287b02824b6c7fdebf0c/stateless_tests_flaky_check__address__actions_/runlog.log
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
f
2022-06-16 11:59:01 +03:00
Antonio Andelic
0b843d316f
Merge branch 'master' into keeper-sequential-consistency-reads
2022-06-16 07:52:56 +00:00
Alexey Milovidov
c26abbfdc5
Merge pull request #38093 from danlark1/master
...
Optimize most important parts with NEON SIMD
2022-06-16 08:26:56 +03:00