Kseniia Sumarokova
44577ea8f6
Update FileSegment.cpp
2022-08-25 13:02:41 +02:00
Kseniia Sumarokova
7e0ff0dfaf
Update FileSegment.cpp
2022-08-25 12:53:51 +02:00
Kseniia Sumarokova
b0f5fd9bf6
Update FileSegment.cpp
2022-08-25 12:41:31 +02:00
alesapin
c4f8811701
Merge pull request #40395 from ClickHouse/data_source_description
...
Add copy shortcut for backups if dest and source support it
2022-08-25 12:36:30 +02:00
Kseniia Sumarokova
f6eea03120
Merge pull request #40569 from kssenii/add-comments
...
Add some logs to fs cache
2022-08-25 12:06:48 +02:00
Frank Chen
221a71f801
Fix typo
2022-08-25 15:05:07 +08:00
Frank Chen
558b179cb1
Fix typo
...
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2022-08-25 10:14:18 +08:00
Frank Chen
99c37ce6c6
Merge branch 'master' into tracing_context_propagation
2022-08-25 10:07:16 +08:00
Antonio Andelic
da0d6056f4
Merge branch 'master' into replicate-tables-readonly
2022-08-24 17:29:33 +00:00
kssenii
ee2f9caa16
Fix
2022-08-24 18:50:22 +02:00
vdimir
7194df1184
Move back TemporaryFile -> TemporaryFileOnDisk
2022-08-24 16:14:11 +00:00
vdimir
1e24a731f7
Move Disks/TemporaryFileOnDisk -> Common/TemporaryFile
2022-08-24 16:14:10 +00:00
vdimir
8701a909a2
enoughSpaceInDirectory works if file is not created yet
2022-08-24 16:14:09 +00:00
vdimir
51c44424cc
More metrics for temp files
2022-08-24 16:14:09 +00:00
vdimir
cd4038d511
Use TemporaryFileOnDisk instead of Poco::TemporaryFile
2022-08-24 16:14:08 +00:00
vdimir
7e0c9062c7
Add ProfileEvents::ExternalSort(Un)CompressedBytes
2022-08-24 16:14:07 +00:00
Kseniia Sumarokova
3b982f6b8b
Update FileCache.cpp
2022-08-24 17:29:43 +02:00
Kseniia Sumarokova
aa0ed28a0b
Update FileSegment.cpp
2022-08-24 16:43:44 +02:00
Kseniia Sumarokova
985276f8e6
Merge branch 'master' into add-comments
2022-08-24 16:33:35 +02:00
Kseniia Sumarokova
6d0907ebef
Update FileSegment.cpp
2022-08-24 15:55:07 +02:00
Kseniia Sumarokova
2cea295a7a
More logs
2022-08-24 15:53:54 +02:00
alesapin
eeb0a3584e
Merge branch 'master' into data_source_description
2022-08-24 15:26:18 +02:00
Kseniia Sumarokova
3e8a2d9093
Update FileSegment.cpp
2022-08-24 14:16:53 +02:00
kssenii
2b792a2bdd
Add more logs
2022-08-24 12:40:44 +02:00
Frank Chen
1cf3659f26
Fix style checking
...
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2022-08-24 18:03:14 +08:00
Frank Chen
dd0f2cbe6b
Fix style checking
...
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2022-08-24 17:04:02 +08:00
Frank Chen
03007606ba
Fix format
...
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2022-08-24 17:01:31 +08:00
Frank Chen
cd19366b44
Move classes into DB::OpenTelemetry namespace
2022-08-24 16:41:40 +08:00
Frank Chen
efc6a60a60
Clean code
2022-08-24 15:59:44 +08:00
Antonio Andelic
fc33941959
Add support for shared drop/create
2022-08-23 13:15:31 +00:00
Kseniia Sumarokova
d814a56277
Merge pull request #40420 from kssenii/fix-cache-bug
...
Fix incorrect fallback to skip cache in case of very high concurrency level
2022-08-23 14:18:38 +02:00
Robert Schulze
635cc7617e
Merge pull request #40464 from ClickHouse/less-string-ref
...
Reduce some usage of StringRef
2022-08-23 12:16:00 +02:00
Maksim Kita
11ba15dde6
Merge branch 'master' into dns-resolver-remove-hints
2022-08-22 22:23:35 +02:00
Alexey Milovidov
0252695c6e
Merge pull request #40224 from ClickHouse/alexey-milovidov-patch-4
...
Check what will be if I enable `concurrency-mt-unsafe` in `clang-tidy`
2022-08-22 23:12:01 +03:00
Maksim Kita
191f39ae8d
DNSResolver remove AI_V4MAPPED, AI_ALL hints
2022-08-22 21:03:25 +02:00
Robert Schulze
ec9ef741fc
Merge branch 'master' into less-string-ref
2022-08-22 20:48:16 +02:00
Kseniia Sumarokova
f85e5f8808
Merge branch 'master' into fix-cache-bug
2022-08-22 16:54:07 +02:00
Azat Khuzhin
ec2e671d20
Remove ThreadStatus::untracked_memory_limit_increase
...
It looks useless nowadays, because operator new cannot throw
MEMORY_LIMIT_EXCEEDED today, and so any code that works on Exception is
likely safe.
Refs: #40249
Refs: #24483
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-08-22 13:26:18 +03:00
kssenii
74087d57c4
Less notify
2022-08-22 12:00:14 +02:00
kssenii
26384b2543
Fix
2022-08-22 00:02:05 +02:00
Alexey Milovidov
996aa2d126
Remove some trash
2022-08-21 20:46:33 +02:00
Robert Schulze
e8b3f56733
Limit suppression to a specific warning
2022-08-21 18:24:17 +00:00
Robert Schulze
4caf2c4c33
Reduce some usage of StringRef
...
See #39535 and #39300
2022-08-21 18:10:32 +00:00
Antonio Andelic
f29ef68936
Merge branch 'master' into keeper-storage
2022-08-21 17:09:36 +00:00
alesapin
78ba732b31
Merge branch 'master' into data_source_description
2022-08-21 12:11:21 +02:00
Alexey Milovidov
0db45afb08
Merge branch 'master' into alexey-milovidov-patch-4
2022-08-20 19:58:31 +03:00
Alexey Milovidov
3649aac593
Fix style
2022-08-20 17:59:51 +02:00
Alexey Milovidov
74e1f4dc61
Fix clang-tidy
2022-08-20 17:09:20 +02:00
Frank Chen
54a530a4e0
Make sure right toString is linked
2022-08-20 14:22:33 +08:00
Frank Chen
0569608192
Use toString
2022-08-20 12:30:39 +08:00
Azat Khuzhin
28f2607d50
Fix rare race in updating peak of memory tracker
...
Previously it was possible that allocation, that had been done with
locked MEMORY_LIMIT_EXCEEDED (LockMemoryExceptionInThread), will update
the peak to the amount that will include some other allocations.
Consider the following example (T1/T2 - different threads):
Initial state:
MemoryTracker::amount = 0
Possible race:
LockMemoryExceptionInThread locker;
T1: MemoryTracker::allocImpl(1MiB) T2: MemoryTracker::allocImpl(1PiB)
will_be = 1T, amount = 1T
will_be = 1T+1MiB, amount=1T+1MiB
peak = will_be
amount = 0 // revert due to exceeding limit
throw(MEMORY_LIMIT_EXCEEDED)
So as you can see T1 includes allocations from T2 into the peak.
Note, that almost every place that has memory_usage column uses
MemoryTracker::peak (not MemoryTracker::amount), this is true for
query_log (for part_log column name is peak_memory_usage, and obviously
it uses MemoryTracker::peak).
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-08-19 20:57:12 +02:00
kssenii
e641664115
Refactor
2022-08-19 18:15:18 +02:00
alesapin
d8664c3227
Add shortcut for backups
2022-08-19 16:58:30 +02:00
Antonio Andelic
5e2c8869a9
Merge branch 'master' into replicate-tables-readonly
2022-08-19 09:41:11 +00:00
Alexander Tokmakov
3d253ec51b
Merge branch 'master' into zookeeper_client_fault_injection
2022-08-18 21:23:50 +03:00
Frank Chen
a3b6ad2a65
Merge branch 'master' into tracing_context_propagation
2022-08-18 20:59:07 +08:00
Nikolai Kochetov
5a85531ef7
Merge pull request #38286 from Avogar/schema-inference-cache
...
Add schema inference cache for s3/hdfs/file/url
2022-08-18 13:07:50 +02:00
Kseniia Sumarokova
d8fd1f9c37
Merge pull request #36171 from kssenii/make-cache-composable
...
Make cache composable, allow not to evict certain files (regarding .idx, .mrk, ..), delete old cache version
2022-08-18 12:37:34 +02:00
Vladimir C
7ac0eb1af1
Merge pull request #40035 from ClickHouse/vdimir/cluster-discovery-observer
...
Add observer mode to (zoo)keeper cluster discovery feature
2022-08-18 10:53:49 +02:00
kssenii
d63d34f7bb
Merge master
2022-08-17 15:49:32 +02:00
avogar
8dd54c043d
Merge branch 'master' of github.com:ClickHouse/ClickHouse into schema-inference-cache
2022-08-17 11:47:40 +00:00
Alexander Tokmakov
747d74121c
Merge branch 'master' into zookeeper_client_fault_injection
2022-08-17 14:41:02 +03:00
Nikolai Kochetov
8ec3f0821c
Merge pull request #40249 from azat/fix-memory-account-on-error
...
Fix memory accounting in case of MEMORY_LIMIT_EXCEEDED errors
2022-08-17 12:51:40 +02:00
Alexander Tokmakov
ae000e9125
Merge branch 'master' into zookeeper_client_fault_injection
2022-08-17 12:48:54 +03:00
Alexander Tokmakov
4482465ce3
Merge pull request #40134 from arthurpassos/fix_cares_reverse_dns_race_condition
...
Fix CaresPTRResolver race condition
2022-08-16 21:06:17 +03:00
Nikolai Kochetov
feab1eb0a3
Update CurrentMemoryTracker.cpp
2022-08-16 18:40:06 +02:00
Nikolai Kochetov
8e4c967fc9
Try to account untracked memory (per thread) more correctly.
2022-08-16 18:09:05 +02:00
Azat Khuzhin
ded70fbd53
Fix memory accounting in case of MEMORY_LIMIT_EXCEEDED errors
...
In case of memory allocation had been failed, it should not be take into
account anywhere, otherwise it will report incorrect memory usage for
query and in logs.
I found this while was digging into OOM in stress tests, and saw memory
allocations that had been significantly greater the memory limit (even
with overcommit).
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-08-16 18:08:56 +02:00
kssenii
43991729ef
Fix
2022-08-16 17:08:12 +02:00
kssenii
93816e7366
Review fixes
2022-08-16 15:11:55 +02:00
kssenii
3901778d49
Add more current metrics for cache
2022-08-16 11:32:23 +02:00
Antonio Andelic
2ed42bca7e
Merge branch 'master' into replicate-tables-readonly
2022-08-16 08:17:53 +00:00
kssenii
eb26b219b9
Merge master
2022-08-16 00:56:27 +02:00
Alexander Tokmakov
c9bb91c31f
Merge pull request #40232 from ClickHouse/fix_a_bug_with_symlinks_detection
...
Fix a bug with symlinks detection
2022-08-15 20:28:30 +03:00
Robert Schulze
70f63d2aae
Merge pull request #40033 from ClickHouse/reenable-avx512-vbmi-columnvector-filter
...
Revert the revert of "ColumnVector: optimize filter with AVX512 VBMI2 compress store"
2022-08-15 18:39:57 +02:00
Alexander Tokmakov
0e22b4b800
Update src/Common/filesystemHelpers.cpp
...
Co-authored-by: alesapin <alesapin@clickhouse.com>
2022-08-15 14:25:25 +03:00
Alexander Tokmakov
6f5a7c3bf7
fix a bug with symlinks detection
2022-08-15 12:30:47 +02:00
Alexey Milovidov
903021f35a
Merge branch 'master' into profile-events-incomplete-result
2022-08-14 11:04:19 +03:00
Alexey Milovidov
1a8ddf2956
Addition to prev. revision
2022-08-14 09:35:22 +02:00
Alexey Milovidov
001aca3b47
ProfileEvents for incomplete data due to query complexity settings
2022-08-14 09:17:02 +02:00
Alexey Milovidov
216c1d8efa
Revert "Use separate counter for RSS in global memory tracker."
2022-08-14 09:50:12 +03:00
Alexey Milovidov
6b8f4f1d59
Merge branch 'master' into server-side-time
2022-08-12 21:36:40 +02:00
Maksim Kita
098436f012
Merge pull request #40129 from kitaisreal/pod-array-assign-empty-array-fix
...
PODArray assign empty array fix
2022-08-12 18:34:19 +02:00
Kseniia Sumarokova
a6cfc7bc3b
Merge pull request #34651 from alexX512/master
...
New caching strategies
2022-08-12 17:23:37 +02:00
Alexey Milovidov
8c3d1f2b5b
Merge branch 'master' into server-side-time
2022-08-12 17:40:02 +03:00
Arthur Passos
07163eb79a
Remove c-ares lib de-initialization
2022-08-12 10:49:39 -03:00
vdimir
cef6c4f8ce
Add observer mode to (zoo)keeper cluster discovery feature
2022-08-12 13:26:12 +00:00
Kruglov Pavel
4c7222d938
Merge pull request #40020 from canhld94/ch_canh_fix_hash
...
fix HashMethodOneNumber with const column
2022-08-12 14:40:24 +02:00
Antonio Andelic
1c0f85ccdb
Attach table in background
2022-08-12 09:32:13 +00:00
kssenii
fd6362d2f2
Merge master
2022-08-12 11:22:39 +02:00
Kseniia Sumarokova
09ad1115d7
Merge pull request #40138 from ClickHouse/kssenii-patch-4
...
fs cache: minor change
2022-08-12 11:18:03 +02:00
Maksim Kita
e2ae0d585b
Merge pull request #40078 from Algunenano/threadlocal_perf
...
Optimize CurrentMemoryTracker alloc and free
2022-08-12 10:24:54 +02:00
Maksim Kita
5dd4f4b10f
Merge branch 'master' into pod-array-assign-empty-array-fix
2022-08-12 10:20:33 +02:00
alexX512
06cb8737a8
Delete TODO
2022-08-11 22:38:13 +00:00
kssenii
b7c91b9cdb
Merge remote-tracking branch 'origin/make-cache-composable' into make-cache-composable
2022-08-11 23:09:35 +02:00
kssenii
768a207176
Merge master
2022-08-11 22:25:53 +02:00
Kseniia Sumarokova
004a4d4947
Update FileCache.cpp
2022-08-11 22:15:33 +02:00
Kseniia Sumarokova
687ea5dd76
Update FileCache.cpp
2022-08-11 22:13:22 +02:00
Arthur Passos
59108345e9
Use one CaresPTRResolver per thread
2022-08-11 13:28:07 -03:00
Kseniia Sumarokova
be69169f97
Merge pull request #38048 from KinderRiven/decoupling_cache_function_and_algorithm
...
Decoupling local cache function and cache algorithm
2022-08-11 16:49:56 +02:00
Maksim Kita
f9326b00c8
PODArray assign empty array fix
2022-08-11 16:26:41 +02:00
Raúl Marín
11a274e990
Clean up constinit usage and add a comment
2022-08-11 13:27:53 +02:00
Nikolai Kochetov
971d94e492
Merge pull request #38682 from ClickHouse/separate-counter-for-rss-in-memory-tracker
...
Use separate counter for RSS in global memory tracker.
2022-08-11 10:11:18 +02:00
Duc Canh Le
84cd867aa8
materialize column instead of handling column in hash method
2022-08-11 10:46:06 +08:00
Alexey Milovidov
8374f31306
Merge pull request #39425 from arenadata/ADQM-485
...
Add support of dates from year 1900 to 2299 for Date32 and DateTime64
2022-08-11 05:01:53 +03:00
Raúl Marín
d7a545e30d
Try to optimize CurrentMemoryTracker alloc and free
2022-08-10 22:05:09 +02:00
Nikolai Kochetov
c9353cbae8
Merge branch 'master' into separate-counter-for-rss-in-memory-tracker
2022-08-10 21:39:50 +02:00
Alexey Milovidov
0b24dbaec8
Merge
2022-08-10 19:41:44 +02:00
KinderRiven
1aa7bbcbbd
fix unique_ptr
2022-08-11 00:11:41 +08:00
KinderRiven
9b7f87677d
fix
2022-08-11 00:11:41 +08:00
KinderRiven
9ba94e64f9
fix
2022-08-11 00:11:41 +08:00
KinderRiven
2ae02a4921
fix style
2022-08-11 00:11:40 +08:00
KinderRiven
76e0aad69e
fix
2022-08-11 00:11:40 +08:00
KinderRiven
fbaa70b313
fix
2022-08-11 00:11:40 +08:00
KinderRiven
f6a58bff4c
fix build
2022-08-11 00:11:40 +08:00
KinderRiven
d2b5581632
fix
2022-08-11 00:11:40 +08:00
KinderRiven
1b01cc8ed9
fix
2022-08-11 00:11:40 +08:00
KinderRiven
164fa1ab0e
fix build and style
2022-08-11 00:11:40 +08:00
KinderRiven
61b580aba4
add note
2022-08-11 00:11:40 +08:00
KinderRiven
081cd4938a
fix style
2022-08-11 00:11:40 +08:00
KinderRiven
9d83b93e88
fix rebase
2022-08-11 00:11:40 +08:00
KinderRiven
50fd740ec3
fix
2022-08-11 00:11:40 +08:00
KinderRiven
c5f9022510
fix
2022-08-11 00:11:40 +08:00
KinderRiven
43cf771657
better
2022-08-11 00:11:40 +08:00
KinderRiven
ffaf44c1c1
fix style
2022-08-11 00:11:40 +08:00
KinderRiven
7f54fa726b
Decoupling cache functions and algorithms
2022-08-11 00:11:40 +08:00
Kseniia Sumarokova
a4d513657c
Merge pull request #40010 from ClickHouse/catboost-further-preparation
...
Further preparation for catboost integration into library-bridge
2022-08-10 17:35:12 +02:00
kssenii
0ff5609738
Finish support for persistent files
2022-08-10 16:54:59 +02:00
Nikita Mikhaylov
8e8e845a89
Merge branch 'master' into separate-counter-for-rss-in-memory-tracker
2022-08-10 16:44:24 +02:00
Duc Canh Le
bf5e9173ad
fix hashMethodOneNumber constructor
2022-08-10 21:01:09 +08:00
Duc Canh Le
ec49f7b3a3
fix style
2022-08-10 19:28:08 +08:00
Duc Canh Le
17fa8076e3
better implementation
2022-08-10 18:59:13 +08:00
Duc Canh Le
7910a93b02
fix other hash methods and add tests
2022-08-10 18:59:13 +08:00
Duc Canh Le
1a08161dbf
simplify solution
2022-08-10 18:59:13 +08:00
Duc Canh Le
cdd5b32d29
fix HashMethodOneNumber with const column
2022-08-10 18:59:13 +08:00
kssenii
7b66b96ec5
Minor change
2022-08-10 12:20:54 +02:00
kssenii
5c3227ba56
Merge master
2022-08-10 12:00:34 +02:00
Antonio Andelic
4023d4a37a
Add support for drop/truncate and tests
2022-08-10 07:24:56 +00:00
Alexey Milovidov
33e43cc6f9
Merge branch 'master' into server-side-time
2022-08-10 07:28:15 +03:00
Robert Schulze
05ea33c897
Revert "Revert "ColumnVector: optimize filter with AVX512VBMI2 compress store""
...
This reverts commit 5524706b78
.
2022-08-09 17:00:13 +00:00
Kruglov Pavel
5ebf230dbb
Merge pull request #39426 from kitaisreal/shell-command-wait-pid-refactoring
...
ShellCommand wait pid refactoring
2022-08-09 11:48:20 +02:00
Roman Vasin
96598e3574
Replace normalizeDayNum() by std::clamp()
2022-08-09 08:52:02 +00:00
alexX512
21624708aa
Fix tidy build
2022-08-09 08:36:49 +00:00
alexX512
c9c26d4898
Fix review
2022-08-08 21:34:24 +00:00
Robert Schulze
e0d5020a92
Add simple versioning to the *-bridge-to-server protocol
...
- In general, it is expected that clickhouse-*-bridges and
clickhouse-server were build from the same source version (e.g. are
upgraded "atomically"). If that is not the case, we should at least
be able to detect the mismatch and abort.
- This commit adds a URL parameter "version", defined in a header shared
by the server and bridges. The bridge returns an error in case of
mismatch.
- The version is *not* send and checked for "ping" requests (used for
handshake), only for regular requests send after handshake. This is
because the internally thrown server-side exception due to HTTP
failure does not propagate the exact HTTP error (it only stores the
error as text), and as a result, the server-side handshake code
simply retries in case of error with exponential backoff and finally
fails with a "timeout error". This is reasonable as pings typically
fail due to time out. However, without a rework of HTTP exceptions,
version mismatch during ping would also appear as "timeout" which is
too misleading. The behavior may be changed later if needed.
- Note that introducing a version parameter does not represent a
protocol upgrade itself. Bridges older than the server will simply
ignore the field. Only servers older than the bridges receive an error
but such a situation should never occur in practice.
2022-08-08 19:40:37 +00:00
Alexey Milovidov
db85ebfaa7
Add server-side time to Progress
2022-08-08 06:55:41 +02:00
Alexey Milovidov
5524706b78
Revert "ColumnVector: optimize filter with AVX512VBMI2 compress store"
2022-08-08 05:03:15 +03:00
alexX512
f16c668a0c
Delete Args from CacheBase
2022-08-07 20:44:26 +00:00
alexX512
1c9d40e4a8
Fix build error
2022-08-07 20:37:11 +00:00
alexX512
269631cfbe
Fix typo
2022-08-07 20:02:32 +00:00
alexX512
62f84b88eb
Review fixes
2022-08-07 20:02:21 +00:00
alexX512
8adf9f2448
fix compile error
2022-08-07 19:59:39 +00:00
alexX512
2be72560f4
Add loading of mark cache policy
2022-08-07 19:59:39 +00:00
alexX512
a057e1607a
Fix remove operation, add evict ion i from cachef number of elements greater than max_elements_size
2022-08-07 19:59:39 +00:00
alexX512
8be62770aa
Test with SLRU and debug logs
2022-08-07 19:59:39 +00:00
alexX512
c044fee7a0
Test with LRU
2022-08-07 19:59:39 +00:00
alexX512
a238e349cd
Unblocking get max_size
2022-08-07 19:59:39 +00:00
alexX512
daab74925c
Test with default cache policy = SLRU
2022-08-07 19:59:39 +00:00
alexX512
994a9fc005
Minor fixes
2022-08-07 19:59:38 +00:00
alexX512
50a8887ab9
Add [[maybe_unused]] locks and delete all objects from cache queue
2022-08-07 19:59:38 +00:00
alexX512
69cc89e36e
Check with default cache policy = LRU
2022-08-07 19:59:38 +00:00
alexX512
a94950d3ac
Test only with LRU
2022-08-07 19:59:38 +00:00
alexX512
9940b69bd7
Include fix in ICachePolicy.h
2022-08-07 19:59:38 +00:00
alexX512
910645a52a
Include fix in SLRUCachePolicy.h
2022-08-07 19:59:38 +00:00
alexX512
4f52efd4bb
Build fix
2022-08-07 19:59:38 +00:00
alexX512
6bf29cb610
Change class LRUCache to class CachBase. Check running CacheBase with default pcahce policy SLRU
2022-08-07 19:59:30 +00:00
alexX512
c606ccc75d
Fix formatting problems
2022-08-07 19:16:40 +00:00
alexX512
1133e42367
Add CacheBase clas instead of LRUCache and SLRUCache fo simpler configuration oache policiesf
2022-08-07 19:16:40 +00:00
alexX512
a0d2da7261
Add tests for SLRU cache
2022-08-07 19:16:40 +00:00
AlPerevyshin
d7bdf9e601
Add SLRU cache
2022-08-07 19:16:38 +00:00
avogar
9b1a267203
Refactor, remove TTL, add size limit, add system table and system query
2022-08-05 16:20:15 +00:00
kssenii
91f4143b6e
Merge master
2022-08-05 12:14:08 +02:00
Alexey Milovidov
3130aa7218
Merge pull request #39732 from HarryLeeIBM/hlee-s390x-unaligned
...
Fix Endian issue in SipHash for s390x
2022-08-05 07:24:57 +03:00
kssenii
7a9b0bc47f
Merge master
2022-08-05 01:48:52 +02:00
Alexey Milovidov
24ee944986
Update TaskStatsInfoGetter.cpp
2022-08-04 07:34:36 +03:00
Fangyuan Deng
0bf5c3d1a1
Merge branch 'master' into fix_netlink_leak
2022-08-04 11:42:17 +08:00
pzhdfy
848a349e42
fix potential netlink leak when init fail in TaskStatsInfoGetter
2022-08-04 11:37:27 +08:00
Alexey Milovidov
f474eb957d
Merge pull request #39758 from ClickHouse/tsan_clang_15
...
Try clang-15 for build with tsan
2022-08-04 01:56:52 +03:00
Nikita Mikhaylov
f144eae388
Fix typo and extra dots in exception messages from OverCommitTracker ( #39858 )
2022-08-03 15:23:07 +02:00
Robert Schulze
00a7c8733b
Merge pull request #39633 from guowangy/filter-vbmi2
...
ColumnVector: optimize filter with AVX512VBMI2 compress store
2022-08-03 09:23:24 +02:00
Frank Chen
035896fa16
Fix build check
2022-08-02 20:18:42 +08:00
Frank Chen
1c9b462c80
Merge remote-tracking branch 'origin/master' into tracing_context_propagation
2022-08-02 20:18:22 +08:00
Alexander Tokmakov
b4c4448f0f
fix
2022-08-02 12:07:38 +02:00
Alexander Tokmakov
82b50e79cf
Merge branch 'master' into tsan_clang_15
2022-08-02 13:00:55 +03:00
Frank Chen
57dde8c250
Clean up header inclusion
...
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2022-08-02 15:43:19 +08:00
Alexey Milovidov
316528817b
Merge pull request #39179 from azat/fsync-profile-events
...
Add profile events for fsync
2022-08-02 05:44:35 +03:00
Wangyang Guo
6a7213291b
Merge master and resolve conflict
2022-08-02 10:40:40 +08:00
Frank Chen
40c6e4c0d6
Merge remote-tracking branch 'origin/master' into tracing_context_propagation
2022-08-02 10:02:09 +08:00
Alexander Tokmakov
ecf7ce1f74
Merge branch 'master' into zookeeper_client_fault_injection
2022-08-01 20:49:01 +02:00
Alexander Tokmakov
0d68b1c67f
fix build with clang-15
2022-08-01 18:00:54 +02:00
Robert Schulze
bf574b9154
Merge pull request #39760 from ClickHouse/bit-fiddling
...
Use std::popcount, ::countl_zero, ::countr_zero functions
2022-08-01 17:04:51 +02:00
Alexander Tokmakov
80f9ba9186
Merge pull request #39690 from ClickHouse/show-addresses-in-stack-traces
...
Configuration parameter to hide addresses in stack traces
2022-08-01 14:20:37 +03:00
Wangyang Guo
b05be56eef
ColumnVector: naming style fix
2022-08-01 10:16:15 +08:00
Robert Schulze
d39259a4c0
More conversions
2022-07-31 22:06:02 +00:00
Azat Khuzhin
3e627e2861
Add profile events for fsync
...
The following new provile events had been added:
- FileSync - Number of times the F_FULLFSYNC/fsync/fdatasync function was called for files.
- DirectorySync - Number of times the F_FULLFSYNC/fsync/fdatasync function was called for directories.
- FileSyncElapsedMicroseconds - Total time spent waiting for F_FULLFSYNC/fsync/fdatasync syscall for files.
- DirectorySyncElapsedMicroseconds - Total time spent waiting for F_FULLFSYNC/fsync/fdatasync syscall for directories.
v2: rewrite test to sh with retries
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-31 23:19:30 +03:00
kssenii
e5f4a619ed
Merge master
2022-07-31 20:24:40 +03:00
Robert Schulze
a7734672b9
Use std::popcount, ::countl_zero, ::countr_zero functions
...
- Introduced with the C++20 <bit> header
- The problem with __builtin_c(l|t)z() is that 0 as input has an
undefined result (*) and the code did not always check. The std::
versions do not have this issue.
- In some cases, we continue to use buildin_c(l|t)z(), (e.g. in
src/Common/BitHelpers.h) because the std:: versions only accept
unsigned inputs (and they also check that) and the casting would be
ugly.
(*) https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html
2022-07-31 15:16:51 +00:00
Robert Schulze
52d08d9db4
Merge pull request #39520 from ClickHouse/no-split-binary
...
Remove SPLIT_BINARY
2022-07-31 14:23:31 +02:00
Alexey Milovidov
d285648a37
Merge branch 'master' into shell-command-wait-pid-refactoring
2022-07-31 02:57:46 +03:00
Robert Schulze
85773e0926
Merge pull request #38615 from liyinsg/simplified_function_registration_interface
...
Simplified function registration interface
2022-07-31 00:18:37 +02:00
Alexey Milovidov
c0d7b6efc3
Suggestions from @tavplubix
2022-07-30 01:45:06 +02:00
Alexey Milovidov
4aa96767d5
Merge branch 'master' of github.com:ClickHouse/ClickHouse into show-addresses-in-stack-traces
2022-07-30 01:44:04 +02:00
HarryLeeIBM
cc73b53116
Fix Endian issue in SipHash for s390x
2022-07-29 13:21:50 -07:00
Robert Schulze
3d1797f75f
Merge remote-tracking branch 'origin/master' into no-split-binary
2022-07-29 12:17:43 +00:00
Robert Schulze
a3c52f1140
Merge pull request #39679 from azat/fix-utils
...
Fix utils build on CI
2022-07-29 14:02:53 +02:00
Azat Khuzhin
498c8b3c52
Fix clang-tidy in utils/examples
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-29 11:36:51 +03:00
Li Yin
4088c0a7f3
Automated function registration
...
Automated register all functions with below naming convention by
iterating through the symbols:
void DB::registerXXX(DB::FunctionFactory &)
2022-07-29 15:39:50 +08:00
Alexey Milovidov
552b517e59
Allow to hide addresses in stack traces
2022-07-29 00:13:02 +02:00
Azat Khuzhin
e1f8cff3b4
Add duration_ms into system.zookeeper_log
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-28 22:34:51 +03:00
Robert Schulze
199e254777
Merge remote-tracking branch 'origin/master' into no-split-binary
2022-07-28 15:54:22 +00:00
Antonio Andelic
15e999f8bc
Merge pull request #39096 from ClickHouse/keeper-version-check
...
Add version for Keeper API
2022-07-28 10:10:49 +02:00
Sergei Trifonov
edf6b02723
Merge pull request #37558 from ClickHouse/concurrency-control
...
add concurrency control for pipeline executors
2022-07-28 08:56:24 +02:00
Yakov Olkhovskiy
f1bc767ccf
Merge pull request #39627 from HarryLeeIBM/hlee-s390x-wide_int
...
Fix wide integer unit tests on s390x platform
2022-07-27 21:30:12 -04:00
Alexander Tokmakov
8fc075a527
Merge pull request #39619 from ClickHouse/try-do-enable-global-allocations-for-attach
...
Replace MemoryTrackerBlockerInThread to LockMemoryExceptionInThread
2022-07-27 13:13:04 +03:00
Kruglov Pavel
381ea139c2
Merge branch 'master' into schema-inference-cache
2022-07-27 11:35:36 +02:00
Antonio Andelic
904a05ac21
Merge pull request #39496 from azat/custom-tld-exclamation-asterisk
...
Add support of !/* (exclamation/asterisk) in custom TLDs
2022-07-27 08:55:49 +02:00
Wangyang Guo
e6752d687f
TargetSpecific: add AVX512VBMI2 support
2022-07-27 13:30:43 +08:00
Wangyang Guo
50fdbcdc12
CpuId: add AVX512VBMI2 detection
2022-07-27 13:30:43 +08:00
HarryLeeIBM
d475086100
Fix wide integer unit test
2022-07-26 12:57:45 -07:00
Nikolai Kochetov
f1818a9103
Make only one possible public ctor in MemoryTrackerBlockerInThread
2022-07-26 19:04:20 +00:00
Nikolai Kochetov
dcb86eca0b
Make only one possible public ctor in MemoryTrackerBlockerInThread
2022-07-26 19:03:26 +00:00
Nikolai Kochetov
fa0707b4e7
Review fixes.
2022-07-26 17:51:09 +00:00
Nikolai Kochetov
33ce662d3e
Replace MemoryTrackerBlockerInThread to LockMemoryExceptionInThread in some places. Reduced MemoryTrackerBlockerInThread level to User.
2022-07-26 15:22:00 +00:00
Kruglov Pavel
c683cb252f
Merge pull request #39227 from amosbird/rename-log1
...
Rename log when rename merge tree tables
2022-07-26 17:12:44 +02:00
Sergei Trifonov
24ab5fbb86
fix finish() condition to account only active threads
2022-07-26 11:32:45 +02:00
Antonio Andelic
35b61cc94e
Merge branch 'master' into keeper-version-check
2022-07-26 07:47:57 +00:00
Azat Khuzhin
1d4a7c7290
Add support of !/* (exclamation/asterisk) in custom TLDs
...
Public suffix list may contain special characters (you may find format
here - [1]):
- asterisk (*)
- exclamation mark (!)
[1]: https://github.com/publicsuffix/list/wiki/Format
It is easier to describe how it should be interpreted with an examples.
Consider the following part of the list:
*.sch.uk
*.kawasaki.jp
!city.kawasaki.jp
And here are the results for `cutToFirstSignificantSubdomainCustom()`:
If you have only asterisk (*):
foo.something.sheffield.sch.uk -> something.sheffield.sch.uk
sheffield.sch.uk -> sheffield.sch.uk
If you have exclamation mark (!) too:
foo.kawasaki.jp -> foo.kawasaki.jp
foo.foo.kawasaki.jp -> foo.foo.kawasaki.jp
city.kawasaki.jp -> city.kawasaki.jp
some.city.kawasaki.jp -> city.kawasaki.jp
TLDs had been verified wit the following script [2], to match with
python publicsuffix2 module.
[2]: https://gist.github.com/azat/c1a7a9f1e3519793134ef4b1df5461a6
v2: fix StringHashTable padding requirements
Fixes : #39468
Follow-up for: #17748
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-26 08:34:30 +03:00
Roman Vasin
b462366415
Merge branch 'master' of github.com:ClickHouse/ClickHouse into ADQM-485
2022-07-25 17:55:47 +00:00
Roman Vasin
b412ea5f6d
Improve generateRandom() for Date32; fix tests 01087_table_function_generate, 01277_fromUnixTimestamp64, 01691_DateTime64_clamp and 01702_toDateTime_from_string_clamping
2022-07-25 17:06:11 +00:00
Sergei Trifonov
d9d1381c3f
Merge branch 'master' into concurrency-control
2022-07-25 14:16:46 +02:00
mergify[bot]
f667eff025
Merge branch 'master' into shell-command-wait-pid-refactoring
2022-07-25 11:42:05 +00:00
Robert Schulze
24c74da473
Merge pull request #39430 from azat/getauxval-fix
...
Fix LSan by fixing getauxval() (resubmit v4)
2022-07-25 10:43:09 +02:00
Alexey Milovidov
388d06fda1
Merge pull request #39535 from ClickHouse/stringref
...
Less usage of StringRef
2022-07-25 04:06:11 +03:00
Azat Khuzhin
93f876b58a
Add a test for LSan
...
(cherry picked from commit 51e7c41883
)
v2: fix type check
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-25 01:21:40 +03:00
Robert Schulze
4333750985
Less usage of StringRef
...
... replaced by std::string_view, see #39262
2022-07-24 18:33:52 +00:00
Robert Schulze
73c04b64e3
Merge pull request #39493 from nicelulu/fix_build_osx
...
Fix build on mac osx
2022-07-24 18:11:34 +02:00
Robert Schulze
c788e05c77
Merge pull request #39292 from zvonand/zvonand-b58-datatype
...
Simplify Base58 encoding/decoding
2022-07-24 18:09:40 +02:00
Andrey Zvonov
032fcab70e
set char to be explicitely signed
2022-07-24 14:23:22 +02:00
Alexey Milovidov
071374b152
Remove SPLIT_BINARY
2022-07-24 01:15:54 +02:00
Constantine Peresypkin
d645d03e89
fix -DENABLE_EXAMPLES=1
in master
2022-07-23 19:00:52 +02:00
Antonio Andelic
6aff87d4b5
Ignore system paths from snapshots and logstore
2022-07-23 13:46:59 +00:00
Antonio Andelic
46a6fbd7aa
Small polish
2022-07-22 11:03:07 +00:00
zhangxiao871
0d4d1e66b5
Fix build
2022-07-22 18:53:44 +08:00
Alexander Tokmakov
bed2206ae9
Merge pull request #39460 from ClickHouse/remove_some_dead_and_commented_code
...
Remove some dead and commented code
2022-07-22 13:24:34 +03:00
Roman Vasin
0b102c6d1f
Fix code style
2022-07-22 08:24:05 +00:00
Antonio Andelic
3040ff0959
Merge branch 'master' into keeper-version-check
2022-07-22 08:07:55 +00:00
Roman Vasin
74d2bf8d95
Fix gTest for DateLUTTest
2022-07-22 07:49:02 +00:00
Kruglov Pavel
9252f42b4c
Merge branch 'master' into schema-inference-cache
2022-07-21 18:59:14 +02:00
Robert Schulze
ea0a3bf600
Merge branch 'master' into stringref-to-string_view
2022-07-21 18:33:06 +02:00
Maksim Kita
0159fe1274
Merge branch 'master' into shell-command-wait-pid-refactoring
2022-07-21 16:38:28 +02:00
Andrey Zvonov
e473606cd1
Merge branch 'master' into zvonand-b58-datatype
2022-07-21 15:55:05 +02:00
Alexander Tokmakov
9e9969cea7
Merge pull request #37827 from arthurpassos/host_regexp_multiple_domains
...
Test host_regexp against all PTR records instead of only one
2022-07-21 16:43:05 +03:00
Alexander Tokmakov
a8da5d96fc
remove some dead and commented code
2022-07-21 15:05:48 +02:00
Maksim Kita
7f4a1b8bb8
ShellCommand wait pid refactoring
2022-07-21 11:45:09 +02:00
Nikolai Kochetov
e15967e9db
Merge pull request #38475 from ClickHouse/additional-filters
...
Additional filters for a table (from setting)
2022-07-21 07:52:04 +02:00
Alexey Milovidov
4305eacd92
Merge pull request #39434 from ClickHouse/revert-39299-fix-lsan-v3
...
Revert "[RFC] Fix LSan by fixing getauxval()"
2022-07-21 06:47:54 +03:00
Alexey Milovidov
f4f5a5e044
Merge pull request #39397 from azat/fix-extractAll
...
Fix UB (stack-use-after-scope) in extactAll()
2022-07-21 05:21:18 +03:00
Alexey Milovidov
72dda35108
Revert "[RFC] Fix LSan by fixing getauxval()"
2022-07-21 04:34:04 +03:00
Nikolai Kochetov
91043351aa
Fixing build.
2022-07-20 20:30:16 +00:00
Kseniia Sumarokova
3759ee76a7
Merge pull request #39386 from ClickHouse/kssenii-patch-4
...
Rethrow exception on cache initialisation on server startup, better error message
2022-07-20 21:26:25 +02:00
Roman Vasin
e3192cf753
Correct docs to reflect new range 1900..2299 for Date32 and DateTime64; Cleanup code
2022-07-20 15:19:02 +00:00
zvonand
11b8d788ca
small improvements
2022-07-20 13:36:47 +02:00
Kruglov Pavel
3046cd6d29
Merge branch 'master' into schema-inference-cache
2022-07-20 13:30:42 +02:00
Kseniia Sumarokova
0840dfe2e0
Update LRUFileCache.cpp
2022-07-20 11:42:24 +02:00
Maksim Kita
d3ce4ffc88
Merge pull request #38929 from pkit/pkit/wait_pid_fix
...
fix long wait for process exit in ShellCommand
2022-07-20 11:26:38 +02:00
Azat Khuzhin
18eb8b6d48
Fix UB (stack-use-after-scope) in extactAll()
...
After #37544 OptimizedRegularExpressionImpl started to be moved, but
StringSearcher is not copyable since it holds pointers that goes out of
scope after move (before Regexps::get() returns std::shared_ptr<Regexp>
but it had been replaced with Regexps::createRegexp() that returns
Regexp object).
<details>
<summary>ASan report</summary>
==48348==ERROR: AddressSanitizer: stack-use-after-scope on address 0x7fff577239a9 at pc 0x00001518209b bp 0x7fff57723820 sp 0x7fff57723818
READ of size 1 at 0x7fff577239a9 thread T0
0 0x1518209a in char8_t const* DB::StringSearcher<true, true>::search<char8_t>(char8_t const*, char8_t const*) const /bld/./src/Common/StringSearcher.h:730:41
1 0x1518dd3f in char8_t const* DB::StringSearcher<true, true>::search<char8_t>(char8_t const*, unsigned long) const /bld/./src/Common/StringSearcher.h:751:16
2 0x1518dd3f in OptimizedRegularExpressionImpl<false>::match(char const*, unsigned long, std::__1::vector<OptimizedRegularExpressionDetails::Match, std::__1::allocator<OptimizedRegularExpressionDetails::Match> >&, unsigned int) const /bld/./src/Common/OptimizedRegularExpression.cpp:463:54
3 0x1811cb42 in DB::ExtractAllImpl::get(char const*&, char const*&) /bld/./src/Functions/FunctionsStringArray.h:588:18
4 0x1811aa62 in DB::FunctionTokens<DB::ExtractAllImpl>::executeImpl(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long) const /bld/./src/Functions/FunctionsStringArray.h:704:30
5 0x14fe17b4 in DB::IFunction::executeImplDryRun(std::__1::vector<DB::ColumnWithTypeAndName, std::__1::allocator<DB::ColumnWithTypeAndName> > const&, std::__1::shared_ptr<DB::IDataType const> const&, unsigned long) const /bld/./src/Functions/IFunction.h:409:16
Address 0x7fff577239a9 is located in stack of thread T0 at offset 201 in frame
0 0x1518d98f in OptimizedRegularExpressionImpl<false>::match(char const*, unsigned long, std::__1::vector<OptimizedRegularExpressionDetails::Match, std::__1::allocator<OptimizedRegularExpressionDetails::Match> >&, unsigned int) const /bld/./src/Common/OptimizedRegularExpression.cpp:439
</details>
CI: https://s3.amazonaws.com/clickhouse-test-reports/39342/c6f7698f9ad6ae22199182ebf7c3b2dac77d69d8/fuzzer_astfuzzerasan,actions//report.html
Fixes : #37544 (cc @rschu1ze)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-20 10:18:44 +03:00
Roman Vasin
5f9c293963
Fix addDays() and addWeeks() in upper and lower limits of Date and Date32
2022-07-19 17:29:08 +00:00
Kseniia Sumarokova
891dbccdc7
Update LRUFileCache.cpp
2022-07-19 19:18:42 +02:00
Robert Schulze
02ed126533
Update src/Common/TraceSender.cpp
...
Co-authored-by: Nikolay Degterinsky <43110995+evillique@users.noreply.github.com>
2022-07-19 18:44:31 +02:00
Sergei Trifonov
499818751e
Merge pull request #39280 from ClickHouse/avg-cpu-progress
...
fix cpu usage metric in client
2022-07-19 15:17:44 +02:00
Kseniia Sumarokova
2e3ceb351e
Update LRUFileCache.cpp
2022-07-19 14:44:56 +02:00
Kseniia Sumarokova
f7bb69ef7a
Update LRUFileCache.cpp
2022-07-19 14:43:11 +02:00
Kseniia Sumarokova
af7acb1782
Update LRUFileCache.cpp
2022-07-19 14:41:31 +02:00
Kseniia Sumarokova
5ad4e9d28f
Update LRUFileCache.cpp
2022-07-19 14:33:50 +02:00
Robert Schulze
7146685f9d
Merge pull request #39299 from azat/fix-lsan-v3
...
[RFC] Fix LSan by fixing getauxval()
2022-07-19 14:27:41 +02:00
Constantine Peresypkin
246614229f
fix long wait for process exit in ShellCommand
...
fixes #38889
2022-07-19 14:07:27 +02:00
kssenii
21ebf8874e
Merge remote-tracking branch 'upstream/master' into make-cache-composable
2022-07-19 13:36:42 +02:00
Nikolai Kochetov
fc6982f9e2
Fixing test.
2022-07-19 10:10:02 +00:00
Antonio Andelic
ce570b6ee3
Add logs and 4LW for api version
2022-07-19 09:02:57 +00:00
Antonio Andelic
e6ded88ea3
Small refactoring
2022-07-19 08:51:12 +00:00
Robert Schulze
81ef1099cc
Even less usage of StringRef
...
--> see #39300
2022-07-19 07:01:06 +00:00
Nikolai Kochetov
5a3cb07711
Fix rss
2022-07-18 21:24:41 +00:00
Nikolai Kochetov
3ee3144708
Merge branch 'master' into separate-counter-for-rss-in-memory-tracker
2022-07-18 23:20:38 +02:00
Nikolai Kochetov
87e5b31598
Do not include memory buffered by allocator into drift.
2022-07-18 21:16:16 +00:00
Robert Schulze
6df3c9d799
Merge pull request #39300 from ClickHouse/stringref-to-stringview
...
First try at reducing the use of StringRef
2022-07-18 19:20:16 +02:00
Roman Vasin
28fd774df8
Correct 01821_to_date_time_ubsan and 01921_datatype_date32 tests
2022-07-18 16:44:14 +00:00
Antonio Andelic
cfc741030f
Revert to the version with path
2022-07-18 14:30:15 +00:00
kssenii
0aed5a602a
Merge master
2022-07-18 16:16:28 +02:00
Azat Khuzhin
51e7c41883
Add a test for LSan
...
v2: fix type check
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-18 16:49:49 +03:00
Sergei Trifonov
30cb4e168e
cleanup
2022-07-18 13:31:32 +02:00
Sergei Trifonov
0ffe5bd39c
fix typo and style
2022-07-18 13:26:33 +02:00
Sergei Trifonov
2593dbcaa6
reuse Common/ExponentiallySmoothedCounter.h
2022-07-18 12:21:44 +02:00
Antonio Andelic
758a61a36f
Close socket if ApiVersion wasn't processed
2022-07-18 09:43:21 +00:00
Antonio Andelic
5170ac0898
Merge branch 'master' into keeper-version-check
2022-07-18 08:57:50 +00:00
Antonio Andelic
b8ffb151dc
Some small polishing
2022-07-18 08:52:52 +00:00
Antonio Andelic
742e3d7643
Merge branch 'master' into keeper-version-check
2022-07-18 08:45:00 +00:00
Antonio Andelic
7d7c4ce4cd
Add ApiVersion request
2022-07-18 08:44:38 +00:00
Alexey Milovidov
f1256c3aef
Merge pull request #37891 from guowangy/lz4-decompress-vbmi
...
LZ4 decompress: add VBMI optimized copyOverlap32Shuffle
2022-07-18 00:37:59 +03:00
Robert Schulze
13482af4ee
First try at reducing the use of StringRef
...
- to be replaced by std::string_view
- suggested in #39262
2022-07-17 17:26:02 +00:00
Robert Schulze
fffeab2e86
Merge pull request #39262 from ClickHouse/stringref-by-value
...
Pass const StringRef by value, not by reference
2022-07-17 16:53:15 +02:00