serxa
23defcbf6a
log progress in AsyncLoader
2023-05-01 12:08:50 +00:00
Sergei Trifonov
b390a07698
Merge branch 'master' into async-loader
2023-05-01 14:04:07 +02:00
Nikolay Degterinsky
f64ef88197
Merge remote-tracking branch 'upstream/master' into bcrypt
2023-04-30 21:39:16 +00:00
alesapin
ec2c860a39
Merge branch 'fix_zero_copy_not_atomic' of github.com:ClickHouse/ClickHouse into fix_zero_copy_not_atomic
2023-04-30 15:02:27 +02:00
alesapin
daa0b095ed
Merge branch 'master' into fix_zero_copy_not_atomic
2023-04-29 21:26:26 +02:00
Kseniia Sumarokova
d4aa96e262
Merge branch 'master' into better-local-object-storage
2023-04-28 21:05:05 +02:00
Robert Schulze
7609832606
Unpoison stack frame ptrs from libunwind for msan
...
Turned out that in msan builds, stack frame pointers explicitly returned
by unw_backtrace() in Common/StackTrace.cpp were unpoisened, but stack
frame pointers returned from the same function under the
STD_EXCEPTION_HAS_STACK_TRACE define in std::exception (see (*)) were
not. I tried unpoisoning directly in libcxx. Since this didn't work,
this PR unpoisons where get_stack_trace_frames() is called.
I don't know why this issue never occurred in CI but it fixes #48385 for
me (where msan complains about reads of uninitialized data when query
cache squashes and compresses chunks produced from SELECT last_error_trace
FROM system.errors).
Fixes : #48385
(*) https://github.com/ClickHouse/llvm-project/pull/7/files#diff-8c81d74ef72967be2fa27a31275000aa18532a583490f9673c62d7b981a5b39c
2023-04-28 17:43:12 +00:00
Alexey Milovidov
cd171bef7d
Merge pull request #45999 from sichenzhao/cgroup_v2
...
Updates AsyncMetrics cgroup-memory to support cgroupv2
2023-04-28 20:39:53 +03:00
alesapin
70ee02a3eb
Add node holder
2023-04-28 17:39:32 +02:00
Alexey Milovidov
c86d3daee5
Update AsynchronousMetrics.cpp
2023-04-28 16:21:37 +03:00
alesapin
ef49603432
Merge branch 'master' into fix_zero_copy_not_atomic
2023-04-28 14:42:30 +02:00
kssenii
298ed454ce
Merge remote-tracking branch 'upstream/master' into better-local-object-storage
2023-04-28 13:26:55 +02:00
Alexey Milovidov
4dccd5f9b0
Update AsynchronousMetrics.cpp
2023-04-27 23:20:38 +03:00
Nikolay Degterinsky
df1e56b985
Merge remote-tracking branch 'upstream/master' into bcrypt
2023-04-27 17:01:58 +00:00
Robert Schulze
b6e65ddeff
Merge branch 'master' into add-examples-to-functions
2023-04-27 16:11:32 +02:00
Kseniia Sumarokova
45e2d296f9
Merge branch 'master' into better-local-object-storage
2023-04-27 14:54:04 +02:00
Alexey Milovidov
c5793f9684
Fix build
2023-04-27 14:56:51 +03:00
Sergei Trifonov
85d7816d65
Update src/Common/AsyncLoader.h
2023-04-27 13:55:26 +02:00
Robert Schulze
096667be45
More fixes
2023-04-27 09:33:52 +00:00
Alexey Milovidov
1bb685a081
Fix error.
2023-04-27 01:32:54 +03:00
Robert Schulze
739bfc9b8d
Fix categories
2023-04-26 21:23:26 +00:00
Robert Schulze
d72030b876
Various fixes
2023-04-26 20:38:41 +00:00
serxa
94f730ded0
Merge branch 'async-loader' of github.com:ClickHouse/ClickHouse into async-loader
2023-04-26 18:40:26 +00:00
serxa
173a8a3795
fix tidy build
2023-04-26 18:39:51 +00:00
alesapin
f3d1d3b58b
Merge branch 'master' into fix_zero_copy_not_atomic
2023-04-26 18:00:31 +02:00
Sergei Trifonov
bb4109b3e1
Merge branch 'master' into async-loader
2023-04-26 15:00:59 +02:00
Alexey Milovidov
88ce78a7be
Merge branch 'master' into cgroup_v2
2023-04-26 13:55:12 +03:00
alesapin
e5f71c3b74
Fix zero copy not atomic
2023-04-26 00:25:04 +02:00
alesapin
3a24f0b6c6
fxi
2023-04-25 15:33:41 +02:00
Anton Popov
cc5acfbe62
add profile event
2023-04-25 01:37:37 +00:00
Alexey Milovidov
dee0b78303
Merge pull request #48745 from Algunenano/arenita
...
Don't count unused arena bytes as read_bytes in AggregateFunction types
2023-04-25 00:53:11 +03:00
Alexey Milovidov
761ed890a7
Update AsynchronousMetrics.cpp
2023-04-25 00:45:57 +03:00
Anton Popov
38622d0770
add settings to delay or throw in case of too many mutations
2023-04-24 18:21:49 +00:00
alesapin
d28b71e69d
Merge pull request #49078 from ClickHouse/fix-false-report
...
Prevent false positive report by static analyzer
2023-04-24 13:08:37 +02:00
Robert Schulze
ded8eca041
Merge pull request #48284 from ClickHouse/rs/qc-quota
...
Query Cache: Allow per-user quotas
2023-04-24 11:32:32 +02:00
Vladimir C
6090648fc8
Merge pull request #43606 from arthurpassos/parsey_key_value_function
2023-04-24 11:03:28 +02:00
Raúl Marín
868b4e41fd
Arena used bytes
2023-04-24 10:46:23 +02:00
Raúl Marín
f0e045bb3d
Merge remote-tracking branch 'blessed/master' into arenita
2023-04-24 10:42:56 +02:00
Alexey Milovidov
ff5ec30b0b
Merge pull request #49005 from ClickHouse/fix-ibm
...
Reimplement #48986
2023-04-24 01:39:22 +03:00
Alexey Milovidov
f4bd5a0667
Prevent false positive report by static analyzer
2023-04-23 22:45:12 +02:00
Alexey Milovidov
c3749d652e
Update AsynchronousMetrics.cpp
2023-04-23 23:24:57 +03:00
Alexey Milovidov
40bae4b611
Merge branch 'master' into cgroup_v2
2023-04-23 21:46:05 +03:00
Robert Schulze
71c4e3dbf4
A bit more memory-safe
2023-04-23 18:13:51 +00:00
Robert Schulze
38d5f12a32
Query Cache: Implement per-user quotas
2023-04-23 16:13:51 +00:00
Alexey Milovidov
c42e1ad131
Merge pull request #48922 from RunningXie/fix_reopen_stale_fd
...
Implement a slight improvement for the rare case when ClickHouse is run inside LXC and LXCFS is used, because LXCFS has a bug
2023-04-23 16:12:49 +03:00
kssenii
d8023806a9
Merge remote-tracking branch 'upstream/master' into better-local-object-storage
2023-04-23 12:39:34 +02:00
Alexey Milovidov
1be5a20546
Simplify the code
2023-04-23 09:01:21 +03:00
Alexey Milovidov
1087c7361d
Update AsynchronousMetrics.cpp
2023-04-23 08:51:17 +03:00
Alexey Milovidov
28e03f5781
Merge branch 'master' into fix-ibm
2023-04-23 06:01:41 +02:00
Sergei Trifonov
7fa5ccb53c
Merge branch 'master' into async-loader
2023-04-22 19:44:44 +02:00
serxa
056268f90b
fix
2023-04-22 17:43:36 +00:00
serxa
64fb8da812
improve AsyncLoader interface
2023-04-22 17:29:53 +00:00
Alexey Milovidov
5fd42a2220
Add ProfileEvents for log messages
2023-04-22 14:55:25 +02:00
Alexey Milovidov
4c7f747fae
Update AsynchronousMetrics.cpp
2023-04-22 14:51:21 +03:00
Dmitry Novik
e23105b322
Merge remote-tracking branch 'origin/master' into revert-47476-revert_46622
2023-04-21 16:40:04 +00:00
Alexey Milovidov
a6fceaa497
Update AsynchronousMetrics.cpp
2023-04-21 13:51:13 +03:00
kssenii
e58c16318f
Merge remote-tracking branch 'upstream/master' into better-local-object-storage
2023-04-21 12:49:40 +02:00
Alexey Milovidov
252c755323
Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-ibm
2023-04-21 12:39:13 +02:00
Alexey Milovidov
530b764953
Fix IBM
2023-04-21 12:38:45 +02:00
Alexey Milovidov
9d6c3d7a4c
Merge pull request #48920 from awfeequdng/bugfix/maskLowBits
...
A non significant change (does not affect anything): add support for signed integers in the maskBits function
2023-04-21 13:22:44 +03:00
Kruglov Pavel
9749448eb0
Merge pull request #47229 from Avogar/non-blocking-connect
...
Improve async reading from socket
2023-04-21 10:38:50 +02:00
pengxiangcai
41cf29179d
typename spelling error: unsigedT -> UnsignedT
2023-04-21 08:51:58 +08:00
Alexey Milovidov
98ae9be734
Revert "Added tests for ClickHouse apps help and fixed help issues"
2023-04-21 01:54:34 +03:00
Arthur Passos
43841a6ee6
Merge branch 'master' into parsey_key_value_function
2023-04-20 10:05:34 -03:00
Antonio Andelic
31548ab17c
Fix result
2023-04-20 12:30:24 +00:00
Antonio Andelic
09156ab2b7
Merge branch 'master' into keeper-check-not-exists
2023-04-20 12:07:16 +00:00
avogar
39ba4c27bc
Fix deadlock, add profile event
2023-04-20 11:56:20 +00:00
Sergei Trifonov
4b51f12479
Update src/Common/AsyncLoader.h
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2023-04-19 13:48:38 +02:00
serxa
5390fccf50
Merge branch 'async-loader' of github.com:ClickHouse/ClickHouse into async-loader
2023-04-19 11:47:08 +00:00
serxa
2e31140ca4
review fixes
2023-04-19 11:46:56 +00:00
Kruglov Pavel
2ad161d2b7
Merge branch 'master' into non-blocking-connect
2023-04-19 13:39:40 +02:00
Sergei Trifonov
31e3a3bc0c
Update src/Common/AsyncLoader.h
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2023-04-19 13:35:58 +02:00
Antonio Andelic
58e9b56fcb
Fix CheckNotExists
2023-04-19 09:06:20 +00:00
serxa
ff7444f077
typo
2023-04-19 06:27:17 +00:00
serxa
c672d117ea
better job shutdown
2023-04-19 06:27:07 +00:00
serxa
8cc9b32af1
fix more comments
2023-04-19 06:26:59 +00:00
serxa
d4daea5f09
fix typos
2023-04-19 06:26:51 +00:00
serxa
baced2c66e
fix data race in test
2023-04-19 06:26:35 +00:00
serxa
326c733be6
fix tidy build
2023-04-19 06:26:27 +00:00
serxa
54c5b1083b
add test for dynamic prioritization
2023-04-19 06:26:14 +00:00
serxa
911c46aa32
fix double finish() during cancellation of a job with multiple paths
2023-04-19 06:26:06 +00:00
serxa
0d7e3d4100
fix comment
2023-04-19 06:25:54 +00:00
serxa
356b934835
split AsyncLoader into .h and .cpp files + more docs
2023-04-19 06:25:45 +00:00
Sergei Trifonov
1efacba196
Update src/Common/AsyncLoader.h
2023-04-19 06:25:37 +00:00
Sergei Trifonov
281b72c413
Update src/Common/AsyncLoader.h
2023-04-19 06:25:28 +00:00
serxa
1a3780ed8e
fix
2023-04-19 06:25:11 +00:00
serxa
800efd5df8
add a good comment
2023-04-19 06:24:50 +00:00
serxa
69719c0819
allow to change max number of workers in runtime
2023-04-19 06:24:42 +00:00
serxa
c736d9fd4f
fix test
2023-04-19 06:24:35 +00:00
serxa
264956fece
add test for static priorities
2023-04-19 06:24:25 +00:00
serxa
b86c737450
make performance-inefficient-vector-operation,-warnings-as-errors happy
2023-04-19 06:24:15 +00:00
serxa
8c1f734f50
add random test with dependent tasks
2023-04-19 06:24:07 +00:00
serxa
d8bf775b9a
fix worker shutdown
2023-04-19 06:24:01 +00:00
serxa
51992ae96d
add ScheduleJobWithCanceledDependencies test
2023-04-19 06:23:47 +00:00
serxa
5837b09880
fix scheduling of jobs with dependencies on finished jobs
2023-04-19 06:23:39 +00:00
serxa
5ba1dc91e3
add LoadStatus::CANCELED
2023-04-19 06:23:32 +00:00
serxa
df3c5212ff
test deps are completed + refactoring
2023-04-19 06:23:22 +00:00
serxa
9ac37ca3b8
add overload test
2023-04-19 06:23:14 +00:00
serxa
4bc52a1b06
add concurrency test
2023-04-19 06:23:07 +00:00
serxa
ba67f95525
allow cancels on RandomTasks test shutdown
2023-04-19 06:22:37 +00:00
serxa
3be0645413
fix remove of executing tasks + add test
2023-04-19 06:22:27 +00:00
serxa
a9c51c4aca
add more tests for cancel
2023-04-19 06:22:15 +00:00
serxa
5e5a38cd64
add tests for job status/wait/cancel
2023-04-19 06:22:01 +00:00
serxa
3226eadd94
add simple randomized test
2023-04-19 06:21:31 +00:00
serxa
2c29c0b2f3
add dependency cycle detection
2023-04-19 06:21:19 +00:00
serxa
b4318b4ab6
fix style
2023-04-19 06:21:08 +00:00
serxa
5d11706929
make t work, add simple test
2023-04-19 06:20:37 +00:00
serxa
3fe36a3db6
wip AsyncLoader
2023-04-19 06:20:17 +00:00
caipengxiang
c9848e0822
bugfix:
...
maskLowBits<UInt8>(3) => 00000111, but maskLowBits<Int8>(3) => 11111111, so I fixed it.
after I fixed it, the result is:
maskLowBits<UInt8>(3) => 00000111, maskLowBits<Int8>(3) => 00000111
2023-04-19 12:22:17 +08:00
xieyichen
11b3469eac
reopen stale fd
2023-04-19 11:13:16 +08:00
Ilya Yatsishin
ebb1b990f2
Merge pull request #45819 from qoega/clickhouse-help
2023-04-18 21:46:36 +02:00
Antonio Andelic
fb16623d48
Add CheckNotExists request to Keeper
2023-04-18 14:45:24 +00:00
kssenii
16b027ed0c
Merge remote-tracking branch 'upstream/master' into better-local-object-storage
2023-04-18 16:25:08 +02:00
Kruglov Pavel
cd88024a33
Merge pull request #48556 from ClickHouse/fix-nested-map-ip-uuid
...
Fix nested map for keys of IP and UUID types
2023-04-18 13:03:50 +02:00
Alexander Gololobov
423193d619
Merge branch 'master' into fix_zookeeper_join_race
2023-04-17 20:50:51 +02:00
kssenii
a470deafbc
Fix style check
2023-04-17 19:11:42 +02:00
Alexander Gololobov
0493b2cbce
A wrapper to allow calling join() from multiple threads
2023-04-17 15:01:45 +02:00
Raúl Marín
39f8c43a60
Merge remote-tracking branch 'blessed/master' into arenita
2023-04-17 10:33:38 +02:00
Michael Kolupaev
e133633359
Parallel decoding with one row group per thread
2023-04-17 04:58:32 +00:00
Michael Kolupaev
2d4fe85513
Something
2023-04-17 04:58:32 +00:00
Dmitry Novik
a1f21a5fc4
Do not log peak for background memory tracker
2023-04-14 16:49:14 +02:00
Dmitry Novik
cf5d9a175a
Revert "Merge pull request #48760 from ClickHouse/revert-46089-background-memory-tracker"
...
This reverts commit a61ed33223
, reversing
changes made to 5f01b8a2b5
.
2023-04-14 16:34:19 +02:00
Sergei Trifonov
7cb6303e4c
Merge branch 'master' into lower-threadpool-mutex-contention
2023-04-14 03:26:39 +02:00
Alexander Tokmakov
af1bf08663
Revert "Add MemoryTracker for the background tasks"
2023-04-13 21:05:02 +03:00
serxa
22b75ec81b
fix typo, add coment, dont reset trace
2023-04-13 15:17:39 +00:00
Arthur Passos
88f1c969cc
Merge branch 'master' into parsey_key_value_function
2023-04-13 10:38:44 -03:00
Sergei Trifonov
c51f1359be
Merge branch 'master' into lower-threadpool-mutex-contention
2023-04-13 14:21:45 +02:00
serxa
32f78afd7e
get rid of redundant code
2023-04-13 12:11:06 +00:00
serxa
ce2073e58f
detach_thread() and notifications order does not matter
2023-04-13 11:45:19 +00:00
Raúl Marín
e239df4c25
Keep count to avoid iterating over lists
2023-04-13 13:41:44 +02:00
Dmitry Novik
467ecf45e1
Merge pull request #46089 from ClickHouse/background-memory-tracker
...
Add MemoryTracker for the background tasks
2023-04-13 13:29:27 +02:00
serxa
5e8339c2d9
Lock ThreadPool mutex once per job instead of twice
2023-04-13 11:27:11 +00:00
Raúl Marín
2b70e08f23
Don't count unreserved bytes in Arenas as read_bytes
2023-04-13 12:43:24 +02:00
Raúl Marín
da9a539cf7
Reduce the usage of Arena.h
2023-04-13 10:31:32 +02:00
Robert Schulze
22c3b10e41
Merge pull request #48628 from ClickHouse/rs/var_uint_max
...
Better handling of values too large for VarInt encoding
2023-04-13 10:17:55 +02:00
Robert Schulze
f41354ccd6
Merge pull request #48671 from ClickHouse/rs/gcc-removal
...
Remove GCC remainders
2023-04-13 10:15:35 +02:00
Robert Schulze
3f7ce60e03
Merge branch 'master' into rs/gcc-removal
2023-04-12 22:17:04 +02:00
Azat Khuzhin
37d430d68a
Use ThreadPool in PipelineExecutor
...
PipelineExecutor is one of the heavy user of the threads, by using pool
there number of used threads for query execution can be tracked in
system.metrics.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-12 20:00:51 +02:00
Robert Schulze
05606a8835
Clean up GCC warning pragmas
2023-04-11 18:21:08 +00:00
Dmitry Novik
52f3406bbb
Fix CurrentMetrics::MergesMutationsMemoryTracking value
2023-04-11 16:10:59 +00:00
Dmitry Novik
06e6794fc0
Merge remote-tracking branch 'origin/master' into background-memory-tracker
2023-04-11 15:29:35 +00:00
Yatsishin Ilya
b5b65d2149
Merge remote-tracking branch 'origin' into clickhouse-help
2023-04-11 11:24:48 +00:00
Sema Checherinda
4dd86a406a
Merge pull request #48543 from azat/mv-uniq-thread-group
...
Use one ThreadGroup while pushing to materialized views (and some refactoring for ThreadGroup)
2023-04-11 11:47:46 +02:00
Robert Schulze
bbbb55cd50
Better handling of values too large for VarInt encoding
...
PR #48154 introduced a sanity check in the form of a debug assertion
that the input values for VarInt encoding are not too big. Such values
should be exceptionally rare in practice but the AST fuzzer managed to
trigger the assertion regardless. The strategy to deal with such values
until now was to bypass the check by limiting the value to the maximum
allowed value (see #48412 ). Because a new AST Fuzzer failure appeared
(#48497 ) and there may be more failures in future, this PR changes the
sanity check from an assert to an exception.
Fixes : #48497
2023-04-11 07:47:33 +00:00
Azat Khuzhin
79b83c4fd2
Remove superfluous includes of logger_userful.h from headers
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-10 17:59:30 +02:00
Arthur Passos
b062ab5b4f
Merge branch 'master' into parsey_key_value_function
2023-04-10 10:32:01 -03:00
Alexey Milovidov
6292a08c9c
Add a test
2023-04-09 03:25:38 +02:00
Alexey Milovidov
09ea79aaf7
Add support for {server_uuid} macro
2023-04-09 03:04:26 +02:00
Alexey Milovidov
5495cb2935
Merge pull request #48521 from ClibMouse/feature/tiny-improvement
...
Use std::string::starts_with instead of a roll your own variant
2023-04-08 22:19:48 +03:00
Yakov Olkhovskiy
054078e06d
fix nested map for keys of IP and UUID types
2023-04-08 05:34:35 +00:00
Vitaly Baranov
62e9c419de
Merge pull request #48519 from azat/build/fwd-decl-threadpool
...
Use forward declaration of ThreadPool
2023-04-08 00:44:01 +02:00
Antonio Andelic
55a999d6a3
Merge branch 'master' into fix-topk-ubsan
2023-04-07 20:05:36 +02:00
Azat Khuzhin
aacf2a0838
Move ThreadGroupSwitcher to ThreadStatus.h (out from MergeTree code)
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-07 15:32:08 +02:00
Azat Khuzhin
5b2b20a0b0
Rename ThreadGroupStatus to ThreadGroup
...
There are methods like getThreadGroup() and ThreadGroupSwitcher class,
so seems that this is logical.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-07 15:31:48 +02:00
alesapin
a0dd1c96bf
Merge pull request #48512 from ClickHouse/serxa-patch-3
...
Fix bytesSize() of zk SetRequest
2023-04-07 11:47:31 +02:00
Azat Khuzhin
011480924a
Use forward declaration of ThreadPool
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-07 11:25:35 +02:00
Antonio Andelic
67e6335250
Fix nullptr to memcpy
2023-04-07 08:32:01 +00:00
robot-clickhouse-ci-2
2cb8252676
Merge pull request #48487 from azat/memory-tracker-fix-deadlock
...
Fix deadlock due to debug tracking of memory allocations
2023-04-06 23:50:44 +02:00
ltrk2
790e570af6
Use std::string::starts_with instead of a roll your own variant
2023-04-06 11:57:44 -07:00
Sergei Trifonov
b6975d36e9
Fix bytesSize() of zk SetRequest
2023-04-06 19:05:32 +02:00
Alexey Milovidov
4ab8b35b00
Merge pull request #48444 from ClickHouse/better-exception-messages-from-keeper-client
...
Better exception messages from Keeper client
2023-04-06 18:36:39 +03:00
Azat Khuzhin
e91c8bfea3
Fix deadlock due to debug tracking of memory allocations
...
Since memory tracker should be blocked globally (by default it is blocked only
for VariableContext::User).
<details>
<summary>stacktrace</summary>
* thread 11, name = 'Collector', stop reason = trace
frame 1: 0x00007ffff7e37ea2 libc.so.6`pthread_mutex_lock + 274
frame 6: 0x000000001ee10298 clickhouse`toStringCached(pointers=0x00007fffedfef068, offset=0, size=45) + 88 at StackTrace.cpp:425
frame 7: 0x000000001ee1128e clickhouse`StackTrace::toString(this=0x00007fffedfef058) const + 46 at StackTrace.cpp:444
frame 8: 0x000000001ede58ff clickhouse`(anonymous namespace)::debugLogBigAllocationWithoutCheck(size=25187648) + 479 at MemoryTracker.cpp:98
frame 9: 0x000000001ede5481 clickhouse`MemoryTracker::allocImpl(this=0x0000000030729e68, size=25187648, throw_if_memory_exceeded=false, query_tracker=0x00007ffff3c22440) + 3937 at MemoryTracker.cpp:352
frame 10: 0x000000001ede477f clickhouse`MemoryTracker::allocImpl(this=0x00007ffff3c1e530, size=25187648, throw_if_memory_exceeded=false, query_tracker=0x00007ffff3c22440) + 607 at MemoryTracker.cpp:198
frame 11: 0x000000001ede477f clickhouse`MemoryTracker::allocImpl(this=0x00007ffff3c22440, size=25187648, throw_if_memory_exceeded=false, query_tracker=0x0000000000000000) + 607 at MemoryTracker.cpp:198
frame 12: 0x000000001ede477f clickhouse`MemoryTracker::allocImpl(this=0x00007fffedff3bf0, size=25187648, throw_if_memory_exceeded=false, query_tracker=0x0000000000000000) + 607 at MemoryTracker.cpp:198
frame 13: 0x000000001ed6c4c9 clickhouse`CurrentMemoryTracker::allocImpl(size=25165824, throw_if_memory_exceeded=false) + 265 at CurrentMemoryTracker.cpp:58
frame 14: 0x000000001ed6c6a1 clickhouse`CurrentMemoryTracker::allocNoThrow(size=25165824) + 33 at CurrentMemoryTracker.cpp:91
frame 15: 0x000000001ed63079 clickhouse`operator new(unsigned long) [inlined] void Memory::trackMemory<>(size=25165824) + 89 at memory.h:177
frame 16: 0x000000001ed63034 clickhouse`operator new(size=25165824) + 20 at new_delete.cpp:74
...
frame 27: 0x000000001ee15895 clickhouse`DB::(anonymous namespace)::collectSymbols(info=0x00007ffff7062000, (null)=<unavailable>, data_ptr=0x00007fffec2b7000) + 53 at SymbolIndex.cpp:483
frame 28: 0x000000002e1a5d57 clickhouse`::dl_iterate_phdr(callback=(clickhouse`DB::(anonymous namespace)::collectSymbols(dl_phdr_info*, unsigned long, void*) at SymbolIndex.cpp:480), data=0x00007fffec2b7000)(dl_phdr_info *, size_t, void *), void *) + 183 at phdr_cache.cpp:76
frame 29: 0x000000001ee15549 clickhouse`DB::SymbolIndex::update(this=0x00007fffec2b7000) + 41 at SymbolIndex.cpp:515
frame 30: 0x000000001ee1be35 clickhouse`DB::SymbolIndex::SymbolIndex(this=<unavailable>) + 53 at SymbolIndex.h:23
frame 31: 0x000000001ee17a6c clickhouse`DB::SymbolIndex::instanceImpl() + 108 at SymbolIndex.cpp:555
frame 32: 0x000000001ee17b4a clickhouse`DB::SymbolIndex::instance() + 10 at SymbolIndex.cpp:561
frame 33: 0x000000001ee1070f clickhouse`void toStringEveryLineImpl<toStringCached(std::__1::array<void*, 45ul> const&, unsigned long, unsigned long)::$_0>(fatal=false, stack_trace=0x00007fffedff1ef8, callback=0x00007fffedff1ee0)::$_0&&) + 207 at StackTrace.cpp:349
frame 34: 0x000000001ee1040b clickhouse`toStringCached(pointers=0x00007fffedff2928, offset=0, size=43) + 459 at StackTrace.cpp:436
frame 35: 0x000000001ee1128e clickhouse`StackTrace::toString(this=0x00007fffedff2918) const + 46 at StackTrace.cpp:444
frame 36: 0x000000001ede58ff clickhouse`(anonymous namespace)::debugLogBigAllocationWithoutCheck(size=25165824) + 479 at MemoryTracker.cpp:98
frame 37: 0x000000001ede5481 clickhouse`MemoryTracker::allocImpl(this=0x00007fffedff3bf0, size=25165824, throw_if_memory_exceeded=false, query_tracker=0x0000000000000000) + 3937 at MemoryTracker.cpp:352
frame 38: 0x000000001ed6c4c9 clickhouse`CurrentMemoryTracker::allocImpl(size=25165824, throw_if_memory_exceeded=false) + 265 at CurrentMemoryTracker.cpp:58
frame 39: 0x000000001ed6c6a1 clickhouse`CurrentMemoryTracker::allocNoThrow(size=25165824) + 33 at CurrentMemoryTracker.cpp:91
frame 40: 0x000000001ed63079 clickhouse`operator new(unsigned long) [inlined] void Memory::trackMemory<>(size=25165824) + 89 at memory.h:177
frame 41: 0x000000001ed63034 clickhouse`operator new(size=25165824) + 20 at new_delete.cpp:74
...
frame 55: 0x0000000025badace clickhouse`DB::WriteBufferFromS3::nextImpl(this=0x00007ffff3c58f00) + 366 at WriteBufferFromS3.cpp:110
</details>
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-06 16:03:46 +02:00
Sema Checherinda
b8729be97a
Merge pull request #47104 from CheSema/thead-group-for-merge
...
merge and mutation make thread group for setting memory trackers right
2023-04-06 15:46:56 +02:00
Robert Schulze
ce2692ba79
Merge pull request #47246 from ClickHouse/rs/m
...
Change the behavior of formatter %M in function formatDateTime() from minutes to month name
2023-04-06 09:19:50 +02:00
Alexey Milovidov
785ea8213b
Better exception messages from Keeper client
2023-04-06 02:29:37 +02:00
Alexander Tokmakov
a154fa73f4
Merge pull request #48401 from ClickHouse/fix_00002_log_messages
...
Find big allocations without memory limits checks
2023-04-05 23:52:39 +03:00
Robert Schulze
809a7fbb7a
Incorporate more review feedback
2023-04-05 19:43:30 +00:00
Sema Checherinda
554c928534
Merge branch 'master' into thead-group-for-merge
2023-04-05 21:09:38 +02:00
robot-clickhouse
3d2c279859
Merge pull request #48242 from azat/better-throttling
...
Per-query/per-server throttling for remote IO/local IO/BACKUPs
2023-04-05 19:32:37 +02:00
Alexander Tokmakov
0e0e89d596
fix
2023-04-05 15:10:09 +02:00
Robert Schulze
598b050628
AST Fuzzer: Fix assertion in TopK serialization
...
Problem:
https://s3.amazonaws.com/clickhouse-test-reports/0/fa5b2bd4a5b02336bca8837c473a7124f8ecedf2/fuzzer_astfuzzerasan/report.html
The new assertion in the Varint code was introduced with (*). It rejects
values whose serialization cannot be deserialized (and this behavior
cannot be changed due to historical reasons). Such values should be
exceptionally rare in practice but AST fuzzer managers to trigger them.
The fix is similar to (**): Bypass the check by limiting the value to
the maximum allowed value.
(if AST fuzzer triggers finds more violations of the assertion, we might
consider throwing an exception instead)
(*) https://github.com/ClickHouse/ClickHouse/pull/48154
(**) https://github.com/ClickHouse/ClickHouse/pull/48154/files#diff-653c0a18dfdaa86262c78dc6b25550add0487f165b4ad053e86f530388f6203a
2023-04-05 08:07:37 +00:00
Azat Khuzhin
a25dd1d348
Add ability to throttle local IO on per-query/server basis
...
Server settings:
- max_local_read_bandwidth_for_server
- max_local_write_bandwidth_for_server
Query settings:
- max_local_read_bandwidth
- max_local_write_bandwidth
This is the preparation for adding ability to throttle BACKUPs
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-05 09:39:15 +02:00
Azat Khuzhin
b3406beeb7
Add per-query network throttling
...
Controlled with:
- max_remote_read_network_bandwidth
- max_remote_write_network_bandwidth
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-05 09:39:15 +02:00
Arthur Passos
34be1893fd
fix unit tests
2023-04-04 20:31:12 -03:00
Arthur Passos
474377c16f
remove debug find functions and replace SearchSymbols assertion with an exception, check with upstream preferred way
2023-04-04 16:14:56 -03:00
Alexander Tokmakov
f5574d0d74
find big allocations without limits checks
2023-04-04 20:10:22 +02:00
Nikita Mikhaylov
fa5b2bd4a5
Added Keeper retries for backups operations ( #47224 )
2023-04-04 18:12:08 +02:00
Arthur Passos
fa1fdf7573
style
2023-04-03 15:28:17 -03:00
Arthur Passos
c80ba3d52e
unit tests passing
2023-04-03 13:48:58 -03:00
Antonio Andelic
f5e97fbb66
Merge branch 'master' into alternative-keeper-configs
2023-04-03 15:06:52 +00:00
Azat Khuzhin
fbc99d7788
Use StrongTypedef for StatusInfo
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-01 16:01:33 +02:00
Azat Khuzhin
c64f9e6f07
Use StrongTypedef for ProfileEvents
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-01 16:01:33 +02:00
Azat Khuzhin
8d0e516310
Use StrongTypedef for CurrentMetrics to avoid possible incorrect usage
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-01 16:01:33 +02:00
Azat Khuzhin
f69441b633
Fix ThreadPool usage in gtest_thread_pool_limit
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-01 16:00:03 +02:00
Azat Khuzhin
ba6ecd2d4e
Fix ThreadPool for DistributedSink
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-01 16:00:03 +02:00
Alexander Tokmakov
29e85bd9a4
Merge branch 'master' into fix-threadpools
2023-03-31 01:03:06 +03:00
Sergei Trifonov
01ed4ffca4
Merge pull request #47870 from ClickHouse/segmentator-fix
...
Fix CPU usage counters update logic
2023-03-30 22:55:58 +02:00
Azat Khuzhin
edeeac84cf
Fix incorrect ThreadPool usage after ThreadPool introspection
...
```
$ gg 'ThreadPool[^()]([A-Za-z_]\+,' src/
src/Interpreters/Context.cpp: shared->load_marks_threadpool = std::make_unique<ThreadPool>(pool_size, pool_size, queue_size);
src/Interpreters/Context.cpp: shared->prefetch_threadpool = std::make_unique<ThreadPool>(pool_size, pool_size, queue_size);
src/Interpreters/Context.cpp: shared->threadpool_writer = std::make_unique<ThreadPool>(pool_size, pool_size, queue_size);
```
Fixes : #47880
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-03-30 20:08:38 +02:00
Alexey Milovidov
e982fb9f1c
Merge pull request #47880 from azat/threadpool-introspection
...
ThreadPool metrics introspection
2023-03-30 01:27:31 +03:00
Alexey Milovidov
13761fe891
Merge pull request #48133 from ClickHouse/allow-to-skip-errors-of-enums
...
Allow skipping errors related to unknown enum values in row input formats
2023-03-29 23:20:05 +03:00
Frank Chen
87d235e842
Some improvements about names of span logs ( #47667 )
2023-03-29 16:28:36 +02:00
Azat Khuzhin
f38a7aeabe
ThreadPool metrics introspection
...
There are lots of thread pools and simple local-vs-global is not enough
already, it is good to know which one in particular uses threads.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-03-29 10:46:59 +02:00
Sergei Trifonov
1ff6519c25
Merge branch 'master' into segmentator-fix
2023-03-29 02:29:43 +02:00
serxa
81af0b6deb
avoid counters updates if not initialized
2023-03-29 00:29:21 +00:00
Alexey Milovidov
cfc4e40811
Allow skipping errors related to unknown enum values in row input formats
2023-03-29 00:31:53 +02:00
Alexander Tokmakov
614f8b313e
Merge pull request #47856 from Algunenano/zk_inject_timeout
...
Allow injecting timeout errors on Keeper
2023-03-28 23:40:51 +03:00
Yatsishin Ilya
26aacd3d2d
Merge remote-tracking branch 'origin/master' into clickhouse-help
2023-03-28 16:57:58 +00:00
avogar
e63231700f
Fix tests
2023-03-28 16:00:00 +00:00
Sema Checherinda
b774310dc9
keep query id for logging when optimize query
2023-03-28 16:44:47 +02:00
Sema Checherinda
10f1030a14
Merge pull request #47147 from aletik256/fix_JSON_searchField
...
fix_JSON_searchField
2023-03-28 14:45:09 +02:00