thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-10 00:12:27 +00:00
Code Issues Packages Projects Releases Wiki Activity
115,128 Commits 9,222 Branches 1,782 Tags 2 GiB
e2e3a03dbe
Commit Graph

5713 Commits

Author SHA1 Message Date
serxa
23defcbf6a log progress in AsyncLoader 2023-05-01 12:08:50 +00:00
Sergei Trifonov
b390a07698
Merge branch 'master' into async-loader 2023-05-01 14:04:07 +02:00
Nikolay Degterinsky
f64ef88197 Merge remote-tracking branch 'upstream/master' into bcrypt 2023-04-30 21:39:16 +00:00
alesapin
ec2c860a39 Merge branch 'fix_zero_copy_not_atomic' of github.com:ClickHouse/ClickHouse into fix_zero_copy_not_atomic 2023-04-30 15:02:27 +02:00
alesapin
daa0b095ed Merge branch 'master' into fix_zero_copy_not_atomic 2023-04-29 21:26:26 +02:00
Kseniia Sumarokova
d4aa96e262
Merge branch 'master' into better-local-object-storage 2023-04-28 21:05:05 +02:00
Robert Schulze
7609832606
Unpoison stack frame ptrs from libunwind for msan 
Turned out that in msan builds, stack frame pointers explicitly returned
by unw_backtrace() in Common/StackTrace.cpp were unpoisened, but stack
frame pointers returned from the same function under the
STD_EXCEPTION_HAS_STACK_TRACE define in std::exception (see (*)) were
not. I tried unpoisoning directly in libcxx. Since this didn't work,
this PR unpoisons where get_stack_trace_frames() is called.

I don't know why this issue never occurred in CI but it fixes #48385 for
me (where msan complains about reads of uninitialized data when query
cache squashes and compresses chunks produced from SELECT last_error_trace
FROM system.errors).

Fixes: #48385

(*) https://github.com/ClickHouse/llvm-project/pull/7/files#diff-8c81d74ef72967be2fa27a31275000aa18532a583490f9673c62d7b981a5b39c
 2023-04-28 17:43:12 +00:00
Alexey Milovidov
cd171bef7d
Merge pull request #45999 from sichenzhao/cgroup_v2 
Updates AsyncMetrics cgroup-memory to support cgroupv2
 2023-04-28 20:39:53 +03:00
alesapin
70ee02a3eb Add node holder 2023-04-28 17:39:32 +02:00
Alexey Milovidov
c86d3daee5
Update AsynchronousMetrics.cpp 2023-04-28 16:21:37 +03:00
alesapin
ef49603432 Merge branch 'master' into fix_zero_copy_not_atomic 2023-04-28 14:42:30 +02:00
kssenii
298ed454ce Merge remote-tracking branch 'upstream/master' into better-local-object-storage 2023-04-28 13:26:55 +02:00
Alexey Milovidov
4dccd5f9b0
Update AsynchronousMetrics.cpp 2023-04-27 23:20:38 +03:00
Nikolay Degterinsky
df1e56b985 Merge remote-tracking branch 'upstream/master' into bcrypt 2023-04-27 17:01:58 +00:00
Robert Schulze
b6e65ddeff
Merge branch 'master' into add-examples-to-functions 2023-04-27 16:11:32 +02:00
Kseniia Sumarokova
45e2d296f9
Merge branch 'master' into better-local-object-storage 2023-04-27 14:54:04 +02:00
Alexey Milovidov
c5793f9684
Fix build 2023-04-27 14:56:51 +03:00
Sergei Trifonov
85d7816d65
Update src/Common/AsyncLoader.h 2023-04-27 13:55:26 +02:00
Robert Schulze
096667be45
More fixes 2023-04-27 09:33:52 +00:00
Alexey Milovidov
1bb685a081
Fix error. 2023-04-27 01:32:54 +03:00
Robert Schulze
739bfc9b8d
Fix categories 2023-04-26 21:23:26 +00:00
Robert Schulze
d72030b876
Various fixes 2023-04-26 20:38:41 +00:00
serxa
94f730ded0 Merge branch 'async-loader' of github.com:ClickHouse/ClickHouse into async-loader 2023-04-26 18:40:26 +00:00
serxa
173a8a3795 fix tidy build 2023-04-26 18:39:51 +00:00
alesapin
f3d1d3b58b Merge branch 'master' into fix_zero_copy_not_atomic 2023-04-26 18:00:31 +02:00
Sergei Trifonov
bb4109b3e1
Merge branch 'master' into async-loader 2023-04-26 15:00:59 +02:00
Alexey Milovidov
88ce78a7be
Merge branch 'master' into cgroup_v2 2023-04-26 13:55:12 +03:00
alesapin
e5f71c3b74 Fix zero copy not atomic 2023-04-26 00:25:04 +02:00
alesapin
3a24f0b6c6 fxi 2023-04-25 15:33:41 +02:00
Anton Popov
cc5acfbe62 add profile event 2023-04-25 01:37:37 +00:00
Alexey Milovidov
dee0b78303
Merge pull request #48745 from Algunenano/arenita 
Don't count unused arena bytes as read_bytes in AggregateFunction types
 2023-04-25 00:53:11 +03:00
Alexey Milovidov
761ed890a7
Update AsynchronousMetrics.cpp 2023-04-25 00:45:57 +03:00
Anton Popov
38622d0770 add settings to delay or throw in case of too many mutations 2023-04-24 18:21:49 +00:00
alesapin
d28b71e69d
Merge pull request #49078 from ClickHouse/fix-false-report 
Prevent false positive report by static analyzer
 2023-04-24 13:08:37 +02:00
Robert Schulze
ded8eca041
Merge pull request #48284 from ClickHouse/rs/qc-quota 
Query Cache: Allow per-user quotas
 2023-04-24 11:32:32 +02:00
Vladimir C
6090648fc8
Merge pull request #43606 from arthurpassos/parsey_key_value_function 2023-04-24 11:03:28 +02:00
Raúl Marín
868b4e41fd Arena used bytes 2023-04-24 10:46:23 +02:00
Raúl Marín
f0e045bb3d Merge remote-tracking branch 'blessed/master' into arenita 2023-04-24 10:42:56 +02:00
Alexey Milovidov
ff5ec30b0b
Merge pull request #49005 from ClickHouse/fix-ibm 
Reimplement #48986
 2023-04-24 01:39:22 +03:00
Alexey Milovidov
f4bd5a0667 Prevent false positive report by static analyzer 2023-04-23 22:45:12 +02:00
Alexey Milovidov
c3749d652e
Update AsynchronousMetrics.cpp 2023-04-23 23:24:57 +03:00
Alexey Milovidov
40bae4b611
Merge branch 'master' into cgroup_v2 2023-04-23 21:46:05 +03:00
Robert Schulze
71c4e3dbf4
A bit more memory-safe 2023-04-23 18:13:51 +00:00
Robert Schulze
38d5f12a32
Query Cache: Implement per-user quotas 2023-04-23 16:13:51 +00:00
Alexey Milovidov
c42e1ad131
Merge pull request #48922 from RunningXie/fix_reopen_stale_fd 
Implement a slight improvement for the rare case when ClickHouse is run inside LXC and LXCFS is used, because LXCFS has a bug
 2023-04-23 16:12:49 +03:00
kssenii
d8023806a9 Merge remote-tracking branch 'upstream/master' into better-local-object-storage 2023-04-23 12:39:34 +02:00
Alexey Milovidov
1be5a20546
Simplify the code 2023-04-23 09:01:21 +03:00
Alexey Milovidov
1087c7361d
Update AsynchronousMetrics.cpp 2023-04-23 08:51:17 +03:00
Alexey Milovidov
28e03f5781 Merge branch 'master' into fix-ibm 2023-04-23 06:01:41 +02:00
Sergei Trifonov
7fa5ccb53c
Merge branch 'master' into async-loader 2023-04-22 19:44:44 +02:00
serxa
056268f90b fix 2023-04-22 17:43:36 +00:00
serxa
64fb8da812 improve AsyncLoader interface 2023-04-22 17:29:53 +00:00
Alexey Milovidov
5fd42a2220 Add ProfileEvents for log messages 2023-04-22 14:55:25 +02:00
Alexey Milovidov
4c7f747fae
Update AsynchronousMetrics.cpp 2023-04-22 14:51:21 +03:00
Dmitry Novik
e23105b322 Merge remote-tracking branch 'origin/master' into revert-47476-revert_46622 2023-04-21 16:40:04 +00:00
Alexey Milovidov
a6fceaa497
Update AsynchronousMetrics.cpp 2023-04-21 13:51:13 +03:00
kssenii
e58c16318f Merge remote-tracking branch 'upstream/master' into better-local-object-storage 2023-04-21 12:49:40 +02:00
Alexey Milovidov
252c755323 Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-ibm 2023-04-21 12:39:13 +02:00
Alexey Milovidov
530b764953 Fix IBM 2023-04-21 12:38:45 +02:00
Alexey Milovidov
9d6c3d7a4c
Merge pull request #48920 from awfeequdng/bugfix/maskLowBits 
A non significant change (does not affect anything): add support for signed integers in the maskBits function
 2023-04-21 13:22:44 +03:00
Kruglov Pavel
9749448eb0
Merge pull request #47229 from Avogar/non-blocking-connect 
Improve async reading from socket
 2023-04-21 10:38:50 +02:00
pengxiangcai
41cf29179d typename spelling error: unsigedT -> UnsignedT 2023-04-21 08:51:58 +08:00
Alexey Milovidov
98ae9be734
Revert "Added tests for ClickHouse apps help and fixed help issues" 2023-04-21 01:54:34 +03:00
Arthur Passos
43841a6ee6 Merge branch 'master' into parsey_key_value_function 2023-04-20 10:05:34 -03:00
Antonio Andelic
31548ab17c Fix result 2023-04-20 12:30:24 +00:00
Antonio Andelic
09156ab2b7 Merge branch 'master' into keeper-check-not-exists 2023-04-20 12:07:16 +00:00
avogar
39ba4c27bc Fix deadlock, add profile event 2023-04-20 11:56:20 +00:00
Sergei Trifonov
4b51f12479
Update src/Common/AsyncLoader.h 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2023-04-19 13:48:38 +02:00
serxa
5390fccf50 Merge branch 'async-loader' of github.com:ClickHouse/ClickHouse into async-loader 2023-04-19 11:47:08 +00:00
serxa
2e31140ca4 review fixes 2023-04-19 11:46:56 +00:00
Kruglov Pavel
2ad161d2b7
Merge branch 'master' into non-blocking-connect 2023-04-19 13:39:40 +02:00
Sergei Trifonov
31e3a3bc0c
Update src/Common/AsyncLoader.h 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2023-04-19 13:35:58 +02:00
Antonio Andelic
58e9b56fcb Fix CheckNotExists 2023-04-19 09:06:20 +00:00
serxa
ff7444f077 typo 2023-04-19 06:27:17 +00:00
serxa
c672d117ea better job shutdown 2023-04-19 06:27:07 +00:00
serxa
8cc9b32af1 fix more comments 2023-04-19 06:26:59 +00:00
serxa
d4daea5f09 fix typos 2023-04-19 06:26:51 +00:00
serxa
baced2c66e fix data race in test 2023-04-19 06:26:35 +00:00
serxa
326c733be6 fix tidy build 2023-04-19 06:26:27 +00:00
serxa
54c5b1083b add test for dynamic prioritization 2023-04-19 06:26:14 +00:00
serxa
911c46aa32 fix double finish() during cancellation of a job with multiple paths 2023-04-19 06:26:06 +00:00
serxa
0d7e3d4100 fix comment 2023-04-19 06:25:54 +00:00
serxa
356b934835 split AsyncLoader into .h and .cpp files + more docs 2023-04-19 06:25:45 +00:00
Sergei Trifonov
1efacba196 Update src/Common/AsyncLoader.h 2023-04-19 06:25:37 +00:00
Sergei Trifonov
281b72c413 Update src/Common/AsyncLoader.h 2023-04-19 06:25:28 +00:00
serxa
1a3780ed8e fix 2023-04-19 06:25:11 +00:00
serxa
800efd5df8 add a good comment 2023-04-19 06:24:50 +00:00
serxa
69719c0819 allow to change max number of workers in runtime 2023-04-19 06:24:42 +00:00
serxa
c736d9fd4f fix test 2023-04-19 06:24:35 +00:00
serxa
264956fece add test for static priorities 2023-04-19 06:24:25 +00:00
serxa
b86c737450 make performance-inefficient-vector-operation,-warnings-as-errors happy 2023-04-19 06:24:15 +00:00
serxa
8c1f734f50 add random test with dependent tasks 2023-04-19 06:24:07 +00:00
serxa
d8bf775b9a fix worker shutdown 2023-04-19 06:24:01 +00:00
serxa
51992ae96d add ScheduleJobWithCanceledDependencies test 2023-04-19 06:23:47 +00:00
serxa
5837b09880 fix scheduling of jobs with dependencies on finished jobs 2023-04-19 06:23:39 +00:00
serxa
5ba1dc91e3 add LoadStatus::CANCELED 2023-04-19 06:23:32 +00:00
serxa
df3c5212ff test deps are completed + refactoring 2023-04-19 06:23:22 +00:00
serxa
9ac37ca3b8 add overload test 2023-04-19 06:23:14 +00:00
serxa
4bc52a1b06 add concurrency test 2023-04-19 06:23:07 +00:00
serxa
ba67f95525 allow cancels on RandomTasks test shutdown 2023-04-19 06:22:37 +00:00
serxa
3be0645413 fix remove of executing tasks + add test 2023-04-19 06:22:27 +00:00
serxa
a9c51c4aca add more tests for cancel 2023-04-19 06:22:15 +00:00
serxa
5e5a38cd64 add tests for job status/wait/cancel 2023-04-19 06:22:01 +00:00
serxa
3226eadd94 add simple randomized test 2023-04-19 06:21:31 +00:00
serxa
2c29c0b2f3 add dependency cycle detection 2023-04-19 06:21:19 +00:00
serxa
b4318b4ab6 fix style 2023-04-19 06:21:08 +00:00
serxa
5d11706929 make t work, add simple test 2023-04-19 06:20:37 +00:00
serxa
3fe36a3db6 wip AsyncLoader 2023-04-19 06:20:17 +00:00
caipengxiang
c9848e0822 bugfix: 
maskLowBits<UInt8>(3) => 00000111, but maskLowBits<Int8>(3) => 11111111, so I fixed it.
    after I fixed it, the result is:
    maskLowBits<UInt8>(3) => 00000111, maskLowBits<Int8>(3) => 00000111
 2023-04-19 12:22:17 +08:00
xieyichen
11b3469eac reopen stale fd 2023-04-19 11:13:16 +08:00
Ilya Yatsishin
ebb1b990f2
Merge pull request #45819 from qoega/clickhouse-help 2023-04-18 21:46:36 +02:00
Antonio Andelic
fb16623d48 Add CheckNotExists request to Keeper 2023-04-18 14:45:24 +00:00
kssenii
16b027ed0c Merge remote-tracking branch 'upstream/master' into better-local-object-storage 2023-04-18 16:25:08 +02:00
Kruglov Pavel
cd88024a33
Merge pull request #48556 from ClickHouse/fix-nested-map-ip-uuid 
Fix nested map for keys of IP and UUID types
 2023-04-18 13:03:50 +02:00
Alexander Gololobov
423193d619
Merge branch 'master' into fix_zookeeper_join_race 2023-04-17 20:50:51 +02:00
kssenii
a470deafbc Fix style check 2023-04-17 19:11:42 +02:00
Alexander Gololobov
0493b2cbce A wrapper to allow calling join() from multiple threads 2023-04-17 15:01:45 +02:00
Raúl Marín
39f8c43a60 Merge remote-tracking branch 'blessed/master' into arenita 2023-04-17 10:33:38 +02:00
Michael Kolupaev
e133633359 Parallel decoding with one row group per thread 2023-04-17 04:58:32 +00:00
Michael Kolupaev
2d4fe85513 Something 2023-04-17 04:58:32 +00:00
Dmitry Novik
a1f21a5fc4 Do not log peak for background memory tracker 2023-04-14 16:49:14 +02:00
Dmitry Novik
cf5d9a175a Revert "Merge pull request #48760 from ClickHouse/revert-46089-background-memory-tracker" 
This reverts commit a61ed33223, reversing
changes made to 5f01b8a2b5.
 2023-04-14 16:34:19 +02:00
Sergei Trifonov
7cb6303e4c
Merge branch 'master' into lower-threadpool-mutex-contention 2023-04-14 03:26:39 +02:00
Alexander Tokmakov
af1bf08663
Revert "Add MemoryTracker for the background tasks" 2023-04-13 21:05:02 +03:00
serxa
22b75ec81b fix typo, add coment, dont reset trace 2023-04-13 15:17:39 +00:00
Arthur Passos
88f1c969cc
Merge branch 'master' into parsey_key_value_function 2023-04-13 10:38:44 -03:00
Sergei Trifonov
c51f1359be
Merge branch 'master' into lower-threadpool-mutex-contention 2023-04-13 14:21:45 +02:00
serxa
32f78afd7e get rid of redundant code 2023-04-13 12:11:06 +00:00
serxa
ce2073e58f detach_thread() and notifications order does not matter 2023-04-13 11:45:19 +00:00
Raúl Marín
e239df4c25 Keep count to avoid iterating over lists 2023-04-13 13:41:44 +02:00
Dmitry Novik
467ecf45e1
Merge pull request #46089 from ClickHouse/background-memory-tracker 
Add MemoryTracker for the background tasks
 2023-04-13 13:29:27 +02:00
serxa
5e8339c2d9 Lock ThreadPool mutex once per job instead of twice 2023-04-13 11:27:11 +00:00
Raúl Marín
2b70e08f23 Don't count unreserved bytes in Arenas as read_bytes 2023-04-13 12:43:24 +02:00
Raúl Marín
da9a539cf7 Reduce the usage of Arena.h 2023-04-13 10:31:32 +02:00
Robert Schulze
22c3b10e41
Merge pull request #48628 from ClickHouse/rs/var_uint_max 
Better handling of values too large for VarInt encoding
 2023-04-13 10:17:55 +02:00
Robert Schulze
f41354ccd6
Merge pull request #48671 from ClickHouse/rs/gcc-removal 
Remove GCC remainders
 2023-04-13 10:15:35 +02:00
Robert Schulze
3f7ce60e03
Merge branch 'master' into rs/gcc-removal 2023-04-12 22:17:04 +02:00
Azat Khuzhin
37d430d68a Use ThreadPool in PipelineExecutor 
PipelineExecutor is one of the heavy user of the threads, by using pool
there number of used threads for query execution can be tracked in
system.metrics.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-12 20:00:51 +02:00
Robert Schulze
05606a8835
Clean up GCC warning pragmas 2023-04-11 18:21:08 +00:00
Dmitry Novik
52f3406bbb Fix CurrentMetrics::MergesMutationsMemoryTracking value 2023-04-11 16:10:59 +00:00
Dmitry Novik
06e6794fc0 Merge remote-tracking branch 'origin/master' into background-memory-tracker 2023-04-11 15:29:35 +00:00
Yatsishin Ilya
b5b65d2149 Merge remote-tracking branch 'origin' into clickhouse-help 2023-04-11 11:24:48 +00:00
Sema Checherinda
4dd86a406a
Merge pull request #48543 from azat/mv-uniq-thread-group 
Use one ThreadGroup while pushing to materialized views (and some refactoring for ThreadGroup)
 2023-04-11 11:47:46 +02:00
Robert Schulze
bbbb55cd50
Better handling of values too large for VarInt encoding 
PR #48154 introduced a sanity check in the form of a debug assertion
that the input values for VarInt encoding are not too big. Such values
should be exceptionally rare in practice but the AST fuzzer managed to
trigger the assertion regardless. The strategy to deal with such values
until now was to bypass the check by limiting the value to the maximum
allowed value (see #48412). Because a new AST Fuzzer failure appeared
(#48497) and there may be more failures in future, this PR changes the
sanity check from an assert to an exception.

Fixes: #48497
 2023-04-11 07:47:33 +00:00
Azat Khuzhin
79b83c4fd2 Remove superfluous includes of logger_userful.h from headers 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-04-10 17:59:30 +02:00
Arthur Passos
b062ab5b4f
Merge branch 'master' into parsey_key_value_function 2023-04-10 10:32:01 -03:00
Alexey Milovidov
6292a08c9c Add a test 2023-04-09 03:25:38 +02:00
Alexey Milovidov
09ea79aaf7 Add support for {server_uuid} macro 2023-04-09 03:04:26 +02:00
Alexey Milovidov
5495cb2935
Merge pull request #48521 from ClibMouse/feature/tiny-improvement 
Use std::string::starts_with instead of a roll your own variant
 2023-04-08 22:19:48 +03:00
Yakov Olkhovskiy
054078e06d fix nested map for keys of IP and UUID types 2023-04-08 05:34:35 +00:00