Commit Graph

3717 Commits

Author SHA1 Message Date
Dmitry Novik
455742ec7a Make unit tests stable 2022-05-03 13:20:57 +00:00
Alexander Tokmakov
94af11dd74 add extra info when sending exception 2022-05-03 15:13:47 +02:00
kssenii
025e632a66 Merge master 2022-05-03 14:46:59 +02:00
kssenii
b0d43e3f40 Better 2022-05-03 14:20:23 +02:00
kssenii
a3c9ff7fe8 Tiny changes 2022-05-03 11:46:22 +02:00
Vitaly Baranov
409edfd3fa Rework RestoreCoordination: make restore deterministic. 2022-05-03 11:01:44 +02:00
Vitaly Baranov
2c754f44fc Make calculation of shard_num & replica_num not dependant on match of the cluster's definitions on nodes. 2022-05-03 11:01:44 +02:00
Larry Luo
ee131eefd8 Removed move for trivially-copyable type and added noexcept for move constructor 2022-05-02 18:46:34 -07:00
Dmitry Novik
2815f22152 Remove incorrect assert 2022-05-02 23:02:43 +00:00
Dmitry Novik
2ed5a4013a
Revert "Revert "Memory overcommit: continue query execution if memory is available"" 2022-05-03 00:45:13 +02:00
alesapin
f0b7af0aa2
Revert "Memory overcommit: continue query execution if memory is available" 2022-05-03 00:36:50 +02:00
alesapin
73f3334d80 Better fix for cache 2022-05-03 00:28:53 +02:00
Dmitry Novik
71b6f89166
Merge pull request #35637 from ClickHouse/memory-overcommit-free
Memory overcommit: continue query execution if memory is available
2022-05-02 19:00:18 +02:00
kssenii
70e2bb6264 Reimplement write-through cache 2022-05-02 18:55:11 +02:00
Ilya Yatsishin
6c3bbb4933 Add Other Time Microseconds Profile Event 2022-05-02 17:13:57 +02:00
Vladimir C
7293a69e5e
Merge pull request #36656 from amosbird/timefunctionunderflow
Saturate date/datetime to zero (part 2)
2022-05-02 17:10:48 +02:00
Maksim Kita
9bea207680
Merge pull request #36839 from rschu1ze/concepts
Replace uses of SFINAE by C++20 concepts
2022-05-02 14:20:12 +02:00
Robert Schulze
ad0b3be79d
Replace uses of SFINAE by C++20 concepts
- enable_if is usually regarded as fragile and unreadable

- C++20 concepts are much easier to read and produce more expressive
  error messages
2022-05-02 09:23:53 +02:00
Robert Schulze
330212e0f4
Remove inherited create() method + disallow copying
The original motivation for this commit was that shared_ptr_helper used
std::shared_ptr<>() which does two heap allocations instead of
make_shared<>() which does a single allocation. Turned out that
1. the affected code (--> Storages/) is not on a hot path (rendering the
performance argument moot ...)
2. yet copying Storage objects is potentially dangerous and was
   previously allowed.

Hence, this change

- removes shared_ptr_helper and as a result all inherited create() methods,

- instead, Storage objects are now created using make_shared<>() by the
  caller (for that to work, many constructors had to be made public), and

- all Storage classes were marked as noncopyable using boost::noncopyable.

In sum, we are (likely) not making things faster but the code becomes
cleaner and harder to misuse.
2022-05-02 08:46:52 +02:00
Alexey Milovidov
ef9633499c Merge branch 'master' into fix-trash-s3-metrics 2022-05-02 03:55:10 +02:00
Mikhail Artemenko
cf48390cb9
Merge branch 'master' into MeiliSearch 2022-05-02 00:28:42 +03:00
KinderRiven
7d0789472c fix 2022-05-01 19:45:16 +08:00
KinderRiven
e1acacf831 fix 2022-05-01 19:45:16 +08:00
KinderRiven
d80aa0fd17 impl system.filesystem_cache_log 2022-05-01 19:45:16 +08:00
KinderRiven
d595df1604 fix 2022-05-01 19:45:16 +08:00
KinderRiven
6a2e4559c8 impl system.cache_log 2022-05-01 19:45:16 +08:00
KinderRiven
ce9a6965d0 impl system.log_table 2022-05-01 19:45:16 +08:00
KinderRiven
a4b8f9e9cf impl system.cache_log 2022-05-01 19:45:16 +08:00
KinderRiven
9c9870f021 build cache_log table 2022-05-01 19:45:16 +08:00
KinderRiven
5a361a0db5 impl cache_log in filecache 2022-05-01 19:45:16 +08:00
KinderRiven
5b7b2d26ea fix add miss header 2022-05-01 19:45:16 +08:00
KinderRiven
df81496ba1 fix bug for cache log table 2022-05-01 19:45:16 +08:00
KinderRiven
de7e89661d resubmit 2022-05-01 19:45:16 +08:00
Mikhail Artemenko
41f657d8ed
Merge branch 'master' into MeiliSearch 2022-05-01 10:01:56 +03:00
kssenii
8cf83a2daf Fix 2022-04-30 22:02:44 +02:00
Robert Schulze
89aa9ae00f
Fixed clang-tidy check "bugprone-branch-clone"
The check is currently *not* part of .clang-tidy. It complains about:
(1) "switch has multiple consecutive identical branches"
(2) "repeated branch in conditional chain"

About (1): Lots of findings in switches were about redundant
"[[fallthrough]]" in places where the compiler would not warn anyways. I
have cleaned these up.

About (2): In if-else_if-else chains, fixing the warning would usually
mean concatenating multiple if-conditions. As this would reduce
readability in most cases, I did not fix these places.

Because of (2), I also refrained from adding "bugprone-branch-clone" to
.clang-tidy.
2022-04-30 19:40:28 +02:00
Kseniia Sumarokova
66f62b2ba6
Merge pull request #36632 from DevTeamBK/Fix-Coverity-Issues
Minor Coverity defects fixes
2022-04-30 15:11:40 +02:00
kssenii
eea0a009dc Debug 2022-04-30 15:02:21 +02:00
mergify[bot]
cc08ccb420
Merge branch 'master' into remove-useless-code-2 2022-04-30 12:48:15 +00:00
Alexey Milovidov
1ddb04b992
Merge pull request #36715 from amosbird/refactorbase
Reorganize source files so that base won't depend on Common
2022-04-30 09:40:58 +03:00
HeenaBansal2009
042157efca Clang -Tidy Fixes 2022-04-29 20:17:45 -07:00
mergify[bot]
e9fde5d067
Merge branch 'master' into memory-overcommit-free 2022-04-29 19:31:59 +00:00
Nikolai Kochetov
5807ca47b0
Merge pull request #36425 from ClickHouse/pool-size-hot-reload
Allow to increase number of threads and tasks for background executors
2022-04-29 17:24:12 +02:00
Kseniia Sumarokova
ca994e0861
Merge pull request #36660 from kssenii/fix-stress-test
Fix stress test after 36639
2022-04-29 12:56:25 +02:00
Amos Bird
4a5e4274f0
base should not depend on Common 2022-04-29 10:26:35 +08:00
Amos Bird
e81929a8b5
Saturate date/datetime to zero (part 2)
For partial hours/minutes timezones.
2022-04-29 10:24:18 +08:00
Kseniia Sumarokova
ace3fe253d
Merge pull request #36737 from KinderRiven/fix_bug_for_local_cache
fix bug for local cache
2022-04-29 02:54:33 +02:00
mergify[bot]
41aa85fd4c
Merge branch 'master' into fix-stress-test 2022-04-29 00:54:09 +00:00
kssenii
1cbf672237 Another fix 2022-04-28 22:51:57 +02:00
Alexey Milovidov
d8fa806fca
Merge pull request #36360 from ClickHouse/optimize-asynchronous-metric-log
Optimize `system.asynchronous_metric_log`
2022-04-28 20:16:17 +03:00
Dmitry Novik
8fb7e2dbab Cleanup 2022-04-28 15:31:13 +00:00
HeenaBansal2009
3ac9434c57 Merge branch 'master' into Fix-Coverity-Issues 2022-04-28 08:14:50 -07:00
kssenii
9b38c9f7eb Fix 2022-04-28 13:06:43 +02:00
Kseniia Sumarokova
0c2f074c60
Update FileCache.cpp 2022-04-28 12:47:02 +02:00
Kseniia Sumarokova
2039e0e974
Update FileCache.h 2022-04-28 12:46:45 +02:00
KinderRiven
63804fd425 fix 2022-04-28 18:45:01 +08:00
KinderRiven
1fbaddf559 fix bug for local_cache 2022-04-28 17:11:25 +08:00
Vitaly Baranov
641a5f5e35
Merge pull request #36198 from vitlibar/backup-improvements-3
Backup for replicated tables and other improvements
2022-04-28 09:23:53 +02:00
Alexey Milovidov
de3a47c67a Merge branch 'master' into optimize-asynchronous-metric-log 2022-04-28 05:03:57 +02:00
kssenii
37d11cfb1c Merge master 2022-04-28 00:44:15 +02:00
mergify[bot]
b7e63154df
Merge branch 'master' into optimize-asynchronous-metric-log 2022-04-27 18:27:27 +00:00
kssenii
da1392877a Better 2022-04-27 20:01:59 +02:00
mergify[bot]
9410e2813f
Merge branch 'master' into fix-stress-test 2022-04-27 14:51:55 +00:00
HeenaBansal2009
4961caec13 Merge branch 'master' into Fix-Coverity-Issues 2022-04-26 11:43:44 -07:00
kssenii
64168faf0a Fix 2022-04-26 20:31:42 +02:00
HeenaBansal2009
a34fca27d5 Incorporated review comments 2022-04-26 11:12:14 -07:00
HeenaBansal2009
13a9a8fd5e Style check fix 2022-04-26 07:13:09 -07:00
HeenaBansal2009
267f426605 Minor Coverity defetcs fixes 2022-04-26 07:13:09 -07:00
mergify[bot]
7571387a6e
Merge branch 'master' into memory-overcommit-free 2022-04-26 10:15:18 +00:00
kssenii
b874750807 Better 2022-04-26 12:13:21 +02:00
kssenii
41bba15e20 Fix 2022-04-26 12:09:58 +02:00
Vladimir C
1aa9d77246
Merge pull request #36453 from zombee0/master 2022-04-26 11:47:42 +02:00
Kseniia Sumarokova
81db081b88
Merge pull request #36639 from kssenii/better-drop-cache
Improve drop cache query
2022-04-26 11:25:46 +02:00
Vitaly Baranov
1c0b731ea6 Fix compilation. 2022-04-26 10:13:08 +02:00
mergify[bot]
4e82b56a8d
Merge branch 'master' into optimize-asynchronous-metric-log 2022-04-26 05:40:49 +00:00
Alexey Milovidov
86bf67c93c
Merge pull request #36526 from Joeywzr/to_start_of_five_minutes
Replace toStartOfFiveMinute to toStartOfFiveMinutes
2022-04-26 05:52:02 +03:00
alesapin
92296484e7
Merge pull request #36348 from rschu1ze/erase_if3
Replace remove-erase idiom by C++20 erase()/erase_if()
2022-04-25 23:34:18 +02:00
alesapin
7a901c0826
Merge pull request #36602 from azat/netlink-strict-resubmit
Strict taskstats parser (resubmit)
2022-04-25 23:28:08 +02:00
alesapin
04afe552f8 Trying to avoid segfaults 2022-04-25 22:51:09 +02:00
kssenii
35b8d302da Fix 2022-04-25 21:57:13 +02:00
alesapin
1fffa56f63 Merge branch 'master' into memory-overcommit-free 2022-04-25 20:21:08 +02:00
mergify[bot]
d2ac9b2223
Merge branch 'master' into to_start_of_five_minutes 2022-04-25 17:36:38 +00:00
mergify[bot]
e79de5a577
Merge branch 'master' into optimize-asynchronous-metric-log 2022-04-25 17:29:11 +00:00
mergify[bot]
fdc895462e
Merge branch 'master' into fix-trash-s3-metrics 2022-04-25 17:28:06 +00:00
kssenii
a76a35d3ce Better drop cache 2022-04-25 19:20:25 +02:00
Vitaly Baranov
d20b3d78c5 Rename some restore settings. 2022-04-25 16:34:17 +02:00
Sergei Trifonov
ad9d57807f
fix typo 2022-04-25 16:06:14 +02:00
Kruglov Pavel
34c342fdd3
Merge pull request #36205 from Avogar/improve-globs
Some refactoring around schema inference with globs
2022-04-25 13:14:46 +02:00
Nikolai Kochetov
d5b2d28f7b
Merge pull request #36579 from amosbird/refactor-executor1
Small refactoring of Processors and QueryPipeline
2022-04-25 12:47:59 +02:00
Antonio Andelic
d60384d336
Merge pull request #36469 from bigo-sg/fix_ranger_generator
Fix bug of range generator
2022-04-25 10:52:34 +02:00
Azat Khuzhin
278101fc0c Add a comment for taskstruct netlink interface rounding to KiB
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-25 08:49:17 +03:00
Azat Khuzhin
d161b798a8 Strict taskstats parser
This will unhide possible issues (since sometimes statistics is
inaccurate).

Also before this patch TASKSTATS_TYPE_AGGR_TGID had been handled, while
it not correct to take into account, but it was not requested anyway.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
(cherry picked from commit bfa65b651a)
2022-04-25 07:56:54 +03:00
Azat Khuzhin
6df54611a9 Log which taskstats provider is in use
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
v2: add missing __builtin_unreachable() for gcc built [1]
    [1]: https://s3.amazonaws.com/clickhouse-builds/36351/99e72a960325cc1fb3cd3d6a1b56cf2b1cc2d733/binary_gcc/build_log.log
(cherry picked from commit 2be022cb56)
2022-04-25 07:56:54 +03:00
mergify[bot]
705d5af3a0
Merge branch 'master' into to_start_of_five_minutes 2022-04-24 22:24:24 +00:00
Alexey Milovidov
74bb6349c2
Revert "Strict taskstats parser" 2022-04-25 01:15:08 +03:00
Amos Bird
a25bb50096
Refactor many exception messages
1. Always use fmt variant
2. Remove redundant period at the end of message
3. Remove useless parenthesis
2022-04-24 19:44:00 +08:00
taiyang-li
6716aa9d95 Merge remote-tracking branch 'ck/master' into fix_ranger_generator 2022-04-24 11:31:19 +08:00
Alexey Milovidov
8af3159916
Merge pull request #36351 from azat/netlink-strict
Strict taskstats parser
2022-04-24 05:14:06 +03:00
Alexey Milovidov
d08da0f63c Everything related to S3 metrics was a trash 2022-04-24 03:46:35 +02:00
Alexey Milovidov
6de4f2bbff Everything related to S3 metrics was a trash 2022-04-24 03:42:18 +02:00
zombee0
c02de61086 fixed the cow_compositions.cpp clone inner column when there is no need
Signed-off-by: zombee0 <flylucas_10@163.com>
2022-04-23 18:42:04 +08:00
alesapin
7380a71333
Merge pull request #31833 from ClickHouse/musl-check
Build clickhouse-keeper with Musl
2022-04-22 22:41:16 +02:00
Nikita Mikhaylov
36bdee0499 Deleted settings 2022-04-22 13:56:16 +00:00
Alexander Tokmakov
b16faab477 fix configs 2022-04-22 15:37:14 +02:00
Kseniia Sumarokova
33bb48106f
Merge pull request #36314 from CurtizJ/print-bad-filenames
Show names of erroneous files in case of parsing errors while executing table functions
2022-04-22 13:24:55 +02:00
taiyang-li
3751c69aaa fix code style 2022-04-22 15:38:19 +08:00
taiyang-li
6bcf24ae36 add test for range generator 2022-04-22 12:33:16 +08:00
Memo
956d525840 fix conflict 2022-04-22 11:22:50 +08:00
Memo
39aadf0975 replaced toStartOfFiveMinute to toStartOfFiveMinutes 2022-04-22 10:49:59 +08:00
Maksim Kita
57444fc7d3
Merge pull request #36444 from rschu1ze/clang-tidy-fixes
Clang tidy fixes
2022-04-21 16:11:27 +02:00
mergify[bot]
1ba1cad5cf
Merge branch 'master' into improve-globs 2022-04-21 11:52:13 +00:00
taiyang-li
adb1288e10 fix bug of range generator 2022-04-21 11:33:15 +08:00
Azat Khuzhin
2be022cb56 Log which taskstats provider is in use
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
v2: add missing __builtin_unreachable() for gcc built [1]
    [1]: https://s3.amazonaws.com/clickhouse-builds/36351/99e72a960325cc1fb3cd3d6a1b56cf2b1cc2d733/binary_gcc/build_log.log
2022-04-20 21:39:26 +03:00
Azat Khuzhin
bfa65b651a Strict taskstats parser
This will unhide possible issues (since sometimes statistics is
inaccurate).

Also before this patch TASKSTATS_TYPE_AGGR_TGID had been handled, while
it not correct to take into account, but it was not requested anyway.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-20 21:39:25 +03:00
Alexander Tokmakov
09daa23bd7 fix 2022-04-20 18:15:21 +02:00
Kruglov Pavel
813e228fcc
Merge branch 'master' into improve-globs 2022-04-20 16:31:47 +02:00
Anton Popov
d4df38a0e6 fix tests 2022-04-20 14:13:04 +00:00
KinderRiven
137a1a3392 fix bug for local cache 2022-04-20 20:22:33 +08:00
KinderRiven
59d56b6391 resubmit 2022-04-20 20:10:31 +08:00
KinderRiven
910c4a6660 fix bug for local cache 2022-04-20 19:49:04 +08:00
KinderRiven
b91f18033c fix bug for local cache 2022-04-20 19:43:07 +08:00
Alexander Tokmakov
1d30a97fd2 Merge branch 'master' into remove-useless-code-2 2022-04-20 11:45:56 +02:00
mergify[bot]
0df5fda6ed
Merge branch 'master' into musl-check 2022-04-20 09:28:40 +00:00
tavplubix
2c54381520
Merge pull request #36200 from ClickHouse/fix_create_as_replicated_default_args
Fix CREATE AS for ReplicatedMergeTree with default arguments
2022-04-20 12:01:13 +03:00
Robert Schulze
b24ca8de52
Fix various clang-tidy warnings
When I tried to add cool new clang-tidy 14 warnings, I noticed that the
current clang-tidy settings already produce a ton of warnings. This
commit addresses many of these. Almost all of them were non-critical,
i.e. C vs. C++ style casts.
2022-04-20 10:29:05 +02:00
Robert Schulze
b6d7367538
Merge remote-tracking branch 'origin/master' into erase_if3
Conflicts:
- Interpreters/ActionsDAG.cpp
2022-04-20 10:02:59 +02:00
Anton Popov
bee4ca9b62 add more tests for error diagnostics in files 2022-04-19 15:56:34 +00:00
Alexey Milovidov
93229133e2 Return two changes 2022-04-19 17:44:27 +02:00
Alexey Milovidov
79be0aa75b Remove Arcadia 2022-04-19 17:44:27 +02:00
Alexey Milovidov
3c1e05eb53 Remove Arcadia 2022-04-19 17:44:27 +02:00
Alexey Milovidov
7629016f66 Fix clang-tidy-14 (part 1) 2022-04-19 17:44:27 +02:00
Alexey Milovidov
b6c06268eb Simplification #36313 2022-04-19 17:44:27 +02:00
Alexey Milovidov
767b2eecff Remove trash 2022-04-19 17:44:27 +02:00
Alexey Milovidov
fb9545e0b4 Remove trash 2022-04-19 17:44:27 +02:00
Alexey Milovidov
611ea86d60 Remove abbreviation 2022-04-19 17:44:26 +02:00
Alexey Milovidov
c46715db82 Remove abbreviation 2022-04-19 17:44:26 +02:00
Anton Popov
3e361c9759 Merge remote-tracking branch 'upstream/master' into HEAD 2022-04-19 14:18:04 +00:00
Robert Schulze
118e94523c
Activate clang-tidy warning "readability-container-contains"
This check suggests replacing <Container>.count() by
<Container>.contains() which is more speaking and in case of
multimaps/multisets also faster.
2022-04-18 23:53:11 +02:00
Alexey Milovidov
f6ab2bd523
Merge pull request #36312 from ClickHouse/remove-arcadia
Remove remaining parts of Arcadia
2022-04-18 07:02:54 +03:00
Alexey Milovidov
36595e4206
Merge pull request #36320 from ClickHouse/fix-clang-tidy-14
Fix clang-tidy-14 (part 1)
2022-04-18 07:02:10 +03:00
Alexey Milovidov
48fd09c7db
Merge pull request #36310 from ClickHouse/cpuid-remove-trash
Remove trash from CPUID
2022-04-18 07:01:23 +03:00
Alexey Milovidov
543e5d7242 Merge branch 'master' into musl-check 2022-04-18 02:40:13 +02:00
Alexey Milovidov
c7bca13a03 Remove abbreviation 2022-04-18 01:03:58 +02:00
Alexey Milovidov
242919eddd Remove abbreviation 2022-04-18 01:02:49 +02:00
Alexey Milovidov
a64765bc6f Allow custom default ORDER BY for system logs 2022-04-17 21:56:36 +02:00
Alexey Milovidov
27afc08d8f Merge branch 'master' into cpuid-remove-trash 2022-04-17 16:30:30 +02:00
Alexey Milovidov
7de4f2b6c1 Merge branch 'master' into remove-arcadia 2022-04-17 16:30:09 +02:00
Robert Schulze
1e1df8e101
Replace remove-erase idiom by C++20 erase()/erase_if()
- makes the code less verbose while being as efficient
2022-04-17 12:04:47 +02:00
Alexey Milovidov
294efeccfe Fix clang-tidy-14 (part 1) 2022-04-16 04:54:04 +02:00
Alexey Milovidov
33ffdcaa4f Return two changes 2022-04-16 02:18:22 +02:00
Anton Popov
2de6668b3f show names of erroneous files 2022-04-16 00:10:47 +00:00
Alexey Milovidov
69b91c5410 Simplification #36313 2022-04-16 02:08:06 +02:00
Alexey Milovidov
40357637ec Remove Arcadia 2022-04-16 00:33:32 +02:00
Alexey Milovidov
cbeeb7ec4f Remove Arcadia 2022-04-16 00:20:47 +02:00
Alexey Milovidov
364f10aff5 Remove trash 2022-04-15 22:56:51 +02:00
Alexey Milovidov
58fc4e033a Remove trash 2022-04-15 22:54:08 +02:00
alesapin
40e5ba21a4
Update src/Common/FileSegment.cpp 2022-04-15 11:55:05 +02:00
Alexander Tokmakov
4ce0b280e6 wait for node to disappear instead of removing 2022-04-14 15:31:05 +02:00
kssenii
3e42a1711d Fix 2022-04-14 13:50:13 +02:00
Kseniia Sumarokova
4d667301cc
Merge pull request #36135 from kssenii/cache-fix
Fix after #35475
2022-04-14 11:11:13 +02:00
Antonio Andelic
4be7cc757d
Merge pull request #36125 from ClickHouse/memory-tracking-overloads
Add new/delete overloads with align
2022-04-14 08:16:51 +02:00
avogar
1c065f8c7a Some refactoring around schema inference with globs 2022-04-13 17:02:48 +00:00
tavplubix
6a165787a6
Merge pull request #36115 from ClickHouse/thread-pool-reset-thread-name
Reset thread name in thread pool
2022-04-13 19:10:04 +03:00
Maksim Kita
ca89af89a1
Merge pull request #36088 from israelg99/fix_executable
Fix CH silently failing when it cannot execute a file
2022-04-13 16:53:00 +02:00
Alexander Tokmakov
40fea2b833 do not unfold {uuid} macro 2022-04-13 16:51:59 +02:00
mergify[bot]
2c5d6d132e
Merge branch 'master' into cache-fix 2022-04-13 13:32:56 +00:00
mergify[bot]
4fb66013fe
Merge branch 'master' into memory-tracking-overloads 2022-04-13 12:40:49 +00:00
tavplubix
43de4827d7
Merge pull request #36074 from ClickHouse/zookeeper_log_thread_query_id
Add thread_id and query_id to zookeeper_log
2022-04-13 15:34:04 +03:00
Alexey Milovidov
ab80b94f45
Update ThreadPool.cpp 2022-04-13 15:18:49 +03:00
Kseniia Sumarokova
07cfa980ac
Update FileCache.h 2022-04-13 12:08:25 +02:00
Kseniia Sumarokova
44ecfd925b
Update FileSegment.cpp 2022-04-13 11:35:46 +02:00
Kseniia Sumarokova
053892777f
Update FileSegment.cpp 2022-04-13 11:35:12 +02:00
Kseniia Sumarokova
a611566266
Update FileCache.h 2022-04-13 11:33:15 +02:00
Mikhail Artemenko
2fd86cc564
Merge branch 'master' into MeiliSearch 2022-04-13 12:05:46 +03:00
Antonio Andelic
835f752a1a Add maybe_unused for align arguments 2022-04-13 06:19:59 +00:00
alesapin
f011864386 Build clickhouse-keeper with musl 2022-04-12 18:48:16 +02:00
mergify[bot]
d5b2355944
Merge branch 'master' into memory-overcommit-free 2022-04-12 16:00:12 +00:00
Dmitry Novik
d9d2bf310d Fix test 2022-04-12 15:58:34 +00:00
Julian Gilyadov
a4f56f3330
Throw exception when file cant be executed instead of displaying success 2022-04-12 17:52:44 +02:00
Antonio Andelic
1b4cebcfb6 Add new/delete overloads 2022-04-12 14:11:10 +00:00
Dmitry Novik
63de3c9746 Add comment with explanation 2022-04-12 13:00:06 +00:00
Dmitry Novik
188e1a87bd Check if enough memory is actually freed within timeout 2022-04-12 12:52:35 +00:00
Kseniia Sumarokova
50b5684d00
Update FileSegment.cpp 2022-04-12 11:59:58 +02:00
Dmitry Novik
9c20a492c8 Add unit tests 2022-04-12 09:46:26 +00:00
Kseniia Sumarokova
a77a228ab8
Update FileSegment.cpp 2022-04-12 09:37:03 +02:00
kssenii
d2ccf876c6 Fix 2022-04-11 17:51:49 +02:00
Raúl Marín
83c814f658 OSX fixes 2022-04-11 15:25:20 +02:00
Raúl Marín
6e346a7fc3 Support atomic replace under OSX 2022-04-11 14:32:55 +02:00
Alexander Tokmakov
802f62a80e fix 2022-04-11 09:42:00 +02:00
Alexey Milovidov
f0fad98a73 Reset thread name in thread pool 2022-04-11 01:03:24 +02:00
Alexander Tokmakov
50a32a74b7 add thread_id and query_id to zookeeper_log 2022-04-08 17:10:08 +02:00
Alexander Tokmakov
49c35f3261 Merge branch 'master' into mvcc_prototype 2022-04-08 13:34:40 +02:00
alesapin
8ec802bc62
Merge pull request #35475 from kssenii/remote-fs-cache-improvements
Allow to write remote fs cache on all write operations. Add `system.remote_filesystem_cache` table. Add `drop remote filesystem cache (<path>)` query. Add `system.remote_data_paths` table.
2022-04-08 12:06:26 +02:00
kssenii
0b7af1f26c Fix checks 2022-04-08 01:58:55 +02:00
kssenii
5dce2f18b5 Better 2022-04-07 18:48:35 +02:00
Alexander Tokmakov
7f54e7b422 Merge branch 'master' into mvcc_prototype 2022-04-07 15:14:06 +02:00
Alexander Tokmakov
8290ffa88d Merge branch 'master' into mvcc_prototype 2022-04-07 13:50:42 +02:00
tavplubix
6e9c028bbb
Merge pull request #35956 from Algunenano/limit_dns_retries
DNS cache: Add option to drop elements after several consecutive failures
2022-04-07 13:20:36 +03:00
Nikolai Kochetov
3e1b3f14c0
Merge pull request #34355 from azat/processors-profiling
Profiling on Processors level
2022-04-07 12:13:14 +02:00
alesapin
d21ed546a4
Merge pull request #35967 from ClickHouse/keeper-polishing
Small code changes in ZooKeeper client/Keeper code
2022-04-07 12:11:10 +02:00
Mikhail Artemenko
0f1119dd2a fix impl after merge 2022-04-07 11:56:30 +03:00
mergify[bot]
585cff1e95
Merge branch 'master' into limit_dns_retries 2022-04-06 18:23:17 +00:00
tavplubix
58db8cd9b3
Merge pull request #35946 from ClickHouse/faster_replicated_ddl
Make some replicated DDL queries faster
2022-04-06 19:25:25 +03:00
Mikhail Artemenko
151eeb1a27
Merge branch 'master' into MeiliSearch 2022-04-06 17:07:55 +03:00
Kseniia Sumarokova
a383d9ee6a
Merge pull request #35916 from filimonov/kafka_metrics
Add some metrics to engine Kafka
2022-04-06 11:58:17 +02:00
Raúl Marín
060f5118f0 Improvements based on PR review 2022-04-06 10:48:12 +02:00
Mikhail Filimonov
3412be9d4d
Change KafkaDirectReads to KafkaConsumersInUse 2022-04-05 20:25:05 +02:00
Raúl Marín
39cc3baf5f Keep the previous exception code 2022-04-05 16:37:53 +02:00
Antonio Andelic
d296eeee2d Small changes for Keeper 2022-04-05 13:56:28 +00:00
Alexander Tokmakov
d9e3e9b69f Merge branch 'master' into faster_replicated_ddl 2022-04-05 14:38:38 +02:00
Alexander Tokmakov
1fe50ad201 Merge branch 'master' into mvcc_prototype 2022-04-05 14:38:02 +02:00
Maksim Kita
c3c284e6e6
Merge pull request #35917 from kitaisreal/executable-user-defined-function-fix
Executable user defined functions prevent executing during analysis
2022-04-05 14:05:52 +02:00
Raúl Marín
8a05cf3927 DNS cache: Add option to drop elements after several consecutive failures 2022-04-05 13:00:27 +02:00
Maksim Kita
6e1d837439 Fixed style check 2022-04-05 11:24:33 +02:00
Alexander Tokmakov
da00beaf7f Merge branch 'master' into mvcc_prototype 2022-04-05 11:14:42 +02:00
Maksim Kita
b160ffd726
Merge pull request #35723 from ClickHouse/array-has-all-sse-avx2-optimizations
Merging #27653
2022-04-05 11:09:14 +02:00
Alexander Tokmakov
4e9ec5dc2f make some replicated DDL faster 2022-04-05 00:51:48 +02:00
larryluogit
808d9afd0f
Fix optin.cplusplus.UninitializedObject issue (#35626)
* Fix optin.cplusplus.UninitializedObject issue

* Enable optin.cplusplus.UninitializedObject
2022-04-04 23:47:14 +03:00
Mikhail Filimonov
53c7376e37
Add some metrics to engine Kafka 2022-04-04 20:25:53 +02:00
Maksim Kita
2a8e479277 ExecutableUserDefinedFunction prevent function execution during query analysis 2022-04-04 15:56:01 +02:00
Alexander Tokmakov
a2167f12b8 Merge branch 'master' into mvcc_prototype 2022-04-04 14:24:23 +02:00
Maksim Kita
47528de78b Fix build 2022-04-04 14:07:05 +02:00
Azat Khuzhin
14538f6456 Add system.processors_profile_log
This is the system table that will contain Processors level profiling.

v2: one entry per Processor, not 3 (PortFull/NeedData/work())
v3: us over ms
v4: Enable processors_profile_log table by default

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-04 13:19:41 +03:00
kssenii
36c583d0de Better version of cache on insert 2022-04-04 10:58:59 +02:00
Antonio Andelic
a926bc19ea Address PR comments 2022-04-04 07:24:42 +00:00
Antonio Andelic
d96b682a55 Refactor 2022-04-04 07:24:42 +00:00
taiyang-li
d6247338de fix failed stateless tests 2022-04-04 07:24:42 +00:00
taiyang-li
eda299b48b fix building 2022-04-04 07:24:42 +00:00
taiyang-li
4547ed370a add hints for column description 2022-04-04 07:24:42 +00:00
Alexey Milovidov
d9e5ca2119
Merge pull request #34394 from holadepo/last_day
Add toLastDayOfMonth function
2022-04-04 07:02:08 +03:00
Alexey Milovidov
5a47958744
Merge pull request #35736 from CurtizJ/quota-written-bytes
Add quota for written bytes
2022-04-03 05:26:49 +03:00
Anton Popov
687942ce70 more strict quota for written bytes 2022-04-01 15:02:49 +00:00
kssenii
6c8e073a61 Merge master 2022-04-01 16:48:29 +02:00
tcoyvwac
cb698c4cc1
Replace throw in debug mode in destructor
FileSegmentsHolder:
- Replaced throw in destructor with assert for all #ifdef modes.
2022-04-01 15:42:34 +02:00
Alexander Tokmakov
6591d1ceb7 Merge branch 'master' into mvcc_prototype 2022-04-01 15:38:46 +02:00
alesapin
f2c6387a8d
Merge pull request #35381 from kssenii/remote-filesystem-cache-fixes
Cache fixes for high concurrency on corner cases
2022-04-01 10:48:31 +02:00
filimonov
24d72a18d0
Update getNumberOfPhysicalCPUCores.cpp
In Kubernetes shares are calculated from resources.requests.cpu, quota & period - from limits.

When you configure only requests without limits - the container still can use all the cores. So you see all the cores the system has.
(because of requests the pod just can't be scheduled on the node which have less cpus)
So the effective CPU count is all the available cores if you don't have limits, or calculated from quota & period if you have the limits.
2022-03-31 17:21:54 +02:00
mergify[bot]
3183b61c74
Merge branch 'master' into mvcc_prototype 2022-03-31 12:27:09 +00:00
Alexander Tokmakov
5a50ad9de3 Merge branch 'master' into mvcc_prototype 2022-03-31 11:35:04 +02:00
alesapin
2365bbf0d3
Merge pull request #34566 from ClickHouse/startup-sanity-checks
add simple sanity checks for server startup
2022-03-31 11:33:59 +02:00
alesapin
81ce991494 Merge branch 'master' into startup-sanity-checks 2022-03-31 00:08:43 +02:00
Nikita Taranov
30f2a942c5
Predict size of hash table for GROUP BY (#33439)
* use AggregationMethod ctor with reserve

* add new settings

* add HashTablesStatistics

* support queries with limit

* support distributed and with external aggregation

* add new profile events

* add some tests

* add perf test

* export cache stats through AsynchronousMetrics

* rm redundant trace

* fix style

* fix 02122_parallel_formatting test

* review fixes

* fix 02122_parallel_formatting test

* apply also to two-level HTs

* try simpler strategy

* increase max_size_to_preallocate_for_aggregation for experiment

* fixes

* Revert "increase max_size_to_preallocate_for_aggregation for experiment"

This reverts commit 6cf6f75704.

* fix test

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-03-30 22:47:51 +02:00
kssenii
0fc92fe2aa Get rid of all "remote" mentions 2022-03-30 17:15:29 +02:00
kssenii
b30948dd53 Relax test 2022-03-30 17:12:23 +02:00
Vladimir C
31c367d3cd
Merge pull request #35651 from amosbird/columntransformerrename 2022-03-30 12:37:30 +02:00
Antonio Andelic
03a09dc914
Merge pull request #35571 from ClickHouse/parallel-s3-downloading
Parallel S3 downloading
2022-03-30 08:14:49 +02:00
kssenii
ce9131f905 Fix unit test 2022-03-29 19:57:55 +02:00
kssenii
f0e0d97752 Fix lask of query id in merges 2022-03-29 19:49:42 +02:00
kssenii
421b1e5a81 Update 2022-03-29 17:33:02 +02:00
Alexander Tokmakov
287d858fda Merge branch 'master' into mvcc_prototype 2022-03-29 16:24:12 +02:00
Antonio Andelic
9990abb76a Use compile-time check for Exception messages, fix wrong messages 2022-03-29 13:16:11 +00:00
alesapin
842c0b8ff0 Merge remote-tracking branch 'origin/startup-sanity-checks' into startup-sanity-checks 2022-03-29 14:29:33 +02:00
alesapin
ed2c461fe1 Merge branch 'master' into startup-sanity-checks 2022-03-29 14:09:20 +02:00
kssenii
7a70e3a6da Fix 2022-03-29 14:08:24 +02:00
kssenii
eab04674fa Fix review comments 2022-03-29 12:38:17 +02:00
taiyang-li
834ac3a1b9 Merge remote-tracking branch 'ck/master' into rocksdb_metacache 2022-03-29 18:32:06 +08:00
Antonio Andelic
362e877529 Merge branch 'master' into parallel-s3-downloading 2022-03-29 07:05:26 +00:00
Alexey Milovidov
5e262fba85
Merge pull request #35204 from azat/build-gcc
Add build with GCC
2022-03-29 04:55:15 +03:00
kssenii
44157e5c80 Merge master 2022-03-29 02:19:32 +02:00
mergify[bot]
9efc8bda78
Merge branch 'master' into remote-filesystem-cache-fixes 2022-03-28 20:53:29 +00:00
Alexander Tokmakov
208b242188 Merge branch 'master' into mvcc_prototype 2022-03-28 19:58:06 +02:00
Dmitry Novik
a767605f0e Remove redundant mutable 2022-03-28 15:36:25 +00:00
Dmitry Novik
b34c1c02b7 Fix typo 2022-03-28 15:05:37 +00:00
Dmitry Novik
2017361e00 Do not cancel query if memory is free 2022-03-28 14:54:13 +00:00
Dmitry Novik
31566cdd78 Fix typo 2022-03-28 13:49:07 +00:00
taiyang-li
38702be40c Merge remote-tracking branch 'ck/master' into rocksdb_metacache 2022-03-28 21:03:22 +08:00
Amos Bird
6003a49a87
Fix style 2022-03-28 17:48:47 +08:00
Antonio Andelic
d9d826c813 Address PR review 2022-03-28 08:19:23 +00:00
Amos Bird
58a78fba12
formatString bug fix and refactoring 2022-03-28 15:47:48 +08:00
Antonio Andelic
cbe8e747b4 Merge branch 'master' into parallel-s3-downloading 2022-03-28 07:04:45 +00:00
Alexey Milovidov
6fbf5bd9ad More correct 2022-03-28 02:11:27 +02:00
Alexey Milovidov
bb35184da1 Add metric about size of async INSERTs 2022-03-28 02:04:19 +02:00
Dmitry Novik
95c9a03b9c Handle corner case 2022-03-27 16:56:41 +00:00
Azat Khuzhin
6da2c48b7a Add missing __builtin_unreachable for switch/case for gcc
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-26 23:42:21 +03:00
Dmitry Novik
0d75e773ab Continue query execution if memory is freed 2022-03-26 18:25:58 +00:00
alesapin
14e84ff6ef
Merge pull request #34802 from ClickHouse/overcommit-doc
Add documentation for memory overcommit
2022-03-25 20:50:44 +01:00
kssenii
141c290344 Fix build 2022-03-25 18:31:15 +01:00
Antonio Andelic
394682b983 Merge branch 'master' into parallel-s3-downloading 2022-03-25 13:00:44 +00:00
Kruglov Pavel
2f2d3cdbca
Merge pull request #35578 from ClickHouse/Avogar-patch-3
Increase fiber stack size a bit in attempt to fix stack overflow in tests with address sanitizer
2022-03-25 00:33:05 +01:00
Maksim Kita
7b03617341
Merge pull request #35539 from kitaisreal/parse-ipv6-long-fix
Fix parsing of IPv6 addresses longer than 39 characters
2022-03-24 22:28:51 +01:00
kssenii
0c6577eee0 Merge master 2022-03-24 20:49:02 +01:00
kssenii
6769b84638 Merge master 2022-03-24 20:47:43 +01:00
Dmitry Novik
56d20e00cc
Update src/Common/OvercommitTracker.cpp
Co-authored-by: alesapin <alesapin@clickhouse.com>
2022-03-24 17:47:48 +01:00
Kruglov Pavel
f07918c659
Increase stack size 2022-03-24 15:50:32 +01:00
kssenii
d6ab6de30a Minor changes 2022-03-24 15:32:08 +01:00
Sergei Trifonov
2e2a1c06a2 Merge branch 'startup-sanity-checks' of github.com:ClickHouse/ClickHouse into startup-sanity-checks 2022-03-24 15:30:59 +01:00
Sergei Trifonov
041e348076 cosmetic bugfix 2022-03-24 15:30:10 +01:00
mergify[bot]
394fdb4fb8
Merge branch 'master' into startup-sanity-checks 2022-03-24 14:23:38 +00:00
Kruglov Pavel
98be162d20
Update comment 2022-03-24 13:49:26 +01:00
Kruglov Pavel
37286c6141
Increase fiber stack size a bit in attempt to fix stack overflow in tests with address sanitizer 2022-03-24 12:55:56 +01:00
tavplubix
7b96d61057
Merge pull request #35478 from ClickHouse/merging_30325
Merging #30325
2022-03-24 14:53:20 +03:00
kssenii
142a5938a1 Merge master 2022-03-24 12:10:26 +01:00
alesapin
d3fbd9b2ad Fix style 2022-03-24 11:25:28 +01:00
alesapin
93d1b1d198 Merge branch 'master' into startup-sanity-checks 2022-03-24 11:17:06 +01:00
Antonio Andelic
c52cee2219 Merge branch 'master' into parallel-s3-downloading 2022-03-24 08:11:52 +00:00
kssenii
d4161b5925 Add optin read_from_cache_if_exists_otherwise_bypass_cache (for merges) 2022-03-23 20:24:00 +01:00
mergify[bot]
b745b67026
Merge branch 'master' into parse-ipv6-long-fix 2022-03-23 18:56:40 +00:00
mergify[bot]
abbed251c1
Merge branch 'master' into asof-join-improve-performance 2022-03-23 18:53:17 +00:00
Sergei Trifonov
c7cf14e743 fix style and darwin build 2022-03-23 19:15:01 +01:00
kssenii
d2a3cfe5dc Cache on all write operations 2022-03-23 19:14:33 +01:00