Commit Graph

30765 Commits

Author SHA1 Message Date
Daniil Rubin
66c9305668 Add Hudi engine 2022-09-07 11:05:40 +03:00
Daniil Rubin
842c636354 Undone hudi 2022-09-07 11:05:40 +03:00
Daniil Rubin
c5ae712651 Add list files on s3 2022-09-07 11:05:40 +03:00
Daniil Rubin
7f46b6a48b Add Hudi engine template 2022-09-07 11:05:40 +03:00
Nikolay Degterinsky
981e9dbce2
Merge pull request #40997 from canhld94/ch_canh_fix_grouping_set
Fix grouping set with group_by_use_nulls
2022-09-07 00:08:31 +02:00
Nikita Taranov
7c4f42d014
Skip empty literals in lz4 decompression (#40142) 2022-09-06 13:58:26 +02:00
Sergei Trifonov
f77809ddbc
Merge pull request #40900 from ClickHouse/s3-detailed-metrics
S3 detailed metrics
2022-09-06 13:20:57 +02:00
Duc Canh Le
6950016b8a fix grouping set with group_by_use_nulls 2022-09-06 09:39:27 +08:00
Alexey Milovidov
59bd94dcf8
Merge pull request #40975 from vitlibar/fix-access-rights-for-describe
Fix access rights for DESCRIBE TABLE
2022-09-06 02:38:34 +03:00
Alexey Milovidov
940a53e519
Merge pull request #40984 from Lucky-Chang/typo_fix
Fix some typos and clang-tidy warnings
2022-09-06 02:37:50 +03:00
Alexey Milovidov
9091995934
Merge pull request #40978 from azat/log-ignore-memory-limits
[RFC] Ignore memory limits while logging
2022-09-06 02:37:16 +03:00
Alexey Milovidov
4c30dbc905
Merge pull request #40968 from ClickHouse/s3-sharding-2
Sharding s3 key names (2)
2022-09-05 22:47:25 +03:00
Alexander Tokmakov
b264be3c63
Merge branch 'master' into zookeeper_client_fault_injection 2022-09-05 22:13:09 +03:00
Kruglov Pavel
cadae5a2f6
Merge pull request #40896 from kssenii/fix-memory-usage-for-remote
Fix extra memory allocation for remote read buffers
2022-09-05 20:38:57 +02:00
Sergei Trifonov
1c247308f1
Merge branch 'master' into s3-detailed-metrics 2022-09-05 15:55:41 +02:00
Kseniia Sumarokova
e12858dca5
Merge pull request #36837 from ClickHouse/stress_s3
Trying stress on top of S3
2022-09-05 15:12:55 +02:00
Anton Popov
f11b7499d1
Merge pull request #40592 from azat/inmemory-WAL-cleanup
Remove completely processed WAL files
2022-09-05 14:15:19 +02:00
Alexander Tokmakov
abffd4595e
Merge branch 'master' into zookeeper_client_fault_injection 2022-09-05 15:03:26 +03:00
Kruglov Pavel
582216a3ca
Merge pull request #39919 from pzhdfy/UniqSketch
UniqThetaSketch support set operation such as union/intersect/not
2022-09-05 13:42:14 +02:00
Nikita Mikhaylov
33e530392e
Check grants for databases in case of renaming databases (#40906) 2022-09-05 12:38:44 +02:00
Azat Khuzhin
313747688e Ignore memory limits while logging
Otherwise there can be pretty noisy messages, like here [1]:

    Cannot add message to the log: Code: 241. DB::Exception: Memory limit (for user) exceeded: would use 42.02 MiB (attempt to allocate chunk of 2101278 bytes), maximum: 4.77 MiB. OvercommitTracker decision: Waiting timeout for memory to be freed is reached. (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/0/efc74e33e580b7d3ff33c3e269db86addfaa6b51/stateless_tests__msan__[2/3].html

And it should be okay to ignore server limits too, since it always
better to have a proper log message in a proper place, not stderr which
does not inspected usually.

Follow-up for: #24069
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-09-05 12:20:15 +02:00
Igor Nikonov
025c7a1ae0
Merge pull request #40305 from ClickHouse/sort_mode_rename
Sorting mode renaming
2022-09-05 11:59:35 +02:00
Kseniia Sumarokova
64168da410
Merge pull request #40950 from kssenii/fix-races-in-write-buffer-from-s3
Fix race in WriteBufferFromS3, add TSA annotations
2022-09-05 11:58:42 +02:00
Antonio Andelic
3a0581e990
Merge pull request #40543 from Lloyd-Pottiger/feat/support-read-only-for-embeddedrocksdb
Add read-only support for EmbeddedRocksDB
2022-09-05 09:31:07 +02:00
Luck-Chang
0800c0fd4c fix some typos 2022-09-05 14:09:42 +08:00
Alexey Milovidov
1364685eff
Merge branch 'master' into s3-sharding-2 2022-09-05 08:39:19 +03:00
Luck-Chang
1ac8e739c9 fix some typos and clang-tidy warnings 2022-09-05 09:50:24 +08:00
Alexey Milovidov
bff96b0af6
Merge branch 'master' into fix-access-rights-for-describe 2022-09-05 04:34:57 +03:00
Alexey Milovidov
79e3723272 Merge branch 'master' into s3-sharding-2 2022-09-05 02:24:35 +02:00
Alexey Milovidov
d20d49f39f
Merge pull request #40958 from arenadata/ADQM-566
stylecheck renewed and version in docker set explicitely
2022-09-05 03:23:38 +03:00
alesapin
45b1f9dc41
Merge branch 'master' into stress_s3 2022-09-04 23:17:10 +02:00
Igor Nikonov
8fece1e2d2
Merge branch 'master' into sort_mode_rename 2022-09-04 21:44:33 +02:00
kssenii
e2df4b1da1 Merge remote-tracking branch 'origin/fix-races-in-write-buffer-from-s3' into fix-races-in-write-buffer-from-s3 2022-09-04 19:50:19 +02:00
kssenii
52375d2eb0 Merge master 2022-09-04 19:48:04 +02:00
Vitaly Baranov
cfb250c46b Fix checking access rights for DESCRIBE TABLE url. 2022-09-04 18:58:39 +02:00
Nikita Taranov
16af4aebc8
impl (#40952) 2022-09-04 16:26:31 +02:00
Azat Khuzhin
2e85f9f0ad Remove completely processed WAL files
Previously all WAL files had been stored, though with the time of use
this can take too much space on disk, and also the startup time will be
increased.

But it is pretty easy to prune old WAL files (the one parts from which
had been completely written to disk already).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-09-04 14:18:14 +02:00
Kseniia Sumarokova
5f757be43d
Merge pull request #40943 from kssenii/fix-deadlock-in-s3-write-buffer
Fix deadlock in WriteBufferFromS3
2022-09-04 13:17:41 +02:00
alesapin
a67703b76b
Merge branch 'master' into stress_s3 2022-09-04 13:00:28 +02:00
Alexey Milovidov
712de53033 Sharding s3 key names 2022-09-04 04:15:45 +02:00
Alexey Milovidov
193cd1b3b2
Merge pull request #39138 from nickitat/control_block_size_in_aggregator
Control block size in aggregator
2022-09-04 04:51:00 +03:00
Alexey Milovidov
dbca269ec7
Merge pull request #40321 from azat/mem/untracked_memory_limit_increase
Remove ThreadStatus::untracked_memory_limit_increase
2022-09-04 04:42:24 +03:00
Alexey Milovidov
097221a3af
Merge pull request #40726 from amosbird/projection-fix-1
Disable projection when grouping set is used.
2022-09-04 04:23:30 +03:00
Igor Nikonov
5d7fa55f36
Merge branch 'master' into sort_mode_rename 2022-09-02 23:19:04 +02:00
Yakov Olkhovskiy
fdcced8962
Merge pull request #40850 from canhld94/ch_canh_fix_set
Fix tryGetPreparedSet crash when same set expression built from different column(s)
2022-09-02 14:40:35 -04:00
Alexander Tokmakov
2044470895 review fixes 2022-09-02 19:18:44 +02:00
Kseniia Sumarokova
1e81e0ef67
Update WriteBufferFromS3.cpp 2022-09-02 18:54:34 +02:00
Kseniia Sumarokova
7dea71c83f
Merge pull request #40884 from TKaxe/fix_removeUnneededColumn
fix bug when remove unneeded columns in subquery
2022-09-02 18:51:18 +02:00
Kseniia Sumarokova
1370bf0de7
Merge pull request #40820 from kssenii/cache-assert-directory-ownership
Create status file for cache directory
2022-09-02 18:42:11 +02:00
kssenii
239847ad16 Fix races 2022-09-02 18:27:07 +02:00