Alexey Milovidov
bff96b0af6
Merge branch 'master' into fix-access-rights-for-describe
2022-09-05 04:34:57 +03:00
Alexey Milovidov
79e3723272
Merge branch 'master' into s3-sharding-2
2022-09-05 02:24:35 +02:00
Alexey Milovidov
d20d49f39f
Merge pull request #40958 from arenadata/ADQM-566
...
stylecheck renewed and version in docker set explicitely
2022-09-05 03:23:38 +03:00
alesapin
45b1f9dc41
Merge branch 'master' into stress_s3
2022-09-04 23:17:10 +02:00
Igor Nikonov
8fece1e2d2
Merge branch 'master' into sort_mode_rename
2022-09-04 21:44:33 +02:00
kssenii
48dc32faf8
Better test
2022-09-04 19:54:34 +02:00
kssenii
e2df4b1da1
Merge remote-tracking branch 'origin/fix-races-in-write-buffer-from-s3' into fix-races-in-write-buffer-from-s3
2022-09-04 19:50:19 +02:00
kssenii
52375d2eb0
Merge master
2022-09-04 19:48:04 +02:00
Vitaly Baranov
cfb250c46b
Fix checking access rights for DESCRIBE TABLE url.
2022-09-04 18:58:39 +02:00
Nikita Taranov
16af4aebc8
impl ( #40952 )
2022-09-04 16:26:31 +02:00
Azat Khuzhin
2e85f9f0ad
Remove completely processed WAL files
...
Previously all WAL files had been stored, though with the time of use
this can take too much space on disk, and also the startup time will be
increased.
But it is pretty easy to prune old WAL files (the one parts from which
had been completely written to disk already).
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-09-04 14:18:14 +02:00
Kseniia Sumarokova
5f757be43d
Merge pull request #40943 from kssenii/fix-deadlock-in-s3-write-buffer
...
Fix deadlock in WriteBufferFromS3
2022-09-04 13:17:41 +02:00
alesapin
a67703b76b
Merge branch 'master' into stress_s3
2022-09-04 13:00:28 +02:00
Alexey Milovidov
712de53033
Sharding s3 key names
2022-09-04 04:15:45 +02:00
Alexey Milovidov
193cd1b3b2
Merge pull request #39138 from nickitat/control_block_size_in_aggregator
...
Control block size in aggregator
2022-09-04 04:51:00 +03:00
Alexey Milovidov
dbca269ec7
Merge pull request #40321 from azat/mem/untracked_memory_limit_increase
...
Remove ThreadStatus::untracked_memory_limit_increase
2022-09-04 04:42:24 +03:00
Alexey Milovidov
660c1439ec
Fix build
2022-09-04 03:27:55 +02:00
Alexey Milovidov
13a129bee7
Merge branch 'master' into remove-useless-line
2022-09-04 03:24:40 +02:00
Alexey Milovidov
097221a3af
Merge pull request #40726 from amosbird/projection-fix-1
...
Disable projection when grouping set is used.
2022-09-04 04:23:30 +03:00
Alexey Milovidov
22f7bfdcb5
Update gtest_sensitive_data_masker.cpp
2022-09-04 03:42:58 +03:00
Alexey Milovidov
c6f7ddc5e6
Merge branch 'master' into sensitive_data_masker_unittest_issue
2022-09-04 03:42:24 +03:00
Azat Khuzhin
9479e2143f
Do not try to load empty debug files
...
This will avoid CANNOT_PARSE_ELF error for builds that has empty debug
file in clickhouse-common-static-dbg package, i.e. debug build.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-09-02 23:23:05 +02:00
Azat Khuzhin
597197c01d
Improve error messages in Elf loader
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-09-02 23:22:25 +02:00
Azat Khuzhin
45afaa6fb8
Fix loading external symbols
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-09-02 23:22:25 +02:00
Igor Nikonov
5d7fa55f36
Merge branch 'master' into sort_mode_rename
2022-09-02 23:19:04 +02:00
Yakov Olkhovskiy
fdcced8962
Merge pull request #40850 from canhld94/ch_canh_fix_set
...
Fix tryGetPreparedSet crash when same set expression built from different column(s)
2022-09-02 14:40:35 -04:00
kssenii
2dcf3fef94
Fix tests
2022-09-02 19:27:52 +02:00
Alexander Tokmakov
2044470895
review fixes
2022-09-02 19:18:44 +02:00
Suzy Wang
83b029e716
re arrange sensitive data masker unit test scenario order
2022-09-02 10:11:57 -07:00
Kseniia Sumarokova
1e81e0ef67
Update WriteBufferFromS3.cpp
2022-09-02 18:54:34 +02:00
Kseniia Sumarokova
7dea71c83f
Merge pull request #40884 from TKaxe/fix_removeUnneededColumn
...
fix bug when remove unneeded columns in subquery
2022-09-02 18:51:18 +02:00
Kseniia Sumarokova
1370bf0de7
Merge pull request #40820 from kssenii/cache-assert-directory-ownership
...
Create status file for cache directory
2022-09-02 18:42:11 +02:00
kssenii
239847ad16
Fix races
2022-09-02 18:27:07 +02:00
Anton Popov
dd776eb3d5
fix enumerateStreams
2022-09-02 16:26:38 +00:00
Yakov Olkhovskiy
5727517713
add variadic constructor
2022-09-02 16:22:57 +00:00
Anton Popov
464818c142
try to fix filling of missed Nested columns with multiple levels
2022-09-02 15:05:58 +00:00
Sergei Trifonov
067e712fdf
Merge branch 'master' into s3-detailed-metrics
2022-09-02 16:59:52 +02:00
Sergei Trifonov
3daf1f5c77
fix build
2022-09-02 16:57:51 +02:00
Ilya Golshtein
74561b40ae
stylecheck renewed
2022-09-02 17:42:18 +03:00
Lloyd-Pottiger
5db220c7fb
Merge branch 'master' of github.com:ClickHouse/ClickHouse into feat/support-read-only-for-embeddedrocksdb
2022-09-02 22:40:28 +08:00
Kruglov Pavel
77071381e4
fix build
2022-09-02 16:37:33 +02:00
kssenii
c9b512e33e
Fix
2022-09-02 16:15:54 +02:00
kssenii
0db38fcbc9
Fix
2022-09-02 15:32:46 +02:00
Anton Popov
9dda9658a8
Merge remote-tracking branch 'upstream/master' into HEAD
2022-09-02 12:48:27 +00:00
Vladimir C
963c0111bf
Merge pull request #39418 from vdimir/join_and_sets
...
Filter joined streams for `full_sorting_join` by each other before sorting
2022-09-02 13:57:06 +02:00
alesapin
3f02995161
Better exceptions handling
2022-09-02 13:54:41 +02:00
Antonio Andelic
b816d46520
Merge branch 'master' into fix-keeper-system-path-check
2022-09-02 13:42:16 +02:00
alesapin
521e2cc22e
Merge branch 'master' into stress_s3
2022-09-02 12:57:29 +02:00
alesapin
45fe9c800a
Merge pull request #40898 from georgthegreat/patch-2
...
Remove remaining usage of // Y_IGNORE magic comment
2022-09-02 12:24:57 +02:00
Antonio Andelic
9ca20b97c2
Merge branch 'master' into keeper-storage
2022-09-02 12:13:07 +02:00
Alexander Tokmakov
3582a51a72
Merge pull request #40920 from ClickHouse/fix-typos
...
Fix typos found by new `codespell`
2022-09-02 13:11:16 +03:00
Lloyd-Pottiger
5a6b2106b5
Merge branch 'master' of github.com:ClickHouse/ClickHouse into feat/support-read-only-for-embeddedrocksdb
2022-09-02 18:10:00 +08:00
Alexander Tokmakov
5b6a8677ef
Merge pull request #40890 from arthurpassos/patch-1
...
CaresPTRResolver small safety improvement
2022-09-02 12:29:23 +03:00
Antonio Andelic
e64436fef3
Fix typos with new codespell
2022-09-02 08:54:48 +00:00
Antonio Andelic
ad2196155c
Add test for system node modification
2022-09-02 08:14:06 +00:00
Antonio Andelic
b7eebfc626
Correctly check if the node is using system path
2022-09-02 07:47:12 +00:00
Antonio Andelic
f5fc3db0dd
Small fix
2022-09-02 07:42:22 +00:00
Antonio Andelic
1a48beec30
Use server UUID
2022-09-02 07:40:39 +00:00
Antonio Andelic
dbf67f26d8
Add prefix path
2022-09-02 07:24:26 +00:00
Robert Schulze
319d8b00a7
Merge pull request #39010 from FrankChen021/tracing_context_propagation
...
Improve the opentelemetry tracing context propagation across threads
2022-09-02 07:56:43 +02:00
Robert Schulze
c7c00f9002
Merge pull request #40739 from ClickHouse/clang-tidy-for-headers
...
Enable clang-tidy for headers
2022-09-02 07:54:50 +02:00
Yakov Olkhovskiy
b6e3680d60
test with TLS
2022-09-02 04:47:04 +00:00
Duc Canh Le
65b64010a0
move out-of-bound check into set::areTypesEqual
2022-09-02 11:56:13 +08:00
Duc Canh Le
763328ef53
Merge branch 'master' into ch_canh_fix_set
2022-09-02 11:45:13 +08:00
Lloyd-Pottiger
59dccd6e49
fix test
...
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
2022-09-02 11:14:49 +08:00
Sergei Trifonov
29e922f05c
support all profile events for disk s3
2022-09-02 02:22:52 +02:00
Yakov Olkhovskiy
e7959d74bc
add composable factory and protocol
2022-09-01 22:06:54 +00:00
Kruglov Pavel
5a9d2460dd
Merge pull request #40851 from Avogar/fix-json-object-inference
...
Fix nested JSON Objects schema inference
2022-09-01 21:07:21 +02:00
Kruglov Pavel
7a4a65bc36
Make better exception message in schema inference
2022-09-01 20:36:08 +02:00
Kruglov Pavel
f53aa86a20
Merge pull request #40485 from arthurpassos/fix-parquet-chunked-array-deserialization
...
Add support for extended (chunked) arrays for Parquet format
2022-09-01 19:40:40 +02:00
Sergei Trifonov
66b5cf566c
fix bugs
2022-09-01 19:02:04 +02:00
Sergei Trifonov
ac9dc53c73
Merge branch 'master' into s3-detailed-metrics
2022-09-01 18:31:02 +02:00
Sergei Trifonov
bcb6475c4a
add separate s3 profile events for disk s3
2022-09-01 18:30:55 +02:00
Dmitry Novik
ddadb362cf
Merge pull request #39762 from quickhouse/betterorderbyoptimization
...
Fixed `Unknown identifier (aggregate-function)` exception which appears when a user tries to calculate WINDOW ORDER BY/PARTITION BY expressions over aggregate functions
2022-09-01 18:08:06 +02:00
Dmitry Novik
2ca619d40f
Merge pull request #40320 from azat/memory-tracker-fix-peak-race
...
Fix rare race in updating peak of memory tracker
2022-09-01 18:07:33 +02:00
Frank Chen
9d63cbe811
Merge 'origin/master' into tracing_context_propagation to resolve conflicts
2022-09-01 23:18:59 +08:00
Frank Chen
1da35a486c
Disable query_span for internal queries
2022-09-01 23:16:05 +08:00
Yuriy Chernyshov
304dc30f0f
Remove remaining usage of // Y_IGNORE magic comment
2022-09-01 18:15:56 +03:00
alesapin
dbe3b777ec
Fix build
2022-09-01 17:07:56 +02:00
kssenii
ec55c90f36
Fix
2022-09-01 17:04:34 +02:00
alesapin
89930c315b
Merge branch 'master' into stress_s3
2022-09-01 17:03:55 +02:00
kssenii
4fa291cf59
Fix hdfs read buffer too
2022-09-01 16:46:46 +02:00
Robert Schulze
1bbbae8da7
Merge pull request #40785 from ClickHouse/remove-unnecessary-context-access-method
...
Minor: remove unnecessary ContextAccess::make() method
2022-09-01 16:41:06 +02:00
Antonio Andelic
9231084c57
Style fix
2022-09-01 14:40:50 +00:00
Antonio Andelic
a0c1ce5eb7
Merge branch 'master' into embeddedrocksdb-delete-update-support
2022-09-01 14:40:24 +00:00
kssenii
3134bdeb13
Fix
2022-09-01 16:35:28 +02:00
Alexander Tokmakov
04d431793b
Update src/Common/ZooKeeper/ZooKeeperImpl.cpp
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2022-09-01 17:12:22 +03:00
Alexander Tokmakov
36454a340d
Update src/Common/ZooKeeper/ZooKeeperArgs.cpp
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2022-09-01 17:12:16 +03:00
Vladimir C
12e6fc4182
Merge branch 'master' into join_and_sets
2022-09-01 14:56:14 +02:00
Vladimir C
f2cf7d7762
Merge pull request #40334 from ClickHouse/vdimir/analyze-stuck-limit
...
Limit number of analyze for one query, att. 2
2022-09-01 14:40:20 +02:00
Kseniia Sumarokova
c6c67a248d
Merge pull request #40792 from canhld94/ch_canh_intersect_distinct
...
Implement intersect + except distinct
2022-09-01 14:35:26 +02:00
Antonio Andelic
62ff61404f
Improve remove recursive
2022-09-01 12:16:13 +00:00
Arthur Passos
fb42afbbac
CaresPTRResolver small safety improvement
...
Previous to #40769 , only `hostent::h_aliases` was being accessed. After that PR got merged, `hostent::h_name` started being accessed as well. This PR moves the first `hostent::h_aliases != nullptr` check that could prevent `hostent::h_name` from being accessed. During debugging, I observed that even when there are not aliases, `hostent::h_aliases` is not null. That's why it hasn't caused any problems, but proposing this change to be on the safe side.
2022-09-01 08:59:14 -03:00
Fangyuan Deng
bc7d661668
Merge branch 'master' into UniqSketch
2022-09-01 19:31:53 +08:00
pzhdfy
acec516271
add docs
2022-09-01 19:31:01 +08:00
Nikita Mikhaylov
c7b0b87574
Better exception handling in DiskObjectStorageTransaction ( #40813 )
...
Co-authored-by: alesapin <alesapin@clickhouse.com>
2022-09-01 13:29:59 +02:00
alesapin
f2bcd63618
Merge pull request #40844 from ClickHouse/fix_incorrect_remove_in_zero_copy
...
Fix incorrect directory remove in zero copy replication
2022-09-01 13:21:58 +02:00
alesapin
1ae7e82126
Merge pull request #40819 from vitlibar/fix-locking-when-writing-backup
...
Fix locking while writing backup in multiple threads
2022-09-01 13:20:21 +02:00
Vitaly Baranov
6dcca686cb
Merge pull request #40827 from vitlibar/fix-incremental-backups-for-log-family
...
Fix incremental backups for Log family.
2022-09-01 13:13:32 +02:00
kssenii
82879543d3
Merge remote-tracking branch 'upstream/master' into improve-marks-cache-loading
2022-09-01 13:13:01 +02:00
kssenii
9ae95f8c04
Fix unit test
2022-09-01 13:11:47 +02:00
kssenii
ca4363e2e3
Merge master
2022-09-01 13:09:00 +02:00
Anton Popov
f7bdf07adc
Merge pull request #38715 from CurtizJ/fix-read-in-order-fixed-prefix
...
Better support of `optimize_read_in_order` in case of fixed prefix of sorting key
2022-09-01 12:59:18 +02:00
Anton Popov
cc49198204
Merge pull request #40754 from CurtizJ/fix-fuzzer-with-object
...
Avoid logical errors in some common operations with `ColumnObject`
2022-09-01 12:57:32 +02:00
Antonio Andelic
6442824da8
Fix KeeperMap with Replicated database
2022-09-01 10:47:05 +00:00
kssenii
0d1f7afb17
Merge remote-tracking branch 'upstream/master' into fix-removing-directories
2022-09-01 12:28:51 +02:00
Kseniia Sumarokova
849262bd99
Merge pull request #40858 from kssenii/move-cache-files
...
Move cache files from Common
2022-09-01 12:28:00 +02:00
Sema Checherinda
7b59fdc042
Merge pull request #40779 from CheSema/detached-parts-metric
...
Metric for the number of detached parts
2022-09-01 12:24:42 +02:00
Duc Canh Le
018f4118bb
Fix view with merge table function ( #40734 )
2022-09-01 12:20:00 +02:00
luocongkai
56e4179cad
fix bug when remove unneeded columns in subquery
2022-09-01 17:10:32 +08:00
Robert Schulze
b3fc6eafe7
Merge pull request #40750 from ClickHouse/last-command-shortcut
...
Make last command available in clickhouse-client interactive mode
2022-09-01 10:27:31 +02:00
Robert Schulze
de64c6b103
Merge branch 'master' into clang-tidy-for-headers
2022-09-01 10:24:56 +02:00
Robert Schulze
560b6082ee
Merge branch 'master' into remove-unnecessary-context-access-method
2022-09-01 10:10:31 +02:00
Antonio Andelic
c7d1dd942c
Address PR comments
2022-09-01 07:56:06 +00:00
Robert Schulze
56eece40ec
Merge pull request #40736 from LevyCory/add-offset-to-formatDateTime
...
Add timezone offset support to `formatDateTime`
2022-09-01 09:50:17 +02:00
Antonio Andelic
cecdcb5059
Merge pull request #40627 from ClickHouse/fix-install-snapshot
...
Keeper fix for install snapshot and preprocessing performance improvements
2022-09-01 09:13:01 +02:00
Robert Coelho
19097c433d
fix compile error
2022-09-01 01:34:41 -04:00
Frank Chen
d9a516288c
Resolve comments
2022-09-01 11:56:10 +08:00
Robert Coelho
e0597700eb
Fix uncaught DNS_ERROR on failed connection to replicas
...
DNS_ERROR would cause the replica to not be marked as unusable, resulting in the replica being repeatedly reattempted on subsequent queries and for connection failover to break.
(This is common in Kubernetes setups where a replica has failed and it's DNS record is returning NXDOMAIN)
On SELECT, would additionally result in an intermittent query error if the failed replica is chosen:
"Code: 198. DB::Exception: Received from localhost:9000. DB::Exception: Not found address of host: chi-clickhouse-main-2-0: While executing Remote. (DNS_ERROR)"
2022-08-31 20:34:14 -04:00
Nikolay Degterinsky
e98ceb2575
Merge pull request #40251 from ClibMouse/Structured-logging-custom-keys
...
Feature Improvement: Custom Key Names for Structured Logging Support
2022-09-01 00:36:23 +02:00
kssenii
10a7e73d52
Fix unit test
2022-08-31 21:07:39 +02:00
Robert Schulze
912663b719
Revert "Move CatBoost evaluation into clickhouse-library-bridge"
2022-08-31 20:54:43 +02:00
alesapin
251bdab9a0
Better grep
2022-08-31 20:08:47 +02:00
kssenii
a78571de11
Fix
2022-08-31 18:47:47 +02:00
Kruglov Pavel
86516d3bb4
Merge pull request #40740 from amosbird/row-policy-index-fix-1
...
Use index when row_policy_filter is always false
2022-08-31 18:46:14 +02:00
Kruglov Pavel
b7ea529307
Merge pull request #40746 from Avogar/fix-csv-schema-inference
...
Fix possible error Attempt to read after eof in CSV schema inference
2022-08-31 18:29:01 +02:00
Kseniia Sumarokova
6e61a4f49f
Update unit test
2022-08-31 18:07:40 +02:00
kssenii
0f747e7987
Move cache files
2022-08-31 17:50:55 +02:00
Antonio Andelic
f043790e6d
Add support for update
2022-08-31 14:33:53 +00:00
Robert Schulze
ca01286028
Merge pull request #39629 from ClickHouse/catboost-bridge
...
Move CatBoost evaluation into clickhouse-library-bridge
2022-08-31 16:16:11 +02:00
avogar
acf87c1d10
Fix nested JSON Objects schema inference
2022-08-31 14:10:29 +00:00
Anton Popov
5a3e24c4e4
Merge pull request #40483 from canhld94/ch_canh_fix_json
...
Fix columnObject crash with array of variadic dimension elems
2022-08-31 16:07:58 +02:00
Duc Canh Le
55c20ce45b
fix tryGetPreparedSet crash when same set expression built from different columns
2022-08-31 21:43:08 +08:00
kssenii
3a183236a2
Fix
2022-08-31 15:39:53 +02:00
alesapin
10b456ce64
Merge branch 'master' into fix-locking-when-writing-backup
2022-08-31 15:12:23 +02:00
Antonio Andelic
0e6b3b870a
Add support for delete in RocksDB
2022-08-31 13:08:27 +00:00
alesapin
c427491bcb
Better
2022-08-31 15:05:45 +02:00
alesapin
a4f71ba6b8
Fix one more bug
2022-08-31 14:53:33 +02:00
alesapin
2d976753e1
Fix incorrect directory remove in zero copy replication
2022-08-31 13:50:59 +02:00
Vitaly Baranov
007ae0e6cc
Fix incremental backups for Log family.
2022-08-31 12:57:28 +02:00
Robert Schulze
cedf75ed5e
Enable clang-tidy for headers
...
clang-tidy now also checks code in header files. Because the analyzer
finds tons of issues, activate the check only for directory "base/" (see
file ".clang-tidy"). All other directories, in particular "src/" are
left to future work.
While many findings were fixed, some were not (and suppressed instead).
Reasons for this include: a) the file is 1:1 copypaste of a 3rd-party
lib (e.g. pcg_extras.h) and fixing stuff would make upgrades/fixes more
difficult b) a fix would have broken lots of using code
2022-08-31 10:48:15 +00:00
Antonio Andelic
7d5e13c6ba
Use async exist for write
2022-08-31 10:40:32 +00:00
Antonio Andelic
9dd1a9859d
Merge branch 'master' into keeper-storage
2022-08-31 10:12:11 +00:00
vdimir
6c15db9b11
Setting max_analyze_depth forbid only second analyze
2022-08-31 11:57:41 +02:00
vdimir
473c9106a5
Revert "Create new query context in StorageRabbitMQ/Nat"
...
This reverts commit 2d6ac7344a25134a625e879da63b9a4aa20d9a69.
2022-08-31 11:57:41 +02:00
vdimir
63f7301ee0
Create new query context in StorageRabbitMQ/Nat
2022-08-31 11:57:41 +02:00
vdimir
60454bd722
Fix error codes in InterpreterSelectQuery.cpp
2022-08-31 11:57:41 +02:00
vdimir
c829a1914d
Disable max_analyze_depth for StorageRabbitMQ/NATS
2022-08-31 11:57:41 +02:00
vdimir
f38f39ed6f
Use max_analyze_depth
2022-08-31 11:57:41 +02:00
vdimir
928adb2b38
[WIP] Limit number of analyze for one query
2022-08-31 11:57:41 +02:00
Kseniia Sumarokova
164ebbe009
Merge branch 'master' into cache-assert-directory-ownership
2022-08-31 11:56:59 +02:00
Kseniia Sumarokova
9abcfce529
Merge pull request #40759 from kssenii/fix-write-through-cache-logical-error
...
Fix logical error from write-though cache from stress test
2022-08-31 11:55:18 +02:00
Antonio Andelic
1af9a81297
Sync after reconnect
2022-08-31 08:29:40 +00:00
Antonio Andelic
1ff14aed02
Fix dropping table
2022-08-31 08:23:44 +00:00
Antonio Andelic
5ef4c52290
Add support for keys limit
2022-08-31 08:14:28 +00:00
Robert Schulze
5ff0ecb867
Merge branch 'master' into remove-unnecessary-context-access-method
2022-08-31 10:14:27 +02:00
Antonio Andelic
97fd3d9123
Merge branch 'master' into fix-install-snapshot
2022-08-31 08:42:00 +02:00
Duc Canh Le
aebd1cb7de
Merge branch 'master' into ch_canh_intersect_distinct
2022-08-31 12:08:20 +08:00
Fangyuan Deng
5c2f35c302
Merge branch 'master' into UniqSketch
2022-08-31 10:23:46 +08:00
Mallik Hassan
87a28c25f5
Update src/Loggers/Loggers.cpp
...
Co-authored-by: Nikolay Degterinsky <43110995+evillique@users.noreply.github.com>
2022-08-30 23:20:19 -03:00
root
f02f7ce99c
addressed minor change requests
2022-08-30 23:20:19 -03:00
root
834ab5cd2b
removed redundant initializer for base class 'OwnPatternFormatter'
2022-08-30 23:20:19 -03:00
root
790a437eb2
Addressed review requests: removed this pointer in functions and config variable in class
2022-08-30 23:20:19 -03:00
root
bb25abd0ae
to resolve build error(no member named 'Util' in namespace 'Poco'), added <Poco/Util/AbstractConfiguration.h> in OwnJSONPatternFormatter.h
2022-08-30 23:20:19 -03:00
root
b8501bea06
style check correction
2022-08-30 23:20:19 -03:00
root
d5db88fa27
Structured-logging-custom-keys PR init
2022-08-30 23:20:19 -03:00
Anton Popov
3504781529
Merge branch 'master' into fix-read-in-order-fixed-prefix
2022-08-30 23:32:43 +02:00
Sema Checherinda
412973daee
Update src/Interpreters/AsynchronousMetrics.cpp
...
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
2022-08-30 21:57:39 +02:00
Sema Checherinda
6b23abe2b6
Update src/Interpreters/AsynchronousMetrics.cpp
...
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
2022-08-30 21:57:33 +02:00
Dmitry Novik
003483b616
Add compatibility setting
2022-08-30 20:49:40 +02:00
kssenii
a5f69517ae
Allow to load marks in threadpool
2022-08-30 20:20:05 +02:00
Sema Checherinda
9bb4f556c3
work on remarks on the review
2022-08-30 20:18:35 +02:00
kssenii
23db13ca69
Merge master
2022-08-30 19:39:41 +02:00
kssenii
19c4442933
Fix
2022-08-30 19:38:57 +02:00
Robert Schulze
7a4978e600
docs: mention where else shortcuts are used
2022-08-30 17:00:25 +00:00
Robert Schulze
413a5759e1
style: eliminate some newlines
...
Co-authored-by: Nikolay Degterinsky <43110995+evillique@users.noreply.github.com>
2022-08-30 17:00:20 +00:00
Robert Schulze
9baf0d745a
feat: make last command available in clickhouse-client interactive mode
2022-08-30 16:59:14 +00:00
Vitaly Baranov
77d741dc25
Add comments.
2022-08-30 18:58:13 +02:00
Vitaly Baranov
86872b2307
Fix locking while writing backup in multiple threads.
2022-08-30 18:10:54 +02:00
Antonio Andelic
d498213973
Style fixes
2022-08-30 13:50:02 +00:00
Antonio Andelic
bf03dfc661
Fix create
2022-08-30 13:41:13 +00:00
Alexander Tokmakov
6a50c20734
Merge branch 'master' into zookeeper_client_fault_injection
2022-08-30 15:43:30 +03:00
Dmitry Novik
0a8378d9cd
Merge branch 'master' into betterorderbyoptimization
2022-08-30 14:23:22 +02:00
Alexander Tokmakov
6fdfb964d0
Revert "Add Annoy index"
2022-08-30 15:10:10 +03:00
vdimir
0f6f3c73b0
Minor fix
2022-08-30 11:57:28 +00:00
Vladimir C
7266c007b7
Merge pull request #39131 from amosbird/cte-global-with-fix1
...
Fix unused unknown columns introduced by WITH statement
2022-08-30 13:52:19 +02:00
Anton Popov
ee691961d9
fix build
2022-08-30 11:43:27 +00:00
Alexander Tokmakov
022f440ad0
Merge pull request #40769 from arthurpassos/caresptrresolver-hosts-patch
...
Fix CaresPTRResolver not reading hosts file
2022-08-30 14:35:10 +03:00
Kseniia Sumarokova
54f6f5bb2c
Merge branch 'master' into fix-write-through-cache-logical-error
2022-08-30 13:30:13 +02:00
Kruglov Pavel
624af5bffd
Update src/Storages/MergeTree/KeyCondition.cpp
2022-08-30 13:18:05 +02:00
Kruglov Pavel
be81d21fdf
Merge pull request #40597 from helifu/master
...
Fix a crash while the grpc port is conflict
2022-08-30 13:17:25 +02:00
Kseniia Sumarokova
5eb188311a
Merge pull request #40586 from kssenii/fix-race
...
Fix possible race in fs cache
2022-08-30 12:58:31 +02:00
Alexander Tokmakov
0ec7f068cc
Merge pull request #40677 from ClickHouse/query-insert-oom
...
Avoid deadlock in case of new query and OOM
2022-08-30 13:54:34 +03:00
Antonio Andelic
53da469147
Merge branch 'master' into keeper-storage
2022-08-30 10:22:04 +00:00
Duc Canh Le
626b3e4147
Merge branch 'master' into ch_canh_intersect_distinct
2022-08-30 18:14:07 +08:00
Duc Canh Le
8590cc46c4
implement intersect + except distinct
2022-08-30 18:09:01 +08:00
Kseniia Sumarokova
487100a4fd
Merge pull request #40764 from kssenii/fix-logical-error-in-cache
...
Remove incorrect check in fs cache
2022-08-30 11:59:56 +02:00
Kseniia Sumarokova
c88db2ef97
Merge pull request #40751 from kssenii/fix-mysql-timeouts
...
Fix issue with mysql db / table function timeouts
2022-08-30 11:59:01 +02:00
Antonio Andelic
80365e0448
Simplify drop and create
2022-08-30 09:19:59 +00:00
Antonio Andelic
21cdb9a500
Don't remove from cache if it wasn't added during rollback
2022-08-30 08:14:53 +00:00
Antonio Andelic
2416ddfae7
Add extra message about startup preprocessing
2022-08-30 08:14:31 +00:00
Robert Schulze
c185353bf0
Minor: remove unnecessary ContextAccess::make() method
2022-08-30 08:06:42 +00:00
Robert Schulze
cc4225109f
Merge pull request #37215 from Vector-Similarity-Search-for-ClickHouse/annoy-2
...
Test failures are unrelated, merging.
2022-08-30 09:25:57 +02:00
Robert Schulze
566daad4a1
Merge pull request #40723 from k-morozov/ref/remove-unnecessary-allocation
...
refactoring: remove unnecessary allocation
2022-08-30 09:22:21 +02:00
Frank Chen
922a0dee74
Fix style
...
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2022-08-30 15:14:12 +08:00
Frank Chen
f17d56b528
Merge branch 'master' into tracing_context_propagation
2022-08-30 14:24:36 +08:00
Frank Chen
83cbdef3c6
Fix context re-initialization for ThreadPool
2022-08-30 12:26:23 +08:00
Cory Levy
cd371110ad
Fix timezone offset formatting
2022-08-29 21:01:27 -04:00
Sema Checherinda
e436b4f4cc
Metric for the number of detached parts
2022-08-30 02:12:57 +02:00
Robert Schulze
4ed1e1a5be
perf: don't copy data around unnecessarily
2022-08-29 20:27:06 +00:00
Robert Schulze
35a37c91f8
chore: incorporate review feedback
2022-08-29 20:27:06 +00:00
robot-clickhouse
64fa077148
style: fix style
2022-08-29 20:27:06 +00:00
Robert Schulze
4d511332c4
chore: delete obsolete modelEvaluate() function
...
- superseded by catboostEvaluate() which no longer uses the internal
repository for external models
- also removed was statement SYSTEM RELOAD MODELS and the monitoring view
SYSTEM.SYSTEMMODELS
2022-08-29 20:27:06 +00:00
Robert Schulze
6b2b3c1eb3
feat: implement catboost in library-bridge
...
This commit moves the catboost model evaluation out of the server
process into the library-bridge binary. This serves two goals: On the
one hand, crashes / memory corruptions of the catboost library no longer
affect the server. On the other hand, we can forbid loading dynamic
libraries in the server (catboost was the last consumer of this
functionality), thus improving security.
SQL syntax:
SELECT
catboostEvaluate('/path/to/model.bin', FEAT_1, ..., FEAT_N) > 0 AS prediction,
ACTION AS target
FROM amazon_train
LIMIT 10
Required configuration:
<catboost_lib_path>/path/to/libcatboostmodel.so</catboost_lib_path>
*** Implementation Details ***
The internal protocol between the server and the library-bridge is
simple:
- HTTP GET on path "/extdict_ping":
A ping, used during the handshake to check if the library-bridge runs.
- HTTP POST on path "extdict_request"
(1) Send a "catboost_GetTreeCount" request from the server to the
bridge, containing a library path (e.g /home/user/libcatboost.so) and
a model path (e.g. /home/user/model.bin). Rirst, this unloads the
catboost library handler associated to the model path (if it was
loaded), then loads the catboost library handler associated to the
model path, then executes GetTreeCount() on the library handler and
finally sends the result back to the server. Step (1) is called once
by the server from FunctionCatBoostEvaluate::getReturnTypeImpl(). The
library path handler is unloaded in the beginning because it contains
state which may no longer be valid if the user runs
catboost("/path/to/model.bin", ...) more than once and if "model.bin"
was updated in between.
(2) Send "catboost_Evaluate" from the server to the bridge, containing
the model path and the features to run the interference on. Step (2)
is called multiple times (once per chunk) by the server from function
FunctionCatBoostEvaluate::executeImpl(). The library handler for the
given model path is expected to be already loaded by Step (1).
Fixes #27870
2022-08-29 20:26:45 +00:00
Dmitry Novik
e25ed9547e
Update src/Interpreters/ProcessList.h
2022-08-29 20:26:37 +02:00
Dmitry Novik
865ee5d0d6
Refactor code
2022-08-29 20:24:35 +02:00
Arthur Passos
961365c7a4
Fix CaresPTRResolver not reading hosts file
2022-08-29 15:11:39 -03:00
Dmitry Novik
1169315580
Add OvercommitTracker blocking
2022-08-29 19:44:05 +02:00
Maksim Kita
88141cae98
Merge pull request #40732 from azat/thread-status-fix-leak
...
Fix memory leak while pushing to MVs w/o query context (from Kafka/...)
2022-08-29 19:36:25 +02:00
Kseniia Sumarokova
c5c48e44ea
Merge branch 'master' into fix-mysql-timeouts
2022-08-29 19:33:29 +02:00
kssenii
db2bc31e17
Remove incorrect assertion
2022-08-29 19:32:47 +02:00
Konstantin Morozov
d185b7a332
refactoring: public ctors
2022-08-29 20:19:20 +03:00
FArthur-cmd
862b53b06f
Merge branch 'annoy-2' of https://github.com/Vector-Similarity-Search-for-ClickHouse/ClickHouse into annoy-2
2022-08-29 16:43:39 +00:00
FArthur-cmd
3305af8db2
fix case when query is already matched
2022-08-29 16:43:24 +00:00
Dmitry Novik
d32492ce8a
Fix GROUPING function SQL compatibility
2022-08-29 18:26:56 +02:00
kssenii
545c6c8be4
Fix
2022-08-29 17:50:27 +02:00
Dmitry Novik
cfe509c3de
Block overcommit tracker in ProcessList near allocations
2022-08-29 17:49:01 +02:00
Filatenkov Artur
d73f661732
Merge branch 'master' into annoy-2
2022-08-29 17:33:13 +03:00
Anton Popov
506bee1e2f
avoid logical errors at some common operation with ColumnObject
2022-08-29 14:31:09 +00:00
kssenii
b1dab84d97
Review fixes
2022-08-29 16:23:14 +02:00
kssenii
0a6c4b9265
Fix
2022-08-29 16:20:53 +02:00
kssenii
877ade9a50
Merge remote-tracking branch 'upstream/master' into fix-race
2022-08-29 16:05:27 +02:00
Antonio Andelic
57471d11db
Fix uncommitted auth
2022-08-29 12:55:30 +00:00
Vladimir C
5cbe7e0846
Merge pull request #40548 from ClickHouse/vdimir/warn-suppress-40330
...
Add config option warning_supress_regexp
2022-08-29 14:02:00 +02:00
Alexander Tokmakov
8bdb589c2b
Merge branch 'master' into zookeeper_client_fault_injection
2022-08-29 13:34:57 +02:00
Antonio Andelic
54ce92a5c4
Merge branch 'master' into fix-install-snapshot
2022-08-29 11:27:34 +00:00
vdimir
24f62e8486
Throw an error in CreatingSetsOnTheFlyTransform in case of input for finished
2022-08-29 11:27:08 +00:00
vdimir
b0e2616aa9
Style fixes in CreateSetAndFilterOnTheFlyTransform and related
2022-08-29 11:26:21 +00:00
Alexander Tokmakov
69387acffa
Revert "Support for DWARF-5 in in house DWARF parser"
2022-08-29 14:25:53 +03:00
Alexander Tokmakov
8d90d30d37
Merge pull request #40589 from ClickHouse/remove_wrong_code_from_mutations
...
Remove wrong code for skipping mutations in MergeTree
2022-08-29 14:18:59 +03:00
avogar
85fa372ad7
Fix possible error Attempt to read after eof in CSV schema inference
2022-08-29 11:18:53 +00:00
Anton Popov
2a3e012931
Merge branch 'master' into fix-read-in-order-fixed-prefix
2022-08-29 13:17:26 +02:00
Alexander Tokmakov
eda0582ec0
Merge pull request #40641 from ClickHouse/fix_startup_of_dropped_replica
...
Do not try to strartup dropped replica
2022-08-29 14:15:15 +03:00
vdimir
3e06cf3a9d
Upd max_rows_in_set_to_optimize_join description
2022-08-29 10:10:47 +00:00
vdimir
7915b6948f
Fix build after rebase
2022-08-29 09:49:16 +00:00
vdimir
afb6b7d9cf
Test plan and pipeline for filtering step for join
2022-08-29 09:49:15 +00:00
vdimir
afeff512b5
Aux port for ReadHeadBalancedProcessor is empty Block
2022-08-29 09:49:14 +00:00
vdimir
95f87dc34e
fix sanitizer assert in CreateSetAndFilterOnTheFlyStep
2022-08-29 09:49:12 +00:00
vdimir
c67ab33d90
small fix CreateSetAndFilterOnTheFlyStep
2022-08-29 09:49:11 +00:00
vdimir
51e02d09f6
set preserves_sorting = true for CreateSetAndFilterOnTheFlyStep
2022-08-29 09:49:10 +00:00
vdimir
714c53ab24
fix typos
2022-08-29 09:49:09 +00:00
vdimir
8e1632f824
Create sets for joins: better code
2022-08-29 09:49:08 +00:00
vdimir
7228091ff1
rename CreateSetAndFilterOnTheFlyTransform
2022-08-29 09:49:07 +00:00
vdimir
67a9acc8db
rename CreatingSetOnTheFlyStep -> CreateSetAndFilterOnTheFlyStep
2022-08-29 09:49:07 +00:00
vdimir
d82a75ae75
cleanup PingPongProcessor
2022-08-29 09:49:06 +00:00
vdimir
e472e13c70
move PingPongProcessor/ReadHeadBalancedProceesor into separate file
2022-08-29 09:49:05 +00:00
vdimir
6ae15db3ad
check join kind for before applying max_rows_in_set_to_optimize_join
2022-08-29 09:49:04 +00:00
vdimir
51a51694d6
Create sets for joins: better code
2022-08-29 09:49:01 +00:00
vdimir
c778bba13f
Create sets for joins: wip
2022-08-29 09:47:00 +00:00
vdimir
31a167848d
Fix set finish condition in CreatingSetsOnTheFlyTransform
2022-08-29 09:46:59 +00:00
vdimir
71708d595f
Create sets for joins: wip
2022-08-29 09:46:59 +00:00
vdimir
8f06430ebd
Create sets for joins: upd
2022-08-29 09:46:58 +00:00
vdimir
3292566603
Format bytes in CreatingSetsOnTheFlyTransform logs
2022-08-29 09:46:57 +00:00
vdimir
031aaf3a45
Add Creating/FilterBySetsOnTheFlyStep for full sorting join
2022-08-29 09:46:57 +00:00
vdimir
c5bc7b0a0c
Resize pipeline after full sort join
2022-08-29 09:46:56 +00:00
Azat Khuzhin
f9812d9917
Fix memory leak while pushing to MVs w/o query context (from Kafka/...)
...
While pushign to MVs, there is a low-level code that create
ThreadGroupStatus/ThreadStatus, it is required to gather some metrics
for system.query_views_log.
But, one should not use ThreadGroupStatus of the MainThreadStatus, since
this structure can hold some state, that may not be cleaned, plus this
may be racy, instead it is better to create new ThreadGroupStatus and
attach it instead.
Also this place misses detachQuery(), and because of this it leaks
ThreadGroupStatus::finished_threads_counters_memory. But it is only the
problem pushing to MVs is done w/o query context (i.e. from Kafka/...),
since when it has query context detachQuery() will be called eventually.
Before this patch series, when I've tried the reproducer with
500 MVs attached to Kafka engine (that @den-crane suggested), jemalloc
report looks like this:
$ ../jeprof --text ~/ch/tmp/upstream/clickhouse-binary --base jeprof.44384.0.i0.heap jeprof.44384.167.i167.heap
Using local file /home/azat/ch/tmp/upstream/clickhouse-binary.
Using local file jeprof.44384.167.i167.heap.
Total: 915.6 MB
910.7 99.5% 99.5% 910.7 99.5% Snapshot (inline)
9.5 1.0% 100.5% 9.5 1.0% std::__1::__libcpp_operator_new (inline)
0.5 0.1% 100.6% 0.5 0.1% DB::TasksStatsCounters::create
And with focus to this place:
$ ../jeprof --focus Snapshot --text ~/ch/tmp/upstream/clickhouse-binary --base jeprof.44384.0.i0.heap jeprof.44384.167.i167.heap
Using local file /home/azat/ch/tmp/upstream/clickhouse-binary.
Using local file jeprof.44384.167.i167.heap.
Total: 915.6 MB
910.7 100.0% 100.0% 910.7 100.0% Snapshot (inline)
0.0 0.0% 100.0% 910.7 100.0% DB::QueryPipeline::reset
0.0 0.0% 100.0% 910.7 100.0% DB::StorageKafka::streamToViews
0.0 0.0% 100.0% 910.7 100.0% DB::StorageKafka::threadFunc
0.0 0.0% 100.0% 910.7 100.0% ProfileEvents::Counters::getPartiallyAtomicSnapshot
0.0 0.0% 100.0% 910.7 100.0% ~ThreadStatus
0.0 0.0% 100.0% 910.7 100.0% ~ViewRuntimeData
0.0 0.0% 100.0% 910.7 100.0% ~ViewRuntimeStats (inline)
Actually this report does not looks great (you understand it because I
stripped it), because --text does not that smart, but if you will use
--pdf for the report you will see the stacktrace (will attach pdf to the
pull request).
But after this patch series the process RSS does not goes beyond
~700MiB.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-08-29 11:36:33 +02:00
Azat Khuzhin
6da5707f8f
Fix possible missing detachQuery() in case of exception in readers
...
This can create leaks, since detachQuery() responsible for cleaning,
i.e. ThreadGroupStatus::finished_threads_counters_memory
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-08-29 11:30:17 +02:00
Azat Khuzhin
b16891da8d
Avoid using of ThreadGroupStatus of the MainThreadStatus
...
One should not use MainThreadStatus, since ThreadGroupStatus can hold
some states, and it is better not to play with this, since this may
create leaks.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-08-29 11:30:17 +02:00
Azat Khuzhin
9fff08eac7
WriteBufferFromS3: remove unused ThreadGroupStatus
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-08-29 11:30:17 +02:00
alesapin
133ca01447
Merge branch 'master' into stress_s3
2022-08-29 11:25:28 +02:00
alesapin
4ed375ca5b
Merge pull request #40720 from ClickHouse/fix_benign_race
...
Fix benign race in database replicated worker
2022-08-29 11:13:07 +02:00
Azat Khuzhin
269453a646
Avoid leaking of ThreadGroupStatus::finished_threads_counters_memory
...
Cleanup them in ThreadStatus::detachQuery(), anyway they cannot be
received after by the client.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-08-29 11:08:50 +02:00
alesapin
7ce0afc0df
Merge pull request #40670 from Avogar/kafka
...
Add setting to disable limit on kafka_num_consumers
2022-08-29 10:53:35 +02:00
Amos Bird
15a69bce84
Use index when row_policy_filter is always false
2022-08-29 16:44:32 +08:00
Antonio Andelic
b4af21b6fb
Small fixes
2022-08-29 07:06:51 +00:00
Amos Bird
abf51bccfd
Disable projection when grouping set is used.
2022-08-29 14:35:48 +08:00
Konstantin Morozov
6636bdec2a
refactoring: fix format
2022-08-29 07:44:40 +03:00
Cory Levy
d0ace9a8fb
Add the %z format descriptor to formatDateTime
2022-08-28 22:28:11 -04:00
Alexey Milovidov
71f6c52c2d
Merge pull request #40727 from amosbird/column-transformer-fix1
...
Correct format of APPLY transformer param
2022-08-29 03:48:04 +03:00
Alexey Milovidov
307e8f2da9
Merge pull request #40729 from ClickHouse/remove-useless-method
...
Remove useless method
2022-08-29 03:46:34 +03:00
Alexey Milovidov
ae97e880d7
Merge pull request #40710 from azat/DWARF-5
...
Support for DWARF-5 in in house DWARF parser
2022-08-29 00:13:33 +03:00
Alexey Milovidov
27782ceef8
Remove useless method
2022-08-28 22:33:42 +02:00
Konstantin Morozov
38a85ade27
refactoring: small up
2022-08-28 22:33:12 +03:00
Alexey Milovidov
f462160bd1
Merge remote-tracking branch 'origin/master' into remove-useless-line
2022-08-28 21:00:29 +02:00
Konstantin Morozov
b4de735a90
refactoring: fix space
2022-08-28 20:55:08 +03:00
Konstantin Morozov
08e9e799d0
refactoring: fix
2022-08-28 20:53:00 +03:00
Konstantin Morozov
75bd61fad5
Merge remote-tracking branch 'origin/ref/remove-unnecessary-allocation' into ref/remove-unnecessary-allocation
...
# Conflicts:
# src/Access/ContextAccess.cpp
# src/Access/ContextAccess.h
2022-08-28 20:50:29 +03:00
Konstantin Morozov
7cd2821aed
refactoring: template make helper
2022-08-28 20:42:55 +03:00
Amos Bird
d1fbe51b81
Correct format of APPLY transformer param
2022-08-29 01:21:12 +08:00
Konstantin Morozov
4050ab819e
refactoring: remove unnecessary allocation
2022-08-28 17:23:17 +03:00
Konstantin Morozov
d4d80dd3c1
refactoring: remove unnecessary allocation
2022-08-28 17:09:13 +03:00
Alexey Milovidov
f518844f06
Merge pull request #40709 from ClickHouse/all_new_table_functions_must_be_documented
...
All new table functions must be documented
2022-08-28 17:05:20 +03:00
Alexey Milovidov
00074a5e14
Merge pull request #40711 from ClickHouse/all_new_functions_must_be_documented
...
All new functions must be documented
2022-08-28 17:05:05 +03:00
Azat Khuzhin
e03870bc8b
Common/Dwarf: fix clang-tidy
2022-08-28 15:51:46 +02:00
alesapin
4f9faab924
Fix benign race in database replicated worker
2022-08-28 15:07:29 +02:00
Kseniia Sumarokova
a0bc5b6ea4
Merge pull request #40705 from azat/stress/max_read_buffer_size
...
Fix LOGICAL_ERROR with max_read_buffer_size=0 during reading marks
2022-08-28 12:41:16 +02:00
Robert Schulze
df934d8762
Merge pull request #40217 from zvonand/zvonand-minmax
...
Fix conversion Date32 / DateTime64 / Date to narrow types
2022-08-28 09:42:39 +02:00
Alexey Milovidov
d3a7945b1b
Merge pull request #40646 from ClickHouse/force-documentation-3
...
Every profile event must have documentation
2022-08-28 04:10:11 +03:00
Alexey Milovidov
b42d26acfe
Remove one line from XML, because we do not care
2022-08-28 02:44:02 +02:00
Alexey Milovidov
5b57c91ad6
Rename table function MeiliSearch to meilisearch
2022-08-28 02:08:51 +02:00