Commit Graph

9361 Commits

Author SHA1 Message Date
Kruglov Pavel
58538d3f90
Merge pull request #40636 from Avogar/fix-url-format-detection
Better format detection for url table function/engine
2022-08-26 12:25:55 +02:00
Vladimir C
e067629e0d
Merge pull request #40239 from vdimir/vdimir/tmp-file-metrics
More metrics for on-disk temporary data
2022-08-26 11:28:01 +02:00
Lloyd-Pottiger
0d33327eda
Merge branch 'master' into feat/support-read-only-for-embeddedrocksdb 2022-08-26 15:36:21 +08:00
Frank Chen
2e874b384d
Update src/Storages/StorageURL.cpp
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
2022-08-26 10:34:46 +08:00
Alexey Milovidov
8afbfebd25 A prototype for embedded documentation 2022-08-26 03:41:52 +02:00
Alexey Milovidov
5b5a16320b Suggestions from @azat 2022-08-26 03:00:56 +02:00
Alexander Tokmakov
b8e3a582f7 do not try to strartup dropped replica 2022-08-25 22:14:39 +02:00
Alexey Milovidov
ebe7d2d25e Fix stack overflow on recursive Buffer tables and system.tables 2022-08-25 21:26:32 +02:00
Azat Khuzhin
09aaa2b5dd Do not obtain storage snapshot for each INSERT block (improves performance)
It depends on how much parts do you have, but for some workload with
InMemory only parts w/o merges, I got 5% increase.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-08-25 21:02:27 +02:00
Alexander Gololobov
6a69e08799
Merge pull request #40559 from ClickHouse/lwd_vertical_merge_fix
Fix vertical merge of parts with lightweight deleted rows
2022-08-25 20:47:44 +02:00
avogar
cef2972af4 Better format detection for url table function/engine 2022-08-25 18:31:06 +00:00
Alexander Tokmakov
eae2667a1c
Merge pull request #40148 from ClickHouse/replicate-tables-readonly
`ReplicatedMergeTree` tables as readonly until attach is finished
2022-08-25 19:39:19 +03:00
Alexander Tokmakov
a6f55b1430 fix 2022-08-25 16:27:12 +02:00
Vladimir C
ddde5096ef
Merge branch 'master' into vdimir/tmp-file-metrics 2022-08-25 15:23:35 +02:00
Frank Chen
92f7ca3616 Move TracingContextOnThread::current() out of class for simplicity
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2022-08-25 20:23:56 +08:00
Alexander Tokmakov
0aea4c7184 fix 2022-08-25 14:22:27 +02:00
Frank Chen
bb00dcc19b Remove using namespace from header
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2022-08-25 20:20:13 +08:00
Frank Chen
c9ea4f9f77 Change compression_method from String to CompressionMethod 2022-08-25 19:18:04 +08:00
alesapin
c4f8811701
Merge pull request #40395 from ClickHouse/data_source_description
Add copy shortcut for backups if dest and source support it
2022-08-25 12:36:30 +02:00
Kseniia Sumarokova
4b62d0f75f
Merge pull request #40511 from maks-buren630501/master
Fix bug (issues 40262)
2022-08-25 12:18:23 +02:00
Antonio Andelic
f90612f577 Remove shutdown assert 2022-08-25 07:07:54 +00:00
Frank Chen
4c1fe0d623 validate parameter
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2022-08-25 14:49:17 +08:00
Frank Chen
99c37ce6c6
Merge branch 'master' into tracing_context_propagation 2022-08-25 10:07:16 +08:00
Kseniia Sumarokova
ba4dfe8308
Update MaterializedPostgreSQLConsumer.cpp 2022-08-25 00:23:59 +02:00
Alexander Tokmakov
3cd26aafe4 remove wrong code for skipping mutations in MergeTree 2022-08-24 20:58:59 +02:00
Antonio Andelic
797e21761a Address PR comments 2022-08-24 18:32:03 +00:00
Nikita Taranov
ac34a17551
Merge branch 'master' into control_block_size_in_aggregator 2022-08-24 20:25:28 +02:00
Antonio Andelic
da0d6056f4 Merge branch 'master' into replicate-tables-readonly 2022-08-24 17:29:33 +00:00
vdimir
7194df1184
Move back TemporaryFile -> TemporaryFileOnDisk 2022-08-24 16:14:11 +00:00
vdimir
1e24a731f7
Move Disks/TemporaryFileOnDisk -> Common/TemporaryFile 2022-08-24 16:14:10 +00:00
vdimir
cd4038d511
Use TemporaryFileOnDisk instead of Poco::TemporaryFile 2022-08-24 16:14:08 +00:00
Lloyd-Pottiger
9d7ace3286
Merge branch 'master' into feat/support-read-only-for-embeddedrocksdb 2022-08-24 23:52:34 +08:00
Lloyd-Pottiger
dc64341876 add tests
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
2022-08-24 23:42:13 +08:00
alesapin
eeb0a3584e Merge branch 'master' into data_source_description 2022-08-24 15:26:18 +02:00
Alexander Gololobov
1c2dd50ca5 Fix vertical merge of parts with lightweight deleted rows 2022-08-24 15:18:33 +02:00
alesapin
cac3005f4d Merge branch 'master' into paranoid_check_for_s3_uploads 2022-08-24 14:43:11 +02:00
Alexander Tokmakov
f9f85a0e8b Revert "Parallel distributed insert select from *Cluster table functions (#39107)"
This reverts commit d3cc234986.
2022-08-24 15:17:15 +03:00
Kseniia Sumarokova
38c4094f89
Update MaterializedPostgreSQLConsumer.h 2022-08-24 12:25:03 +02:00
Kseniia Sumarokova
de9d254290
Update MaterializedPostgreSQLConsumer.cpp 2022-08-24 12:24:44 +02:00
Kseniia Sumarokova
4486f26d51
Update MaterializedPostgreSQLConsumer.cpp 2022-08-24 12:23:08 +02:00
Frank Chen
cd19366b44 Move classes into DB::OpenTelemetry namespace 2022-08-24 16:41:40 +08:00
Frank Chen
efc6a60a60 Clean code 2022-08-24 15:59:44 +08:00
Kruglov Pavel
72f02bd6eb
Merge pull request #40414 from Avogar/improve-schema-inference-cache
Improve schema inference cache, respect format settings that can change the schema
2022-08-23 17:04:58 +02:00
Lloyd-Pottiger
52ec268d75 format check
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
2022-08-23 21:26:15 +08:00
alesapin
07286dd042 Add paranoid option to check blobs to S3 after upload 2022-08-23 15:12:09 +02:00
Lloyd-Pottiger
ac3cf0beb4 add read_only option
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
2022-08-23 20:58:57 +08:00
Maksim Buren
0827e87d45
delete miss log 2022-08-23 14:20:32 +03:00
maks-buren630501
b2b97e53ea Fix bug whith wrong postgresql answer 2022-08-23 14:13:16 +03:00
Kseniia Sumarokova
4711f57763
Update PostgreSQLReplicationHandler.cpp 2022-08-23 10:43:12 +02:00
Maksim Buren
9e54792359
Troubleshooting racing
consumer_task use replication_handler_initialized in thread start in method assertInitialized(), by this sometimes assertInitialized throw error because replication_handler_initialized was still "false".
2022-08-23 09:34:03 +03:00
Antonio Andelic
0765495be1
Merge pull request #39986 from Lloyd-Pottiger/feat/support-ttl-for-embeddedrocksdb
Support TTL for EmbeddedRocksDB
2022-08-23 08:27:47 +02:00
Alexey Milovidov
0252695c6e
Merge pull request #40224 from ClickHouse/alexey-milovidov-patch-4
Check what will be if I enable `concurrency-mt-unsafe` in `clang-tidy`
2022-08-22 23:12:01 +03:00
Lloyd-Pottiger
85a1a36167 remove read_only option
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
2022-08-22 23:31:55 +08:00
alesapin
b4d6afd5b3 Black 2022-08-22 13:51:48 +02:00
Alexey Milovidov
e9d0c50486 Add a test for #24093 2022-08-22 02:32:45 +02:00
Alexey Milovidov
7cd4619d9f
Merge pull request #34551 from ClickHouse/xdbc-constraints
Allow constraints for ODBC and JDBC tables
2022-08-21 21:35:06 +03:00
Robert Schulze
e8b3f56733
Limit suppression to a specific warning 2022-08-21 18:24:17 +00:00
Robert Schulze
3aa5acdb35
Merge pull request #40391 from ClickHouse/reduce-StringRef
Reduce some usage of StringRef
2022-08-21 20:04:57 +02:00
Alexey Milovidov
d157c6a7a0
Update StorageXDBC.cpp 2022-08-21 16:53:48 +03:00
FArthur-cmd
f59735d4f9 fix style 2022-08-21 12:45:35 +00:00
FArthur-cmd
fb7785c740 add support queries with parameters, add setting to avoid memory overflow 2022-08-21 11:07:03 +00:00
alesapin
78ba732b31 Merge branch 'master' into data_source_description 2022-08-21 12:11:21 +02:00
Robert Schulze
bba04db0b4
Try to fix tests 2022-08-21 09:23:37 +00:00
FArthur-cmd
1e8910f048 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into annoy-2 2022-08-21 08:45:03 +00:00
Alexey Milovidov
ab91c99495
Merge branch 'master' into control_block_size_in_aggregator 2022-08-20 21:28:27 +03:00
Alexey Milovidov
01700078d9
Merge branch 'master' into xdbc-constraints 2022-08-20 21:26:07 +03:00
Alexey Milovidov
0db45afb08
Merge branch 'master' into alexey-milovidov-patch-4 2022-08-20 19:58:31 +03:00
alesapin
e4d486fbba
Merge pull request #40346 from ClickHouse/fix_ttl_vertical_merge_bug
Fix bug with multiple TTL merges on expired column
2022-08-20 17:09:32 +02:00
Alexey Milovidov
74e1f4dc61 Fix clang-tidy 2022-08-20 17:09:20 +02:00
Alexey Milovidov
6f39e3e039
Merge pull request #40419 from ClickHouse/fix-parallel-replicas
Fix obvious trash in Parallel replicas
2022-08-20 14:10:14 +03:00
Frank Chen
024fef1aec
Merge branch 'master' into tracing_context_propagation 2022-08-20 14:23:36 +08:00
Nikita Mikhaylov
4a453f0776 Fix obvious trash 2022-08-19 18:08:13 +00:00
avogar
5ab87f1da4 Small refactoring 2022-08-19 16:42:23 +00:00
alesapin
d8664c3227 Add shortcut for backups 2022-08-19 16:58:30 +02:00
Antonio Andelic
d07fd218a2 Fix initialization 2022-08-19 14:56:17 +00:00
Alexey Milovidov
2d1628439b
Merge pull request #40357 from ClickHouse/22.9-prepare
Update version after release
2022-08-19 15:51:30 +03:00
Duc Canh Le
24682556a4
Fix storage merge on view cannot use primary index (#40233)
* fix storage merge on view cannot use index

* minor fix code + tests

* fix view stage

* fix stage

* fix style

* fix tests
2022-08-19 14:24:22 +02:00
Antonio Andelic
eeb716d36b Remove unused error code 2022-08-19 12:03:02 +00:00
Antonio Andelic
ee3df77fda Simplify logic 2022-08-19 11:12:20 +00:00
Robert Schulze
77e64935e1
Reduce some usage of StringRef 2022-08-19 09:56:59 +00:00
Antonio Andelic
5e2c8869a9 Merge branch 'master' into replicate-tables-readonly 2022-08-19 09:41:11 +00:00
Antonio Andelic
3f7d28cc6b Address PR comments 2022-08-19 09:39:59 +00:00
Antonio Andelic
a8f3b62bf6 Use global zookeeper 2022-08-19 08:49:51 +00:00
Antonio Andelic
6f533313e9 Restart everything during attach 2022-08-19 08:17:02 +00:00
Alexander Tokmakov
b9e090699a
Merge branch 'master' into fix_incorrect_replica_idenifier 2022-08-18 21:18:10 +03:00
Alexander Tokmakov
e311d06eaa
Merge pull request #40246 from ClickHouse/parallel_drop_ranges_execution
Allow parallel execution of disjoint drop ranges
2022-08-18 21:17:41 +03:00
Alexey Milovidov
d62d3ad461 Update version to 22.9.1.1 2022-08-18 19:39:03 +02:00
Alexander Tokmakov
629690c32b fix "incorrect replica identifier" 2022-08-18 16:13:27 +02:00
alesapin
a2e08299f8
Update src/Storages/MergeTree/IMergedBlockOutputStream.cpp
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2022-08-18 15:36:55 +02:00
Frank Chen
a3b6ad2a65
Merge branch 'master' into tracing_context_propagation 2022-08-18 20:59:07 +08:00
alesapin
1751ba3e1e
Merge branch 'master' into fix_ttl_vertical_merge_bug 2022-08-18 14:24:05 +02:00
alesapin
20a2e786c9 Fix bug with multiple TTL merges on expired column 2022-08-18 14:21:20 +02:00
Nikolai Kochetov
5a85531ef7
Merge pull request #38286 from Avogar/schema-inference-cache
Add schema inference cache for s3/hdfs/file/url
2022-08-18 13:07:50 +02:00
Alexander Tokmakov
6e5e0689a3
Merge branch 'master' into parallel_drop_ranges_execution 2022-08-18 13:43:29 +03:00
Antonio Andelic
7dce8e5ebd Use just atomic for first try 2022-08-18 06:57:40 +00:00
FArthur-cmd
05331603f5 fix size 2022-08-18 06:16:18 +00:00
Lloyd-Pottiger
26d55f3354 address comments
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
2022-08-17 23:25:42 +08:00
Lloyd-Pottiger
19c4385881
Merge branch 'master' into feat/support-ttl-for-embeddedrocksdb 2022-08-17 22:17:10 +08:00
kssenii
d63d34f7bb Merge master 2022-08-17 15:49:32 +02:00
avogar
8dd54c043d Merge branch 'master' of github.com:ClickHouse/ClickHouse into schema-inference-cache 2022-08-17 11:47:40 +00:00
Igor Nikonov
46ed4f6cdf
Merge pull request #38719 from ClickHouse/skipping_sorting_step
SortingStep: deduce way to sort based on input stream sort description
2022-08-17 12:58:11 +02:00
alesapin
4398a357c8
Merge pull request #40151 from ClickHouse/fix_cannot_quickly_remove_directory
Fix "Cannot quickly remove directory"
2022-08-17 12:11:18 +02:00
Alexander Tokmakov
caa270b72a
Merge pull request #39933 from ClickHouse/auto_convert_ordinary_to_atomic
Add flag that enables automatic conversion from Ordinary to Atomic
2022-08-17 12:40:49 +03:00
Alexander Tokmakov
d23296ef85
Merge pull request #37198 from ClickHouse/replicated_database_improvements
Improvements for Replicated database engine and ON CLUSTER queries
2022-08-17 12:30:04 +03:00
Antonio Andelic
1936ea6377 Fix test issues 2022-08-17 08:28:53 +00:00
Nikita Mikhaylov
a7c3f23dfb
Merge branch 'master' into auto_convert_ordinary_to_atomic 2022-08-17 00:38:25 +02:00
Nikita Taranov
6bdbaccc37 use max_block_size from settings 2022-08-16 18:56:22 +02:00
Kruglov Pavel
cebe9d273c
Fix build 2022-08-16 16:46:17 +02:00
Kruglov Pavel
af2bc42ee1
Fix special build 2022-08-16 16:32:13 +02:00
Kruglov Pavel
b580789802
Fix special build 2022-08-16 16:31:22 +02:00
Alexander Tokmakov
832d7c6e7c Merge branch 'master' into replicated_database_improvements 2022-08-16 15:49:13 +02:00
Antonio Andelic
b0b9f96837 Use setting for retry period, create independent zookeeper sessions 2022-08-16 13:07:51 +00:00
Alexander Tokmakov
387b69caea fix race condition 2022-08-16 14:03:31 +02:00
Alexander Tokmakov
d4fb75c8db Revert "fix race with check thread"
This reverts commit 45284108ba.
2022-08-16 12:49:17 +02:00
Antonio Andelic
b33f3a4e16 Extract common KV storage logic 2022-08-16 09:32:01 +00:00
Antonio Andelic
9ac4ca265a Small initialization fixes 2022-08-16 08:19:02 +00:00
Antonio Andelic
2ed42bca7e Merge branch 'master' into replicate-tables-readonly 2022-08-16 08:17:53 +00:00
kssenii
eb26b219b9 Merge master 2022-08-16 00:56:27 +02:00
Igor Nikonov
7a9bdbe81a Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-08-15 20:59:37 +00:00
Alexander Tokmakov
589c3408d2
Merge pull request #40234 from ClickHouse/better_message_on_restore_covered
Better error message when restoring covered parts
2022-08-15 22:01:48 +03:00
Alexander Tokmakov
b3f3b60ac3 allow parallel execution of disjoint drop ranges 2022-08-15 20:32:03 +02:00
Alexey Milovidov
a0693c3a84
Merge pull request #40225 from ClickHouse/simplify-code
Simplify the code and check what will happen
2022-08-15 19:05:38 +03:00
FArthur-cmd
1d048432a1 update contrib and update cmake 2022-08-15 14:35:51 +00:00
Alexander Tokmakov
c18f3bef91 fix 2022-08-15 16:03:03 +02:00
avogar
c4ff3ffeea Rename settings 2022-08-15 12:45:18 +00:00
Alexander Tokmakov
edaff70010 better error message when restoring covered parts 2022-08-15 13:53:14 +02:00
Nikita Mikhaylov
d3cc234986
Parallel distributed insert select from *Cluster table functions (#39107) 2022-08-15 12:41:17 +02:00
Alexey Milovidov
24371f6cf0 Simplify the code and check what will happen 2022-08-15 06:56:29 +02:00
Alexey Milovidov
3d5ceb8fee Merge branch 'master' into disable-zero-copy-replication 2022-08-14 01:40:20 +02:00
Igor Nikonov
d83bea626c Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-08-13 21:46:34 +00:00
Alexey Milovidov
201cf69854
Merge pull request #39470 from nicelulu/issues_39469
Fix issues 39469
2022-08-13 03:13:28 +03:00
Alexey Milovidov
0e450787f3 Edit the documentation 2022-08-12 21:24:27 +02:00
Alexey Milovidov
a88e713102 Disable zero-copy replication by default 2022-08-12 21:22:11 +02:00
Alexander Tokmakov
46aaf842f8
Merge pull request #40031 from ClickHouse/fix_old_tmp_dirs_cleanup
Fix old temporary directories cleanup
2022-08-12 20:31:20 +03:00
Kseniia Sumarokova
a6cfc7bc3b
Merge pull request #34651 from alexX512/master
New caching strategies
2022-08-12 17:23:37 +02:00
Anton Popov
4bd50bb06c
Merge branch 'master' into distinct_sorted_simplify 2022-08-12 17:11:18 +02:00
Antonio Andelic
c6b3a897d2 Test replicated table without ZK, small fixes 2022-08-12 11:56:46 +00:00
Alexander Tokmakov
e691888267 fix build 2022-08-12 13:03:57 +02:00
Alexander Tokmakov
d24b9874bc fix Cannot quickly remove directory 2022-08-12 12:47:25 +02:00
Nikolai Kochetov
86061d945e
Merge pull request #39826 from amosbird/projection-row-policy-fix
Primary key analysis for row_policy_filter and additional_filter
2022-08-12 11:32:58 +02:00
Antonio Andelic
1c0f85ccdb Attach table in background 2022-08-12 09:32:13 +00:00
kssenii
768a207176 Merge master 2022-08-11 22:25:53 +02:00
Kseniia Sumarokova
be69169f97
Merge pull request #38048 from KinderRiven/decoupling_cache_function_and_algorithm
Decoupling local cache function and cache algorithm
2022-08-11 16:49:56 +02:00
Igor Nikonov
75f6fcfa70 Merge remote-tracking branch 'origin/master' into skipping_sorting_step 2022-08-11 12:35:55 +00:00
Vladimir C
1b44cb5c97
Merge pull request #39974 from ClickHouse/vdimir/setting-num-with-auto
Add setting type to support special 'auto' value
2022-08-11 14:25:10 +02:00
Amos Bird
fa8fab2e8f
Fix KeyCondition with other filters 2022-08-11 19:20:44 +08:00
Jianmei Zhang
d1051d822c
Use getSerializedFileExtension() to get correct file extension for index (#40095) 2022-08-11 12:39:40 +02:00
Maksim Kita
6bec0f5854
Merge pull request #38956 from vdimir/dict-join-refactoring
Join with dictionary refactoring
2022-08-11 11:54:11 +02:00
Vladimir C
2d44e6c458
Merge pull request #39343 from vdimir/refactor-prepared-sets
Refactor PreparedSets/SubqueryForSet
2022-08-11 11:19:18 +02:00
Alexey Milovidov
8374f31306
Merge pull request #39425 from arenadata/ADQM-485
Add support of dates from year 1900 to 2299 for Date32 and DateTime64
2022-08-11 05:01:53 +03:00