thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 19:02:04 +00:00
Code Issues Packages Projects Releases Wiki Activity
71,950 Commits 9,326 Branches 1,783 Tags 2.1 GiB
e9721804b6
Commit Graph

5205 Commits

Author SHA1 Message Date
kssenii
0d5000f917 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into mysql 2021-08-24 07:30:42 +00:00
kssenii
14b87feb27 Fix build 2021-08-24 07:30:33 +00:00
kssenii
b536653851 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into disk-over-web-server 2021-08-24 07:29:50 +00:00
Vladimir C
98acccbdcc
Merge branch 'master' into s3partitionedwrite 2021-08-24 10:03:57 +03:00
kssenii
5307d31924 Throw for alter and silence for drop 2021-08-23 23:48:53 +00:00
kssenii
962f415740 Add test and settings 2021-08-23 22:18:04 +00:00
Dmitry Novik
e78f9b3d4b code cleanup 2021-08-24 00:42:03 +03:00
Vitaly Baranov
63e8bc1f20 Added new commands BACKUP and RESTORE. 2021-08-23 23:41:45 +03:00
Anton Popov
0db8b524f0
Merge pull request #27984 from BraulioVM/optimize/has-all-bloom-filter 
Make `hasAll` filter condition leverage bloom filters
 2021-08-23 22:19:50 +03:00
Alexander Tokmakov
d95131dd4a better check for alter partition version 2021-08-23 20:51:20 +03:00
Dmitry Novik
6dad70a1fb Add check whether enums have the same underlying type or not 2021-08-23 17:55:21 +03:00
Alexander Tokmakov
4a4a0b4823 fix 2021-08-23 16:32:07 +03:00
Dmitry Novik
ac2e9a27e6 Fixes #26672. Lower restrictions in StorageInMemoryMetadata::check. 2021-08-23 16:08:23 +03:00
Alexander Tokmakov
cc9c2fd63b make code better 2021-08-23 15:57:50 +03:00
Ilya Yatsishin
315a835c95
Merge pull request #28020 from qoega/fix-style 2021-08-23 15:48:41 +03:00
Yatsishin Ilya
17bb938541 fix style 2021-08-23 13:59:01 +03:00
mergify[bot]
95e9d517ff
Merge branch 'master' into fix-clang-tidy 2021-08-23 10:54:36 +00:00
Maksim Kita
2a6aa50d49 Merge branch 'master' into create-user-defined-lambda-function 2021-08-23 11:08:49 +03:00
alesapin
31b12b5240
Merge pull request #27787 from kafka1991/master 
modify column datatype to Nullable(datatype) or Nullable(datatype) to Nullable(datatype) donot allways need mutations.
 2021-08-23 10:35:39 +03:00
Vitaly Baranov
89166a655a Init SettingFieldBool with true or false, not with 1 or 0. 2021-08-23 09:47:36 +03:00
vdimir
6fe63a80bc Fix validateKey/Bucket for S3 2021-08-23 09:22:15 +03:00
vdimir
6ba6577978 Do not allow slashes in bucket formatted from PARTITION BY 2021-08-23 09:22:15 +03:00
vdimir
c4c42b61c4 Validate uft8 in partition key from PARTITION BY for s3 2021-08-23 09:22:15 +03:00
Vladimir Chebotarev
f6cb82aba5 Minor fix. 2021-08-23 09:22:15 +03:00
Vladimir Chebotarev
e203c330c2 Validation. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
6e10d28c78 Fix. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
ea67849fd9 Typo. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
119172543b Better fix. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
0b75c66378 Revert "Update src/Storages/StorageS3.cpp " 
This reverts commit bbbfe1891bfdc868878cfc8d52c254babd4ca181.
 2021-08-23 09:22:14 +03:00
Vladimir C
067cc763cc Update src/Storages/StorageS3.cpp 
StringRef in fmt::format throws `fmt::v7::format_error: cannot switch from manual to automatic argument indexing`
 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
d8bc601dbf Minor. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
4b1cc38056 Fixes. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
cebe1e7123 Style fix. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
b34f60ad46 Fixed find. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
be343766a6 Update src/Storages/StorageS3.cpp 
Co-authored-by: Vladimir C <vdimir@yandex-team.ru>
 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
24fcf152ed Minor fix. 2021-08-23 09:22:13 +03:00
Vladimir Chebotarev
5ce9e266b4 Less strict validation. 2021-08-23 09:22:13 +03:00
vdimir
4c9fb7ff58 Fix error code in StorageS3.cpp 2021-08-23 09:22:13 +03:00
vdimir
eb31b194b9 Add validation to insert partition by key to s3 2021-08-23 09:22:13 +03:00
Vladimir Chebotarev
d1efe3afa7 Review fixes. 2021-08-23 09:22:13 +03:00
Vladimir Chebotarev
e99433e094 Better functionality. 2021-08-23 09:22:13 +03:00
Vladimir Chebotarev
328213f5d9 Fixes. 2021-08-23 09:22:13 +03:00
Vladimir Chebotarev
1146efacff Partitioned writes for S3. 2021-08-23 09:22:13 +03:00
Vitaly Baranov
ff1fad8521 Use setSetting() with implicit conversion true/false -> Field. 2021-08-23 08:57:00 +03:00
kssenii
c5ee335c68 Fix tests 2021-08-22 21:41:09 +03:00
kssenii
14afb3c509 File progress for local fix 2021-08-22 20:33:55 +03:00
Nikita Mikhaylov
80ebc6ccea Done 2021-08-22 15:09:49 +00:00
Braulio Valdivielso
d48f6122e3 clean up implementation a bit 2021-08-22 10:06:30 +01:00
Braulio Valdivielso
e95563fb82 leverage bloom filter index for 'hasAll' 
Fixes #26151. For now, this is a draft implementation. There is some
cleanup that can be done, but the new test is passing
 2021-08-21 20:14:56 +01:00
Nikita Mikhaylov
cfa571cac4
Merge pull request #27895 from nikitamikhaylov/temporary_remove_yaml_fuzzer 
Build fuzzers with clang-tidy
 2021-08-21 16:51:31 +03:00
alexey-milovidov
5ac6a99542
Merge pull request #27927 from ClickHouse/fix_27832 
Сheck cluster name before creating Distributed
 2021-08-21 10:40:23 +03:00
Nikita Mikhaylov
f72457fa98 Lower the size of the binaries 2021-08-21 02:20:23 +00:00
kssenii
945e2c4ce5 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into disk-over-web-server 2021-08-20 18:01:23 +00:00
Ilya Yatsishin
624cb43f7f
Merge pull request #27885 from azat/kafka-write-assert 
Fix incorrect assertion during writing to StorageKafka
 2021-08-20 18:53:22 +03:00
Alexander Tokmakov
42378b5913 fix 2021-08-20 17:05:53 +03:00
Alexander Tokmakov
59eb3aa9a9 avoid too long waiting for inactive replicas 2021-08-20 15:59:57 +03:00
Alexander Tokmakov
8c6dd18917 check cluster name before creating Distributed 2021-08-20 14:55:04 +03:00
mergify[bot]
66c6a85211
Merge branch 'master' into master 2021-08-20 11:50:30 +00:00
Maksim Kita
5f2b28639f Merge branch 'master' into create-user-defined-lambda-function 2021-08-20 11:53:08 +03:00
Azat Khuzhin
b60e5ac801 Fix incorrect assertion during writing to StorageRabbitMQ 2021-08-20 07:14:20 +03:00
Azat Khuzhin
5947e54c1b Fix incorrect assertion during writing to StorageKafka 
The problem it does not triggered in CI always because buffers was not
destroyed by that time.

Fixes: #26547
 2021-08-19 22:07:20 +03:00
nvartolomei
c09c90125f
Merge branch 'master' into nv/last-queue-update-exception 2021-08-19 10:29:16 +01:00
Nikolai Kochetov
7dcff5c90e
Merge pull request #27742 from amosbird/projection-improvement3 
Improve projection analysis.
 2021-08-19 12:11:20 +03:00
alexey-milovidov
215385e4d0
Merge pull request #27794 from filimonov/kafka_null_messages 
Avoid nullptr dereference during processing of NULL messages in Kafka for some formats
 2021-08-19 11:25:07 +03:00
Nikolai Kochetov
3e44ef437e
Merge pull request #26639 from ClickHouse/fix-bad-cast 
Fix bad cast
 2021-08-19 11:17:26 +03:00
alesapin
0aec151719
Merge pull request #27808 from ClickHouse/fix_replicas_may_diverge 
Fix a couple of bugs that may cause replicas to diverge
 2021-08-19 09:58:07 +03:00
Maksim Kita
01682a86b3 Updated user defined functions implementation 2021-08-19 00:54:55 +03:00
Alexander Tokmakov
0ed046eb7b remove irrelevant comments 2021-08-18 15:33:11 +03:00
alexey-milovidov
bcfab277e7
Update ReadBufferFromKafkaConsumer.cpp 2021-08-18 15:25:54 +03:00
Nikolai Kochetov
a46fe11e2c Try fix test_mutations_with_merge_background_task 2021-08-18 13:30:02 +03:00
Alexander Tokmakov
09ff66da0e fix a couple of bugs that may cause replicas to diverge 2021-08-18 12:50:46 +03:00
Mikhail Filimonov
05d77d2873
Safer processing of NULL messages in Kafka for some formats 2021-08-18 11:26:57 +02:00
alesapin
5cc98c67a4 Fix 2021-08-18 11:56:18 +03:00
mergify[bot]
f11e396151
Merge branch 'master' into nv/last-queue-update-exception 2021-08-18 07:00:50 +00:00
vicgao
c16bd25399 modify column datatype to Nullable(datatype) or Nullable(datatype) to Nullable(datatype) donot allways need mutations. 2021-08-17 21:14:13 +08:00
Nikita Mikhaylov
cc8ad0a4e7
Merge pull request #27733 from Algunenano/i27620_take2 
Refactor arrayJoin check on partition expressions
 2021-08-17 15:57:16 +03:00
vicgao
b5610d9a83 modify column datatype to Nullable(datatype) or Nullable(datatype) to Nullable(datatype) donot allways need mutations. 2021-08-17 20:52:16 +08:00
Nikolai Kochetov
98eb619b4b
Merge pull request #23367 from Avogar/short-circuit 
Implement short circuit function evaluation
 2021-08-17 12:36:07 +03:00
Amos Bird
ea7dc495cf
Better code. 2021-08-17 12:32:49 +08:00
Raúl Marín
90f224807a Style 2021-08-16 17:31:37 +02:00
kssenii
bec34531bc Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into disk-over-web-server 2021-08-16 14:48:32 +00:00
Amos Bird
90881aab09
Better code style 2021-08-16 22:17:07 +08:00
Raúl Marín
3ec904a953 Only analyze if tuple has arguments 2021-08-16 14:47:41 +02:00
Amos Bird
b162a2b699
Improve projection analysis. 
Remove duplicate index analysis and avoid possible invalid limit checks
during projection analysis.
 2021-08-16 20:11:57 +08:00
Raúl Marín
34099bc196 Refactor arrayJoin check on partition expressions 2021-08-16 12:46:18 +02:00
Alexey Milovidov
d184b79bba Progress on async reads. 2021-08-16 03:00:32 +03:00
alexey-milovidov
4a36366925
Merge pull request #27648 from Algunenano/i27620 
Disable arrayJoin on partition expressions
 2021-08-16 02:53:36 +03:00
Alexey Milovidov
0cd3f25ba4 Merge remote-tracking branch 'origin/master' into async-reads 2021-08-16 02:41:01 +03:00
alexey-milovidov
76b0502482
Merge pull request #27690 from kitaisreal/remove-dense-hash-map-set 
Removed DenseHashMap, DenseHashSet
 2021-08-16 02:39:41 +03:00
Maksim Kita
5f5470c2cd Removed DenseHashMap, DenseHashSet 2021-08-15 13:32:56 +03:00
Alexey Milovidov
020beb885d Merge branch 'master' into async-reads 2021-08-15 12:08:19 +03:00
mergify[bot]
bf426cfb96
Merge branch 'master' into fix_intersecting_parts 2021-08-15 09:07:20 +00:00
alexey-milovidov
d88cf81d71
Merge branch 'master' into fix-bad-cast 2021-08-15 09:06:42 +03:00
mergify[bot]
353a324887
Merge branch 'master' into i27620 2021-08-15 02:32:26 +00:00
alexey-milovidov
ab4415a22b
Update MergeTreeData.cpp 2021-08-15 05:28:38 +03:00
kssenii
daac6323fa Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into disk-over-web-server 2021-08-14 13:15:04 +00:00
kssenii
200d75646a Fix 2021-08-13 20:58:03 +00:00
Alexander Tokmakov
c534363abe fix intersecting parts 2021-08-13 20:33:10 +03:00
Raúl Marín
73cb7d55ec Disable arrayJoin on partition expressions 2021-08-13 16:18:46 +02:00
mergify[bot]
319e5af6eb
Merge branch 'master' into short-circuit 2021-08-13 13:01:46 +00:00
Nikolai Kochetov
66fdb859f3
Merge pull request #27528 from amosbird/projection-fix12 
Fix projection materialization with missing columns
 2021-08-13 15:20:51 +03:00
Kseniia Sumarokova
0917488cec
Merge pull request #27382 from kssenii/cast-internal 
add Cast internal function
 2021-08-13 13:13:51 +03:00
Nikolai Kochetov
ad00aaa18c
Merge pull request #27575 from kitaisreal/removed-some-data-streams 
Removed some data streams
 2021-08-13 12:59:00 +03:00
Nikolai Kochetov
d207fbc699
Merge pull request #27152 from qoega/fix-access-gtest-in-arcadia 
Fix arcadia src/Access gtest
 2021-08-13 10:51:29 +03:00
alexey-milovidov
f20eae9a45
Merge pull request #27577 from ClickHouse/remove-streams-from-lv 
Remove streams from lv
 2021-08-13 03:01:33 +03:00
Nikolai Kochetov
224e0a547a Fix build. 2021-08-12 16:02:17 +03:00
Nikolai Kochetov
9c066d476e Fix tests. 2021-08-12 13:46:58 +03:00
Nikolai Kochetov
5e5a6ff654 Merge branch 'master' into qoega-fix-access-gtest-in-arcadia 2021-08-12 12:02:15 +03:00
mergify[bot]
80eaea1c51
Merge branch 'master' into short-circuit 2021-08-12 08:38:45 +00:00
Maksim Kita
124a87684f Removed some data streams 2021-08-11 23:39:01 +03:00
kssenii
dcc6f597dc Move templates from FunctionsConvertion 2021-08-11 19:09:51 +00:00
kssenii
7555fc19fd Revert "Move some templates" 
This reverts commit 7f247becca.
 2021-08-11 18:49:25 +00:00
Nikolai Kochetov
beed4c21a5 Use processors in Storage::watch 2021-08-11 20:28:54 +03:00
mergify[bot]
68a52fc065
Merge branch 'master' into projection-fix12 2021-08-11 17:08:04 +00:00
Alexander Kuzmenkov
8f9845722a
Merge pull request #27250 from azat/minmax-nullable-new-format 
Add new index data skipping minmax index format for proper Nullable support
 2021-08-11 13:56:16 +03:00
Nikolai Kochetov
81b85c30f7 Remove streams from LV part 1. 2021-08-11 11:35:10 +03:00
kssenii
7d8b694a9f Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into cast-internal 2021-08-11 07:37:21 +00:00
kssenii
7f247becca Move some templates 2021-08-11 07:32:46 +00:00
Alexey Milovidov
08f8511854 Fix Style 2021-08-11 06:52:28 +03:00
alexey-milovidov
54d8282209
Merge pull request #27248 from azat/mt-mutation-stuck-fix 
Fix mutation stuck on invalid partitions in non-replicated MergeTree
 2021-08-11 06:10:15 +03:00
alexey-milovidov
a0a5c0da32
Update BackgroundJobsExecutor.cpp 2021-08-11 06:08:30 +03:00
Alexey Milovidov
c590a239d5 Merge branch 'master' into fix-bad-cast 2021-08-11 05:48:37 +03:00
Amos Bird
e63c26edb7
Fix projection materialization with missing columns 2021-08-10 21:47:27 +08:00
Nikolai Kochetov
8613cfd4e2
Merge branch 'master' into fix-access-gtest-in-arcadia 2021-08-10 16:13:11 +03:00
Maksim Kita
87fb75b272
Merge pull request #27191 from FArthur-cmd/memory-tracker-26043 
Client with flag for memory tracker
 2021-08-10 14:01:02 +03:00
kssenii
eb4746ac69 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into cast-internal 2021-08-10 09:38:41 +00:00
Nikolai Kochetov
a1ec7f75c5 Merge branch 'master' into qoega-fix-access-gtest-in-arcadia 2021-08-10 11:31:47 +03:00
Nikolai Kochetov
bb4c11cd27
Merge pull request #27298 from ClickHouse/fix-23515 
Fix bug from #23515.
 2021-08-09 23:25:30 +03:00
Nikolai Kochetov
8cc493a3cd Try fix build. 2021-08-09 18:09:29 +03:00
Pavel Kruglov
70b51133c1 Try to simplify code 2021-08-09 18:01:08 +03:00
Pavel Kruglov
0662df8b76 Fix performance with JIT, add arguments to function isSuitableForShortCircuitArgumentsExecution 2021-08-09 17:54:14 +03:00
Pavel Kruglov
e792fa588f Mark all Functions as sutable or not for executing as short circuit arguments 2021-08-09 17:50:09 +03:00
Maksim Kita
b8b24684b4
Merge pull request #27273 from ClickHouse/remove-streams-from-dicts 
Remove streams from dicts
 2021-08-09 15:58:37 +03:00
Nicolae Vartolomei
3f291b024a
Use plain mutex instead of MultiVersion 2021-08-09 13:58:23 +01:00
kssenii
d767f6704f Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into disk-over-web-server 2021-08-09 12:14:23 +00:00
Nikita Mikhaylov
73d3f2c60f
Merge pull request #26313 from fastio/control_execution_period_of_clearOldTemporaryDirectories 
Control the execution period of clear old temporary directories by parameter
 2021-08-09 14:29:24 +03:00
Nikolai Kochetov
4a00e402ae Fix spelling. 2021-08-09 12:09:09 +03:00
Nikolai Kochetov
372293b85d
Merge branch 'master' into fix-27179 2021-08-09 11:59:22 +03:00
Alexey Milovidov
8b9da6fe26 Merge branch 'master' into async-reads 2021-08-09 03:33:32 +03:00
mergify[bot]
316c254536
Merge branch 'master' into fix-bad-cast 2021-08-08 23:45:19 +00:00
kssenii
7991bb6e83 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into cast-internal 2021-08-08 21:30:17 +00:00
Azat Khuzhin
038241b6ed Add new index data skipping minmax index format for proper Nullable support 
Note, that it cannot be done w/o new extension, since index does not
have any header.

v2: use IDisk interface for existence check
v3: remove extra file existence check
v4: fix MATERIALIZE INDEX
 2021-08-08 19:30:12 +03:00
Azat Khuzhin
91d7f3daa7 Fix lock-order-inversion while notifying about finished mutations 
TSAN reports [1]:

  WARNING: ThreadSanitizer: lock-order-inversion (potential deadlock) (pid=36)
    Cycle in lock order graph: M16388 (0x7b7400011d68) => M1030334152907497744 (0x000000000000) => M16388

    Mutex M1030334152907497744 acquired here while holding mutex M16388 in thread T4:
      0 pthread_mutex_lock <null> (clickhouse+0x967d536)
      1 std::__1::__libcpp_mutex_lock(pthread_mutex_t*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__threading_support:405:10 (clickhouse+0x1b25c7d9)
      2 std::__1::mutex::lock() obj-x86_64-linux-gnu/../contrib/libcxx/src/mutex.cpp:33:14 (clickhouse+0x1b25c7d9)
      3 std::__1::lock_guard<std::__1::mutex>::lock_guard(std::__1::mutex&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__mutex_base:91:27 (clickhouse+0x15220cd9)
      4 DB::StorageMergeTree::getIncompleteMutationsStatus(long, std::__1::set<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std:>
      5 DB::StorageMergeTree::waitForMutation(long, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_0::operator()() const obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:464:36 (clickhouse+0x1521e2b1)
      6 void std::__1::condition_variable::wait<DB::StorageMergeTree::waitForMutation(long, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)::$_0>(std::__1::unique_lock<std::__1::mutex>&, DB::StorageMergeTree::waitForMutation(lon>
      7 DB::StorageMergeTree::waitForMutation(long, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:469:29 (clickhouse+0x1521e2b1)
      8 DB::StorageMergeTree::mutate(DB::MutationCommands const&, std::__1::shared_ptr<DB::Context const>) obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:496:9 (clickhouse+0x15221738)
      9 DB::InterpreterAlterQuery::execute() obj-x86_64-linux-gnu/../src/Interpreters/InterpreterAlterQuery.cpp:113:16 (clickhouse+0x141182f6)
      10 DB::executeQueryImpl(char const*, char const*, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, bool, DB::ReadBuffer*) obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:560:32 (clickhouse+0x149152f6)
      11 DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, bool) obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:909:30 (clickhous>
      12 DB::TCPHandler::runImpl() obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:313:24 (clickhouse+0x153270af)

      Hint: use TSAN_OPTIONS=second_deadlock_stack=1 to get more informative warning message

    Mutex M16388 acquired here while holding mutex M1030334152907497744 in thread T59:
      0 pthread_mutex_lock <null> (clickhouse+0x967d536)
      1 std::__1::__libcpp_mutex_lock(pthread_mutex_t*) obj-x86_64-linux-gnu/../contrib/libcxx/include/__threading_support:405:10 (clickhouse+0x1b25c7d9)
      2 std::__1::mutex::lock() obj-x86_64-linux-gnu/../contrib/libcxx/src/mutex.cpp:33:14 (clickhouse+0x1b25c7d9)
      3 std::__1::unique_lock<std::__1::mutex>::unique_lock(std::__1::mutex&) obj-x86_64-linux-gnu/../contrib/libcxx/include/__mutex_base:119:61 (clickhouse+0x15226c31)
      4 DB::StorageMergeTree::selectPartsToMutate(std::__1::shared_ptr<DB::StorageInMemoryMetadata const> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, std::__1::shared_ptr<DB::RWLockImpl::LockHolderImpl>&) obj-x86_64-linux->
      5 DB::StorageMergeTree::scheduleDataProcessingJob(DB::IBackgroundJobExecutor&) obj-x86_64-linux-gnu/../src/Storages/StorageMergeTree.cpp:1060:28 (clickhouse+0x15228b10)
      6 DB::BackgroundJobsExecutor::scheduleJob() obj-x86_64-linux-gnu/../src/Storages/MergeTree/BackgroundJobsExecutor.cpp:229:17 (clickhouse+0x14f441dc)
      7 DB::IBackgroundJobExecutor::backgroundTaskFunction() obj-x86_64-linux-gnu/../src/Storages/MergeTree/BackgroundJobsExecutor.cpp:185:10 (clickhouse+0x14f438ed)

  [1]: https://clickhouse-test-reports.s3.yandex.net/27248/4f3b80ff33c846465983aa2bc9ae9490e1118b15/fuzzer_tsan/report.htmlfail1
 2021-08-08 19:28:29 +03:00
Azat Khuzhin
7964355ecf Fix mutation stuck on invalid partitions in non-replicated MergeTree 
v2: Do not try to process empty mutations
    Found with flaky check [1].

      [1]: https://clickhouse-test-reports.s3.yandex.net/27248/66e8c0833392c20ba8dba3780f2b0d5c18f8194e/functional_stateless_tests_flaky_check_(address).html#fail1
 2021-08-08 19:28:15 +03:00
Azat Khuzhin
5139067631 Guard BackgroundJobsExecutor from thread termination in case of uncaught exception 2021-08-08 19:27:58 +03:00
Azat Khuzhin
702d9955c0 Fix distributed queries with zero shards and aggregation 2021-08-08 19:22:49 +03:00
mergify[bot]
ec331132c1
Merge branch 'master' into fix-bad-cast 2021-08-08 15:20:27 +00:00
Azat Khuzhin
3be3c503aa Fix some comments 2021-08-08 09:58:07 +03:00
alexey-milovidov
a7e26506cf
Merge pull request #26951 from ianton-ru/zero_copy_by_default 
Set allow_remote_fs_zero_copy_replication to true by default
 2021-08-08 04:46:49 +03:00
alexey-milovidov
ba3fbbaab0
Update ReplicatedMergeTreeMergeStrategyPicker.cpp 2021-08-08 04:43:59 +03:00
alexey-milovidov
e6e9a27d8f
Merge pull request #27377 from evillique/bz2 
Add support for bzip2 compression method
 2021-08-08 04:40:13 +03:00
Alexey Milovidov
1ad1e62b47 Fix unit test 2021-08-08 04:02:48 +03:00
Alexey Milovidov
24cef99065 Merge branch 'master' into fix-bad-cast 2021-08-08 04:00:29 +03:00
alexey-milovidov
c5207fc237
Merge pull request #26466 from azat/optimize-dist-select 
Rework SELECT from Distributed optimizations
 2021-08-08 03:59:32 +03:00
mergify[bot]
9869c8bb10
Merge branch 'master' into zero_copy_by_default 2021-08-07 16:53:44 +00:00
alexey-milovidov
24cc073a22
Merge pull request #27328 from ClickHouse/fix_partition_id_validation 
Fix partition id validation
 2021-08-07 18:45:19 +03:00
kssenii
9ca422f0c5 Introduce CAST for internal usage 2021-08-07 09:03:10 +00:00
Nikolay Degterinsky
9a45458faf Add bzip2 file compression 2021-08-07 00:25:52 +00:00
Alexander Tokmakov
5e2bfd5ba1 fix partition id validation 2021-08-06 23:56:51 +03:00
kssenii
073d7fdd5e Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into disk-over-web-server 2021-08-06 19:42:29 +00:00
Nikolai Kochetov
3cbeab8959 Fix incorrect row-level filtering 2021-08-06 21:16:06 +03:00
Nikolai Kochetov
157ecef3c2
Merge pull request #27227 from ClickHouse/do-not-update-settings-for-view 
Revert #24095. User-level settings will affect queries from view.
 2021-08-06 20:57:36 +03:00
mergify[bot]
c696817a79
Merge branch 'master' into control_execution_period_of_clearOldTemporaryDirectories 2021-08-06 16:16:22 +00:00
Nikolai Kochetov
7320411a7f Fix bug from #23515. 2021-08-06 16:39:11 +03:00
vdimir
ffab19caad
Fix hasJoin in StorageView.cpp 2021-08-06 13:48:47 +03:00
vdimir
76ca40857a
Better error message for changed json_use_nulls in StorageView, check ast 2021-08-06 12:16:26 +03:00
tavplubix
14ef66e49b
Merge pull request #25895 from ClickHouse/improve_create_or_replace 
Improve CREATE OR REPLACE query
 2021-08-06 11:58:46 +03:00
Nikolai Kochetov
39ff5ab478 Merge branch 'master' into remove-streams-from-dicts 2021-08-06 11:46:27 +03:00
Nikolai Kochetov
13f95f3fdf Streams -> Processors for dicts, part 3. 2021-08-06 11:41:45 +03:00
Alexey Milovidov
cea9690327 Merge branch 'master' into async-reads 2021-08-05 23:58:57 +03:00
alexey-milovidov
1a3d8cebd2
Merge pull request #27197 from azat/nullable-index-fix 
Fix on-disk format breakage for secondary indices over Nullable column
 2021-08-05 21:52:18 +03:00
Artur
d2869d2e2b add test for memory tracker in client 2021-08-05 16:20:24 +00:00
vdimir
a427b19d3a
Check join_use_nulls changed in StorageView::read 2021-08-05 18:38:52 +03:00
Kseniia Sumarokova
5edd9e0513
Merge pull request #27198 from kssenii/postgres-db-schema 
Support schema for postgres database engine
 2021-08-05 18:30:47 +03:00
Kruglov Pavel
59a94bd322
Merge pull request #27183 from ClickHouse/fix_detached_parts_name_partsing 
Fix part name parsing in system.detached_parts
 2021-08-05 18:21:03 +03:00
Nikolai Kochetov
fc12310ee8 Revert #24095. User-level settings will affect queries from view. 2021-08-05 17:17:22 +03:00
Alexander Kuzmenkov
d852207b0e
Merge pull request #25721 from CurtizJ/read-in-order-perf-2 
Reduce memory usage in queries with `ORDER BY primary_key
 2021-08-05 16:13:48 +03:00
Maksim Kita
7fdf3cc263
Merge pull request #27180 from kitaisreal/storage-system-replicas-added-column-replica-is-active 
Storage system replicas added column replica is active
 2021-08-05 12:46:53 +03:00
terrylin
dd539f0a0d improvement of materilize ttl 2021-08-05 15:33:02 +08:00
terrylin
bd3d9a4518 materialize ttl recalculate only (optional) 2021-08-05 14:17:48 +08:00
Azat Khuzhin
dee27fcbb9 Fix on-disk format breakage for secondary indices over Nullable column 
[1] breaks on disk format (and the relevant change in the:

  [1]: https://github.com/ClickHouse/ClickHouse/pull/12455#discussion_r682830812

Too bad that I checked this patchset only for compatibility after
reverting this patch [2] (use case: I've applied it manually, then
revert it, and data skipping indexes over Nullable column had been
broken)

  [2]: https://github.com/ClickHouse/ClickHouse/pull/12455#issuecomment-823423772

But this patchset actually breaks compatibility with older versions of
clickhouse for Nullable data skipping indexes after simple upgrade:

Here is a simple reproducer:

    --
    -- run this with 21.6 or similar (i.e. w/o this patch)
    --

    CREATE TABLE data
    (
        `key` Int,
        `value` Nullable(Int),
        INDEX value_index value TYPE minmax GRANULARITY 1
    )
    ENGINE = MergeTree
    ORDER BY key;

    INSERT INTO data SELECT
        number,
        number
    FROM numbers(10000);

    SELECT * FROM data WHERE value = 20000 SETTINGS force_data_skipping_indices = 'value_index' SETTINGS force_data_skipping_indices = 'value_index', max_rows_to_read=1;

Now upgrade and run the query again:

    SELECT * FROM data WHERE value = 20000 SETTINGS force_data_skipping_indices = 'value_index' SETTINGS force_data_skipping_indices = 'value_index', max_rows_to_read=1;

And it will fail because of on disk format changes:

    $ ll --time-style=+ data/*/data/all_1_1_0/skp*.idx
    -rw-r----- 1 azat azat 36  data/with_nullable_patch/data/all_1_1_0/skp_idx_value_index.idx
    -rw-r----- 1 azat azat 37  data/without_nullable_patch/data/all_1_1_0/skp_idx_value_index.idx

    $ md5sum data/*/data/all_1_1_0/skp*.idx
    a19c95c4a14506c65665a1e30ab404bf  data/with_nullable_patch/data/all_1_1_0/skp_idx_value_index.idx
    e50e2fcfa873b232196623d56ab26105  data/without_nullable_patch/data/all_1_1_0/skp_idx_value_index.idx

Note, that there is no stable release with this patch included yet, so
no need to backport.

Also note that you may create data skipping indexes over Nullable
column even before [3].

  [3]: https://github.com/ClickHouse/ClickHouse/pull/12455

v2: break cases when granulas has Null in values due to backward
compatibility
 2021-08-05 00:19:38 +03:00
kssenii
f06703a7c9 Support schema for postgres database engine 2021-08-04 20:52:45 +00:00
Azat Khuzhin
7a47327980 Fix used_storages by using log_queries from local context 
I don't have log_queries enabled by default.
 2021-08-04 21:44:19 +03:00
Anton Popov
9586bb7e90 fix pvs 2021-08-04 20:42:22 +03:00
terrylin
6f42ec6b9b fix compile error 2021-08-05 00:16:21 +08:00
Anton Popov
915ef50c2d optimize reading in order of key more 2021-08-04 18:18:03 +03:00
Alexander Tokmakov
23f8b3d07d fix part name parsing in system.detached_parts 2021-08-04 17:42:48 +03:00
terrylin
88b9e7f24f materialize ttl recalculate only optional 2021-08-04 22:16:13 +08:00
Anton Ivashkin
61016da2b1 Set allow_remote_fs_zero_copy_replication to true by default 2021-08-04 17:08:09 +03:00
Maksim Kita
3f48c85722 StorageSystemReplicas added column replica_is_active 2021-08-04 16:19:42 +03:00
Maksim Kita
4dc4854fad
Merge pull request #27168 from excitoon-favorites/safers3readbuffer 
Safer `ReadBufferFromS3` for merges and backports
 2021-08-04 15:01:21 +03:00
Yatsishin Ilya
55e06ea16b fix 2021-08-04 13:06:01 +03:00
Vladimir Chebotarev
c2410920d3 Safer ReadBufferFromS3 for merges and backports. 2021-08-04 09:14:20 +03:00
Alexey Milovidov
fd3b5680e2 Merge branch 'master' into async-reads 2021-08-03 23:13:31 +03:00
mergify[bot]
dc57254982
Merge branch 'master' into improve_create_or_replace 2021-08-03 11:39:07 +00:00
Maksim Kita
d9383270f6
Merge pull request #27054 from MaxWk/feature/add-default-database-to-system-users 
add column default_database to system.users
 2021-08-03 13:37:15 +03:00
Azat Khuzhin
97851bde08 Fix Distributed over Distributed for WithMergeableStateAfterAggregation* stages 
In case if one Distributed has multiple shards, and underlying
Distributed has only one, there can be the case when the query will be
tried to process from Complete to WithMergeableStateAfterAggregation,
which is obviously wrong.
 2021-08-03 10:10:08 +03:00
Kseniia Sumarokova
5d3abc14c7
Merge pull request #22045 from OmarBazaraa/mongo-ssl 
Support SSL connection in MongoDB engine
 2021-08-03 08:29:34 +03:00
tavplubix
58b50bad3a
Merge pull request #27033 from ClickHouse/database_replicated_fixes 
Fix assertions in Replicated database
 2021-08-02 23:33:18 +03:00
Azat Khuzhin
ff12f5102a Avoid running LIMIT BY/DISTINCT step on the initiator for optimize_distributed_group_by_sharding_key 
Before the following queries was running LimitBy/Distinct step on the
initator:

  select distinct sharding_key from dist order by k

While this can be omitted.
 2021-08-02 21:04:30 +03:00