Azat Khuzhin
07e8b2b3c7
Do not try to remove temporary paths that is currently in written by merge/mutation
...
v2: rebase against MergeTask
v3: rebase due to conflicts in src/Storages/MergeTree/MergeTreeDataMergerMutator.cpp
v4:
- rebase due to conflicts in src/Storages/MergeTree/MergeTask.cpp
- drop common/scope_guard_safe.h (not used)
2021-10-16 00:43:52 +03:00
Azat Khuzhin
0b3bf43d61
Rewrite MergeTreeData::clearOldTemporaryDirectories() to use early continue
2021-10-16 00:43:51 +03:00
Nikolai Kochetov
fd14faeae2
Remove DataStreams folder.
2021-10-15 23:18:20 +03:00
mergify[bot]
477ea62257
Merge branch 'master' into remove-sample-by
2021-10-15 15:14:17 +00:00
kssenii
759f747080
Fix build
2021-10-15 17:12:34 +03:00
kssenii
7b81766168
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into remote-connection
2021-10-15 12:36:44 +00:00
kssenii
ccb02cac56
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http
2021-10-15 12:36:30 +00:00
kssenii
881ae8617e
Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read
2021-10-15 15:09:56 +03:00
kssenii
edfb1a5521
Better performance for smaller number of threads
2021-10-15 15:07:39 +03:00
Nikita Mikhaylov
d1138a8a25
Merge pull request #29691 from azat/fix-merge-mutate-memory-tracking
...
Fix memory tracking for merges and mutations
2021-10-15 13:53:52 +03:00
Nikolai Kochetov
ad8a344b46
Move TTL streams and algo
2021-10-15 13:11:57 +03:00
Maksim Kita
7beaeb8b13
Merge branch 'master' into indexcache
2021-10-15 12:59:09 +03:00
Maksim Kita
27fdeef4b6
Merge pull request #29650 from kitaisreal/add-default-database-visitor-support-dict-get
...
AddDefaultDatabaseVisitor support dictGet
2021-10-15 12:28:58 +03:00
kssenii
3995506d37
Adjust range reader for remote fs reads
2021-10-15 11:36:26 +03:00
Azat Khuzhin
2a7a1d8df5
Avoid losing any allocations context from merges/mutations
2021-10-15 01:43:28 +03:00
Azat Khuzhin
8a209a78d7
Set query_id for mutations/merges
...
This will allow to distinguish allocations in trace_log.
2021-10-15 01:43:28 +03:00
Azat Khuzhin
8cc45bea7b
Avoid accounting memory from another mutation/merge
...
Before this path it was possible that one merge/mutation may account
memory from another, due to ThreadStatus::untracked_memory.
And this cause flakiness of 01200_mutations_memory_consumption.
2021-10-15 01:43:28 +03:00
Azat Khuzhin
fd38cbb0df
Fix memory tracking for merges and mutations (by destroying earlier)
...
It fixes only some tiny allocations, and so it should not affect any
huge mutations/merges.
And plus, this should not be a real fix, since peak_memory_usage is
obtained before even destrying this objects, and destroing objects will
unlikely update peak memory usage (although it is possible).
v0: do this in dtors
v2: do this earlier
2021-10-15 01:43:27 +03:00
Azat Khuzhin
e4b39ca99c
Remove unused Block member from MutateTask
...
v0: Fix excessive memory usage in MutateTask
This is relevant only for 01200_mutations_memory_consumption test, since
this is just a block.
v2: just remove unused Block member, since other part had been fixed in #29768
2021-10-15 01:43:27 +03:00
alexey-milovidov
bc1662b9fe
Merge pull request #29946 from azat/log-fix-data-race
...
Fix data-race between LogSink::writeMarks() and LogSource in StorageLog
2021-10-15 01:30:58 +03:00
alexey-milovidov
8d694cbcdc
Merge pull request #29959 from azat/log-data-race-file-checker-fix
...
Fix possible data-race between FileChecker and StorageLog/StorageStripeLog
2021-10-15 01:30:23 +03:00
mergify[bot]
92dd528c78
Merge branch 'master' into remove_trash
2021-10-14 14:14:07 +00:00
Anton Popov
476d7a411f
allow to remove SAMPLE BY expression
2021-10-14 17:05:20 +03:00
Maksim Kita
c1f04452de
AddDefaultDatabaseVisitor support dictGet
2021-10-14 16:31:32 +03:00
feng lv
400a31db06
reduce poll timeout
...
fix
2021-10-14 11:45:18 +00:00
Anton Popov
c2d376d0d6
Merge branch 'master' into better-interfaces
2021-10-14 14:33:36 +03:00
Nikolai Kochetov
ab28c6c855
Remove BlockInputStream interfaces.
2021-10-14 13:25:43 +03:00
Nikolai Kochetov
3d3e143c29
Merge pull request #30001 from ClickHouse/remove-streams-from-formats
...
Remove streams from formats.
2021-10-14 12:39:07 +03:00
Maksim Kita
f589084a94
Merge pull request #29899 from Enmk/Governance/dictionary_comment
...
Governance/dictionary comment
2021-10-14 12:35:10 +03:00
Anton Popov
92413aed68
better interfaces for IDataType and ISerialization
2021-10-14 05:36:49 +03:00
feng lv
21f7ecbd9f
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into filelog-engine
2021-10-14 02:09:51 +00:00
feng lv
3116767834
fix
2021-10-14 02:09:43 +00:00
Maksim Kita
659a46a972
WriteBufferToRabbitMQProduced fixed queue usage
2021-10-14 00:33:18 +03:00
Maksim Kita
04047f76c7
Fixed tests
2021-10-14 00:33:18 +03:00
Maksim Kita
d55561422f
Fixed tests
2021-10-14 00:33:18 +03:00
Maksim Kita
c9b6c2661b
Refactor ConcurrentBoundedQueue
2021-10-14 00:33:18 +03:00
Azat Khuzhin
eba6410fd5
Fix data-race between LogSink::writeMarks() and LogSource::readData() in StorageLog
...
CI founds [1], TSan report:
WARNING: ThreadSanitizer: data race (pid=497)
Write of size 8 at 0x7b1c00e4e8d8 by thread T922:
4 DB::LogSink::writeMarks() obj-x86_64-linux-gnu/../src/Storages/StorageLog.cpp:462:72 (clickhouse+0x1665cdaa)
5 DB::LogSink::consume() obj-x86_64-linux-gnu/../src/Storages/StorageLog.cpp:324:5 (clickhouse+0x1665c216)
Previous read of size 8 at 0x7b1c00e4e8d8 by thread T711:
1 DB::LogSource::readData()::$_0::operator()(bool) const::'lambda'()::operator()() const obj-x86_64-linux-gnu/../src/Storages/StorageLog.cpp:188:26 (clickhouse+0x16661baf)
9 DB::LogSource::readData() obj-x86_64-linux-gnu/../src/Storages/StorageLog.cpp:204:20 (clickhouse+0x1665bbcb)
[1]: https://clickhouse-test-reports.s3.yandex.net/29930/4bc90d1dd7dbd4b8a9b6920d00ca24e8b160358e/stress_test_(thread).html#fail1
Fixes : #29470
v2: rework locking, but this produce deadlock for 01499_log_deadlock test
v3: introduce per-file marks mutex
2021-10-14 00:17:45 +03:00
Azat Khuzhin
e4db862d85
Fix possible data-race between FileChecker and StorageLog/StorageStripeLog
2021-10-14 00:16:58 +03:00
Nikolay Degterinsky
63faf0212d
Merge branch 'master' into query_parameters
2021-10-13 23:55:09 +03:00
Nikolai Kochetov
2957971ee3
Remove some last streams.
2021-10-13 21:22:02 +03:00
alesapin
1801ea4955
Merge branch 'master' into remove_trash
2021-10-13 18:10:31 +03:00
feng lv
168020e0db
fix
2021-10-13 14:02:32 +00:00
alesapin
d2dfbb5ab6
Remove trash from MergeTreeReadPool
2021-10-13 15:25:44 +03:00
feng lv
c82fd3491d
fix
...
fix
2021-10-13 11:18:19 +00:00
feng lv
868dbc4bad
use poll replace select
...
fix
fix
2021-10-13 09:13:35 +00:00
mergify[bot]
cfda5592fe
Merge branch 'master' into fix-writing-storage-log-marks
2021-10-13 07:18:33 +00:00
feng lv
74e1900034
fix
2021-10-13 05:23:00 +00:00
Amos Bird
23602f4607
Speed up part loading for JBOD
2021-10-13 12:33:54 +08:00
feng lv
0409b775bf
fix
2021-10-13 02:11:15 +00:00
Nikolay Degterinsky
49c1a52f34
Merge branch 'master' into query_parameters
2021-10-13 03:04:40 +03:00
Azat Khuzhin
e9ce859b02
Fix race between MOVE PARTITION and merges/mutations for MergeTree
...
From time to time 00975_move_partition_merge_tree test got failed, the
reason is that there is merge in progress that writes data to the source
table again:
<details>
2021.10.12 13:20:40.243839 [ 26955 ] {2dbfea50-639e-49ac-9e82-933a00ea04a3} <Debug> executeQuery: (from [::1]:38060) (comment: 00975_move_partition_merge_tree.sql) ALTER TABLE test_move_partition_src MOVE PARTITION 1 TO TABLE test_move_partition_dest;
2021.10.12 13:20:40.244482 [ 26955 ] {2dbfea50-639e-49ac-9e82-933a00ea04a3} <Debug> test_t1tzb2.test_move_partition_dest (cab3b005-d54b-4cdc-8ab3-b005d54becdc): Cloning part /var/lib/clickhouse/store/467/467d7145-b47e-444e-867d-7145b47ea44e/1_2_2_0/ to /var/lib/clickhouse/store/cab/cab3b005-d54b-4cdc-8ab3-b005d54becdc/tmp_move_from_1_21_21_0
...
2021.10.12 13:20:40.373487 [ 378 ] {} <Trace> test_t1tzb2.test_move_partition_src (467d7145-b47e-444e-867d-7145b47ea44e) (MergerMutator): Merged 6 parts: from 1_2_2_0 to 1_11_11_0
...
2021.10.12 13:20:40.379750 [ 26955 ] {2dbfea50-639e-49ac-9e82-933a00ea04a3} <Debug> test_t1tzb2.test_move_partition_dest (cab3b005-d54b-4cdc-8ab3-b005d54becdc): Cloning part /var/lib/clickhouse/store/467/467d7145-b47e-444e-867d-7145b47ea44e/1_15_15_0/ to /var/lib/clickhouse/store/cab/cab3b005-d54b-4cdc-8ab3-b005d54becdc/tmp_move_from_1_28_28_0
</details>
And also remove cleaning of mutations since this will cause deadlock
after doing MOVE PARTITION under
currently_processing_in_background_mutex.
CI: https://clickhouse-test-reports.s3.yandex.net/0/a59c6b1c8eb47ebf77189a491ee0c3980b38e91c/functional_stateless_tests_
Fixes: 00975_move_partition_merge_tree
2021-10-13 00:14:24 +03:00
feng lv
b2a17cb320
fix build and conflict
2021-10-12 16:50:50 +00:00
Kseniia Sumarokova
bb15b57753
Merge pull request #30016 from ucasfl/sample-by-tuple
...
Fix crash of sample by tuple()
2021-10-12 19:46:21 +03:00
Nikolai Kochetov
f2d8cc547f
Fix Kafka source.
2021-10-12 18:09:28 +03:00
Nikolai Kochetov
c36bb9206e
Merge pull request #29879 from amosbird/queryidholderfix
...
Fix potential leak of query_id_holder
2021-10-12 15:42:02 +03:00
Nikolai Kochetov
077aba4a97
Merge pull request #29520 from amosbird/projection-improve4
...
Get rid of naming limitation of projections.
2021-10-12 14:25:29 +03:00
Nikolai Kochetov
93b3fcc195
Fix rabbitmq
2021-10-12 13:05:20 +03:00
kssenii
9814cb1b45
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http
2021-10-12 09:49:08 +00:00
Vladimir C
969999ff10
Merge pull request #29544 from Algunenano/join_deadlock
2021-10-12 12:34:04 +03:00
Maksim Kita
91a48e9651
Merge pull request #29896 from kitaisreal/system-data-skipping-indices-added-size
...
System table data skipping indices added size
2021-10-12 12:16:57 +03:00
Maksim Kita
c15b67c182
Fix naming
2021-10-12 11:42:24 +03:00
alesapin
5ac0d36392
Merge pull request #30005 from ClickHouse/more_strict_check_for_intersecting_parts
...
More strict check for intersecting parts
2021-10-12 11:38:09 +03:00
kssenii
e7dbfff5d2
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http
2021-10-12 06:19:15 +00:00
feng lv
b5d69d599e
fix sample by tuple()
...
add tests
fix
fix
fix
2021-10-12 05:19:32 +00:00
alexey-milovidov
f1b550cdaf
Merge pull request #29974 from ClickHouse/minimized-toolchain
...
Experiment with minimized toolchain (2)
2021-10-11 22:04:01 +03:00
Alexander Tokmakov
706e2b6b88
more strict check for intersecting parts
2021-10-11 21:42:46 +03:00
Nikolai Kochetov
ec18340351
Remove streams from formats.
2021-10-11 19:11:50 +03:00
kssenii
6d93253a67
Fix tests
2021-10-11 11:34:08 +00:00
kssenii
1de55cfeb4
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into remote-connection
2021-10-11 11:33:35 +00:00
Alexey Milovidov
2370a8d6da
Fix PVS-Studio
2021-10-11 14:28:46 +03:00
Amos Bird
72bccaa501
Fix path name
2021-10-11 19:12:08 +08:00
Maksim Kita
b0d887a0fe
Added tests
2021-10-11 14:00:10 +03:00
Amos Bird
83717b7c3b
Get rid of naming limitation of projections.
2021-10-11 18:32:17 +08:00
Kseniia Sumarokova
20e8074b5e
Merge pull request #29912 from taiyang-li/remove_unused_var
...
Remove unused variable in s3cluster
2021-10-11 12:45:04 +03:00
Maksim Kita
ce0c41e1ad
Fixed tests
2021-10-11 11:39:50 +03:00
Maksim Kita
2d069acc22
System table data skipping indices added size
2021-10-11 11:39:50 +03:00
Vasily Nemkov
2c55807b2a
Create dictionary with comment
2021-10-11 11:26:59 +03:00
Amos Bird
ebfb013ea1
Fix potential leak of query_id_holder
2021-10-11 15:56:00 +08:00
Vladimir C
dc4bb920d3
Merge branch 'master' into join_deadlock
2021-10-11 10:18:17 +03:00
Vladimir C
6c7fbf0b88
Merge pull request #29956 from azat/fix-tuple-filter
2021-10-11 10:02:47 +03:00
feng lv
a57e97d2e0
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into filelog-engine
2021-10-11 06:09:43 +00:00
feng lv
d8bd1474bc
fix
2021-10-11 04:18:21 +00:00
feng lv
4d9604e298
fix
2021-10-11 04:06:31 +00:00
kssenii
448d63a0fc
Range read with upper bound offset
2021-10-11 01:53:51 +03:00
alexey-milovidov
0adbcd58c9
Merge pull request #29930 from azat/buffer-fix-data-race
...
Fix data-race between flush() and startup() in StorageBuffer
2021-10-11 01:11:39 +03:00
kssenii
aa4b797808
Turn off tasks stealing for remote disk
2021-10-10 23:22:58 +03:00
Azat Khuzhin
6c33eaee32
Fix filtering by tuple (some conditions was lost during analyzing)
...
Fixes : #29281
Fixes: test_cluster_copier/test_three_nodes.py::test
2021-10-10 22:37:52 +03:00
Nikolai Kochetov
a95c28ec4b
Merge pull request #29898 from ClickHouse/remove-native-stream
...
Remove some more streams.
2021-10-10 21:01:16 +03:00
feng lv
045704274c
implement directory watcher
2021-10-10 16:31:32 +00:00
kssenii
eaebd84cd5
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into retriable-http
2021-10-10 15:17:35 +00:00
Azat Khuzhin
4bc90d1dd7
Fix data-race between flush() and startup() in StorageBuffer
...
Stress tests found [1], TSan report:
==================
WARNING: ThreadSanitizer: data race (pid=485)
Read of size 8 at 0x7b5001280bd8 by thread T567 (mutexes: write M612061890855345680):
1 std::__1::shared_ptr<DB::BackgroundSchedulePoolTaskInfo>::operator bool() const obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2851:62 (clickhouse+0x159140a6)
2 bool std::__1::operator!=<DB::BackgroundSchedulePoolTaskInfo>() obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3447:30 (clickhouse+0x159140a6)
3 DB::BackgroundSchedulePoolTaskHolder::operator bool() const obj-x86_64-linux-gnu/../src/Core/BackgroundSchedulePool.h:164:46 (clickhouse+0x159140a6)
4 DB::StorageBuffer::flush() obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:675:10 (clickhouse+0x159140a6)
Previous write of size 8 at 0x7b5001280bd8 by thread T586 (mutexes: write M191819750614415520):
2 std::__1::shared_ptr<DB::BackgroundSchedulePoolTaskInfo>::operator=(std::__1::shared_ptr<DB::BackgroundSchedulePoolTaskInfo>&&) obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3243:34 (clickhouse+0x15913e22)
3 DB::BackgroundSchedulePoolTaskHolder::operator=() obj-x86_64-linux-gnu/../src/Core/BackgroundSchedulePool.h:156:110 (clickhouse+0x15913e22)
4 DB::StorageBuffer::startup() obj-x86_64-linux-gnu/../src/Storages/StorageBuffer.cpp:668:18 (clickhouse+0x15913e22)
5 DB::InterpreterCreateQuery::doCreateTable() obj-x86_64-linux-gnu/../src/Interpreters/InterpreterCreateQuery.cpp:1092:10 (clickhouse+0x149bef7b)
6 DB::InterpreterCreateQuery::createTable() obj-x86_64-linux-gnu/../src/Interpreters/InterpreterCreateQuery.cpp:952:20 (clickhouse+0x149ba9f5)
7 DB::InterpreterCreateQuery::execute() obj-x86_64-linux-gnu/../src/Interpreters/InterpreterCreateQuery.cpp:1302:16 (clickhouse+0x149c1086)
[1]: https://clickhouse-test-reports.s3.yandex.net/0/1c9778603ff49563d1d3d0d357de0608167e504d/stress_test_(thread).html
Fixes : #29416
2021-10-10 04:03:36 +03:00
Maksim Kita
0358d664b9
Merge pull request #29882 from MaxWk/add-alias-for-name
...
Add table alias to system.tables and database alias to system.databases
2021-10-09 12:44:40 +03:00
Vasily Nemkov
67ff0f5dba
Refactoring: moved comment
out of ASTStorage to ASTCreateQuery
2021-10-09 12:25:27 +03:00
Nikolai Kochetov
939d38b13e
Fix gtest build.
2021-10-09 10:01:58 +03:00
taiyang-li
860f74db62
remove unused var
2021-10-09 11:15:13 +08:00
tavplubix
fa1396a9dd
Merge pull request #29790 from ClickHouse/fix-bad-cast-in-parser-create
...
Fix bad cast in ParserCreateQuery
2021-10-08 23:53:38 +03:00
Nikolai Kochetov
c6bce1a4cf
Update Native.
2021-10-08 20:21:19 +03:00
Nikolai Kochetov
340b53ef85
Remove some more streams.
2021-10-08 17:03:54 +03:00
kssenii
f3f820c6e0
HTTP read buffer retries
2021-10-08 15:41:55 +03:00
MaxWk
f0e193d5a5
add database alias
2021-10-08 20:25:37 +08:00
Nikolai Kochetov
9eddee5517
Merge pull request #29768 from ClickHouse/remove-merging-streams
...
Remove some merging streams
2021-10-08 13:15:10 +03:00
MaxWk
b081f58db5
add alias
2021-10-08 16:49:02 +08:00
Maksim Kita
717998a37f
Merge pull request #29825 from azat/memory_profiler_step-cleaner-api
...
Make memory_profiler_step API cleaner
2021-10-07 22:44:47 +03:00
tavplubix
de2fd9d762
Merge pull request #29812 from ClickHouse/better_diagnostic_for_optimize
...
Better diagnostic for OPTIMIZE
2021-10-07 22:22:25 +03:00
Kseniia Sumarokova
185d58a7cc
Merge pull request #29766 from ClickHouse/remove_redundant_seeks
...
Less seeks in compressed buffers
2021-10-07 17:04:50 +03:00
Alexander Tokmakov
7e2bc184ec
fix another suspicious places, add test
2021-10-07 16:43:49 +03:00
feng lv
4233e2cca4
fix
2021-10-07 13:23:25 +00:00
feng lv
4841b31b56
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into filelog-engine
2021-10-07 13:20:08 +00:00
tavplubix
754e038eec
Merge pull request #29803 from ClickHouse/intersects-previous-part-usability
...
Improve usability of error messages when error is caused by sophisticated interventions
2021-10-07 15:24:17 +03:00
feng lv
bd1ea5cd16
try fix undefine in ifstream
2021-10-07 12:15:36 +00:00
Alexander Tokmakov
5365cc686f
Merge branch 'master' into improvement_for_strange_mutations
2021-10-07 13:26:58 +03:00
Alexander Tokmakov
4119848a41
better diagnostic for OPTIMIZE
2021-10-07 13:08:10 +03:00
Vladimir C
0422939893
Merge pull request #29281 from lingtaolf/bugfix/tupleFilter
2021-10-07 12:24:26 +03:00
Nikolai Kochetov
78e1db209f
Remove more data streams ( #29491 )
...
* Remove more streams.
* Fixing build.
* Fixing build.
* Rename files.
* Fix fast test.
* Fix StorageKafka.
* Try fix kafka test.
* Move createBuffer to KafkaSource ctor.
* Revert "Move createBuffer to KafkaSource ctor."
This reverts commit 81fa94d27e
.
* Revert "Try fix kafka test."
This reverts commit 2107e54969
.
* Comment some rows in test.
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-10-07 11:26:08 +03:00
Azat Khuzhin
71cffbf521
Make memory_profiler_step API cleaner
...
Right now to configure memory_profiler_step/total_memory_profiler_step
you need to call:
MemoryTracker::setOrRaiseProfilerLimit()
MemoryTracker::setProfilerStep()
But it is easy to forget about setOrRaiseProfilerLimit(), since there is
no even any comments about this.
So instead, make setOrRaiseProfilerLimit() private and call it from
setProfilerStep()
2021-10-07 10:52:16 +03:00
feng lv
127a0f3591
fix
2021-10-07 07:31:41 +00:00
Maksim Kita
4ec7311d4d
Merge pull request #29738 from kitaisreal/added-function-ngram
...
Added function ngram
2021-10-07 10:21:00 +03:00
feng lv
d9959a4f22
fix
2021-10-07 05:56:09 +00:00
feng lv
28f8e1c297
fix
...
update test
2021-10-07 05:33:12 +00:00
Maksim Kita
b7b39da8cc
Fixed build
2021-10-06 23:22:29 +03:00
Nikolai Kochetov
d0c6f11fcb
More.
2021-10-06 20:59:27 +03:00
Alexey Milovidov
1455cfeddd
Improve usability of error messages when error is caused by sophisticated interventions
2021-10-06 15:28:52 +03:00
Maksim Kita
33353be4ed
Fixed tests
2021-10-06 15:12:07 +03:00
kssenii
1f91e2de46
Update install.sh
2021-10-06 12:08:37 +00:00
kssenii
c96b403809
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into remote-connection
2021-10-06 11:54:28 +00:00
feng lv
c801fa65df
fix and add new test
...
fix
fix
2021-10-06 10:56:42 +00:00
Nikolai Kochetov
6b619512aa
Fix style.
2021-10-06 12:37:56 +03:00
feng lv
acdb8233c3
fix
...
fix
fix
2021-10-06 09:06:33 +00:00
kssenii
5a87b6eb4f
Predefined configuration for table function remote
2021-10-06 05:55:09 +00:00
Maksim Kita
90eb74c933
Fixed tests
2021-10-06 00:37:45 +03:00
feng lv
f2a62ef9ee
try to fix data race found in DirectoryWatcher
2021-10-05 17:38:18 +00:00
mergify[bot]
ab55225c8f
Merge branch 'master' into remove-merging-streams
2021-10-05 14:16:34 +00:00
Nikolai Kochetov
96d070a5ba
Fix tests.
2021-10-05 16:58:24 +03:00
feng lv
0709b2e690
fix
2021-10-05 13:28:45 +00:00
mergify[bot]
3b1f49aac7
Merge branch 'master' into nv/parts-uuid-move-shard-kill
2021-10-05 12:36:02 +00:00
alesapin
9aab0eb0a2
Fixup
2021-10-05 13:41:09 +03:00
alesapin
8186339307
Less seeks in compressed buffers
2021-10-05 12:11:25 +03:00
Vitaly Baranov
8a01b32cba
Merge pull request #28637 from vitlibar/fix-materialized-column-as-sharding-key
...
Fix materialized column as sharding key
2021-10-05 10:53:24 +03:00
feng lv
86d31445fd
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into filelog-engine
2021-10-05 03:58:27 +00:00
Alexander Tokmakov
bb32432943
more fixes
2021-10-05 00:13:18 +03:00
Nikolai Kochetov
2001ebbf9d
Fix build.
2021-10-04 21:52:31 +03:00
Nikita Mikhaylov
b31d11478b
Merge pull request #29680 from azat/defines
...
Cleanup common defines
2021-10-04 20:13:29 +03:00
Nikita Mikhaylov
36516dd503
Merge pull request #29681 from azat/MemoryTracker-mutate-merge
...
Apply max_untracked_memory/memory_profiler_step/memory_profiler_sample_probability during mutate/merge
2021-10-04 20:04:57 +03:00
Maksim Kita
3fe9fdac96
Added function ngram
2021-10-04 18:53:17 +03:00
Vitaly Baranov
200f655a2f
Fix writing marks in StorageLog.
2021-10-04 18:45:04 +03:00
Nikolay Degterinsky
e03bf0c526
Merge branch 'master' into query_parameters
2021-10-04 17:23:17 +03:00
feng lv
873805d4da
update poco
...
update
fix unbundled build
2021-10-04 13:55:49 +00:00
Anton Popov
362e84a336
Merge pull request #29429 from Enmk/Governance/database_comment
...
Implemented creating databases with comments
2021-10-04 15:12:33 +03:00
feng lv
45a49b399a
fix typo
2021-10-04 08:23:02 +00:00
Vitaly Baranov
1636ee24bb
Fix using materialized column as sharding key.
2021-10-04 10:56:42 +03:00
feng lv
f40a17c300
fix bug in updateFileInfos
...
update
2021-10-04 07:45:37 +00:00
Kseniia Sumarokova
5ab17d795f
Merge pull request #29673 from ClickHouse/url-fix-initialization
...
Fix initialization of `StorageURLSource`
2021-10-04 10:23:52 +03:00
feng lv
8c82e8b9bc
fix
...
fix
2021-10-03 17:49:00 +00:00
kssenii
dbc114e6fc
Fix special build check
2021-10-03 16:49:02 +00:00
kssenii
7ea9037939
Fix
2021-10-03 16:41:12 +00:00
Azat Khuzhin
117e9e77c8
Apply max_untracked_memory/memory_profiler_{step,sample_probability} during mutate/merge
2021-10-03 17:39:07 +03:00
kssenii
0cb7193fde
Fix url with failover case
2021-10-03 13:58:03 +00:00
Azat Khuzhin
ae5ee2dd28
Move macros for distributed engine into separate header
2021-10-03 14:34:03 +03:00
Azat Khuzhin
6a9dd9828d
Move protocol macros into separate header
...
Defines.h is a very common header, so lots of modules will be recompiled
on changes.
Move macros for protocol into separate header, this should significantly
decreases number of units to compile on it's changes.
2021-10-03 14:34:03 +03:00
feng lv
9683513c52
fix build
2021-10-03 11:03:20 +00:00
feng lv
8a4092c55b
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into filelog-engine
2021-10-03 10:00:16 +00:00
feng lv
3a56cc5141
fix
...
update test
2021-10-03 09:49:11 +00:00
Alexey Milovidov
868215d0dc
Fix initialization of StorageURLSource
2021-10-03 07:28:28 +03:00
Maksim Kita
395119339f
Merge branch 'master' into rename-common
2021-10-02 17:15:10 +03:00
alexey-milovidov
86d1ce63cf
Merge pull request #29649 from CurtizJ/merging-29376
...
Merging #29376
2021-10-02 12:02:42 +03:00
Alexey Milovidov
2ad5dbdfe8
Rename "common" to "base"
2021-10-02 11:30:38 +03:00
Alexey Milovidov
fe6b7c77c7
Rename "common" to "base"
2021-10-02 10:13:14 +03:00
Anton Popov
bb92ff52d8
Update IMergeTreeDataPart.cpp
2021-10-02 00:50:19 +03:00
Anton Popov
d404efbf7e
add test for freeze in-memory parts
2021-10-01 22:05:36 +03:00
Anton Popov
f0363c524d
Merge branch 'master' of git://github.com/mo-avatar/ClickHouse into merging-29376
2021-10-01 20:29:33 +03:00
Maksim Kita
57666725f9
Merge pull request #29623 from kitaisreal/bloom-filter-indexes-map-support-fix-unsafe-identifier-cast
...
BloomFilter indexes map data type support fix unsafe identifier cast
2021-10-01 19:28:04 +03:00
Nikita Mikhaylov
b2f7b6fae8
Fix race between MergeTask and storage destruction ( #29614 )
2021-10-01 18:55:04 +03:00
feng lv
9c1dcf41b8
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into filelog-engine
2021-10-01 14:47:21 +00:00
feng lv
5bb5e04fb3
fix assert
2021-10-01 14:47:05 +00:00
vdimir
7444d64156
Fix for recursive tuples in MergeTreeWhereOptimizer
2021-10-01 17:01:03 +03:00
vdimir
23ce41e3bb
MergeTreeWhereOptimizer supports constant tuple in both sides of equals
2021-10-01 17:01:03 +03:00
vdimir
64ffacd9e0
MergeTreeWhereOptimizer::tryAnalyzeTuple, fix fuzz
2021-10-01 17:01:02 +03:00
vdimir
e326b12a37
Fix style in MergeTreeWhereOptimizer.cpp
2021-10-01 17:01:02 +03:00
凌涛
f660224a5f
add tests
2021-10-01 17:01:01 +03:00
凌涛
ea68944055
fixed index didn't work in tuple where
2021-10-01 17:01:01 +03:00
Maksim Kita
c793c8e7fa
BloomFilter indexes map data type support fix unsafe identifier cast
2021-10-01 14:40:00 +03:00
feng lv
6eeade23fa
revert drop table immediately
...
remove unused code
fix pvs check
fix
2021-10-01 11:36:49 +00:00
Vasily Nemkov
2f16806cd0
Implemented creating databases with comments
2021-10-01 11:51:26 +03:00
feng lv
ac9af1ca38
fix wrong offset
2021-10-01 08:24:57 +00:00
feng lv
8c20a184ca
optimize event handle
2021-10-01 05:06:19 +00:00
Alexey Milovidov
4271f8c738
Merge remote-tracking branch 'origin/master' into yandex-to-clickhouse-in-configs
2021-10-01 01:33:58 +03:00
Nikita Mikhaylov
9756b8dc33
Added an ability to execute more merges and mutations than threads, added new settings ( #29140 )
2021-10-01 00:26:24 +03:00
Kseniia Sumarokova
3e0d4faa1e
Merge pull request #29503 from azat/external-db-log-query
...
Log queries to external databases (since they may be rewritten)
2021-09-30 21:08:57 +03:00
feng lv
4390c8f029
fix
2021-09-30 17:13:45 +00:00
feng lv
ee02e52d3d
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into filelog-engine
2021-09-30 16:02:31 +00:00
feng lv
1758ff2d32
fix
2021-09-30 16:02:17 +00:00
Alexey Milovidov
551c52c236
Merge remote-tracking branch 'origin/master' into yandex-to-clickhouse-in-configs
2021-09-30 17:45:14 +03:00
Raúl Marín
6f2447c027
clang-tidy fix
2021-09-30 15:48:54 +02:00
Raúl Marín
f58742014c
Consistent naming
2021-09-30 11:05:21 +02:00
Nikolay Degterinsky
7f3d395bf3
Merge branch 'master' into query_parameters
2021-09-30 02:01:33 +03:00
Azat Khuzhin
a4f2663b82
Log queries to external databases (since they may be rewriten)
...
By rewritten means that some expression from WHERE can be removed, yes
there is external_table_strict_query, but sometimes it is too strict.
2021-09-29 21:56:28 +03:00
Azat Khuzhin
6ddfbf39c8
Cover IS NULL/IS NOT NULL for external dbs in unit tests
2021-09-29 21:45:09 +03:00
Azat Khuzhin
24f55e29d1
Add support for IS NULL/IS NOT NULL for external dbs
2021-09-29 21:45:09 +03:00
Nikolai Kochetov
68f8b9d235
Update ColumnGathererStream
2021-09-29 20:45:01 +03:00
Raúl Marín
0ee5c0bff5
Use RWLock in StorageJoin to avoid deadlocks
2021-09-29 19:30:07 +02:00
mergify[bot]
6a2ac6e871
Merge branch 'master' into yandex-to-clickhouse-in-configs
2021-09-29 13:29:08 +00:00
Vladimir C
27f0f9fcf3
Merge pull request #21320 from arenadata/ADQM-138
2021-09-29 11:53:54 +03:00
Maksim Kita
0279afa738
Merge pull request #29474 from kitaisreal/bloom-filter-indexes-updated-tests
...
Bloom filter indexes updated tests
2021-09-28 22:40:27 +03:00
Maksim Kita
9ecd0a3f4f
Merge pull request #29220 from ClickHouse/full-text-bloom-filter-index-map-data-type
...
Merging #28511
2021-09-28 22:40:17 +03:00
alexey-milovidov
1e6f9ac635
Merge branch 'master' into yandex-to-clickhouse-in-configs
2021-09-28 20:26:44 +03:00
mo-avatar
6c16ec83eb
in memory part lost when freeze
2021-09-28 22:52:01 +08:00
vdimir
71b6c9414c
Minor changes related to JOIN ON ORs
2021-09-28 14:11:31 +03:00
vdimir
8e2637aab2
Store all related to one join disjunct in JoinOnClause, pt1
2021-09-28 14:11:31 +03:00
Ilya Golshtein
3766d47f31
ORs in JOINs
2021-09-28 14:11:28 +03:00
Maksim Kita
f2402e1a10
Fixed tests
2021-09-28 12:52:07 +03:00
Maksim Kita
71fb9e6059
Bloom filter indexes updated tests
2021-09-28 12:38:27 +03:00
feng lv
1f90c8dc22
refactor
...
fix style
2021-09-28 09:17:50 +00:00
Maksim Kita
9f9f36b480
Merge pull request #29464 from azat/std-cerr-cout-style-check
...
Add std::cerr/std::cout style check
2021-09-28 12:12:44 +03:00
Vladimir C
6c2db9c29b
Merge pull request #29428 from vdimir/rocksdb-istream-to-source
2021-09-28 12:06:41 +03:00
Azat Khuzhin
9a23b20965
Remove debug from MergeTreeIndexConditionBloomFilter::traverseASTEquals()
2021-09-28 10:28:28 +03:00
feng lv
ef5c71a9ab
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into filelog-engine
2021-09-28 02:06:28 +00:00
alexey-milovidov
995913e89c
Merge branch 'master' into yandex-to-clickhouse-in-configs
2021-09-28 02:27:36 +03:00
Maksim Kita
7861c26eaf
Merge pull request #29431 from kitaisreal/map-bloom-filter-index-map-values-equals-support
...
Map bloom filter index mapValues equals function support
2021-09-27 22:03:53 +03:00
Nikolai Kochetov
236d71ea94
Merge pull request #28582 from ClickHouse/rewrite-pushing-to-views
...
Rewrite PushingToViews
2021-09-27 21:19:11 +03:00
alesapin
ddd80c8e72
Merge pull request #29264 from Enmk/governance/table_comment_update
...
Implemented modifying table comments with `ALTER TABLE t MODIFY COMMENT 'value'`
2021-09-27 18:02:14 +03:00
Maksim Kita
e290f19957
Merge branch 'master' into full-text-bloom-filter-index-map-data-type
2021-09-27 17:50:07 +03:00
Maksim Kita
b28d6e76cc
Map bloom filter full text index added support for mapKeys, mapValues
2021-09-27 17:26:25 +03:00
Maksim Kita
cd9c796e6f
Map bloom filter index mapValues equals function support
2021-09-27 17:23:56 +03:00
vdimir
aea2b952fd
Merge EmbeddedRocksDBBlockInputStream and EmbeddedRocksDBSource
2021-09-27 16:16:34 +03:00
Kseniia Sumarokova
beda6bee2e
Merge pull request #29276 from kssenii/hdfs-fix-assertion
...
Fix !hasPendingData() failed assertion in hdfs read buffer
2021-09-27 15:47:17 +03:00
feng lv
814f6d5068
fix
2021-09-27 12:09:01 +00:00
Nikita Mikhaylov
3741b8a17e
Merge pull request #29337 from amosbird/projection-fix18
...
Fix projection merges and mutations.
2021-09-27 13:45:13 +03:00
vdimir
8fe17dd3d1
Revert "Rename EmbeddedRocksDBBlockInputStream -> EmbeddedRocksDBSource"
...
This reverts commit 3ab217031e50a0b4d19fb24d72088e353cdfa510.
2021-09-27 12:49:14 +03:00
vdimir
dc7393e810
Rename EmbeddedRocksDBBlockInputStream -> EmbeddedRocksDBSource
2021-09-27 12:49:13 +03:00
vdimir
222ab78931
Convert EmbeddedRocksDBBlockInputStream to ISource
2021-09-27 12:49:13 +03:00
feng lv
30428ad1b0
fix
2021-09-27 08:44:48 +00:00
Nikolai Kochetov
78a7665f43
Merge branch 'master' into rewrite-pushing-to-views
2021-09-27 10:56:50 +03:00
feng lv
56be3fab00
fix
2021-09-27 07:17:16 +00:00
feng lv
1eb96c2fce
fix
2021-09-27 04:39:50 +00:00
feng lv
c8062ad397
fix
2021-09-27 04:25:59 +00:00
feng lv
69985ac4ae
fix
2021-09-27 02:27:24 +00:00
Alexey Milovidov
1227a383e3
Merge branch 'remove-trash-db-regexp' of github.com:yandex/ClickHouse into remove-trash-db-regexp
2021-09-27 02:17:19 +03:00
Alexey Milovidov
16f0f0af7f
Merge remote-tracking branch 'origin/master' into remove-trash-db-regexp
2021-09-27 02:12:13 +03:00
mergify[bot]
64669a28ce
Merge branch 'master' into yandex-to-clickhouse-in-configs
2021-09-26 22:59:23 +00:00
feng lv
da2c8f23cc
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into filelog-engine
2021-09-26 15:00:37 +00:00
Nikolai Kochetov
efbd8e4911
Remove ExceptionHandlingSink.
2021-09-26 17:54:59 +03:00
feng lv
6c4b0f2d37
drop table immediately
...
fix style
update
2021-09-26 14:13:12 +00:00
feng lv
30527879b9
fix style
2021-09-26 12:19:19 +00:00
feng lv
7af96b12b1
add meta file
...
fix and update tests
2021-09-26 11:28:46 +00:00
mergify[bot]
15e5204a7c
Merge branch 'master' into issue-29227-rocksdb-truncate
2021-09-26 00:10:30 +00:00
Alexey Milovidov
fb7ce28198
Merge branch 'master' into yandex-to-clickhouse-in-configs
2021-09-26 02:10:11 +03:00
Kseniia Sumarokova
a7809d3d63
Merge pull request #29365 from kssenii/fix-mysql
...
Fix integration test
2021-09-26 00:52:10 +03:00
alexey-milovidov
e4363872ca
Merge branch 'master' into yandex-to-clickhouse-in-configs
2021-09-25 19:58:36 +03:00
alexey-milovidov
9dac348893
Update StorageMerge.cpp
2021-09-25 19:41:50 +03:00
kssenii
30b92d24f1
Fix
2021-09-25 14:46:03 +00:00
Amos Bird
34017c942e
More fixes
2021-09-25 13:38:56 +08:00
Alexey Milovidov
4f7830ae3a
Merge branch 'master' into yandex-to-clickhouse-in-configs
2021-09-25 06:58:30 +03:00
Alexey Milovidov
fe44be522e
Fix assert in table function merge
with database regexp
2021-09-25 05:48:24 +03:00
Alexey Milovidov
cd7f9d981c
Remove ya.make
2021-09-25 04:22:54 +03:00
Anton Popov
7628273df4
Merge pull request #29266 from CurtizJ/normalize-asts
...
Normalize ASTs in ColumnsDescription
2021-09-25 04:01:54 +03:00
Alexey Milovidov
82d57c6e47
Merge branch 'master' into yandex-to-clickhouse-in-configs
2021-09-25 01:38:39 +03:00
feng lv
0009cab962
add virtual columns
2021-09-24 16:44:22 +00:00
Amos Bird
23d3d894e6
Fix projection merges and mutations.
2021-09-24 22:45:50 +08:00
Anton Popov
62dea79b78
better formatting
2021-09-24 14:41:49 +03:00
Anton Popov
4017b1eb89
Update src/Storages/ColumnsDescription.cpp
...
Co-authored-by: Vladimir C <vdimir@clickhouse.com>
2021-09-24 14:08:33 +03:00
Nikolai Kochetov
9d3e8fb9c3
Tiny fix.
2021-09-24 13:41:52 +03:00
Nikolai Kochetov
cc53ba283d
Try fix other tests.
2021-09-24 13:23:28 +03:00
Vladimir C
604dc8861d
Apply suggestions from code review
...
Co-authored-by: Azat Khuzhin <a3at.mail@gmail.com>
2021-09-24 13:07:24 +03:00
Nikolai Kochetov
e617aeb7a5
Merge pull request #29139 from amosbird/projection-fix17
...
Disable projections when ARRAY JOIN is used.
2021-09-24 11:15:23 +03:00
feng lv
9456c67486
fix
...
fix test
fix
fix
fix
exception handle
2021-09-24 02:59:47 +00:00
alexey-milovidov
b8083929a6
Merge branch 'master' into yandex-to-clickhouse-in-configs
2021-09-23 20:49:40 +03:00
Amos Bird
983ee004c7
Add empty key description builder.
2021-09-24 01:35:41 +08:00
Dmitrii Kovalkov
e3ae32f763
Allow nulls first
2021-09-23 19:39:15 +03:00
vdimir
9ca9a66a70
Fix race in StorageSystemRocksDB truncate
2021-09-23 17:03:33 +03:00
vdimir
81a051e3ee
Use dynamic_pointer_cast instead of dynamic_cast in StorageSystemRocksDB
2021-09-23 16:40:10 +03:00
Anton Popov
0390035963
fix unit test
2021-09-23 16:11:31 +03:00
Kruglov Pavel
55553026ee
Merge pull request #29206 from azat/strict-external-queries
...
Add ability to disable converting expressions to local filter for external queries
2021-09-23 13:22:12 +03:00
Nikolai Kochetov
998d29ebc7
Merge branch 'master' into rewrite-pushing-to-views
2021-09-23 13:10:27 +03:00
Maksim Kita
3d79812427
Merge branch 'master' into full-text-bloom-filter-index-map-data-type
2021-09-23 12:38:03 +03:00
Maksim Kita
0341b99c69
Merge pull request #29280 from kitaisreal/full-text-bloom-filter-added-support-for-array-data-type
...
FullText bloom filter index added support for Array data type
2021-09-23 12:34:06 +03:00
Maksim Kita
d9e265df2c
Fixed debug message
2021-09-23 12:13:01 +03:00
Maksim Kita
d59986f224
Merge pull request #29278 from kitaisreal/bloom-filter-map-added-support-for-has-function
...
Bloom filter map added support for has function
2021-09-23 12:03:56 +03:00
kssenii
6c1312760f
Remove debug
2021-09-23 07:52:52 +00:00
Kseniia Sumarokova
f7ff14b407
Merge pull request #28577 from kssenii/named-collections
...
Predefined connections and other parameters for external data sources
2021-09-23 10:25:23 +03:00
kssenii
d71b6ad657
Fix
2021-09-23 07:21:08 +00:00
Maksim Kita
ebbefd49c3
FullText bloom filter index added support for Array data type
2021-09-23 02:06:54 +03:00
Vasily Nemkov
cb1ca9b33e
Implemented modifying table comments with ALTER TABLE t MODIFY COMMENT 'value'
2021-09-23 01:14:00 +03:00
Maksim Kita
d5a7e92b97
Bloom filter map added support for has function
2021-09-23 01:10:14 +03:00
Maksim Kita
1bf375e2b7
Merge pull request #29254 from kitaisreal/map-bloom-filter-index-updated
...
Updated bloom filter index for map type to work with mapKeys, mapValues functions
2021-09-23 00:31:14 +03:00
Nikita Mikhaylov
32aa2ca939
Merge pull request #29257 from nikitamikhaylov/race-merge-task
...
Fix segfault in fast test
2021-09-23 00:12:31 +03:00
Nikolai Kochetov
4f802d1cea
Fix test_executable_table_function
2021-09-22 22:31:12 +03:00
hhell
fc166570d8
Fix typo in an error message
...
Related: https://github.com/ClickHouse/ClickHouse/issues/20551
2021-09-22 21:33:15 +03:00
Maksim Kita
93208fca4b
Bloom filter for map data type dded support for IN, mapContains functions
2021-09-22 20:25:10 +03:00
Kseniia Sumarokova
b41a446e8b
Delete getEngineConnectionArguments.h
2021-09-22 19:52:55 +03:00
Anton Popov
8943389d03
normalize ASTs in ColumnsDescription
2021-09-22 18:55:20 +03:00
kssenii
a84fc77b52
Fixes
2021-09-22 18:10:25 +03:00
tavplubix
ba82781e11
Merge pull request #29061 from ClickHouse/remove_some_trash
...
Fix `system.distributed_ddl_queue`
2021-09-22 16:58:29 +03:00
mergify[bot]
fd39fc58d7
Merge branch 'master' into yandex-to-clickhouse-in-configs
2021-09-22 13:35:20 +00:00
mergify[bot]
a3f62af09c
Merge branch 'master' into race-merge-task
2021-09-22 12:42:45 +00:00
Nikita Mikhaylov
558c74e826
Done
2021-09-22 11:52:47 +00:00
Nikolay Degterinsky
1b1d90deb8
Merge branch 'master' into query_parameters
2021-09-22 13:53:56 +03:00
Maksim Kita
92fd6b376f
Updated bloom filter index for map type to work with mapKeys function
2021-09-22 12:45:48 +03:00
kssenii
b51840a205
Merge branch 'master' of github.com:ClickHouse/ClickHouse into named-collections
2021-09-22 11:45:22 +03:00
feng lv
f637f4f2d1
remove unused variable
...
update test
update
2021-09-22 07:24:18 +00:00
feng lv
dfdf756b6a
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into filelog-engine
2021-09-22 02:51:51 +00:00