thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 19:32:07 +00:00
Code Issues Packages Projects Releases Wiki Activity
71,535 Commits 9,319 Branches 1,783 Tags 2.1 GiB
f8d4f04294
Commit Graph

2265 Commits

Author SHA1 Message Date
mergify[bot]
8182b6e890
Merge branch 'master' into better_remove_empty_parts 2021-06-30 08:42:26 +00:00
alesapin
0af53ace55 Fix comment 2021-06-29 22:49:41 +03:00
alesapin
ca1d21b663
Update src/Storages/MergeTree/MergeTreeData.cpp 
Co-authored-by: tavplubix <avtokmakov@yandex-team.ru>
 2021-06-29 22:48:46 +03:00
alesapin
a6834213a1 Add tests 2021-06-29 22:47:54 +03:00
Nikolai Kochetov
d5bd2b1fa9
Merge pull request #25719 from ClickHouse/remove-PrewhereDAGInfo 
Remove PrewhereDAGInfo.
 2021-06-29 18:58:03 +03:00
alesapin
5822f0ba29 Replace lost parts with empty parts instead of hacking replication queue 2021-06-29 18:14:44 +03:00
Nikolai Kochetov
bf827936b7 Rename PrewhereActions 2021-06-29 14:53:34 +03:00
alesapin
7680eab0e4 one more fix 2021-06-29 13:03:00 +03:00
alesapin
941a6e539f Fix 2021-06-29 10:36:20 +03:00
alesapin
72b281987e Add more debug 2021-06-28 17:15:51 +03:00
alesapin
d20feeb42c Relax check during DROP PART 2021-06-28 15:05:22 +03:00
alesapin
b11254d191
Merge branch 'master' into fix_aggregation_ttl 2021-06-28 13:31:29 +03:00
alesapin
4c213b639f
Merge pull request #25548 from nikitamikhaylov/background-processing 
A little improvement in BackgroundJobsExecutor
 2021-06-28 13:24:21 +03:00
alexey-milovidov
a11351a473
Merge pull request #25484 from vdimir/issue-19589 
Fix assert with non uint8 in prewhere
 2021-06-28 00:03:12 +03:00
alesapin
c977c33d6d Fix bug in execution of TTL GROUP BY 2021-06-27 19:18:15 +03:00
Anton Popov
2165ff679a
Merge branch 'master' into read-in-order-perf-2 2021-06-27 17:45:00 +03:00
alexey-milovidov
6662e7ab0f
Merge pull request #25732 from matwey/missed_include_optional_2 
Fix some more missed includes
 2021-06-26 14:04:06 +03:00
Matwey V. Kornilov
a29bd2c4b3 Add missed #include <limits> 
Signed-off-by: Matwey V. Kornilov <matwey.kornilov@gmail.com>
 2021-06-26 11:25:54 +03:00
Anton Popov
fafae3ab8d fix tests 2021-06-25 20:00:23 +03:00
Anton Popov
fa8ad1a7c6 reduce memory usage in queries with 'ORDER BY primary_key' 2021-06-25 19:23:41 +03:00
Ivan
14e43cfafa
Add CI check for darwin-aarch64 (#25560) 
* Add support for darwin-aarch64

* Fix PVS warnings

* Add build configuration

* Fix packager args
 2021-06-25 18:21:56 +03:00
Nikolai Kochetov
3544b059ce Merge branch 'master' into remove-PrewhereDAGInfo 2021-06-25 17:52:03 +03:00
Nikolai Kochetov
6bc0a628cd Remove PrewhereDAGInfo. 2021-06-25 17:49:28 +03:00
alesapin
f6afc381be
Merge pull request #25665 from ClickHouse/fix_24869 
Fix REPLACE_RANGE entry removal
 2021-06-25 11:11:25 +03:00
alesapin
d138ba79c7
Merge pull request #25549 from ClickHouse/add_test_for_concurrent_merges 
Fix bug in TTL merges assignment
 2021-06-25 11:02:29 +03:00
Alexander Tokmakov
76156af5cc cancel merges on drop partition 2021-06-24 17:07:43 +03:00
Ivan
e9e49a9085
Fix ANTLR parser and enable it back in CI (#25638) 
* Fix

* Skip more tests
 2021-06-24 14:45:29 +03:00
Alexander Tokmakov
7e6963bb39 fix REPLACE_RANGE entry removal 2021-06-24 11:16:54 +03:00
alesapin
e6da53dc46 better 2021-06-24 00:09:06 +03:00
alesapin
24876291d1 Followup fix 2021-06-24 00:01:07 +03:00
alesapin
4be4bc21e2 Fix for fix 2021-06-23 23:57:49 +03:00
alesapin
81c74435a3 Fix drop part bug 2021-06-23 22:25:30 +03:00
Nikita Mikhaylov
f4d6a6e128 support dynamic reloading of server config 2021-06-22 23:24:47 +00:00
Nikita Mikhaylov
a39bca0acf better 2021-06-22 23:24:47 +00:00
Nikita Mikhaylov
c66a3b22b5 done 2021-06-22 23:24:47 +00:00
mergify[bot]
c15493f0df
Merge branch 'master' into add_test_for_concurrent_merges 2021-06-22 15:49:35 +00:00
alesapin
47b29092b4 Fix bug in MergerMutator parts selector 2021-06-22 16:47:42 +03:00
Neng Liu
7ed1728a37 Merge branch 'master' into datatype-date32 
# Conflicts:
#	src/DataTypes/IDataType.h
#	src/Functions/CustomWeekTransforms.h
 2021-06-22 10:00:18 +08:00
Anton Popov
662d5b8495 fix ALTER MODIFY COLUMN of columns, that participates in TTL expressions 2021-06-21 18:20:18 +03:00
vdimir
46a5dd6701
Fix MergeTreeBaseSelectProcessor::executePrewhereActions 2021-06-21 16:01:02 +03:00
Anton Popov
ffa56bde24 fix usage of index with array columns and ARRAY JOIN 2021-06-21 15:34:05 +03:00
vdimir
eb85c5a8e7
Fix assert with non uint8 in prewhere 2021-06-21 15:32:29 +03:00
Mike Kot
4c391f8e99
SYSTEM RESTORE REPLICA replica [ON CLUSTER cluster] (#13652) 
* initial commit: add setting and stub

* typo

* added test stub

* fix

* wip merging new integration test and code proto

* adding steps interpreters

* adding firstly proposed solution (moving parts etc)

* added checking zookeeper path existence

* fixing the include

* fixing and sorting includes

* fixing outdated struct

* fix the name

* added ast ptr as level of indirection

* fix ref

* updating the changes

* working on test stub

* fix iterator -> reference

* revert rocksdb submodule update

* fixed show privileges test

* updated the test stub

* replaced rand() with thread_local_rng(), updated the tests

updated the test

fixed test config path

test fix

removed error messages

fixed the test

updated the test

fixed string literal

fixed literal

typo: =

* fixed the empty replica error message

* updated the test and the code with logs

* updated the possible test cases, updated

* added the code/test milestone comments

* updated the test (added more testcases)

* replaced native assert with CH one

* individual replicas recursive delete fix

* updated the AS db.name AST

* two small logging fixes

* manually generated AST fixes

* Updated the test, added the possible algo change

* Some thoughts about optimizing the solution:

ALTER MOVE PARTITION .. TO TABLE -> move to detached/ + ALTER ... ATTACH

* fix

* Removed the replica sync in test as it's invalid

* Some test tweaks

* tmp

* Rewrote the algo by using the executeQuery instead of

hand-crafting the ASTPtr.

Two questions still active.

* tr: logging active parts

* Extracted the parts moving algo into a separate helper function

* Fixed the test data and the queries slightly

* Replaced query to system.parts to direct invocation,

started building the test that breaks on various parts.

* Added the case for tables when at least one replica is alive

* Updated the test to test replicas restoration by detaching/attaching

* Altered the test to check restoration without replica restart

* Added the tables swap in the start if the server failed last time

* Hotfix when only /replicas/replica... path was deleted

* Restore ZK paths while creating a replicated MergeTree table

* Updated the docs, fixed the algo for individual replicas restoration case

* Initial parts table storage fix, tests sync fix

* Reverted individual replica restoration to general algo

* Slightly optimised getDataParts

* Trying another solution with parts detaching

* Rewrote algo without any steps, added ON CLUSTER support

* Attaching parts from other replica on restoration

* Getting part checksums from ZK

* Removed ON CLUSTER, finished working solution

* Multiple small changes after review

* Fixing parallel test

* Supporting rewritten form on cluster

* Test fix

* Moar logging

* Using source replica as checksum provider

* improve test, remove some code from parser

* Trying solution with move to detached + forget

* Moving all parts (not only Committed) to detached

* Edited docs for RESTORE REPLICA

* Re-merging

* minor fixes

Co-authored-by: Alexander Tokmakov <avtokmakov@yandex-team.ru>
 2021-06-20 11:24:43 +03:00
Amos Bird
f2ed5ef42b
Nullable primary key with correct KeyCondition 2021-06-18 23:04:24 +08:00
Latysheva Alexandra
48d3419160
Merge branch 'master' into alexelex-encryption-1 2021-06-18 11:28:08 +06:00
Neng Liu
b6eab5fd2f add datetype32 function support 2021-06-17 16:05:30 +08:00
Maksim Kita
67e9b85951 Merge ext into common 2021-06-16 23:28:41 +03:00
Kseniia Sumarokova
d6240f2bd4
Merge pull request #25309 from azat/fs-fix 
Catch ErrnoException during parts cleaning
 2021-06-16 22:03:54 +03:00
tavplubix
07ce932282
Merge pull request #25306 from ClickHouse/tavplubix-patch-2 
Forcefully stop queue tasks on restarting
 2021-06-16 14:22:36 +03:00
Nikolai Kochetov
8bf55ae2b5
Merge pull request #25166 from ClickHouse/always-detach-parts-with-wrong-partition-id 
Always detach parts with wrong partition ID
 2021-06-16 11:06:07 +03:00
Kseniia Sumarokova
e08c05cdf5
Merge pull request #25295 from ClickHouse/remove-some-code-from-key-condition 
Remove some code from KeyCondition.
 2021-06-16 10:12:12 +03:00
Azat Khuzhin
de9689379f Catch ErrnoException during parts cleaning 
Avoids noisy messages like:

    2021.06.15 09:27:35.546982 [ 58176 ] {} <Error> auto DB::IBackgroundJobExecutor::jobExecutingTask()::(anonymous class)::operator()() const: Code: 481, e.displayText() = DB::ErrnoException: Cannot check modification time for file: /var/lib/clickhouse/store/c52/c52619f8-99bd-435e-8526-19f899bdb35e/tmp_insert_202106_7010_7010_0/, errno: 2, strerror: No such file or directory, Stack trace (when copying this message, always include the lines below):

    0. DB::Exception::Exception() @ 0x903209a in /usr/lib/debug/.build-id/be/ea088a1830cb2f.debug
    1. DB::throwFromErrnoWithPath() @ 0x9033089 in /usr/lib/debug/.build-id/be/ea088a1830cb2f.debug
    2. FS::getModificationTime() @ 0x907b166 in /usr/lib/debug/.build-id/be/ea088a1830cb2f.debug
    3. DB::DiskLocal::getLastModified() @ 0xf9a6669 in /usr/lib/debug/.build-id/be/ea088a1830cb2f.debug
    4. DB::MergeTreeData::clearOldTemporaryDirectories(long) @ 0x1069a6de in /usr/lib/debug/.build-id/be/ea088a1830cb2f.debug
 2021-06-16 06:56:55 +03:00
tavplubix
60c8dc1c0b
Update ReplicatedMergeTreeRestartingThread.cpp 2021-06-16 00:48:38 +03:00
Nikolai Kochetov
96d98ff020 Add comment 2021-06-15 21:42:26 +03:00
mergify[bot]
7959d92029
Merge branch 'master' into minor-changes-3 2021-06-15 18:07:24 +00:00
Nikolai Kochetov
80a13c489b Revert back moduloLegacy check for canConstantBeWrappedByFunctions. 2021-06-15 18:21:31 +03:00
Nikolai Kochetov
481b87b37a Remove some code from keyCondition. 2021-06-15 16:47:37 +03:00
Nikita Mikhaylov
a52bba91b7
Merge pull request #16401 from abyss7/ast-table-identifier-2 
ASTTableIdentifier part #2: Introduce ASTTableIdentifier
 2021-06-15 13:51:30 +03:00
tavplubix
e2ecc51a1f
Merge pull request #25087 from ClickHouse/srmt_remove_copypaste 
Remove copypaste from StorageReplicatedMergeTree
 2021-06-15 12:43:55 +03:00
Alexey Milovidov
4e982a3ae6 Merge branch 'master' into minor-changes-3 2021-06-15 00:45:45 +03:00
Alexey Milovidov
f49463d1e5 Fix clang-tidy 2021-06-14 07:28:17 +03:00
Alexey Milovidov
447d7bb8cd Minor changes 2021-06-14 07:13:35 +03:00
Alexey Milovidov
2211ba53d4 Copy paste some code 2021-06-14 05:48:10 +03:00
mergify[bot]
ab2a1a410f
Merge branch 'master' into test-key-condition-mon 2021-06-12 00:32:38 +00:00
alexey-milovidov
30f0124e77
Merge branch 'master' into always-detach-parts-with-wrong-partition-id 2021-06-12 02:55:55 +03:00
alexey-milovidov
eb7b491da9
Merge pull request #24900 from Enmk/bloom_filter_hasAny 
Added support of hasAny function to bloom_filter index.
 2021-06-12 00:58:25 +03:00
Vasily Nemkov
2ac288c303 index is not used if types in hasAny are unsuported 
updated tests
 2021-06-11 18:29:52 +03:00
Alexander Tokmakov
73ff1728ae rename flag to more generic name 2021-06-11 15:41:48 +03:00
Nikolai Kochetov
79c69ea316 Fix build. 2021-06-11 11:29:39 +03:00
Vasily Nemkov
55064e49a7 Fixed the test 
Previous test version wasn't causing bug on unfixed code, this new one does.
 2021-06-10 19:09:43 +03:00
Nikolai Kochetov
fea19c2037
Merge pull request #25163 from amosbird/projection-fix10 
Disable projection when final is used in select query
 2021-06-10 18:44:29 +03:00
Vasily Nemkov
0ab28360c9 Added support of hasAny function to bloom_filter index. 
Also fixed bug discovered by AST fuzzer.
 2021-06-10 18:22:04 +03:00
Ivan
a7fa4e641c
Merge branch 'master' into ast-table-identifier-2 2021-06-10 16:43:22 +03:00
Nikolai Kochetov
6222fcffe0 Never remove parts on start at all. Always move. 2021-06-10 15:47:55 +03:00
Nikolai Kochetov
aa90309abf Never remove parts on start at all. Always move. 2021-06-10 15:46:03 +03:00
Nikolai Kochetov
2936fdd16c Always detach parts with wrong partition id. 2021-06-10 15:04:36 +03:00
Nikolai Kochetov
847dfb9914 Merge remote-tracking branch 'origin' into always-detach-parts-with-wrong-partition-id 2021-06-10 12:37:46 +03:00
Nikolai Kochetov
d52b87d47d Always detach proken parts with wrong partition id. 2021-06-10 12:37:33 +03:00
Amos Bird
4e358b28f1
Disable projection with final is used 2021-06-10 17:11:17 +08:00
Alexey Milovidov
6c5b2b827c Move code to more appropriate place 2021-06-10 02:58:51 +03:00
alexey-milovidov
0636f40349
Merge pull request #25127 from ClickHouse/fix-partition-hash-for-uuid 
Revert old UUID hash used for partition ID.
 2021-06-10 02:31:01 +03:00
Nikolai Kochetov
12791431da Revert old UUID hash used for partition ID. 2021-06-09 16:51:40 +03:00
Alexander Tokmakov
cef22688ff make code less ugly 2021-06-09 15:36:47 +03:00
Alexander Tokmakov
3ade38df82 remove copypaste 2021-06-08 22:17:45 +03:00
alexey-milovidov
e25d9843d1
Merge pull request #25025 from azat/parallel-kill-mutation 
Distinguish KILL MUTATION for different tables.
 2021-06-08 21:40:53 +03:00
Nikolai Kochetov
6197d20c18
Update KeyCondition.cpp 2021-06-08 15:48:14 +03:00
Azat Khuzhin
1062d0ec91 Distinguish KILL MUTATION for different tables. 
Before this patch KILL MUTATION marks mutation as canceled just by name
(and part numbers) so if you have multiple tables with the same part
name, then killing mutation for one table, will mark it as killed for
another too.

Fix this by comparing StorageID too (it is better to use StorageID over
database/table to avoid ambiguity by using UUIDs for comparing).

Here is a failure of the 01414_freeze_does_not_prevent_alters on CI [1].

  [1]: https://clickhouse-test-reports.s3.yandex.net/24069/9fb69dcf98c71a939d200cad3c8491bf43a44622/functional_stateless_tests_(ubsan).html#fail1
 2021-06-08 10:51:22 +03:00
Nikolai Kochetov
c4832fd3c0 Added test. 2021-06-07 21:24:32 +03:00
Ivan
579250168f
Merge branch 'master' into ast-table-identifier-2 2021-06-07 20:30:39 +03:00
Latysheva Alexandra
1a2cbe9275
Merge branch 'master' into alexelex-encryption-1 2021-06-07 21:18:54 +06:00
Amos Bird
840895509e
Fix using aggregate projection when group by without key 2021-06-07 22:44:42 +08:00
Nikolai Kochetov
0d2a839ca4 Fix tests. 2021-06-07 16:41:40 +03:00
alesapin
170c49db69
Merge pull request #24961 from ClickHouse/more_general_check_in_queue 
Trying to fix 'Tagging already tagged part'
 2021-06-07 11:52:05 +03:00
Alexey Milovidov
17962459f5 Merge branch 'master' into issue-16775 2021-06-06 02:18:28 +03:00
Nikolai Kochetov
397f6133e0 Refactor canConstantBeWrappedByMonotonicFunctions function. 2021-06-04 20:56:56 +03:00
alesapin
1e69128443 Trying to fix 'Tagging already tagged part' 2021-06-04 14:49:00 +03:00
Kruglov Pavel
8acf031c1b
Merge pull request #24398 from Avogar/fix-deltasum 
Fix logical error AggregateFunctionFactory returned nullptr
 2021-06-03 19:40:06 +03:00
Nikolai Kochetov
3e5a1cda60 Revert 2021-06-03 17:44:59 +03:00
Nikolai Kochetov
726e22ea1d Always return false for canConstantBeWrappedByMonotonicFunctions. 2021-06-03 16:26:04 +03:00
tavplubix
1aff716f18
Merge pull request #24777 from ClickHouse/fix_intersection_with_lost_part 
Do not crash on intersecting virtual parts
 2021-06-03 15:42:46 +03:00
alesapin
282e8b52c4
Merge pull request #24872 from ClickHouse/fix_background_pool 
Paranoid fix for background pool task decrement
 2021-06-03 11:43:58 +03:00
alesapin
4a1bf51ad1
Merge branch 'master' into limited_fetches 2021-06-02 23:01:47 +03:00
Ivan Lezhankin
ae38b4a3f1 Merge remote-tracking branch 'upstream/master' into ast-table-identifier-2 2021-06-02 16:07:59 +03:00
Nikolai Kochetov
134ad18249
Merge pull request #24574 from ClickHouse/refactor-MergeTreeDataSelectExecutor 
Refactor MergeTreeDataSelectExecutor
 2021-06-02 13:17:54 +03:00
alesapin
68d251ec11 Paranoid fix for background pool task decrement 2021-06-02 12:40:03 +03:00
alesapin
1c0973162d
Merge pull request #24803 from ClickHouse/followupfix_zookeeper_timeouts 
Don't capture temporary references in ZooKeeper client callbacks.
 2021-06-02 01:08:17 +03:00
tavplubix
a4ea5783d5
Update ActiveDataPartSet.cpp 2021-06-01 20:03:52 +03:00
Alexander Tokmakov
7d7c5638a5 Merge branch 'master' into fix_intersection_with_lost_part 2021-06-01 16:32:59 +03:00
tavplubix
275a616bb6
Update ActiveDataPartSet.cpp 2021-06-01 16:27:09 +03:00
Alexander Tokmakov
5969891611 do not crash on intersecting parts 2021-06-01 16:25:23 +03:00
Nikolai Kochetov
e615858b10 Merge branch 'master' into refactor-MergeTreeDataSelectExecutor 2021-06-01 16:25:04 +03:00
Alexander Tokmakov
cdd46aa117 Revert "try fix intersecting virtual parts" 
This reverts commit 2571ad7d43.
 2021-06-01 14:52:25 +03:00
Ivan Lezhankin
365e52817b More fixes due to "in" function arguments being incorrectly checked as ASTIdentifier 2021-06-01 14:20:03 +03:00
alexey-milovidov
c7bd3b8674
Merge pull request #23315 from CurtizJ/empty-parts-fix 
Fix waiting while dropping empty parts
 2021-06-01 13:52:49 +03:00
Nikolai Kochetov
b54ff696e5 Merge branch 'master' into refactor-MergeTreeDataSelectExecutor 2021-06-01 12:08:37 +03:00
Nikolai Kochetov
d4cbce3761
Merge pull request #24782 from amosbird/projection-fix7 
Fix empty part set with force_use_projection = 1
 2021-06-01 11:36:50 +03:00
alesapin
4e038b2a61 Remove copy-paste and rename (review fixes) 2021-06-01 10:30:06 +03:00
tavplubix
e9ff0b6d70
Merge pull request #23657 from kssenii/poco-file-to-std-fs 
Poco::File to std::filesystem
 2021-05-31 23:17:02 +03:00
Nikolai Kochetov
b9aa404508 Review fixes. 2021-05-31 21:25:43 +03:00
Nikolai Kochetov
afc1fe7f3d Make ContextPtr const by default. 2021-05-31 17:49:02 +03:00
alesapin
2fe75e33ee Replace Poco::Event with std future in synchronous calls in ZooKeeper client 2021-05-31 16:03:30 +03:00
Amos Bird
a206b76e5b
Fix empty part set with force_use_projection = 1 2021-05-31 10:31:45 +08:00
Alexander Tokmakov
2571ad7d43 try fix intersecting virtual parts 2021-05-31 00:30:50 +03:00
Alexander Tokmakov
16647fe8ce some unrelated fixes 2021-05-31 00:29:37 +03:00
kssenii
2a631aaf08 Final fixes 2021-05-29 00:34:44 +03:00
Nikolai Kochetov
58fbc544cc Add more comments. 2021-05-28 20:16:09 +03:00
Nikolai Kochetov
295a302bc8 Remove settings from ReadFromMergeTree. 2021-05-28 17:34:02 +03:00
Pavel Kruglov
42d57138f2 replace all isDateOrDateTime invocations to explicit calls to isDate and isDateTime and isDateTime64 2021-05-28 15:34:49 +03:00
Nikolai Kochetov
91e8c2c75d Remove MergingFinal step. 2021-05-28 12:52:33 +03:00
Nikolai Kochetov
94f1ac5a16 Remove some commented code. 2021-05-28 12:41:07 +03:00
Nikolai Kochetov
c80ab6166a Fix style. 2021-05-28 12:26:44 +03:00
Nikolai Kochetov
4e28b7cb02 Fix _sample_factor column. 2021-05-28 12:23:46 +03:00
Anton Popov
179f172c4c fix build 2021-05-28 02:10:44 +03:00
Anton Popov
336abfa475 Merge remote-tracking branch 'upstream/master' into HEAD 2021-05-28 01:58:30 +03:00
Nikolai Kochetov
936bd8a011 Merge branch 'master' into refactor-MergeTreeDataSelectExecutor 2021-05-27 20:47:39 +03:00
Nikolai Kochetov
1aeb705b20 Fix some tests. 2021-05-27 19:53:58 +03:00
Nikolai Kochetov
cbdf3752ef Part 3. 2021-05-27 16:40:33 +03:00
kssenii
a0d8d689d3 Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-27 15:44:12 +03:00
alesapin
6924dd51a2 Merge branch 'master' into limited_fetches 2021-05-27 11:17:36 +03:00
alesapin
ef17daaaa0
Merge pull request #24496 from ClickHouse/configurable_max_parts_to_merge 
Add `max_parts_to_merge_at_once` setting
 2021-05-27 10:56:58 +03:00
kssenii
b28157a0df Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-27 00:01:25 +03:00
alesapin
42b2383e01 Basic throttler implementation 2021-05-26 23:37:44 +03:00
kssenii
73f16ee9ee Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-26 23:08:08 +03:00
Nikolai Kochetov
a51a6ea0b7 Part 2. 2021-05-26 21:14:43 +03:00
alesapin
8ba6ed3fc2 Simplier test 2021-05-26 16:53:05 +03:00
tavplubix
a21e4b333e
Update ReplicatedMergeTreeQueue.cpp 2021-05-26 15:10:38 +03:00
alesapin
08c37a4c9e Merge branch 'master' into configurable_max_parts_to_merge 2021-05-26 11:30:39 +03:00
alesapin
069d45790f
Merge pull request #24157 from kssenii/modulo-partition-key 
Adjust modulo function when used in partition key
 2021-05-26 10:30:54 +03:00
alesapin
5638c18140
Merge pull request #22864 from ianton-ru/s3_zero_copy_replication 
Fix Zero-Copy replication with several S3 volumes (issue 22679)
 2021-05-26 10:28:23 +03:00
kssenii
940eab6924 Review fixes 2021-05-26 09:26:19 +03:00
Alexey Milovidov
20f59ffbb9 Merge branch 'master' into issue-16775 2021-05-26 03:46:43 +03:00
tavplubix
84f9d38620
Merge pull request #24321 from amosbird/fixdroppartition 
Fix drop partition in rare case
 2021-05-26 01:34:22 +03:00
Nikolai Kochetov
34eaa48294 Part 1. 2021-05-25 19:34:43 +03:00
alesapin
6e29404a75 Add max_parts_to_merge_at_once setting 2021-05-25 16:21:17 +03:00
Alexander Tokmakov
4849ea2e03 fix unexpected behavior in mutation finalization 2021-05-25 14:10:45 +03:00
kssenii
70469429c1 Fixes 2021-05-24 23:39:56 +00:00
kssenii
c3fc2ff04b Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into modulo-partition-key 2021-05-24 22:08:49 +00:00
kssenii
31107816bb fs::absolute to fs::canonical 2021-05-24 20:37:54 +03:00
Anton Ivashkin
c936dab6d5 Fix part type in Fetcher::downloadPartToS3 2021-05-24 19:37:37 +03:00
Anton Ivashkin
a98286b4c4 Merge master 2021-05-24 17:46:29 +03:00
Alexey Milovidov
d32819f068 Mark false positives for PVS-Studio 2021-05-24 06:59:12 +03:00
Alexey Milovidov
273226de32 Remove string parameter for Density 2021-05-24 06:43:25 +03:00
Alexey Milovidov
6f70feed2f Merge branch 'master' into issue-16775 2021-05-24 05:56:57 +03:00
Alexey Milovidov
ff7b5d34bb Mark false positives for PVS-Studio 2021-05-24 02:56:03 +03:00
kssenii
659e57e60f Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into modulo-partition-key 2021-05-23 22:46:00 +00:00
kssenii
1ffaf1d793 Better check 2021-05-23 21:53:32 +00:00
kssenii
bbf56e35ac Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-23 10:54:48 +03:00
alesapin
6c0b683d3a
Merge pull request #17871 from nvartolomei/nv/parts-uuid-move-shard 
Part movement between shards
 2021-05-23 10:20:57 +03:00
mergify[bot]
3c6e6464a7
Merge branch 'master' into s3_zero_copy_replication 2021-05-23 07:09:17 +00:00
Alexey Milovidov
40d4f0678f Remove overload (harmful) 2021-05-23 04:25:06 +03:00
kssenii
1c43b333ea Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-22 21:26:48 +03:00
kssenii
5606e248a7 Fix 2021-05-22 16:19:56 +00:00
kssenii
b72f6ea78e Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into modulo-partition-key 2021-05-22 16:15:15 +00:00
alexey-milovidov
a25f1910cc
Merge pull request #24320 from ClickHouse/remove-auto-array 
Remove AutoArray
 2021-05-22 02:19:59 +03:00
kssenii
fcfec83875 Fix comparisons with modulo key (version 2) 2021-05-21 16:40:47 +00:00
kssenii
94b5763dbd Revert "Different approach to fix modulo" 
This reverts commit 665f1aa0f6.
 2021-05-21 16:14:01 +00:00
kssenii
30845a383f Fix comparisons with modulo key 2021-05-21 15:01:41 +00:00
kssenii
665f1aa0f6 Different approach to fix modulo 2021-05-21 14:30:08 +00:00
Anton Ivashkin
29336a4a34 Use keep_s3_on_delet flag instead of DeleteOnDestroyKeepS3 state 2021-05-21 15:29:10 +03:00
kssenii
bd0bc8e3e7 Revert "Adjust minMax index" 
This reverts commit c69f587f8a.
 2021-05-21 12:01:08 +00:00
alesapin
20a34e951a Split fetch part into two stages, don't use replicated block output stream 2021-05-21 12:30:49 +03:00
Alexey Milovidov
ea30bb7abf Improve implementation 2021-05-21 04:17:18 +03:00
alesapin
deb784a321 Merge branch 'master' into nvartolomei-parts-move 2021-05-20 21:24:55 +03:00
kssenii
3dee003f9b Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-20 19:20:09 +03:00
kssenii
c69f587f8a Adjust minMax index 2021-05-20 15:32:18 +00:00
Nikolai Kochetov
818d081292
Merge pull request #24296 from amosbird/projection-fix5 
Skip data finalization when doing projection materialization.
 2021-05-20 10:50:42 +03:00
Alexey Milovidov
1006a970f7 Remove AutoArray 2021-05-20 09:30:13 +03:00
kssenii
56b388addf Add check for nullptr 2021-05-19 19:32:04 +00:00
kssenii
477214a57f Fix partitionPrunner with modulo 2021-05-19 17:36:32 +00:00
Amos Bird
c68ef31065
Skip data finalization when doing projection materialization. 2021-05-20 00:28:50 +08:00
mergify[bot]
230bec70fb
Merge branch 'master' into modulo-partition-key 2021-05-19 10:08:41 +00:00
alesapin
4d6c71801c
Merge branch 'master' into nv/parts-uuid-move-shard 2021-05-19 13:03:14 +03:00
Alexandra
3f00f064af wide parts validation with file opening via disk 2021-05-18 20:06:09 +00:00
tavplubix
8b13e73ac2
Merge pull request #24208 from ClickHouse/fix_intersecting_parts_again 
Addition to #23997
 2021-05-18 21:13:25 +03:00
Kseniia Sumarokova
0ab15d0bc4
Merge branch 'master' into modulo-partition-key 2021-05-18 17:45:32 +03:00
Anton Ivashkin
d256cf4edf Fix PVS check (Two or more case-branches perform the same actions) 2021-05-18 17:41:09 +03:00
alesapin
c1c6a0d8e3
Merge pull request #24248 from ucasFL/remove 
remove useless code
 2021-05-18 16:09:00 +03:00
feng lv
403d0a305b remove useless code 2021-05-18 12:19:04 +00:00
kssenii
5f083b779d A little better, fix checks 2021-05-18 14:25:19 +03:00
alesapin
36aeb59705 Fix lazy test 2021-05-18 11:03:38 +03:00
alesapin
22cc1c953f Revert "PVS Studio fixes" 
This reverts commit 87c6e39a29.
 2021-05-18 10:34:16 +03:00
Alexander Tokmakov
20b9af7b26 do not allow concurrent alter partition 2021-05-18 00:20:58 +03:00
alesapin
87c6e39a29 PVS Studio fixes 2021-05-17 23:23:32 +03:00
kssenii
9b8df78fdd Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-17 17:42:05 +03:00
kssenii
8c955680ad Add comments and remove adjusted deserializarion 2021-05-17 14:33:04 +00:00
Anton Popov
e27acc26be better dropPart and dropPartition 2021-05-17 17:26:36 +03:00
Anton Popov
e043cc45bf Merge remote-tracking branch 'upstream/master' into HEAD 2021-05-17 16:11:12 +03:00
Anton Ivashkin
8ed4a5de62 Fix Zero Copy after merge master 2021-05-17 16:01:08 +03:00
alesapin
5563a0d7f3 Fix style 2021-05-17 14:44:47 +03:00
alesapin
023b592f97 Fix style 2021-05-17 14:42:14 +03:00
alesapin
d8994350c1 Useful comments 2021-05-17 14:39:37 +03:00
Ilya Yatsishin
8ede5c736a
Merge pull request #23992 from MaxWk/enable-datatime64-version-column 
enable DateTime64 to be version column
 2021-05-17 14:34:15 +03:00
alesapin
77b1aeaac2 Use same zookeeper object to have an exception if something went wrong 2021-05-17 14:30:23 +03:00
alesapin
67d34c0136 merge with master 2021-05-17 14:13:18 +03:00
alesapin
17f229857c Merge branch 'master' into nvartolomei-parts-move 2021-05-17 13:52:48 +03:00
Anton Ivashkin
39a30b77fe Merge master 2021-05-17 11:47:48 +03:00
alesapin
eae2df6478
Merge pull request #23997 from ClickHouse/fix_intersecting_parts 
Try fix intersecting parts
 2021-05-17 10:46:58 +03:00
Nikolai Kochetov
98e6c27a95
Merge pull request #24168 from amosbird/projection-fix3 
Fix distributed processing when using projection
 2021-05-17 09:45:39 +03:00
Maksim Kita
20cced0a4c
Merge pull request #24155 from kitaisreal/function-refactoring 
IFunction refactoring
 2021-05-17 09:26:57 +03:00
kssenii
3b1bf2bae6 Poco::Path substitution 2021-05-16 23:38:23 +03:00
kssenii
807684812e Fix clang tidy 2021-05-16 18:18:11 +00:00
kssenii
d44f1680da Use unsigned type for (de)serialization 2021-05-16 18:09:55 +00:00
Amos Bird
07b1be5a76
Fix distributed processing when using projection 2021-05-16 22:40:06 +08:00
alesapin
29726b5510
Merge pull request #24122 from ClickHouse/fix_table_startup 
Set readonly mode if restarting thread failed to activate replica
 2021-05-16 16:45:51 +03:00
kssenii
22b6d0eb25 Adjust modulo recursively 2021-05-16 09:20:59 +00:00
Nikolai Kochetov
093b7e120a
Merge pull request #24062 from amosbird/projection-fix1 
Fix empty key projection query analysis
 2021-05-16 07:37:03 +03:00
kssenii
add5970b5f fs::absolute 2021-05-16 03:12:13 +03:00
kssenii
c7f51fe60e Modulo adjustment for partition key 2021-05-15 19:51:02 +00:00
Maksim Kita
947f28d430 IFunction refactoring 2021-05-15 20:33:15 +03:00
Alexander Tokmakov
df5f3fbc9d review suggestions 2021-05-14 19:11:40 +03:00
Amos Bird
a113acc40c
Fix empty key 2021-05-14 22:54:14 +08:00
Amos Bird
8de016d33a
Fix empty key projection query analysis 2021-05-14 22:26:26 +08:00
Alexander Tokmakov
41776ba5d9 fix 2021-05-14 15:55:30 +03:00
Alexander Tokmakov
970ad94851 Merge branch 'master' into fix_intersecting_parts 2021-05-14 13:17:02 +03:00
alesapin
d4c6a5a05e Better logging 2021-05-14 11:38:53 +03:00
alesapin
67e4393769 If table was not active set readonly mode 2021-05-14 11:32:41 +03:00
kssenii
f33a517d33 Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-14 02:58:26 +03:00
Alexander Tokmakov
2da12f0b59 fix intersection of drop range and merge 2021-05-13 17:04:36 +03:00
Alexander Tokmakov
e114c7eb8b fix virtual parts in REPLACE_RANGE 2021-05-13 14:29:59 +03:00
feng lv
c6f8ab9826 fix 2021-05-13 02:05:53 +00:00
Nikolai Kochetov
690410009c Restor is_temporary flag for fetched part. 2021-05-12 19:17:01 +03:00
kssenii
0527f0ea33 Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-12 16:54:18 +03:00
feng lv
9035e3a643 fix test and conflict 2021-05-11 16:22:24 +00:00
Nikolai Kochetov
817bc1377c Fix part check for projections. 2021-05-11 17:07:19 +03:00
kssenii
393b026f2e Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-11 16:49:23 +03:00
alesapin
46e136b5c4
Merge branch 'master' into nv/parts-uuid-move-shard 2021-05-11 15:36:40 +03:00
Nikolai Kochetov
a7d717d767 Remove unused code. 2021-05-11 14:59:56 +03:00
Nikolai Kochetov
89f16014b8 fix fetch for projection. 2021-05-11 14:44:59 +03:00
Amos Bird
074c2ead9c
Fix 2021-05-11 18:22:23 +08:00
Amos Bird
5e9fab6d10
Comment debug info out 2021-05-11 18:12:29 +08:00
Amos Bird
5836e2e277
Fix issues in normal projection 2021-05-11 18:12:29 +08:00
Amos Bird
51d685bb9c
Even more fixes 2021-05-11 18:12:29 +08:00
Amos Bird
ddd4256a15
More fixes 2021-05-11 18:12:28 +08:00
Amos Bird
718c284437
Fix more tests 2021-05-11 18:12:28 +08:00
Amos Bird
ba17acbd63
Fix tests 2021-05-11 18:12:28 +08:00
Amos Bird
a3fd0b6f2e
Fix tests 2021-05-11 18:12:28 +08:00
Amos Bird
42f161b1dc
Fix more 2021-05-11 18:12:28 +08:00
Amos Bird
ef918aeaff
Rebase and fix 2021-05-11 18:12:28 +08:00
Amos Bird
9c069ebdbf
support prewhere, row_filter, read_in_order and decent projection selection 
TODO set index analysis in projection
 2021-05-11 18:12:27 +08:00
Amos Bird
f7f949c1f9
Fix aggregation keys order 2021-05-11 18:12:27 +08:00
Amos Bird
e1e560765c
Fix missing columns 2021-05-11 18:12:27 +08:00
Amos Bird
35961c0c5d
Use ActionsDAG to rewrite projection queries 2021-05-11 18:12:27 +08:00
Nikolai Kochetov
29d7038e30
Add ProjectionDescription::Type enum. Remove some checks. 2021-05-11 18:12:27 +08:00
Amos Bird
ebaf42a448
Reformat and fix some tests 2021-05-11 18:12:27 +08:00
Nikolai Kochetov
672cfedd13
Disable normal projection by the number of granules. 2021-05-11 18:12:26 +08:00
Nikolai Kochetov
95431168c2
Try to enable normal projections. 2021-05-11 18:12:26 +08:00
Amos Bird
483bd165e7
Check if pipeline is simple and add more comments 2021-05-11 18:12:26 +08:00
Nikolai Kochetov
427f444ba4
Fix prewhere. 2021-05-11 18:12:26 +08:00
Nikolai Kochetov
3d36fb57eb
Cleanup code. 2021-05-11 18:12:26 +08:00
Nikolai Kochetov
3296c9292f
Try to merge projectons faster. 2021-05-11 18:12:26 +08:00
Amos Bird
264cff6415
Projections 
TODO (suggested by Nikolai)

1. Build query plan fro current query (inside storage::read) up to WithMergableState
2. Check, that plan is simple enough: Aggregating - Expression - Filter - ReadFromStorage (or simplier)
3. Check, that filter is the same as filter in projection, and also expression calculates the same aggregation keys as in projection
4. Return WithMergableState if projection applies

3 will be easier to do with ActionsDAG, cause it sees all functions, and dependencies are direct (but it is possible with ExpressionActions also)

Also need to figure out how prewhere works for projections, and
row_filter_policies.

wip
 2021-05-11 18:12:23 +08:00
Kruglov Pavel
49e7ed6e60
Merge pull request #23673 from amosbird/partitionvalue 
Add _partition_value virtual column
 2021-05-11 11:23:00 +03:00
Alexander Tokmakov
eef899ae63 better code, add test 2021-05-10 21:03:37 +03:00
万康
308b964ca4 enable DateTime64 to be version column 2021-05-10 20:30:52 +08:00
kssenii
ab1a05a1f4 Poco::Path to fs::path, less concatination 2021-05-09 14:59:49 +03:00
Alexey Milovidov
e23a0d8f7e Fix warnings by PVS-Studio 2021-05-08 23:57:08 +03:00
Alexey Milovidov
513c6891db Fix warnings by PVS-Studio 2021-05-08 23:54:49 +03:00
Alexey Milovidov
ad88819ee4 Fix a bunch of warnings from PVS-Studio 2021-05-08 19:13:10 +03:00
Alexey Milovidov
e905883c75 More fixes for PVS-Studio 2021-05-08 19:12:31 +03:00
Alexey Milovidov
8d62c42eb9 A bunch of changes for PVS-Studio 2021-05-08 19:11:50 +03:00
Alexey Milovidov
604daa9581 Fix a bunch of warnings from PVS-Studio 2021-05-08 19:09:17 +03:00
Alexey Milovidov
2fbecf36ca More fixes for PVS-Studio 2021-05-08 18:20:40 +03:00
kssenii
266567e04c Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-08 17:49:05 +03:00
Alexey Milovidov
9e1a5f41fc A bunch of changes for PVS-Studio 2021-05-08 17:43:03 +03:00
Alexey Milovidov
47a4c101b9 Merge branch 'master' into normalize-bigint 2021-05-08 16:41:15 +03:00
tavplubix
f8c7725ef5
Merge pull request #23742 from ClickHouse/update_metadata_on_replica_recovery 
Update metadata on replica recovery
 2021-05-08 16:34:03 +03:00
kssenii
02288359c5 Less manual concatenation of paths 2021-05-08 13:59:55 +03:00
kssenii
35f999bf04 Poco::createFile to fs::createFile 2021-05-08 02:41:47 +03:00
Alexander Tokmakov
24bcd9f7c6 better comments, fix pvs warnings 2021-05-07 16:39:21 +03:00
kssenii
140bf7e2ea Less Poco 2021-05-07 15:29:26 +03:00
kssenii
9ec92ec514 Fix tests, less manual concatination of paths 2021-05-05 18:39:30 +03:00
Alexey Milovidov
02ce9cc725 Untangle UUID 2021-05-04 01:46:51 +03:00
Alexey Milovidov
f4100cf70c Remove harmful std::regex 2021-05-02 22:04:26 +03:00
Alexey Milovidov
1fda866881 Merge branch 'master' into normalize-bigint 2021-05-02 21:24:45 +03:00
feng lv
39f68bf5ff fix conflict 2021-05-02 16:33:45 +00:00
kssenii
ee06936596 Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-01 17:24:31 +03:00
Maksim Kita
14e879cef9
Merge pull request #23781 from ucasFL/fix-clear-column 
Fix CLEAR COLUMN does not work after #21303
 2021-04-30 17:28:59 +03:00
Nikolai Kochetov
4bb56849b3
Merge pull request #23763 from amosbird/rowfilterfix 
Fix crash when prewhere and row policy filter are both in effect with empty result
 2021-04-30 16:25:36 +03:00
feng lv
aed2f337e9 Fix CLEAR COLUMN does not work after #21303 2021-04-30 05:02:32 +00:00
Amos Bird
70ec13f3d3
Fix empty block with row_filter 2021-04-29 19:40:14 +08:00