thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 19:02:04 +00:00
Code Issues Packages Projects Releases Wiki Activity
58,108 Commits 9,326 Branches 1,783 Tags 2.1 GiB
a26c9e64a9
Commit Graph

1434 Commits

Author SHA1 Message Date
alesapin
54455b4740 Add test for already working code 2020-12-23 14:53:49 +03:00
Anton Popov
518cac6150 fix for review 2020-12-23 13:11:24 +03:00
Alexander Kuzmenkov
51b2329295
Merge branch 'master' into aku/window-prototype 2020-12-23 09:26:11 +03:00
Amos Bird
8b5714b2ac
Fix 2-arg functions with constant in PK analysis 2020-12-23 12:29:29 +08:00
Nikita Mikhailov
dcfbe782c6 Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel-parsing-input-format 2020-12-23 05:20:22 +03:00
roverxu
e38ecee30e fix style 2020-12-23 09:20:45 +08:00
Anton Popov
78c4b9f2bc do not allow merge from wide to compact parts 2020-12-23 03:16:15 +03:00
Anton Popov
5101804e30
Merge pull request #18366 from CurtizJ/remove-unused-code 
Remove unused code
 2020-12-23 02:45:33 +03:00
Anton Popov
1be39fddac fix subcolumns with some storages 2020-12-22 19:42:37 +03:00
Anton Popov
b6ff6300b2 Merge remote-tracking branch 'upstream/master' into HEAD 2020-12-22 18:06:21 +03:00
Anton Popov
40b5ffe3b5 minor renames 2020-12-22 18:06:12 +03:00
Anton Popov
371050a16e remove unused code 2020-12-22 15:22:54 +03:00
徐炘
81f9623e2e
Merge branch 'master' into dev/add_metrics_for_parts 2020-12-22 19:04:28 +08:00
alesapin
35413635e7 Add ability to change some types for primary key 2020-12-22 13:41:12 +03:00
flynn
4e580f7b7c
Add asynchronous metrics on MergeTree tables data volume (#17639) 
Co-authored-by: Alexander Kazakov <Akazz@users.noreply.github.com>
 2020-12-22 13:34:35 +03:00
root
d6a69ee164 remove code 2020-12-22 18:12:28 +08:00
Amos Bird
7a81e0a0f5
Extend mergetree setting alter restriction 2020-12-21 18:45:39 +08:00
roverxu
f3fef24d30 fix deadlock 2020-12-21 16:30:27 +08:00
alesapin
07b5c03c54 Better adjustment of the last mark 2020-12-21 11:24:52 +03:00
Alexander Kuzmenkov
85c1bc1253 Merge remote-tracking branch 'origin/master' into tmp 2020-12-21 10:46:21 +03:00
roverxu
8e72b6d31b Merge remote-tracking branch 'origin' into dev/add_metrics_for_parts 2020-12-21 09:25:54 +08:00
alesapin
23eebb3f90 Merge branch 'master' into delayed_last_mark_for_wide_parts 2020-12-20 23:45:44 +03:00
alesapin
0041c293ab Fix gcc-10 build 2020-12-20 11:01:39 +03:00
Alexey Milovidov
9be5fa9ef2 Merge branch 'master' into Enmk-Optimize_deduplicate 2020-12-20 09:57:10 +03:00
alesapin
ca6708e594 Fix check and granule adjust for non adaptive parts 2020-12-19 21:59:20 +03:00
Vasily Nemkov
e5ec81f7cd Single quotes around column names 2020-12-19 20:58:23 +02:00
alesapin
b2d53d802d Fix for non-adaptive granularity 2020-12-18 23:32:52 +03:00
Anton Popov
25ddd78976 fix nested 2020-12-18 23:09:34 +03:00
Nikolai Kochetov
d7fc426458
Merge pull request #18216 from ClickHouse/fix-18137 
Fix 18137
 2020-12-18 18:49:02 +03:00
alesapin
cd598034cd Fix add tests 2020-12-18 17:44:31 +03:00
alesapin
c1e7d5d3c1 Working code 2020-12-18 16:49:45 +03:00
Anton Popov
009efeff40 Merge remote-tracking branch 'upstream/master' into HEAD 2020-12-18 16:14:09 +03:00
Anton Popov
31e6956ac8 fix alters of nested 2020-12-18 15:29:57 +03:00
Vasily Nemkov
e166aae3f9 Using CSV-like strings for list of columns to deduplicate by instead of JSON-like notation. 2020-12-18 13:44:56 +02:00
alesapin
26637bd7ee
Merge pull request #17943 from ClickHouse/try_rewrite_writer 
Small simplification of MergeTreeDataWriter
 2020-12-18 12:41:16 +03:00
Nikolai Kochetov
eef58c85f0 Put thread back to set if it had stolen all tasks of backoff thread. 2020-12-18 11:33:55 +03:00
Alexander Kuzmenkov
5e19eaf2f0 Merge remote-tracking branch 'origin/master' into tmp 2020-12-18 03:49:59 +03:00
Nikita Mikhailov
6a55e018e4 Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel-parsing-input-format 2020-12-18 02:25:32 +03:00
alexey-milovidov
6fc3ca8b7b
Merge pull request #16724 from kitaisreal/added-function-accurate-cast-or-null 
Added function accurateCastOrNull, allow different types inside IN subquery
 2020-12-18 00:45:27 +03:00
alesapin
8140fbd0fb
Merge pull request #18200 from ClickHouse/more_logs_during_quorum 
More logs during quorum insert
 2020-12-18 00:22:19 +03:00
alexey-milovidov
dd78c9dc75
Merge pull request #17566 from ClickHouse/relax-too-many-parts-1 
Relax "Too many parts" threshold, part 1
 2020-12-17 20:44:44 +03:00
Anton Popov
a42b00c9aa Merge remote-tracking branch 'upstream/master' into HEAD 2020-12-17 20:43:23 +03:00
Anton Popov
6de8b05b7c improvements in subcolumns reading 2020-12-17 20:10:33 +03:00
alesapin
aecdd02be4 Better message 2020-12-17 19:14:42 +03:00
alesapin
f45993cd1e More logs during quorum insert 2020-12-17 19:13:01 +03:00
alesapin
911a1cd48f Suppress gdb warning 2020-12-17 13:25:31 +03:00
alesapin
b74e7accf1 Review fixes 2020-12-17 11:17:39 +03:00
alesapin
f2957d6f05 Merge branch 'master' into try_rewrite_writer 2020-12-17 10:40:25 +03:00
Alexey Milovidov
b25f126569 Added a test 2020-12-17 04:52:27 +03:00
Alexey Milovidov
9a017b77df Merge branch 'master' into relax-too-many-parts-1 2020-12-17 04:40:39 +03:00
Alexander Kuzmenkov
a1326414ff Merge remote-tracking branch 'origin/master' into tmp 2020-12-17 00:46:14 +03:00
Alexander Kuzmenkov
fef24c81e3 some grammar 2020-12-17 00:44:05 +03:00
alesapin
849db47f8a Better exception messages 2020-12-16 17:31:17 +03:00
roverxu
1065ddef65 fix bug when drop data 2020-12-16 22:02:34 +08:00
Alexey Milovidov
37363ae89d Comment update 2020-12-16 09:43:16 +03:00
Alexey Milovidov
4bd5d3b662 Comment update 2020-12-16 06:36:43 +03:00
Alexey Milovidov
88c5031dfa Add a commit #16595 2020-12-16 06:27:28 +03:00
alesapin
1bd2f49fb5 Merge branch 'master' into try_rewrite_writer 2020-12-15 19:05:04 +03:00
Nikolai Kochetov
6defcbb662
Merge branch 'master' into optimize-data-on-insert 2020-12-15 16:50:42 +03:00
alesapin
8670836573 Add some comments 2020-12-15 13:34:28 +03:00
alesapin
74c2211510 More clear 2020-12-15 12:54:48 +03:00
alesapin
a50187c995 Slightly better 2020-12-15 11:04:37 +03:00
Nikita Mikhaylov
5a47928431 save 2020-12-15 00:56:47 +03:00
Nikita Mikhaylov
f40f3ced2a fix JSONEachRowArray 2020-12-15 00:56:46 +03:00
Nikita Mikhaylov
e0addac6fc save changes 2020-12-15 00:56:46 +03:00
Maksim Kita
18dc118298 Fixed compile issues 2020-12-14 22:12:15 +03:00
Maksim Kita
0464859cfe Updated usage of different types during IN query 
1. Added accurateCast function.
2. Use accurateCast in Set during execute.
3. Added accurateCast tests.
4. Updated select_in_different_types tests.
 2020-12-14 22:12:15 +03:00
Maksim Kita
f4b8e8ef99 Allow different types inside IN subquery 2020-12-14 22:12:15 +03:00
alesapin
34d1ad1aca Fix writer compact 2020-12-14 17:27:39 +03:00
alesapin
9e98cdad5a Accident changes 2020-12-14 16:05:07 +03:00
alesapin
47e3f619e5 Remove redundant code 2020-12-14 16:01:01 +03:00
alesapin
a87fb3fae0 Fix style 2020-12-14 15:52:15 +03:00
alesapin
303c12d2e4 Slightly better 2020-12-14 15:51:14 +03:00
alesapin
6fb23da851 Refactoring 2020-12-14 15:03:49 +03:00
alesapin
70be03aa7d Working compact parts 2020-12-14 14:36:15 +03:00
alesapin
fbcf85cb42 Something working for wide parts 2020-12-14 14:06:02 +03:00
alesapin
12b6a54a24 Merge branch 'master' into try_rewrite_writer 2020-12-14 10:30:01 +03:00
alesapin
fd695c9ff4 Dive into debug 2020-12-14 10:28:42 +03:00
Azat Khuzhin
5b3ab48861 More forward declaration for generic headers 
The following headers are pretty generic, so use forward declaration as
much as possible:
- Context.h
- Settings.h
- ConnectionTimeouts.h
(Also this shows that some missing some includes -- this has been fixed)

And split ConnectionTimeouts.h into ConnectionTimeoutsContext.h (since
module part cannot be added for it, due to recursive build dependencies
that will be introduced)

Also remove Settings from the RemoteBlockInputStream/RemoteQueryExecutor
and just pass the context, since settings was passed only in speicifc
places, that can allow making a copy of Context (i.e. Copier).

Approx results (How much units will be recompiled after changing file X?):

- ConnectionTimeouts.h
  - mainline: 100

- Context.h:
  - mainline: ~800
  - patched:  415

- Settings.h:
  - mainline: 900-1K
  - patched:  440 (most of them because of the Context.h)
 2020-12-12 17:43:10 +03:00
alesapin
a24a76828e Fix 2020-12-11 18:18:51 +03:00
alesapin
4b83707c63 Compileable code for some reason 2020-12-11 18:00:58 +03:00
alesapin
57c30c4164 Garbage 2020-12-11 16:20:19 +03:00
alesapin
7ffd76a605 Fix bug 2020-12-11 11:41:02 +03:00
alesapin
cbba73672e Merge branch 'master' into try_rewrite_writer 2020-12-11 11:09:47 +03:00
Alexander Kuzmenkov
d757f32300 Remove some redundant includes to speed up build 2020-12-11 02:58:11 +03:00
alexey-milovidov
c35c52be87
Merge pull request #17867 from ucasFL/mergetree-setting 
Add settings `min_compress_block_size` and `max_compress_block_size` to MergeTreeSettings
 2020-12-10 22:54:05 +03:00
alexey-milovidov
f1ee1ef16a
Update MergeTreeSettings.h 2020-12-10 22:50:54 +03:00
alesapin
1cb06bd975 Split one method 2020-12-10 19:29:10 +03:00
alesapin
90fa9a2073 Merge branch 'master' into try_rewrite_writer 2020-12-10 18:36:10 +03:00
alesapin
f3d1dc360c Fix style 2020-12-10 12:24:52 +03:00
alesapin
0704b478ee Less constuctors 2020-12-10 12:22:43 +03:00
alesapin
2787b615b0 Make interface as simple as possible 2020-12-10 11:57:52 +03:00
alesapin
fef65b0cbd
Merge pull request #17919 from ClickHouse/additional_size_check 
Add additional columns size check for MergeTree in debug mode
 2020-12-09 22:40:06 +03:00
alesapin
7979b3655c Better formating 2020-12-09 21:22:07 +03:00
alesapin
3b88c077ed Mechanical refactoring 2020-12-09 21:19:49 +03:00
alesapin
af73db93e6 Mechanical refactoring 2020-12-09 21:10:09 +03:00
Pavel Kruglov
c8c543ca94 Add throwing exceptions 2020-12-09 18:07:58 +03:00
alesapin
b307e545a9 Fix check 2020-12-09 14:46:04 +03:00
alesapin
0f4056fd95 Add additional size check in debug mode 2020-12-09 14:23:37 +03:00
alesapin
7783ddb3af
Revert "Fix index granularity calculation on block borders" 2020-12-09 14:05:48 +03:00
Vasily Nemkov
59fc301344 Fixed test to be less flaky 
Also logging expanded list of columns passed from `DEDUPLICATE BY` to actual deduplication routines.
 2020-12-08 19:44:34 +03:00
Kruglov Pavel
e19eb6f17a
Merge branch 'master' into optimize-data-on-insert 2020-12-08 15:57:46 +03:00
Vasily Nemkov
8c5daf0925 Fixed building tests with GCC-10 
Also a minor cleanup of the test code.
 2020-12-08 13:15:18 +03:00
Anton Popov
06d5b87bc9 fix nested and subcolumns 2020-12-07 22:02:26 +03:00
Anton Popov
d7aad3bf79 Merge remote-tracking branch 'upstream/master' into HEAD 2020-12-07 19:13:01 +03:00
Vasily Nemkov
168155eeec Minor: cleanup 2020-12-07 18:07:40 +03:00
Vasily Nemkov
dbdc018ab8 Fixed and refined unite-test 
* no more undefined values for attributes in ReplicatedMergeTreeLogEntry
* validation of string serialization format
 2020-12-07 17:38:25 +03:00
Anton Popov
66e0add2ba fix nested 2020-12-07 16:35:12 +03:00
feng lv
50b64ba1f8 fix 2020-12-07 13:18:31 +00:00
feng lv
d7b30f74d1 add min-max_compress_block_size to mergetree settings 2020-12-07 12:47:51 +00:00
Vasily Nemkov
70ea507dae OPTIMIZE DEDUPLICATE BY columns 
Extended OPTIMIZE ... DEDUPLICATE syntax to allow explicit (or implicit with asterisk/column transformers) list of columns to check for duplicates on.

Following syntax variants are now supported:

OPTIMIZE TABLE table DEDUPLICATE; -- the old one
OPTIMIZE TABLE table DEDUPLICATE BY *;
OPTIMIZE TABLE table DEDUPLICATE BY * EXCEPT colX;
OPTIMIZE TABLE table DEDUPLICATE BY * EXCEPT (colX, colY);
OPTIMIZE TABLE table DEDUPLICATE BY col1,col2,col3;
OPTIMIZE TABLE table DEDUPLICATE BY COLUMNS('column-matched-by-regex');
OPTIMIZE TABLE table DEDUPLICATE BY COLUMNS('column-matched-by-regex') EXCEPT colX;
OPTIMIZE TABLE table DEDUPLICATE BY COLUMNS('column-matched-by-regex') EXCEPT (colX, colY);

Note that * behaves just like in SELECT: MATERIALIZED, and ALIAS columns are not used for expansion.
Also, it is an error to specify empty list of columns, or write an expression that results in an empty list of columns, or deduplicate by an ALIAS column.
Column transformers other than EXCEPT are not supported.
 2020-12-07 09:44:07 +03:00
feng lv
a634e4ed80 fix incorrect initialize MergeTreeWriterSettings 2020-12-06 08:57:02 +00:00
alesapin
0eec52b1fd
Merge pull request #17737 from ClickHouse/fix_segfault_in_distributed_out_stream 
Fix segfault when 'not enough space'
 2020-12-05 16:14:30 +03:00
Anton Popov
d1ef6a897e
Merge pull request #17802 from detailyang/patch-1 
hotfix:check in_memory_parts_enable_wal
 2020-12-05 03:05:38 +03:00
Anton Popov
60b0cbb1c1
Merge pull request #15939 from Avogar/optimize_final_optimization 
Optimize final optimization
 2020-12-05 02:26:27 +03:00
Pavel Kruglov
4b58528b9e Rename getPos to getRow, change mergeBlock, pass setting instead of context 2020-12-04 19:25:30 +03:00
Pavel Kruglov
905ba78adc Merge branch 'master' of github.com:ClickHouse/ClickHouse into optimize-data-on-insert 2020-12-04 18:56:46 +03:00
detailyang
e7a151fd1e
hotfix:check in_memory_parts_enable_wal 2020-12-04 23:14:27 +08:00
alesapin
42544a9833 Merge branch 'master' into fix_segfault_in_distributed_out_stream 2020-12-04 17:41:10 +03:00
Pavel Kruglov
9dbced0474 Pass setting instead of context 2020-12-04 17:01:59 +03:00
roverxu
2a6c1463a1 Add metrics for part number in MergeTree 2020-12-04 11:42:58 +08:00
Ivan
315ff4f0d9
ANTLR4 Grammar for ClickHouse and new parser (#11298) 2020-12-04 05:15:44 +03:00
Anton Popov
cd1917c7a6
Merge branch 'master' into optimize_final_optimization 2020-12-03 16:52:51 +03:00
Alexander Tokmakov
bfbf150c67 fix segfault when 'not enough space' 2020-12-02 17:49:43 +03:00
Alexander Tokmakov
19c8399eb0 Merge branch 'master' into database_replicated 2020-12-01 17:33:07 +03:00
Alexey Milovidov
6590fdfdd2 Fix logging 2020-11-30 11:44:01 +03:00
alesapin
25f40db2fb
Merge pull request #17499 from ClickHouse/concurrent_mutation_and_random_kill 
Fix kill mutation on concurrent alter queries
 2020-11-30 10:51:50 +03:00
Alexey Milovidov
2e6bedce91 Relax "Too many parts" threshold, part 1 2020-11-29 18:08:02 +03:00
alexey-milovidov
fabceebbce
Merge pull request #17145 from amosbird/cddt 
Fix unmatched type comparison in KeyCondition
 2020-11-29 14:29:35 +03:00
alexey-milovidov
f4a61ac3c3
Merge pull request #17527 from ucasFL/spelling 
fix spelling errors
 2020-11-29 13:45:42 +03:00
feng lv
7e3524caa1 fix spelling errors 2020-11-28 08:17:20 +00:00
alexey-milovidov
c189f6405f
Merge pull request #16767 from azat/optimize_trivial_count_query-fix 
Fix optimize_trivial_count_query with partition predicate
 2020-11-28 08:43:12 +03:00
Nikita Mikhaylov
6f3db7ff50
Merge pull request #17330 from CurtizJ/move-to-prewhere 
Allow to move conditions to prewhere with compact parts
 2020-11-28 02:02:42 +03:00
alesapin
6567796014 Fix kill mutation on concurrent alter queries 2020-11-27 18:46:52 +03:00
Alexander Tokmakov
9e3fd3c170 Merge branch 'master' into database_replicated 2020-11-27 17:08:34 +03:00
Anton Popov
1a4ed07a98 add some more comments 2020-11-27 15:47:27 +03:00
alexey-milovidov
dfae1efbbd
Merge pull request #17070 from fastio/master 
Support multiple ZooKeeper clusters
 2020-11-27 10:38:01 +03:00
Amos Bird
022ba2b0a9
Fix unmatched type comparison in KeyCondition 2020-11-26 16:15:50 +08:00
alesapin
960d077612 Better functions 2020-11-26 10:25:57 +03:00
alexey-milovidov
75a78e6c20
Update BackgroundJobsExecutor.h 2020-11-26 07:09:05 +03:00
Azat Khuzhin
0b47f4a9e9 Fix optimize_trivial_count_query with partition predicate 
Consider the following example:

    CREATE TABLE test(p DateTime, k int) ENGINE MergeTree PARTITION BY toDate(p) ORDER BY k;
    INSERT INTO test VALUES ('2020-09-01 00:01:02', 1), ('2020-09-01 20:01:03', 2), ('2020-09-02 00:01:03', 3);

- SELECT count() FROM test WHERE toDate(p) >= '2020-09-01' AND p <= '2020-09-01 00:00:00'
  In this case rpn will be (FUNCTION_IN_RANGE, FUNCTION_UNKNOWN (due to strict), FUNCTION_AND)
  and for optimize_trivial_count_query we cannot use index if there is at least one FUNCTION_UNKNOWN.
  since there is no post processing and return count() based on only the first predicate is wrong.

  Before this patch FUNCTION_UNKNOWN was allowed for optimize_trivial_count_query, and the result was wrong.

And two examples above just to show the difference, the behaviour hadn't been changed with this patch:

- SELECT * FROM test WHERE toDate(p) >= '2020-09-01' AND p <= '2020-09-01 00:00:00'
  In this case will be (FUNCTION_IN_RANGE, FUNCTION_IN_RANGE (due to non-strict), FUNCTION_AND)
  so it will prune everything out and nothing will be read.

- SELECT * FROM test WHERE toDate(p) >= '2020-09-01' AND toUnixTimestamp(p)%5==0
  In this case will be (FUNCTION_IN_RANGE, FUNCTION_UNKNOWN, FUNCTION_AND)
  and all, two, partitions will be scanned, but due to filtering later none of rows will be matched.
 2020-11-25 23:09:17 +03:00
tavplubix
6477251ea1
Update BackgroundJobsExecutor.h 2020-11-25 23:06:00 +03:00
Alexander Tokmakov
5ccf975a4a Merge branch 'master' into database_replicated 2020-11-24 14:38:02 +03:00
Anton Popov
852a08eacb allow to move conditions to prewhere with compact parts 2020-11-23 21:27:59 +03:00
Anton Popov
b384beb564 Merge remote-tracking branch 'upstream/master' into HEAD 2020-11-23 17:46:51 +03:00
Amos Bird
172b7e9ed1
global in set index. 2020-11-23 22:05:08 +08:00
tavplubix
5cc9cb01cd
Merge pull request #16751 from amosbird/globalcontext 
Make global_context consistent.
 2020-11-22 18:46:17 +03:00
alesapin
9c8b0da382
Merge pull request #16033 from nvartolomei/nv/parts-uuid 
Add unique identifiers IMergeTreeDataPart structure
 2020-11-22 16:13:19 +03:00
Anton Popov
1f65478b49 implement nested with multiple nesting 2020-11-21 20:27:42 +03:00
Pavel Kruglov
623dc2df7a Update comment 2020-11-20 17:32:39 +03:00
Pavel Kruglov
ca3fe49a2a Make setting global 2020-11-20 17:29:13 +03:00
Nicolae Vartolomei
7c8bc1c04e Use JSON metadata in WAL 2020-11-20 13:49:17 +00:00
Nicolae Vartolomei
040aba9f85 Add uuid.txt to checksums for parts stored on disk 
We are breaking backwards compatibility anyway (but agted by a setting)
 2020-11-20 13:49:17 +00:00
Nicolae Vartolomei
94293ca3ce Assign UUIDs to parts only when configured to do so 
Avoid breaking backwards compatibility by default for now.
 2020-11-20 13:49:17 +00:00
Pavel Kruglov
e5588b94aa Fix style and tests 2020-11-20 13:38:53 +03:00
Amos Bird
1d9d586e20
Make global_context consistent. 2020-11-20 18:23:14 +08:00
Pavel Kruglov
625962fb59 Remove debug logging 2020-11-20 03:19:36 +03:00
Pavel Kruglov
3ee4972586 Merge branch 'master' of github.com:ClickHouse/ClickHouse into optimize-data-on-insert 2020-11-20 01:23:51 +03:00
Pavel Kruglov
4c30857759 Minor change 2020-11-20 01:22:40 +03:00
alesapin
3f01096c86 Less verbose logging when fetch is impossible 2020-11-19 20:03:20 +03:00
alesapin
2623d35f68
Merge pull request #17120 from ClickHouse/fix_granularity_on_block_borders 
Fix index granularity calculation on block borders
 2020-11-19 18:36:07 +03:00
Nicolae Vartolomei
746f8e45f5 All new parts must have uuids 2020-11-19 13:18:03 +00:00
Nicolae Vartolomei
425dc4b11b Add unique identifiers IMergeTreeDataPart structure 
For now uuids are not generated at all, they are present only if the
part is updated manually (as you can see in the integration test).

The only place where they can be seen today by an end user is in
`system.parts` table. I was looking for hiding this column behind an
option but couldn't find an easy way to do that.

Likely this is also required for WAL, but need to think how not to break
compatibility.

Relates to #13574, https://github.com/ClickHouse/ClickHouse/issues/13574

Next 1: In the upcoming PR the plan is to integrate de-duplication based on
these fingerprints in the query pipeline.

Next 2: We'll enable automatic generation of uuids and come up with a
way for conditionally sending uuids when processing distributed queries
only when part movement is in progress.
 2020-11-19 13:14:25 +00:00
Peng Jian
a0683ce460 Support mulitple ZooKeeper clusters 2020-11-19 15:44:47 +08:00
alesapin
7080f424e2 Fix bug for skip indices and make code more complex 2020-11-18 15:04:13 +03:00
Pavel Kruglov
547ec19fb3 Merge branch 'master' of github.com:ClickHouse/ClickHouse into optimize-data-on-insert 2020-11-18 12:01:59 +03:00
filimonov
258170b325
Update ReplicatedMergeTreeMergeStrategyPicker.cpp 2020-11-18 08:45:44 +01:00
Mikhail Filimonov
234c671e52
After CR fixes 2020-11-18 08:45:44 +01:00
Mikhail Filimonov
97fef77ed1
execute_merges_on_single_replica 2020-11-18 08:45:33 +01:00
Pavel Kruglov
53b8ff42c2 Add test and set setting to 0 in some other tests 2020-11-17 19:56:36 +03:00
alesapin
389b883534 Fix index granularity calculation on block borders 2020-11-17 12:37:13 +03:00
Anton Popov
de5ead0c40 add setting remove_empty_parts 2020-11-15 05:24:47 +03:00
alexey-milovidov
915d96fee0
Merge pull request #16955 from nikitamikhaylov/fix-typos 
Fix typos
 2020-11-14 17:17:37 +03:00
alexey-milovidov
8ef4bfc0b5
Merge pull request #16968 from amosbird/blocktypemismatch 
Fix missing conversion of unmutated parts
 2020-11-14 17:16:51 +03:00
Nikolai Kochetov
ed98b48845
Merge pull request #16618 from ClickHouse/actions-dag-f14 
Actions dag 2
 2020-11-13 18:34:51 +03:00
alesapin
b2a8b6135d
then 2020-11-13 16:24:14 +03:00
alesapin
b8e33bd35b Add comments 2020-11-13 15:42:26 +03:00
Pavel Kruglov
264e9daf6f Fix style 2020-11-13 11:16:30 +03:00
Pavel Kruglov
6a57c0a8cf Move merge in MergeTreeDataWriter 2020-11-13 10:55:56 +03:00
alesapin
8d3858fc22
Merge pull request #16895 from CurtizJ/remove-empty-parts 
Remove empty parts after they were pruned by TTL [2]
 2020-11-13 10:00:18 +03:00
Amos Bird
fdf979d5ce
Fix missing conversion of unmutated parts 2020-11-13 11:45:50 +08:00
nikitamikhaylov
c0940f2ebd done 2020-11-13 02:27:18 +03:00
Pavel Kruglov
8d5e0784d3 Add setting optimize_on_insert 2020-11-12 23:37:23 +03:00
alesapin
bbd85c3b2d
Merge pull request #16390 from ClickHouse/add_separate_pool_for_fetches 
Add separate pool for fetches
 2020-11-12 22:42:28 +03:00
Alexander Tokmakov
0731bfb7be Merge branch 'master' into database_replicated 2020-11-12 21:04:18 +03:00
Anton Popov
4cecd89e84 fix dropping of empty parts 2020-11-12 20:37:54 +03:00
Anton Popov
0aa35d53c7 Merge remote-tracking branch 'upstream/master' into HEAD 2020-11-12 14:31:58 +03:00
alesapin
2cac5e5d97
Merge pull request #16875 from CurtizJ/refactor-alter-partition 
Avoid code duplication in alterPartition
 2020-11-12 13:19:59 +03:00
alesapin
a8d466a645 Merge branch 'master' into add_separate_pool_for_fetches 2020-11-12 12:51:02 +03:00
Nikolai Kochetov
46f70dd0de Merge branch 'master' into actions-dag-f14 2020-11-12 11:54:44 +03:00
tavplubix
058aa8f85e
Merge pull request #16824 from ClickHouse/replace_stringstreams_with_buffers 
Replace std::*stringstreams with DB::*Buffers
 2020-11-12 01:11:44 +03:00
Anton Popov
0a6d93d929
Update MergeTreeData.h 2020-11-11 21:17:41 +03:00
Anton Popov
2497598a49 Remove empty parts after they were pruned by TTL 2020-11-11 19:19:43 +03:00
Anton Popov
c01870076d
Update MergeTreeData.h 2020-11-11 19:09:01 +03:00
Anton Popov
6f0dc08ba7 avoid code duplication in alterPartition 2020-11-11 16:34:07 +03:00
Nikolai Kochetov
1846bb3cac Merge branch 'master' into actions-dag-f14 2020-11-11 13:08:57 +03:00
Pavel Kruglov
c648f62629 Merge branch 'master' of github.com:ClickHouse/ClickHouse into optimize_final_optimization 2020-11-10 22:58:21 +03:00
Alexander Tokmakov
b94cc5c4e5 remove more stringstreams 2020-11-10 21:22:26 +03:00
Nikolai Kochetov
c6575c9032 Update ExpressionActions constructur 2020-11-10 19:27:55 +03:00
Nikolai Kochetov
1db8e77371 Add comments. Update ActionsDAG::Index 2020-11-10 17:54:59 +03:00
Pavel Kruglov
9120189d8a Add SelectPartsDecision enum class 2020-11-10 17:42:56 +03:00
Anton Popov
9b354be66f Merge remote-tracking branch 'upstream/master' into HEAD 2020-11-10 16:21:11 +03:00
Nikolai Kochetov
07fe3a6347 Fix build. 2020-11-10 15:14:05 +03:00
Nikolai Kochetov
195c941c4e Merge branch 'master' into storage-read-query-plan 2020-11-10 15:02:22 +03:00
alesapin
2e357516a6 Merge branch 'master' into add_separate_pool_for_fetches 2020-11-10 13:53:08 +03:00
Vladimir Chebotarev
059357d51e
ALTER UPDATE/DELETE ... IN PARTITION with partition pruning in ReplicatedMergeTree (#13403) 
Co-authored-by: Alexander Kazakov <Akazz@users.noreply.github.com>
 2020-11-10 13:23:46 +03:00
Nikolai Kochetov
363c1e05c0 Try fix tests. 2020-11-10 12:35:05 +03:00
Alexander Tokmakov
5cdfcfb307 remove other stringstreams 2020-11-09 22:12:44 +03:00
Nikolai Kochetov
e41b1ae52b Empty commit. 2020-11-09 19:35:43 +03:00
Alexander Tokmakov
62ff00ee8b use WriteBuffer in formatAST(...) 2020-11-09 19:05:40 +03:00
alesapin
ea7fe94aa9
Merge pull request #15511 from nvartolomei/nv/drop-detached-part-quorum-possible-solution 
ALTER TABLE ... DROP|DETACH PART
 2020-11-09 18:00:30 +03:00
alexey-milovidov
e47ed55d32
Merge pull request #14977 from achimbab/indexOf_bloomfilter 
Fix indexOf() to use BloomFilter
 2020-11-09 17:58:42 +03:00
Nikolai Kochetov
99cc9b1ec0 Fix build 2020-11-09 16:20:56 +03:00
Nikolai Kochetov
8c4db34f9d Update after merge. 2020-11-09 14:58:11 +03:00
Nikolai Kochetov
6717c7a0af Merge branch 'master' into actions-dag-f14 2020-11-09 14:57:48 +03:00
alesapin
e347f4b72f Add some comments 2020-11-09 12:14:20 +03:00
alesapin
72046313d4 Merge branch 'master' into nvartolomei-drop-part 2020-11-09 10:02:02 +03:00
alesapin
4e846d2197 Merge branch 'master' into add_separate_pool_for_fetches 2020-11-09 10:00:21 +03:00
alexey-milovidov
0e6ae4aff7
Merge pull request #16253 from amosbird/pf 
Prune partition in verbatim way.
 2020-11-08 18:58:02 +03:00
Alexey Milovidov
3f5839c83c Add one more case 2020-11-08 16:45:57 +03:00
Alexey Milovidov
8e1a2fc9e4 Fix UB and overflow 2020-11-08 16:41:25 +03:00
alexey-milovidov
8264bd900b
Update MergeTreeIndexConditionBloomFilter.cpp 2020-11-08 16:17:03 +03:00
alexey-milovidov
85f7eadc4f
Update MergeTreeIndexConditionBloomFilter.cpp 2020-11-08 16:14:32 +03:00
alexey-milovidov
dfa35a5e6d
Update MergeTreeIndexConditionBloomFilter.cpp 2020-11-08 16:11:18 +03:00
Alexey Milovidov
cc012e2cc7 Merge branch 'master' into achimbab-indexOf_bloomfilter 2020-11-08 14:42:03 +03:00
alexey-milovidov
f4ba5f1f9a
Merge pull request #16772 from ClickHouse/fix-stringstream 
Fix "server failed to start" error
 2020-11-08 14:27:08 +03:00
Alexey Milovidov
5314185e25 Merge branch 'master' into azat-optimize_skip_unused_shards-optimization 2020-11-08 00:17:59 +03:00
Alexey Milovidov
fd84d16387 Fix "server failed to start" error 2020-11-07 03:14:53 +03:00
Anton Popov
245c395a68 Merge remote-tracking branch 'upstream/master' into HEAD 2020-11-06 22:00:32 +03:00
Anton Popov
01a04e06ef implement nested with multiple nesting 2020-11-06 21:57:04 +03:00
alexey-milovidov
7fb53b205c
Merge pull request #16637 from azat/mt-read_in_order-spread-fix 
Fix spreading for ReadInOrderOptimizer with expression in ORDER BY
 2020-11-06 17:36:03 +03:00
Nikolai Kochetov
c10f733587 Merge branch 'master' into storage-read-query-plan 2020-11-06 15:43:46 +03:00
Nikolai Kochetov
9aeb757da4 Merge branch 'master' into actions-dag-f14 2020-11-06 15:04:20 +03:00
Amos Bird
2b0085c106
Pruning is different from counting 2020-11-06 19:58:03 +08:00
Amos Bird
aa436a3cb1
Transform single point 2020-11-06 14:59:55 +08:00
Amos Bird
30bf5e6d26
Prune partition in verbatim way. 2020-11-06 09:56:13 +08:00
alexey-milovidov
ee3e289ec2
Merge pull request #15983 from ClickHouse/no_background_pool_no_more 
Replace BackgroundProcessingPool with SchedulePool task and ThreadPool.
 2020-11-05 21:38:22 +03:00
Anton Popov
bea0984d12
Merge pull request #16642 from kitaisreal/bloom-filter-index-added-big-int-and-uuid-support 
BloomFilter index added big integers and UUID support
 2020-11-05 15:26:08 +03:00
Maksim Kita
7dd749fc4c BloomFilter index added big integers and UUID support 2020-11-04 23:41:10 +03:00
Alexey Milovidov
1bcf22d42f Fix 'max_parallel_replicas' without sampling. 2020-11-04 18:59:14 +03:00
alesapin
20043e77bb Merge branch 'master' into nvartolomei-drop-part 2020-11-04 10:22:23 +03:00
Azat Khuzhin
2389406c21 Fix spreading for ReadInOrderOptimizer with expression in ORDER BY 
This will fix optimize_read_in_order/optimize_aggregation_in_order with
max_threads>0 and expression in ORDER BY
 2020-11-04 07:07:26 +03:00
Nikolai Kochetov
54a9b80a11 Fix build 2020-11-03 22:30:58 +03:00
Nikolai Kochetov
4984fdc693 Fix build 2020-11-03 22:28:50 +03:00
Nikolai Kochetov
7b61f5d641 Refactor more 2020-11-03 22:05:47 +03:00
alesapin
faca9d9b91 Check max bytes on disk 2020-11-03 18:41:46 +03:00
Nikolai Kochetov
6767a226fc Merge branch 'master' into actions-dag-f14 2020-11-03 15:21:06 +03:00
Nikolai Kochetov
07a7c46b89 Refactor ExpressionActions [Part 3] 2020-11-03 14:28:28 +03:00
alexey-milovidov
11fc6fd8c9
Merge pull request #16584 from ClickHouse/drop_table_task_better_scheduling 
Better scheduling of drop table task
 2020-11-03 14:22:51 +03:00
Anton Popov
85ab1e9bc3
Merge pull request #15938 from Avogar/select_final 
Select final
 2020-11-03 13:49:47 +03:00