thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 19:32:07 +00:00
Code Issues Packages Projects Releases Wiki Activity
48,693 Commits 9,319 Branches 1,783 Tags 2.1 GiB
8bc9ede711
Commit Graph

766 Commits

Author SHA1 Message Date
Alexander Tokmakov
5b92f43f12 Merge branch 'master' into database_atomic_improvements 2020-08-03 14:35:40 +03:00
Nikolai Kochetov
e411916bde Refactor Pipe [part 1]. 2020-08-03 14:33:11 +03:00
Nikolai Kochetov
cf49a83980
Merge pull request #13201 from azat/fix-logging-in-MergeTreeDataSelectExecutor 
Fix logging in MergeTreeDataSelectExecutor for multiple threads (attach to thread group)
 2020-08-03 10:08:58 +03:00
Alexey Milovidov
77516b48fa Merge branch 'master' into merge-tree-settings-sanity-check 2020-08-03 00:30:56 +03:00
Alexey Milovidov
0fd7af8a63 Merge branch 'master' into stack-frams-size-limit 2020-08-03 00:06:00 +03:00
Alexey Milovidov
4ed0bf3af1 Better code 2020-08-03 00:01:39 +03:00
Alexey Milovidov
3c489ce159 Fix assertion in KeyCondition 2020-08-02 23:55:20 +03:00
Alexey Milovidov
5f808aa503 Fix bad code 2020-08-02 23:41:52 +03:00
alesapin
75a9053511
Merge pull request #13150 from nvartolomei/nv/freeze-verbose-absolute-path 
Use absolute paths to backup locations (shadow)
 2020-08-02 19:25:24 +03:00
Alexey Milovidov
41111d1e57 Merge branch 'master' into stack-frams-size-limit 2020-08-02 19:18:33 +03:00
Alexey Milovidov
778abb346f Merge branch 'master' into merge-tree-settings-sanity-check 2020-08-02 17:13:17 +03:00
Azat Khuzhin
e37c42c56c Fix logging in MergeTreeDataSelectExecutor for multiple threads (attach to thread group) 2020-08-02 13:40:01 +03:00
Azat Khuzhin
101217470e Use "Not using primary index on part" over "Not using index on part" (add "primary") 2020-08-02 13:40:01 +03:00
Vitaly Baranov
18b21511a9
Merge pull request #13013 from vitlibar/implement-custom-settings 
Implement custom settings
 2020-08-02 05:01:14 +03:00
Alexey Milovidov
d3c0635fc0 Merge branch 'master' into stack-frams-size-limit 2020-08-02 04:35:09 +03:00
Alexey Milovidov
8fe201f1b9 Merge branch 'master' into merge-tree-settings-sanity-check 2020-08-01 19:20:11 +03:00
Nikolai Kochetov
c786f05388
Merge pull request #13189 from ClickHouse/refactor-pipes-2 
Remove TreeExecutorBIS
 2020-08-01 11:56:50 +03:00
Vitaly Baranov
56665a15f7 Rework and rename the template class SettingsCollection => BaseSettings. 2020-07-31 20:54:18 +03:00
Vitaly Baranov
90602b869a Make SettingsChanges a class. 2020-07-31 19:11:27 +03:00
Vitaly Baranov
4ad99f1472 Make type names of settings shorter: SettingUInt64 => UInt64. 2020-07-31 19:11:27 +03:00
Nikolai Kochetov
39530f837e Remove TreeExecutorBlockInputStream. 2020-07-31 16:23:19 +03:00
alesapin
45894fcf99 Better comments 2020-07-31 15:22:32 +03:00
alexey-milovidov
9c9ef0db9b
Merge branch 'master' into database_atomic_improvements 2020-07-31 14:56:49 +03:00
alesapin
14928874a1 Fix mutations status check 2020-07-31 14:37:16 +03:00
Alexey Milovidov
2c40539df6 Check ALTERs; update test 2020-07-31 00:42:55 +03:00
Alexey Milovidov
c491b2c153 Lower stack frame size 2020-07-30 23:58:08 +03:00
Alexey Milovidov
b9f49d31df Sanity checks for MergeTreeSettings 2020-07-30 22:08:13 +03:00
alexey-milovidov
fc35ce69fa
Merge pull request #12666 from ClickHouse/forward_alter_error_to_client 
Forward exceptions to client if exception happened during ALTER
 2020-07-30 21:36:06 +03:00
Nicolae Vartolomei
85174d64a0 Use absolute paths to backup locations (shadow) 
Tests are frankenstain. Had to hide absolute path `/tmp/clickhouse.test..IP69I/data/shadow/test_01417_single_part_7/data/default/table_for_freeze/7_8_8_0` to make them deterministic and independend of the method they are run (local vs remote target)
 2020-07-30 18:03:26 +01:00
alexey-milovidov
4c3241808d
Merge pull request #12728 from ClickHouse/prevent-inconsistent-merge-tree-settings 
Safe logic for incorrect MergeTreeSettings
 2020-07-30 17:55:39 +03:00
alexey-milovidov
e6265bf299
Merge pull request #13100 from ClickHouse/keep-less-amount-of-logs-in-zookeeper 
Keep less amount of logs in ZooKeeper
 2020-07-30 15:41:55 +03:00
alexey-milovidov
a22bdb1989
Merge pull request #13081 from CurtizJ/fix-index_analysis 
Fix wrong index analysis with functions
 2020-07-30 15:30:55 +03:00
Alexey Milovidov
6517ddcbd4 Keep less amount of logs in ZooKeeper 2020-07-30 05:47:05 +03:00
Alexander Kuzmenkov
881220eec2
Merge pull request #12755 from bobrik/ivan/obvious-skip-overall 
Show marks before applying skipping indices
 2020-07-29 20:59:08 +03:00
Anton Popov
4c266d1e5d fix wrong index analysis with functions 2020-07-29 19:09:38 +03:00
alesapin
7a806eb90f
Merge pull request #13017 from ClickHouse/freeze_partition_with_name 
Alter table freeze with verbose output
 2020-07-29 18:42:48 +03:00
Alexander Kuzmenkov
1b9269ae0c fixup 2020-07-28 19:58:19 +03:00
Alexander Kuzmenkov
297cf65f1f Merge remote-tracking branch 'origin/master' into HEAD 2020-07-28 19:56:35 +03:00
alesapin
c75830e878 Alter table freeze with verbose output 2020-07-28 18:10:36 +03:00
alesapin
585d9247ec
Merge pull request #12959 from ClickHouse/test_for_backup 
Check backup restore in integration test.
 2020-07-28 17:59:52 +03:00
Alexander Kazakov
2bde393499
Merging #12548 - Correction to merge_with_ttl_timeout logic by @excitoon (#12982) 
* Fixed `merge_with_ttl_timeout` logic.
* Separate TTL-merge timers for each partition

Co-authored-by: Vladimir Chebotarev <vladimir.chebotarev@gmail.com>
 2020-07-28 17:38:34 +03:00
Alexander Kuzmenkov
ba7c33f806
Merge pull request #12754 from bobrik/ivan/obvious-skip 
Show total granules examined by skipping indices
 2020-07-28 17:14:25 +03:00
alexey-milovidov
7a22717180
Merge pull request #12760 from ClickHouse/sticky_mutations_bug 
Sticking mutations bug
 2020-07-28 14:42:54 +03:00
alesapin
1bece3de0a Remove strange code and test 2020-07-28 12:36:08 +03:00
Alexey Milovidov
798ef1b9f9 Adjustments; added test result 2020-07-27 23:18:23 +03:00
Alexey Milovidov
ec84ed30cb Merge branch 'master' into prevent-inconsistent-merge-tree-settings 2020-07-27 21:39:25 +03:00
Nikolai Kochetov
abdd160db1
Merge pull request #12589 from bobrik/ivan/parallel-ranges 
Parallelize PK range and skipping index stages
 2020-07-27 19:23:00 +03:00
alexey-milovidov
7dac65ce0f
Merge pull request #12659 from zhang2014/fix/ISSUES-10572 
ISSUES-10572 fix bloom filter with const column
 2020-07-27 14:38:56 +03:00
alesapin
1707f84a44 Less strict check and rare rename bug 2020-07-27 12:42:37 +03:00
alesapin
e0bf5913e7 Fix sticking mutations 2020-07-26 17:21:57 +03:00
Alexander Tokmakov
6bfe88e83f Merge branch 'master' into database_atomic_improvements 2020-07-26 13:40:05 +03:00
alesapin
c66dc23a47 Fix error style 2020-07-25 23:56:29 +03:00
alesapin
eae7dc852e Fix build 2020-07-25 23:54:33 +03:00
alesapin
393a9195ca Better exception message 2020-07-25 17:45:46 +03:00
alesapin
78d357f0d2 Add a test for sticking mutations bug 2020-07-25 17:42:20 +03:00
Ivan Babrou
e835ec0b56 Show marks before applying skipping indices 
This change makes skipping index efficiency more obvious, changing this:

```
Selected 30 parts by date, 30 parts by key, 592 marks to read from 541 ranges
```

Into this:

```
Selected 30 parts by date, 30 parts by key, 48324 marks by primary key, 592 marks to read from 541 ranges
```
 2020-07-24 15:45:38 -07:00
Ivan Babrou
67d4529783 Show total granules examined by skipping indices 
This change makes skipping index efficiency more obvious, changing this:

```
Index `idx_duration` has dropped 59 granules.
```

Into this:

```
Index `idx_duration` has dropped 59 / 61 granules.
```
 2020-07-24 14:50:32 -07:00
Anton Popov
d3fbeb296e write compressed blocks for every column in granule 2020-07-23 19:56:12 +03:00
Anton Popov
61018c275d Merge remote-tracking branch 'upstream/master' into polymorphic-parts-4 2020-07-23 18:38:29 +03:00
Anton Popov
1d0493cfcc better code in ReaderCompact and perf test added 2020-07-23 18:37:44 +03:00
Nikolai Kochetov
dad9d369a1 Merge branch 'master' into bobrik-parallel-randes 2020-07-23 16:21:32 +03:00
Alexey Milovidov
6b4be2b9fe Safe logic for incorrect MergeTreeSettings #10897 2020-07-23 15:31:09 +03:00
Azat Khuzhin
c97fb4b0e5
Remove unused bits in MergeTreeRangeReader (#12671) 
Refs: #7769
Cc: @amosbird
 2020-07-23 11:20:09 +03:00
alesapin
978f286625 Trying to fix FreeBSD build 2020-07-23 11:00:18 +03:00
alesapin
6fdcb59df7 Merge branch 'master' into forward_alter_error_to_client 2020-07-23 10:48:58 +03:00
Alexander Tokmakov
237accd154 Merge branch 'master' into database_atomic_improvements 2020-07-23 06:33:20 +03:00
alesapin
355150afce Fix deadlock and better code 2020-07-22 22:29:54 +03:00
Artem Zuikov
2afd123eda
Refactoring: extract TreeOptimizer from SyntaxAnalyzer (#12645) 2020-07-22 20:13:05 +03:00
alesapin
8c8bdd5070 Fix tests and work with multiple mutations 2020-07-22 18:19:54 +03:00
Nikolai Kochetov
76d784be9c
Merge pull request #12613 from ClickHouse/remove-sort-description-from-streams 
Remove sort description from streams
 2020-07-22 17:35:05 +03:00
alesapin
de846e5e3c Forward exception to client during alter 2020-07-22 15:36:19 +03:00
Nikolai Kochetov
b27066389a Do not create ThreadPool for single thread. 2020-07-22 14:51:35 +03:00
zhang2014
580889f1e4 ISSUES-10572 fix bloom filter with const column 2020-07-22 17:42:12 +08:00
Alexander Tokmakov
9bcaaea3e0 Merge branch 'master' into database_atomic_improvements 2020-07-22 05:24:48 +03:00
alesapin
be106b662e Comment and style 2020-07-21 18:19:41 +03:00
Nikolai Kochetov
486a4932c3 Fix tests. 2020-07-21 17:08:18 +03:00
alesapin
daaa5cc382 Add setting allow_non_metadata_alters 2020-07-21 17:05:30 +03:00
Nikolai Kochetov
755f15def3 Make MergeTreeSetIndex::checkInRange const. 2020-07-21 14:22:45 +03:00
alesapin
379a5a750a Fix error message about granularity 2020-07-21 14:10:23 +03:00
Nikolai Kochetov
12c5e376c6 Remove mutable from RPNElement. 2020-07-21 14:02:58 +03:00
Nikolai Kochetov
cab9146ff3 Remove SortDescription from IBlockInputStream. 2020-07-21 11:05:52 +03:00
Nikolai Kochetov
d6583698a9 Merge branch 'master' into bobrik-parallel-randes 2020-07-20 19:05:28 +03:00
Nikolai Kochetov
0cc55781d8 Try fix tests. 2020-07-20 18:09:00 +03:00
Alexander Kuzmenkov
40ee6eba19
Merge pull request #12545 from ClickHouse/fix_load_part_without_checksums 
Fix loading parts without checksums
 2020-07-20 14:40:53 +03:00
Ivan Babrou
72622a9b00 Parallelize PK range and skipping index stages 
This runs PK lookup and skipping index stages on parts
in parallel, as described in #11564.

While #12277 sped up PK lookups, skipping index stage
may still be a bottleneck in a select query. Here we
parallelize both stages between parts.

On a query that uses a bloom filter skipping index to pick
2,688 rows out of 8,273,114,994 on a two day time span,
this change reduces latency from 10.5s to 1.5s.
 2020-07-19 21:49:41 -07:00
Alexander Tokmakov
c27de9b424 fixes 2020-07-17 16:11:44 +03:00
alesapin
ac802a8e96
Merge pull request #12543 from ClickHouse/fix_marks_type_on_mutation 
Fix incorrect adaptive granularity by default
 2020-07-17 11:09:54 +03:00
Alexander Tokmakov
a612080df1 Merge branch 'master' into database_atomic_improvements 2020-07-17 00:43:59 +03:00
Anton Popov
97e8a88b30
Merge pull request #12277 from bobrik/ivan/exact-range-speedup 
WIP: Optimize PK lookup for queries that match exact PK range
 2020-07-16 19:17:50 +03:00
alesapin
b3ec3b1ed1 Fix loading parts without checksums 2020-07-16 13:54:49 +03:00
alesapin
42983c5416 Fix incorrect adaptive granularity by default 2020-07-16 12:10:15 +03:00
Anton Popov
c6423b2b28 lower read buffer size, while reading from compact parts 2020-07-15 14:53:23 +03:00
Alexander Tokmakov
09abe71b60 Merge branch 'master' into database_atomic_improvements 2020-07-15 13:07:08 +03:00
alesapin
614540eddf
Merge pull request #12382 from ClickHouse/clear-all-columns 
Better errors for CLEAR/DROP columns (possibly in partitions)
 2020-07-15 12:52:06 +03:00
alesapin
9e41fbca55 Remove check for drop detached partition 2020-07-14 16:56:30 +03:00
alesapin
014bb070ec Fix tests 2020-07-14 11:19:39 +03:00
alesapin
1f576ee039 Some intermediate solution 2020-07-13 20:27:52 +03:00
alesapin
4a53264a86 Remove redundant and duplicated code 2020-07-13 19:19:08 +03:00
Alexander Tokmakov
fdb02edc7c Merge branch 'master' into database_atomic_improvements 2020-07-13 17:28:48 +03:00
Alexander Tokmakov
04c3e7cab6 add implicit uuid macro 2020-07-13 16:18:38 +03:00
Anton Popov
a9530d2883 in-memory parts: fix reading from nested 2020-07-13 12:10:55 +03:00
alexey-milovidov
ae7eff98ed
Merge pull request #12433 from amosbird/np 
Allow nullable key in MergeTree
 2020-07-13 04:36:00 +03:00
Amos Bird
cac5a89169
Allow nullable key in MergeTree 2020-07-12 22:21:51 +08:00
Ivan Babrou
8784994d65 Allow conditions outside of PK with exact range 
Conditions that are outside of PK are marked as `unknown` in `KeyCondition`,
so it's safe to allow them, as long as they are always combined by `AND`.
 2020-07-11 18:59:26 -07:00
Ivan Babrou
d9d8d0242e Optimize PK lookup for queries that match exact PK range 
Existing code that looks up marks that match the query has a pathological
case, when most of the part does in fact match the query.

The code works by recursively splitting a part into ranges and then discarding
the ranges that definitely do not match the query, based on primary key.

The problem is that it requires visiting every mark that matches the query,
making the complexity of this sort of look up O(n).

For queries that match exact range on the primary key, we can find
both left and right parts of the range with O(log 2) complexity.

This change implements exactly that.

To engage this optimization, the query must:

* Have a prefix list of the primary key.
* Have only range or single set element constraints for columns.
* Have only AND as a boolean operator.

Consider a table with `(service, timestamp)` as the primary key.

The following conditions will be optimized:

* `service = 'foo'`
* `service = 'foo' and timestamp >= now() - 3600`
* `service in ('foo')`
* `service in ('foo') and timestamp >= now() - 3600 and timestamp <= now`

The following will fall back to previous lookup algorithm:

* `timestamp >= now() - 3600`
* `service in ('foo', 'bar') and timestamp >= now() - 3600`
* `service = 'foo'`

Note that the optimization won't engage when PK has a range expression
followed by a point expression, since in that case the range is not continuous.

Trace query logging provides the following messages types of messages,
each representing a different kind of PK usage for a part:

```
Used optimized inclusion search over index for part 20200711_5710108_5710108_0 with 9 steps
Used generic exclusion search over index for part 20200711_5710118_5710228_5 with 1495 steps
Not using index on part 20200710_5710473_5710473_0
```

Number of steps translates to computational complexity.

Here's a comparison for before and after for a query over 24h of data:

```
Read 4562944 rows, 148.05 MiB in 45.19249672 sec.,   100966 rows/sec.,   3.28 MiB/sec.
Read 4183040 rows, 135.78 MiB in 0.196279627 sec., 21311636 rows/sec., 691.75 MiB/sec.
```

This is especially useful for queries that read data in order
and terminate early to return "last X things" matching a query.

See #11564 for more thoughts on this.
 2020-07-11 12:26:54 -07:00
Anton Popov
fbec940e0f fix reading and check query for compact parts with different codecs 2020-07-11 02:33:36 +03:00
alexey-milovidov
e22547c29d
Merge pull request #12388 from ClickHouse/bloom-filter-arg-check 
Check arguments of bloom filter index
 2020-07-10 20:54:16 +03:00
alexey-milovidov
caef1d8e24
Update MergeTreeIndexFullText.cpp 2020-07-10 20:53:58 +03:00
Anton Popov
24f627e52c fix reading from compact parts with different codecs 2020-07-10 18:57:10 +03:00
Alexey Milovidov
47eaffbe63 Additional checks 2020-07-10 11:21:40 +03:00
Alexey Milovidov
4b86f36d37 Check arguments of bloom filter index 2020-07-10 11:13:21 +03:00
Alexey Milovidov
276b3a0215 Avoid exception when negative or floating point constant is used in WHERE condition for indexed tables #11905 2020-07-10 09:30:49 +03:00
alexey-milovidov
c16d8e094b
Merge pull request #12308 from ClickHouse/fix-codec-bad-exception-code 
Fix wrong exception code in codecs Delta, DoubleDelta #12110
 2020-07-10 08:40:46 +03:00
Anton Popov
80a62977f1 fix hashing in DataPartWriterCompact 2020-07-09 21:26:54 +03:00
Anton Popov
9f72baad1f Merge remote-tracking branch 'upstream/master' into HEAD 2020-07-09 20:51:36 +03:00
Alexey Milovidov
6c458a6287 Fix race condition in ReplicatedMergeTreeQueue 2020-07-09 07:43:28 +03:00
Alexey Milovidov
ea970fd57c Remove bad ugliness 2020-07-09 04:00:16 +03:00
alesapin
84f8bf1e14
Merge pull request #10697 from CurtizJ/polymorphic-parts 
Polymorphic parts (in-memory format)
 2020-07-08 13:25:45 +03:00
Anton Popov
416d9648c2 remove questionable functionality 2020-07-07 12:58:14 +03:00
Anton Popov
d6434f61dc support codecs in compact parts 2020-07-07 03:15:02 +03:00
alesapin
3f90f65678 Merge remote-tracking branch 'origin' into CurtizJ-polymorphic-parts 2020-07-06 21:34:17 +03:00
Nikita Mikhaylov
2b539e6ace fix style 2020-07-06 20:26:15 +03:00
Nikita Mikhaylov
d31ed58f01 done 2020-07-06 17:33:31 +03:00
myrrc
8c3417fbf7
ILIKE operator (#12125) 
* Integrated CachingAllocator into MarkCache

* fixed build errors

* reset func hotfix

* upd: Fixing build

* updated submodules links

* fix 2

* updating grabber allocator proto

* updating lost work

* updating CMake to use concepts

* some other changes to get it building (integration into MarkCache)

* further integration into caches

* updated Async metrics, fixed some build errors

* and some other errors revealing

* added perfect forwarding to some functions

* fix: forward template

* fix: constexpr modifier

* fix: FakePODAllocator missing member func

* updated PODArray constructor taking alloc params

* fix: PODArray overload with n restored

* fix: FakePODAlloc duplicating alloc() func

* added constexpr variable for alloc_tag_t

* split cache values by allocators, provided updates

* fix: memcpy

* fix: constexpr modifier

* fix: noexcept modifier

* fix: alloc_tag_t for PODArray constructor

* fix: PODArray copy ctor with different alloc

* fix: resize() signature

* updating to lastest working master

* syncing with 273267

* first draft version

* fix: update Searcher to case-insensitive

* added ILIKE test

* fixed style errors, updated test, split like and ilike,  added notILike

* replaced inconsistent comments

* fixed show tables ilike

* updated missing test cases

* regenerated ya.make

* Update 01355_ilike.sql

Co-authored-by: myrrc <me-clickhouse@myrrec.space>
Co-authored-by: alexey-milovidov <milovidov@yandex-team.ru>
 2020-07-05 18:57:59 +03:00
alexey-milovidov
e05283d5d3
Merge pull request #12060 from ClickHouse/fix-12053 
Check wrong type for filter.
 2020-07-04 23:46:34 +03:00
Anton Popov
73676f5022
Improve performace of reading in order of sorting key. (#11696) 
* simplify reading in order of sorting key

* add perf test for reading many parts

* Revert "simplify reading in order of sorting key"

This reverts commit 7267d7c46e.

* add threshold for preliminary merge for reading in order

* better threshold

* limit threads in test
 2020-07-04 15:48:51 +03:00
alesapin
8dc204350f
Fix version column in replicated version collapsing merge tree (#12121) 2020-07-04 10:35:17 +03:00
Anton Popov
ca346ea13c rename fsync-related settings 2020-07-03 02:41:37 +03:00
Anton Popov
4422df2e37 Merge remote-tracking branch 'upstream/master' into HEAD 2020-07-02 20:18:21 +03:00
Anton Popov
71059e4dff
Merge pull request #12062 from nvartolomei/nv/set-index-tuple-types 
Try fix pk in tuple performance
 2020-07-02 15:51:03 +03:00
Nikolai Kochetov
79cd33a567 Fix tests. 2020-07-02 10:44:47 +03:00
Nicolae Vartolomei
3854ce6d84 Rewrite Set lookup to make it more readable 2020-07-01 15:05:54 +01:00
Anton Popov
d45697a1b5 in-memory-parts: lazy creation of WAL 2020-06-30 21:47:12 +03:00
Nikolai Kochetov
3585700506 Fix tests. 2020-06-30 19:41:43 +03:00
Nicolae Vartolomei
8f1845185e Try fix pk in tuple performance 
Possible approach for fixing #10574

The problem is that prepared sets are built correctly, it is a hash map of key -> set
where key is a hash of AST and list of data types (when we a list of
tuples of literals).

However, when the key is built from the index to try and find if there
exists a prepared set that would match it looks for data types of the
primary key (see how data_types is populated) because the primary key
has only one field (v in my example) it can not find the prepared set.

The patch looks for any prepared indexes where data types match for the
subset of fields found in primary key, we are not interested in other
fields anyway for the purpose of primary key pruning.
 2020-06-30 16:33:38 +01:00
Nikolai Kochetov
66da0733ab Check type of filter for prewhere. 2020-06-30 17:20:27 +03:00
Vladimir Chebotarev
bb8da71eff Moves task shall be started if new storage policy needs them. 2020-06-30 15:36:44 +03:00
Anton Popov
53e955c6dd several fixes 2020-06-29 23:36:18 +03:00
Anton Popov
a43cb93be5 remove questionable functionality 2020-06-29 18:46:50 +03:00
Alexey Milovidov
cc597bb30b Implement for ReplicatedMergeTree 2020-06-28 22:39:31 +03:00
Alexey Milovidov
1f9c27de34 Fix clang-tidy 2020-06-28 22:39:31 +03:00
Alexey Milovidov
b6e31f0a5e Only allow Enum extension within the same width 2020-06-28 22:39:31 +03:00
Alexey Milovidov
e4d8cc2b44 Slightly more safe 2020-06-28 22:39:31 +03:00
Alexey Milovidov
97ad23b905 Allow to ALTER partition key in some cases 2020-06-28 22:39:31 +03:00
Alexey Milovidov
678d127503 Whitespace 2020-06-28 22:39:31 +03:00
Alexey Milovidov
c4d5ca485e Remove the usage of obsolete setting 2020-06-28 22:39:31 +03:00
Alexey Milovidov
1462a66d1e Fix typos 2020-06-27 22:05:00 +03:00
Anton Popov
b2aa565a37 allow to turn on fsync on inserts, merges and fetches 2020-06-27 17:52:16 +03:00
Alexey Milovidov
46caa211d5 Better diagnostics of "Replica {} appears to be already active" message 2020-06-27 16:55:00 +03:00
alesapin
9710a67704 Fix style check 2020-06-26 15:27:12 +03:00
alesapin
6caf2e7a3f Fix protocol check 2020-06-26 14:38:37 +03:00
alesapin
6f1824f0ea Correct merge with master 2020-06-26 14:30:23 +03:00
alesapin
e9c47dc89c Merge branch 'master' into CurtizJ-polymorphic-parts 2020-06-26 14:27:19 +03:00
Alexey Milovidov
8eed47857b Fix estimation of the number of marks for various thresholds 2020-06-25 23:20:22 +03:00
Alexey Milovidov
8872417d00 Respect direct_io/mmap settings while reading secondary indices 2020-06-25 22:31:54 +03:00
Alexey Milovidov
5608f15749 Revive mmap IO 2020-06-25 22:15:41 +03:00
Anton Popov
f8f67a788e allow to turn on fsync on inserts and merges 2020-06-25 19:55:45 +03:00
alesapin
63f8f89ca0
bump CI 2020-06-24 19:06:01 +03:00
alesapin
08f5ed6da3 Fix unitialized memory 2020-06-24 17:12:44 +03:00
Nikolai Kochetov
f7f16fdbf7
Merge pull request #11895 from ClickHouse/fix-nullable-prewhere-type-3 
Fix nullable prewhere type 3
 2020-06-24 14:32:08 +03:00
Anton Popov
8c3f496826 add test with alters and in-memory parts 2020-06-24 01:16:29 +03:00
Nikolai Kochetov
658a2d0dc0 Fix anotehr one nullable prewhere column. 2020-06-23 21:52:32 +03:00
alesapin
4185fe9975 Merge branch 'master' into atomic_metadata5 2020-06-23 11:13:05 +03:00
alesapin
292016e045 Add missed lock 2020-06-23 11:04:43 +03:00
Anton Popov
78d28be8cf check in-memory parts, comments and style-fixes 2020-06-22 21:57:50 +03:00
Nikolai Kochetov
ae21aca3a3 Fix header for nullable prewhere column. 2020-06-22 19:01:59 +03:00
alesapin
c76b4b3481 Clang-tidy fixes 2020-06-22 18:51:11 +03:00
alesapin
b1e8976df4 Merge with master 2020-06-22 12:04:27 +03:00
alesapin
b9e74f4e82 Merge branch 'master' into atomic_metadata5 2020-06-22 12:03:53 +03:00
Alexey Milovidov
0e6fb617aa Merge branch 'master' into manual-write-duplicate-parts-to-replicas 2020-06-22 01:54:03 +03:00
Alexey Milovidov
ad672f7ce2 Whitespace 2020-06-20 10:26:33 +03:00
Alexey Milovidov
b8769e1ada Fix style 2020-06-20 10:24:51 +03:00
alesapin
4c0879ae30 Better logging in storages 2020-06-19 20:17:13 +03:00
alesapin
a2e7e9f232 Remove unused constructor and rename method 2020-06-19 18:21:48 +03:00
alesapin
3c47faa9da Move partition key initialization into registerMergeTree 2020-06-19 18:14:08 +03:00
Alexey Milovidov
5211a42c04 Remove leader election, step 3 2020-06-19 17:18:58 +03:00
alesapin
c666763cc0 Remove unused method better 2020-06-18 20:09:06 +03:00
alesapin
e888dafdc2 Remove unused method 2020-06-18 20:03:42 +03:00
alesapin
d79982f497 Better locks in Storages 2020-06-18 19:10:47 +03:00
alesapin
d4c49816ab Really atomic metadata 2020-06-18 14:02:31 +03:00
alesapin
ebb36bec8a Merge branch 'master' into atomic_metadata5 2020-06-18 11:57:16 +03:00
Azat Khuzhin
caa88777be Parse metadata stored in zookeeper before checking for equality 
After #11325 trailing whitespaces has been removed for data skipping
indicies, and it may be different, if you have multiple skip indices,
and in this case new server will not load such tables, because metadata
will be different.

Fix this by re-parse metadata in zookeeper.
 2020-06-17 21:03:22 +03:00
alesapin
dffdece350 getColumns in StorageInMemoryMetadta (only compilable) 2020-06-17 19:39:58 +03:00
alesapin
33c27de54d Check methods in metadata 2020-06-17 17:32:25 +03:00
alesapin
eaaef83742 Settings changes in StorageInMemoryMetadata 2020-06-17 16:46:01 +03:00
alesapin
ed8f3b2fc4 TTL in storage in memory metadata 2020-06-17 16:39:26 +03:00
alesapin
1afdebeebd Primary key in storage metadata 2020-06-17 15:39:20 +03:00
alesapin
1da393b218 Sampling key in StorageInMemoryMetadata 2020-06-17 15:07:09 +03:00
alesapin
eca6e9087e Fix race condition 2020-06-17 14:52:19 +03:00
alesapin
ba04d02f1e Compilable sorting key in metadata 2020-06-17 14:05:11 +03:00
alesapin
ab61abccc1 Partition key in StorageInMemoryMetadata 2020-06-17 13:34:23 +03:00
alesapin
62f2c17a66 Secondary indices in StorageInMemoryMetadata 2020-06-17 12:38:47 +03:00
alesapin
2fac3290f0 Merge branch 'master' into atomic_metadata5 2020-06-17 11:14:03 +03:00
alexey-milovidov
e07d02015c
Merge pull request #11661 from ClickHouse/return-not-nullable-from-count-distinct-2 
Return non-Nullable results from COUNT(DISTINCT), more complete.
 2020-06-17 09:25:45 +03:00
alexey-milovidov
598fd75caf
Merge pull request #11639 from ClickHouse/remove-leader-election-2 
Remove leader election, step 2: allow multiple leaders
 2020-06-17 09:25:23 +03:00
Alexey Milovidov
0a6054eece Merge branch 'master' into return-not-nullable-from-count-distinct-2 2020-06-16 22:31:33 +03:00
alesapin
d214a504d7
Merge pull request #11681 from ClickHouse/fix_mutations_premature_finish 
Fix mutations premature "finish"
 2020-06-16 21:26:14 +03:00
alesapin
ccc2bda666 getConstraints() in StorageInMemoryMetadata (suspicious commit, but pretend to work) 2020-06-16 19:55:04 +03:00
alesapin
1ddeb3d149 Buildable getSampleBlock in StorageInMemoryMetadata 2020-06-16 18:51:29 +03:00
alesapin
71f99a274d Compileable getSampleBlockWithColumns in StorageInMemoryMetadata 2020-06-16 17:25:08 +03:00
alesapin
0bcd22008a Get column dependencies in StorageInMemoryMetadata 2020-06-16 15:19:21 +03:00
alesapin
eae901f58d Merge branch 'master' into atomic_metadata5 2020-06-16 14:44:45 +03:00
alesapin
0a33ccb77b Fix remove method 2020-06-16 14:12:52 +03:00
alesapin
d1052f2dfe Better logic 2020-06-16 13:50:47 +03:00
alesapin
d5b76b2fe1 Fix parts count in mutations 2020-06-16 13:34:59 +03:00
alesapin
0cb14c836c
Merge pull request #11691 from ClickHouse/fix-race-condition-system-sync-replicas 
Fix race condition in SYSTEM SYNC REPLICA
 2020-06-16 13:29:18 +03:00
alesapin
362aaf1385
Merge pull request #11516 from ClickHouse/consistent_metadata4 
Storage metadata as a single struct.
 2020-06-16 13:28:22 +03:00
alesapin
605c904fc6 Add method for compatibility 2020-06-16 11:39:12 +03:00
Anton Popov
b19d48a11c Merge remote-tracking branch 'upstream/master' into HEAD 2020-06-16 06:37:55 +03:00
Alexey Milovidov
125b007360 Merge branch 'master' into remove-leader-election-2 2020-06-16 05:56:29 +03:00
Alexey Milovidov
22a92faab6 Avoid connection to replica when fetches are cancelled 2020-06-16 05:14:53 +03:00
Alexey Milovidov
4360e326ed Fix clang-static-analyzer 2020-06-16 04:17:02 +03:00
Alexey Milovidov
6da1a0f153 Fix style 2020-06-16 04:13:45 +03:00
Alexey Milovidov
53d985909e Fix race condition in SYSTEM SYNC REPLICA 2020-06-16 01:54:19 +03:00
Anton Popov
a3ac224ae4 in-memory parts: sync insert to replicated 2020-06-15 22:57:12 +03:00
Alexey Milovidov
d10109dc38 Remove excessive statement #11131 2020-06-15 22:18:33 +03:00
alesapin
36ba0192df Metadata in read and write methods of IStorage 2020-06-15 22:08:58 +03:00
Alexey Milovidov
e44e1ad0d4 Don't ignore duplicate parts written to replicas 2020-06-15 21:57:38 +03:00
alesapin
b47a7327fd All set methods in metadata 2020-06-15 21:08:05 +03:00
alesapin
33a74a3ea0 TTL methods in StorageInMemoryMetadata 2020-06-15 20:50:53 +03:00
alesapin
e667eb57b2 Working set*Keys methods 2020-06-15 20:24:49 +03:00
alesapin
5fc41c7ecc Move set*Key methods to StorageInMemoryMetadata 2020-06-15 20:17:06 +03:00
alesapin
aa30649ce5 Working setColumns, setConstraints, setIndices 2020-06-15 20:10:14 +03:00
alesapin
af2fe2ba55 Compilable setColumns, setConstraints, setIndices 2020-06-15 19:55:33 +03:00
alesapin
8945d0073a Fix misunderstanding bug in mutations finalization 2020-06-15 16:37:40 +03:00
alesapin
bc9a3c0458 Merge branch 'master' into consistent_metadata4 2020-06-15 11:20:30 +03:00
alexey-milovidov
fd6115cccf
Merge pull request #11648 from ClickHouse/simplify-code-functions-comparison-3 
Allow comparison with String in index analysis; simplify code.
 2020-06-15 05:47:03 +03:00
Alexey Milovidov
5866401f60 Less noise in cleanup thread 2020-06-15 05:12:06 +03:00
Alexey Milovidov
689b6901f8 Fix typo 2020-06-15 04:04:42 +03:00
Alexey Milovidov
1c438a133e Leader election both backward and forward compatible 2020-06-15 03:59:12 +03:00
Alexey Milovidov
b596831342 Merge branch 'master' into remove-leader-election-2 2020-06-15 03:16:41 +03:00
Alexey Milovidov
8dac30ae95 Split file for better build times 2020-06-14 21:42:10 +03:00
Alexey Milovidov
f6c52fe1c2 Allow comparison with String in index analysis; simplify code #11630 2020-06-14 21:31:42 +03:00
Alexey Milovidov
394fb64a9c Better way of implementation 2020-06-14 20:42:11 +03:00
Alexey Milovidov
2cd82a25f5 Remove trivial count query optimization if row-level security is set #11352 2020-06-13 23:59:20 +03:00
alexey-milovidov
f53da4d36b
Merge pull request #11608 from ClickHouse/fix-nullable-prewhere-type 
Fix nullable prewhere type
 2020-06-13 20:29:12 +03:00
alesapin
18b58e8483 Revert too strict lock 2020-06-13 12:12:45 +03:00
alesapin
5d3de747cd Merge remote-tracking branch 'origin/master' into consistent_metadata4 2020-06-13 11:50:27 +03:00
Alexey Milovidov
6f0db5ef10 Fix error 2020-06-13 04:35:49 +03:00
Alexey Milovidov
6ff671b092 Improvement 2020-06-13 04:35:49 +03:00
Alexey Milovidov
ab00e34305 Miscellaneous 2020-06-13 04:35:49 +03:00
Alexey Milovidov
85c0706901 Step 2: allow multiple leaders 2020-06-13 04:35:49 +03:00
Alexey Milovidov
b8b55a5b99 More LeaderElection to Storage/MergeTree 2020-06-13 04:35:49 +03:00
Alexey Milovidov
52ac009754 Update versions on merge and mutation 2020-06-13 04:34:05 +03:00
Alexey Milovidov
6869c122a8 Step 1 towards removing leader election: check and modify version of the "log" node 2020-06-13 04:34:05 +03:00
Alexey Milovidov
72257061d5 Avoid errors due to implicit int<->bool conversions when using ZK API 2020-06-12 18:09:12 +03:00
Nikolai Kochetov
83155e139c Try fix tests. 2020-06-12 17:59:14 +03:00
alesapin
0edf5ff7a2 Fix race condition 2020-06-12 17:32:47 +03:00
alesapin
465c4b65b7 Slightly better interfaces and comments 2020-06-12 12:37:52 +03:00
Nikolai Kochetov
7b8c1657a5 Fix nullable prewhere column type. 2020-06-11 19:34:36 +03:00