Pavel Kruglov
9dbced0474
Pass setting instead of context
2020-12-04 17:01:59 +03:00
Anton Popov
cd1917c7a6
Merge branch 'master' into optimize_final_optimization
2020-12-03 16:52:51 +03:00
alexey-milovidov
f4a61ac3c3
Merge pull request #17527 from ucasFL/spelling
...
fix spelling errors
2020-11-29 13:45:42 +03:00
feng lv
7e3524caa1
fix spelling errors
2020-11-28 08:17:20 +00:00
nikitamikhaylov
72c7cd6693
replace Context& to Settings&
2020-11-25 16:47:32 +03:00
nikitamikhaylov
68bef22fda
Merge branch 'master' of github.com:ClickHouse/ClickHouse into merging-sequential-consistency
2020-11-23 16:28:35 +03:00
Pavel Kruglov
ca3fe49a2a
Make setting global
2020-11-20 17:29:13 +03:00
Nicolae Vartolomei
94293ca3ce
Assign UUIDs to parts only when configured to do so
...
Avoid breaking backwards compatibility by default for now.
2020-11-20 13:49:17 +00:00
Pavel Kruglov
4c30857759
Minor change
2020-11-20 01:22:40 +03:00
Nicolae Vartolomei
746f8e45f5
All new parts must have uuids
2020-11-19 13:18:03 +00:00
Pavel Kruglov
547ec19fb3
Merge branch 'master' of github.com:ClickHouse/ClickHouse into optimize-data-on-insert
2020-11-18 12:01:59 +03:00
Pavel Kruglov
8d5e0784d3
Add setting optimize_on_insert
2020-11-12 23:37:23 +03:00
Anton Popov
4cecd89e84
fix dropping of empty parts
2020-11-12 20:37:54 +03:00
Anton Popov
0aa35d53c7
Merge remote-tracking branch 'upstream/master' into HEAD
2020-11-12 14:31:58 +03:00
alesapin
2cac5e5d97
Merge pull request #16875 from CurtizJ/refactor-alter-partition
...
Avoid code duplication in alterPartition
2020-11-12 13:19:59 +03:00
tavplubix
058aa8f85e
Merge pull request #16824 from ClickHouse/replace_stringstreams_with_buffers
...
Replace std::*stringstreams with DB::*Buffers
2020-11-12 01:11:44 +03:00
Anton Popov
2497598a49
Remove empty parts after they were pruned by TTL
2020-11-11 19:19:43 +03:00
Anton Popov
6f0dc08ba7
avoid code duplication in alterPartition
2020-11-11 16:34:07 +03:00
Pavel Kruglov
287d71b363
Fix build
2020-11-11 13:34:32 +03:00
Pavel Kruglov
dd06ff4f94
Minor change
2020-11-11 00:02:11 +03:00
Pavel Kruglov
1c35b47073
Add checking select decision in merge
2020-11-10 23:52:26 +03:00
Pavel Kruglov
c648f62629
Merge branch 'master' of github.com:ClickHouse/ClickHouse into optimize_final_optimization
2020-11-10 22:58:21 +03:00
Alexander Tokmakov
b94cc5c4e5
remove more stringstreams
2020-11-10 21:22:26 +03:00
Pavel Kruglov
9120189d8a
Add SelectPartsDecision enum class
2020-11-10 17:42:56 +03:00
Nikolai Kochetov
195c941c4e
Merge branch 'master' into storage-read-query-plan
2020-11-10 15:02:22 +03:00
Nikolai Kochetov
046830967c
Try fix tests.
2020-11-10 13:26:26 +03:00
Alexander Tokmakov
62ff00ee8b
use WriteBuffer in formatAST(...)
2020-11-09 19:05:40 +03:00
alesapin
72046313d4
Merge branch 'master' into nvartolomei-drop-part
2020-11-09 10:02:02 +03:00
alexey-milovidov
0e6ae4aff7
Merge pull request #16253 from amosbird/pf
...
Prune partition in verbatim way.
2020-11-08 18:58:02 +03:00
alexey-milovidov
f4ba5f1f9a
Merge pull request #16772 from ClickHouse/fix-stringstream
...
Fix "server failed to start" error
2020-11-08 14:27:08 +03:00
Alexey Milovidov
5314185e25
Merge branch 'master' into azat-optimize_skip_unused_shards-optimization
2020-11-08 00:17:59 +03:00
Alexey Milovidov
fd84d16387
Fix "server failed to start" error
2020-11-07 03:14:53 +03:00
Nikolai Kochetov
c10f733587
Merge branch 'master' into storage-read-query-plan
2020-11-06 15:43:46 +03:00
Amos Bird
2b0085c106
Pruning is different from counting
2020-11-06 19:58:03 +08:00
Amos Bird
30bf5e6d26
Prune partition in verbatim way.
2020-11-06 09:56:13 +08:00
alesapin
da8ac948bf
Disallow to drop parts with parallel quorum
2020-11-03 12:24:10 +03:00
alesapin
dac6705995
DROP PART and clear blocks in a single zookeeper op
2020-11-02 20:30:59 +03:00
alesapin
67d22b3e57
Merge branch 'master' into nvartolomei-drop-part
2020-11-02 19:09:09 +03:00
hchen9
feaab260d8
Fix conflict
2020-10-23 12:54:33 -07:00
alesapin
39e47c5338
Less strange lambdas
2020-10-23 11:54:00 +03:00
alesapin
7a63b373f9
Merge branch 'blind_fix_of_read_in_order' into no_background_pool_no_more
2020-10-22 15:47:11 +03:00
Nikolai Kochetov
7fa045cff8
Merge branch 'master' into storage-read-query-plan
2020-10-22 13:31:10 +03:00
alexey-milovidov
adeba6bdd8
Merge pull request #15074 from amosbird/btc
...
Extend trivial count optimization.
2020-10-22 02:50:57 +03:00
alesapin
c77b154398
Better locks in merge tree
2020-10-21 00:10:55 +03:00
alesapin
ce8ee01d75
Fix background moves start
2020-10-20 14:27:50 +03:00
alesapin
d126aa8a09
Merge branch 'master' into no_background_pool_no_more
2020-10-20 12:54:42 +03:00
alesapin
514a95c292
Fixup
2020-10-19 12:06:51 +03:00
Alexander Tokmakov
026896835c
fixes
2020-10-16 15:20:30 +03:00
alesapin
7edbe4d276
Add some comments
2020-10-16 13:12:31 +03:00
alesapin
4c1f5cab13
Sleep less in background task
2020-10-15 19:10:22 +03:00
Pavel Kruglov
25c39689c3
Check expired ttl
2020-10-15 15:15:02 +03:00
alesapin
84b0c22044
fix possible deadlock
2020-10-15 13:54:50 +03:00
alesapin
c5419083e3
Better method name
2020-10-15 10:43:50 +03:00
alesapin
7265792510
Less files
2020-10-14 21:33:41 +03:00
alesapin
a535a089d1
Some style fixes
2020-10-14 19:09:18 +03:00
alesapin
483893cdd4
Slightly refactored code
2020-10-14 17:56:42 +03:00
alesapin
0b14a31ba9
Get rid of background processing pool
2020-10-14 15:44:10 +03:00
alesapin
4014e0f08d
Something very similar to working code
2020-10-14 15:32:35 +03:00
alesapin
efd3126b5d
Moving pool
2020-10-14 10:22:48 +03:00
Pavel Kruglov
1ccf10505a
add tests and comments
2020-10-13 21:26:40 +03:00
Pavel Kruglov
6d9b310f61
Add optimize_skip_merged_partitions
2020-10-13 21:26:35 +03:00
alesapin
fd35368c59
First simple implementation
2020-10-13 17:25:42 +03:00
alesapin
dcf489ac8f
Remove unused methods
...
(cherry picked from commit 77f5841a46
2020-10-13 15:10:59 +03:00
alesapin
ef355e71d6
Buildable code
2020-10-13 15:10:46 +03:00
alesapin
48aaa0ce16
Split merge to separate method
...
(cherry picked from commit 14a7a25c4b
2020-10-13 15:06:32 +03:00
alesapin
cb7db6d232
Split select and process merges
...
(cherry picked from commit 1889eb0ff1
2020-10-13 15:04:16 +03:00
Nikolai Kochetov
7e58f99f64
Merge branch 'master' into storage-read-query-plan
2020-10-12 13:12:39 +03:00
Pervakov Grigorii
3a2068f19d
Use tmp disk for vertical merge files
2020-10-08 13:49:41 +03:00
Amos Bird
867216103f
Extend trivial count optimization.
2020-10-08 18:08:17 +08:00
hchen9
ed2ce4a6fc
Resolve conflict in StorageMemory
2020-10-07 17:19:59 -07:00
Nikolai Kochetov
7e02152b50
Merge branch 'master' into storage-read-query-plan
2020-10-07 14:27:21 +03:00
Nikolai Kochetov
c5cb05f5f3
Try fix tests.
2020-10-07 14:26:29 +03:00
alesapin
8ec58c17f3
More consistent metadata usage
2020-10-05 19:41:46 +03:00
Azat Khuzhin
b838214a35
Pass non-const SelectQueryInfo (and drop mutable qualifiers)
2020-10-02 22:42:35 +03:00
hchen9
b3949db00f
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into trivial_count_seq_consistency
2020-10-01 21:15:26 -07:00
Nikolai Kochetov
ec64def384
Use QueryPlan while reading from MergeTree.
2020-10-01 20:34:22 +03:00
hchen9
a5ac39b564
Pass Context parameter for IStorage.totalRows and IStorage.totalBytes
2020-09-30 16:47:42 -07:00
alesapin
f5c38b34ae
Fix race condition during write ahead log cleanup and rename
2020-09-25 15:07:38 +03:00
Nicolae Vartolomei
97d0b5ab23
Remove unused query argument
2020-09-08 10:56:10 +01:00
Nicolae Vartolomei
1876374770
ALTER TABLE ... DROP|DETACH PART for MergeTree
2020-09-08 10:56:09 +01:00
alesapin
82c56349a5
Some comments
2020-09-04 13:08:09 +03:00
alesapin
61ecaebcb1
Simplify settings for TTL merges
2020-09-04 09:55:19 +03:00
alesapin
f4c7ff0376
Add fixed size of Merge TTLS
2020-09-03 16:00:13 +03:00
alesapin
232c264d7d
Disable force TTL on optimise
2020-08-28 16:45:42 +03:00
alexey-milovidov
0e3b9d0da6
Merge pull request #13747 from ClickHouse/fix-race-condition-in-test
...
Fix race condition in test
2020-08-15 15:55:47 +03:00
Alexey Milovidov
9e6d4802ac
Do not drop cache if DROP/RENAME in Atomic database
2020-08-15 09:56:47 +03:00
Alexey Milovidov
ee084f0ec9
Fix race condition in DETACH and background merge
2020-08-15 08:21:02 +03:00
Nikolai Kochetov
09fbce1b1e
Merge branch 'master' into refactor-pipes-3
2020-08-04 11:32:34 +03:00
Nikolai Kochetov
2cca4d5fcf
Refactor Pipe [part 2].
2020-08-03 16:54:14 +03:00
alesapin
45894fcf99
Better comments
2020-07-31 15:22:32 +03:00
alexey-milovidov
bbc8473388
Update StorageMergeTree.cpp
2020-07-31 15:00:07 +03:00
alesapin
14928874a1
Fix mutations status check
2020-07-31 14:37:16 +03:00
alexey-milovidov
fc35ce69fa
Merge pull request #12666 from ClickHouse/forward_alter_error_to_client
...
Forward exceptions to client if exception happened during ALTER
2020-07-30 21:36:06 +03:00
alesapin
7a806eb90f
Merge pull request #13017 from ClickHouse/freeze_partition_with_name
...
Alter table freeze with verbose output
2020-07-29 18:42:48 +03:00
alesapin
ba8a29a3a4
Better logging on mutation skip
2020-07-29 16:12:55 +03:00
alesapin
7fc1e45c3b
Review fixe
2020-07-29 11:32:52 +03:00
alesapin
c75830e878
Alter table freeze with verbose output
2020-07-28 18:10:36 +03:00
alesapin
355150afce
Fix deadlock and better code
2020-07-22 22:29:54 +03:00
alesapin
8c8bdd5070
Fix tests and work with multiple mutations
2020-07-22 18:19:54 +03:00
alesapin
7585fc4b52
Fix style
2020-07-22 16:22:05 +03:00
alesapin
de846e5e3c
Forward exception to client during alter
2020-07-22 15:36:19 +03:00
alesapin
4a53264a86
Remove redundant and duplicated code
2020-07-13 19:19:08 +03:00
Anton Popov
416d9648c2
remove questionable functionality
2020-07-07 12:58:14 +03:00
Anton Popov
4422df2e37
Merge remote-tracking branch 'upstream/master' into HEAD
2020-07-02 20:18:21 +03:00
Vladimir Chebotarev
bb8da71eff
Moves task shall be started if new storage policy needs them.
2020-06-30 15:36:44 +03:00
Anton Popov
a43cb93be5
remove questionable functionality
2020-06-29 18:46:50 +03:00
Alexey Milovidov
97ad23b905
Allow to ALTER partition key in some cases
2020-06-28 22:39:31 +03:00
alesapin
6f1824f0ea
Correct merge with master
2020-06-26 14:30:23 +03:00
alesapin
e9c47dc89c
Merge branch 'master' into CurtizJ-polymorphic-parts
2020-06-26 14:27:19 +03:00
alesapin
292016e045
Add missed lock
2020-06-23 11:04:43 +03:00
alesapin
c666763cc0
Remove unused method better
2020-06-18 20:09:06 +03:00
alesapin
e888dafdc2
Remove unused method
2020-06-18 20:03:42 +03:00
alesapin
54e5fe7dbc
Less locks
2020-06-18 19:19:40 +03:00
alesapin
d79982f497
Better locks in Storages
2020-06-18 19:10:47 +03:00
alesapin
aab4ce6394
Truncate with metadata
2020-06-18 13:29:13 +03:00
alesapin
ed8f3b2fc4
TTL in storage in memory metadata
2020-06-17 16:39:26 +03:00
alesapin
eca6e9087e
Fix race condition
2020-06-17 14:52:19 +03:00
alesapin
ab61abccc1
Partition key in StorageInMemoryMetadata
2020-06-17 13:34:23 +03:00
alesapin
1ddeb3d149
Buildable getSampleBlock in StorageInMemoryMetadata
2020-06-16 18:51:29 +03:00
alesapin
71f99a274d
Compileable getSampleBlockWithColumns in StorageInMemoryMetadata
2020-06-16 17:25:08 +03:00
alesapin
0bcd22008a
Get column dependencies in StorageInMemoryMetadata
2020-06-16 15:19:21 +03:00
Anton Popov
b19d48a11c
Merge remote-tracking branch 'upstream/master' into HEAD
2020-06-16 06:37:55 +03:00
alesapin
36ba0192df
Metadata in read and write methods of IStorage
2020-06-15 22:08:58 +03:00
alesapin
33a74a3ea0
TTL methods in StorageInMemoryMetadata
2020-06-15 20:50:53 +03:00
alesapin
ec933d9d03
Better naming
2020-06-10 12:09:51 +03:00
alesapin
d2fcf5aea5
Fixes for gcc
2020-06-09 20:28:29 +03:00
alesapin
f7a397242f
Merge branch 'master' into consistent_metadata4
2020-06-09 18:59:59 +03:00
alesapin
223f45685f
Review fixes
2020-06-09 05:03:22 +03:00
Anton Popov
66e31d4311
in-memory parts: several fixes
2020-06-05 23:49:24 +03:00
alesapin
abaf47f0cd
Make metadata single structure
2020-06-05 20:29:40 +03:00
Nikita Mikhaylov
4e4218e10d
Merge pull request #11385 from ClickHouse/akz/bg-processing-pool-update1
...
Extended BackgroundProcessingPool::startTask() to control task execution concurrency
2020-06-04 16:19:29 +04:00
Alexey Milovidov
484d269c22
Fix typo
2020-06-04 01:15:13 +03:00
Alexey Milovidov
a0fd9b021a
Fix MSan failure in MergeTree
2020-06-04 01:11:06 +03:00
Anton Popov
11c4e9dde3
in-memory parts: fix 'check' query
2020-06-03 22:19:49 +03:00
Anton Popov
1ce09e1faa
Merge remote-tracking branch 'upstream/master' into polymorphic-parts
2020-06-03 16:27:54 +03:00
Alexander Kuzmenkov
c7d9094a7a
Merge pull request #11259 from ClickHouse/consistent_metadata3
...
More consistent metadata for secondary indices
2020-06-03 12:23:21 +03:00
Alexander Kazakov
9fbd72ab34
Renamed method
2020-06-02 18:18:36 +03:00
alexey-milovidov
d47d0d7108
Merge pull request #11346 from ClickHouse/optimize-final-should-force-merge-2
...
OPTIMIZE FINAL should force merge even if concurrent merges are performed (experimental, try 2)
2020-06-02 16:40:20 +03:00
alesapin
e5eb6071a4
Merge with master
2020-06-02 10:46:58 +03:00
Alexey Milovidov
78d966f4e5
Better code
2020-06-02 03:41:52 +03:00
alesapin
f287a28fc5
Stop merges before taking exclusive lock :(
2020-06-01 18:16:10 +03:00
alesapin
3847ea892d
Merge branch 'master' into consistent_metadata3
2020-06-01 13:17:59 +03:00
Anton Popov
4769ce7271
in-memory parts: several fixes
2020-05-29 02:59:13 +03:00
Alexander Tokmakov
c071e0a473
add context to DatabaseCatalog methods
2020-05-29 02:01:18 +03:00
alesapin
52ca6b2051
I'm able to build it
2020-05-28 15:37:05 +03:00
Anton Popov
d8342e5b12
in-memory parts: better restore and clear stale wal files
2020-05-27 23:05:55 +03:00
Alexey Milovidov
7e1813825b
Return old names of macros
2020-05-24 01:24:01 +03:00
Alexey Milovidov
af2a59e49b
Progress on task
2020-05-24 00:50:34 +03:00
Alexey Milovidov
bab24879e9
Progress on task
2020-05-24 00:16:05 +03:00
Alexey Milovidov
e1695feb7f
Apply all transformations again
2020-05-23 23:40:32 +03:00
Alexey Milovidov
85f84550ba
Progress on task
2020-05-23 23:37:37 +03:00
Alexey Milovidov
8042e5febe
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "\3{}\5{}", \4, \6);/'
2020-05-23 19:58:15 +03:00
Alexey Milovidov
e391b77d81
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+"\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)"\);/\1_FORMATTED(\2, "\3{}\5", \4);/'
2020-05-23 19:56:05 +03:00
Alexey Milovidov
ee4ffbc332
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "\3{}", \4);/'
2020-05-23 19:47:56 +03:00
Alexey Milovidov
8d2e80a5e2
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+"\)' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+, "[^"]+")\)/\1_FORMATTED(\2)/'
2020-05-23 19:42:39 +03:00
alesapin
b9dfa61774
Merge pull request #11042 from CurtizJ/modify-ttl
...
Apply TTL for old data, after ALTER MODIFY TTL query
2020-05-22 11:12:14 +03:00
Azat Khuzhin
d93b9a57f6
Forward declaration for Context as much as possible.
...
Now after changing Context.h 488 modules will be recompiled instead of 582.
2020-05-21 01:53:18 +03:00
Anton Popov
d62fe980ef
Merge remote-tracking branch 'upstream/master' into modify-ttl
2020-05-20 16:52:30 +03:00
Anton Popov
4fb6492b08
in-memory parts: delay merges
2020-05-20 15:02:02 +03:00
Anton Popov
58ed04dc24
materialize TTL after its modifying
2020-05-19 04:53:01 +03:00
alesapin
07cb21ccb7
Merge branch 'master' into refactor-reservations
2020-05-18 11:43:48 +03:00
Alexey Milovidov
70c31cbdc4
Fix error
2020-05-17 22:44:31 +03:00
Alexey Milovidov
20dbd6c3ef
Fix error
2020-05-17 22:41:49 +03:00
Alexey Milovidov
88ee76186b
Try to fix use-after-free error in MergeTree
2020-05-17 07:12:33 +03:00
Anton Popov
14e1a46a8c
Merge remote-tracking branch 'upstream/master' into polymorphic-parts
2020-05-15 03:48:27 +03:00
Gleb Novikov
d255bb998a
VolumePtr instead of DiskPtr in IMergedBlockOutputStream and inherited classes
2020-05-11 20:20:26 +03:00
Gleb Novikov
57b6fb6d80
Merge branch 'master' into refactor-reservations
2020-05-10 13:01:45 +03:00
Gleb Novikov
390b39b272
VolumePtr instead of DiskPtr in MergeTreeData*
2020-05-10 00:24:15 +03:00
Vladimir Chebotarev
28631f461e
Fixed handling of condition variable for synchronous mutations.
2020-05-08 12:01:06 +03:00
Anton Popov
387041a97e
Merge remote-tracking branch 'upstream/master' into polymorphic-parts
2020-05-06 19:22:07 +03:00
Gleb Novikov
c637d99e07
Volumes and storages refactoring:
...
1. Moved Volume to separate file
2. Created IVolume interface and implemented current behaviour in implementation of new interface — VolumeJBOD
3. Replaced all old volume usages with new VolumeJBOD. Where it is unnecessary to have JBOD — left just IVolume.
4. Removed old Volume completely
5. Moved StoragePolicy to separated files
6. Moved DiskSelector to separated files
7. Removed DiskSpaceMonitor file
2020-05-04 23:15:38 +03:00
Anton Popov
aa723d245f
Merge remote-tracking branch 'upstream/master' into polymorphic-parts
2020-04-30 18:27:39 +03:00
Alexander Tokmakov
04d6b59ac0
Merge branch 'master' into database_atomic
2020-04-23 17:31:37 +03:00
Anton Popov
4069dbcc58
in-memory parts: add waiting for insert
2020-04-20 04:38:38 +03:00
Nikita Mikhaylov
c153cc8fe5
fixed :)
2020-04-16 22:54:11 +03:00
Alexander Tokmakov
5e6d4b9449
Merge branch 'master' into database_atomic
2020-04-12 16:35:44 +03:00
Alexander Kazakov
497df3086f
Merge branch 'master' into timed_rwlock
...
Change-Id: I620bfde2121ff013773b001d514b40b1e796a58b
2020-04-10 11:38:20 +03:00
Alexander Kazakov
26dd6140b2
Added new config settings to control timeouts
...
* "lock_acquire_timeout" controls for how long a query will continue to
acquire each lock on its argument tables
* "lock_acquire_timeout_for_background_operations" is a per-table
setting for storages of *MergeTree family
2020-04-09 21:10:27 +03:00
Alexander Tokmakov
dd1590830b
Merge branch 'master' into database_atomic
2020-04-08 22:00:46 +03:00
Alexander Tokmakov
08bae4668d
Merge branch 'master' into database_atomic
2020-04-06 16:18:07 +03:00
alesapin
1cb072d58a
Merge branch 'master' into alter_rename_column
2020-04-06 11:40:27 +03:00
Ivan Lezhankin
06446b4f08
dbms/ → src/
2020-04-03 18:14:31 +03:00
