Alexander Tokmakov
5e622d4c98
fix another issue
2021-10-27 15:59:26 +03:00
alesapin
2af950d4d5
Merge pull request #30658 from ClickHouse/std_mutex_alter_lock
...
Change `alter_lock` from `RWLock` to `std::timed_mutex`
2021-10-27 11:14:11 +03:00
alesapin
a29711f1d0
Merge pull request #30651 from ClickHouse/fix_intersecting_parts_to_do
...
Fix intersecting parts in `parts_to_do`
2021-10-27 10:52:21 +03:00
alesapin
8749f4a31a
Merge pull request #29043 from nvartolomei/nv/parts-uuid-move-shard-kill
...
Part movements between shards improvements and cancel support
2021-10-27 10:46:00 +03:00
Dmitry Novik
e971871a95
Merge pull request #30668 from azat/prewhere-always-true-where
...
Fix PREWHERE with WHERE in case of always true PREWHERE
2021-10-26 17:18:50 +03:00
Alexander Tokmakov
6be1e98342
fix
2021-10-26 17:06:48 +03:00
mergify[bot]
ce88a84e88
Merge branch 'master' into nv/parts-uuid-move-shard-kill
2021-10-26 11:09:19 +00:00
Alexander Tokmakov
2e7e195e77
change alter_lock to std::timed_mutex
2021-10-26 13:37:00 +03:00
Azat Khuzhin
16cd70aee9
Fix PREWHERE with WHERE in case of always true PREWHERE
2021-10-26 00:02:41 +03:00
Alexander Tokmakov
5cdd9a4b99
fix intersecting parts in parts_to_do
2021-10-25 19:00:55 +03:00
Alexander Tokmakov
94088f8197
minor fixes
2021-10-25 17:01:36 +03:00
Alexander Tokmakov
075c3a37f4
Merge branch 'master' into fix_some_fetches_may_stuck
2021-10-25 15:28:54 +03:00
alesapin
28bde879ed
Merge pull request #30455 from kssenii/deprecate-setting
...
Deprecate write_final_mark setting
2021-10-25 10:51:49 +03:00
kssenii
52296e8b04
Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read
2021-10-25 00:22:32 +03:00
kssenii
13a2a3d307
Merge branch 'master' of github.com:ClickHouse/ClickHouse into deprecate-setting
2021-10-24 23:05:21 +03:00
kssenii
658b33717f
A little nicer
2021-10-24 20:43:34 +03:00
Azat Khuzhin
f9b27608f5
Fix reading of columns if offset_in_compressed_file is the same for mark range
...
Fixes: 00078_group_by_arrays
2021-10-24 18:58:22 +03:00
Azat Khuzhin
93c34824c0
Fix explicit 0 size for createReadBufferFromFileBase()
2021-10-24 18:58:22 +03:00
Azat Khuzhin
4599e9191b
Fix explicit 0 size for readFile()
2021-10-24 18:58:22 +03:00
Kseniia Sumarokova
153dd9a895
Update MergeTreeSettings.h
2021-10-22 16:27:01 +03:00
kssenii
39b9e9c258
Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read
2021-10-22 15:40:41 +03:00
Azat Khuzhin
496aa43a9f
Remove check_columns argument from MergeTree code (false was never passed)
2021-10-21 21:24:59 +03:00
kssenii
56f0560c46
Asserts and read till end option
2021-10-21 20:54:03 +03:00
Dmitry Novik
58ee783be4
Merge pull request #30293 from amosbird/projection-fix19
...
Fix projections with JOIN alias columns
2021-10-21 17:49:21 +03:00
Anton Popov
b1a74af44d
Merge pull request #30180 from CurtizJ/remove-sample-by
...
Allow to remove `SAMPLE BY` expression
2021-10-21 17:09:53 +03:00
kssenii
e4e157688d
Review fixes
2021-10-21 01:57:43 +03:00
Nikita Mikhaylov
56869c0ec4
Merge pull request #30360 from nikitamikhaylov/allow-graphite-final
...
Allow to use FINAL clause in SELECT from GraphiteMergeTree
2021-10-20 19:57:44 +03:00
kssenii
0a90be8317
Remove write_final_mark setting
2021-10-20 14:13:04 +00:00
Amos Bird
2f615e9176
Use original_query for projection analysis
2021-10-20 21:30:15 +08:00
Nikolai Kochetov
8986de7e50
Merge pull request #30384 from amosbird/projection-fix21
...
Fix crash when minmax_count projection is used with primary key in partition expr
2021-10-20 16:17:13 +03:00
Kseniia Sumarokova
5324cc8359
Merge pull request #30282 from kssenii/fix-local-less-threads
...
Less threads in clickhouse-local, fix Ok. printing
2021-10-19 22:54:28 +03:00
Amos Bird
05d93796dc
Fix minmax_count projection with primary key in partition expr
2021-10-19 23:36:07 +08:00
Nikita Mikhaylov
75487be899
White list of storages that supports final
2021-10-19 12:16:33 +00:00
Nikita Mikhaylov
4fbd332bf1
Done
2021-10-19 11:53:55 +00:00
Nikita Mikhaylov
ea2a647724
Merge pull request #30192 from azat/granula-adjust-LOGICAL_ERROR-fix
...
Fix reading from MergeTree with max_read_buffer_size=0
2021-10-19 13:25:55 +03:00
kssenii
e53335bc6f
Better way
2021-10-19 08:21:38 +00:00
Nikolai Kochetov
cc7a3ea9ff
Merge pull request #30322 from ClickHouse/update-mergine-sorted-memory-optimization
...
Update memory optimisation for MergingSorted.
2021-10-19 11:12:50 +03:00
kssenii
df5dc4e991
Adjust MergeTreeIndexReader
2021-10-19 10:51:45 +03:00
kssenii
49106f407f
Fix some checks
2021-10-19 09:34:27 +03:00
Alexander Tokmakov
89fe606d15
try fix 'some fetches may stuck'
2021-10-18 23:16:02 +03:00
Nikita Mikhaylov
d004fd96ed
Merge pull request #30318 from azat/fix-shared-build
...
Move SquashingTransform to Interpreters (to fix split build)
2021-10-18 22:41:01 +03:00
Azat Khuzhin
e576fd17bd
Do not allow reading to empty buffer in MergeTreeReaderStream
...
Changelog:
- Verify that all rows was read in MergeTreeReaderWide
- Ignore some exceptions for Wide parts
- Take max_rows_to_read into account
- Verify that requested rows is not zero (otherwise it is too tricky)
- Simply verify that buffer is not empty
2021-10-18 21:19:34 +03:00
Azat Khuzhin
cd4b33c8c9
Verify that all rows was read in MergeTreeReaderCompact
...
v0: Use fmt-like style exception in MergeTreeReaderCompact
v2: Update the check
2021-10-18 21:19:34 +03:00
Azat Khuzhin
41e6df0f4b
Remove unused offset_columns from MergeTreeReaderWide::readRows()
2021-10-18 21:19:34 +03:00
kssenii
d5d4817350
Add disk proxies back, add setReadUntilPosition to ReadBuffer
2021-10-18 18:35:11 +03:00
Anton Popov
d71ffc355a
Merge remote-tracking branch 'upstream/master' into HEAD
2021-10-18 15:18:22 +03:00
Nikolai Kochetov
6e479b301a
Update memory optimisation for MergingSorted.
2021-10-18 12:54:12 +03:00
Anton Popov
71b474793c
Merge pull request #30174 from CurtizJ/better-interfaces
...
Better interfaces for `IDataType` and `ISerialization`
2021-10-18 12:07:04 +03:00
kssenii
3d39ea8e37
Fix WHERE
2021-10-18 11:06:30 +03:00
Azat Khuzhin
6d4af3bac1
Move SquashingTransform to Interpreters (to fix split build)
...
clickhouse_common_io requires clickhouse_core:
ld.lld: error: undefined symbol: DB::blocksHaveEqualStructure(DB::Block const&, DB::Block const&)
>>> referenced by SquashingTransform.cpp:92 (/src/ch/clickhouse/src/Common/SquashingTransform.cpp:92)
>>> src/CMakeFiles/clickhouse_common_io.dir/Common/SquashingTransform.cpp.o:(void DB::SquashingTransform::append<DB::Block&&>(DB::Block&&))
>>> referenced by SquashingTransform.cpp:92 (/src/ch/clickhouse/src/Common/SquashingTransform.cpp:92)
>>> src/CMakeFiles/clickhouse_common_io.dir/Common/SquashingTransform.cpp.o:(void DB::SquashingTransform::append<DB::Block const&>(DB::Block const&))
while clickhouse_core requires clickhouse_common_io:
"clickhouse_core" of type SHARED_LIBRARY
depends on "roaring" (weak)
depends on "clickhouse_common_io" (weak)
depends on "clickhouse_common_config" (weak)
depends on "clickhouse_common_zookeeper" (weak)
depends on "clickhouse_dictionaries_embedded" (weak)
depends on "clickhouse_parsers" (weak)
Follow-up for: #30247 (cc @KochetovNicolai)
2021-10-18 10:28:36 +03:00
alexey-milovidov
7beffb04ad
Merge pull request #29495 from ClickHouse/improvement_for_strange_mutations
...
Minor improvement for strange mutations
2021-10-17 20:57:06 +03:00
Nikolai Kochetov
2a95d7fc87
Merge pull request #29918 from amosbird/projection-improve5
...
Add primary key to minmax_count_projection
2021-10-17 14:04:18 +03:00
Nikolai Kochetov
bfcbf5abe0
Merge branch 'master' into removing-data-streams-folder
2021-10-17 10:42:37 +03:00
alexey-milovidov
4f11cfa59d
Merge pull request #28760 from azat/mutator-forbid-cleaner
...
Forbid cleaning of tmp directories that can be used by an active mutation/merge.
2021-10-17 03:18:01 +03:00
kssenii
4a0bdd296d
Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read
2021-10-16 18:35:48 +03:00
Nikolai Kochetov
a08c98d760
Move some files.
2021-10-16 17:03:50 +03:00
alesapin
9418eda122
Merge pull request #30111 from ClickHouse/remove_trash
...
Remove trash from MergeTreeReadPool
2021-10-16 15:41:41 +03:00
Amos Bird
9ceb668a4d
Fix TSan
2021-10-16 18:13:41 +08:00
Amos Bird
77c4a5fa18
Better
2021-10-16 18:13:41 +08:00
Amos Bird
77a2022cf5
Add primary key to minmax_count_projection
2021-10-16 18:13:41 +08:00
Nikolai Kochetov
067eaadadd
Merge branch 'master' into removing-data-streams-folder
2021-10-16 09:46:05 +03:00
Nikolai Kochetov
c668696047
Merge pull request #30171 from ClickHouse/remove-stream-interfaces
...
Remove stream interfaces
2021-10-16 09:34:01 +03:00
Anton Popov
d2ba3bcaa4
Merge pull request #28363 from amosbird/fastload2
...
Speed up part loading for JBOD
2021-10-16 02:37:32 +03:00
Azat Khuzhin
07e8b2b3c7
Do not try to remove temporary paths that is currently in written by merge/mutation
...
v2: rebase against MergeTask
v3: rebase due to conflicts in src/Storages/MergeTree/MergeTreeDataMergerMutator.cpp
v4:
- rebase due to conflicts in src/Storages/MergeTree/MergeTask.cpp
- drop common/scope_guard_safe.h (not used)
2021-10-16 00:43:52 +03:00
Azat Khuzhin
0b3bf43d61
Rewrite MergeTreeData::clearOldTemporaryDirectories() to use early continue
2021-10-16 00:43:51 +03:00
Nikolai Kochetov
fd14faeae2
Remove DataStreams folder.
2021-10-15 23:18:20 +03:00
mergify[bot]
477ea62257
Merge branch 'master' into remove-sample-by
2021-10-15 15:14:17 +00:00
kssenii
759f747080
Fix build
2021-10-15 17:12:34 +03:00
kssenii
881ae8617e
Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read
2021-10-15 15:09:56 +03:00
kssenii
edfb1a5521
Better performance for smaller number of threads
2021-10-15 15:07:39 +03:00
Nikita Mikhaylov
d1138a8a25
Merge pull request #29691 from azat/fix-merge-mutate-memory-tracking
...
Fix memory tracking for merges and mutations
2021-10-15 13:53:52 +03:00
Nikolai Kochetov
ad8a344b46
Move TTL streams and algo
2021-10-15 13:11:57 +03:00
Maksim Kita
7beaeb8b13
Merge branch 'master' into indexcache
2021-10-15 12:59:09 +03:00
kssenii
3995506d37
Adjust range reader for remote fs reads
2021-10-15 11:36:26 +03:00
Azat Khuzhin
2a7a1d8df5
Avoid losing any allocations context from merges/mutations
2021-10-15 01:43:28 +03:00
Azat Khuzhin
8a209a78d7
Set query_id for mutations/merges
...
This will allow to distinguish allocations in trace_log.
2021-10-15 01:43:28 +03:00
Azat Khuzhin
8cc45bea7b
Avoid accounting memory from another mutation/merge
...
Before this path it was possible that one merge/mutation may account
memory from another, due to ThreadStatus::untracked_memory.
And this cause flakiness of 01200_mutations_memory_consumption.
2021-10-15 01:43:28 +03:00
Azat Khuzhin
fd38cbb0df
Fix memory tracking for merges and mutations (by destroying earlier)
...
It fixes only some tiny allocations, and so it should not affect any
huge mutations/merges.
And plus, this should not be a real fix, since peak_memory_usage is
obtained before even destrying this objects, and destroing objects will
unlikely update peak memory usage (although it is possible).
v0: do this in dtors
v2: do this earlier
2021-10-15 01:43:27 +03:00
Azat Khuzhin
e4b39ca99c
Remove unused Block member from MutateTask
...
v0: Fix excessive memory usage in MutateTask
This is relevant only for 01200_mutations_memory_consumption test, since
this is just a block.
v2: just remove unused Block member, since other part had been fixed in #29768
2021-10-15 01:43:27 +03:00
Anton Popov
7aa6068fb2
Merge remote-tracking branch 'upstream/master' into HEAD
2021-10-14 19:44:08 +03:00
mergify[bot]
92dd528c78
Merge branch 'master' into remove_trash
2021-10-14 14:14:07 +00:00
Anton Popov
476d7a411f
allow to remove SAMPLE BY expression
2021-10-14 17:05:20 +03:00
Anton Popov
c2d376d0d6
Merge branch 'master' into better-interfaces
2021-10-14 14:33:36 +03:00
Nikolai Kochetov
ab28c6c855
Remove BlockInputStream interfaces.
2021-10-14 13:25:43 +03:00
Nikolai Kochetov
3d3e143c29
Merge pull request #30001 from ClickHouse/remove-streams-from-formats
...
Remove streams from formats.
2021-10-14 12:39:07 +03:00
Anton Popov
92413aed68
better interfaces for IDataType and ISerialization
2021-10-14 05:36:49 +03:00
Nikolai Kochetov
2957971ee3
Remove some last streams.
2021-10-13 21:22:02 +03:00
alesapin
1801ea4955
Merge branch 'master' into remove_trash
2021-10-13 18:10:31 +03:00
alesapin
d2dfbb5ab6
Remove trash from MergeTreeReadPool
2021-10-13 15:25:44 +03:00
Amos Bird
23602f4607
Speed up part loading for JBOD
2021-10-13 12:33:54 +08:00
Kseniia Sumarokova
bb15b57753
Merge pull request #30016 from ucasfl/sample-by-tuple
...
Fix crash of sample by tuple()
2021-10-12 19:46:21 +03:00
Nikolai Kochetov
c36bb9206e
Merge pull request #29879 from amosbird/queryidholderfix
...
Fix potential leak of query_id_holder
2021-10-12 15:42:02 +03:00
Nikolai Kochetov
077aba4a97
Merge pull request #29520 from amosbird/projection-improve4
...
Get rid of naming limitation of projections.
2021-10-12 14:25:29 +03:00
Maksim Kita
91a48e9651
Merge pull request #29896 from kitaisreal/system-data-skipping-indices-added-size
...
System table data skipping indices added size
2021-10-12 12:16:57 +03:00
Maksim Kita
c15b67c182
Fix naming
2021-10-12 11:42:24 +03:00
alesapin
5ac0d36392
Merge pull request #30005 from ClickHouse/more_strict_check_for_intersecting_parts
...
More strict check for intersecting parts
2021-10-12 11:38:09 +03:00
feng lv
b5d69d599e
fix sample by tuple()
...
add tests
fix
fix
fix
2021-10-12 05:19:32 +00:00
alexey-milovidov
f1b550cdaf
Merge pull request #29974 from ClickHouse/minimized-toolchain
...
Experiment with minimized toolchain (2)
2021-10-11 22:04:01 +03:00
Alexander Tokmakov
706e2b6b88
more strict check for intersecting parts
2021-10-11 21:42:46 +03:00
Nikolai Kochetov
ec18340351
Remove streams from formats.
2021-10-11 19:11:50 +03:00
Alexey Milovidov
2370a8d6da
Fix PVS-Studio
2021-10-11 14:28:46 +03:00
Amos Bird
72bccaa501
Fix path name
2021-10-11 19:12:08 +08:00
Maksim Kita
b0d887a0fe
Added tests
2021-10-11 14:00:10 +03:00
Amos Bird
83717b7c3b
Get rid of naming limitation of projections.
2021-10-11 18:32:17 +08:00
Maksim Kita
ce0c41e1ad
Fixed tests
2021-10-11 11:39:50 +03:00
Maksim Kita
2d069acc22
System table data skipping indices added size
2021-10-11 11:39:50 +03:00
Amos Bird
ebfb013ea1
Fix potential leak of query_id_holder
2021-10-11 15:56:00 +08:00
kssenii
448d63a0fc
Range read with upper bound offset
2021-10-11 01:53:51 +03:00
kssenii
aa4b797808
Turn off tasks stealing for remote disk
2021-10-10 23:22:58 +03:00
Azat Khuzhin
6c33eaee32
Fix filtering by tuple (some conditions was lost during analyzing)
...
Fixes : #29281
Fixes: test_cluster_copier/test_three_nodes.py::test
2021-10-10 22:37:52 +03:00
Nikolai Kochetov
a95c28ec4b
Merge pull request #29898 from ClickHouse/remove-native-stream
...
Remove some more streams.
2021-10-10 21:01:16 +03:00
tavplubix
fa1396a9dd
Merge pull request #29790 from ClickHouse/fix-bad-cast-in-parser-create
...
Fix bad cast in ParserCreateQuery
2021-10-08 23:53:38 +03:00
Nikolai Kochetov
c6bce1a4cf
Update Native.
2021-10-08 20:21:19 +03:00
Nikolai Kochetov
340b53ef85
Remove some more streams.
2021-10-08 17:03:54 +03:00
Nikolai Kochetov
9eddee5517
Merge pull request #29768 from ClickHouse/remove-merging-streams
...
Remove some merging streams
2021-10-08 13:15:10 +03:00
Maksim Kita
717998a37f
Merge pull request #29825 from azat/memory_profiler_step-cleaner-api
...
Make memory_profiler_step API cleaner
2021-10-07 22:44:47 +03:00
tavplubix
de2fd9d762
Merge pull request #29812 from ClickHouse/better_diagnostic_for_optimize
...
Better diagnostic for OPTIMIZE
2021-10-07 22:22:25 +03:00
Kseniia Sumarokova
185d58a7cc
Merge pull request #29766 from ClickHouse/remove_redundant_seeks
...
Less seeks in compressed buffers
2021-10-07 17:04:50 +03:00
Alexander Tokmakov
7e2bc184ec
fix another suspicious places, add test
2021-10-07 16:43:49 +03:00
tavplubix
754e038eec
Merge pull request #29803 from ClickHouse/intersects-previous-part-usability
...
Improve usability of error messages when error is caused by sophisticated interventions
2021-10-07 15:24:17 +03:00
Alexander Tokmakov
5365cc686f
Merge branch 'master' into improvement_for_strange_mutations
2021-10-07 13:26:58 +03:00
Alexander Tokmakov
4119848a41
better diagnostic for OPTIMIZE
2021-10-07 13:08:10 +03:00
Vladimir C
0422939893
Merge pull request #29281 from lingtaolf/bugfix/tupleFilter
2021-10-07 12:24:26 +03:00
Azat Khuzhin
71cffbf521
Make memory_profiler_step API cleaner
...
Right now to configure memory_profiler_step/total_memory_profiler_step
you need to call:
MemoryTracker::setOrRaiseProfilerLimit()
MemoryTracker::setProfilerStep()
But it is easy to forget about setOrRaiseProfilerLimit(), since there is
no even any comments about this.
So instead, make setOrRaiseProfilerLimit() private and call it from
setProfilerStep()
2021-10-07 10:52:16 +03:00
Maksim Kita
4ec7311d4d
Merge pull request #29738 from kitaisreal/added-function-ngram
...
Added function ngram
2021-10-07 10:21:00 +03:00
Maksim Kita
b7b39da8cc
Fixed build
2021-10-06 23:22:29 +03:00
Nikolai Kochetov
d0c6f11fcb
More.
2021-10-06 20:59:27 +03:00
Alexey Milovidov
1455cfeddd
Improve usability of error messages when error is caused by sophisticated interventions
2021-10-06 15:28:52 +03:00
Maksim Kita
33353be4ed
Fixed tests
2021-10-06 15:12:07 +03:00
Nikolai Kochetov
6b619512aa
Fix style.
2021-10-06 12:37:56 +03:00
Maksim Kita
90eb74c933
Fixed tests
2021-10-06 00:37:45 +03:00
mergify[bot]
ab55225c8f
Merge branch 'master' into remove-merging-streams
2021-10-05 14:16:34 +00:00
Nikolai Kochetov
96d070a5ba
Fix tests.
2021-10-05 16:58:24 +03:00
mergify[bot]
3b1f49aac7
Merge branch 'master' into nv/parts-uuid-move-shard-kill
2021-10-05 12:36:02 +00:00
alesapin
9aab0eb0a2
Fixup
2021-10-05 13:41:09 +03:00
alesapin
8186339307
Less seeks in compressed buffers
2021-10-05 12:11:25 +03:00
Alexander Tokmakov
bb32432943
more fixes
2021-10-05 00:13:18 +03:00
Nikolai Kochetov
2001ebbf9d
Fix build.
2021-10-04 21:52:31 +03:00
Maksim Kita
3fe9fdac96
Added function ngram
2021-10-04 18:53:17 +03:00
Azat Khuzhin
117e9e77c8
Apply max_untracked_memory/memory_profiler_{step,sample_probability} during mutate/merge
2021-10-03 17:39:07 +03:00
Maksim Kita
395119339f
Merge branch 'master' into rename-common
2021-10-02 17:15:10 +03:00
alexey-milovidov
86d1ce63cf
Merge pull request #29649 from CurtizJ/merging-29376
...
Merging #29376
2021-10-02 12:02:42 +03:00
Alexey Milovidov
2ad5dbdfe8
Rename "common" to "base"
2021-10-02 11:30:38 +03:00
Alexey Milovidov
fe6b7c77c7
Rename "common" to "base"
2021-10-02 10:13:14 +03:00
Anton Popov
bb92ff52d8
Update IMergeTreeDataPart.cpp
2021-10-02 00:50:19 +03:00
Anton Popov
d404efbf7e
add test for freeze in-memory parts
2021-10-01 22:05:36 +03:00
Anton Popov
f0363c524d
Merge branch 'master' of git://github.com/mo-avatar/ClickHouse into merging-29376
2021-10-01 20:29:33 +03:00
Maksim Kita
57666725f9
Merge pull request #29623 from kitaisreal/bloom-filter-indexes-map-support-fix-unsafe-identifier-cast
...
BloomFilter indexes map data type support fix unsafe identifier cast
2021-10-01 19:28:04 +03:00
Nikita Mikhaylov
b2f7b6fae8
Fix race between MergeTask and storage destruction ( #29614 )
2021-10-01 18:55:04 +03:00
vdimir
7444d64156
Fix for recursive tuples in MergeTreeWhereOptimizer
2021-10-01 17:01:03 +03:00
vdimir
23ce41e3bb
MergeTreeWhereOptimizer supports constant tuple in both sides of equals
2021-10-01 17:01:03 +03:00
vdimir
64ffacd9e0
MergeTreeWhereOptimizer::tryAnalyzeTuple, fix fuzz
2021-10-01 17:01:02 +03:00
vdimir
e326b12a37
Fix style in MergeTreeWhereOptimizer.cpp
2021-10-01 17:01:02 +03:00
凌涛
f660224a5f
add tests
2021-10-01 17:01:01 +03:00
凌涛
ea68944055
fixed index didn't work in tuple where
2021-10-01 17:01:01 +03:00
Maksim Kita
c793c8e7fa
BloomFilter indexes map data type support fix unsafe identifier cast
2021-10-01 14:40:00 +03:00
Alexey Milovidov
4271f8c738
Merge remote-tracking branch 'origin/master' into yandex-to-clickhouse-in-configs
2021-10-01 01:33:58 +03:00
Nikita Mikhaylov
9756b8dc33
Added an ability to execute more merges and mutations than threads, added new settings ( #29140 )
2021-10-01 00:26:24 +03:00
Nikolai Kochetov
68f8b9d235
Update ColumnGathererStream
2021-09-29 20:45:01 +03:00
Anton Popov
914781052e
Merge remote-tracking branch 'upstream/master' into HEAD
2021-09-29 17:37:07 +03:00
mergify[bot]
6a2ac6e871
Merge branch 'master' into yandex-to-clickhouse-in-configs
2021-09-29 13:29:08 +00:00
Maksim Kita
0279afa738
Merge pull request #29474 from kitaisreal/bloom-filter-indexes-updated-tests
...
Bloom filter indexes updated tests
2021-09-28 22:40:27 +03:00
Maksim Kita
9ecd0a3f4f
Merge pull request #29220 from ClickHouse/full-text-bloom-filter-index-map-data-type
...
Merging #28511
2021-09-28 22:40:17 +03:00
alexey-milovidov
1e6f9ac635
Merge branch 'master' into yandex-to-clickhouse-in-configs
2021-09-28 20:26:44 +03:00
mo-avatar
6c16ec83eb
in memory part lost when freeze
2021-09-28 22:52:01 +08:00
Maksim Kita
f2402e1a10
Fixed tests
2021-09-28 12:52:07 +03:00
Maksim Kita
71fb9e6059
Bloom filter indexes updated tests
2021-09-28 12:38:27 +03:00
Azat Khuzhin
9a23b20965
Remove debug from MergeTreeIndexConditionBloomFilter::traverseASTEquals()
2021-09-28 10:28:28 +03:00
alexey-milovidov
995913e89c
Merge branch 'master' into yandex-to-clickhouse-in-configs
2021-09-28 02:27:36 +03:00
Maksim Kita
7861c26eaf
Merge pull request #29431 from kitaisreal/map-bloom-filter-index-map-values-equals-support
...
Map bloom filter index mapValues equals function support
2021-09-27 22:03:53 +03:00
Nikolai Kochetov
236d71ea94
Merge pull request #28582 from ClickHouse/rewrite-pushing-to-views
...
Rewrite PushingToViews
2021-09-27 21:19:11 +03:00
Maksim Kita
e290f19957
Merge branch 'master' into full-text-bloom-filter-index-map-data-type
2021-09-27 17:50:07 +03:00
Maksim Kita
b28d6e76cc
Map bloom filter full text index added support for mapKeys, mapValues
2021-09-27 17:26:25 +03:00
Maksim Kita
cd9c796e6f
Map bloom filter index mapValues equals function support
2021-09-27 17:23:56 +03:00
Nikita Mikhaylov
3741b8a17e
Merge pull request #29337 from amosbird/projection-fix18
...
Fix projection merges and mutations.
2021-09-27 13:45:13 +03:00
Nikolai Kochetov
78a7665f43
Merge branch 'master' into rewrite-pushing-to-views
2021-09-27 10:56:50 +03:00
Amos Bird
34017c942e
More fixes
2021-09-25 13:38:56 +08:00
Alexey Milovidov
82d57c6e47
Merge branch 'master' into yandex-to-clickhouse-in-configs
2021-09-25 01:38:39 +03:00
Amos Bird
23d3d894e6
Fix projection merges and mutations.
2021-09-24 22:45:50 +08:00
Nikolai Kochetov
e617aeb7a5
Merge pull request #29139 from amosbird/projection-fix17
...
Disable projections when ARRAY JOIN is used.
2021-09-24 11:15:23 +03:00
alexey-milovidov
b8083929a6
Merge branch 'master' into yandex-to-clickhouse-in-configs
2021-09-23 20:49:40 +03:00
Dmitrii Kovalkov
e3ae32f763
Allow nulls first
2021-09-23 19:39:15 +03:00
Nikolai Kochetov
998d29ebc7
Merge branch 'master' into rewrite-pushing-to-views
2021-09-23 13:10:27 +03:00
Maksim Kita
3d79812427
Merge branch 'master' into full-text-bloom-filter-index-map-data-type
2021-09-23 12:38:03 +03:00
Maksim Kita
0341b99c69
Merge pull request #29280 from kitaisreal/full-text-bloom-filter-added-support-for-array-data-type
...
FullText bloom filter index added support for Array data type
2021-09-23 12:34:06 +03:00
Maksim Kita
d9e265df2c
Fixed debug message
2021-09-23 12:13:01 +03:00
Maksim Kita
ebbefd49c3
FullText bloom filter index added support for Array data type
2021-09-23 02:06:54 +03:00
Maksim Kita
d5a7e92b97
Bloom filter map added support for has function
2021-09-23 01:10:14 +03:00
Maksim Kita
1bf375e2b7
Merge pull request #29254 from kitaisreal/map-bloom-filter-index-updated
...
Updated bloom filter index for map type to work with mapKeys, mapValues functions
2021-09-23 00:31:14 +03:00
Maksim Kita
93208fca4b
Bloom filter for map data type dded support for IN, mapContains functions
2021-09-22 20:25:10 +03:00
Nikita Mikhaylov
558c74e826
Done
2021-09-22 11:52:47 +00:00
Maksim Kita
92fd6b376f
Updated bloom filter index for map type to work with mapKeys function
2021-09-22 12:45:48 +03:00
Alexey Milovidov
e9e77b4403
.tech -> .com
2021-09-22 03:22:57 +03:00
Maksim Kita
84e37c2d87
Fixed tests
2021-09-21 21:28:52 +03:00
Maksim Kita
6dadc521fb
MergeTreeIndexFullText updated for data type map
2021-09-21 16:55:48 +03:00
凌涛
e3e3b84f63
fix tests
2021-09-21 16:55:48 +03:00
凌涛
9461d6dbf2
modify code style
2021-09-21 16:55:47 +03:00
凌涛
e84f1000fb
optimze code to make sure only map function will be processed
2021-09-21 16:55:47 +03:00
凌涛
a3e657778a
bloom filter skipping index support Map with key of String type
2021-09-21 16:55:07 +03:00
凌涛
8f87d44958
bloom filter skipping index support Map with key of String type
2021-09-21 16:55:07 +03:00
Maksim Kita
15c9315ea1
MergeTreeIndexConditionBloomFilter updated for Map data type
2021-09-21 15:07:59 +03:00
fuwhu
8962672e59
add bloom filter skip index for map data type.
2021-09-21 10:59:16 +03:00
Anton Popov
55cf857aba
Merge pull request #28707 from azat/max_suspicious_broken_parts_bytes
...
Implement max_suspicious_broken_parts_bytes
2021-09-20 19:55:07 +03:00
Nikolai Kochetov
a790d391c0
Merge branch 'master' into rewrite-pushing-to-views
2021-09-20 19:43:15 +03:00
Anton Popov
6f9e53197c
Merge remote-tracking branch 'upstream/master' into HEAD
2021-09-20 17:17:05 +03:00
Anton Popov
e3e6258b95
Merge pull request #29163 from CurtizJ/small-refactoring
...
Slightly better code near reading of subcolumns
2021-09-19 16:14:52 +03:00
Anton Popov
f50098a059
fix style check
2021-09-19 00:13:00 +03:00
Anton Popov
1c80a95b36
slightly better code near reading of subcolumns
2021-09-18 22:34:54 +03:00
Amos Bird
6b3cd34cff
Disable projections when ARRAY JOIN is used.
2021-09-18 08:15:59 +08:00
Azat Khuzhin
b108c5e1b0
Implement max_suspicious_broken_parts_bytes
...
Usually broken parts are found from just created parts, which are
usually small enough, so it make sense to have a limit in bytes for
broken parts.
Default value was set to 1G.
2021-09-17 22:06:32 +03:00
Nikolai Kochetov
a8443bef4d
Fix build.
2021-09-17 20:52:26 +03:00
Nicolae Vartolomei
9a02061d9c
Rewrite part movement between shards logic and add kill support
...
See https://github.com/ClickHouse/ClickHouse/pull/24585#issuecomment-857735081
for extra context about the current implementation.
2021-09-17 18:11:32 +01:00
Nicolae Vartolomei
0381c634d4
Add support for user defined identifier on log entries
...
Sometimes we want to push a log entry once and only once. Because it is
not possible to create a sequential node in ZooKeeper and store its name
to a well known location in the same transaction we'll do it in the
other order. First somehow generate a unique identifier, then submit a
log entry with that identifier. Later, we can search through log entries
using the identifier we provided to find the node.
Required for part movement between shards.
2021-09-17 15:32:35 +01:00
Nikolai Kochetov
a8c3b02598
Merge branch 'master' into rewrite-pushing-to-views
2021-09-17 15:38:11 +03:00
mergify[bot]
c23f1fc160
Merge branch 'master' into fix_part_checker
2021-09-17 07:23:12 +00:00
Nikita Mikhaylov
c52b8ec083
Introduced MergeTask and MutateTask ( #25165 )
...
Introduced MergeTask and MutateTask
2021-09-17 00:19:58 +03:00
Nikolai Kochetov
341553febd
Fix build.
2021-09-16 20:40:42 +03:00
alesapin
606ab6d3e4
Typo fix
2021-09-16 18:40:25 +03:00
alesapin
8930a0d2bd
More precise initialization
2021-09-16 18:39:11 +03:00
Anton Popov
eef436fe22
Merge remote-tracking branch 'upstream/master' into HEAD
2021-09-16 18:07:42 +03:00
alesapin
b4cbc86164
Fix bug
2021-09-16 17:59:55 +03:00
Anton Popov
f6191b98e7
Merge remote-tracking branch 'upstream/master' into HEAD
2021-09-15 17:47:22 +03:00
Kruglov Pavel
8a67c3cf44
Merge pull request #28636 from amosbird/nullable-index-fix
...
Fix nullable/lowcardinality primary key with constant conversion
2021-09-15 15:56:19 +03:00
Anton Popov
fc17936c12
Merge remote-tracking branch 'upstream/master' into HEAD
2021-09-14 23:02:30 +03:00
Nikolai Kochetov
b997214620
Rename QueryPipeline to QueryPipelineBuilder.
2021-09-14 20:48:18 +03:00
Nikolai Kochetov
7fc830ae2b
Rename QueryPipeline to QueryPipelineBuilder.
2021-09-14 19:28:41 +03:00
mergify[bot]
4993c7c527
Merge branch 'master' into fix_part_checker
2021-09-14 14:13:52 +00:00
Nikolai Kochetov
0e267c50b4
Merge branch 'master' into rewrite-pushing-to-views
2021-09-14 16:13:54 +03:00
alesapin
9441378b39
Add logs to queue because I cannot understand anything without them
2021-09-14 12:04:11 +03:00
Kseniia Sumarokova
d3e604e14b
Merge pull request #28603 from kssenii/fix-web-disk-2
...
Some fixes for static files disk
2021-09-14 08:24:24 +03:00
mergify[bot]
c54434084f
Merge branch 'master' into fix_part_checker
2021-09-13 13:19:34 +00:00
alesapin
1d05fab295
Merge pull request #28864 from ClickHouse/fix_replace_range_drop_part_reordering
...
Fix reordering of REPLACE_RANGE and DROP PART
2021-09-13 15:11:29 +03:00
alesapin
fa8b698b7e
Merge branch 'fix_part_checker' of github.com:ClickHouse/ClickHouse into fix_part_checker
2021-09-13 11:00:41 +03:00
alesapin
ab41384f63
Move queue initialization to restarting thread
2021-09-13 11:00:07 +03:00
alesapin
91e10adca5
Merge branch 'master' into fix_part_checker
2021-09-13 10:19:04 +03:00
kssenii
9ec6a9b4c9
Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-web-disk-2
2021-09-13 08:32:34 +03:00
alexey-milovidov
cdcfdbec7e
Merge pull request #28873 from CurtizJ/fix-subcolumns-compact-parts
...
Fix reading of subcolumns from compact parts
2021-09-12 15:54:07 +03:00
alexey-milovidov
ea13a8b562
Merge pull request #28659 from myrrc/improvement/tostring_to_magic_enum
...
Improving CH type system with concepts
2021-09-12 15:26:29 +03:00
mergify[bot]
267e56df8c
Merge branch 'master' into fix-web-disk-2
2021-09-11 15:45:43 +00:00
Anton Popov
8999a83320
add comment
2021-09-10 20:22:28 +03:00
Anton Popov
a63887413c
fix reading of subcolumns from compact parts
2021-09-10 20:19:18 +03:00
Anton Popov
0bb74f8eaf
Merge pull request #28762 from CurtizJ/fix-nested-1
...
Fix usage of nested columns with non-array columns with the same prefix [2]
2021-09-10 17:34:43 +03:00
Alexander Tokmakov
518f542c3e
fix reordering of REPLACE_RANGE and DROP PART
2021-09-10 17:32:45 +03:00
alesapin
d1dea0e95f
Update src/Storages/MergeTree/ReplicatedMergeTreePartCheckThread.cpp
...
Co-authored-by: Azat Khuzhin <a3at.mail@gmail.com>
2021-09-10 15:32:24 +03:00
Anton Popov
5cff615eca
Merge remote-tracking branch 'upstream/master' into HEAD
2021-09-10 13:40:48 +03:00
alexey-milovidov
2236a5df98
Merge pull request #28085 from myrrc/improvement/merge-tree-part-opt-parse
...
optional<> semantics for parsing MergeTreePartInfo and DetachedPartInfo
2021-09-10 00:04:03 +03:00
Nikolai Kochetov
f569a3e3f7
Merge branch 'master' into rewrite-pushing-to-views
2021-09-09 20:30:23 +03:00
alesapin
59edf6b5f6
Better logic on part checker
2021-09-09 18:19:12 +03:00
kssenii
0c013ed91b
Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-web-disk-2
2021-09-09 15:26:36 +03:00
Nikolai Kochetov
13eb93a9c0
Merge pull request #28700 from amosbird/projection-fix16
...
Fix crash on exception with projection aggregate
2021-09-09 15:24:05 +03:00
Anton Popov
8e5b3b2f6c
Merge remote-tracking branch 'upstream/master' into HEAD
2021-09-09 15:12:34 +03:00
Nikolai Kochetov
abfb1d20ea
Merge pull request #25734 from vdimir/join-arrayjoin-type-cast
...
Move join type inference to ExpressionAnalyzer from syntax stage + small refactor
2021-09-09 14:38:43 +03:00
Anton Popov
8203bd1ac6
Merge remote-tracking branch 'upstream/master' into HEAD
2021-09-09 14:04:37 +03:00
Anton Popov
2ccdb6c22d
fix clang-tidy
2021-09-09 12:40:26 +03:00
Nikita Mikhaylov
4db5062d6b
Merge pull request #28374 from nikitamikhaylov/global-merge-executor
...
Introduced global executor for background MergeTree-related operations
2021-09-09 11:30:21 +03:00
alesapin
407c0bbb56
Merge pull request #28734 from ClickHouse/fix_race_on_mutation_pointer
...
Fix race on mutation_pointer update
2021-09-09 10:36:09 +03:00
Amos Bird
91293c7449
Fix crash on exception with projection aggregate
2021-09-09 10:43:56 +08:00
Nikita Mikhaylov
f12504b002
Fix stupid race
2021-09-08 20:30:54 +00:00
Anton Popov
9816014f6c
fix nested and scalar columns with dot, which is missed in part
2021-09-08 21:42:34 +03:00
kssenii
6108a4139c
Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-web-disk-2
2021-09-08 20:54:07 +03:00
Nikita Mikhaylov
1416b1ccba
Update MergeTreeBackgroundExecutor.cpp
2021-09-08 20:21:25 +03:00
Nikita Mikhaylov
dae9a01b4b
Fix build
2021-09-08 16:13:32 +00:00
Nikita Mikhaylov
3b7fa06540
Review fixes
2021-09-08 12:42:03 +00:00
alesapin
f8a6dfb0c4
Fix race on mutation_pointer update
2021-09-08 14:37:16 +03:00
Nikita Mikhaylov
ed37a22c37
Delete debug print
2021-09-08 09:35:18 +00:00
Nikita Mikhaylov
6062dd0021
Better
2021-09-08 00:21:21 +00:00
Azat Khuzhin
db0767a194
Implement detach_not_byte_identical_parts
...
Maybe useful for further analysis of non byte-identical parts.
2021-09-07 23:29:57 +03:00
Nikita Mikhaylov
231f64cf37
Fix unit test
2021-09-07 13:51:30 +00:00
Nikita Mikhaylov
9b38248e64
Remove scheduler thread
2021-09-07 13:13:23 +00:00
Nikita Mikhaylov
8a0765c60d
Review fixes (partially)
2021-09-07 12:45:39 +00:00
mergify[bot]
e9fbdec13a
Merge branch 'master' into improvement/merge-tree-part-opt-parse
2021-09-07 09:52:24 +00:00
Nikita Mikhaylov
cfe2207be3
Better
2021-09-06 22:07:41 +00:00
Nikita Mikhaylov
324ab97f50
Fix gcc build
2021-09-06 16:44:46 +00:00
Nikita Mikhaylov
7c39f84a5e
Merge upstream/master into global-merge-executor (using imerge)
2021-09-06 15:37:18 +00:00
Mike Kot
8e9aacadd1
Initial: replacing hardcoded toString for enums with magic_enum
2021-09-06 16:24:03 +02:00
Nikita Mikhaylov
17d65b3d7b
Allow allocations
2021-09-06 14:04:37 +00:00
Nikita Mikhaylov
ea0fbf81af
Renaming
2021-09-06 12:01:16 +00:00
Nikita Mikhaylov
0249015515
added priority queue
2021-09-06 11:37:51 +00:00
Nikolai Kochetov
2178ed21df
Merge pull request #28560 from amosbird/projection-fix15
...
Fix wrong header of minmax_count projection
2021-09-06 13:00:44 +03:00
mergify[bot]
8eba5f29a1
Merge branch 'master' into improvement/merge-tree-part-opt-parse
2021-09-06 09:53:46 +00:00
Amos Bird
8fac97692a
Fix nullable primary key with constant conversion
2021-09-06 15:11:37 +08:00
Nikita Mikhaylov
3eec8a3c2f
Use boost::circular_buffer
2021-09-03 22:15:20 +00:00
kssenii
6ee94c923f
Fix
2021-09-04 00:51:11 +03:00
Anton Popov
59128412ea
Merge remote-tracking branch 'upstream/master' into HEAD
2021-09-03 22:17:11 +03:00
Nikolai Kochetov
66a76ab70f
Rewrite PushingToViewsBlockOutputStream part 6
2021-09-03 20:29:36 +03:00
Kruglov Pavel
f559c34113
Merge pull request #28302 from amosbird/binaryconstantwrap
...
Always monotonic for non-zero division
2021-09-03 20:02:54 +03:00
Nikita Mikhaylov
26ab6ebc0a
finalize in destructor
2021-09-03 14:20:34 +00:00
Nikita Mikhaylov
b867eabe73
Use Poco::Event
2021-09-03 13:02:49 +00:00
alesapin
dd7493d871
Merge pull request #27038 from excitoon-favorites/altermaterializecolumn
...
`ALTER TABLE ... MATERIALIZE COLUMN`
2021-09-03 15:50:52 +03:00
Nikita Mikhaylov
218f0adca7
Merge upstream/master into global-merge-executor (using imerge)
2021-09-03 12:29:48 +00:00
Nikita Mikhaylov
ca7adb76b9
Owning future
2021-09-03 12:27:49 +00:00
Mike Kot
1f148719ce
Review fixes
2021-09-03 13:33:40 +02:00
Amos Bird
bb87d01579
Fix wrong header of minmax_count projection
2021-09-03 18:49:01 +08:00
Nikita Mikhaylov
d777c6cd70
Better [2]
2021-09-03 00:51:12 +00:00
Nikita Mikhaylov
60765b8c6e
Better
2021-09-03 00:45:19 +00:00
Nikita Mikhaylov
292a24abe8
Merge upstream/master into global-merge-executor (using imerge)
2021-09-03 00:34:24 +00:00
Nikita Mikhaylov
9257188512
Comments
2021-09-03 00:28:29 +00:00
Nikita Mikhaylov
7baad6adde
Fix build
2021-09-02 22:56:03 +00:00
Nikita Mikhaylov
cc7c221fad
Own PriorityQueue + prettifying the code
2021-09-02 21:31:32 +00:00
Nikita Mikhaylov
6624fa12ba
deleted set
2021-09-02 18:49:37 +00:00