Alexander Gololobov
a7adc0a91b
Cleanups
2022-12-28 18:07:36 +01:00
Alexander Gololobov
b22711baa3
Reset need_filter flag when filter is applied
2022-12-28 18:07:36 +01:00
Alexander Gololobov
ada6422985
Restored old logic for filling _part_offset
2022-12-28 18:07:36 +01:00
Alexander Gololobov
4cebc6f3a4
Cleanups
2022-12-28 18:07:36 +01:00
Alexander Gololobov
13e457c754
Cleanups
2022-12-28 18:07:36 +01:00
Alexander Gololobov
a1cdad4815
Fix for reading _part_offset along with newly added column that has only default value
2022-12-28 18:07:35 +01:00
Alexander Gololobov
ac1549f6b3
Skip filtering if there are no rows after optimize()
2022-12-28 18:07:35 +01:00
Alexander Gololobov
f273f8712d
Avoid filtering same column in block_before_prewhere if it is present in the result
2022-12-28 18:07:35 +01:00
Alexander Gololobov
f3646248c5
Avoid unneeded work if all rows were filtered
2022-12-28 18:07:35 +01:00
Alexander Gololobov
75152ddabb
Apply filter only if needed
2022-12-28 18:07:35 +01:00
Alexander Gololobov
a18850458c
Test applying current filter at each step
2022-12-28 18:07:35 +01:00
Alexander Gololobov
29b5c4af07
Test dirty intermediate changes
2022-12-28 18:07:35 +01:00
Alexander Gololobov
c561acb774
Properly handle low cardinality column as prewhere filter
2022-12-28 18:07:35 +01:00
Alexander Gololobov
aa276b230b
Don't need to save filter and rows_per_granule from previous step
2022-12-28 18:07:35 +01:00
Alexander Gololobov
c4a01cbd5b
Fix for propely cleaning rows_per_granule_original between prewhere steps
2022-12-28 18:07:35 +01:00
Alexander Gololobov
abbb58107c
Fix for "out of bound" in ColumnVector::insertRangeFrom called from shrink()
2022-12-28 18:07:35 +01:00
Alexander Gololobov
79874e8733
Fix for "Invalid number of rows in Chunk"
2022-12-28 18:07:35 +01:00
Alexander Gololobov
bdf51545f7
Added FilterWithCachedCount class instead of caching counts in filter_bytes_map
2022-12-28 18:07:35 +01:00
Azat Khuzhin
31a88d4eae
Fix PREWHERE with row-level filters (when row filter is always true/false)
...
In case of row-level filters optimized out, i.e. converted to
always true/false, it is possible for MergeTreeRangeReader to reuse
incorrect statistics for the filter (countBytesInResultFilter()), and
because of this it simply does not apply other filters, since it assume
that this filter does not need to filter anything.
Fixes : #40956
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-28 18:07:35 +01:00
Alexander Gololobov
d44392b366
Checking the fix for "Invalid number of rows in Chunk"
2022-12-28 18:07:35 +01:00
Anton Popov
38f0cf9b10
fix test 01130_in_memory_parts
2022-12-28 16:40:31 +00:00
Dmitry Novik
6efd64a61c
Merge branch 'master' into parts-removal-limit
2022-12-28 17:34:48 +01:00
Han Fei
17358f5e98
clean headers
2022-12-28 17:29:07 +01:00
Han Fei
66111ef241
deduplicate async inserts in the same block earlier
2022-12-28 17:25:36 +01:00
Anton Popov
2adf0e9db3
do not wait loading parts at shutdown
2022-12-28 16:07:14 +00:00
alesapin
3a1e9f4a4d
Fix style
2022-12-28 14:08:13 +01:00
Nikolai Kochetov
c203aa054a
Use sequential source for mutations.
2022-12-27 20:28:25 +00:00
Han Fei
4d8513024c
fix bug that async blocks cleanup not work
2022-12-27 20:12:09 +01:00
alesapin
3a7f26ff3a
Split test into two
2022-12-27 16:15:23 +01:00
Raúl Marín
948ce8db93
Revert "Merge pull request #38953 from ClickHouse/add-allocation-ptr-to-trace-log"
...
This reverts commit e99849d0317ad3ff8a9e
2022-12-27 14:28:13 +01:00
alesapin
560223ac19
Merge branch 'fix-restart-after-quorum-insert' of github.com:CurtizJ/ClickHouse into CurtizJ-fix-restart-after-quorum-insert
2022-12-27 13:48:55 +01:00
Alexey Milovidov
bfe895975e
Merge pull request #44575 from ClickHouse/minimal-changes
...
Non-significant changes
2022-12-27 15:34:27 +03:00
Alexey Milovidov
464a513f0e
Merge pull request #44615 from ClickHouse/projection-grouping-sets
...
Disable projections in presense of any grouping sets, including WITH ROLLUP, WITH CUBE and WITH TOTALS
2022-12-27 15:31:03 +03:00
Alexey Milovidov
1b21cc018e
Merge pull request #44342 from ClickHouse/add-test-29883
...
Add a test for #29883
2022-12-27 14:55:14 +03:00
Raúl Marín
fc1fa82a39
Merge branch 'master' into perf_experiment
2022-12-27 10:51:58 +01:00
Alexey Milovidov
593675a7c5
Disable projections in presense of any grouping sets, including WITH ROLLUP, WITH CUBE and WITH TOTALS
2022-12-26 22:31:41 +01:00
Igor Nikonov
ba88bc17d4
Fix typo
2022-12-26 21:01:00 +00:00
Igor Nikonov
3204f7353b
Fix: handle exceptions from unlockSharedData() during error handling
...
+ renamePartToTemporary() lambda to reused code
2022-12-26 20:49:04 +00:00
Igor Nikonov
0d428318a9
Merge remote-tracking branch 'origin/master' into fix-restart-after-quorum-insert
2022-12-26 19:18:35 +00:00
Alexey Milovidov
52b306d544
Improve variable name
2022-12-26 18:00:18 +01:00
Alexey Milovidov
71850f6913
Check for bad index_granularity
2022-12-25 16:06:34 +01:00
Alexey Milovidov
f22fc35900
Minimal changes
2022-12-25 14:18:05 +01:00
Nikolai Kochetov
03352a15b6
Use sequential source for mutations.
2022-12-23 22:39:27 +00:00
Alexander Tokmakov
a0668af762
Update MergeTreeData.cpp
2022-12-23 23:06:37 +03:00
Alexander Tokmakov
b2af49a4cf
Merge branch 'master' into parallel_parts_cleanup_with_zero_copy
2022-12-23 20:02:17 +01:00
Alexander Tokmakov
ca989e9212
less runtime format strings
2022-12-23 19:50:34 +01:00
Anton Popov
0722020cae
Merge remote-tracking branch 'upstream/master' into HEAD
2022-12-23 18:15:55 +00:00
alesapin
2fecf420cb
Merge pull request #44513 from ClickHouse/small_polishing
...
Minor code polishing
2022-12-23 13:47:26 +01:00
Vladimir C
7482ea54ab
Merge pull request #43972 from ClickHouse/vdimir/tmp-data-in-fs-cache-2
2022-12-23 11:59:27 +01:00
Alexander Tokmakov
6b6504f0d1
Merge branch 'master' into parallel_parts_cleanup_with_zero_copy
2022-12-23 02:04:43 +01:00
Dmitry Novik
6c23721255
Merge pull request #43761 from ClickHouse/refector-function-node
...
Refactor FunctionNode
2022-12-22 23:53:27 +01:00
alesapin
a32fab90d5
Merge pull request #44436 from ClickHouse/fix_bug_with_max_task_count_update
...
Fix bug with merge/mutate pool size increase
2022-12-22 23:48:54 +01:00
Dmitry Novik
d3502937c7
Update MergeTreeIndexSet
2022-12-22 22:17:18 +00:00
Dmitry Novik
cff882d506
Merge remote-tracking branch 'origin/master' into refector-function-node
2022-12-22 21:34:29 +00:00
alesapin
8d0ad4bbcd
Strange why I have to supress in .cpp
2022-12-22 16:14:19 +01:00
alesapin
fa467b3398
Minor code polishing
2022-12-22 14:31:42 +01:00
Nikolai Kochetov
773de7a8a6
Merge pull request #44472 from CurtizJ/fix-mutations
...
Fix mutations with setting `max_streams_for_merge_tree_reading`
2022-12-22 14:05:20 +01:00
alesapin
049103c550
Merge pull request #44495 from azat/mergetree/static-storage-fix
...
Do not try to remove WAL/move broken parts for static storage
2022-12-22 12:27:08 +01:00
vdimir
88008726d8
Merge branch 'master' into vdimir/tmp-data-in-fs-cache-2
2022-12-22 10:58:39 +00:00
alesapin
380c6cebe4
Merge pull request #44393 from ClickHouse/small_improvement
...
Lock table for share during startup for database ordinary
2022-12-22 11:42:42 +01:00
alesapin
bf188c46a4
Merge branch 'master' into fix_bug_with_max_task_count_update
2022-12-22 11:40:08 +01:00
Maksim Kita
e5ebf20044
Merge pull request #44097 from kitaisreal/analyzer-support-set-index
...
Analyzer support Set index
2022-12-22 13:30:15 +03:00
vdimir
d30d25dbbe
Temporary files evict fs cache
2022-12-22 10:22:49 +00:00
Yakov Olkhovskiy
a8cb29da4b
Merge branch 'master' into refactoring-ip-types
2022-12-21 23:56:24 -05:00
alesapin
9b97cd98c8
Update src/Storages/MergeTree/MergeTreeDataMergerMutator.h
2022-12-21 23:27:17 +01:00
alesapin
97d43a6748
Add comment
2022-12-21 23:26:26 +01:00
alesapin
edcd0d95ca
Revert "Remove explicit dependency on fast test to debug the issue"
...
This reverts commit 2fe969a2e1
2022-12-21 23:25:50 +01:00
Azat Khuzhin
1985d321a6
Do not try to remove WAL for static storage
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-21 18:09:38 +01:00
Azat Khuzhin
e06c169b43
Do not try to move parts to detached for static storage
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-12-21 18:09:38 +01:00
alesapin
a4fbea2184
Relax synchronization to avoid deadlock
2022-12-21 15:59:28 +01:00
Alexander Tokmakov
4a38a03f68
fix
2022-12-21 15:42:25 +01:00
Anton Popov
4b7f1e0cf2
fix mutations with setting 'max_streams_for_merge_tree_reading'
2022-12-21 01:59:53 +00:00
Dmitry Novik
42812dfa08
Add logging
2022-12-21 00:14:01 +00:00
Alexey Milovidov
a6f50edde3
Merge branch 'master' into add-test-29883
2022-12-21 01:45:18 +03:00
Alexander Tokmakov
cdc3912743
fix incorrect usages of getPartName()
2022-12-20 22:44:27 +01:00
Alexander Tokmakov
5f5335876a
Merge branch 'master' into parallel_parts_cleanup_with_zero_copy
2022-12-21 00:35:21 +03:00
Alexander Tokmakov
3f851d5a09
parallel parts cleanup with zero copy replication
2022-12-20 22:04:26 +01:00
alesapin
2fe969a2e1
Remove explicit dependency on fast test to debug the issue
2022-12-20 22:00:57 +01:00
Dmitry Novik
2ec720c5c4
Add part removal limit for one CleanupThread iteration
2022-12-20 19:15:13 +00:00
Dmitry Novik
4793412887
Merge branch 'master' into refector-function-node
2022-12-20 18:26:19 +01:00
alesapin
f06b311abc
It's really needed
2022-12-20 18:09:38 +01:00
alesapin
2818520572
Fix bug with merge/mutate pool size increase
2022-12-20 14:51:11 +01:00
alesapin
9d4404ce0d
Merge branch 'master' into small_improvement
2022-12-20 12:12:39 +01:00
alesapin
a8515e6925
Merge branch 'master' into fix_race_background_temp_removal
2022-12-20 12:12:14 +01:00
Raúl Marín
45d27f461b
Merge branch 'master' into perf_experiment
2022-12-20 09:07:48 +00:00
alesapin
c8d9ccf3cd
Lock table for share during startup for database ordinary
2022-12-19 18:51:46 +01:00
alesapin
38fd4e0162
Fix race which can lead to merged parts premature removal
2022-12-19 15:25:01 +01:00
Han Fei
98efee2d75
Merge branch 'master' into hanfei/fix-async-flaky
2022-12-19 14:14:29 +01:00
Han Fei
1cb98eb2d6
address comments
2022-12-19 14:05:50 +01:00
Han Fei
ee250174ab
fix flack test "02481_async_insert_dedup.python"
2022-12-18 22:44:51 +01:00
Alexey Milovidov
3e8dcdc155
Add earlier check
2022-12-18 08:01:43 +01:00
Dmitry Novik
875a24a650
Merge remote-tracking branch 'origin/master' into refector-function-node
2022-12-16 16:07:30 +00:00
Anton Popov
b7e76e3539
do not hold data parts during insert
2022-12-15 23:57:10 +00:00
Nikolai Kochetov
e99849d031
Merge pull request #38953 from ClickHouse/add-allocation-ptr-to-trace-log
...
Track allocation ptr in system.trace_log. Add aggregate function flamegraph
2022-12-15 18:16:05 +01:00
Dmitry Novik
7ad3ff8a9e
Merge pull request #44244 from ClickHouse/part-log-removal-info
...
Add more information about part removal into system.parts table
2022-12-15 18:06:55 +01:00
Alexander Tokmakov
688e488e93
Merge pull request #44220 from ClickHouse/correct-readonlyreplica-metric
...
Fix `ReadonlyReplica` metric
2022-12-15 19:05:20 +03:00
Dmitry Novik
b87739d31c
Add HAS_SKIPPED_MUTATION_PARENT state
2022-12-15 16:05:15 +00:00
Kruglov Pavel
b63ff31061
Merge pull request #44192 from Avogar/fix-bad-cast
...
Fix possible Bad cast from type DB::IAST const* to DB::ASTLiteral const*
2022-12-15 14:02:41 +01:00
Dmitry Novik
5b7a990425
Add more information about part removal into system.parts table
2022-12-14 22:30:43 +00:00
Dmitry Novik
1df038e39c
Merge remote-tracking branch 'origin/master' into refector-function-node
2022-12-14 14:55:30 +00:00
Anton Popov
c6b6b0ad7d
Merge pull request #44173 from CurtizJ/fix-fetch-projection
...
Fix `LOGICAL_ERROR` if fetch of projection was stopped
2022-12-14 15:55:26 +01:00
Maksim Kita
9b5ebf88cf
Fixed tests
2022-12-14 15:54:01 +01:00
Anton Popov
3593377f23
Merge pull request #44159 from ClickHouse/prevent-delete-nested-if-empty-part
...
Prevent dropping nested column if it creates empty part
2022-12-14 15:04:15 +01:00
Antonio Andelic
d0905fe408
Fix typo
2022-12-14 13:43:23 +01:00
Antonio Andelic
ff42a68940
Fix ReadonlyReplica
2022-12-14 12:02:12 +00:00
Alexander Tokmakov
816eba6149
Merge pull request #44202 from ClickHouse/fix_an_assertion_in_transactions
...
Fix an assertion in transactions
2022-12-14 14:25:09 +03:00
Maksim Kita
ddad879cb4
Analyzer support Set index
2022-12-14 11:16:53 +01:00
Alexander Tokmakov
f4c8281ccc
avoid outdating part twice
2022-12-13 21:24:26 +01:00
Nikolay Degterinsky
9b6d31b95d
Merge branch 'master' into perf_experiment
2022-12-13 17:15:07 +01:00
Antonio Andelic
2c57f2bd1f
Remove wrong assert
2022-12-13 14:11:08 +00:00
avogar
a1e5cb8c79
Fix possible Bad cast from type DB::IAST const* to DB::ASTLiteral const*
2022-12-13 13:12:44 +00:00
Alexander Tokmakov
a4df1cff52
Update ReplicatedMergeTreeQueue.cpp
2022-12-13 16:04:47 +03:00
Alexander Tokmakov
371e85d854
Merge pull request #44157 from ClickHouse/fix_session_expired_on_startup
...
Ignore "session expired" errors after BC check
2022-12-13 15:10:36 +03:00
Dmitry Novik
3d2fccab87
Merge branch 'master' into refector-function-node
2022-12-12 21:36:39 +01:00
Anton Popov
659a79e53f
fix LOGICAL_ERROR if fetch of projection was stopped
2022-12-12 18:31:20 +00:00
Nikolai Kochetov
e451137040
Add comments.
2022-12-12 18:26:07 +00:00
Anton Popov
0b396bab14
Merge remote-tracking branch 'upstream/master' into HEAD
2022-12-12 18:08:27 +00:00
Anton Popov
a2fee110b8
add query for waiting of loading parts and fix test_transactions
2022-12-12 14:33:42 +00:00
Vitaly Baranov
7d701e9b32
Merge pull request #44065 from vitlibar/fix-race-in-s3-multipart-upload
...
Fix race in s3 multipart upload
2022-12-12 15:28:05 +01:00
Alexander Tokmakov
d54f739836
ensure force_restore_data is removed from zk
2022-12-12 14:27:35 +01:00
Antonio Andelic
e18ac19ab3
Prevent dropping nested column if it can create empty part
2022-12-12 11:55:33 +00:00
Sema Checherinda
02133f68dd
Merge branch 'master' into mutation-breaks-source-part
2022-12-09 17:20:57 +01:00
Nikolai Kochetov
42d1b38ada
Merge pull request #43965 from ClickHouse/respect-adaptive-granularity-for-disk-remote
...
Respect adaptive granularity for disk remote.
2022-12-09 16:54:34 +01:00
Nikolai Kochetov
9e04473e8c
Fix comment.
2022-12-09 15:44:59 +00:00
Sergei Trifonov
7c37e48092
Merge pull request #43903 from ClickHouse/merge_tree_settings_constraints
...
Constraints on merge tree settings
2022-12-09 14:16:14 +01:00
Maksim Kita
3a3c6eb458
Merge pull request #43341 from kitaisreal/analyzer-support-indexes
...
Analyzer added indexes support
2022-12-09 16:05:40 +03:00
Han Fei
6740ae9fd7
Merge pull request #43304 from hanfei1991/hanfei/aysnc-insert
...
Support dedup for async insert.
2022-12-09 12:50:48 +01:00
Vitaly Baranov
517e84cfb9
Finalize buffers after writing.
2022-12-09 03:02:45 +01:00
Alexey Milovidov
f880644d3a
Merge branch 'master' into nv/fix-mutate-fetch
2022-12-09 00:51:54 +03:00
Sergei Trifonov
bae1519970
Merge branch 'master' into merge_tree_settings_constraints
2022-12-08 22:25:16 +01:00
Dmitry Novik
2826942ca3
Fixup assert condition
2022-12-08 16:41:54 +00:00
Maksim Kita
99de697ea2
Analyzer added indexes support
2022-12-08 17:30:32 +01:00
Yakov Olkhovskiy
0641066183
Merge branch 'master' into refactoring-ip-types
2022-12-08 11:12:05 -05:00
Sema Checherinda
46f96064ea
do not hardlink serialization.json in new part
2022-12-08 12:48:34 +01:00
Han Fei
1bde6d012d
fix
2022-12-08 09:57:33 +01:00
serxa
77d551788a
Fix relaxed "too many parts" threshold
2022-12-08 00:40:43 +00:00
Han Fei
3c11aca040
address comments
2022-12-07 23:40:52 +01:00
Han Fei
7a3dcd196e
Update src/Storages/MergeTree/EphemeralLockInZooKeeper.cpp
...
Co-authored-by: alesapin <alesapin@gmail.com>
2022-12-07 22:30:03 +01:00
Han Fei
6d6bcd9ba2
Update src/Storages/MergeTree/ReplicatedMergeTreeSink.cpp
...
Co-authored-by: alesapin <alesapin@gmail.com>
2022-12-07 22:28:55 +01:00
Dmitry Novik
25ecb75ca8
Merge remote-tracking branch 'origin/master' into refector-function-node
2022-12-07 18:36:50 +00:00
Dmitry Novik
a438b6efd8
Try to fix InvertedDAG construction
2022-12-07 18:27:37 +00:00
serxa
2306bd74d7
fix for ALTER TABLE ... MODIFY SETTING name=value
2022-12-06 20:46:10 +00:00
serxa
1b5b43ec87
fix checks on CREATE + added resolveSetting.h + improved system table
2022-12-06 19:09:18 +00:00
Nikolai Kochetov
9a4b3493fd
Merge branch 'master' into add-allocation-ptr-to-trace-log
2022-12-06 17:09:54 +01:00
Nikolai Kochetov
0ed82f3cc0
Merge branch 'master' into aggregating-in-order-from-query-plan
2022-12-06 16:36:49 +01:00
Yakov Olkhovskiy
031d2802d0
Merge pull request #43642 from HarryLeeIBM/hlee-s390x-bloom-filter
...
Fix endian issue in bloom filter serialization for s390x
2022-12-06 08:23:44 -05:00
Nikolai Kochetov
f75250eaf4
Fix style.
2022-12-05 21:27:54 +00:00
Nikolai Kochetov
b797157437
Respect adaptive index granularity settings while reading from remote disk.
2022-12-05 20:10:31 +00:00
Sema Checherinda
be00335ca4
can_remove_anything is false for temporary parts
2022-12-05 15:56:04 +01:00
Sema Checherinda
005cded791
Merge branch 'master' into logical-error-race-tmp-dirs
2022-12-04 18:18:42 +01:00
