Maksim Kita
ebbefd49c3
FullText bloom filter index added support for Array data type
2021-09-23 02:06:54 +03:00
Maksim Kita
d5a7e92b97
Bloom filter map added support for has function
2021-09-23 01:10:14 +03:00
Maksim Kita
1bf375e2b7
Merge pull request #29254 from kitaisreal/map-bloom-filter-index-updated
...
Updated bloom filter index for map type to work with mapKeys, mapValues functions
2021-09-23 00:31:14 +03:00
Maksim Kita
93208fca4b
Bloom filter for map data type dded support for IN, mapContains functions
2021-09-22 20:25:10 +03:00
Nikita Mikhaylov
558c74e826
Done
2021-09-22 11:52:47 +00:00
Maksim Kita
92fd6b376f
Updated bloom filter index for map type to work with mapKeys function
2021-09-22 12:45:48 +03:00
Alexey Milovidov
e9e77b4403
.tech -> .com
2021-09-22 03:22:57 +03:00
Maksim Kita
84e37c2d87
Fixed tests
2021-09-21 21:28:52 +03:00
Maksim Kita
6dadc521fb
MergeTreeIndexFullText updated for data type map
2021-09-21 16:55:48 +03:00
凌涛
e3e3b84f63
fix tests
2021-09-21 16:55:48 +03:00
凌涛
9461d6dbf2
modify code style
2021-09-21 16:55:47 +03:00
凌涛
e84f1000fb
optimze code to make sure only map function will be processed
2021-09-21 16:55:47 +03:00
凌涛
a3e657778a
bloom filter skipping index support Map with key of String type
2021-09-21 16:55:07 +03:00
凌涛
8f87d44958
bloom filter skipping index support Map with key of String type
2021-09-21 16:55:07 +03:00
Maksim Kita
15c9315ea1
MergeTreeIndexConditionBloomFilter updated for Map data type
2021-09-21 15:07:59 +03:00
fuwhu
8962672e59
add bloom filter skip index for map data type.
2021-09-21 10:59:16 +03:00
Anton Popov
55cf857aba
Merge pull request #28707 from azat/max_suspicious_broken_parts_bytes
...
Implement max_suspicious_broken_parts_bytes
2021-09-20 19:55:07 +03:00
Nikolai Kochetov
a790d391c0
Merge branch 'master' into rewrite-pushing-to-views
2021-09-20 19:43:15 +03:00
Anton Popov
e3e6258b95
Merge pull request #29163 from CurtizJ/small-refactoring
...
Slightly better code near reading of subcolumns
2021-09-19 16:14:52 +03:00
Anton Popov
f50098a059
fix style check
2021-09-19 00:13:00 +03:00
Anton Popov
1c80a95b36
slightly better code near reading of subcolumns
2021-09-18 22:34:54 +03:00
Amos Bird
6b3cd34cff
Disable projections when ARRAY JOIN is used.
2021-09-18 08:15:59 +08:00
Azat Khuzhin
b108c5e1b0
Implement max_suspicious_broken_parts_bytes
...
Usually broken parts are found from just created parts, which are
usually small enough, so it make sense to have a limit in bytes for
broken parts.
Default value was set to 1G.
2021-09-17 22:06:32 +03:00
Nikolai Kochetov
a8443bef4d
Fix build.
2021-09-17 20:52:26 +03:00
Nicolae Vartolomei
9a02061d9c
Rewrite part movement between shards logic and add kill support
...
See https://github.com/ClickHouse/ClickHouse/pull/24585#issuecomment-857735081
for extra context about the current implementation.
2021-09-17 18:11:32 +01:00
Nicolae Vartolomei
0381c634d4
Add support for user defined identifier on log entries
...
Sometimes we want to push a log entry once and only once. Because it is
not possible to create a sequential node in ZooKeeper and store its name
to a well known location in the same transaction we'll do it in the
other order. First somehow generate a unique identifier, then submit a
log entry with that identifier. Later, we can search through log entries
using the identifier we provided to find the node.
Required for part movement between shards.
2021-09-17 15:32:35 +01:00
Nikolai Kochetov
a8c3b02598
Merge branch 'master' into rewrite-pushing-to-views
2021-09-17 15:38:11 +03:00
mergify[bot]
c23f1fc160
Merge branch 'master' into fix_part_checker
2021-09-17 07:23:12 +00:00
Nikita Mikhaylov
c52b8ec083
Introduced MergeTask and MutateTask ( #25165 )
...
Introduced MergeTask and MutateTask
2021-09-17 00:19:58 +03:00
Nikolai Kochetov
341553febd
Fix build.
2021-09-16 20:40:42 +03:00
alesapin
606ab6d3e4
Typo fix
2021-09-16 18:40:25 +03:00
alesapin
8930a0d2bd
More precise initialization
2021-09-16 18:39:11 +03:00
alesapin
b4cbc86164
Fix bug
2021-09-16 17:59:55 +03:00
Anton Popov
f6191b98e7
Merge remote-tracking branch 'upstream/master' into HEAD
2021-09-15 17:47:22 +03:00
Kruglov Pavel
8a67c3cf44
Merge pull request #28636 from amosbird/nullable-index-fix
...
Fix nullable/lowcardinality primary key with constant conversion
2021-09-15 15:56:19 +03:00
Anton Popov
fc17936c12
Merge remote-tracking branch 'upstream/master' into HEAD
2021-09-14 23:02:30 +03:00
Nikolai Kochetov
b997214620
Rename QueryPipeline to QueryPipelineBuilder.
2021-09-14 20:48:18 +03:00
Nikolai Kochetov
7fc830ae2b
Rename QueryPipeline to QueryPipelineBuilder.
2021-09-14 19:28:41 +03:00
mergify[bot]
4993c7c527
Merge branch 'master' into fix_part_checker
2021-09-14 14:13:52 +00:00
Nikolai Kochetov
0e267c50b4
Merge branch 'master' into rewrite-pushing-to-views
2021-09-14 16:13:54 +03:00
alesapin
9441378b39
Add logs to queue because I cannot understand anything without them
2021-09-14 12:04:11 +03:00
Kseniia Sumarokova
d3e604e14b
Merge pull request #28603 from kssenii/fix-web-disk-2
...
Some fixes for static files disk
2021-09-14 08:24:24 +03:00
mergify[bot]
c54434084f
Merge branch 'master' into fix_part_checker
2021-09-13 13:19:34 +00:00
alesapin
1d05fab295
Merge pull request #28864 from ClickHouse/fix_replace_range_drop_part_reordering
...
Fix reordering of REPLACE_RANGE and DROP PART
2021-09-13 15:11:29 +03:00
alesapin
fa8b698b7e
Merge branch 'fix_part_checker' of github.com:ClickHouse/ClickHouse into fix_part_checker
2021-09-13 11:00:41 +03:00
alesapin
ab41384f63
Move queue initialization to restarting thread
2021-09-13 11:00:07 +03:00
alesapin
91e10adca5
Merge branch 'master' into fix_part_checker
2021-09-13 10:19:04 +03:00
kssenii
9ec6a9b4c9
Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-web-disk-2
2021-09-13 08:32:34 +03:00
alexey-milovidov
cdcfdbec7e
Merge pull request #28873 from CurtizJ/fix-subcolumns-compact-parts
...
Fix reading of subcolumns from compact parts
2021-09-12 15:54:07 +03:00
alexey-milovidov
ea13a8b562
Merge pull request #28659 from myrrc/improvement/tostring_to_magic_enum
...
Improving CH type system with concepts
2021-09-12 15:26:29 +03:00
mergify[bot]
267e56df8c
Merge branch 'master' into fix-web-disk-2
2021-09-11 15:45:43 +00:00
Anton Popov
8999a83320
add comment
2021-09-10 20:22:28 +03:00
Anton Popov
a63887413c
fix reading of subcolumns from compact parts
2021-09-10 20:19:18 +03:00
Anton Popov
0bb74f8eaf
Merge pull request #28762 from CurtizJ/fix-nested-1
...
Fix usage of nested columns with non-array columns with the same prefix [2]
2021-09-10 17:34:43 +03:00
Alexander Tokmakov
518f542c3e
fix reordering of REPLACE_RANGE and DROP PART
2021-09-10 17:32:45 +03:00
alesapin
d1dea0e95f
Update src/Storages/MergeTree/ReplicatedMergeTreePartCheckThread.cpp
...
Co-authored-by: Azat Khuzhin <a3at.mail@gmail.com>
2021-09-10 15:32:24 +03:00
Anton Popov
5cff615eca
Merge remote-tracking branch 'upstream/master' into HEAD
2021-09-10 13:40:48 +03:00
alexey-milovidov
2236a5df98
Merge pull request #28085 from myrrc/improvement/merge-tree-part-opt-parse
...
optional<> semantics for parsing MergeTreePartInfo and DetachedPartInfo
2021-09-10 00:04:03 +03:00
Nikolai Kochetov
f569a3e3f7
Merge branch 'master' into rewrite-pushing-to-views
2021-09-09 20:30:23 +03:00
alesapin
59edf6b5f6
Better logic on part checker
2021-09-09 18:19:12 +03:00
kssenii
0c013ed91b
Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-web-disk-2
2021-09-09 15:26:36 +03:00
Nikolai Kochetov
13eb93a9c0
Merge pull request #28700 from amosbird/projection-fix16
...
Fix crash on exception with projection aggregate
2021-09-09 15:24:05 +03:00
Anton Popov
8e5b3b2f6c
Merge remote-tracking branch 'upstream/master' into HEAD
2021-09-09 15:12:34 +03:00
Nikolai Kochetov
abfb1d20ea
Merge pull request #25734 from vdimir/join-arrayjoin-type-cast
...
Move join type inference to ExpressionAnalyzer from syntax stage + small refactor
2021-09-09 14:38:43 +03:00
Anton Popov
2ccdb6c22d
fix clang-tidy
2021-09-09 12:40:26 +03:00
Nikita Mikhaylov
4db5062d6b
Merge pull request #28374 from nikitamikhaylov/global-merge-executor
...
Introduced global executor for background MergeTree-related operations
2021-09-09 11:30:21 +03:00
alesapin
407c0bbb56
Merge pull request #28734 from ClickHouse/fix_race_on_mutation_pointer
...
Fix race on mutation_pointer update
2021-09-09 10:36:09 +03:00
Amos Bird
91293c7449
Fix crash on exception with projection aggregate
2021-09-09 10:43:56 +08:00
Nikita Mikhaylov
f12504b002
Fix stupid race
2021-09-08 20:30:54 +00:00
Anton Popov
9816014f6c
fix nested and scalar columns with dot, which is missed in part
2021-09-08 21:42:34 +03:00
kssenii
6108a4139c
Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-web-disk-2
2021-09-08 20:54:07 +03:00
Nikita Mikhaylov
1416b1ccba
Update MergeTreeBackgroundExecutor.cpp
2021-09-08 20:21:25 +03:00
Nikita Mikhaylov
dae9a01b4b
Fix build
2021-09-08 16:13:32 +00:00
Nikita Mikhaylov
3b7fa06540
Review fixes
2021-09-08 12:42:03 +00:00
alesapin
f8a6dfb0c4
Fix race on mutation_pointer update
2021-09-08 14:37:16 +03:00
Nikita Mikhaylov
ed37a22c37
Delete debug print
2021-09-08 09:35:18 +00:00
Nikita Mikhaylov
6062dd0021
Better
2021-09-08 00:21:21 +00:00
Azat Khuzhin
db0767a194
Implement detach_not_byte_identical_parts
...
Maybe useful for further analysis of non byte-identical parts.
2021-09-07 23:29:57 +03:00
Nikita Mikhaylov
231f64cf37
Fix unit test
2021-09-07 13:51:30 +00:00
Nikita Mikhaylov
9b38248e64
Remove scheduler thread
2021-09-07 13:13:23 +00:00
Nikita Mikhaylov
8a0765c60d
Review fixes (partially)
2021-09-07 12:45:39 +00:00
mergify[bot]
e9fbdec13a
Merge branch 'master' into improvement/merge-tree-part-opt-parse
2021-09-07 09:52:24 +00:00
Nikita Mikhaylov
cfe2207be3
Better
2021-09-06 22:07:41 +00:00
Nikita Mikhaylov
324ab97f50
Fix gcc build
2021-09-06 16:44:46 +00:00
Nikita Mikhaylov
7c39f84a5e
Merge upstream/master into global-merge-executor (using imerge)
2021-09-06 15:37:18 +00:00
Mike Kot
8e9aacadd1
Initial: replacing hardcoded toString for enums with magic_enum
2021-09-06 16:24:03 +02:00
Nikita Mikhaylov
17d65b3d7b
Allow allocations
2021-09-06 14:04:37 +00:00
Nikita Mikhaylov
ea0fbf81af
Renaming
2021-09-06 12:01:16 +00:00
Nikita Mikhaylov
0249015515
added priority queue
2021-09-06 11:37:51 +00:00
Nikolai Kochetov
2178ed21df
Merge pull request #28560 from amosbird/projection-fix15
...
Fix wrong header of minmax_count projection
2021-09-06 13:00:44 +03:00
mergify[bot]
8eba5f29a1
Merge branch 'master' into improvement/merge-tree-part-opt-parse
2021-09-06 09:53:46 +00:00
Amos Bird
8fac97692a
Fix nullable primary key with constant conversion
2021-09-06 15:11:37 +08:00
Nikita Mikhaylov
3eec8a3c2f
Use boost::circular_buffer
2021-09-03 22:15:20 +00:00
kssenii
6ee94c923f
Fix
2021-09-04 00:51:11 +03:00
Anton Popov
59128412ea
Merge remote-tracking branch 'upstream/master' into HEAD
2021-09-03 22:17:11 +03:00
Nikolai Kochetov
66a76ab70f
Rewrite PushingToViewsBlockOutputStream part 6
2021-09-03 20:29:36 +03:00
Kruglov Pavel
f559c34113
Merge pull request #28302 from amosbird/binaryconstantwrap
...
Always monotonic for non-zero division
2021-09-03 20:02:54 +03:00
Nikita Mikhaylov
26ab6ebc0a
finalize in destructor
2021-09-03 14:20:34 +00:00
Nikita Mikhaylov
b867eabe73
Use Poco::Event
2021-09-03 13:02:49 +00:00
alesapin
dd7493d871
Merge pull request #27038 from excitoon-favorites/altermaterializecolumn
...
`ALTER TABLE ... MATERIALIZE COLUMN`
2021-09-03 15:50:52 +03:00
Nikita Mikhaylov
218f0adca7
Merge upstream/master into global-merge-executor (using imerge)
2021-09-03 12:29:48 +00:00
Nikita Mikhaylov
ca7adb76b9
Owning future
2021-09-03 12:27:49 +00:00
Mike Kot
1f148719ce
Review fixes
2021-09-03 13:33:40 +02:00
Amos Bird
bb87d01579
Fix wrong header of minmax_count projection
2021-09-03 18:49:01 +08:00
Nikita Mikhaylov
d777c6cd70
Better [2]
2021-09-03 00:51:12 +00:00
Nikita Mikhaylov
60765b8c6e
Better
2021-09-03 00:45:19 +00:00
Nikita Mikhaylov
292a24abe8
Merge upstream/master into global-merge-executor (using imerge)
2021-09-03 00:34:24 +00:00
Nikita Mikhaylov
9257188512
Comments
2021-09-03 00:28:29 +00:00
Nikita Mikhaylov
7baad6adde
Fix build
2021-09-02 22:56:03 +00:00
Nikita Mikhaylov
cc7c221fad
Own PriorityQueue + prettifying the code
2021-09-02 21:31:32 +00:00
Nikita Mikhaylov
6624fa12ba
deleted set
2021-09-02 18:49:37 +00:00
Nikita Mikhaylov
a6fe91ca47
get rid of half of allocations
2021-09-02 17:40:29 +00:00
alesapin
464b8167d1
Update comment
2021-09-02 18:29:26 +03:00
alesapin
ba78a737dc
Fix detach/attach for ReplicatedVersionedCollapsingMergeTree after alter
2021-09-02 17:47:00 +03:00
alesapin
b592400e54
Merge pull request #28404 from ClickHouse/remove_outdated_settings
...
Remove obsolete settings for replicated fetches.
2021-09-02 14:03:12 +03:00
Nikita Mikhaylov
7f21cd7f3d
save
2021-09-02 10:39:27 +00:00
Nikolai Kochetov
5f0b1bc4e3
Merge pull request #26286 from amosbird/projection-improve2
...
Virtual projection for min max indices.
2021-09-02 13:03:30 +03:00
Anton Popov
5e421ab272
Merge remote-tracking branch 'upstream/master' into HEAD
2021-09-02 02:42:50 +03:00
mergify[bot]
5d299fbdee
Merge branch 'master' into remove_outdated_settings
2021-09-01 14:07:48 +00:00
Nikita Mikhaylov
c7727f093d
Merge upstream/master into global-merge-executor (using imerge)
2021-08-31 23:34:11 +00:00
Nikita Mikhaylov
ac5316ce98
Stress test + reworker executor
2021-08-31 23:20:23 +00:00
alexey-milovidov
4cc0b0298c
Merge pull request #28269 from amosbird/fixweirdcode
...
Better nullable primary key implementation
2021-09-01 00:48:45 +03:00
Nikita Mikhaylov
28eff0e369
save
2021-08-31 18:07:24 +00:00
alesapin
2c7a51d816
Merge pull request #28366 from azat/remove-Temporary-part-fix-v2
...
Fix removing of parts in a Temporary state (v2 followup)
2021-08-31 19:13:15 +03:00
Nikita Mikhaylov
dbc950caa4
added a test
2021-08-31 14:54:24 +00:00
Nikolai Kochetov
5be05c2ef9
Rewrite PushingToViewsBlockOutputStream part 3.
2021-08-31 16:50:07 +03:00
Mike Kot
4194ed2ec7
Merge remote-tracking branch 'upstream/master' into improvement/merge-tree-part-opt-parse
2021-08-31 14:35:14 +02:00
Nikita Mikhaylov
e39f9aabde
delete useless file
2021-08-31 12:29:51 +00:00
alesapin
9a3bbf44e3
Better description
2021-08-31 15:28:17 +03:00
vdimir
e8e26463bf
Change signature of ASTSelectQuery::arrayJoinExpressionList
2021-08-31 15:27:44 +03:00
alesapin
921e51e061
Remove some obsolete settings for replicated fetches
2021-08-31 15:22:56 +03:00
Nikita Mikhaylov
3027efe860
after merge with master
2021-08-31 12:09:35 +00:00
Nikita Mikhaylov
f8d4f04294
Merge upstream/master into global-merge-executor (using imerge)
2021-08-31 11:52:11 +00:00
Nikita Mikhaylov
1adb9bfe23
better
2021-08-31 11:02:39 +00:00
alesapin
525999145e
Merge pull request #26791 from ClickHouse/async-reads
...
Experiment with asynchronous readers
2021-08-31 13:17:45 +03:00
Nikolai Kochetov
8143c2e2b8
Merge pull request #28353 from ClickHouse/another-try-to-fix-BackgroundPoolTask-decrement
...
Another try to fix BackgroundPoolTask decrement.
2021-08-31 11:34:26 +03:00
Alexey Milovidov
ca6f6306f3
Merge branch 'master' into async-reads
2021-08-31 02:52:58 +03:00
Nikita Mikhaylov
c4416906c8
done
2021-08-30 19:37:03 +00:00
Azat Khuzhin
2c0390f326
Fix removing of parts in a Temporary state (v2 followup)
...
CI reports [1]:
<Fatal> : Logical error: 'Deleting data part all_0_0_0_3 doesn't exist'.
...
<Fatal> BaseDaemon: 8. ./obj-x86_64-linux-gnu/../src/Storages/MergeTree/MergeTreeData.cpp:1254: DB::MergeTreeData::removePartsFinally() @ 0x1f94dffa in /usr/bin/clickhouse
[1]: https://clickhouse-test-reports.s3.yandex.net/0/855a53ff8160c4638fe345b0d26e062804ba790a/stress_test_(debug).html#fail1
Follow-up for: #28221
2021-08-30 21:07:26 +03:00
Azat Khuzhin
0828548b88
Fix intersecting parts due to new part had been replaced with an empty part
...
AFAICS the problem is that some parts may be replaced with empty parts
(after #25820 ), and removed by the cleanup thread, due to it is empty
[1] (while it should not be deleted since it can download source part):
<details>
```
2021.08.18 20:11:22.687933 [ 341 ] {} <Trace> test_dpefxp.alter_table_1 (0758ca24-90e7-452c-8758-ca2490e7252c): Created log entry for mutation -1_115_115_0_146
...
2021.08.18 20:11:22.707609 [ 22825 ] {} <Trace> test_dpefxp.alter_table_6 (766ae414-e113-4965-b66a-e414e1137965): Executing log entry to mutate part -1_115_115_0 to -1_115_115_0_146
2021.08.18 20:11:22.707643 [ 22825 ] {} <Debug> test_dpefxp.alter_table_6 (766ae414-e113-4965-b66a-e414e1137965): Source part -1_115_115_0 for -1_115_115_0_146 is not ready; will try to fetch it instead
...
2021.08.18 20:11:22.709397 [ 333 ] {} <Trace> test_dpefxp.alter_table_6 (ReplicatedMergeTreeQueue): Not executing log entry queue-0000001579 for part -1_115_115_0 because it is covered by part -1_115_115_0_146 that is currently executing.
2021.08.18 20:11:22.718861 [ 22825 ] {} <Information> test_dpefxp.alter_table_6 (766ae414-e113-4965-b66a-e414e1137965): DB::Exception: No active replica has part -1_115_115_0_146 or covering part
...
2021.08.18 20:11:27.936829 [ 295 ] {} <Information> test_dpefxp.alter_table_6 (766ae414-e113-4965-b66a-e414e1137965): Going to replace lost part -1_115_115_0_146 with empty part
2021.08.18 20:11:27.957839 [ 295 ] {} <Information> test_dpefxp.alter_table_6 (766ae414-e113-4965-b66a-e414e1137965): Created empty part -1_115_115_0_146 instead of lost part
...
2021.08.18 20:11:28.731635 [ 257 ] {} <Trace> test_dpefxp.alter_table_6 (ReplicatedMergeTreeCleanupThread): Cleared 190 old blocks from ZooKeeper
...
2021.08.18 20:11:28.734507 [ 257 ] {} <Trace> test_dpefxp.alter_table_6 (766ae414-e113-4965-b66a-e414e1137965): Will try to insert a log entry to DROP_RANGE for part: -1_115_115_0_146
2021.08.18 20:11:28.779373 [ 22837 ] {} <Debug> test_dpefxp.alter_table_6 (766ae414-e113-4965-b66a-e414e1137965): Removed 1 parts inside -1_115_115_0_146.
...
2021.08.18 20:11:28.792600 [ 273 ] {} <Trace> test_dpefxp.alter_table_6 (766ae414-e113-4965-b66a-e414e1137965): Created log entry /clickhouse/tables/00993_system_parts_race_condition_drop_zookeeper_test_dpefxp/alter_table/log/log-0000003459 for merge -1_111_118_1
...
2021.08.18 20:11:28.910988 [ 354 ] {} <Error> test_dpefxp.alter_table_7 (ReplicatedMergeTreeQueue): Code: 49. DB::Exception: Part -1_111_118_1 intersects next part -1_115_115_0_146. It is a bug. (LOGICAL_ERROR), Stack trace (when copying this message, always include the lines below):
2021.08.18 20:11:31.282160 [ 305 ] {} <Error> test_dpefxp.alter_table_2 (ReplicatedMergeTreeQueue): Code: 49. DB::Exception: Part -1_111_118_1 intersects next part -1_115_115_0_146. It is a bug. (LOGICAL_ERROR), Stack trace (when copying this message, always include the lines below):
```
</details>
[1]: https://clickhouse-test-reports.s3.yandex.net/27752/59e3cb18f4e53c453951267b5599afeb664290d8/functional_stateless_tests_(release,_wide_parts_enabled).html
2021-08-30 21:05:46 +03:00
Nikolai Kochetov
ab45c412d0
Another try to fix BackgroundPoolTask decrement.
2021-08-30 15:45:32 +03:00
alesapin
531079c452
Merge pull request #28221 from azat/remove-Temporary-part-fix
...
Fix removing of parts in a Temporary state
2021-08-30 12:39:08 +03:00
Vladimir Chebotarev
de5daa22ab
Revert "MATERIALIZE COLUMN xxx FINAL"
...
This reverts commit bf1430c1ec
2021-08-30 11:50:26 +03:00
Vladimir Chebotarev
0d6e0181d6
MATERIALIZE COLUMN xxx FINAL
2021-08-30 11:49:59 +03:00
Vladimir Chebotarev
ce0850007f
1. Draw some circles.
2021-08-30 11:49:26 +03:00
alexey-milovidov
77f26e8673
Merge pull request #28299 from amosbird/fixtmprowsource
...
Use real tmp file instead of predefined “rows_sources" for vertical merges
2021-08-29 21:49:59 +03:00
Alexey Milovidov
e4f3d97b52
Merge branch 'master' of github.com:yandex/ClickHouse into async-reads
2021-08-29 21:42:14 +03:00
alexey-milovidov
3d3afbdb35
Merge pull request #28193 from amosbird/alterprimarykey2
...
Reserve protocol number for ALTER PRIMARY KEY.
2021-08-29 20:37:19 +03:00
Amos Bird
9e7ae5dd6d
Use real tmp file instead of predefined one
2021-08-29 16:54:15 +08:00
Amos Bird
4624bf70b0
Always monotonic for non-zero division
2021-08-28 23:33:18 +08:00
Amos Bird
f2374a6916
Better nullable primary key implementation.
2021-08-28 17:48:28 +08:00
Amos Bird
b68857d086
Simplify projection, add minmax_count projection.
2021-08-28 11:25:37 +08:00
Alexey Milovidov
79e0433ba7
Merge branch 'master' of github.com:yandex/ClickHouse into async-reads
2021-08-28 01:19:16 +03:00
Amos Bird
ce96b2c35a
bump
2021-08-27 20:44:26 +08:00
tavplubix
703101fe4d
Merge pull request #27931 from ClickHouse/wait_for_all_replicas_timeouts
...
Avoid too long waiting for inactive replicas
2021-08-27 14:31:36 +03:00
Azat Khuzhin
19458a0c61
Fix removing of parts in a Temporary state
...
Parts in a temporary state is not exists in data_parts_by_info, so do
not try to search there and throw LOGICAL_ERROR in case of failure:
<details>
```
02:45:49.037546 [ 5890 ] {} <Error> test_iy9rta.concurrent_kill_4 (526aa7c8-db2a-4f0e-926a-a7c8db2a9f0e): Code: 40. DB::Exception: Part all_0_0_0_1 from r11 has different columns hash. (CHECKSUM_DOESNT_MATCH) (version 21.10.1.7910 (official build)). Data after mutation is not byte-identical to data on another replicas. We will download merged part from replica to force byte-identical result.
02:45:49.049422 [ 5890 ] {} <Trace> test_iy9rta.concurrent_kill_4 (526aa7c8-db2a-4f0e-926a-a7c8db2a9f0e): Trying to immediately remove part all_0_0_0_1 (state Temporary)
02:45:49.060210 [ 5890 ] {} <Fatal> : Logical error: 'Part all_0_0_0_1 doesn't exist'.
02:47:01.572508 [ 29208 ] {} <Fatal> BaseDaemon: (version 21.10.1.7910 (official build), build id: 9309CECED9A0D32CBB00BB8EC77B91456679868E) (from thread 5890) (no query) Received signal Aborted (6)
...
02:47:06.815000 [ 29208 ] {} <Fatal> BaseDaemon: 8. ./obj-x86_64-linux-gnu/../src/Storages/MergeTree/MergeTreeData.cpp:2593: DB::MergeTreeData::tryRemovePartImmediately(std::__1::shared_ptr<DB::IMergeTreeDataPart const>&&) @ 0x1f959e0d in /usr/bin/clickhouse
02:47:06.816309 [ 29209 ] {} <Fatal> BaseDaemon: 8. ./obj-x86_64-linux-gnu/../src/Storages/MergeTree/MergeTreeData.cpp:2593: DB::MergeTreeData::tryRemovePartImmediately(std::__1::shared_ptr<DB::IMergeTreeDataPart const>&&) @ 0x1f959e0d in /usr/bin/clickhouse
02:47:09.455665 [ 29208 ] {} <Fatal> BaseDaemon: 9. ./obj-x86_64-linux-gnu/../src/Storages/StorageReplicatedMergeTree.cpp:1939: DB::StorageReplicatedMergeTree::tryExecutePartMutation(DB::ReplicatedMergeTreeLogEntry const&) @ 0x1f5f2bf6 in /usr/bin/clickhouse
02:47:09.468738 [ 29209 ] {} <Fatal> BaseDaemon: 9. ./obj-x86_64-linux-gnu/../src/Storages/StorageReplicatedMergeTree.cpp:1939: DB::StorageReplicatedMergeTree::tryExecutePartMutation(DB::ReplicatedMergeTreeLogEntry const&) @ 0x1f5f2bf6 in /usr/bin/clickhouse
02:47:11.776857 [ 29208 ] {} <Fatal> BaseDaemon: 10. ./obj-x86_64-linux-gnu/../src/Storages/StorageReplicatedMergeTree.cpp:1581: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x1f5e484c in /usr/bin/clickhouse
02:47:11.904232 [ 29209 ] {} <Fatal> BaseDaemon: 10. ./obj-x86_64-linux-gnu/../src/Storages/StorageReplicatedMergeTree.cpp:1581: DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry&) @ 0x1f5e484c in /usr/bin/clickhouse
02:47:13.941811 [ 29208 ] {} <Fatal> BaseDaemon: 11. ./obj-x86_64-linux-gnu/../src/Storages/StorageReplicatedMergeTree.cpp:3176: DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>)::$_18::operator() shared_ptr<DB::ReplicatedMergeTreeLogEntry>&) const @ 0x1f65faa5 in /usr/bin/clickhouse
02:47:14.477158 [ 29209 ] {} <Fatal> BaseDaemon: 11. ./obj-x86_64-linux-gnu/../src/Storages/StorageReplicatedMergeTree.cpp:3176: DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>)::$_18::operator() shared_ptr<DB::ReplicatedMergeTreeLogEntry>&) const @ 0x1f65faa5 in /usr/bin/clickhouse
02:47:16.475373 [ 29208 ] {} <Fatal> BaseDaemon: 12. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3676: decltype(std::__1::forward<DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>))(std::__1::forward<std::__1::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&>(fp0))) std::__1::__invoke<DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>)::$_18&, std::__1::shared_ptr<DB::ReplicatedMEntry>&>(DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>)::$_18&, std::__1::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&) @ 0x1f65fa32 in /usr/bin/clickhouse
02:47:16.970325 [ 29209 ] {} <Fatal> BaseDaemon: 12. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3676: decltype(std::__1::forward<DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>))(std::__1::forward<std::__1::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&>(fp0))) std::__1::__invoke<DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>)::$_18&, std::__1::shared_ptr<DB::ReplicatedMEntry>&>(DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>)::$_18&, std::__1::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&) @ 0x1f65fa32 in /usr/bin/clickhouse
02:47:18.979481 [ 29208 ] {} <Fatal> BaseDaemon: 13. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/__functional_base:317: bool std::__1::__invoke_void_return_wrapper<bool>::__call<DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplireeQueue::SelectedEntry>)::$_18&, std::__1::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&>(DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>)::$_18&, std::__1::shared_ptr<DB::ReplicatedMergeTreeLogE x1f65f9f2 in /usr/bin/clickhouse
02:47:19.450807 [ 29209 ] {} <Fatal> BaseDaemon: 13. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/__functional_base:317: bool std::__1::__invoke_void_return_wrapper<bool>::__call<DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplireeQueue::SelectedEntry>)::$_18&, std::__1::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&>(DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::SelectedEntry>)::$_18&, std::__1::shared_ptr<DB::ReplicatedMergeTreeLogE x1f65f9f2 in /usr/bin/clickhouse
02:47:21.055007 [ 29208 ] {} <Fatal> BaseDaemon: 14. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:1608: std::__1::__function::__default_alloc_func<DB::StorageReplicatedMergeTree::processQueueEntry(std::__1::shared_ptr<DB::ReplicatedMergeTreeQueue::ry>)::$_18, bool (std::__1::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&)>::operator()(std::__1::shared_ptr<DB::ReplicatedMergeTreeLogEntry>&) @ 0x1f65f9b0 in /usr/bin/clickhouse
02:47:23.546946 [ 413 ] {} <Fatal> Application: Child process was terminated by signal 6.
```
</details>
CI: https://clickhouse-test-reports.s3.yandex.net/0/4a8b82232c11512232df3ecdf4ffaec287116ad5/stress_test_(debug).html#fail1
2021-08-27 10:08:44 +03:00
Alexey Milovidov
46732e781f
Merge branch 'master' of github.com:yandex/ClickHouse into async-reads
2021-08-27 01:57:48 +03:00
Mike Kot
1f3701d485
Disk empty name fix
2021-08-26 19:28:33 +02:00
Amos Bird
103e0abe1c
Reserve protocol number for ALTER PRIMARY KEY.
2021-08-26 22:08:58 +08:00
Amos Bird
0169fce78e
Projection bug fixes and refactoring.
2021-08-26 19:09:31 +08:00
Ilya Yatsishin
5553a064be
Merge pull request #28139 from lthaooo/fix_prewhere_total_size_of_queried_columns_bug
...
Fix bug in MergeTreeWhereOptimizer
2021-08-26 10:42:23 +03:00
Anton Popov
6d3274c22c
Merge remote-tracking branch 'upstream/master' into HEAD
2021-08-26 03:31:46 +03:00
Alexey Milovidov
7dd0146223
Fix error
2021-08-26 01:56:08 +03:00
Alexey Milovidov
3cb9cf7242
Merge branch 'master' of github.com:yandex/ClickHouse into async-reads
2021-08-26 01:55:46 +03:00
Maksim Kita
2403686bb9
Merge pull request #28135 from amosbird/projection-improve3
...
Get rid of useless projection columns during merge
2021-08-25 22:14:38 +03:00
Mike Kot
407af64280
Merge remote-tracking branch 'upstream/master' into improvement/merge-tree-part-opt-parse
2021-08-25 18:24:43 +02:00
lthaooo
eecd5b21c9
fix total_size_of_queried_columns bug
2021-08-25 23:02:13 +08:00
Amos Bird
a6b2a6ee36
Get rid of useless projection columns during merge
2021-08-25 20:54:27 +08:00
lthaooo
314ba1e682
Merge branch 'master' into materialize_ttl_recalculate_only
2021-08-25 10:56:24 +08:00
Alexey Milovidov
848c2ec23e
Tuneable prefetch
2021-08-25 02:38:08 +03:00
Alexey Milovidov
fd27a2faef
Remove imperfect code
2021-08-25 02:05:55 +03:00
Alexey Milovidov
7c1d0a3baf
Progress on development
2021-08-25 01:24:47 +03:00
Alexey Milovidov
f7ca674b0c
Progress on development
2021-08-25 01:07:06 +03:00
Alexey Milovidov
8f57216180
Progress on development
2021-08-25 00:45:58 +03:00
Alexey Milovidov
4d9ad3725d
Merge branch 'master' into async-reads
2021-08-25 00:45:40 +03:00
Mike Kot
d621605954
Merge remote-tracking branch 'upstream/master' into improvement/merge-tree-part-opt-parse
2021-08-24 19:43:33 +02:00
Mike Kot
f85e2e027c
Optional semantics for [Detached]MergeTreePartInfo
2021-08-24 19:43:09 +02:00
Vitaly Baranov
64f69462c6
Merge pull request #27921 from vitlibar/add-tests-for-field-init
...
Allow implicit cast bool to Field
2021-08-24 17:15:44 +03:00
mergify[bot]
ae22bab2c4
Merge branch 'master' into materialize_ttl_recalculate_only
2021-08-24 08:49:19 +00:00
Vitaly Baranov
63e8bc1f20
Added new commands BACKUP and RESTORE.
2021-08-23 23:41:45 +03:00
Anton Popov
0db8b524f0
Merge pull request #27984 from BraulioVM/optimize/has-all-bloom-filter
...
Make `hasAll` filter condition leverage bloom filters
2021-08-23 22:19:50 +03:00
Alexander Tokmakov
4a4a0b4823
fix
2021-08-23 16:32:07 +03:00
Alexander Tokmakov
cc9c2fd63b
make code better
2021-08-23 15:57:50 +03:00
Yatsishin Ilya
17bb938541
fix style
2021-08-23 13:59:01 +03:00
Amos Bird
a2256b1307
Dedicated Mark/Uncompressed cache for skip indices
2021-08-23 15:27:31 +08:00
Vitaly Baranov
89166a655a
Init SettingFieldBool with true or false, not with 1 or 0.
2021-08-23 09:47:36 +03:00
Vitaly Baranov
ff1fad8521
Use setSetting() with implicit conversion true/false -> Field.
2021-08-23 08:57:00 +03:00
Braulio Valdivielso
d48f6122e3
clean up implementation a bit
2021-08-22 10:06:30 +01:00
Braulio Valdivielso
e95563fb82
leverage bloom filter index for 'hasAll'
...
Fixes #26151 . For now, this is a draft implementation. There is some
cleanup that can be done, but the new test is passing
2021-08-21 20:14:56 +01:00
Alexander Tokmakov
59eb3aa9a9
avoid too long waiting for inactive replicas
2021-08-20 15:59:57 +03:00
nvartolomei
c09c90125f
Merge branch 'master' into nv/last-queue-update-exception
2021-08-19 10:29:16 +01:00
Nikolai Kochetov
7dcff5c90e
Merge pull request #27742 from amosbird/projection-improvement3
...
Improve projection analysis.
2021-08-19 12:11:20 +03:00
alesapin
0aec151719
Merge pull request #27808 from ClickHouse/fix_replicas_may_diverge
...
Fix a couple of bugs that may cause replicas to diverge
2021-08-19 09:58:07 +03:00
Alexander Tokmakov
09ff66da0e
fix a couple of bugs that may cause replicas to diverge
2021-08-18 12:50:46 +03:00
alesapin
5cc98c67a4
Fix
2021-08-18 11:56:18 +03:00
mergify[bot]
f11e396151
Merge branch 'master' into nv/last-queue-update-exception
2021-08-18 07:00:50 +00:00
Nikita Mikhaylov
cc8ad0a4e7
Merge pull request #27733 from Algunenano/i27620_take2
...
Refactor arrayJoin check on partition expressions
2021-08-17 15:57:16 +03:00
Nikolai Kochetov
98eb619b4b
Merge pull request #23367 from Avogar/short-circuit
...
Implement short circuit function evaluation
2021-08-17 12:36:07 +03:00
Amos Bird
ea7dc495cf
Better code.
2021-08-17 12:32:49 +08:00
Raúl Marín
90f224807a
Style
2021-08-16 17:31:37 +02:00
Amos Bird
90881aab09
Better code style
2021-08-16 22:17:07 +08:00
Raúl Marín
3ec904a953
Only analyze if tuple has arguments
2021-08-16 14:47:41 +02:00
Amos Bird
b162a2b699
Improve projection analysis.
...
Remove duplicate index analysis and avoid possible invalid limit checks
during projection analysis.
2021-08-16 20:11:57 +08:00
Raúl Marín
34099bc196
Refactor arrayJoin check on partition expressions
2021-08-16 12:46:18 +02:00
Alexey Milovidov
d184b79bba
Progress on async reads.
2021-08-16 03:00:32 +03:00
alexey-milovidov
4a36366925
Merge pull request #27648 from Algunenano/i27620
...
Disable arrayJoin on partition expressions
2021-08-16 02:53:36 +03:00
Alexey Milovidov
0cd3f25ba4
Merge remote-tracking branch 'origin/master' into async-reads
2021-08-16 02:41:01 +03:00
alexey-milovidov
76b0502482
Merge pull request #27690 from kitaisreal/remove-dense-hash-map-set
...
Removed DenseHashMap, DenseHashSet
2021-08-16 02:39:41 +03:00
Maksim Kita
5f5470c2cd
Removed DenseHashMap, DenseHashSet
2021-08-15 13:32:56 +03:00
Alexey Milovidov
020beb885d
Merge branch 'master' into async-reads
2021-08-15 12:08:19 +03:00
mergify[bot]
bf426cfb96
Merge branch 'master' into fix_intersecting_parts
2021-08-15 09:07:20 +00:00
mergify[bot]
353a324887
Merge branch 'master' into i27620
2021-08-15 02:32:26 +00:00
alexey-milovidov
ab4415a22b
Update MergeTreeData.cpp
2021-08-15 05:28:38 +03:00
Alexander Tokmakov
c534363abe
fix intersecting parts
2021-08-13 20:33:10 +03:00
Raúl Marín
73cb7d55ec
Disable arrayJoin on partition expressions
2021-08-13 16:18:46 +02:00
mergify[bot]
319e5af6eb
Merge branch 'master' into short-circuit
2021-08-13 13:01:46 +00:00
Nikolai Kochetov
66fdb859f3
Merge pull request #27528 from amosbird/projection-fix12
...
Fix projection materialization with missing columns
2021-08-13 15:20:51 +03:00
Kseniia Sumarokova
0917488cec
Merge pull request #27382 from kssenii/cast-internal
...
add Cast internal function
2021-08-13 13:13:51 +03:00
Nikolai Kochetov
5e5a6ff654
Merge branch 'master' into qoega-fix-access-gtest-in-arcadia
2021-08-12 12:02:15 +03:00
mergify[bot]
80eaea1c51
Merge branch 'master' into short-circuit
2021-08-12 08:38:45 +00:00
kssenii
dcc6f597dc
Move templates from FunctionsConvertion
2021-08-11 19:09:51 +00:00
kssenii
7555fc19fd
Revert "Move some templates"
...
This reverts commit 7f247becca
2021-08-11 18:49:25 +00:00
mergify[bot]
68a52fc065
Merge branch 'master' into projection-fix12
2021-08-11 17:08:04 +00:00
Alexander Kuzmenkov
8f9845722a
Merge pull request #27250 from azat/minmax-nullable-new-format
...
Add new index data skipping minmax index format for proper Nullable support
2021-08-11 13:56:16 +03:00
kssenii
7d8b694a9f
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into cast-internal
2021-08-11 07:37:21 +00:00
kssenii
7f247becca
Move some templates
2021-08-11 07:32:46 +00:00
Alexey Milovidov
08f8511854
Fix Style
2021-08-11 06:52:28 +03:00
alexey-milovidov
54d8282209
Merge pull request #27248 from azat/mt-mutation-stuck-fix
...
Fix mutation stuck on invalid partitions in non-replicated MergeTree
2021-08-11 06:10:15 +03:00
alexey-milovidov
a0a5c0da32
Update BackgroundJobsExecutor.cpp
2021-08-11 06:08:30 +03:00
Amos Bird
e63c26edb7
Fix projection materialization with missing columns
2021-08-10 21:47:27 +08:00
kssenii
eb4746ac69
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into cast-internal
2021-08-10 09:38:41 +00:00
Nikolai Kochetov
a1ec7f75c5
Merge branch 'master' into qoega-fix-access-gtest-in-arcadia
2021-08-10 11:31:47 +03:00
Nikolai Kochetov
bb4c11cd27
Merge pull request #27298 from ClickHouse/fix-23515
...
Fix bug from #23515 .
2021-08-09 23:25:30 +03:00
Nikolai Kochetov
8cc493a3cd
Try fix build.
2021-08-09 18:09:29 +03:00
Pavel Kruglov
70b51133c1
Try to simplify code
2021-08-09 18:01:08 +03:00
Pavel Kruglov
0662df8b76
Fix performance with JIT, add arguments to function isSuitableForShortCircuitArgumentsExecution
2021-08-09 17:54:14 +03:00
Pavel Kruglov
e792fa588f
Mark all Functions as sutable or not for executing as short circuit arguments
2021-08-09 17:50:09 +03:00
Nicolae Vartolomei
3f291b024a
Use plain mutex instead of MultiVersion
2021-08-09 13:58:23 +01:00
Nikolai Kochetov
4a00e402ae
Fix spelling.
2021-08-09 12:09:09 +03:00
Nikolai Kochetov
372293b85d
Merge branch 'master' into fix-27179
2021-08-09 11:59:22 +03:00
Alexey Milovidov
8b9da6fe26
Merge branch 'master' into async-reads
2021-08-09 03:33:32 +03:00
kssenii
7991bb6e83
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into cast-internal
2021-08-08 21:30:17 +00:00
Azat Khuzhin
038241b6ed
Add new index data skipping minmax index format for proper Nullable support
...
Note, that it cannot be done w/o new extension, since index does not
have any header.
v2: use IDisk interface for existence check
v3: remove extra file existence check
v4: fix MATERIALIZE INDEX
2021-08-08 19:30:12 +03:00
Azat Khuzhin
5139067631
Guard BackgroundJobsExecutor from thread termination in case of uncaught exception
2021-08-08 19:27:58 +03:00
alexey-milovidov
ba3fbbaab0
Update ReplicatedMergeTreeMergeStrategyPicker.cpp
2021-08-08 04:43:59 +03:00
mergify[bot]
9869c8bb10
Merge branch 'master' into zero_copy_by_default
2021-08-07 16:53:44 +00:00
kssenii
9ca422f0c5
Introduce CAST for internal usage
2021-08-07 09:03:10 +00:00
Alexander Tokmakov
5e2bfd5ba1
fix partition id validation
2021-08-06 23:56:51 +03:00
Nikolai Kochetov
3cbeab8959
Fix incorrect row-level filtering
2021-08-06 21:16:06 +03:00
Nikolai Kochetov
7320411a7f
Fix bug from #23515 .
2021-08-06 16:39:11 +03:00
