Amos Bird
ef918aeaff
Rebase and fix
2021-05-11 18:12:28 +08:00
Amos Bird
9c069ebdbf
support prewhere, row_filter, read_in_order and decent projection selection
...
TODO set index analysis in projection
2021-05-11 18:12:27 +08:00
Amos Bird
f7f949c1f9
Fix aggregation keys order
2021-05-11 18:12:27 +08:00
Amos Bird
e1e560765c
Fix missing columns
2021-05-11 18:12:27 +08:00
Amos Bird
35961c0c5d
Use ActionsDAG to rewrite projection queries
2021-05-11 18:12:27 +08:00
Nikolai Kochetov
bf95b684a7
Remove copy constructor from ProjectionDescription
2021-05-11 18:12:27 +08:00
Nikolai Kochetov
29d7038e30
Add ProjectionDescription::Type enum. Remove some checks.
2021-05-11 18:12:27 +08:00
Amos Bird
ebaf42a448
Reformat and fix some tests
2021-05-11 18:12:27 +08:00
Nikolai Kochetov
672cfedd13
Disable normal projection by the number of granules.
2021-05-11 18:12:26 +08:00
Nikolai Kochetov
95431168c2
Try to enable normal projections.
2021-05-11 18:12:26 +08:00
Amos Bird
483bd165e7
Check if pipeline is simple and add more comments
2021-05-11 18:12:26 +08:00
Nikolai Kochetov
427f444ba4
Fix prewhere.
2021-05-11 18:12:26 +08:00
Nikolai Kochetov
3d36fb57eb
Cleanup code.
2021-05-11 18:12:26 +08:00
Nikolai Kochetov
3296c9292f
Try to merge projectons faster.
2021-05-11 18:12:26 +08:00
Amos Bird
cd6414639e
add metadata_snapshot to getQueryProcessingStage
2021-05-11 18:12:26 +08:00
Amos Bird
264cff6415
Projections
...
TODO (suggested by Nikolai)
1. Build query plan fro current query (inside storage::read) up to WithMergableState
2. Check, that plan is simple enough: Aggregating - Expression - Filter - ReadFromStorage (or simplier)
3. Check, that filter is the same as filter in projection, and also expression calculates the same aggregation keys as in projection
4. Return WithMergableState if projection applies
3 will be easier to do with ActionsDAG, cause it sees all functions, and dependencies are direct (but it is possible with ExpressionActions also)
Also need to figure out how prewhere works for projections, and
row_filter_policies.
wip
2021-05-11 18:12:23 +08:00
Maksim Kita
fdaffe869d
Merge pull request #24007 from ClickHouse/fix-bad-code-2
...
Fix bad code
2021-05-11 12:03:53 +03:00
Kruglov Pavel
49e7ed6e60
Merge pull request #23673 from amosbird/partitionvalue
...
Add _partition_value virtual column
2021-05-11 11:23:00 +03:00
Alexey Milovidov
eb8a4ffe4b
Fix bad code
2021-05-11 02:43:58 +03:00
Alexey Milovidov
b2ca5cd98b
Merge branch 'master' into normalize-bigint
2021-05-11 02:05:40 +03:00
Alexander Tokmakov
eef899ae63
better code, add test
2021-05-10 21:03:37 +03:00
Vladimir
d02d14fa00
Merge pull request #23846 from excitoon-favorites/s3regions
2021-05-10 12:14:08 +03:00
alexey-milovidov
1ea637d996
Merge pull request #23970 from ClickHouse/pvs-studio-fixes-6
...
PVS-Studio fixes, part 6
2021-05-09 21:45:19 +03:00
Alexey Milovidov
9753ddc8a0
Merge branch 'master' of github.com:yandex/ClickHouse into normalize-bigint
2021-05-09 18:54:29 +03:00
Alexey Milovidov
e23a0d8f7e
Fix warnings by PVS-Studio
2021-05-08 23:57:08 +03:00
Alexey Milovidov
367f7fe6c9
Fix warnings by PVS-Studio
2021-05-08 23:57:08 +03:00
Alexey Milovidov
513c6891db
Fix warnings by PVS-Studio
2021-05-08 23:54:49 +03:00
Alexey Milovidov
037d06bb2a
Fix warnings by PVS-Studio
2021-05-08 23:50:12 +03:00
Alexey Milovidov
2d3647c5b0
Merge branch 'master' into normalize-bigint
2021-05-08 21:22:32 +03:00
Alexey Milovidov
8e8160be28
Fix error
2021-05-08 21:11:47 +03:00
Alexey Milovidov
b96087025e
Fix error
2021-05-08 21:11:31 +03:00
Maksim Kita
0a2ea38cc9
Merge pull request #23916 from azat/dict-found_rate
...
Measure found rate for dictionaries
2021-05-08 19:23:52 +03:00
Alexey Milovidov
ad88819ee4
Fix a bunch of warnings from PVS-Studio
2021-05-08 19:13:10 +03:00
Alexey Milovidov
f805f4aa4f
Fix error found by PVS-Studio
2021-05-08 19:13:00 +03:00
Alexey Milovidov
3025f9e141
More fixes for PVS-Studio
2021-05-08 19:12:49 +03:00
Alexey Milovidov
e905883c75
More fixes for PVS-Studio
2021-05-08 19:12:31 +03:00
Alexey Milovidov
022667a190
Fix some PVS-Studio warnings
2021-05-08 19:12:19 +03:00
Alexey Milovidov
8d62c42eb9
A bunch of changes for PVS-Studio
2021-05-08 19:11:50 +03:00
Alexey Milovidov
125b2ab214
Suppressions for PVS-Studio
2021-05-08 19:10:23 +03:00
Alexey Milovidov
604daa9581
Fix a bunch of warnings from PVS-Studio
2021-05-08 19:09:17 +03:00
Alexey Milovidov
f59c55abb5
Fix error found by PVS-Studio
2021-05-08 18:35:38 +03:00
Alexey Milovidov
45dea86395
More fixes for PVS-Studio
2021-05-08 18:35:09 +03:00
Alexey Milovidov
2fbecf36ca
More fixes for PVS-Studio
2021-05-08 18:20:40 +03:00
Alexey Milovidov
3e8deef6fe
Fix some PVS-Studio warnings
2021-05-08 17:54:08 +03:00
Alexey Milovidov
9e1a5f41fc
A bunch of changes for PVS-Studio
2021-05-08 17:43:03 +03:00
Azat Khuzhin
e08389b2d2
Add interface for rate of found elements in the dictionaries
...
- IDictionary abstraction
- skeleton implementation into each dictionary
- system.dictionaries.found_rate
- documentation changes
2021-05-08 17:09:01 +03:00
Alexey Milovidov
9d9256d499
Suppressions for PVS-Studio
2021-05-08 17:01:25 +03:00
alexey-milovidov
680860aae7
Merge pull request #23912 from azat/CU
...
Cleanup IDatabase.h from extra headers
2021-05-08 16:42:56 +03:00
Alexey Milovidov
47a4c101b9
Merge branch 'master' into normalize-bigint
2021-05-08 16:41:15 +03:00
tavplubix
f8c7725ef5
Merge pull request #23742 from ClickHouse/update_metadata_on_replica_recovery
...
Update metadata on replica recovery
2021-05-08 16:34:03 +03:00
Alexander Tokmakov
11124f7015
fix
2021-05-07 20:09:39 +03:00
Alexander Tokmakov
24bcd9f7c6
better comments, fix pvs warnings
2021-05-07 16:39:21 +03:00
kssenii
912878ad7f
Review fixes
2021-05-07 12:05:34 +00:00
kssenii
8662078b35
Review fixes
2021-05-07 09:55:18 +00:00
Azat Khuzhin
e398635cb3
Make StorageS3.cpp self compilable (missing AbstractConfiguration.h)
2021-05-07 10:07:44 +03:00
kssenii
660442a16c
Review fixes
2021-05-06 12:48:48 +00:00
Azat Khuzhin
613789dfbb
Make StorageEmbeddedRocksDB.cpp self compilable (missing ReadBufferFromString.h)
2021-05-06 09:07:59 +03:00
Azat Khuzhin
c18ed16699
Make StorageRabbitMQ.cpp self compilable (missing AbstractConfiguration.h)
2021-05-06 09:07:58 +03:00
kssenii
8900e93bd8
New impl
2021-05-06 00:52:59 +00:00
Alexey Milovidov
35aba776e5
Merge branch 'master' into normalize-bigint
2021-05-05 15:01:23 +03:00
alexey-milovidov
bf8c28a17f
Merge pull request #23885 from azat/dist-broken-metrics
...
Add metrics for broken files during async distributed sends
2021-05-05 13:41:15 +03:00
Alexey Milovidov
54fb40c041
Fix OpenTelemetry
2021-05-05 01:42:14 +03:00
Azat Khuzhin
9c6e8e1462
Add BrokenDistributedFilesToInsert new metric
...
Number of files for asynchronous insertion into Distributed tables that
has been marked as broken. This metric will starts from 0 on start.
Number of files for every shard is summed.
2021-05-04 22:48:07 +03:00
Azat Khuzhin
74269882f7
Add broken_data_files/broken_data_compressed_bytes into distribution_queue
2021-05-04 22:48:07 +03:00
Alexey Milovidov
db3baabae7
Support for big integers and decimals in GenerateRandom
2021-05-04 20:41:46 +03:00
Alexey Milovidov
2b118f11b4
PVS-Studio
2021-05-04 15:14:55 +03:00
Alexey Milovidov
86169cc36e
Merge branch 'master' into normalize-bigint
2021-05-04 14:53:42 +03:00
Alexey Milovidov
37d6d54ac6
PVS-Studio
2021-05-04 14:29:50 +03:00
Alexey Milovidov
6ca37b9512
Untangle UUID
2021-05-04 01:59:38 +03:00
Alexey Milovidov
02ce9cc725
Untangle UUID
2021-05-04 01:46:51 +03:00
Raúl Marín
69a7bc001b
Fix typo
2021-05-03 12:50:44 +02:00
Raúl Marín
867ddab00a
StorageMergeTree::selectPartsToMutate: Reduce the amount of logs when busy
...
In a busy system without enough threads to process mutations at the moment,
you would get a log message per data part with pending mutations, which in
some extreme cases could be dozens per call.
2021-05-03 12:49:12 +02:00
Azat Khuzhin
5e33604c4d
Add file paths into logs on failed distributed async sends
2021-05-03 08:55:38 +03:00
Alexey Milovidov
f4100cf70c
Remove harmful std::regex
2021-05-02 22:04:26 +03:00
Alexey Milovidov
1fda866881
Merge branch 'master' into normalize-bigint
2021-05-02 21:24:45 +03:00
feng lv
39f68bf5ff
fix conflict
2021-05-02 16:33:45 +00:00
Vladimir Chebotarev
a6ab040b07
Added region parameter for S3 storage and disk.
2021-05-01 20:45:35 +03:00
Maksim Kita
14e879cef9
Merge pull request #23781 from ucasFL/fix-clear-column
...
Fix CLEAR COLUMN does not work after #21303
2021-04-30 17:28:59 +03:00
Nikolai Kochetov
4bb56849b3
Merge pull request #23763 from amosbird/rowfilterfix
...
Fix crash when prewhere and row policy filter are both in effect with empty result
2021-04-30 16:25:36 +03:00
Maksim Kita
9c808c4944
Merge pull request #23786 from kitaisreal/add-examples-folder-filter-to-ya-make-in
...
Add examples folder filter to ya.make.in
2021-04-30 15:27:54 +03:00
alexey-milovidov
dd31634ba4
Merge pull request #23746 from oxidecomputer/master
...
Adds support for building on Solaris-derived systems
2021-04-30 13:15:52 +03:00
Kseniia Sumarokova
fa5cea7a0a
Merge pull request #23761 from kssenii/hdfs-fixes
...
Fix hdfs read with Values format
2021-04-30 11:41:23 +03:00
Maksim Kita
318c4bb80d
Add examples folder filter to ya.make.in
2021-04-30 11:25:52 +03:00
feng lv
aed2f337e9
Fix CLEAR COLUMN does not work after #21303
2021-04-30 05:02:32 +00:00
tavplubix
f5ba30e4c0
fix comment
2021-04-29 20:36:55 +03:00
tavplubix
c7af5e623c
Update StorageReplicatedMergeTree.cpp
2021-04-29 17:01:51 +03:00
Amos Bird
70ec13f3d3
Fix empty block with row_filter
2021-04-29 19:40:14 +08:00
kssenii
c416fc3441
Fix hdfs read with Values format
2021-04-29 11:31:23 +00:00
Maksim Kita
1db6eb3666
Merge pull request #23744 from azat/dist-INSERT-preserve-error
...
Preserve errors for INSERT into Distributed
2021-04-29 10:26:34 +03:00
Benjamin Naecker
57fd59b73b
Adds support for building on Solaris-derived systems
...
This commit adds a number of changes to platform-detection and
compile-time directives required to support building ClickHouse on
Solaris-derived systems, most notably illumos.
2021-04-28 16:32:41 -07:00
Maksim Kita
1f1a443798
Merge pull request #23152 from amosbird/distributedwriteha
...
Skip unavaiable replicas when writing to distributed tables
2021-04-28 22:46:50 +03:00
Azat Khuzhin
73ab415c4c
Preserve errors for INSERT into Distributed
...
Before this patch (and after #22208 ) the INSERT may fail with "Cannot
schedule a task" because the pool in DistributedBlockOutputStream
already throws exception and simply fail in writeSuffix().
2021-04-28 22:33:29 +03:00
Alexander Tokmakov
bbf3bbc76d
update metadata on replica recovery
2021-04-28 20:49:27 +03:00
alexey-milovidov
f541e4017c
Merge pull request #23710 from ClickHouse/fix-bad-error-message
...
Fix illiterate error message #23700
2021-04-28 14:32:33 +03:00
Alexander Kuzmenkov
526f519c1f
Merge pull request #23446 from kssenii/globs-for-url
...
Support globs and failover for URL table function
2021-04-28 13:51:49 +03:00
Maksim Kita
b544037f03
Merge pull request #23436 from kitaisreal/storage-dictionary-updated
...
Refactored StorageDictionary
2021-04-28 13:11:48 +03:00
alexey-milovidov
fdc5a44202
Merge pull request #23644 from nikitamikhaylov/delete-unit-tests
...
Move non gtest unit tests to /examples folder
2021-04-28 07:28:18 +03:00
Alexey Milovidov
2cf9da72ff
Fix illiterate error message #23700
2021-04-28 06:49:46 +03:00
Maksim Kita
2e5f97c6ff
Fixed code review issues
2021-04-27 22:09:03 +03:00
Maksim Kita
8e508a37b7
Fixed dictionaries_lazy_load during StorageDictionary CREATE query
2021-04-27 12:15:35 +03:00
