thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 10:52:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
64,180 Commits 9,328 Branches 1,783 Tags 2.1 GiB
20a34e951a
Commit Graph

3948 Commits

Author SHA1 Message Date
Amos Bird
f7f949c1f9
Fix aggregation keys order 2021-05-11 18:12:27 +08:00
Amos Bird
e1e560765c
Fix missing columns 2021-05-11 18:12:27 +08:00
Amos Bird
35961c0c5d
Use ActionsDAG to rewrite projection queries 2021-05-11 18:12:27 +08:00
Nikolai Kochetov
bf95b684a7
Remove copy constructor from ProjectionDescription 2021-05-11 18:12:27 +08:00
Nikolai Kochetov
29d7038e30
Add ProjectionDescription::Type enum. Remove some checks. 2021-05-11 18:12:27 +08:00
Amos Bird
ebaf42a448
Reformat and fix some tests 2021-05-11 18:12:27 +08:00
Nikolai Kochetov
672cfedd13
Disable normal projection by the number of granules. 2021-05-11 18:12:26 +08:00
Nikolai Kochetov
95431168c2
Try to enable normal projections. 2021-05-11 18:12:26 +08:00
Amos Bird
483bd165e7
Check if pipeline is simple and add more comments 2021-05-11 18:12:26 +08:00
Nikolai Kochetov
427f444ba4
Fix prewhere. 2021-05-11 18:12:26 +08:00
Nikolai Kochetov
3d36fb57eb
Cleanup code. 2021-05-11 18:12:26 +08:00
Nikolai Kochetov
3296c9292f
Try to merge projectons faster. 2021-05-11 18:12:26 +08:00
Amos Bird
cd6414639e
add metadata_snapshot to getQueryProcessingStage 2021-05-11 18:12:26 +08:00
Amos Bird
264cff6415
Projections 
TODO (suggested by Nikolai)

1. Build query plan fro current query (inside storage::read) up to WithMergableState
2. Check, that plan is simple enough: Aggregating - Expression - Filter - ReadFromStorage (or simplier)
3. Check, that filter is the same as filter in projection, and also expression calculates the same aggregation keys as in projection
4. Return WithMergableState if projection applies

3 will be easier to do with ActionsDAG, cause it sees all functions, and dependencies are direct (but it is possible with ExpressionActions also)

Also need to figure out how prewhere works for projections, and
row_filter_policies.

wip
 2021-05-11 18:12:23 +08:00
Maksim Kita
fdaffe869d
Merge pull request #24007 from ClickHouse/fix-bad-code-2 
Fix bad code
 2021-05-11 12:03:53 +03:00
Kruglov Pavel
49e7ed6e60
Merge pull request #23673 from amosbird/partitionvalue 
Add _partition_value virtual column
 2021-05-11 11:23:00 +03:00
Alexey Milovidov
eb8a4ffe4b Fix bad code 2021-05-11 02:43:58 +03:00
Alexey Milovidov
b2ca5cd98b Merge branch 'master' into normalize-bigint 2021-05-11 02:05:40 +03:00
Alexander Tokmakov
eef899ae63 better code, add test 2021-05-10 21:03:37 +03:00
万康
308b964ca4 enable DateTime64 to be version column 2021-05-10 20:30:52 +08:00
Vladimir
d02d14fa00
Merge pull request #23846 from excitoon-favorites/s3regions 2021-05-10 12:14:08 +03:00
alexey-milovidov
1ea637d996
Merge pull request #23970 from ClickHouse/pvs-studio-fixes-6 
PVS-Studio fixes, part 6
 2021-05-09 21:45:19 +03:00
Alexey Milovidov
9753ddc8a0 Merge branch 'master' of github.com:yandex/ClickHouse into normalize-bigint 2021-05-09 18:54:29 +03:00
Alexey Milovidov
e23a0d8f7e Fix warnings by PVS-Studio 2021-05-08 23:57:08 +03:00
Alexey Milovidov
367f7fe6c9 Fix warnings by PVS-Studio 2021-05-08 23:57:08 +03:00
Alexey Milovidov
513c6891db Fix warnings by PVS-Studio 2021-05-08 23:54:49 +03:00
Alexey Milovidov
037d06bb2a Fix warnings by PVS-Studio 2021-05-08 23:50:12 +03:00
Alexey Milovidov
2d3647c5b0 Merge branch 'master' into normalize-bigint 2021-05-08 21:22:32 +03:00
Alexey Milovidov
8e8160be28 Fix error 2021-05-08 21:11:47 +03:00
Alexey Milovidov
b96087025e Fix error 2021-05-08 21:11:31 +03:00
Maksim Kita
0a2ea38cc9
Merge pull request #23916 from azat/dict-found_rate 
Measure found rate for dictionaries
 2021-05-08 19:23:52 +03:00
Alexey Milovidov
ad88819ee4 Fix a bunch of warnings from PVS-Studio 2021-05-08 19:13:10 +03:00
Alexey Milovidov
f805f4aa4f Fix error found by PVS-Studio 2021-05-08 19:13:00 +03:00
Alexey Milovidov
3025f9e141 More fixes for PVS-Studio 2021-05-08 19:12:49 +03:00
Alexey Milovidov
e905883c75 More fixes for PVS-Studio 2021-05-08 19:12:31 +03:00
Alexey Milovidov
022667a190 Fix some PVS-Studio warnings 2021-05-08 19:12:19 +03:00
Alexey Milovidov
8d62c42eb9 A bunch of changes for PVS-Studio 2021-05-08 19:11:50 +03:00
Alexey Milovidov
125b2ab214 Suppressions for PVS-Studio 2021-05-08 19:10:23 +03:00
Alexey Milovidov
604daa9581 Fix a bunch of warnings from PVS-Studio 2021-05-08 19:09:17 +03:00
Alexey Milovidov
f59c55abb5 Fix error found by PVS-Studio 2021-05-08 18:35:38 +03:00
Alexey Milovidov
45dea86395 More fixes for PVS-Studio 2021-05-08 18:35:09 +03:00
Alexey Milovidov
2fbecf36ca More fixes for PVS-Studio 2021-05-08 18:20:40 +03:00
Alexey Milovidov
3e8deef6fe Fix some PVS-Studio warnings 2021-05-08 17:54:08 +03:00
Alexey Milovidov
9e1a5f41fc A bunch of changes for PVS-Studio 2021-05-08 17:43:03 +03:00
Azat Khuzhin
e08389b2d2 Add interface for rate of found elements in the dictionaries 
- IDictionary abstraction
- skeleton implementation into each dictionary
- system.dictionaries.found_rate
- documentation changes
 2021-05-08 17:09:01 +03:00
Alexey Milovidov
9d9256d499 Suppressions for PVS-Studio 2021-05-08 17:01:25 +03:00
alexey-milovidov
680860aae7
Merge pull request #23912 from azat/CU 
Cleanup IDatabase.h from extra headers
 2021-05-08 16:42:56 +03:00
Alexey Milovidov
47a4c101b9 Merge branch 'master' into normalize-bigint 2021-05-08 16:41:15 +03:00
tavplubix
f8c7725ef5
Merge pull request #23742 from ClickHouse/update_metadata_on_replica_recovery 
Update metadata on replica recovery
 2021-05-08 16:34:03 +03:00
Alexander Tokmakov
11124f7015 fix 2021-05-07 20:09:39 +03:00
Alexander Tokmakov
24bcd9f7c6 better comments, fix pvs warnings 2021-05-07 16:39:21 +03:00
kssenii
912878ad7f Review fixes 2021-05-07 12:05:34 +00:00
kssenii
8662078b35 Review fixes 2021-05-07 09:55:18 +00:00
Azat Khuzhin
e398635cb3 Make StorageS3.cpp self compilable (missing AbstractConfiguration.h) 2021-05-07 10:07:44 +03:00
kssenii
660442a16c Review fixes 2021-05-06 12:48:48 +00:00
Azat Khuzhin
613789dfbb Make StorageEmbeddedRocksDB.cpp self compilable (missing ReadBufferFromString.h) 2021-05-06 09:07:59 +03:00
Azat Khuzhin
c18ed16699 Make StorageRabbitMQ.cpp self compilable (missing AbstractConfiguration.h) 2021-05-06 09:07:58 +03:00
kssenii
8900e93bd8 New impl 2021-05-06 00:52:59 +00:00
Azat Khuzhin
eefd67fce5 Disable optimize_distributed_group_by_sharding_key with window functions 2021-05-06 00:44:22 +03:00
Alexey Milovidov
35aba776e5 Merge branch 'master' into normalize-bigint 2021-05-05 15:01:23 +03:00
alexey-milovidov
bf8c28a17f
Merge pull request #23885 from azat/dist-broken-metrics 
Add metrics for broken files during async distributed sends
 2021-05-05 13:41:15 +03:00
kssenii
0b6fb7660b Fix checks 2021-05-05 09:15:56 +00:00
Alexey Milovidov
54fb40c041 Fix OpenTelemetry 2021-05-05 01:42:14 +03:00
kssenii
973ee4e9c4 Remove strange code, some fixes 2021-05-04 20:24:57 +00:00
kssenii
2112521304 Allow all RabbitMQ setup to be done only by user 2021-05-04 19:54:16 +00:00
Azat Khuzhin
9c6e8e1462 Add BrokenDistributedFilesToInsert new metric 
Number of files for asynchronous insertion into Distributed tables that
has been marked as broken. This metric will starts from 0 on start.
Number of files for every shard is summed.
 2021-05-04 22:48:07 +03:00
Azat Khuzhin
74269882f7 Add broken_data_files/broken_data_compressed_bytes into distribution_queue 2021-05-04 22:48:07 +03:00
kssenii
a3feaa48f4 Allow user to define specific queue settings 2021-05-04 18:57:49 +00:00
Alexey Milovidov
db3baabae7 Support for big integers and decimals in GenerateRandom 2021-05-04 20:41:46 +03:00
kssenii
13ba5287e3 Add cleanup in case of drop table 2021-05-04 16:26:47 +00:00
kssenii
4aad69dc90 Better commnets 2021-05-04 13:03:53 +00:00
Alexey Milovidov
2b118f11b4 PVS-Studio 2021-05-04 15:14:55 +03:00
Alexey Milovidov
86169cc36e Merge branch 'master' into normalize-bigint 2021-05-04 14:53:42 +03:00
Alexey Milovidov
37d6d54ac6 PVS-Studio 2021-05-04 14:29:50 +03:00
Alexey Milovidov
6ca37b9512 Untangle UUID 2021-05-04 01:59:38 +03:00
Alexey Milovidov
02ce9cc725 Untangle UUID 2021-05-04 01:46:51 +03:00
Raúl Marín
69a7bc001b Fix typo 2021-05-03 12:50:44 +02:00
Raúl Marín
867ddab00a StorageMergeTree::selectPartsToMutate: Reduce the amount of logs when busy 
In a busy system without enough threads to process mutations at the moment,
you would get a log message per data part with pending mutations, which in
some extreme cases could be dozens per call.
 2021-05-03 12:49:12 +02:00
Azat Khuzhin
5e33604c4d Add file paths into logs on failed distributed async sends 2021-05-03 08:55:38 +03:00
Alexey Milovidov
f4100cf70c Remove harmful std::regex 2021-05-02 22:04:26 +03:00
Alexey Milovidov
1fda866881 Merge branch 'master' into normalize-bigint 2021-05-02 21:24:45 +03:00
feng lv
39f68bf5ff fix conflict 2021-05-02 16:33:45 +00:00
Vladimir Chebotarev
a6ab040b07 Added region parameter for S3 storage and disk. 2021-05-01 20:45:35 +03:00
Maksim Kita
14e879cef9
Merge pull request #23781 from ucasFL/fix-clear-column 
Fix CLEAR COLUMN does not work after #21303
 2021-04-30 17:28:59 +03:00
Nikolai Kochetov
4bb56849b3
Merge pull request #23763 from amosbird/rowfilterfix 
Fix crash when prewhere and row policy filter are both in effect with empty result
 2021-04-30 16:25:36 +03:00
Maksim Kita
9c808c4944
Merge pull request #23786 from kitaisreal/add-examples-folder-filter-to-ya-make-in 
Add examples folder filter to ya.make.in
 2021-04-30 15:27:54 +03:00
alexey-milovidov
dd31634ba4
Merge pull request #23746 from oxidecomputer/master 
Adds support for building on Solaris-derived systems
 2021-04-30 13:15:52 +03:00
Kseniia Sumarokova
fa5cea7a0a
Merge pull request #23761 from kssenii/hdfs-fixes 
Fix hdfs read with Values format
 2021-04-30 11:41:23 +03:00
Maksim Kita
318c4bb80d Add examples folder filter to ya.make.in 2021-04-30 11:25:52 +03:00
feng lv
aed2f337e9 Fix CLEAR COLUMN does not work after #21303 2021-04-30 05:02:32 +00:00
tavplubix
f5ba30e4c0
fix comment 2021-04-29 20:36:55 +03:00
tavplubix
c7af5e623c
Update StorageReplicatedMergeTree.cpp 2021-04-29 17:01:51 +03:00
Amos Bird
70ec13f3d3
Fix empty block with row_filter 2021-04-29 19:40:14 +08:00
kssenii
c416fc3441 Fix hdfs read with Values format 2021-04-29 11:31:23 +00:00
Maksim Kita
1db6eb3666
Merge pull request #23744 from azat/dist-INSERT-preserve-error 
Preserve errors for INSERT into Distributed
 2021-04-29 10:26:34 +03:00
Benjamin Naecker
57fd59b73b Adds support for building on Solaris-derived systems 
This commit adds a number of changes to platform-detection and
compile-time directives required to support building ClickHouse on
Solaris-derived systems, most notably illumos.
 2021-04-28 16:32:41 -07:00
Maksim Kita
1f1a443798
Merge pull request #23152 from amosbird/distributedwriteha 
Skip unavaiable replicas when writing to distributed tables
 2021-04-28 22:46:50 +03:00
Azat Khuzhin
73ab415c4c Preserve errors for INSERT into Distributed 
Before this patch (and after #22208) the INSERT may fail with "Cannot
schedule a task" because the pool in DistributedBlockOutputStream
already throws exception and simply fail in writeSuffix().
 2021-04-28 22:33:29 +03:00
Alexander Tokmakov
bbf3bbc76d update metadata on replica recovery 2021-04-28 20:49:27 +03:00
alexey-milovidov
f541e4017c
Merge pull request #23710 from ClickHouse/fix-bad-error-message 
Fix illiterate error message #23700
 2021-04-28 14:32:33 +03:00
Alexander Kuzmenkov
526f519c1f
Merge pull request #23446 from kssenii/globs-for-url 
Support globs and failover for URL table function
 2021-04-28 13:51:49 +03:00
Maksim Kita
b544037f03
Merge pull request #23436 from kitaisreal/storage-dictionary-updated 
Refactored StorageDictionary
 2021-04-28 13:11:48 +03:00
alexey-milovidov
fdc5a44202
Merge pull request #23644 from nikitamikhaylov/delete-unit-tests 
Move non gtest unit tests to /examples folder
 2021-04-28 07:28:18 +03:00
Alexey Milovidov
2cf9da72ff Fix illiterate error message #23700 2021-04-28 06:49:46 +03:00
Maksim Kita
2e5f97c6ff Fixed code review issues 2021-04-27 22:09:03 +03:00
Nicolae Vartolomei
1fa5871ff7 Fix bad rebase and introduce part_moves_between_shards_enable setting 2021-04-27 14:20:13 +01:00
Nicolae Vartolomei
94f3db3c78 Use ZK check requests as barriers to prevent issuing tasks based on stale state information 2021-04-27 14:20:13 +01:00
alesapin
d12deed53d bump ci 2021-04-27 14:20:12 +01:00
alesapin
b930ca5d59 Followup fix 2021-04-27 14:20:12 +01:00
Nicolae Vartolomei
53d57ffb52 Part movement between shards 
Integrate query deduplication from #17348
 2021-04-27 14:20:12 +01:00
Maksim Kita
8e508a37b7 Fixed dictionaries_lazy_load during StorageDictionary CREATE query 2021-04-27 12:15:35 +03:00
kssenii
8c25d8d139 Show error description in exception 2021-04-27 08:49:24 +00:00
Amos Bird
8a3b5c1fab
Add _partition_value virtual column 2021-04-27 16:15:59 +08:00
Alexey Milovidov
a5a4e64ba7 Fix a few PVS-Studio warnings 2021-04-27 07:22:32 +03:00
alexey-milovidov
186b1128d0
Merge pull request #23310 from amosbird/fixbugindex 
Don't relax NOT conditions during partition pruning.
 2021-04-27 07:13:18 +03:00
kssenii
4cf8f5370a Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into globs-for-url 2021-04-27 01:07:51 +00:00
Nikita Mikhaylov
9f55424250 move to examples everywhere 2021-04-27 01:51:42 +03:00
Maksim Kita
71bccfdea1 Fixed integration test 2021-04-26 23:35:56 +03:00
Maksim Kita
c93666e0de Fixed code review issues 2021-04-26 13:52:41 +03:00
Maksim Kita
375b77dbcb Fixed integration tests 2021-04-26 13:52:41 +03:00
Maksim Kita
cad6b5a590 Fixed tests 2021-04-26 13:52:41 +03:00
Maksim Kita
e69581d72f Fixed tests 2021-04-26 13:52:41 +03:00
Maksim Kita
43680d8ced Fixed tests 2021-04-26 13:52:41 +03:00
Maksim Kita
fa39c30b4b Fixed tests 2021-04-26 13:52:41 +03:00
Maksim Kita
64f49a981b Fixed tests 2021-04-26 13:52:41 +03:00
Maksim Kita
2e3e90b25b Fixed tests 2021-04-26 13:52:41 +03:00
Maksim Kita
a9917a6540 Refactored StorageDictionary 2021-04-26 13:52:41 +03:00
tavplubix
81726de8fe
Merge pull request #23366 from amosbird/pid 
Fix invalid virtual column expr
 2021-04-26 11:10:24 +03:00
Vladimir
3c48b88929
Merge pull request #22208 from azat/s3-writer 2021-04-26 10:31:45 +03:00
Alexey Milovidov
9eb60086b0 Attempt to normalize big integers 2021-04-25 13:02:19 +03:00
Azat Khuzhin
a4036a8199 Correctly stop ParallelFormattingOutputFormat from StorageS3BlockOutputStream 2021-04-25 12:49:57 +03:00
Alexey Milovidov
649550c5ab Attempt to normalize big integers 2021-04-25 12:30:43 +03:00
Azat Khuzhin
7a8da84eee Respect lock_acquire_timeout_for_background_operations for OPTIMIZE 2021-04-25 10:12:15 +03:00
feng lv
3df8423b6e fix tests build 2021-04-25 06:51:21 +00:00
Alexey Milovidov
8f01af62d9 Merge branch 'master' into normalize-bigint 2021-04-25 06:57:44 +03:00
alexey-milovidov
1ffa63b20a
Merge pull request #23598 from ClickHouse/paths-with-whitespaces 
Somewhat better handling of paths in CMake (incomplete)
 2021-04-25 01:58:28 +03:00
Alexey Milovidov
2d3623b011 Remove build dependency on CMAKE_SYSTEM_NAME 2021-04-24 23:05:08 +03:00
Alexey Milovidov
3705dbfabc Avoid possible build dependency on locale and filesystem order 2021-04-24 23:01:07 +03:00
Alexey Milovidov
4ff812db7f Maybe better support for paths with whitespaces 2021-04-24 22:47:52 +03:00
alexey-milovidov
280c0666a5
Merge pull request #23559 from ClickHouse/reproducible-build 
Reproducible build
 2021-04-24 21:26:31 +03:00
alexey-milovidov
cae5260817
Update KeyCondition.cpp 2021-04-24 05:34:35 +03:00
alexey-milovidov
20e897bc5a
Merge pull request #23301 from ClickHouse/fix_race_in_deduplication 
Deduplication: check block version on removal
 2021-04-24 04:59:00 +03:00
alexey-milovidov
2cfc2c8825
Merge pull request #23397 from CurtizJ/better-move-to-prewhere 
Allow to move more conditions to PREWHERE
 2021-04-24 04:39:33 +03:00
mergify[bot]
a5da512e39
Merge branch 'master' into pid 2021-04-24 00:37:39 +00:00
alexey-milovidov
c66e715f49
Merge pull request #23433 from ClickHouse/tavplubix-patch-1 
Fix `Cannot unlink file` in `dropIfEmpty()` with multidisk confuguration
 2021-04-24 03:34:06 +03:00
alexey-milovidov
cd1f14ba61
Update MergeTreeData.cpp 2021-04-24 03:33:56 +03:00
Alexey Milovidov
4648782a05 Remove build date from system.build_options 2021-04-24 02:10:32 +03:00
feng lv
3916870100 fix 
fix

fix
 2021-04-23 14:01:19 +00:00
feng lv
0f5c05854c fix conflict 2021-04-23 12:22:51 +00:00
feng lv
4ffe199d39 Implement table comments 2021-04-23 12:18:23 +00:00