fuqi
487d6bdcd9
add test case
...
fix order key check
2021-03-24 16:36:45 +08:00
alexey-milovidov
612d4fb073
Update IMergeTreeDataPart.cpp
2021-03-24 02:03:14 +03:00
alesapin
36898bdc4a
Merge pull request #21142 from Jokser/unfreeze-partitions
...
Add ALTER TABLE UNFREEZE command.
2021-03-23 14:05:38 +03:00
Pavel Kovalenko
a92cf30b67
Code review fixes.
2021-03-23 13:33:07 +03:00
Alexey Milovidov
8d0210b510
Expose DateTime64 minmax part index in system.parts and system.parts_columns #18244
2021-03-23 01:16:41 +03:00
Alexey Milovidov
ce67afd9e3
Fix bad code #18244
2021-03-23 00:49:18 +03:00
Mike Kot
c55a73b752
Added the solution to handle the corruption case
...
When the part data (e.g. data.bin) is corrupted, but the checksums.txt
is present -- explicitly deleting the checksums.txt.
Removed the extra logging, changes some exceptions message.
2021-03-22 17:23:43 +03:00
Mike Kot
5789507e8b
Investigating, why the checksums may match when they shouldn't.
2021-03-22 17:23:43 +03:00
Nikolai Kochetov
c3c393a7aa
Merge branch 'master' into refactor-actions-dag
2021-03-18 14:33:07 +03:00
fuqi
cfa92f0045
fix code style
2021-03-18 00:25:43 +08:00
fuqi
b8edc12812
fix code style
2021-03-17 23:56:55 +08:00
Mike Kot
e309dd275a
Trying to investigate the errors after attaching the part.
...
+ A new entry is added to the part_log.
2021-03-17 18:49:04 +03:00
fuqi
e116e8d5e8
fix bug
2021-03-17 22:19:10 +08:00
Mike Kot
831b90f272
Replaced the iteration algorithm in part finder
...
Now, instead of iterating through the directories, we iterate though
directories of on of the table disks (which doesn't give us a
substantial boost but is a bit neater to read).
- Updated the system.replication_queue command types.
- Fixed the part ptr being empty (added the checksum loading and
initialization).
- Removed extra logging.
2021-03-17 16:55:26 +03:00
Mike Kot
2ccdb7ef5c
Multiple small code and test updates
...
- Updated the docs to make everything clear.
- Multiple small logger fixes.
- Changed the attach_part command -- now it's after check for the
covering parts -- motivation is to do less work with the checksums
fetching.
- Better logging in the integration test.
2021-03-17 16:52:35 +03:00
fuqi
9ab713c2e1
optimize select final with prewhere primary key
2021-03-17 18:57:58 +08:00
fuqi
5e20ea2c33
optimize select final with prewhere primary key
2021-03-17 18:49:24 +08:00
Pavel Kovalenko
5627ae9067
Make MergeTreeData::getPartitionMatcher const
2021-03-17 12:12:55 +03:00
alexey-milovidov
d02726bcac
Merge pull request #9404 from Enmk/DateTime64_extended_range
...
Date time64 extended range
2021-03-17 11:06:03 +03:00
Anton Popov
6247cd5503
Merge remote-tracking branch 'upstream/master' into HEAD
2021-03-16 16:50:15 +03:00
Pavel Kovalenko
a43c85b105
Add test for freeze/unfreeze partition with old MergeTree syntax
2021-03-16 16:36:05 +03:00
Alexey Milovidov
ea049b2cc0
Merge branch 'master' into DateTime64_extended_range
2021-03-16 14:08:02 +03:00
Pavel Kovalenko
39f3b17041
Merge remote-tracking branch 'origin/master' into unfreeze-partitions
2021-03-16 10:56:24 +03:00
alesapin
8fd3fbc5b3
Merge pull request #20387 from Avogar/optimize-on-insert
...
Don't create empty parts on INSERT
2021-03-16 10:20:34 +03:00
Amos Bird
f00e108410
Fix scalar subquery index analysis
2021-03-16 14:07:30 +08:00
Anton Popov
173d2ea1f4
Merge remote-tracking branch 'upstream/master' into HEAD
2021-03-16 02:50:14 +03:00
alexey-milovidov
378e10f929
Merge pull request #21351 from fuwhu/refine-load-data-parts
...
Refine MergeTreeData::loadDataParts to not parse format version file and detached directory
2021-03-15 22:38:30 +03:00
Alexey Milovidov
671395e8c8
Most likely improve performance
2021-03-15 22:23:27 +03:00
Mike Kot
406d037ebb
Merge remote-tracking branch 'upstream/master' into feature/attach-partition-local
2021-03-15 18:41:47 +03:00
alesapin
d502a217e5
Merge branch 'master' into Avogar-optimize-on-insert
2021-03-15 11:38:25 +03:00
alesapin
06eb2d8dfd
Merge branch 'master' into s3_zero_copy_replication
2021-03-13 22:32:54 +03:00
Anton Popov
81ac6382a3
slightly better performance
2021-03-13 21:05:18 +03:00
Anton Popov
1b07d28043
fix unwanted changes
2021-03-13 02:59:42 +03:00
Anton Popov
ed42437219
Merge remote-tracking branch 'upstream/master' into HEAD
2021-03-13 01:41:26 +03:00
alexey-milovidov
f6bedf55e9
Merge pull request #21642 from azat/fsync_part_directory-horizontal-merge-fix
...
Fix fsync_part_directory for horizontal merge
2021-03-12 21:22:55 +03:00
Anton Ivashkin
ace628d565
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into s3_zero_copy_replication
2021-03-12 12:58:32 +03:00
alesapin
cca05da7fa
Merge pull request #16481 from amosbird/jbodbalance
...
Auto-balance big parts in JBOD array
2021-03-12 11:07:47 +03:00
Azat Khuzhin
69b2b2a159
Fix fsync_part_directory for horizontal merge
2021-03-11 21:41:27 +03:00
Nikolai Kochetov
4967227fe9
Remove commented code.
2021-03-11 20:11:57 +03:00
alesapin
5704405f8c
Disable new setting by default
2021-03-11 12:29:27 +03:00
Anton Ivashkin
cef0dda933
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into s3_zero_copy_replication
2021-03-10 14:10:29 +03:00
Anton Ivashkin
deb0b83a12
Fix refreshState logic
2021-03-10 14:08:49 +03:00
Nikolai Kochetov
4f133ae036
Merge branch 'master' into refactor-actions-dag
2021-03-10 12:40:55 +03:00
Nikolai Kochetov
5ef9d87790
Fix some other tests.
2021-03-10 11:41:24 +03:00
fuwhu
69f516cd47
Refine MergeTreeData::loadDataParts to not parse format version file and detached directory
2021-03-10 10:10:25 +08:00
Amos Bird
e52b2fb8ae
update
2021-03-10 02:56:46 +08:00
Anton Ivashkin
265d293934
Use 'merge on single replica' option instead of zookeeper lock
2021-03-09 20:39:55 +03:00
Anton Popov
df6663dcb6
refactoring of serializations
2021-03-09 20:02:26 +03:00
Anton Popov
bc417cf54a
refactoring of serializations
2021-03-09 17:46:52 +03:00
Amos Bird
2ec20c5d23
update and add tests
2021-03-08 17:38:07 +08:00
Amos Bird
2b11a2a855
Fix build
2021-03-08 11:10:35 +08:00
Amos Bird
e6522e1ebe
JBOD data balancer
2021-03-08 11:10:35 +08:00
Mikhail f. Shiryaev
efc04b29a6
Add check for growing age AND precision in rollup config
2021-03-06 18:35:53 +01:00
alesapin
5b3161e0b5
Get rid of const_cast
2021-03-05 20:24:06 +03:00
Anton Ivashkin
d08b481660
Fixes by review responces
2021-03-05 19:20:38 +03:00
Nikolai Kochetov
a669f7d641
Merge branch 'master' into refactor-actions-dag
2021-03-05 18:21:14 +03:00
Nikolai Kochetov
9a39459888
Refactor ActionsDAG
2021-03-04 20:38:12 +03:00
Amos Bird
909cb3c243
Fix again....
2021-03-04 22:27:07 +08:00
Amos Bird
fac832227c
Fix again
2021-03-04 19:43:03 +08:00
Amos Bird
9205fad8c7
Better
2021-03-04 19:43:03 +08:00
Amos Bird
4c9b0f666c
Fix error
2021-03-04 19:43:03 +08:00
Amos Bird
93b661ad5a
partition id pruning
2021-03-04 19:43:03 +08:00
Anton Ivashkin
e69124a0a6
Merge master
2021-03-04 13:26:40 +03:00
Nikolai Kochetov
15921fbfcb
Remove index by name from ActionsDAG
2021-03-03 23:01:07 +03:00
Mike Kot
6ea574525c
Small fixes regarding the review
2021-03-03 16:51:41 +03:00
Pavel Kovalenko
c4523ec74c
Add 'const auto' for variable.
2021-03-03 10:49:23 +03:00
Pavel Kovalenko
182ba7b5b4
Fixed argument name.
2021-03-03 10:35:31 +03:00
alesapin
6b1005aea5
Merge pull request #21334 from ClickHouse/fix_alter_partition_key
...
Fix alter modify query for partition key and other metadata fields
2021-03-03 10:15:11 +03:00
Anton Popov
a4c00ab5dc
Merge pull request #21303 from ucasFL/forbid
...
Forbid to drop a column if it's referenced by materialized view
2021-03-03 02:55:06 +03:00
Pavel Kovalenko
483e63ca45
ALTER TABLE UNFREEZE rework.
2021-03-03 00:10:09 +03:00
Pavel Kovalenko
a3c0c35dff
ALTER TABLE UNFREEZE rework.
2021-03-02 23:28:42 +03:00
Nikolai Kochetov
4775ea305e
Remove index by name from ActionsDAG
2021-03-02 20:51:54 +03:00
alesapin
c29d7c7f49
Shutup clang tidy
2021-03-02 19:13:36 +03:00
alesapin
4e33587043
Comments
2021-03-02 13:57:09 +03:00
Nikolai Kochetov
ffbea93b10
Merge pull request #19576 from traceon/row-policy-with-prewhere
...
Allow row policies with PREWHERE
2021-03-02 13:51:37 +03:00
alesapin
06678d650d
Merge branch 'master' into fix_alter_partition_key
2021-03-02 13:43:41 +03:00
alesapin
9ebf1b4fad
Get rid of separate minmax index fields
2021-03-02 13:33:54 +03:00
feng lv
a26c9e64a9
fix
...
fix
2021-03-02 03:20:03 +00:00
Mike Kot
4e1ac185b5
Merge remote-tracking branch 'upstream/master' into feature/attach-partition-local
2021-03-01 20:58:07 +03:00
Anton Ivashkin
98065ec56e
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into s3_zero_copy_replication
2021-03-01 13:33:40 +03:00
Anton Ivashkin
3c11d44494
Add description for getUniqueId method, fix typos
2021-03-01 13:31:36 +03:00
alesapin
9c8afbeb53
Fix alter modify query for partition key and other metadata fields
2021-03-01 12:59:19 +03:00
Nikolai Kochetov
976dbe8077
Merge pull request #20341 from ClickHouse/filter-push-down
...
Filter push down
2021-03-01 12:35:06 +03:00
alexey-milovidov
174f49a9f1
Merge pull request #21308 from fuwhu/refine-wal-restore-code
...
remove unused code in MergeTreeWriteAheadLog::restore
2021-03-01 01:44:23 +03:00
Denis Glazachev
eb80c189e4
Merge branch 'master' into row-policy-with-prewhere
2021-02-28 20:14:31 +04:00
fuwhu
2df33be7c2
Remove unused code in MergeTreeWriteAheadLog::restore
2021-02-28 23:17:31 +08:00
Azat Khuzhin
8b58dba094
Drop unused HexWriteBuffer
2021-02-28 10:52:09 +03:00
feng lv
51021c1164
forbid to drop a column if it's referenced by materialized view
2021-02-28 05:24:39 +00:00
Nikolai Kochetov
d91b8a3acb
Merge branch 'master' into filter-push-down
2021-02-26 19:33:27 +03:00
Nikolai Kochetov
d328bfa41f
Review fixes. Add setting max_optimizations_to_apply.
2021-02-26 19:29:56 +03:00
tavplubix
09f62447af
Merge pull request #21060 from mxzlxy/fix-bug-for-replicated-engine-parameter
...
fix default_replica_path and default_replica_name values are useless …
2021-02-26 17:18:56 +03:00
Anton Ivashkin
5b267b7eec
Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into s3_zero_copy_replication
2021-02-26 13:15:18 +03:00
Anton Ivashkin
c891cf4557
Fixes by review response
2021-02-26 12:48:57 +03:00
Pavel Kovalenko
ed44845f2c
Fixed backup_path for unfreeze
2021-02-25 22:21:22 +03:00
Mike Kot
b2c898f58c
Adding the part when found
2021-02-25 21:25:55 +03:00
alesapin
653d962cdd
Fix style and add low-level check
2021-02-25 17:43:58 +03:00
alesapin
5c6c318737
Restrict mutations for engines which doesn't support them
2021-02-25 13:07:48 +03:00
Pavel Kovalenko
0df783e0ef
Fix code style
2021-02-24 18:38:40 +03:00
Pavel Kovalenko
078dfce038
Add ALTER TABLE UNFREEZE command.
2021-02-24 17:28:30 +03:00
alesapin
427e5f295c
Merge pull request #21007 from CurtizJ/alter-bad-conversions
...
Do not start mutation for alters with wrong type conversion
2021-02-24 10:08:15 +03:00
tavplubix
4661073750
Update src/Storages/MergeTree/registerStorageMergeTree.cpp
2021-02-23 00:09:20 +03:00
jennyma
6433ab3da6
fix default_replica_path and default_replica_name values are useless on Replicated(*)MergeTree engine when the engine needs specify other parameters
2021-02-22 18:34:01 +08:00
Denis Glazachev
c1158b50f4
Merge branch 'master' into row-policy-with-prewhere
...
* master: (155 commits)
Update version_date.tsv after release 20.8.13.15
Update version_date.tsv after release 20.12.7.3
Update version_date.tsv after release 21.1.5.4
Update version_date.tsv after release 21.2.4.6
fix
Add test to skip list
Fix WriteBufferFromHTTPServerResponse usage in other places (add missing finalize())
Fix WriteBufferFromHTTPServerResponse usage in odbc-bridge
Update config.xml
Suppress signed overflow in AggregateFunctionGroupArrayMoving 2
Update BaseDaemon.cpp
review suggestions
Fix bash syntax in 01731_async_task_queue_wait
Do not use view() in 01731_async_task_queue_wait to fix ANTLR parser
Increase buffer for uncaught exception / std::terminate
Even more better
Fix uncaught exception when HTTP client goes away
test for decimal ( p , s) in dictionaries
Just little better
Fixed style check
...
2021-02-21 13:25:47 +04:00
Denis Glazachev
dfde9de0e9
Remove commented-out code
2021-02-20 23:30:21 +04:00
tavplubix
a697b578bc
Merge pull request #16193 from ClickHouse/database_replicated
...
DatabaseReplicated: continuation of #10485
2021-02-20 19:39:34 +03:00
Anton Popov
48e188681c
do not start mutation for alters with wrong type conversion
2021-02-20 19:11:08 +03:00
Nikolai Kochetov
673e24d7ef
Refactor
2021-02-20 14:00:16 +03:00
Denis Glazachev
85ea5cafc4
Always apply row_level_filter immediately
2021-02-20 05:23:48 +04:00
Denis Glazachev
8cc72b8fe0
Merge branch 'master' into row-policy-with-prewhere
...
* master: (160 commits)
Make Poco HTTP Server zero-copy again (#19516 )
Fixed documentation
ccache 4.2+ does not requires any quirks for SOURCE_DATE_EPOCH
Add a function `htmlOrXmlCoarseParse` to extract content from html or xml format string. (#19600 )
Reinterpret function added Decimal, DateTim64 support
Add test
Update InterpreterSelectQuery.cpp
Improved serialization for data types combined of Arrays and Tuples. Improved matching enum data types to protobuf enum type. Fixed serialization of the Map data type. Omitted values are now set by default.
Log stdout and stderr when failed to start docker in integration tests.
Added comment
Don't backport base commit of branch in the same branch (#20628 )
Fix fasttest retry for failed tests
Dictionary create source with functions crash fix
Added error reinterpretation tests
Update run.sh
Updated documentation
fix subquery with limit
Rename untyped function reinterpretAs into reinterpret
ignore data store files
Support vhost
...
2021-02-19 19:49:37 +04:00
Denis Glazachev
5e5b3b80ce
Remove debug printouts
2021-02-19 19:46:52 +04:00
Mike Kot
f1ef382cf9
Added part_checksum to Replicated...Entry serialization.
2021-02-19 16:04:12 +03:00
Ivan
414f470c79
Make Poco HTTP Server zero-copy again ( #19516 )
...
* Refactoring: part 1
* Refactoring: part 2
* Handle request using ReadBuffer interface
* Struggles with ReadBuffer's
* Fix URI parsing
* Implement parsing of multipart/form-data
* Check HTTP_LENGTH_REQUIRED before eof() or will hang
* Fix HTTPChunkedReadBuffer
* Fix build and style
* Fix test
* Resist double-eof
* Fix arcadian build
2021-02-19 15:51:26 +03:00
Mike Kot
ca83775711
Multiple small hotfixes
...
Small fixes
Some fix for old bug
Another old code fix
2021-02-16 16:39:18 +03:00
Mike Kot
a1fafc47c6
Correct checksums pre-calculation for ATTACH_PART in ...OutputStream
2021-02-16 15:40:00 +03:00
Alexander Tokmakov
bf6f64a3fb
Merge branch 'master' into database_replicated
2021-02-16 01:28:19 +03:00
Nikolai Kochetov
85277d6a41
Comment debug output.
2021-02-15 22:54:47 +03:00
Nikolai Kochetov
ae73600fb0
Refactor row level security actions.
2021-02-15 22:48:06 +03:00
Mike Kot
0a02fe913a
Add some code for the checksum pre-calculation in ...BlockOutputStream
...
Added the comment explaining the double-get for the zookeeper header for
the part.
2021-02-15 20:31:58 +03:00
Mike Kot
73f4740be5
Trying to re-write the solution by adding new command type
...
ATTACH_PART into the replicated log.
The LogEntry now also has the pre-calculated part checksum for this
entry type, which is later used while searching in the detached/ folder
2021-02-15 18:06:48 +03:00
Mike Kot
cd32803709
Merge remote-tracking branch 'upstream/master' into feature/attach-partition-local
2021-02-15 15:42:17 +03:00
Mike Kot
feff4c6a22
Started adding the new "ATTACH_PART" command into the replicated log
...
The original ticket idea was to search for the possibly available data
into the /detached folders for the GET_PART command, but
@tavplubix pointed out this would be quite expensive for an every
fetch.
So a new command is going to be introduced, ATTACH_PART, which will
cover ALTER TABLE ATTACH PART and only for which the search will start.
2021-02-15 01:59:13 +03:00
Denis Glazachev
45e90961f7
Store and process alias_actions in FilterInfo
2021-02-14 21:16:40 +04:00
Denis Glazachev
99a04b6c64
Merge branch 'master' into row-policy-with-prewhere
...
* master: (759 commits)
Suppress UBSan report in Decimal comparison
Suppress UBSan report in Decimal comparison
Fix UBSan report in arrayDifference
Update README.md
Non significant change in AggregationCommon
Print stack trace on SIGTRAP
Fix dependent test
Fix tests for better parallel run
Add test for already working code
Revert "Fix access control manager destruction order"
Update index.md
Update index.md
Update index.md
Bit more complicated example for isIPv4String - ru
Bit more complicated example for isIPv4String
cleanup
Replace database with ordinary
Added comments
Split tests to make them stable
Fixes
...
# Conflicts:
# src/Storages/MergeTree/MergeTreeRangeReader.cpp
2021-02-14 02:24:00 +04:00
Denis Glazachev
79592b73f8
Store filter info in prewhere info instead of multiple prewheres
...
Some cleanups
2021-02-14 02:07:13 +04:00
Azat Khuzhin
68f23b7087
Improve logging during MergeTree reading
...
- Remove "Not using primary index on part {}" message (too noisy)
- Add number of total marks before filtering by primary key into the
common message
- Make "Index {} has dropped {} / {} granules." not per-part, but
per-query
2021-02-13 18:08:55 +03:00
Pavel Kruglov
f0163c2acf
Don't create empty parts on INSERT
2021-02-12 17:02:04 +03:00
alesapin
a2bb586170
Merge pull request #20335 from ClickHouse/fix_scheduling_of_non_successful_background_tasks
...
Fix backoff for failed background tasks in replicated merge tree
2021-02-12 10:06:04 +03:00
alexey-milovidov
7b55eb8c58
Merge pull request #20178 from amosbird/delayinactive
...
Delay or throw insertion when too many inactive parts
2021-02-11 20:52:04 +03:00
alesapin
3253638969
Fix backoff for failed background tasks in replicated merge tree
2021-02-11 14:46:18 +03:00
alesapin
56453256b7
Merge pull request #20300 from ClickHouse/restrict_to_drop_version_column
...
Don't allow to drop or rename version column
2021-02-11 11:04:00 +03:00
alesapin
121c5c9605
Merge pull request #20279 from ClickHouse/relax_check_in_data_writer
...
Relax check in data writer for non adaptive tables
2021-02-11 10:57:07 +03:00
alesapin
c2bb2c2902
Merge pull request #20097 from ClickHouse/remove-adding-missed-step
...
Build actions dag to evaluate missing defaults.
2021-02-11 10:51:21 +03:00
alexey-milovidov
e33477a16e
Merge pull request #20282 from ClickHouse/dont_allow_ttl_for_old_syntax
...
Don't allow TTL in old syntax
2021-02-11 00:46:30 +03:00
Anton Popov
365e2fb746
Merge pull request #20231 from fuwhu/refine-loadDataParts-code
...
refine code in MergeTreeData::loadDataParts to avoid parsing WAL file as data part
2021-02-10 22:42:31 +03:00
alesapin
891fce3275
Don't allow to drop or rename version column
2021-02-10 22:23:48 +03:00
alesapin
a63d73db58
Don't allow TTL in old syntax
2021-02-10 15:09:12 +03:00
alesapin
420f2958e2
Merge pull request #19991 from ClickHouse/constant-propagation-fix
...
Do not use inputs which values are known constants in ActionsDAG.
2021-02-10 14:13:52 +03:00
alesapin
21faea1f68
Relax check in data writer for non adaptive tables
2021-02-10 13:43:31 +03:00
Amos Bird
afb5846a24
refactor
2021-02-10 11:22:24 +08:00
HuFuwang
4ac8860cea
refine code in MergeTreeData::loadDataParts to avoid parsing WAL file as data part.
2021-02-10 10:13:43 +08:00
tavplubix
86530e603c
Merge pull request #20027 from bharatnc/ncb/event_time_micro_part_log
...
add event_time_microseconds field to system.part_log table
2021-02-09 14:46:40 +03:00
Nikolai Kochetov
229acbad5d
Merge pull request #20077 from ClickHouse/remove-adding-const-column-transform
...
Remove AddingConstColumn step and transform.
2021-02-09 11:28:44 +03:00
Alexander Tokmakov
5e0cbfe972
Merge branch 'master' into database_replicated
2021-02-08 23:01:12 +03:00
Nicolae Vartolomei
135f82ce94
Correctly document that settings apply only for fetch part requests
2021-02-08 19:44:03 +00:00
Nicolae Vartolomei
b153e8c190
Add support for custom fetchPart timeouts
2021-02-08 19:44:02 +00:00
Alexander Tokmakov
78c1d69b8c
better code
2021-02-08 22:36:17 +03:00
Nicolae Vartolomei
c336cac306
Remove dead code
2021-02-08 17:36:58 +00:00
Nicolae Vartolomei
a8f37e7704
Remove dead code
2021-02-08 17:19:35 +00:00
bharatnc
60c8dfa22c
PartLog - populate event_time_microseconds for MergeParts
2021-02-08 07:25:44 -08:00
Nikolai Kochetov
d7fa440356
Merge branch 'master' into remove-adding-const-column-transform
2021-02-08 15:49:23 +03:00
Anton Ivashkin
4d44d75bc7
Fix build after merge one more time
2021-02-08 14:45:10 +03:00
Amos Bird
109a392e0c
Fix ubsan
2021-02-08 19:41:16 +08:00
Anton Ivashkin
1b6ce056a5
Merge master
2021-02-08 12:34:20 +03:00
Amos Bird
ab5c7b75a4
Delay or throw insertion when too many inactive parts
2021-02-07 19:03:55 +08:00
Maxim Akhmedov
0cd3601928
Allow using MergeTreeWhereOptimizer not only with MergeTree-based storages
2021-02-06 18:08:42 +03:00
alesapin
011109c82a
Merge pull request #17348 from xjewer/alex/CLICKHOUSE-606_deduplication_UUID
...
CLICKHOUSE-606: query deduplication based on parts' UUID
2021-02-05 22:47:34 +03:00
Anton Ivashkin
e64c63c611
Merge master
2021-02-05 20:10:06 +03:00
Anton Ivashkin
df6c882aab
Fix build after merge
2021-02-05 18:52:40 +03:00
Nikolai Kochetov
85c175883e
Rename functions.
2021-02-05 18:11:26 +03:00
alesapin
7cbc135e72
More isolated code
2021-02-05 12:54:34 +03:00
Nikolai Kochetov
d9aa1096cf
Build actions dag to evaluate missing defaults.
2021-02-04 23:36:50 +03:00
vdimir
30f98795bc
Merge remote-tracking branch 'upstream/master' into disk-s3-backup-restore-metadata
2021-02-04 18:38:28 +03:00
Nikolai Kochetov
e3d6ffd9dc
Remove AddingConstColumn step and transform.
2021-02-04 17:25:11 +03:00
Alexey Milovidov
474f151707
More docs for MergeTreeSettings
2021-02-04 08:26:22 +03:00
Alexey Milovidov
dc2afc4795
Fix double whitespace #18383
2021-02-03 21:12:14 +03:00
Alexander Tokmakov
d010f97db0
Merge branch 'master' into database_replicated
2021-02-03 20:13:25 +03:00
Pavel Kovalenko
b7151aa754
Merge remote-tracking branch 'origin/master' into disk-s3-backup-restore-metadata
...
# Conflicts:
# src/Disks/DiskDecorator.h
# src/Disks/IDisk.h
# src/Disks/S3/DiskS3.cpp
2021-02-03 14:22:18 +03:00
Alexander Tokmakov
6456ccf0da
better test
2021-02-02 22:39:04 +03:00
Alexey Milovidov
12485eee6b
Fix some of the issues found by Coverity
2021-02-02 22:07:23 +03:00
Aleksei Semiglazov
921518db0a
CLICKHOUSE-606: query deduplication based on parts' UUID
...
* add the query data deduplication excluding duplicated parts in MergeTree family engines.
query deduplication is based on parts' UUID which should be enabled first with merge_tree setting
assign_part_uuids=1
allow_experimental_query_deduplication setting is to enable part deduplication, default ot false.
data part UUID is a mechanism of giving a data part a unique identifier.
Having UUID and deduplication mechanism provides a potential of moving parts
between shards preserving data consistency on a read path:
duplicated UUIDs will cause root executor to retry query against on of the replica explicitly
asking to exclude encountered duplicated fingerprints during a distributed query execution.
NOTE: this implementation don't provide any knobs to lock part and hence its UUID. Any mutations/merge will
update part's UUID.
* add _part_uuid virtual column, allowing to use UUIDs in predicates.
Signed-off-by: Aleksei Semiglazov <asemiglazov@cloudflare.com>
address comments
2021-02-02 16:53:39 +00:00
Nikolai Kochetov
b2d7790794
Fix test.
2021-02-02 17:30:54 +03:00
Maksim Kita
d0151de4bb
Merge pull request #19608 from kreuzerkrieg/Add_IStoragePolicy_interface
...
Add IStoragePolicy interface
2021-02-02 11:03:20 +03:00
Pavel Kruglov
a3f1b825cc
Fix build
2021-02-01 21:17:12 +03:00
alexey-milovidov
6b2f0435c8
Merge pull request #19375 from Avogar/select-final
...
Improve do_not_merge_across_partitions_select_final optimization.
2021-02-01 20:31:08 +03:00
alesapin
2aa8a6304b
Merge pull request #15450 from CurtizJ/fix-ttl-group-by
...
Fix some cases of TTL expressions
2021-02-01 16:48:07 +03:00
Nikolai Kochetov
19e4a33f9d
Merge pull request #19544 from amosbird/limitconcurrency
...
Per MergeTree table query limit
2021-02-01 16:09:12 +03:00
alesapin
d9598c47b4
Merge pull request #19702 from ClickHouse/fix_rare_bug_after_part_corruption
...
Fix incorrect virtual_parts after part corruption
2021-02-01 10:09:48 +03:00
alexey-milovidov
d19feb724b
Merge pull request #19799 from CurtizJ/fix-uint8-filtering
...
Fix filtering by Uint8 greater than 127
2021-01-29 21:34:42 +03:00
Denis Glazachev
31706e69bc
Merge branch 'master' into row-policy-with-prewhere
...
* master: (75 commits)
Temporary retrun dicttoxml to integration tests runner
Fix option
Fix dependency on ODBC for Yandex internal build
Remove useless headers
Add test-connect tool
Revert "Remove old non-automated test"
Fix trivial bug in arrayEnumerateUniq
Fix UBSan report in "round"
Translate comment in test
Added test to ANTLR skip_list.json
Update Client.cpp
Fixed race in tests
Fixed test_kafka_flush_by_time test.
LowCardinality UUID fix
Update mergetree.md
Clickhouse client query param CTE added test
correct style zh doc typo
Fixed data race in function DictGetNoType
add empty line after error messages in client
Fixed style issues
...
# Conflicts:
# src/Storages/MergeTree/MergeTreeReadPool.h
2021-01-29 22:26:11 +04:00
Pavel Kruglov
78371e15dc
Update test, reduce num_threads_for_lonely_parts if data is small
2021-01-29 21:00:08 +03:00
alesapin
f4236fd765
Fix style
2021-01-29 20:12:53 +03:00
Nikolai Kochetov
47a7273fe3
Fix build
2021-01-29 19:21:52 +03:00
Nikolai Kochetov
afdc9635cb
Update MergeTreeRangeReader
2021-01-29 19:12:53 +03:00
alesapin
c373d92a80
Less strict check
2021-01-29 18:50:08 +03:00
Nikolai Kochetov
4929fe2063
Update MergeTreeRangeReader
2021-01-29 18:13:09 +03:00
Pavel Kruglov
71f4acd48b
Use one pool for lonely parts, update tests
2021-01-29 17:30:14 +03:00
Pavel Kruglov
a437ee4e31
Merge branch 'master' of github.com:ClickHouse/ClickHouse into select-final
2021-01-29 14:25:47 +03:00
Kruglov Pavel
caef103837
Merge branch 'master' into Add_IStoragePolicy_interface
2021-01-29 14:00:12 +03:00
Anton Popov
031132038b
fix filtering by uint8 greater than 127
2021-01-29 10:39:18 +03:00
alexey-milovidov
fa7a795c3a
Merge pull request #19588 from ClickHouse/disable-checksums-on-read
...
Allow to disable checksums on read
2021-01-29 02:37:26 +03:00
alesapin
2881c830e3
Merge branch 'master' into fix_rare_bug_after_part_corruption
2021-01-28 23:16:52 +03:00
Denis Glazachev
009d1e3fc5
Merge branch 'master' into row-policy-with-prewhere
...
* master: (247 commits)
Update 01014_lazy_database_basic.sh
Update 00459_group_array_insert_at.sql
Same for ALTER
Syntax and links
Added test.
Check where and prewhere identifiers exist.
Remove redundant lsof
Add lsof to fasttest
Add new line to status
Make integration odbc tests idempotent
Minor code improvement in JOIN
Arcadia does not have bitmaps
Make Fuzzer more reliable
Add missing lsof for fasttest docker image
Update test
The test most likely would not work in Arcadia
Less parser depth
Fix stack overflow in coroutine
Fuzzer: better messages.
Fixed build.
...
2021-01-28 21:48:12 +04:00
alesapin
be1104d6c1
Fix typos
2021-01-28 19:07:47 +03:00
alesapin
c1d18fc8a2
Fix very rare case
2021-01-28 18:10:53 +03:00
alesapin
a7ca26fec4
Merge pull request #19708 from ClickHouse/fix_flaky_test_multiple_ttl
...
Fix flaky test test_concurrent_ttl_merges
2021-01-28 15:26:56 +03:00
alesapin
d14f4a525b
Merge branch 'master' into fix_rare_bug_after_part_corruption
2021-01-28 11:09:41 +03:00
alesapin
402e031a22
Throw exception only in debug mode
2021-01-28 11:07:18 +03:00
Alexey Milovidov
065d22cf2b
Merge branch 'master' into disable-checksums-on-read
2021-01-28 03:07:31 +03:00
Anton Popov
e5125b8c73
add comments and test for compatibility
2021-01-28 02:39:15 +03:00
alesapin
c74631c650
Fix test and add logical error
2021-01-27 21:54:05 +03:00