Alexey Milovidov
62b246c5a4
Avoid excessive "stat" syscall\
2019-02-27 23:02:48 +03:00
Nikita Vasilev
a1b0dede07
Data Skipping Indices ( #4143 )
...
* made index parser
* added index parsing
* some fixes
* added index interface and factory
* fixed compilation
* ptrs
* added indexParts
* indextypes
* index condition
* IndexCondition
* added indexes in selectexecutor
* fix
* changed comment
* fix
* added granularity
* comments
* fix
* fix
* added writing indexes
* removed indexpart class
* fix
* added setSkipIndexes
* add rw for MergeTreeIndexes
* fixes
* upd error
* fix
* fix
* reading
* test index
* fixed nullptr error
* fixed
* fix
* unique names
* asts -> exprlist
* minmax index
* fix
* fixed select
* fixed merging
* fixed mutation
* working minmax
* removed test index
* fixed style
* added indexes to checkDataPart
* added tests for minmax index
* fixed constructor
* fix style
* fixed includes
* fixed setSkipIndexes
* added indexes meta to zookeeper
* added parsing
* removed throw
* alter cmds parse
* fix
* added alter
* fix
* alters fix
* fix alters
* fix "after"
* fixed alter
* alter fix + test
* fixes
* upd setSkipIndexes
* fixed alter bug with drop all indices
* fix metadata editing
* new test and repl fix
* rm test files
* fixed repl alter
* fix
* fix
* indices
* MTReadStream
* upd test for bug
* fix
* added useful parsers and ast classes
* fix
* fix comments
* replaced columns
* fix
* fixed parsing
* fixed printing
* fix err
* basic IndicesDescription
* go to IndicesDescr
* moved indices
* go to indicesDescr
* fix test minmax_index*
* fixed MT alter
* fixed bug with replMT indices storing in zk
* rename
* refactoring
* docs ru
* docs ru
* docs en
* refactor
* rename tests
* fix docs
* refactoring
* fix
* fix
* fix
* fixed style
* unique idx
* unique
* fix
* better minmax calculation
* upd
* added getBlock
* unique_condition
* added termForAST
* unique
* fixed not
* uniqueCondition::mayBeTrueOnGranule
* fix
* fixed bug with double column
* is always true
* fix
* key set
* spaces
* test
* tests
* fix
* unique
* fix
* fix
* fixed bug with duplicate column
* removed unused data
* fix
* fixes
* __bitSwapLastTwo
* fix
2019-02-05 17:50:25 +03:00
Alexey Zatelepin
8957e73681
a tool to convert an old month-partition part to the custom-partitioned format [#CLICKHOUSE-4231]
2019-01-30 16:51:39 +03:00
alexey-milovidov
7cc867de0a
Merge branch 'master' into background-processing-pool-backoff
2018-12-14 17:41:55 +03:00
Emmanuel Donin de Rosière
982cd749be
Better management of DateTime partitionning #3786
...
Adding min_time and max_date in system.parts table
2018-12-10 11:20:19 +01:00
Amos Bird
3cf2177268
Flexible fields assignment
...
Field f;
before:
f = Int64(-42);
after:
f = -42;
2018-10-24 08:47:33 +08:00
Alexey Milovidov
99a64d3b92
Presumably better #2278
2018-10-17 06:13:00 +03:00
Alexey Milovidov
a2faa511cc
Less number of "stat" calls for each INSERT into MergeTree #2281
2018-07-08 06:56:24 +03:00
Alexey Zatelepin
a9b348fe5e
check if partition key condition can be true in parallelogram instead of range [#CLICKHOUSE-3757]
2018-06-04 19:20:02 +03:00
Alexey Zatelepin
2e77c508ad
add comments [#CLICKHOUSE-3747]
2018-06-04 14:43:09 +03:00
Alexey Zatelepin
f6c72f8e56
Merge branch 'master' into replicated-mergetree-mutations
...
Conflicts:
dbms/src/Common/ActionBlocker.h
dbms/src/Storages/MergeTree/ActiveDataPartSet.cpp
dbms/src/Storages/MergeTree/ActiveDataPartSet.h
dbms/src/Storages/MergeTree/MergeTreeData.h
dbms/src/Storages/MergeTree/MergeTreeDataPart.cpp
dbms/src/Storages/MergeTree/ReplicatedMergeTreeAlterThread.cpp
dbms/src/Storages/MergeTree/ReplicatedMergeTreeLogEntry.cpp
dbms/src/Storages/MergeTree/ReplicatedMergeTreeLogEntry.h
dbms/src/Storages/MergeTree/ReplicatedMergeTreeQueue.cpp
dbms/src/Storages/MergeTree/ReplicatedMergeTreeQueue.h
dbms/src/Storages/MergeTree/ReplicatedMergeTreeRestartingThread.cpp
dbms/src/Storages/StorageMergeTree.cpp
dbms/src/Storages/StorageReplicatedMergeTree.cpp
dbms/src/Storages/StorageReplicatedMergeTree.h
2018-05-28 18:37:30 +03:00
Alexey Zatelepin
aff6ad42ed
support empty parts [#CLICKHOUSE-3747]
2018-05-25 23:13:00 +03:00
Alexey Milovidov
13762471da
Changed DayNum_t to DayNum as Amos Bird suggested [#CLICKHOUSE-3]
2018-05-25 16:29:15 +03:00
Vitaliy Lyudvichenko
c888903488
Add query ALTER TABLE t1 [REPLACE|ATTACH] PARTITION FROM t2
...
* Add ATTACH PARTITION FROM table for MergeTree. [#CLICKHOUSE-3546]
* Implemented replicated case on non-leader replica. [#CLICKHOUSE-3546]
* Disable merges in the dropping range. [#CLICKHOUSE-3546]
* DROP PARTITION is atomic and simpler now. [#CLICKHOUSE-3546]
* Implemented more SYSTEM queries. [#CLICKHOUSE-2931] [#CLICKHOUSE-3546]
SYSTEM queries:
RESTART REPLICAS
SYNC REPLICA db.name
STOP MERGES [db.name]
START MERGES [db.name]
STOP FETCHES [db.name]
START FETCHES [db.name]
STOP REPLICATED SENDS [db.name]
START REPLICATED SENDS [db.name]
STOP REPLICATION QUEUES [db.name]
START REPLICATION QUEUES [db.name]
* Fixed a bunch of bugs in REPLACE PARTITION. [#CLICKHOUSE-3546]
* Add tests for REPLACE PARTITION and SYSTEM. [#CLICKHOUSE-3546]
* Add system.part_log logging. [#CLICKHOUSE-3546]
* Fixed long wait in SYNC REPLICA. [#CLICKHOUSE-3546]
* Add requested changes. [#CLICKHOUSE-3546]
Fixed clickhouse-client bad return code.
* Add requested chenges. [#CLICKHOUSE-3546]
* Add requested chenges. [#CLICKHOUSE-3546]
2018-05-21 16:49:54 +03:00
Alex Zatelepin
c5b04f7f17
Add info about total uncompressed part size to system tables ( #2052 )
...
* column size calculating routines now take into account multiple streams [#CLICKHOUSE-2831]
* rename for clarity
* use more appropriate method for calculating table size
* add info about total uncompressed part size to system.parts and system.parts_columns [#CLICKHOUSE-2831]
* rename columns for clarity [#CLICKHOUSE-2831]
* count shared substreams only once [#CLICKHOUSE-2831]
2018-03-26 17:18:04 +03:00
Vitaliy Lyudvichenko
7c2a99c9b8
Introduced MinimalisticDataPartChecksums. [#CLICKHOUSE-3660]
2018-03-23 00:26:46 +03:00
Vitaliy Lyudvichenko
7abbe093e9
Avoid magic constants in system.parts. [#CLICKHOUSE-3]
...
Resolves #2039
2018-03-14 19:04:19 +03:00
Vitaliy Lyudvichenko
56bfb05ebb
Fix insane DateTime in system.parts. [#CLICKHOUSE-3]
2018-03-14 19:00:02 +03:00
Alexey Milovidov
07d8db9f2a
Explicit semantic for TSan [#CLICKHOUSE-2]
2018-03-03 20:44:53 +03:00
Alexey Milovidov
c897e5ca39
Better [#CLICKHOUSE-2]
2018-03-03 18:36:20 +03:00
Vitaliy Lyudvichenko
61233b886a
Better system.part_log. [#CLICKHOUSE-3342]
2018-01-25 21:40:02 +03:00
Nikolai Kochetov
de62ae6b1b
added system.parts_columns as a copy of system.parts [#CLICKHOUSE-3328]
2017-12-04 18:44:31 +03:00
Alexey Milovidov
5510bea3f8
Unification of serde of data types: development [#CLICKHOUSE-2838].
2017-12-03 01:12:27 +03:00
Vitaliy Lyudvichenko
b738d1baed
Add multi index for data_parts storage. And fixed bugs. [#CLICKHOUSE-3452]
...
Fixed handling of obsolete parts.
Fixed conflict resolution between simultaneous PreCommitted covering parts.
Fixed memory leak caused by ordinary MergeTree parts stucked in Deleting state.
Added hidden _state column into system.parts.
2017-11-29 14:25:11 +03:00
Alexey Milovidov
729b114c15
Removed unsupported functionality [#CLICKHOUSE-2].
2017-11-18 05:34:00 +03:00
Vitaliy Lyudvichenko
397b8bcf92
Smarter parts deletion. [#CLICKHOUSE-3178]
2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
db3a67a421
Add clearer RangeFiltered implementation. [#CLICKHOUSE-3178]
2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
e2a12d1088
Introduced logic for MergeTree parts states. [#CLICKHOUSE-3178]
2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
5787e8b257
Add state for MergeTree parts. [#CLICKHOUSE-3178]
...
And Removed obsolete code.
2017-10-26 17:16:06 +03:00
Alexey Zatelepin
b6b5d1cf25
store rows count for part on disk to allow tables without fixed size columns [#CLICKHOUSE-3000]
2017-10-24 17:11:53 +03:00
Alexey Zatelepin
1be62b567e
safeguards to protect against distinct partition values with the same partition_id [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
f0995f2741
partition serialization into id and text [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
71ebf67f2e
encapsulate partition and minmax_idx load/store code [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
e9f93028b8
serialize partition and minmax index values [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
2f80c150db
save MergeTree data format version and parse part names depending on it [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
d2847f22af
PR fixes [#CLICKHOUSE-3000]
2017-08-31 23:32:03 +03:00
Alexey Zatelepin
645e4b7a53
generalize MinMax index [#CLICKHOUSE-3000]
2017-08-31 23:32:03 +03:00
Alexey Zatelepin
5471979f46
generic partitioning of inserted blocks [#CLICKHOUSE-3000]
2017-08-31 23:32:03 +03:00
Alexey Zatelepin
be1c62d7f8
split partition and minmax index [#CLICKHOUSE-3000]
2017-08-31 23:32:03 +03:00
Alexey Zatelepin
99969ab77a
Introduce MergeTreePartitionIndex. Isolate its loading code. [#CLICKHOUSE-3000]
2017-08-31 23:32:03 +03:00
Alexey Milovidov
164ad05ae1
Little better [#CLICKHOUSE-3231].
2017-08-16 22:22:49 +03:00
Alexey Zatelepin
f25f0cd759
reduce number of places where partitioning by month is assumed [#CLICKHOUSE-3000]
2017-08-15 22:30:52 +03:00
Alexey Milovidov
56a5bb53fa
Miscellaneous [#CLICKHOUSE-2].
2017-08-10 00:07:01 +03:00
Alexey Milovidov
dd1f54bce2
Changed Poco::RWLock to std::shared_lock [#APRILDEV-91].
2017-07-28 20:34:02 +03:00
Nikolai Kochetov
985e449bd3
added IDataType::updateAvgValueSizeHints; renamed PODArray::allocated_size to PODArray::allocated_bytes; renamed renamed IColumn::allocatedSize to IColumn::allocatedBytes; removed IColumn.h from IDataType.h
2017-07-13 19:49:09 +03:00
Alexey Milovidov
3c660fd65b
Fixed translation errors [#CLICKHOUSE-3091].
2017-06-25 03:17:08 +03:00
Alexey Milovidov
6df5d556e5
Changed namespace [#DEVTOOLS-3381].
2017-06-21 11:35:38 +03:00
Alexey Milovidov
604de2bfeb
Better [#DEVTOOLS-3381].
2017-06-21 04:24:05 +03:00
Alexey Milovidov
fa64d14f26
More readable temporary part names [#CLICKHOUSE-2].
2017-05-26 03:47:06 +03:00
Vitaliy Lyudvichenko
d9637c2980
MergeTreeDataPart refactoring. [#CLICKHOUSE-2989]
...
Use relative_path to store actual part path.
Use renameTo() to move parts.
2017-05-24 23:23:58 +03:00