Alexey Milovidov
f5a06cee5b
Simplification of Nullable [#CLICKHOUSE-2]
2017-12-07 11:47:13 +03:00
Alexey Milovidov
e776a4fc67
Simplification of Nullable [#CLICKHOUSE-2]
2017-12-07 07:26:30 +03:00
Alexander Avdonkin
89db5040df
Implemented MySQL and ODBC storages and table functions
2017-12-05 16:32:02 +03:00
alexey-milovidov
6c86bc2506
Merge pull request #1584 from yandex/table-system-models
...
added system.models
2017-12-05 00:25:04 +03:00
alexey-milovidov
b7f656d289
Merge pull request #1541 from yandex/fix-replica-queue-load
...
Add idempotent retries of updating the replica queue in ZooKeeper
2017-12-05 00:16:03 +03:00
Alexey Milovidov
7e2100d7f7
Fixed exception message [#CLICKHOUSE-2].
2017-12-04 23:37:28 +03:00
alexey-milovidov
575a67b5d2
Merge branch 'master' into support-table-functions-for-desc-and-insert
2017-12-04 23:34:27 +03:00
Alexey Zatelepin
fb9c7ab81d
remove experimental_allow_extended_storage_definition_syntax Setting (turn on by default) [#CLICKHOUSE-3000]
2017-12-04 22:45:18 +03:00
proller
826936cf9b
Better fix of std::numeric_limits<__uint128_t> ( #1585 )
...
* Fix std::numeric_limits<__uint128_t>
* clean
* Better fix of std::numeric_limits<__uint128_t>
2017-12-04 19:12:56 +03:00
Nikolai Kochetov
de62ae6b1b
added system.parts_columns as a copy of system.parts [#CLICKHOUSE-3328]
2017-12-04 18:44:31 +03:00
Nikolai Kochetov
d2710bd0f4
added system.models [#CLICKHOUSE-3477]
2017-12-04 16:32:49 +03:00
proller
4c3dfb97df
Clang: Fix std::numeric_limits<__uint128_t> ( #1583 )
...
Fix std::numeric_limits<__uint128_t>
2017-12-04 16:03:40 +03:00
Alexey Milovidov
06dfe7f562
Unification of serde of data types: development [#CLICKHOUSE-2838].
2017-12-03 08:55:49 +03:00
Alexey Milovidov
a80259d0cb
Unification of serde of data types: development [#CLICKHOUSE-2838].
2017-12-03 07:57:33 +03:00
Alexey Milovidov
418e5012e7
Unification of serde of data types: development [#CLICKHOUSE-2838].
2017-12-03 07:43:38 +03:00
Alexey Milovidov
3717d80017
Unification of serde of data types: development [#CLICKHOUSE-2838].
2017-12-03 07:40:21 +03:00
Alexey Milovidov
c472172fa8
Unification of serde of data types: development [#CLICKHOUSE-2838].
2017-12-03 07:25:41 +03:00
Alexey Milovidov
a597f3e7d9
Unification of serde of data types: development [#CLICKHOUSE-2838].
2017-12-03 07:23:03 +03:00
Alexey Milovidov
3786786aeb
Unification of serde of data types: development [#CLICKHOUSE-2838].
2017-12-03 07:19:46 +03:00
Alexey Milovidov
294c0ba5cf
Unification of serde of data types: development [#CLICKHOUSE-2838].
2017-12-03 06:43:48 +03:00
Alexey Milovidov
ab022d0882
Unification of serde of data types: development [#CLICKHOUSE-2838].
2017-12-03 05:59:59 +03:00
Alexey Milovidov
1aa99092c3
Unification of serde of data types: development [#CLICKHOUSE-2838].
2017-12-03 05:15:35 +03:00
Alexey Milovidov
a9ddaa2b19
Unification of serde of data types: development [#CLICKHOUSE-2838].
2017-12-03 04:49:54 +03:00
Alexey Milovidov
a813287afd
Rewrite MergeTreePartChecker [#CLICKHOUSE-2838].
2017-12-03 03:48:19 +03:00
Alexey Milovidov
464d962010
Rewrite MergeTreePartChecker [#CLICKHOUSE-2838].
2017-12-03 03:46:34 +03:00
Alexey Milovidov
5510bea3f8
Unification of serde of data types: development [#CLICKHOUSE-2838].
2017-12-03 01:12:27 +03:00
Alexey Milovidov
b932a193e2
Better [#CLICKHOUSE-2].
2017-12-02 06:25:52 +03:00
Alexey Milovidov
c748493a86
Better [#CLICKHOUSE-2].
2017-12-02 05:47:12 +03:00
Alexey Milovidov
7196602e12
Better [#CLICKHOUSE-2].
2017-12-02 00:51:50 +03:00
Alexey Milovidov
54e601019c
Better [#CLICKHOUSE-2].
2017-12-02 00:40:58 +03:00
Alexey Milovidov
a85773bdbc
Fixed error [#CLICKHOUSE-2838].
2017-12-02 00:30:43 +03:00
Alexey Milovidov
acd78e4891
Better [#CLICKHOUSE-2].
2017-12-02 00:13:25 +03:00
Alexey Milovidov
1efef27af9
Better [#CLICKHOUSE-2]
2017-12-01 23:38:50 +03:00
Alexey Milovidov
bdd25b15bd
Better [#CLICKHOUSE-2].
2017-12-01 22:34:51 +03:00
Alexey Milovidov
1a6cb2989b
Better [#CLICKHOUSE-2].
2017-12-01 21:36:55 +03:00
Nikolai Kochetov
5cbacbc2d7
fixed test [#CLICKHOUSE-3397]
2017-12-01 21:21:37 +03:00
alexey-milovidov
09f030a3ee
Merge branch 'master' into data_parts_multi_index
2017-12-01 19:53:04 +03:00
Alexey Milovidov
98291f1a24
Cleanups [#CLICKHOUSE-2838].
2017-12-01 17:21:24 +03:00
Alexey Milovidov
6fb18a7dc2
Added comments [#CLICKHOUSE-2838].
2017-12-01 17:15:38 +03:00
Nikolai Kochetov
470c633e09
merged with master
2017-12-01 16:33:47 +03:00
Alexey Milovidov
d4089695ad
Cleanups [#CLICKHOUSE-2838].
2017-12-01 16:32:37 +03:00
Nikolai Kochetov
f025f22007
Merge branch 'master' into support-table-functions-for-desc-and-insert
2017-12-01 16:24:50 +03:00
Alexey Milovidov
241e0638b7
Merge branch 'master' of github.com:yandex/ClickHouse into unify-data-types-that-serialized-with-multiple-streams
2017-12-01 15:36:15 +03:00
Kirill Shvakov
cd7f7b076e
Merge branch 'add-description-to-settings-table'
2017-11-30 13:55:42 +02:00
Kirill Shvakov
4d73516508
Add description to system.settings table
2017-11-30 13:50:02 +02:00
Vitaliy Lyudvichenko
ddb02d8ccf
Small enhancements. [#CLICKHOUSE-3452]
2017-11-29 14:54:37 +03:00
Vitaliy Lyudvichenko
b738d1baed
Add multi index for data_parts storage. And fixed bugs. [#CLICKHOUSE-3452]
...
Fixed handling of obsolete parts.
Fixed conflict resolution between simultaneous PreCommitted covering parts.
Fixed memory leak caused by ordinary MergeTree parts stucked in Deleting state.
Added hidden _state column into system.parts.
2017-11-29 14:25:11 +03:00
Alexey Milovidov
fb4951b679
Unification of serde of data types: development [#CLICKHOUSE-2838].
2017-11-29 02:50:17 +03:00
Alexey Milovidov
7b4bffe088
Unification of serde of data types: development [#CLICKHOUSE-2838].
2017-11-29 02:31:21 +03:00
Alexey Zatelepin
01d42242b1
mark queue as dirty if status of ZK operation is unknown [#CLICKHOUSE-3405]
2017-11-28 17:09:53 +03:00
Alexey Milovidov
36d4f89c7a
Unification of serde of data types: development [#CLICKHOUSE-2838].
2017-11-28 05:13:46 +03:00
Alexey Milovidov
71378d329c
Merge branch 'unify-data-types-that-serialized-with-multiple-streams' of github.com:yandex/ClickHouse into unify-data-types-that-serialized-with-multiple-streams
2017-11-28 02:47:21 +03:00
Alexey Milovidov
719d2654f8
Unification of serde of data types: development [#CLICKHOUSE-2838].
2017-11-28 02:47:14 +03:00
Alexey Milovidov
55971073fc
Unification of serde of data types: development [#CLICKHOUSE-2838].
2017-11-28 00:21:09 +03:00
Alexey Milovidov
95d33b3e65
Merge remote-tracking branch 'origin/master' into unify-data-types-that-serialized-with-multiple-streams
2017-11-27 22:33:38 +03:00
Alexey Milovidov
42cc689474
Miscellaneous [#CLICKHOUSE-2].
2017-11-26 22:22:33 +03:00
Alexey Milovidov
6266ecc5ed
Miscellaneous [#CLICKHOUSE-2].
2017-11-25 02:03:58 +03:00
proller
827a5d0612
Fix warnings-errors in gtest
2017-11-24 23:40:14 +03:00
proller
6ff1968196
Compile and likk optimize and fixes
2017-11-24 21:08:01 +03:00
proller
0d3c4f4d67
Core/FieldVisitors.h -> Common/FieldVisitors.h
...
Core/Progress.h -> IO/Progress.h
tests: sudo --non-interactive
Fix freebsd link
2017-11-24 16:55:31 +03:00
Alexey Milovidov
859bf86a43
Fixed error found by ASan with debug libc++ (actually not dangerous) [#CLICKHOUSE-2].
2017-11-23 18:24:55 +03:00
Alexey Zatelepin
78aaa42ddd
add idempotent retries of updating the replica queue in ZooKeeper [#CLICKHOUSE-3405]
2017-11-23 16:19:29 +03:00
Alexey Milovidov
e15d637714
Miscellaneous [#CLICKHOUSE-2].
2017-11-23 05:12:09 +03:00
Alexey Milovidov
4bf120b285
Miscellaneous [#CLICKHOUSE-2].
2017-11-23 05:11:16 +03:00
Alexey Milovidov
4aa520e7be
Merge remote-tracking branch 'origin/master' into unify-data-types-that-serialized-with-multiple-streams
2017-11-21 22:40:59 +03:00
Alexey Milovidov
4bc98c9253
Unification of serde of data types: development [#CLICKHOUSE-2838].
2017-11-21 05:25:36 +03:00
Alexey Milovidov
63f79b7b4f
Unification of serde of data types: development [#CLICKHOUSE-2838].
2017-11-21 05:23:41 +03:00
Amos Bird
dc6847d5c6
Fixes StorageJoin's sample block order.
...
Here is a reproducible test case.
```
create table e (s UInt64, t UInt64) Engine = Memory;
create table v (s UInt64, w Float64, c UInt64) Engine = Join(Any, Inner, s);
insert into e values (1, 2), (1, 3), (1, 4), (2, 1), (2, 4), (3, 1), (4, 2), (4, 3);
insert into v values (1, 0.5, 3), (2, 0.5, 2), (3, 1, 1), (4, 0.5, 2);
select *, w, c from e any inner join v using (s);
```
# before this patch
```
┌─s─┬─t─┬─────w─┬──────────c─┐
│ 1 │ 2 │ 4e-45 │ 1051372192 │
│ 1 │ 3 │ 4e-45 │ 1051372192 │
│ 1 │ 4 │ 4e-45 │ 1051372192 │
│ 2 │ 1 │ 3e-45 │ 1056964608 │
│ 2 │ 4 │ 3e-45 │ 1056964608 │
│ 3 │ 1 │ 1e-45 │ 1065353216 │
│ 4 │ 2 │ 3e-45 │ 1056964608 │
│ 4 │ 3 │ 3e-45 │ 1056964608 │
└───┴───┴───────┴────────────┘
```
# after this patch
```
┌─s─┬─t─┬───w─┬─c─┐
│ 1 │ 2 │ 0.5 │ 3 │
│ 1 │ 3 │ 0.5 │ 3 │
│ 1 │ 4 │ 0.5 │ 3 │
│ 2 │ 1 │ 0.5 │ 2 │
│ 2 │ 4 │ 0.5 │ 2 │
│ 3 │ 1 │ 1 │ 1 │
│ 4 │ 2 │ 0.5 │ 2 │
│ 4 │ 3 │ 0.5 │ 2 │
└───┴───┴─────┴───┘
```
2017-11-20 19:46:24 +08:00
Alexey Milovidov
3b5f82000c
Fixed linking [#CLICKHOUSE-2].
2017-11-20 08:22:54 +03:00
Alexey Milovidov
3403f68c3f
Fixed linking [#CLICKHOUSE-2].
2017-11-20 08:21:50 +03:00
Alexey Milovidov
87ff2f35ec
Miscellaneous [#CLICKHOUSE-2].
2017-11-20 07:41:56 +03:00
Alexey Milovidov
2106849ec2
Miscellaneous [#CLICKHOUSE-2].
2017-11-20 07:40:51 +03:00
Alexey Milovidov
b3f6844171
Merge remote-tracking branch 'origin/master' into unify-data-types-that-serialized-with-multiple-streams
2017-11-20 07:18:37 +03:00
Alexey Milovidov
fd3239dd46
Unification of serde of data types: development [#CLICKHOUSE-2838].
2017-11-20 07:18:27 +03:00
Alexey Milovidov
01e8adbfff
Removed "experimental" namespace [#CLICKHOUSE-2].
2017-11-20 07:15:43 +03:00
Alexey Milovidov
8e6466454a
Unification of serde of data types: development [#CLICKHOUSE-2838].
2017-11-20 05:15:15 +03:00
Alexey Milovidov
2ca6952deb
Merge branch 'master' into unify-data-types-that-serialized-with-multiple-streams
2017-11-20 02:16:18 +03:00
Alexey Milovidov
6c8b5e29d8
Additions after merge [#CLICKHOUSE-2].
2017-11-20 00:17:58 +03:00
alexey-milovidov
fc0a51f1b9
Merge branch 'master' into fix4
2017-11-19 23:57:44 +03:00
Alexey Milovidov
60825748d4
Merge branch 'fix11' of https://github.com/proller/ClickHouse into proller-fix11
2017-11-19 22:57:28 +03:00
Alexey Milovidov
94d3b49e99
BackgroundSchedulePool: miscellaneous [#CLICKHOUSE-2].
2017-11-19 03:27:43 +03:00
Alexey Milovidov
9b92c0fecd
BackgroundSchedulePool: miscellaneous [#CLICKHOUSE-2].
2017-11-19 03:22:13 +03:00
Alexey Milovidov
f497b0c63c
BackgroundSchedulePool: miscellaneous; added TODO [#CLICKHOUSE-2].
2017-11-19 03:08:37 +03:00
Silviu Caragea
dfecd0b2f2
Merge with master
2017-11-18 10:16:06 +02:00
Alexey Milovidov
729b114c15
Removed unsupported functionality [#CLICKHOUSE-2].
2017-11-18 05:34:00 +03:00
Alexey Milovidov
076e0ae014
More clear [#CLICKHOUSE-2].
2017-11-18 00:20:28 +03:00
Alexey Milovidov
7c9ba15ba1
Style [#CLICKHOUSE-2].
2017-11-17 23:42:03 +03:00
proller
aa3dd03389
Split lib dbms to dbms + dbms_common_io
2017-11-17 22:19:49 +03:00
proller
79b3ef2f57
Merge remote-tracking branch 'upstream/master' into fix4
2017-11-17 15:31:13 +03:00
Silviu Caragea
64619c614f
Basic idea is :
...
- We add BackgroundSchedulePool which can execute a function at a specific point in time. Basically all tasks are added in a queue and precessed by worker threads.
- The most important difference between this and BackgroundProcessingPool is that we have the guarantee that the same function is not executed from many workers in the same time
- Each of the following classes instead starting a thread will register a task in BackgroundSchedulePool and when they need to run will call schedule or scheduleAfter(duration) functions
This commit is moving all threads created by ReplicatedMergeTree to BackgroundSchedulePool tasks
NOTE: I did a minimum number of changes to be much simple to review the code
2017-11-17 10:58:35 +02:00
Marek Vavruša
c07b48075b
StorageKafka: fixed messages not showing when EOF is reached in current batch
...
This prevents messages from being read if there's less than 16 messages to read in Kafka.
Updated documentation with more detailed description.
2017-11-15 23:17:22 +03:00
alexey-milovidov
dbee90ba5b
Update ReplicatedMergeTreeBlockOutputStream.cpp
2017-11-15 23:13:10 +03:00
Alexey Zatelepin
99b2d135df
clear deduplication blocks when doing DROP PARTITION [#CLICKHOUSE-3208]
2017-11-15 23:13:10 +03:00
Alexey Milovidov
2846c7560f
Fixed wording [#CLICKHOUSE-2].
2017-11-15 22:47:49 +03:00
proller
468e2ab044
Merge remote-tracking branch 'upstream/master' into fix4
2017-11-14 14:50:19 +03:00
Alexey Milovidov
c4b4764a0a
Fixed error after merge [#CLICKHOUSE-2].
2017-11-14 03:11:33 +03:00
Nikolai Kochetov
14d2b830bb
fix build
2017-11-10 14:44:32 +03:00
Nikolai Kochetov
cdd852902d
Merge branch 'master' into catboost-models
2017-11-10 14:40:08 +03:00
Nikolai Kochetov
ad381f90e5
changed column names in catboost pool [#CLICKHOUSE-3305]
2017-11-10 14:31:02 +03:00
Nikolai Kochetov
2ae3f0b3b4
fixed tests [#CLICKHOUSE-3305]
...
added aliases for catboost pool [#CLICKHOUSE-3305]
2017-11-10 14:30:48 +03:00
Alexey Milovidov
ac1b69247c
Addition to prev. revision [#CLICKHOUSE-3402].
2017-11-09 22:25:21 +03:00
Alexey Milovidov
783e812eb7
Fixed error [#CLICKHOUSE-2]
2017-11-04 19:46:14 +03:00
Alexey Milovidov
1aaff5762c
Attempt to change ext::shared_ptr_helper (incomplete) [#CLICKHOUSE-2].
2017-11-04 06:20:18 +03:00
Alexey Milovidov
fa8bcccdb9
Attempt to change ext::shared_ptr_helper (incomplete) [#CLICKHOUSE-2].
2017-11-04 00:50:22 +03:00
Nikolai Kochetov
1cee62eb0a
supported table_function in insert query [#CLICKHOUSE-3397]
2017-11-03 16:25:39 +03:00
Nikolai Kochetov
1d01d228a4
removed macro DEFINE_AST_QUERY_WITH_TABLE_AND_OUTPUT
...
supported table_functions in desctibe table query [#CLICKHOUSE-3397]
2017-11-03 16:24:52 +03:00
Vitaliy Lyudvichenko
a4636f8656
Fixed incorrect INSERT into Distributed table in async mode with local replicas in a shard. #1404 [#CLICKHOUSE-2]
...
Resolves #1404
2017-11-03 00:32:59 +03:00
proller
c974f0e88e
Merge remote-tracking branch 'upstream/master' into fix4
2017-11-02 17:42:36 +03:00
Vitaliy Lyudvichenko
9cf14d2c61
Remove obsolete code. [#CLICKHOUSE-2]
2017-11-02 17:35:12 +03:00
proller
e671c48a6e
Merge remote-tracking branch 'upstream/master' into fix4
2017-11-02 14:50:47 +03:00
KochetovNicolai
bfb4e8f3fe
zookeeper chroot fix ( #1430 )
...
* added root and identity section to zookeeper configuration [#CLICKHOUSE-3031]
* added root and identity section to zookeeper configuration [#CLICKHOUSE-3031]
* moved root section in zookeeper configuration [#CLICKHOUSE-3031]
* fixed root section in zookeeper configuration [#CLICKHOUSE-3031]
* added tests [#CLICKHOUSE-3031]
* replaced tabs to spaces in config files [#CLICKHOUSE-3031]
* fix build [#CLICKHOUSE-3031]
* added chech that root starts with '/' in zookeeper config file [#CLICKHOUSE-3031]
* added comments [#CLICKHOUSE-3031]
* Check if zookeeper chroot exists before start. Remove tailing slash form chroot. [#CLICKHOUSE-3400]
* added starting '/' to zookeeper path in ReplicatedMergeTree
2017-11-02 01:59:27 +03:00
proller
6da4f5a661
add locks
2017-11-01 22:56:07 +03:00
proller
3b58e9cd67
Fix atomic usage
2017-11-01 20:15:05 +03:00
proller
c5dcde0c78
fix atomic usage
2017-11-01 16:58:02 +03:00
proller
9bc029ba17
Merge remote-tracking branch 'upstream/master' into fix4
2017-11-01 14:43:48 +03:00
Alexey Milovidov
6aff58c747
Added backward compatibility for tables created with wrong sampling key; fixed tiny error [#CLICKHOUSE-2].
2017-11-01 04:45:10 +03:00
proller
47eae26d75
Fix build
2017-10-31 22:19:36 +03:00
proller
6debbf819a
Merge remote-tracking branch 'origin/fix4' into fix4
2017-10-31 18:36:04 +03:00
proller
9e73e08269
wip
2017-10-31 17:40:44 +03:00
Nikolai Kochetov
554bb889ac
merged with master
2017-10-31 15:04:13 +03:00
Nikolai Kochetov
2bd2bc6191
renamed feature columns, check CatBoost pool in data directory [#CLICKHOUSE-3305]
...
renamed feature columns, check CatBoost pool in data directory [#CLICKHOUSE-3305]
2017-10-31 14:50:23 +03:00
Nikolai Kochetov
9baa5b9e14
add non-feature columns as materialized to StorageCatBoostPool [#CLICKHOUSE-3305]
2017-10-31 14:50:22 +03:00
Nikolai Kochetov
d54e27cdcb
added TableFunctionCatBoostPool [#CLICKHOUSE-3305]
...
fix column descriptions parsing [#CLICKHOUSE-3305]
2017-10-31 14:50:22 +03:00
Nikolai Kochetov
8ebce6ef5d
StorageCatBoostPool [#CLICKHOUSE-3305]
2017-10-31 14:50:21 +03:00
proller
bd1d699441
Merge remote-tracking branch 'upstream/master' into fix4
2017-10-30 22:30:28 +03:00
Alexey Zatelepin
f2ebffb7f0
save TO table of a materialized view to the ATTACH query [#CLICKHOUSE-2]
2017-10-30 21:39:39 +03:00
proller
723cbb310d
Merge remote-tracking branch 'upstream/master' into fix4
2017-10-30 21:19:28 +03:00
alexey-milovidov
56ef2e9196
Merge pull request #1380 from yandex/catboost-models
...
Catboost models
2017-10-30 19:16:41 +03:00
proller
5d137e22eb
More atomics
2017-10-30 17:38:16 +03:00
proller
cda6473668
Merge remote-tracking branch 'upstream/master' into fix4
2017-10-30 16:45:22 +03:00
Alexey Zatelepin
141b85189e
always allow attaching tables with extended storage definition syntax [#CLICKHOUSE-3000]
2017-10-30 16:44:43 +03:00
proller
8b3d3cf56f
Merge remote-tracking branch 'upstream/master' into fix4
2017-10-30 16:19:19 +03:00
Vitaliy Lyudvichenko
4a55e5f94b
Hotfix of CREATE MATERIALIZED VIEW a TO b. [#CLICKHOUSE-2]
2017-10-30 15:14:36 +03:00
proller
b7dd5cd0ba
Merge remote-tracking branch 'upstream/master' into fix4
2017-10-30 13:42:34 +03:00
proller
956e1513c5
Merge remote-tracking branch 'upstream/master' into fix4
2017-10-30 13:22:20 +03:00
Alexey Milovidov
e36906cdf5
Fixed build [#CLICKHOUSE-2].
2017-10-28 06:59:42 +03:00
proller
ba49157f84
Merge remote-tracking branch 'upstream/master' into fix4
2017-10-28 01:06:09 +03:00
Alexey Milovidov
9e3daca437
Fixed build [#CLICKHOUSE-2].
2017-10-28 00:36:43 +03:00
Alexey Milovidov
b2892e058a
Fixed bad file name [#CLICKHOUSE-2].
2017-10-28 00:31:10 +03:00
Alexey Milovidov
5ab78c249d
Fixed build after merge [#CLICKHOUSE-2].
2017-10-28 00:18:06 +03:00
alexey-milovidov
9e200bdfd9
Merge branch 'master' into materialized-view-to
2017-10-27 23:21:27 +03:00
proller
c878af8740
Merge remote-tracking branch 'upstream/master' into fix4
2017-10-27 23:13:35 +03:00
alexey-milovidov
3b99b723d7
Merge pull request #1245 from yandex/CLICKHOUSE-3178
...
Atomic parts addition and removing
2017-10-27 23:03:33 +03:00
alexey-milovidov
b4bb6796c1
Merge pull request #1407 from yandex/CLICKHOUSE-3381
...
Ability to turn off deduplication, smart deduplication for MV
2017-10-27 23:03:25 +03:00
proller
07009d5072
Bughunt tsan
2017-10-27 18:34:26 +03:00
proller
1736d7d312
Allow build with system lib rdkafka
2017-10-27 17:40:41 +03:00
Alexey Milovidov
41c05f22be
Removed very old obsolete tests [#CLICKHOUSE-2].
2017-10-26 21:49:48 +03:00
Alexey Milovidov
6022201c26
Removed very old obsolete test [#CLICKHOUSE-2].
2017-10-26 21:38:40 +03:00
Alexey Milovidov
a27981b4ef
Get rid of double whitespaces [#CLICKHOUSE-2].
2017-10-26 21:36:23 +03:00
Vitaliy Lyudvichenko
51265506f8
Convey session setting up to inner table creation for MV. [#CLICKHOUSE-3381]
2017-10-26 18:39:56 +03:00
Vitaliy Lyudvichenko
a0d06c5ee0
Don't deduplicate a block inserted from MV if the source block is Ok. [#CLICKHOUSE-3381]
2017-10-26 17:58:00 +03:00
Nikolai Kochetov
371e234f01
changed ExternalLoader::getObjectsMap return type [#CLICKHOUSE-3305]
2017-10-26 17:26:03 +03:00
Vitaliy Lyudvichenko
602560cb2e
Update system.parts table. [#CLICKHOUSE-3178]
2017-10-26 17:20:21 +03:00
Vitaliy Lyudvichenko
483dfd1b44
Fixed "No part error" in replicated sender. [#CLICKHOUSE-3178]
2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
397b8bcf92
Smarter parts deletion. [#CLICKHOUSE-3178]
2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
d6ea9c3dcf
Fixed awful copypaste. [#CLICKHOUSE-3178]
2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
b5c7eaa80e
Force data parts cleanup before repeat fetch. [#CLICKHOUSE-3178]
2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
71833a7f0f
More correct procedure of parts deleting. [#CLICKHOUSE-3178]
2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
db3a67a421
Add clearer RangeFiltered implementation. [#CLICKHOUSE-3178]
2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
e2a12d1088
Introduced logic for MergeTree parts states. [#CLICKHOUSE-3178]
2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
5787e8b257
Add state for MergeTree parts. [#CLICKHOUSE-3178]
...
And Removed obsolete code.
2017-10-26 17:16:06 +03:00
Alexey Zatelepin
ec963e3397
save evaluated storage parameters in ATTACH queries [#CLICKHOUSE-3000]
2017-10-26 17:07:15 +03:00
proller
9ccba5fad1
Fix includes
2017-10-26 14:46:57 +03:00
Alexey Zatelepin
6ada6d9aa5
remove dead code [#CLICKHOUSE-2]
2017-10-26 07:53:52 +03:00
Alexey Zatelepin
01c39dcf1c
fix CREATE TABLE bugs and idiosyncrasies: [#CLICKHOUSE-3000]
...
* fix segfault on ATTACH MATERIALIZED VIEW
* allow only TEMPORARY TABLEs (not databases or views)
* forbid choosing ENGINE for TEMPORARY tables
* fix segfault on CREATE TABLE t1 AS t2 where t2 is a View
2017-10-26 07:53:52 +03:00
Alexey Milovidov
65982a1642
Fixed exception messages [#CLICKHOUSE-2].
2017-10-25 22:17:37 +03:00
proller
4ffbca2d8c
Fix warnings ( #1406 )
...
* Log query id in executeQuery; Better type mismatch error; change format in report tool
* Better log query_id
* fix message
* Use c++11 thread_local instaed of gcc's __thread
* lock mutex before notifying waiting thread in sync insertion into distributed [#CLICKHOUSE-3379]
* Cmake: fix build without downloaded submodules (#1379 )
* fix
* ZooKeeper: fixed stack smashing with tryGet()
The tryGet() operation creates a 1MB buffer on stack. This may or
may not work depending on the default stack size for threads,
whether the stack protector is enabled or not, recursion depth,
and the actual value size.
This is probably going to slow down some ZK operations, but I don't
see how else this could work reliably with the existing API.
* increased timeout for test_insertion_sync_fails_with_timeout
* Update CHANGELOG_RU.md
* Update ZooKeeper.cpp
* Fix warnings
* Fixes
* Dont strip debug info from asan, tsan and other builds except releases
* Fix asan error causd by test 00144
* Fix empty log message (#CLICKHOUSE-3378)
2017-10-25 21:39:10 +03:00
alexey-milovidov
b5155916a8
Merge branch 'master' into materialized-view-to
2017-10-24 22:39:24 +03:00
Alexey Zatelepin
6bb34a4494
Merge branch 'master' into arbitrary-partitioning
...
Conflicts:
dbms/src/Storages/StorageFactory.cpp
2017-10-24 17:23:18 +03:00
Alexey Zatelepin
b6b5d1cf25
store rows count for part on disk to allow tables without fixed size columns [#CLICKHOUSE-3000]
2017-10-24 17:11:53 +03:00
alexey-milovidov
3c2155df0b
Update StorageKafka.cpp
2017-10-23 20:47:33 +03:00
alexey-milovidov
139e2080ae
Update StorageKafka.cpp
2017-10-23 20:47:33 +03:00
Marek Vavruša
3926851343
StorageKafka: use config string instead of just server name
2017-10-23 20:47:33 +03:00
Marek Vavruša
a7c391fbf7
StorageKafka: use std::vector instead of stack array for error string buffers
2017-10-23 20:47:33 +03:00
Marek Vavruša
f4c1149606
Settings: renamed schema to format_schema
2017-10-23 20:47:33 +03:00
Alexey Milovidov
c57a62611b
Removed StorageTrivialBuffer because it is underdeveloped [#CLICKHOUSE-2].
2017-10-23 20:38:56 +03:00
proller
c66228a746
Log query id in executeQuery; Better type mismatch error; change format in report tool ( #1372 )
...
* Log query id in executeQuery; Better type mismatch error; change format in report tool
* Better log query_id
* fix message
* Use c++11 thread_local instaed of gcc's __thread
* lock mutex before notifying waiting thread in sync insertion into distributed [#CLICKHOUSE-3379]
* Cmake: fix build without downloaded submodules (#1379 )
* fix
2017-10-23 20:32:39 +03:00
Nikolai Kochetov
5966574405
fix timeout lock in DistributedBlockOutputStream
2017-10-23 16:47:00 +03:00
Nikolai Kochetov
3317251e2f
fix jobs flags in DistributedBlockOutputStream
2017-10-23 15:00:56 +03:00
Marek Vavruša
975a7ada42
StorageMaterializedView: allow CREATE MATERIALIZED VIEW x TO y
...
This allows creation of materialized views without inner tables,
using an existing table for materialized rows instead.
This is useful for cases when you want to detach the materializing
view, but keep the already materialized data readable, especially
when the inner table is replicated.
2017-10-21 13:34:59 -07:00
Alexey Zatelepin
dd42d53856
additional help messages for extended syntax [#CLICKHOUSE-3000]
2017-10-20 23:02:55 +03:00
Alexey Zatelepin
c61d4106e8
allow loading all MergeTree settings from a query [#CLICKHOUSE-3000]
2017-10-20 21:28:45 +03:00
Alexey Zatelepin
a7c85e0c89
save index_granularity to ATTACH query if it wasn't set explicitly [#CLICKHOUSE-3000]
2017-10-20 21:28:44 +03:00
Alexey Zatelepin
1f76900500
move CREATE query rewriting for views to Interpreter [#CLICKHOUSE-2]
2017-10-20 21:28:43 +03:00
Alexey Zatelepin
75c65c7b59
use extended storage definition syntax for MergeTree engines [#CLICKHOUSE-3000]
2017-10-20 21:28:42 +03:00
Alexey Zatelepin
6b9ead4bd8
parse PARTITION BY, ORDER BY, SAMPLE BY, SETTINGS clauses in a storage definition [#CLICKHOUSE-3000]
2017-10-20 21:28:42 +03:00
Alexey Zatelepin
48dea685d0
use more typesafe AST objects in ASTCreateQuery [#CLICKHOUSE-3000]
2017-10-20 21:28:41 +03:00
Nikolai Kochetov
74a4c7508a
merged with master
2017-10-20 19:18:09 +03:00
Nikolai Kochetov
4262c00122
lock mutex before notifying waiting thread in sync insertion into distributed [#CLICKHOUSE-3379]
2017-10-19 22:52:45 +03:00
Alexey Zatelepin
e51f95872f
fix backward compatibility for new part format: [#CLICKHOUSE-3000]
...
* do not write out partition and minmax index files for old MergeTreeData format
* download the part from another replica if there are missing or extra files
2017-10-19 21:24:30 +03:00
Marek Vavruša
927c0f7b74
dbms: Partially revert previous change
...
Instead of trying to convert long/unsigned long to Poco types, just
don't use them.
2017-10-13 23:46:10 -07:00
Marek Vavruša
8a0571af24
dbms: Better fix for missing unsigned long to Poco::UInt64 alias
2017-10-13 21:39:14 -07:00
Marek Vavruša
70543200be
dbms: Fix build with clang after Poco update
...
In the current version of Poco, unsigned long no longer aliases to
UInt64 with LP64. The size_t aliases to unsigned long with clang,
so all the uses of size_t instead of UInt64 when interacting with
Poco interfaces are gone. I replaced uses with UInt64 where it makes
sense, and added an overloaded function for readVarUInt() to support size_t.
2017-10-13 18:26:24 -07:00
Marek Vavruša
2ac5a743cc
dbms: fix build with OS X built-in clang
...
The built-in clang doesn’t support value() for
`std::experimental::optional`. It however supports
dereference operator, which is basically the
same thing:
```
/clickhouse/dbms/src/DataStreams/NullableAdapterBlockInputStream.cpp:83:67: error: call to unavailable member function 'value':
res.insert({elem.column, elem.type, rename[i].value()});
~~~~~~~~~~^~~~~
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/experimental/optional:547:17: note: candidate function has been explicitly made unavailable
value_type& value()
^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/experimental/optional:539:33: note: candidate function has been explicitly made unavailable
constexpr value_type const& value() const
```
2017-10-13 18:12:29 -07:00
proller
4c7850fdc2
Optional kafka ( #1355 )
...
* Zookeeper include fix
* Update submodule contrib/poco
* Fix zookeeper submodule ho-to-make doc
* Update zookeeper submodule
* Update submodule zookeeper
* Fix compile with external zookeeper
* Optional kafka
* fix
* Fix
* fix
2017-10-14 03:53:01 +03:00
alexey-milovidov
9ccdb5420b
Merge branch 'master' into upstream-add-kafka
2017-10-13 23:52:42 +03:00
alexey-milovidov
adedf47715
Update StorageKafka.cpp
2017-10-13 23:18:43 +03:00
proller
49f8080e52
Small debud improvement
2017-10-13 21:44:26 +03:00
Amos Bird
f30841d5ab
Fix SummingMergeTree argument checking logic.
...
This patch fixes the Nested Column Name checking logic, which allows nested
columns be explicitly specified in SummingMergeTree engine.
2017-10-13 15:42:32 +03:00
Vladimir Chebotarev
1e0241e262
dbms: CompressionSettingsSelector. [#METR-21516]
2017-10-13 04:08:23 +03:00
Vladimir Chebotarev
4e313f3732
dbms: Added compression level for ZSTD. [#METR-26742]
2017-10-13 04:08:23 +03:00
Alexey Milovidov
d3d976a6fb
Misc [#CLICKHOUSE-2].
2017-10-12 23:45:55 +03:00
proller
b993667156
Try fix strange terminate ( #1329 )
...
* Tests: External: rename --use_http => --no_http and fix
* Try fix strange terminate
2017-10-12 23:42:58 +03:00
Alexey Milovidov
3a6705af0f
Improvement [#CLICKHOUSE-2].
2017-10-12 23:34:01 +03:00
Vitaliy Lyudvichenko
62ea1133dd
Disable part sendings and fetches before ALTER. [#CLICKHOUSE-3343]
2017-10-12 23:26:54 +03:00
KochetovNicolai
83925f1d5e
Fix array writing ( #1314 )
...
* changed MergedBlockOutputStream [#CLICKHOUSE-3341]
* fix build [#CLICKHOUSE-3341]
* fix build [#CLICKHOUSE-3341]
* fix build [#CLICKHOUSE-3341]
* fix build [#CLICKHOUSE-3341]
* fix build [#CLICKHOUSE-3341]
* fix build [#CLICKHOUSE-3341]
* fix build [#CLICKHOUSE-3341]
* fix build [#CLICKHOUSE-3341]
* fix build [#CLICKHOUSE-3341]
* disabled checkNoMultidimensionalArrays [#CLICKHOUSE-3341]
* fix IMergedBlockOutputStream::writeDataImpl [#CLICKHOUSE-3341]
* fix IMergedBlockOutputStream::writeDataImpl [#CLICKHOUSE-3341]
* fix IMergedBlockOutputStream::writeDataImpl [#CLICKHOUSE-3341]
* fix IMergedBlockOutputStream::writeDataImpl [#CLICKHOUSE-3341]
* added test [#CLICKHOUSE-3341]
* fixed test [#CLICKHOUSE-3341]
* refactoring and comments [#CLICKHOUSE-3341]
* fix build [#CLICKHOUSE-3341]
* Update ColumnArray.h
2017-10-12 23:13:26 +03:00
alexey-milovidov
c1285c7d6b
Update MergeTreeDataMerger.cpp
2017-10-12 22:28:24 +03:00
Vitaliy Lyudvichenko
cabf6d7799
Add better logging if OPTIMIZE cannot be executed. [#CLICKHOUSE-2]
2017-10-12 22:28:24 +03:00
Vitaliy Lyudvichenko
a65e8d9ffc
Less noisy logging. [#CLICKHOSUE-2]
2017-10-12 22:28:24 +03:00
Alexey Milovidov
d468473d62
Fixed FREEZE PARTITION: using only active data parts; acquire snapshot of parts [#CLICKHOUSE-3369].
2017-10-12 21:21:17 +03:00
Alexey Milovidov
4814eb7072
Whitespaces [#CLICKHOUSE-2].
2017-10-11 17:41:49 +03:00
Marek Vavruša
7fc08389c9
Storages: added Kafka table engine (read-only)
...
Added a new table engine backed by Apache Kafka,
it allows near real-time ingestion or ETL with
exactly-once semantics.
It doesn’t store any data on disk, so data must
be materialized into a physical table to allow
random access.
Currently it only supports reading, no insertion.
2017-10-10 08:33:00 -07:00
Marek Vavruša
4fb570fa5e
IStorage: allow notification on dependency change
...
This allows sending notifications to supported
table engines when their dependencies change.
For example, a table can be notified when a
MATERIALIZED VIEW is attached to it.
This is a building block for building pipelines.
2017-10-10 08:33:00 -07:00
proller
19042816c4
Fix compile with boost 1.65.1+ and clang 3.8 ( https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=222439 ) ( #1325 )
2017-10-06 19:26:32 +03:00
Nikolai Kochetov
61bef2adf1
added ExternalLoader [#CLICKHOUSE-3305]
2017-10-06 13:31:06 +03:00
Alexey Zatelepin
6e57272052
don't use partition value as a prefix for FREEZE PARTITION [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
f1a8b9bfa1
add comments [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
61f0b32757
forbid ALTER of partition key columns for now [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
1be62b567e
safeguards to protect against distinct partition values with the same partition_id [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
728a23f60e
allow creation of custom-partitioned tables with experimental syntax [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
a10ba0cd91
serialize partition key in Replicated tables metadata [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
909b46de0f
add MergeTreeDataFormatVersion constant [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
68cb592277
allow arbitrary partition literals in ALTER PARTITION queries [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
f0995f2741
partition serialization into id and text [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
25bb444d5a
absence of partitioning works [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
71ebf67f2e
encapsulate partition and minmax_idx load/store code [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
6906921932
load partition and minmax index from disk [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
e9f93028b8
serialize partition and minmax index values [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
2f80c150db
save MergeTree data format version and parse part names depending on it [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Nikolai Kochetov
322e56c692
fix reading with preferred_block_size_bytes = 0
2017-09-24 02:16:30 +03:00
Alexey Zatelepin
df58949a68
make exit out of the loop possible when ZooKeeper is unavailable [#CLICKHOUSE-3259]
2017-09-20 06:05:20 +03:00
Alexey Zatelepin
ba87fa395c
be more conservative when deleting replication queue items in PartCheckThread [#CLICKHOUSE-3258]
2017-09-20 06:05:20 +03:00
proller
0a0e04f1a5
Fix includes
2017-09-18 21:52:47 +03:00
Alexey Milovidov
86c46ad1bd
Unification [#CLICKHOUSE-2].
2017-09-15 15:16:12 +03:00
Vitaliy Lyudvichenko
826c354ff5
Fixed merges progress bar. [#CLICKHOUSE-2]
2017-09-15 01:04:13 +03:00
Vadim Skipin
e0a1e6982b
Context parameter added to the IDatabase methods
2017-09-11 21:29:26 +03:00
proller
5054d50307
Remove too noisy message (#CLICKHOUSE-3304)
2017-09-11 21:26:43 +03:00
Alexey Milovidov
cd540c2674
Replaced Mersenne Twister to PCG-Random [#CLICKHOUSE-2].
2017-09-10 02:17:38 +03:00
Alexey Milovidov
5a8f47a640
Fixed not an issue, shown by TSan [#CLICKHOUSE-2].
2017-09-10 02:07:32 +03:00
Alexey Milovidov
a6d5e9c7e8
Fixed not an issue, shown by TSan [#CLICKHOUSE-2].
2017-09-10 02:04:22 +03:00
Alexey Milovidov
94b975e7f9
Removed function deprecated in C++ [#CLICKHOUSE-3174].
2017-09-09 02:31:18 +03:00
Alexey Milovidov
ed1c0820f0
Fixed minor issue found by Coverity [#CLICKHOUSE-3285].
2017-09-09 02:25:42 +03:00
Alexey Milovidov
a81028377e
Implemented some suggestions from clang-tidy (part 9) [#CLICKHOUSE-3301].
2017-09-08 08:03:21 +03:00
Alexey Milovidov
dc4976f562
Implemented some suggestions from clang-tidy (part 8) [#CLICKHOUSE-3301].
2017-09-08 07:58:57 +03:00
Alexey Milovidov
d29c77adea
Implemented some suggestions from clang-tidy (part 3) [#CLICKHOUSE-3301].
2017-09-08 06:47:27 +03:00
Alexey Milovidov
6307661c35
Addition to prev. revision [#CLICKHOUSE-3300].
2017-09-08 00:26:25 +03:00
Alexey Milovidov
8b313ab99e
Fixed warnings found by cppcheck [#CLICKHOUSE-3300].
2017-09-08 00:04:48 +03:00
Alexey Milovidov
0dcee75180
Supporessed useless false positive errors found by cppcheck [#CLICKHOUSE-3300].
2017-09-07 22:13:37 +03:00
Alexey Milovidov
d82b87aaef
Fixed redundand code in better way [#CLICKHOUSE-3281].
2017-09-06 06:24:39 +03:00
Vitaliy Lyudvichenko
d55d5eca46
Remove extra Context reference and add required ones. Part 2. [#CLICKHOUSE-3117]
2017-09-06 04:18:07 +03:00
