Commit Graph

9795 Commits

Author SHA1 Message Date
Marek Vavruša
9c57def8df StorageKafka: update comments 2017-12-21 00:46:50 +03:00
Marek Vavruša
c6e959d1e9 StorageKafka: extended configuration, parallel consumers, offset tracking
This contains many fixes and corrections for the Kafka engine.
Most notably it now supports extended configuration similarly to GraphiteMergeTree.
Now it also allows specification of consumer count to parallelize consumption of
multiple partitions both in materialized views and in SELECT queries.

The offsets are now committed in the insertSuffix() method after all rows
are successfully read. If an exception is thrown during reading, affected consumer
unsubscribes from all assignments and rejoins the consumer group to rewind offsets.
This means that the consumer won't lose messages in case of write failures.
2017-12-21 00:46:50 +03:00
Marek Vavruša
6d4f106f11 StorageMaterializedView: update dependencies on DETACH TABLE
Previously the dependencies were updated only on DROP TABLE, so
detaching a materialized view and inserting to source table thrown an exception.
2017-12-21 00:46:50 +03:00
Marek Vavruša
fb1aa84973 Fixed clang build on macOS
- Make sure to use dynamic linking on macOS to avoid OpenSSL static linking bug
- Use -std=c++17 in CFLAGS for files in contrib
- Avoid bad support for thread_local on macOS with clang altogether
2017-12-21 00:46:50 +03:00
Amos Bird
e7af27c4e0 Better iostream debugger with adl.
This patch fixes some issues related to DUMP() debugging. Opening std and
placing all Core related classed inside DB namespace enable proper adl support.
It also adds better sfinae guards to Field constructor which somehow breaks <<
overloads.
2017-12-20 21:44:42 +03:00
Alexey Zatelepin
c422b5b2e4 take PreCommitted parts into account when calculating total merge size [#CLICKHOUSE-3508] 2017-12-20 20:52:28 +03:00
Alexey Zatelepin
835b957e95 add comments [#CLICKHOUSE-3508] 2017-12-20 20:40:36 +03:00
Alexey Zatelepin
d35baafde8 check PreCommitted parts when executing log entries and in PartChecker [#CLICKHOUSE-3508] 2017-12-20 20:04:14 +03:00
Alexey Zatelepin
2340e55d0e revert to old behaviour of getPartIfExists() [#CLICKHOUSE-3508] 2017-12-20 20:04:14 +03:00
Silviu Caragea
d60dfc9f0a Fix potential memory leak 2017-12-20 12:46:29 +02:00
Kirill Shvakov
d40d0657e0 Merge branch 'master' of https://github.com/yandex/ClickHouse 2017-12-20 10:21:25 +02:00
Kirill Shvakov
59aa1359c8 Access to TEMPORARY TABLES for read-only users 2017-12-20 09:52:06 +02:00
Alexey Milovidov
3124ffea67 Fixed error in prev. revision [#CLICKHOUSE-2]. 2017-12-20 02:14:37 +03:00
Alexey Milovidov
6c8a0e70fd Additions after merge [#CLICKHOUSE-2]. 2017-12-19 23:09:15 +03:00
alexey-milovidov
7a97d08b04
Merge branch 'master' into CLICKHOUSE-3361 2017-12-19 23:03:17 +03:00
Alexey Milovidov
c8d8016bbb Fixed bad code [#CLICKHOUSE-2]. 2017-12-19 22:56:48 +03:00
Nikolai Kochetov
39ec3f7d74 changed exception message, added comments 2017-12-19 22:51:59 +03:00
Alexey Milovidov
b48d934c3d Merge branch 'master' of github.com:yandex/ClickHouse 2017-12-19 22:46:11 +03:00
Alexey Milovidov
548cb4e658 Fixed bad code in Client.cpp [#CLICKHOUSE-2]. 2017-12-19 22:46:02 +03:00
Silviu Caragea
b08463ed7b Fix for https://github.com/yandex/ClickHouse/issues/1648 2017-12-19 22:41:05 +03:00
Alexey Milovidov
0d1a285b23 Removed ignoring of EPIPE in WriteBufferFromFileDescriptor [#CLICKHOUSE-2]. 2017-12-19 21:04:14 +03:00
Alexey Milovidov
2ac61adfec Merge branch 'master' into ks1322-master 2017-12-19 20:49:02 +03:00
Alexey Milovidov
1029b2a5ef Fixed warning in SpecializedAggregator.h [#CLICKHOUSE-2]. 2017-12-19 20:48:08 +03:00
alexey-milovidov
34d54ea349 Update CMakeLists.txt 2017-12-19 20:42:35 +03:00
proller
631f307b9f Copy more platform-specific headers 2017-12-19 20:42:35 +03:00
Alexey Milovidov
1a6f38b27f Merge branch 'master' into ks1322-master 2017-12-19 20:38:29 +03:00
Alexey Milovidov
e25514d28a Addition to the fix of generic variant of topK aggregate function [#CLICKHOUSE-2]. 2017-12-19 20:21:47 +03:00
Alexey Milovidov
089e78dc16 Clarification [#CLICKHOUSE-2]. 2017-12-19 04:57:06 +03:00
Alexey Milovidov
aba63cc76f Better semantic of sharing columns: fixed error [#CLICKHOUSE-2]. 2017-12-19 04:53:54 +03:00
Alexey Milovidov
50d09f08f5 Updated test [#CLICKHOUSE-2]. 2017-12-19 04:44:05 +03:00
Alexey Milovidov
80e48cc899 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-19 04:42:24 +03:00
Alexey Milovidov
4e8a70a29f Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-19 04:13:20 +03:00
Alexey Milovidov
d9732198ac Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-19 03:51:12 +03:00
Alexey Milovidov
61c2b55a57 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-19 03:43:04 +03:00
Alexey Milovidov
02821f4861 Fixed error [#CLICKHOUSE-2]. 2017-12-19 03:26:16 +03:00
Alexey Milovidov
93a300bea4 Miscellaneous [#CLICKHOUSE-2]. 2017-12-19 00:49:34 +03:00
Alexey Milovidov
521b3aee84 Fixed error found by M.Filimonov (equality of array sizes are not checked in n-ary version of aggregate functions with -Array combinator) [#CLICKHOUSE-2]. 2017-12-19 00:44:18 +03:00
Alexey Milovidov
6c28c72fdc Added test [#CLICKHOUSE-2]. 2017-12-19 00:16:20 +03:00
Alexey Milovidov
b83142ea62 Fixed error found by M.Filimonov (generic implementation of topK aggregate function was totally wrong) [#CLICKHOUSE-2]. 2017-12-19 00:12:15 +03:00
Alexey Milovidov
339e22d014 Fixed error found by M.Filimonov [#CLICKHOUSE-2]. 2017-12-19 00:04:19 +03:00
Alexey Milovidov
d84ef5d8e2 Merge branch 'master' of github.com:yandex/ClickHouse 2017-12-18 23:43:19 +03:00
Alexey Milovidov
09fa34c764 Fixed error found by M.Filimonov (missing check for maximum size of FixedString leading to overflow in arithmetic) [#CLICKHOUSE-2]. 2017-12-18 23:43:10 +03:00
proller
ab5cbcc627 Try fix internal compiler [tests run] (#1651)
* Try fic interal compiler

* Debian: depend on tzdata package

* fix order

* Try fix

* fix

* try

* fix

* fix

* fix

* fix
2017-12-18 23:30:19 +03:00
proller
bee53d483f Allow run tests without extract-from-config 2017-12-18 15:26:34 +03:00
Alexey Milovidov
bc160f5e4d Fixed error [#CLICKHOUSE-2]. 2017-12-18 09:49:38 +03:00
Alexey Milovidov
84016cc96f Fixed error [#CLICKHOUSE-2]. 2017-12-18 09:39:07 +03:00
alexey-milovidov
adfc743e06
Merge pull request #1658 from yandex/cow_columns
Better semantic of sharing columns.
2017-12-18 09:26:27 +03:00
Alexey Milovidov
32fcc7b3f0 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-18 09:23:19 +03:00
Alexey Milovidov
27301ac380 Updated test [#CLICKHOUSE-2]. 2017-12-18 08:43:41 +03:00
Alexey Milovidov
816169a3be Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-18 08:38:42 +03:00
Alexey Milovidov
11d3b9c710 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-18 08:37:20 +03:00
Alexey Milovidov
f9170190c6 Miscellaneous [#CLICKHOUSE-2]. 2017-12-18 07:17:56 +03:00
Alexey Milovidov
c5d5296889 Miscellaneous [#CLICKHOUSE-2]. 2017-12-18 07:07:26 +03:00
Alexey Milovidov
88384bad05 Removed duplicate test [#CLICKHOUSE-2]. 2017-12-18 05:46:34 +03:00
Alexey Milovidov
280b88454b Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-18 05:43:40 +03:00
Alexey Milovidov
7c802ee3de Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-18 05:37:08 +03:00
Alexey Milovidov
f95ddfce10 Make DataTypes totally immutable [#CLICKHOUSE-2]. 2017-12-18 04:11:48 +03:00
ks1322
2fd62ccd9f added expect test 2017-12-17 23:46:52 +03:00
Alexey Milovidov
32bc1df714 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-17 13:51:19 +03:00
Alexey Milovidov
6bad555b9d Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-17 13:39:22 +03:00
Alexey Milovidov
f501f6990e Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-17 13:20:48 +03:00
Alexey Milovidov
6f7d2b99ea Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-17 13:14:12 +03:00
Alexey Milovidov
75c0ad10ab Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-17 12:01:12 +03:00
Alexey Milovidov
c0670435ab Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-17 11:44:06 +03:00
Alexey Milovidov
8484f086a5 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-17 10:59:14 +03:00
Alexey Milovidov
6ad5be5eee Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-17 10:15:13 +03:00
Alexey Milovidov
d497d010d7 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-17 08:21:04 +03:00
ks1322
62a2932299 ignore EPIPE write error for custom pagers in client 2017-12-16 16:06:30 +03:00
ks1322
2a42fe8f3f ignore SIGPIPE signal for custom pagers 2017-12-16 16:06:30 +03:00
Alexey Milovidov
5f4a536f75 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-16 09:33:54 +03:00
Alexey Milovidov
62f7c25522 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-16 09:19:21 +03:00
Alexey Milovidov
6fe31d257d Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-16 09:10:05 +03:00
Alexey Milovidov
342f0e431a Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-16 09:08:33 +03:00
Alexey Milovidov
7d69e041f5 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-16 09:02:54 +03:00
Alexey Milovidov
278f8b9fcb Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-16 08:52:47 +03:00
Alexey Milovidov
ddfb1fab9e Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-16 08:46:46 +03:00
Alexey Milovidov
bce801a663 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-16 08:34:07 +03:00
Alexey Milovidov
14b0397500 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-16 08:21:04 +03:00
Alexey Milovidov
45805a367f Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-16 07:59:32 +03:00
Alexey Milovidov
5b32914089 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-16 07:29:34 +03:00
Alexey Milovidov
3f0c4bf61c Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-16 05:34:02 +03:00
Alexey Milovidov
30639dea0f Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-16 05:24:02 +03:00
Alexey Milovidov
3e3042f01a Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-16 04:21:59 +03:00
Alexey Milovidov
8302769563 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-16 03:49:03 +03:00
Alexey Milovidov
645f620873 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-16 00:46:56 +03:00
Alexey Milovidov
afec433d6f Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-16 00:33:05 +03:00
Alexey Milovidov
7284a64740 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-16 00:32:25 +03:00
Alexey Milovidov
ecd56de6a8 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-16 00:11:24 +03:00
Alexey Milovidov
c4bbc1c963 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 23:48:46 +03:00
Alexey Milovidov
8ce60e3af0 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 22:53:11 +03:00
Alexey Milovidov
0b76b61c73 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 22:46:24 +03:00
proller
e7f82b6a8c
Allow run test without package install and on custom ports. (#1643)
* ExtractFromConfig --try

* Add symlinks to package

* clickhouse-test --order option
2017-12-15 22:34:19 +03:00
Alexey Milovidov
9c9896c1a9 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 22:17:15 +03:00
Alexey Milovidov
d0ca8b2533 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 21:23:05 +03:00
proller
254cced2cd Add internal compiler include dir (to fix missing stddef.h) 2017-12-15 18:43:47 +03:00
Alexey Milovidov
b2aff7521b Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 06:47:43 +03:00
Alexey Milovidov
7e685c383b Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 06:19:14 +03:00
Alexey Milovidov
e3d931d7fc Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 06:12:04 +03:00
Alexey Milovidov
96ee187d1a Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 06:04:33 +03:00
Alexey Milovidov
a086e3e6d8 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 05:52:38 +03:00
Alexey Milovidov
401fd2d6d8 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 05:36:40 +03:00
Alexey Milovidov
a92055c768 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 05:04:28 +03:00
Alexey Milovidov
6756b37925 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 04:34:30 +03:00
Alexey Milovidov
4148c6ce81 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 03:06:56 +03:00
Alexey Milovidov
8926af2ced Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 03:01:59 +03:00
Alexey Milovidov
c2985857f8 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 00:51:30 +03:00
Alexey Milovidov
01a0b91c83 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-15 00:20:14 +03:00
Alexey Milovidov
d164024e3d Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-14 23:58:18 +03:00
Alexey Milovidov
037d304afa Merge remote-tracking branch 'origin/master' into cow_columns 2017-12-14 22:26:16 +03:00
proller
7b47e102c9 Fix internal compiler option 2017-12-14 21:42:48 +03:00
Alexey Milovidov
54476f3c1e Removed note about experimental state of ZSTD [#CLICKHOUSE-2]. 2017-12-14 20:42:27 +03:00
Nikolai Kochetov
08fed58a25 use columns from left block out of columns with same name at Join [#CLICKHOUSE-3361] 2017-12-14 18:28:30 +03:00
Silviu Caragea
a1104644d9 Add tests 2017-12-14 11:02:12 +02:00
Alexey Milovidov
744c2d3d02 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-14 07:30:01 +03:00
Alexey Milovidov
911d80ab51 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-14 07:25:22 +03:00
Alexey Milovidov
fb38ef9a8e Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-14 06:56:56 +03:00
Alexey Milovidov
bed2d644e6 Merge remote-tracking branch 'origin/master' into cow_columns 2017-12-14 04:44:35 +03:00
Alexey Milovidov
f3bb24a3bf Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-14 04:43:19 +03:00
Silviu Caragea
10156f1f56 Fix for topK and groupUniqArray over Nullable columns 2017-12-13 23:48:35 +02:00
Alexey Milovidov
aea1d0ce4a Fixed error [#CLICKHOUSE-2]. 2017-12-14 00:26:17 +03:00
Alexey Milovidov
1172e6559d Reverted Client.cpp [#CLICKHOUSE-2]. 2017-12-14 00:12:21 +03:00
Alexey Milovidov
cfedf4d5b1 Merge branch 'fix14' of https://github.com/proller/ClickHouse into proller-fix14 2017-12-14 00:10:37 +03:00
Alexey Milovidov
9cfd3da183 Merge branch 'master' of github.com:yandex/ClickHouse 2017-12-14 00:09:41 +03:00
Alexey Milovidov
0dd0d76b19 Added changes from proller [#CLICKHOUSE-2]. 2017-12-14 00:09:31 +03:00
alexey-milovidov
58cb0ab57d
Merge pull request #1602 from kshvakov/master
client #1600: add network-bandwidth option
2017-12-13 23:54:03 +03:00
alexey-milovidov
7908818ac1
Update Client.cpp 2017-12-13 23:53:53 +03:00
alexey-milovidov
1612aab3c5
Merge pull request #1636 from bocharov/master
Fix mark cache settings initialization.
2017-12-13 23:45:45 +03:00
Alexey Milovidov
b3277408c4 Added fix from proller [#CLICKHOUSE-2]. 2017-12-13 23:43:51 +03:00
Alexey Milovidov
4d40879135 Proper fix for external dictionaries with complex keys [#CLICKHOUSE-2]. 2017-12-13 23:40:03 +03:00
Alex Bocharov
3a320733f3 Fix mark cache settings initialization.
Before this change setting mark_cache_min_lifetime wasn't applied from the user settings, but only from the default.
There reason was that corresponding Settings initialization was after creation of the MarkCache.
This fix moves Settings initialization above creation of the MarkCache.
2017-12-13 20:32:26 +00:00
Alexey Milovidov
9cc424b4b5 Fixed error: loading "preprocessed" files if dictionary/model config was specified with glob [#CLICKHOUSE-3501]. 2017-12-13 23:21:03 +03:00
Alexey Milovidov
f68cf599e6 Removed unused method [#CLICKHOUSE-2] 2017-12-13 22:55:16 +03:00
Alexey Milovidov
0b8688e89b Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-13 22:14:12 +03:00
proller
4f83806a9e Merge remote-tracking branch 'upstream/master' into fix14 2017-12-13 22:08:53 +03:00
proller
741117a390
Fix freebsd build. Little cmake fixes. (#1634) 2017-12-13 22:07:12 +03:00
proller
5453a41208 fix 2017-12-13 19:41:29 +03:00
proller
b911b03eed Print error 2017-12-13 19:40:25 +03:00
proller
4076e14b6b Merge remote-tracking branch 'upstream/master' into fix14 2017-12-13 16:10:02 +03:00
robot-metrika-test
cbb02b580c Auto version update to [54326] 2017-12-13 12:55:19 +03:00
Alexey Milovidov
819287b697 Better semantic of sharing columns: development [#CLICKHOUSE-2]. 2017-12-13 04:27:53 +03:00
alexey-milovidov
62fb610e4e
Merge pull request #1447 from yandex/support-table-functions-for-desc-and-insert
Support table functions for desc and insert
2017-12-11 23:28:10 +03:00
proller
cbea7bdeb7 Add options to internal compiler: -nostdinc -nostdinc++ -Bprefix= --isysroot= 2017-12-11 23:27:17 +03:00
Nikolai Kochetov
cee84e1120 fix build [#CLICKHOUSE-3397] 2017-12-11 21:40:28 +03:00
Nikolai Kochetov
6c8e5594f9 fix build [#CLICKHOUSE-3397] 2017-12-11 21:38:43 +03:00
Alexey Milovidov
7d04e67262 Merge branch 'write-buffer-from-arena' 2017-12-11 21:25:03 +03:00
Alexey Milovidov
90848adc6b Fixed translation [#CLICKHOUSE-2]. 2017-12-11 21:19:35 +03:00
Nikolai Kochetov
f71b8bf58f merged with master 2017-12-11 21:13:24 +03:00
Nikolai Kochetov
68ccadfc6c removed owned_cluster_for_insertion from StorageDistributed [#CLICKHOUSE-3397] 2017-12-11 21:09:20 +03:00
Alexey Milovidov
3b65d95525 Translated comments and by the way made code better [#CLICKHOUSE-2]. 2017-12-11 21:05:16 +03:00
Nikolai Kochetov
1ba891f3fc removed owned_cluster_for_insertion from StorageDistributed [#CLICKHOUSE-3397] 2017-12-11 20:29:24 +03:00