Alexey Zatelepin
141b85189e
always allow attaching tables with extended storage definition syntax [#CLICKHOUSE-3000]
2017-10-30 16:44:43 +03:00
proller
8b3d3cf56f
Merge remote-tracking branch 'upstream/master' into fix4
2017-10-30 16:19:19 +03:00
Vitaliy Lyudvichenko
4a55e5f94b
Hotfix of CREATE MATERIALIZED VIEW a TO b. [#CLICKHOUSE-2]
2017-10-30 15:14:36 +03:00
proller
b7dd5cd0ba
Merge remote-tracking branch 'upstream/master' into fix4
2017-10-30 13:42:34 +03:00
proller
956e1513c5
Merge remote-tracking branch 'upstream/master' into fix4
2017-10-30 13:22:20 +03:00
Alexey Milovidov
e36906cdf5
Fixed build [#CLICKHOUSE-2].
2017-10-28 06:59:42 +03:00
proller
ba49157f84
Merge remote-tracking branch 'upstream/master' into fix4
2017-10-28 01:06:09 +03:00
Alexey Milovidov
9e3daca437
Fixed build [#CLICKHOUSE-2].
2017-10-28 00:36:43 +03:00
Alexey Milovidov
b2892e058a
Fixed bad file name [#CLICKHOUSE-2].
2017-10-28 00:31:10 +03:00
Alexey Milovidov
5ab78c249d
Fixed build after merge [#CLICKHOUSE-2].
2017-10-28 00:18:06 +03:00
alexey-milovidov
9e200bdfd9
Merge branch 'master' into materialized-view-to
2017-10-27 23:21:27 +03:00
proller
c878af8740
Merge remote-tracking branch 'upstream/master' into fix4
2017-10-27 23:13:35 +03:00
alexey-milovidov
3b99b723d7
Merge pull request #1245 from yandex/CLICKHOUSE-3178
...
Atomic parts addition and removing
2017-10-27 23:03:33 +03:00
alexey-milovidov
b4bb6796c1
Merge pull request #1407 from yandex/CLICKHOUSE-3381
...
Ability to turn off deduplication, smart deduplication for MV
2017-10-27 23:03:25 +03:00
proller
07009d5072
Bughunt tsan
2017-10-27 18:34:26 +03:00
proller
1736d7d312
Allow build with system lib rdkafka
2017-10-27 17:40:41 +03:00
Alexey Milovidov
41c05f22be
Removed very old obsolete tests [#CLICKHOUSE-2].
2017-10-26 21:49:48 +03:00
Alexey Milovidov
6022201c26
Removed very old obsolete test [#CLICKHOUSE-2].
2017-10-26 21:38:40 +03:00
Alexey Milovidov
a27981b4ef
Get rid of double whitespaces [#CLICKHOUSE-2].
2017-10-26 21:36:23 +03:00
Vitaliy Lyudvichenko
51265506f8
Convey session setting up to inner table creation for MV. [#CLICKHOUSE-3381]
2017-10-26 18:39:56 +03:00
Vitaliy Lyudvichenko
a0d06c5ee0
Don't deduplicate a block inserted from MV if the source block is Ok. [#CLICKHOUSE-3381]
2017-10-26 17:58:00 +03:00
Nikolai Kochetov
371e234f01
changed ExternalLoader::getObjectsMap return type [#CLICKHOUSE-3305]
2017-10-26 17:26:03 +03:00
Vitaliy Lyudvichenko
602560cb2e
Update system.parts table. [#CLICKHOUSE-3178]
2017-10-26 17:20:21 +03:00
Vitaliy Lyudvichenko
483dfd1b44
Fixed "No part error" in replicated sender. [#CLICKHOUSE-3178]
2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
397b8bcf92
Smarter parts deletion. [#CLICKHOUSE-3178]
2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
d6ea9c3dcf
Fixed awful copypaste. [#CLICKHOUSE-3178]
2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
b5c7eaa80e
Force data parts cleanup before repeat fetch. [#CLICKHOUSE-3178]
2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
71833a7f0f
More correct procedure of parts deleting. [#CLICKHOUSE-3178]
2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
db3a67a421
Add clearer RangeFiltered implementation. [#CLICKHOUSE-3178]
2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
e2a12d1088
Introduced logic for MergeTree parts states. [#CLICKHOUSE-3178]
2017-10-26 17:16:06 +03:00
Vitaliy Lyudvichenko
5787e8b257
Add state for MergeTree parts. [#CLICKHOUSE-3178]
...
And Removed obsolete code.
2017-10-26 17:16:06 +03:00
Alexey Zatelepin
ec963e3397
save evaluated storage parameters in ATTACH queries [#CLICKHOUSE-3000]
2017-10-26 17:07:15 +03:00
proller
9ccba5fad1
Fix includes
2017-10-26 14:46:57 +03:00
Alexey Zatelepin
6ada6d9aa5
remove dead code [#CLICKHOUSE-2]
2017-10-26 07:53:52 +03:00
Alexey Zatelepin
01c39dcf1c
fix CREATE TABLE bugs and idiosyncrasies: [#CLICKHOUSE-3000]
...
* fix segfault on ATTACH MATERIALIZED VIEW
* allow only TEMPORARY TABLEs (not databases or views)
* forbid choosing ENGINE for TEMPORARY tables
* fix segfault on CREATE TABLE t1 AS t2 where t2 is a View
2017-10-26 07:53:52 +03:00
Alexey Milovidov
65982a1642
Fixed exception messages [#CLICKHOUSE-2].
2017-10-25 22:17:37 +03:00
proller
4ffbca2d8c
Fix warnings ( #1406 )
...
* Log query id in executeQuery; Better type mismatch error; change format in report tool
* Better log query_id
* fix message
* Use c++11 thread_local instaed of gcc's __thread
* lock mutex before notifying waiting thread in sync insertion into distributed [#CLICKHOUSE-3379]
* Cmake: fix build without downloaded submodules (#1379 )
* fix
* ZooKeeper: fixed stack smashing with tryGet()
The tryGet() operation creates a 1MB buffer on stack. This may or
may not work depending on the default stack size for threads,
whether the stack protector is enabled or not, recursion depth,
and the actual value size.
This is probably going to slow down some ZK operations, but I don't
see how else this could work reliably with the existing API.
* increased timeout for test_insertion_sync_fails_with_timeout
* Update CHANGELOG_RU.md
* Update ZooKeeper.cpp
* Fix warnings
* Fixes
* Dont strip debug info from asan, tsan and other builds except releases
* Fix asan error causd by test 00144
* Fix empty log message (#CLICKHOUSE-3378)
2017-10-25 21:39:10 +03:00
alexey-milovidov
b5155916a8
Merge branch 'master' into materialized-view-to
2017-10-24 22:39:24 +03:00
Alexey Zatelepin
6bb34a4494
Merge branch 'master' into arbitrary-partitioning
...
Conflicts:
dbms/src/Storages/StorageFactory.cpp
2017-10-24 17:23:18 +03:00
Alexey Zatelepin
b6b5d1cf25
store rows count for part on disk to allow tables without fixed size columns [#CLICKHOUSE-3000]
2017-10-24 17:11:53 +03:00
alexey-milovidov
3c2155df0b
Update StorageKafka.cpp
2017-10-23 20:47:33 +03:00
alexey-milovidov
139e2080ae
Update StorageKafka.cpp
2017-10-23 20:47:33 +03:00
Marek Vavruša
3926851343
StorageKafka: use config string instead of just server name
2017-10-23 20:47:33 +03:00
Marek Vavruša
a7c391fbf7
StorageKafka: use std::vector instead of stack array for error string buffers
2017-10-23 20:47:33 +03:00
Marek Vavruša
f4c1149606
Settings: renamed schema to format_schema
2017-10-23 20:47:33 +03:00
Alexey Milovidov
c57a62611b
Removed StorageTrivialBuffer because it is underdeveloped [#CLICKHOUSE-2].
2017-10-23 20:38:56 +03:00
proller
c66228a746
Log query id in executeQuery; Better type mismatch error; change format in report tool ( #1372 )
...
* Log query id in executeQuery; Better type mismatch error; change format in report tool
* Better log query_id
* fix message
* Use c++11 thread_local instaed of gcc's __thread
* lock mutex before notifying waiting thread in sync insertion into distributed [#CLICKHOUSE-3379]
* Cmake: fix build without downloaded submodules (#1379 )
* fix
2017-10-23 20:32:39 +03:00
Nikolai Kochetov
5966574405
fix timeout lock in DistributedBlockOutputStream
2017-10-23 16:47:00 +03:00
Nikolai Kochetov
3317251e2f
fix jobs flags in DistributedBlockOutputStream
2017-10-23 15:00:56 +03:00
Marek Vavruša
975a7ada42
StorageMaterializedView: allow CREATE MATERIALIZED VIEW x TO y
...
This allows creation of materialized views without inner tables,
using an existing table for materialized rows instead.
This is useful for cases when you want to detach the materializing
view, but keep the already materialized data readable, especially
when the inner table is replicated.
2017-10-21 13:34:59 -07:00
Alexey Zatelepin
dd42d53856
additional help messages for extended syntax [#CLICKHOUSE-3000]
2017-10-20 23:02:55 +03:00
Alexey Zatelepin
c61d4106e8
allow loading all MergeTree settings from a query [#CLICKHOUSE-3000]
2017-10-20 21:28:45 +03:00
Alexey Zatelepin
a7c85e0c89
save index_granularity to ATTACH query if it wasn't set explicitly [#CLICKHOUSE-3000]
2017-10-20 21:28:44 +03:00
Alexey Zatelepin
1f76900500
move CREATE query rewriting for views to Interpreter [#CLICKHOUSE-2]
2017-10-20 21:28:43 +03:00
Alexey Zatelepin
75c65c7b59
use extended storage definition syntax for MergeTree engines [#CLICKHOUSE-3000]
2017-10-20 21:28:42 +03:00
Alexey Zatelepin
6b9ead4bd8
parse PARTITION BY, ORDER BY, SAMPLE BY, SETTINGS clauses in a storage definition [#CLICKHOUSE-3000]
2017-10-20 21:28:42 +03:00
Alexey Zatelepin
48dea685d0
use more typesafe AST objects in ASTCreateQuery [#CLICKHOUSE-3000]
2017-10-20 21:28:41 +03:00
Nikolai Kochetov
74a4c7508a
merged with master
2017-10-20 19:18:09 +03:00
Nikolai Kochetov
4262c00122
lock mutex before notifying waiting thread in sync insertion into distributed [#CLICKHOUSE-3379]
2017-10-19 22:52:45 +03:00
Alexey Zatelepin
e51f95872f
fix backward compatibility for new part format: [#CLICKHOUSE-3000]
...
* do not write out partition and minmax index files for old MergeTreeData format
* download the part from another replica if there are missing or extra files
2017-10-19 21:24:30 +03:00
Marek Vavruša
927c0f7b74
dbms: Partially revert previous change
...
Instead of trying to convert long/unsigned long to Poco types, just
don't use them.
2017-10-13 23:46:10 -07:00
Marek Vavruša
8a0571af24
dbms: Better fix for missing unsigned long to Poco::UInt64 alias
2017-10-13 21:39:14 -07:00
Marek Vavruša
70543200be
dbms: Fix build with clang after Poco update
...
In the current version of Poco, unsigned long no longer aliases to
UInt64 with LP64. The size_t aliases to unsigned long with clang,
so all the uses of size_t instead of UInt64 when interacting with
Poco interfaces are gone. I replaced uses with UInt64 where it makes
sense, and added an overloaded function for readVarUInt() to support size_t.
2017-10-13 18:26:24 -07:00
Marek Vavruša
2ac5a743cc
dbms: fix build with OS X built-in clang
...
The built-in clang doesn’t support value() for
`std::experimental::optional`. It however supports
dereference operator, which is basically the
same thing:
```
/clickhouse/dbms/src/DataStreams/NullableAdapterBlockInputStream.cpp:83:67: error: call to unavailable member function 'value':
res.insert({elem.column, elem.type, rename[i].value()});
~~~~~~~~~~^~~~~
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/experimental/optional:547:17: note: candidate function has been explicitly made unavailable
value_type& value()
^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/experimental/optional:539:33: note: candidate function has been explicitly made unavailable
constexpr value_type const& value() const
```
2017-10-13 18:12:29 -07:00
proller
4c7850fdc2
Optional kafka ( #1355 )
...
* Zookeeper include fix
* Update submodule contrib/poco
* Fix zookeeper submodule ho-to-make doc
* Update zookeeper submodule
* Update submodule zookeeper
* Fix compile with external zookeeper
* Optional kafka
* fix
* Fix
* fix
2017-10-14 03:53:01 +03:00
alexey-milovidov
9ccdb5420b
Merge branch 'master' into upstream-add-kafka
2017-10-13 23:52:42 +03:00
alexey-milovidov
adedf47715
Update StorageKafka.cpp
2017-10-13 23:18:43 +03:00
proller
49f8080e52
Small debud improvement
2017-10-13 21:44:26 +03:00
Amos Bird
f30841d5ab
Fix SummingMergeTree argument checking logic.
...
This patch fixes the Nested Column Name checking logic, which allows nested
columns be explicitly specified in SummingMergeTree engine.
2017-10-13 15:42:32 +03:00
Vladimir Chebotarev
1e0241e262
dbms: CompressionSettingsSelector. [#METR-21516]
2017-10-13 04:08:23 +03:00
Vladimir Chebotarev
4e313f3732
dbms: Added compression level for ZSTD. [#METR-26742]
2017-10-13 04:08:23 +03:00
Alexey Milovidov
d3d976a6fb
Misc [#CLICKHOUSE-2].
2017-10-12 23:45:55 +03:00
proller
b993667156
Try fix strange terminate ( #1329 )
...
* Tests: External: rename --use_http => --no_http and fix
* Try fix strange terminate
2017-10-12 23:42:58 +03:00
Alexey Milovidov
3a6705af0f
Improvement [#CLICKHOUSE-2].
2017-10-12 23:34:01 +03:00
Vitaliy Lyudvichenko
62ea1133dd
Disable part sendings and fetches before ALTER. [#CLICKHOUSE-3343]
2017-10-12 23:26:54 +03:00
KochetovNicolai
83925f1d5e
Fix array writing ( #1314 )
...
* changed MergedBlockOutputStream [#CLICKHOUSE-3341]
* fix build [#CLICKHOUSE-3341]
* fix build [#CLICKHOUSE-3341]
* fix build [#CLICKHOUSE-3341]
* fix build [#CLICKHOUSE-3341]
* fix build [#CLICKHOUSE-3341]
* fix build [#CLICKHOUSE-3341]
* fix build [#CLICKHOUSE-3341]
* fix build [#CLICKHOUSE-3341]
* fix build [#CLICKHOUSE-3341]
* disabled checkNoMultidimensionalArrays [#CLICKHOUSE-3341]
* fix IMergedBlockOutputStream::writeDataImpl [#CLICKHOUSE-3341]
* fix IMergedBlockOutputStream::writeDataImpl [#CLICKHOUSE-3341]
* fix IMergedBlockOutputStream::writeDataImpl [#CLICKHOUSE-3341]
* fix IMergedBlockOutputStream::writeDataImpl [#CLICKHOUSE-3341]
* added test [#CLICKHOUSE-3341]
* fixed test [#CLICKHOUSE-3341]
* refactoring and comments [#CLICKHOUSE-3341]
* fix build [#CLICKHOUSE-3341]
* Update ColumnArray.h
2017-10-12 23:13:26 +03:00
alexey-milovidov
c1285c7d6b
Update MergeTreeDataMerger.cpp
2017-10-12 22:28:24 +03:00
Vitaliy Lyudvichenko
cabf6d7799
Add better logging if OPTIMIZE cannot be executed. [#CLICKHOUSE-2]
2017-10-12 22:28:24 +03:00
Vitaliy Lyudvichenko
a65e8d9ffc
Less noisy logging. [#CLICKHOSUE-2]
2017-10-12 22:28:24 +03:00
Alexey Milovidov
d468473d62
Fixed FREEZE PARTITION: using only active data parts; acquire snapshot of parts [#CLICKHOUSE-3369].
2017-10-12 21:21:17 +03:00
Alexey Milovidov
4814eb7072
Whitespaces [#CLICKHOUSE-2].
2017-10-11 17:41:49 +03:00
Marek Vavruša
7fc08389c9
Storages: added Kafka table engine (read-only)
...
Added a new table engine backed by Apache Kafka,
it allows near real-time ingestion or ETL with
exactly-once semantics.
It doesn’t store any data on disk, so data must
be materialized into a physical table to allow
random access.
Currently it only supports reading, no insertion.
2017-10-10 08:33:00 -07:00
Marek Vavruša
4fb570fa5e
IStorage: allow notification on dependency change
...
This allows sending notifications to supported
table engines when their dependencies change.
For example, a table can be notified when a
MATERIALIZED VIEW is attached to it.
This is a building block for building pipelines.
2017-10-10 08:33:00 -07:00
proller
19042816c4
Fix compile with boost 1.65.1+ and clang 3.8 ( https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=222439 ) ( #1325 )
2017-10-06 19:26:32 +03:00
Nikolai Kochetov
61bef2adf1
added ExternalLoader [#CLICKHOUSE-3305]
2017-10-06 13:31:06 +03:00
Alexey Zatelepin
6e57272052
don't use partition value as a prefix for FREEZE PARTITION [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
f1a8b9bfa1
add comments [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
61f0b32757
forbid ALTER of partition key columns for now [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
1be62b567e
safeguards to protect against distinct partition values with the same partition_id [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
728a23f60e
allow creation of custom-partitioned tables with experimental syntax [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
a10ba0cd91
serialize partition key in Replicated tables metadata [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
909b46de0f
add MergeTreeDataFormatVersion constant [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
68cb592277
allow arbitrary partition literals in ALTER PARTITION queries [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
f0995f2741
partition serialization into id and text [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
25bb444d5a
absence of partitioning works [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
71ebf67f2e
encapsulate partition and minmax_idx load/store code [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
6906921932
load partition and minmax index from disk [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
e9f93028b8
serialize partition and minmax index values [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Alexey Zatelepin
2f80c150db
save MergeTree data format version and parse part names depending on it [#CLICKHOUSE-3000]
2017-09-26 20:40:02 +03:00
Nikolai Kochetov
322e56c692
fix reading with preferred_block_size_bytes = 0
2017-09-24 02:16:30 +03:00
