* Make separate libs Common/ZooKeeper and Common/ConfigProcessor
renamed: dbms/src/Common/StringUtils.h -> dbms/src/Common/StringUtils/StringUtils.h
* Fix space
* Fix data in system.build_options
* merge
* merge
This makes one pointless check before the first block is read, but
is necessary to prevent reading blocks from storages like Kafka where
messages are read only once.
This contains many fixes and corrections for the Kafka engine.
Most notably it now supports extended configuration similarly to GraphiteMergeTree.
Now it also allows specification of consumer count to parallelize consumption of
multiple partitions both in materialized views and in SELECT queries.
The offsets are now committed in the insertSuffix() method after all rows
are successfully read. If an exception is thrown during reading, affected consumer
unsubscribes from all assignments and rejoins the consumer group to rewind offsets.
This means that the consumer won't lose messages in case of write failures.
- Make sure to use dynamic linking on macOS to avoid OpenSSL static linking bug
- Use -std=c++17 in CFLAGS for files in contrib
- Avoid bad support for thread_local on macOS with clang altogether
Both UnionBlockInputStream and ParallelAggregatingBlockInputStream rely on ParallelInputsProcessor to do stream preparation in parallel, which seems to be absent. This patch fixes it.
Fix thread unsafty issue in https://github.com/yandex/ClickHouse/pull/1516
Here is a list of inputstreams with meaningful work in readPrefix
```
IProfilingBlockInputStream
AsynchronousBlockInputStream
BlockInputStreamFromRowInputStream
CatBoostDatasetBlockInputStream
CreatingSetsBlockInputStream
CSVRowInputStream
DictionaryBlockInputStreamBase
InputStreamFromASTInsertQuery
KafkaBlockInputStream
MergingAggregatedMemoryEfficientBlockInputStream
RemoteBlockInputStream
StorageFileBlockInputStream
```
This addresses one of the remarks in the PR.
All format schemas are required to be in the <format_schema_path> directory.
This makes loading schema files less tedious, as the path can be omitted.
Cap'n Proto is a binary message format.
Like Protocol Buffers and Thrift (but unlike JSON or MessagePack), Cap'n Proto messages are strongly-typed and not self-describing. Due to this, it requires a schema setting to specify schema file and the root object. The schema is parsed on runtime and cached for each SQL statement.
Previously the destination columns were only computed for the first
block, so that subsequently written blocks failed to write
aggregation results to corrent columns.
By default sum(x) promotes the result type to
largest possible integral type to avoid
arithmetic overflow when summing values from
smaller data types.
This is generally desirable behaviour, but it doesn’t
work with summing merge tree, as the result is
expected to be of same type as the input.
This fixes a variant of SummingMergeTree() in which
the columns to sum are explicitly configured.
Previously columns not in that list were ignored,
instead of writing last value.
This also fixes summation of invalid maps with
with only one key column and no value columns.
Modified test to work around compaction limitation
in which a zero-value column isn’t compacted
immediately if the inputs are non-zero but the
output is zero (+1 -1).
* Log query id in executeQuery; Better type mismatch error; change format in report tool
* Better log query_id
* fix message
* Use c++11 thread_local instaed of gcc's __thread
* lock mutex before notifying waiting thread in sync insertion into distributed [#CLICKHOUSE-3379]
* Cmake: fix build without downloaded submodules (#1379)
* fix
* ZooKeeper: fixed stack smashing with tryGet()
The tryGet() operation creates a 1MB buffer on stack. This may or
may not work depending on the default stack size for threads,
whether the stack protector is enabled or not, recursion depth,
and the actual value size.
This is probably going to slow down some ZK operations, but I don't
see how else this could work reliably with the existing API.
* increased timeout for test_insertion_sync_fails_with_timeout
* Update CHANGELOG_RU.md
* Update ZooKeeper.cpp
* Fix warnings
* Fixes
* Dont strip debug info from asan, tsan and other builds except releases
* Fix asan error causd by test 00144
* Fix empty log message (#CLICKHOUSE-3378)
When using a LIMIT to reduce the number of rows
read, JSONEachRow format would always read an
extra row as it triggered a scan to seek the
EOF or beginning of the next row. This is not
an issue with physical tables, but it is
for streaming tables as an extra row is
unintentionally consumed on each read.
In the current version of Poco, unsigned long no longer aliases to
UInt64 with LP64. The size_t aliases to unsigned long with clang,
so all the uses of size_t instead of UInt64 when interacting with
Poco interfaces are gone. I replaced uses with UInt64 where it makes
sense, and added an overloaded function for readVarUInt() to support size_t.
The built-in clang doesn’t support value() for
`std::experimental::optional`. It however supports
dereference operator, which is basically the
same thing:
```
/clickhouse/dbms/src/DataStreams/NullableAdapterBlockInputStream.cpp:83:67: error: call to unavailable member function 'value':
res.insert({elem.column, elem.type, rename[i].value()});
~~~~~~~~~~^~~~~
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/experimental/optional:547:17: note: candidate function has been explicitly made unavailable
value_type& value()
^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/experimental/optional:539:33: note: candidate function has been explicitly made unavailable
constexpr value_type const& value() const
```
