unbundled jemalloc cannot be used, since it includes jemalloc_cpp.o that
contains new/delete overrides, so as clickhouse [1]:
ld.lld: error: duplicate symbol: operator delete(void*, unsigned long)
>>> defined at new_delete.cpp:147 (../src/Common/new_delete.cpp:147)
>>> new_delete.cpp.o:(operator delete(void*, unsigned long)) in archive src/libclickhouse_new_delete.a
>>> defined at jemalloc_cpp.o:(.text+0x160) in archive /usr/lib/x86_64-linux-gnu/libjemalloc.a
[1]: https://clickhouse-builds.s3.yandex.net/15828/35335f07dbf8cab89b4188a674b033c28409dc7b/clickhouse_build_check/build_log_793952627_1602401325.txt
INTERFACE_COMPILE_DEFINITIONS does not work IMPORTED targets:
From 3.6:
Specify compile definitions to use when compiling a given <target>.
The named <target> must have been created by a command such as
add_executable() or add_library() and must not be an Imported Target.
Since 3.11:
Specify compile definitions to use when compiling a given <target>.
The named <target> must have been created by a command such as
add_executable() or add_library() and must not be an ALIAS target.
And this causes the -Wundef warning:
src/Functions/FunctionsHashing.h:1326:5: warning: 'USE_XXHASH' is not defined, evaluates to 0 [-Wundef]
[1] is required for gcc10 build, due to incorrect `static_ltree` declaration:
ld.lld: error: duplicate symbol: static_ltree
>>> defined at trees.h:6 (../contrib/zlib-ng/trees.h:6)
>>> trees.c.o:(static_ltree) in archive contrib/zlib-ng/libzlib.a
>>> defined at deflate_quick.c:168 (../contrib/zlib-ng/arch/x86/deflate_quick.c:168)
>>> deflate_quick.c.o:(.rodata+0x0) in archive contrib/zlib-ng/libzlib.a
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
[1]: https://github.com/ClickHouse-Extras/zlib-ng/pull/2
Squashed:
- cmake: make sure protobuf compiler and snappy are there for unbundled build
- cmake: Turn ON internal GRPC library by default until relevant features will be implemented
- cmake: allow unbundled grpc
Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
Squashed:
- cmake: move USE_INTERNAL_POCO_LIBRARY option to separate file
- point contrib/poco to "add <atomic> to TCPServerDispatcher"
Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
This reverts commit 89419ceb9b
(curl part moved to separate commit)
Squashed:
- termcap removed
- fix for cassandra on apple merged
- cmake: fix "Can't find system zlib library" in unbundled build
- cmake: fix condition when testing for rdkafka platform
- cmake: PROTOBUF_OLD_ABI_COMPAT shouldn't be enabled for internal protobuf
Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
All changes in ClickHouse-Extra has been merged into upstream no need in
using the fork.
There are not a lot of changes for now, and cppkafka does not looks like
in the active development stage, but just in case better to sync with
upstream.
FWIW cppkafka usage is pretty minimal and it's usage is questionable,
since librdkafka is under active development while cppkafka is not, and
later still does not support some features that is used in clickhouse:
- interceptors (for logging proxying)
* commit 'ceac649c01b0158090cd271776f3219f5e7ff57c': (75 commits)
[docs] split misc statements (#12403)
Update 00405_pretty_formats.reference
Update PrettyCompactBlockOutputFormat.cpp
Update PrettyBlockOutputFormat.cpp
Update DataTypeNullable.cpp
Update 01383_remote_ambiguous_column_shard.sql
add output_format_pretty_grid_charset setting in docs
add setting output_format_pretty_grid_charset
Added a test for #11135
Update index.md
RIGHT and FULL JOIN for MergeJoin (#12118)
Update MergeTreeIndexFullText.cpp
restart the tests
[docs] add syntax highlight (#12398)
query fuzzer
Fix std::bad_typeid when JSON functions called with argument of wrong type.
Allow typeid_cast() to cast nullptr to nullptr.
fix another context-related segfault
[security docs] actually, only admins can create advisories
query fuzzer
...
This will fix CMAKE_POLICY_DEFAULT_CMP0077 for snappy:
CMake Warning (dev) at contrib/snappy/CMakeLists.txt:11 (option):
Policy CMP0077 is not set: option() honors normal variables. Run
"cmake
--help-policy CMP0077" for policy details. Use the cmake_policy
command to
set the policy and suppress this warning.
For compatibility with older versions of CMake, option is clearing the
normal variable 'BUILD_SHARED_LIBS'.
This warning is for project developers. Use -Wno-dev to suppress it.
MOTIVATION:
- remove double-conversion external dependency
- remove flatc (but flatbuffers is still required, arrow just shipped
with generated files and that's it)
CHANGED:
- remove pre-generated headers, it is shipped with the arrow
- remove flatc (see above)
NOTES (see tests changes):
- and snappy error is reported as unsupported compression.
Otherwise fails:
FAILED: src/CMakeFiles/dbms.dir/Storages/RabbitMQ/RabbitMQBlockInputStream.cpp.o
...
In file included from ../src/Storages/RabbitMQ/RabbitMQHandler.h:9,
from ../src/Storages/RabbitMQ/StorageRabbitMQ.h:11,
from ../src/Storages/RabbitMQ/RabbitMQBlockInputStream.h:5,
from ../src/Storages/RabbitMQ/RabbitMQBlockInputStream.cpp:3:
../contrib/AMQP-CPP/include/amqpcpp/libuv.h:22:10: fatal error: uv.h: No such file or directory
cmake reports:
CMake Warning (dev) at contrib/re2/CMakeLists.txt:15 (option):
Policy CMP0077 is not set: option() honors normal variables. Run "cmake
--help-policy CMP0077" for policy details. Use the cmake_policy command to
set the policy and suppress this warning.
For compatibility with older versions of CMake, option is clearing the
normal variable 'BUILD_SHARED_LIBS'.
This warning is for project developers. Use -Wno-dev to suppress it.
Othewise cmake reports:
-- Configuring done
CMake Warning (dev) in contrib/googletest/googletest/CMakeLists.txt:
Policy CMP0022 is not set: INTERFACE_LINK_LIBRARIES defines the link
interface. Run "cmake --help-policy CMP0022" for policy details. Use the
cmake_policy command to set the policy and suppress this warning.
Target "gtest" has an INTERFACE_LINK_LIBRARIES property which differs from
its LINK_INTERFACE_LIBRARIES_DEBUG properties.
INTERFACE_LINK_LIBRARIES:
global-group;Threads::Threads
LINK_INTERFACE_LIBRARIES_DEBUG:
Threads::Threads
This warning is for project developers. Use -Wno-dev to suppress it.
-- Generating done
-- Build files have been written to: /src/ch/clickhouse/.cmake-tmp
gcc10 reports:
FAILED: src/CMakeFiles/unit_tests_dbms.dir/Columns/tests/gtest_column_unique.cpp.o
<snip>
../contrib/googletest/googletest/include/gtest/gtest-printers.h:287:7: error: use of deleted function ‘std::basic_ostream<char, _Traits>& std::operator<<(std::basic_ostream<char, _Traits>&, char8_t) [with _Traits = std::char_traits<char>]’
287 | *os << value;
| ~~~~^~~~~~~~
In file included from ../base/common/../common/StringRef.h:6,
from ../src/Columns/IColumn.h:7,
from ../src/Columns/IColumnUnique.h:2,
from ../src/Columns/ColumnUnique.h:2,
from ../src/Columns/tests/gtest_column_unique.cpp:1:
/usr/include/c++/10.1.0/ostream:544:5: note: declared here
544 | operator<<(basic_ostream<char, _Traits>&, char8_t) = delete;
|
Otherwise cmake reports:
CMake Warning at contrib/googletest/googletest/CMakeLists.txt:54 (project):
VERSION keyword not followed by a value or was followed by a value that
expanded to nothing.
(since GOOGLETEST_VERSION is set in contrib/googletest/CMakeLists.txt)
By default IMPORTED target has a scope in the directory in which it is created
and below. This leads to the following issues when building UNBUNDLED:
Target "clickhouse" links to target "Poco::Data::ODBC" but the target was not
found. Perhaps a find_package() call is missing for an IMPORTED target, or an
ALIAS target is missing?
In case of OSX jemalloc register itself as a default zone allocator.
But when you link statically then zone_register() will not be called,
and even will be optimized out:
$ nm clickhouse.patched | grep -c zone_register
0
Fix this, by manually calling it.
v2: extern C
Right now it aborts (Abort trap: 6) on osx, the issue does not looks
like [1], but should be investigated, so disable for now.
What is interesting that it works under debugger, so I guess it is
indeed the order of symbol replacements...
[1]: https://github.com/jemalloc/jemalloc/issues/420
And since it does not supported it produces the following warning:
<jemalloc>: No getcpu support: percpu_arena:percpu
<jemalloc>: perCPU arena getcpu() not available. Setting narenas to 8.
However it is uses narenas=ncpu instead of default narenas=ncpu*4
* master: (114 commits)
Update alter.md (#11823)
Update PODArray.h
Added a test for #6549
Update create.md
Update create.md
replacingmergetree translate into "替换合并树“ better (#11814)
Whitespace
Fix style
Added a test
Fix another 10% of bad code #11529
Update join.md (#11798)
Update memory.md (#11800)
Whitespace
Check for tests with "fail" in their names
Fix 10% of bad code #11756
Update PODArray.h
avoid duplicate short system name
adapt recent result as well
[website] tune benchmark pages styling + add extra checks
Added MSan suppression for Hyperscan
...
# Conflicts:
# src/Common/ErrorCodes.cpp
* master: (114 commits)
Update PushingToViewsBlockOutputStream.cpp
Update PushingToViewsBlockOutputStream.cpp
make clang-10 happy
Fix sync_async test (remove timeout)
CLICKHOUSEDOCS-631: temporary_files_codec, join_on_disk_max_files_to_merge settings. (#11242)
Suppress output of cancelled queries in clickhouse-client #9473
Better log messages in ConfigReloader
fix select from StorageJoin
Fix unit tests under MSan
Added test.
Fix build.
Fix arguments for AggregateFunctionQuantile/
Update style.md
Add a guide on error messages.
Report dictionary name on dictionary load errors.
more types in ASOF JOIN (#11301)
Fix part_log test
Update test.
Add perftest.
Parallel processing for PushingToViewsBlockOutputStream::writeSuffix
...
This patch can go upstream, regardless of the jemalloc update
eventually.
Also I noticed that header with jemalloc version differs (someone forgot
to update?)
* Added flag for safer rdkafka destruction, but more testing detected another hang case, which has no straigt solutions and can be workarounded currenly only by draining the consumer queue, so destructor is back
* After review fixes
* After review fixes2
* After review fixes3
