Commit Graph

1332 Commits

Author SHA1 Message Date
Ilya Golshtein
ff92812ac5 openssl to link with krb5 2020-07-31 13:49:08 +03:00
Ilya Golshtein
09bf3dc0d8 remove odd krb5 files from build 2020-07-31 13:49:08 +03:00
Ilya Golshtein
1a6307c848 headers hell fight 2020-07-31 13:49:08 +03:00
Ilya Golshtein
d7d7283494 generate forgotten headers 2020-07-31 13:49:08 +03:00
Ilya Golshtein
de7544d944 cyrus-sasl depends on krb5 2020-07-31 13:49:08 +03:00
Ilya Golshtein
65eb845ce5 find_program for awk 2020-07-31 13:49:08 +03:00
Ilya Golshtein
8cd51af951 Built against submodules 2020-07-31 13:49:08 +03:00
Ilya Golshtein
017a433403 some cleanup, disabling sasl really works 2020-07-31 13:49:08 +03:00
Ilya Golshtein
439eb38bde config.h and autoconf.h moved to cmake dirs 2020-07-31 13:49:07 +03:00
Ilya Golshtein
d93f0c7548 initial - kerberized kafka works 2020-07-31 13:49:07 +03:00
Azat Khuzhin
3d3c25b52d Switch cppkafka to upstream and bump to upstream/master
All changes in ClickHouse-Extra has been merged into upstream no need in
using the fork.

There are not a lot of changes for now, and cppkafka does not looks like
in the active development stage, but just in case better to sync with
upstream.

FWIW cppkafka usage is pretty minimal and it's usage is questionable,
since librdkafka is under active development while cppkafka is not, and
later still does not support some features that is used in clickhouse:
- interceptors (for logging proxying)
2020-07-31 00:41:44 +03:00
Alexey Milovidov
bbf9fd3c17 Merge branch 'master' into kafka-fix-msan 2020-07-30 22:13:41 +03:00
alesapin
d52f67d24e Fix abtesting build 2020-07-29 14:00:04 +03:00
Alexey Milovidov
ab913c78f4 Fix MSan error in "rdkafka" library #12990 2020-07-28 14:13:32 +03:00
philip.han
b71fe9dab9 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into bayesAB 2020-07-23 22:21:15 +09:00
Vitaly Baranov
0b17595a91
Merge pull request #12423 from vitlibar/update-simdjson
Update simdjson
2020-07-20 10:15:38 +03:00
Vitaly Baranov
4cc101f1d4 Fix build scripts. 2020-07-17 10:30:13 +03:00
Vitaly Baranov
5c2c8edf51 Update SIMDJSON to the latest master 2020-07-17 10:30:08 +03:00
Denis Glazachev
59cb758cf7 Merge branch 'master' into ldap-per-user-authentication 2020-07-16 02:29:24 +04:00
Nikita Mikhaylov
ed05465479 Merge branch 'master' of github.com:ClickHouse/ClickHouse into achimbab-bayesAB 2020-07-15 16:34:56 +00:00
Nikita Mikhaylov
fd125eac1c system 2020-07-15 12:15:43 +00:00
Nikita Mikhaylov
d21c743aaa fix build 2020-07-14 15:09:08 +00:00
alexey-milovidov
1893d89ce3
Merge pull request #12448 from ClickHouse/fix-trash-rabbitmq
Fix trash from RabbitMQ
2020-07-14 01:11:37 +03:00
Alexey Milovidov
cb46bca157 Merge branch 'master' into fix-trash-rabbitmq 2020-07-13 19:51:17 +03:00
alexey-milovidov
3787f55ea4
Merge pull request #12446 from ClickHouse/update-fmt
Update fmtlib to master
2020-07-13 19:09:16 +03:00
Denis Glazachev
f787702922 Merge branch 'master' into ldap-per-user-authentication
* master: (27 commits)
  Whitespaces
  Fix typo
  Fix UBSan report in base64
  Correct default secure port for clickhouse-benchmark #11044
  Remove test with bug #10697
  Update in-functions.md (#12430)
  Allow nullable key in MergeTree
  Update arithmetic-functions.md
  [docs] add rabbitmq docs (#12326)
  Lower block sizes and look what will happen #9248
  Fix lifetime_bytes/lifetime_rows for Buffer direct block write
  Retrigger CI
  Fix up  test_mysql_protocol failed
  Implement lifetime_rows/lifetime_bytes for Buffer engine
  Add comment regarding proxy tunnel usage in PocoHTTPClient.cpp
  Add lifetime_rows/lifetime_bytes interface (exported via system.tables)
  Tiny IStorage refactoring
  Trigger integration-test-runner image rebuild.
  Delete log.txt
  Fix test_mysql_client/test_python_client error
  ...
2020-07-13 15:46:27 +04:00
Alexey Milovidov
e75caae079 Fix UBSan report in HDFS 2020-07-13 08:46:51 +03:00
Alexey Milovidov
8f2055b0a0 Fix trash from RabbitMQ 2020-07-13 04:11:48 +03:00
Alexey Milovidov
bfaa3d6b55 Update fmtlib to master 2020-07-13 03:46:57 +03:00
Alexey Milovidov
3858164438 Fix UBSan report in base64 2020-07-12 22:38:37 +03:00
Denis Glazachev
edb6ef8c09 Merge commit 'ceac649c01b0158090cd271776f3219f5e7ff57c' into ldap-per-user-authentication
* commit 'ceac649c01b0158090cd271776f3219f5e7ff57c': (75 commits)
  [docs] split misc statements (#12403)
  Update 00405_pretty_formats.reference
  Update PrettyCompactBlockOutputFormat.cpp
  Update PrettyBlockOutputFormat.cpp
  Update DataTypeNullable.cpp
  Update 01383_remote_ambiguous_column_shard.sql
  add output_format_pretty_grid_charset setting in docs
  add setting output_format_pretty_grid_charset
  Added a test for #11135
  Update index.md
  RIGHT and FULL JOIN for MergeJoin (#12118)
  Update MergeTreeIndexFullText.cpp
  restart the tests
  [docs] add syntax highlight (#12398)
  query fuzzer
  Fix std::bad_typeid when JSON functions called with argument of wrong type.
  Allow typeid_cast() to cast nullptr to nullptr.
  fix another context-related segfault
  [security docs] actually, only admins can create advisories
  query fuzzer
  ...
2020-07-11 21:32:36 +04:00
philip.han
1e631e2c33 Fix stats-cmake/CMakeLists.txt 2020-07-10 12:52:49 +09:00
philip.han
6c814c9f9f Add bayesAB function for bayesian_ab_testing 2020-07-09 22:11:40 +09:00
Denis Glazachev
5a4762bb8a
Merge branch 'master' into ldap-per-user-authentication 2020-07-09 14:50:02 +04:00
Azat Khuzhin
0aa97af108 Update config.h for arrow
Yes ARROW_FULL_SO_VERSION/ARROW_SO_VERSION is empty right now, like
other version variables (ARROW_VERSION_*)

Refs: #12181
2020-07-09 10:19:44 +03:00
Azat Khuzhin
a645759a42 Set CMAKE_POLICY_DEFAULT_CMP0022/CMAKE_POLICY_DEFAULT_CMP0077 globally
This will fix CMAKE_POLICY_DEFAULT_CMP0077 for snappy:

    CMake Warning (dev) at contrib/snappy/CMakeLists.txt:11 (option):
      Policy CMP0077 is not set: option() honors normal variables.  Run
    "cmake
      --help-policy CMP0077" for policy details.  Use the cmake_policy
    command to
      set the policy and suppress this warning.

      For compatibility with older versions of CMake, option is clearing the
      normal variable 'BUILD_SHARED_LIBS'.
    This warning is for project developers.  Use -Wno-dev to suppress it.
2020-07-09 10:16:39 +03:00
alexey-milovidov
36205e3ddf
Merge pull request #12181 from azat/bump-arrow-to-0.17
Bump arrow to 0.17 (and flatbuffers to v1.12, required by arrow)
2020-07-09 04:24:47 +03:00
Azat Khuzhin
fd5e08884a Bump arrow to 0.17 (and flatbuffers to v1.12, required by arrow)
MOTIVATION:
- remove double-conversion external dependency
- remove flatc (but flatbuffers is still required, arrow just shipped
  with generated files and that's it)

CHANGED:
- remove pre-generated headers, it is shipped with the arrow
- remove flatc (see above)

NOTES (see tests changes):
- and snappy error is reported as unsupported compression.
2020-07-07 23:54:32 +03:00
Denis Glazachev
9effacfbc1 Merge branch 'master' into ldap-per-user-authentication
* master: (1102 commits)
  Update README.md
  Update README.md
  Update README.md
  Update index.md
  [docs] add intrdocution for statements page (#12189)
  Revert "Run perf tests with memory sampling (for allocations >1M)"
  Sanitize LINK_LIBRARIES property for the directories (#12160)
  [docs] refactor Domains overview (#12186)
  DOCS-647: toStartOfSecond (#12190)
  [docs] add intrdocution for commercial page (#12187)
  DOCSUP-1348 Russian translation for new functions (#133) (#12194)
  changelog fixes
  Update index.md (#12191)
  Update zh kafka.md title (#12192)
  Added test for #3767
  style fix for #12152
  Tests for fixed issues #10846 and #7347
  changelog fixes
  [docs] introduction for special table engines (#12170)
  [docs] introduction for third-party interfaces (#12175)
  ...

# Conflicts:
#	src/Access/ya.make
#	src/Common/ErrorCodes.cpp
2020-07-08 00:42:09 +04:00
Azat Khuzhin
32227b9c38 AMQP requires libuv
Otherwise fails:

    FAILED: src/CMakeFiles/dbms.dir/Storages/RabbitMQ/RabbitMQBlockInputStream.cpp.o
    ...
    In file included from ../src/Storages/RabbitMQ/RabbitMQHandler.h:9,
                     from ../src/Storages/RabbitMQ/StorageRabbitMQ.h:11,
                     from ../src/Storages/RabbitMQ/RabbitMQBlockInputStream.h:5,
                     from ../src/Storages/RabbitMQ/RabbitMQBlockInputStream.cpp:3:
    ../contrib/AMQP-CPP/include/amqpcpp/libuv.h:22:10: fatal error: uv.h: No such file or directory
2020-07-07 21:44:24 +03:00
Azat Khuzhin
0a3c1ef450 Set CMP0077 for re2
cmake reports:

    CMake Warning (dev) at contrib/re2/CMakeLists.txt:15 (option):
      Policy CMP0077 is not set: option() honors normal variables.  Run "cmake
      --help-policy CMP0077" for policy details.  Use the cmake_policy command to
      set the policy and suppress this warning.

      For compatibility with older versions of CMake, option is clearing the
      normal variable 'BUILD_SHARED_LIBS'.
    This warning is for project developers.  Use -Wno-dev to suppress it.
2020-07-06 23:34:40 +03:00
Azat Khuzhin
3b75856fca Force CMP0022 for googletest (to avoid using LINK_INTERFACE_LIBRARIES(_<CONFIG>)?)
Othewise cmake reports:

    -- Configuring done
    CMake Warning (dev) in contrib/googletest/googletest/CMakeLists.txt:
      Policy CMP0022 is not set: INTERFACE_LINK_LIBRARIES defines the link
      interface.  Run "cmake --help-policy CMP0022" for policy details.  Use the
      cmake_policy command to set the policy and suppress this warning.

      Target "gtest" has an INTERFACE_LINK_LIBRARIES property which differs from
      its LINK_INTERFACE_LIBRARIES_DEBUG properties.

      INTERFACE_LINK_LIBRARIES:

        global-group;Threads::Threads

      LINK_INTERFACE_LIBRARIES_DEBUG:

        Threads::Threads

    This warning is for project developers.  Use -Wno-dev to suppress it.

    -- Generating done
    -- Build files have been written to: /src/ch/clickhouse/.cmake-tmp
2020-07-06 23:34:40 +03:00
Azat Khuzhin
b173385431 Bump googletest to master (to fix gcc10 builds)
gcc10 reports:

    FAILED: src/CMakeFiles/unit_tests_dbms.dir/Columns/tests/gtest_column_unique.cpp.o
    <snip>
    ../contrib/googletest/googletest/include/gtest/gtest-printers.h:287:7: error: use of deleted function ‘std::basic_ostream<char, _Traits>& std::operator<<(std::basic_ostream<char, _Traits>&, char8_t) [with _Traits = std::char_traits<char>]’
      287 |   *os << value;
          |   ~~~~^~~~~~~~
    In file included from ../base/common/../common/StringRef.h:6,
                     from ../src/Columns/IColumn.h:7,
                     from ../src/Columns/IColumnUnique.h:2,
                     from ../src/Columns/ColumnUnique.h:2,
                     from ../src/Columns/tests/gtest_column_unique.cpp:1:
    /usr/include/c++/10.1.0/ostream:544:5: note: declared here
      544 |     operator<<(basic_ostream<char, _Traits>&, char8_t) = delete;
          |
2020-07-06 23:34:40 +03:00
Azat Khuzhin
a0f257b237 Set GOOGLETEST_VERSION for googletest
Otherwise cmake reports:

    CMake Warning at contrib/googletest/googletest/CMakeLists.txt:54 (project):
      VERSION keyword not followed by a value or was followed by a value that
      expanded to nothing.

(since GOOGLETEST_VERSION is set in contrib/googletest/CMakeLists.txt)
2020-07-06 23:34:40 +03:00
Nikita Mikhaylov
53522c728b
Merge pull request #11662 from Avogar/orc_output_format
Add ORCBlockOutputFormat
2020-07-06 18:51:35 +04:00
Nikita Mikhaylov
dac7a39256 fix ubsan final 2020-07-06 12:29:22 +03:00
alexey-milovidov
1719ab1ec5
Merge pull request #12041 from azat/curl-proper-cmake
Rewrite curl dependency in a more ch compatible way
2020-07-06 06:31:51 +03:00
Alexey Milovidov
351eda5d8a Replace exit to abort in libdivide 2020-07-05 03:02:50 +03:00
Alexey Milovidov
c5cf7199d8 Update libdivide to the latest master 2020-07-05 02:58:10 +03:00
alexey-milovidov
6ae9bfe3b3
Merge pull request #12131 from matwey/fix/cmake_unixodbc
Minor cmake fixes for UNBUNDLED build
2020-07-04 22:29:28 +03:00
Azat Khuzhin
9258368de2 Fail if curl library was enabled and was not found in case of unbundled build 2020-07-04 16:48:25 +03:00
Matwey V. Kornilov
20c8e22942 poco-cmake: Fix Poco::Data::ODBC target when UNBUNDLED
By default IMPORTED target has a scope in the directory in which it is created
and below. This leads to the following issues when building UNBUNDLED:

Target "clickhouse" links to target "Poco::Data::ODBC" but the target was not
found.  Perhaps a find_package() call is missing for an IMPORTED target, or an
ALIAS target is missing?
2020-07-04 13:58:53 +03:00
Matwey V. Kornilov
338e772393 contrib/unixodbc-cmake: Fix build when UNBUNDLED
target_compile_definitions may only set INTERFACE properties on IMPORTED targets
2020-07-04 10:21:56 +03:00
Azat Khuzhin
0700a705bc Rewrite curl dependency in a more ch compatible way
- add support of unbundled curl
- add CURL::libcurl libraries
- avoid explicit linkage of daemon with curl (added with sentry)
- set CACHE variables for non-direct users:
  - mariadb-connector-c
  - aws-s3-cmake
  - sentry-native

Cc: @alexey-milovidov
Cc: @alesapin (requires docker image update)
Cc: @abyss7

Refs: #11300
Refs: #8011
Refs: #8905

v2: replace cmake/find/curl.cmake with proper contrib/curl-cmake (as
pointed by @abyss7, cmake/find/*.cmake is deprecated)
2020-07-03 00:57:08 +03:00
alesapin
64583ceb22 Merge branch 'master' into add-storage-rabbitmq-read-only 2020-07-02 16:08:29 +03:00
Alexander Kuzmenkov
baec4093dd
Merge pull request #11993 from ClickHouse/aku/more-jemalloc-tweaks
jemalloc: set muzzy_decay_ms = 10s
2020-06-30 22:51:21 +03:00
Alexander Kazakov
8b2a5d81df
Update contrib/poco to ClickHouse-Extras/poco #22 (#12037)
* Updated ClickHouse-Extras/poco #22
2020-06-30 14:27:40 +03:00
kssenii
3d2cc9d4b2 Remove libevent library 2020-06-29 12:11:17 +00:00
Alexander Kuzmenkov
8c0177b216
Update contrib/jemalloc-cmake/CMakeLists.txt 2020-06-29 15:01:29 +03:00
alexey-milovidov
77c4861430
Merge pull request #11975 from oandrew/fix-word-breaks
Extend word break config to all non-alphanumeric chars
2020-06-27 19:13:44 +03:00
Alexander Kuzmenkov
8b82ff215f muzzy decay 10 ms 2020-06-27 04:08:21 +03:00
Andrew Onyshchuk
c985dddd45 Update replxx submodule 2020-06-26 13:14:31 -05:00
alexey-milovidov
751d36f7c7
Merge pull request #11954 from oandrew/avro-uuid
Avro UUID support
2020-06-26 14:09:54 +03:00
Andrew Onyshchuk
fc1ae85600 Avro UUID support 2020-06-25 14:25:45 -05:00
alexey-milovidov
a34032cace
Merge pull request #11300 from blinkov/sentry
Opt-in support for sending crash reports
2020-06-25 17:20:50 +03:00
Azat Khuzhin
236cac3d52 Fix jemalloc under OSX (by registering it as default zone explicitly)
In case of OSX jemalloc register itself as a default zone allocator.

But when you link statically then zone_register() will not be called,
and even will be optimized out:

  $ nm clickhouse.patched  | grep -c zone_register
  0

Fix this, by manually calling it.

v2: extern C
2020-06-24 21:05:35 +03:00
alesapin
3fc65b3269 Merge branch 'master' into kssenii-rabbit-mq 2020-06-24 20:14:28 +03:00
Ivan Blinkov
45d1ca2567 merge master 2020-06-24 00:18:16 +03:00
Azat Khuzhin
f76511668f jemalloc: disable it for darwin
Right now it aborts (Abort trap: 6) on osx, the issue does not looks
like [1], but should be investigated, so disable for now.

What is interesting that it works under debugger, so I guess it is
indeed the order of symbol replacements...

  [1]: https://github.com/jemalloc/jemalloc/issues/420
2020-06-23 11:40:34 +03:00
Azat Khuzhin
c230d76cb9 jemalloc: disable percpu_arena on non-linux (not supported anyway)
And since it does not supported it produces the following warning:
  <jemalloc>: No getcpu support: percpu_arena:percpu
  <jemalloc>: perCPU arena getcpu() not available. Setting narenas to 8.

However it is uses narenas=ncpu instead of default narenas=ncpu*4
2020-06-23 11:14:22 +03:00
Azat Khuzhin
5b5ba93523 jemalloc: explicitly disable pthread_setname_np on freebsd (requires 12.1) 2020-06-23 05:45:56 +03:00
Azat Khuzhin
8b6e2e6c67 jemalloc: drop superfluous include directory 2020-06-23 01:47:56 +03:00
Azat Khuzhin
77a3f606f4 jemalloc: enable by default for freebsd/darwin 2020-06-23 01:47:56 +03:00
Azat Khuzhin
1354a103ab jemalloc: add darwin pregenerated header 2020-06-23 01:47:56 +03:00
Azat Khuzhin
637f2cefeb jemalloc: add freebsd pregenerated header 2020-06-23 01:47:56 +03:00
Azat Khuzhin
f0bb511468 jemalloc: fix JEMALLOC_NOTHROW/JEMALLOC_CXX_THROW for freebsd/darwin 2020-06-23 01:47:56 +03:00
Azat Khuzhin
1becc6ca02 jemalloc: jemalloc_preamble.h is common 2020-06-23 01:47:56 +03:00
Azat Khuzhin
6704c42a9e jemalloc: fix HAVE_CPU_SPINWAIT for linux aarch64
Fixes: #11163
2020-06-22 23:24:43 +03:00
Ivan Blinkov
2c0ff29c48 Merge branch 'master' of github.com:ClickHouse/ClickHouse into sentry 2020-06-22 10:06:36 +03:00
Denis Glazachev
6c1b2a4831 Merge branch 'master' into ldap-per-user-authentication
* master: (114 commits)
  Update alter.md (#11823)
  Update PODArray.h
  Added a test for #6549
  Update create.md
  Update create.md
  replacingmergetree translate into "替换合并树“ better (#11814)
  Whitespace
  Fix style
  Added a test
  Fix another 10% of bad code #11529
  Update join.md (#11798)
  Update memory.md (#11800)
  Whitespace
  Check for tests with "fail" in their names
  Fix 10% of bad code #11756
  Update PODArray.h
  avoid duplicate short system name
  adapt recent result as well
  [website] tune benchmark pages styling + add extra checks
  Added MSan suppression for Hyperscan
  ...

# Conflicts:
#	src/Common/ErrorCodes.cpp
2020-06-20 22:01:12 +04:00
Alexey Milovidov
abb59a2621 Give up on UBSan for Hyperscan 2020-06-19 21:45:09 +03:00
Alexey Milovidov
e85227ce2e Remove one compile option 2020-06-19 16:44:30 +03:00
Alexey Milovidov
cebec2115b Added missing compile options from original Hyperscan 2020-06-19 14:57:31 +03:00
Denis Glazachev
5db60202b6 Merge branch 'master' into ldap-per-user-authentication 2020-06-19 00:11:08 +04:00
Alexey Milovidov
39562b4711 Update submodule 2020-06-17 10:12:24 +03:00
Alexey Milovidov
e070c0fa8f Update submodule 2020-06-17 10:10:36 +03:00
Ivan Blinkov
e98a91e9f7 Merge branch 'master' of github.com:ClickHouse/ClickHouse into sentry 2020-06-12 16:53:00 +03:00
alesapin
29311f71bf
Merge pull request #11585 from azat/unbundled-boost
Add unbundled boost support
2020-06-11 23:43:28 +03:00
Ivan Blinkov
b70ed921ea Merge branch 'master' of github.com:ClickHouse/ClickHouse into sentry 2020-06-11 12:22:13 +03:00
Ivan Blinkov
3e5d735871 back to upstream 2020-06-11 12:21:23 +03:00
kssenii
18820814f5 Merge 2020-06-10 23:25:33 +00:00
Denis Glazachev
9e3a28a6b8 Merge branch 'master' into ldap-per-user-authentication
* master: (414 commits)
  Update file.md
  Update merge.md
  Update dictionary.md
  Update external-data.md
  Update distributed.md
  Update null.md
  Update set.md
  Update join.md
  Update url.md
  Update view.md
  Update materializedview.md
  Update memory.md
  Update buffer.md
  Update generate.md
  removed a sentence about global lock during rename (#11577)
  greatCircleAngle en translation (#11584)
  Update configuration-files.md
  try fix flacky test
  Update why.html
  Update rich.html
  ...

# Conflicts:
#	src/Common/ErrorCodes.cpp
#	utils/ci/jobs/quick-build/run.sh
2020-06-11 03:06:17 +04:00
Azat Khuzhin
8ec3ab9c9f Require less recent boost - 1.67 2020-06-10 23:57:54 +03:00
Azat Khuzhin
8418612e09 Add unbundled boost support
Follow-up-for: #11390
Cc: @@abyss7
2020-06-10 20:56:47 +03:00
Ivan
2de9ddce53
Update CMakeLists.txt (#11560)
Update CMakeLists.txt (#11560)
2020-06-10 20:30:34 +03:00
Nikita Mikhaylov
9df693ced5
Merge pull request #11561 from Jokser/s3-http-client-memory-optimization
Avoid copying whole response stream into memory in S3 HTTP client.
2020-06-10 16:53:33 +04:00
Ivan Blinkov
90ee4d52f4 merge master 2020-06-10 13:18:41 +03:00
Ivan Blinkov
f872c639ed Try to disable linker options from sentry 2020-06-09 20:44:56 +03:00
Alexander Tokmakov
6bb68329f8 Merge branch 'master' into merging_external_source_cassandra 2020-06-09 19:51:14 +03:00
Ivan Blinkov
6f7064928f Merge branch 'master' of github.com:ClickHouse/ClickHouse into sentry 2020-06-09 19:07:57 +03:00