Commit Graph

769 Commits

Author SHA1 Message Date
Alexander Tokmakov
a7d3a024c7 Revert "Revert "Test and doc for PR12771 krb5 + cyrus-sasl + kerberized kafka""
This reverts commit c298c633a7.
2020-09-29 11:56:37 +03:00
tavplubix
c298c633a7
Revert "Test and doc for PR12771 krb5 + cyrus-sasl + kerberized kafka" 2020-09-24 16:10:24 +03:00
Ilya Yatsishin
7dad84ea6b
Merge pull request #14265 from arenadata/ADQM-160
Test and doc for PR12771 krb5 + cyrus-sasl + kerberized kafka
2020-09-23 19:02:58 +03:00
Ilya Golshtein
e2ee97202b USE_LIBGSASL not required to use internal librdkafka 2020-09-22 23:16:34 +03:00
alesapin
269d225637 Merge branch 'master' into myrrc-feature-cmake-flags-doc-generator 2020-09-22 17:00:02 +03:00
myrrc
eb1bb88fa0 updated the gtest find cmake file 2020-09-19 20:45:11 +03:00
myrrc
c78d9f9f38 moved the generators to docs/ folder 2020-09-19 19:42:36 +03:00
Alexey Milovidov
059646743b Remove OpenCL 2020-09-19 17:20:00 +03:00
Alexey Milovidov
cac65b38f2 Merge branch 'master' into clang-tidy-11 2020-09-19 00:02:44 +03:00
myrrc
f6bce8bd1e reverted USE to ENABLE 2020-09-18 17:18:59 +03:00
myrrc
3953ed485b added sub-tables in report, replaced some USE_ -> ENABLE_ for
consistency
2020-09-18 16:09:22 +03:00
Alexey Milovidov
799a675b5d Merge branch 'master' into AES_encrypt_decrypt 2020-09-17 21:42:51 +03:00
myrrc
7cd5c0d34f Merge remote-tracking branch 'upstream/master' into feature/cmake-flags-doc-generator 2020-09-17 19:17:09 +03:00
myrrc
ac606dca77 updated cmake/ files, added some comments 2020-09-17 18:37:23 +03:00
Vitaly Baranov
3356d75b23
Merge pull request #13156 from azat/cluster-secure
Secure inter-cluster query execution (with initial_user as current query user) [v3]
2020-09-17 17:11:00 +03:00
Alexey Milovidov
9ceb4934c2 Use clang-tidy 11 2020-09-17 13:32:16 +03:00
Simon Liu
c41e230589 Modify the minimum version of the Clang compiler that Clickhouse can support (8 -> 9),
The main reason is that in Clang 8.0 "#pragma clang attribute" cannot be used in the namesapce scope.
2020-09-16 22:14:30 +08:00
alesapin
a75254766e
Merge pull request #14609 from ClickHouse/update_to_gcc_10
Update default build to gcc-10
2020-09-15 09:49:43 +03:00
Azat Khuzhin
85b63ecadf Bump revision manually (to make it compatible with other version from this tag)
As @alexey-milovidov wrote.

P.S. I did not use ./utils/release/release_lib.sh (via ./release), since
it is pretty complex (it fetches tags and so on)

The following macros has been updated:
- VERSION_REVISION -- usually bumped with utils/release/release_lib.sh
  (interesting that even if there were no changes)
- DBMS_MIN_REVISION_WITH_INTERSERVER_SECRET
- DBMS_TCP_PROTOCOL_VERSION -- does not affect anything, some internal
  yandex stuff
2020-09-15 01:36:28 +03:00
alesapin
fb7fc28e6f
Update warnings.cmake 2020-09-14 12:20:43 +03:00
alesapin
8075ce2809
Update warnings.cmake 2020-09-12 15:42:32 +03:00
alesapin
e25b1da29f Disable -Wstringop-overflow for gcc-10 2020-09-11 13:53:26 +03:00
Azat Khuzhin
0f4fdcbf38 Pass -fsanitize-blacklist for TSAN only under clang (gcc does not support this)
And no such check for -fsnaitize=memory, since gcc does not support it
anyway.
2020-09-10 02:07:28 +03:00
alesapin
4ba8f8960b Increase frame-larger-than 2020-09-09 12:53:24 +03:00
robot-clickhouse
8def718661 Auto version update to [20.10.1.1] [54440] 2020-09-08 17:04:41 +03:00
robot-clickhouse
7bc3c43a38 Auto version update to [20.9.1.4585] [54439] 2020-09-08 17:04:06 +03:00
alesapin
b91f37169d Fix cmake 2020-09-08 10:09:40 +03:00
Vasily Nemkov
b147ffcd43 encrypt, aes_encrypt_mysql, decrypt, aes_decrypt_mysql functions
Functions to encrypt/decrypt any input data with OpenSSL's ciphers
with custom key, iv, and add (-gcm mode only).

_mysql versions are 100% compatitable with corresponding MySQL functions

Supported modes depend on OpenSSL version, but generally are:
aes-{128,192,56}-{ecb,cbc,cfb1,cfb8,cfb128,ofb,gcm}

Please note that in a -gcm mode a 16-byte tag is appended to the ciphertext
on encryption and is expected to be found at the end of ciphertext on decryption.

Added tests that verify compatibility with MySQL functions,
and test vectors for GCM mode from OpenSSL.

Added masking rules for aes_X funtions
Rules are installed by default to config.d/query_masking_rules.xml
2020-09-01 12:42:15 +03:00
robot-clickhouse
e2fa0eae2f Auto version update to [20.9.1.1] [54439] 2020-08-31 23:07:41 +03:00
robot-clickhouse
f63e5faac0 Auto version update to [20.8.1.4513] [54438] 2020-08-31 23:07:05 +03:00
alesapin
30cf59fc9a Revert version update 2020-08-31 22:51:56 +03:00
robot-clickhouse
59a3937adb Auto version update to [20.8.1.4513] [54438] 2020-08-31 19:41:50 +03:00
Konstantin Podshumok
a9d2de595d
xcmake: enbale ccache by default if it is installed
Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-27 14:27:00 +03:00
Konstantin Podshumok
8ce844a40d
cmake: when USE_STATIC_LIBRARIES prefer bundled brotli
even if UNBUNDLED=ON because libbrotli*.a and similar are not
available in most of the systems

Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:44 +03:00
Konstantin Podshumok
e296997b6c
cmake: fix potential curl linking problems
Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:44 +03:00
Konstantin Podshumok
68a921cdea
cmake: use bundled rdkafaka for in unbundled tests for now
Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:43 +03:00
Konstantin Podshumok
5097793872
cmake: Fix odbc in unbudled
Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:43 +03:00
Konstantin Podshumok
5d13c1f03f
cmake: unbundled: libxml2 requires liblzma when building static
Squashed:
- fixup hdfs3-libxml2

Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:43 +03:00
Konstantin Podshumok
ef858027d4
unbundled: fix detection of mariadb headers
Squashed:
- unbundled: add warning about mariadbclient dependencies

Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:42 +03:00
Konstantin Podshumok
d989e59699
unbundled: link with termcap when using static llvm
ld.lld: error: undefined symbol: setupterm
referenced by Process.cpp.o:(llvm::sys::Process::FileDescriptorHasColors(int)) in archive /usr/lib/llvm-9/lib/libLLVMSupport.a

Squashed:
- fixup termcap + external llvm

Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:42 +03:00
Konstantin Podshumok
d739c6a38f
Revert "Merge pull request #13689 from ClickHouse/remove-termcap"
This reverts commit da8d8f35dd, reversing
changes made to c3d0a87b8b.
2020-08-25 20:16:42 +03:00
Konstantin Podshumok
cc6a59d59b
cmake: disable internal ORC when using external Parquet
Squashed:
- fix internal poco linking with `zlib`

Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:42 +03:00
Konstantin Podshumok
e0939a8666
unbundled: add thrift when using external parquet
Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:41 +03:00
Konstantin Podshumok
17c46faaa7
cmake: fix parquet/arrow variables
squashed:

- sync FindArrow.cmake and FindParquet.cmake with arrow repo
- unbundled: add arrow to dbms link libraries
- cmake: fix-up unbundled Arrow support

Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:41 +03:00
Konstantin Podshumok
3e3b67601f
cmake: USE_INTERNAL_GSASL_LIBRARY by default when USE_STATIC_LIBRARIES
Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:41 +03:00
Konstantin Podshumok
2b867cb2db
cmake: disable external avro as we don't support it yet
Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:41 +03:00
Konstantin Podshumok
2c9c7c9e95
cmake: USE_INTERNAL_H3_LIBRARY is always ON by default
as I don't know any system with package for it

Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:40 +03:00
Konstantin Podshumok
e101db600c
cmake: set USE_INTERNAL_SPARSEHASH_LIBRARY=ON by default
as we don't know any distribution with sparsehash-c11 package

Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:40 +03:00
Konstantin Podshumok
f40a9f5b20
cmake: set USE_INTERNAL_ORC and USE_INTERNAL_HDFS3 to OFF by default even in unbundled
Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:40 +03:00
Konstantin Podshumok
139ba52944
cmake: unbundled + use_static_libs = no ldap
Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:40 +03:00
Konstantin Podshumok
0c30721b91
cmake: set ENABLE_ORC to OFF by default on platforms that don't support protobuf or parquet
Squashed:
- cmake: Better diagnostic message about broken ORC dependencies

Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:39 +03:00
Konstantin Podshumok
3ec7a4bc53
cmake: make sure ENABLE_GRPC is off by default when PROTOBUF is OFF by default
Squashed:
- cmake: make sure protobuf compiler and snappy are there for unbundled build
- cmake: Turn ON internal GRPC library by default until relevant features will be implemented
- cmake: allow unbundled grpc

Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:39 +03:00
Konstantin Podshumok
986bdf86ee
cmake: don't enable internal llvm with disabled embedded compiler
Even if UNBUNDLED is off

Squashed:
- cmake: fix misleading error message about internal llvm on unsupported arches
- cmake: re-do some checks for LLVM options

Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:39 +03:00
Konstantin Podshumok
1913522233
cmake: always use internal poco library as it went too far from upstream
Squashed:
- cmake: move USE_INTERNAL_POCO_LIBRARY option to separate file
- point contrib/poco to "add <atomic> to TCPServerDispatcher"

Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:38 +03:00
Konstantin Podshumok
528f9d587f
cmake: find/s3: use FAIL_ON_UNSUPPORTED_OPTIONS_COMBINATION
Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:38 +03:00
Konstantin Podshumok
d456aa86ef
Revert "Revert "cmake: Add option to fail configuration instead of auto-reconfiguration""
This reverts commit 89419ceb9b
(curl part moved to separate commit)

Squashed:
- termcap removed
- fix for cassandra on apple merged
- cmake: fix "Can't find system zlib library" in unbundled build
- cmake: fix condition when testing for rdkafka platform
- cmake: PROTOBUF_OLD_ABI_COMPAT shouldn't be enabled for internal protobuf

Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:37 +03:00
alexey-milovidov
fb0e68f808
Merge pull request #12771 from arenadata/ADQM-109
krb5 + cyrus-sasl + kerberized kafka
2020-08-20 20:13:01 +03:00
Ilya Yatsishin
e0d86f8b45
Merge pull request #13708 from qoega/cassandra-macos-rpath
Fix cassandra build on macos
2020-08-14 15:49:47 +03:00
Yatsishin Ilya
4c1e5d9d56 Fix cassandra build on macos 2020-08-14 09:25:08 +03:00
Alexey Milovidov
ad20a384c1 Remove useless code 2020-08-14 02:55:10 +03:00
alexey-milovidov
89419ceb9b
Revert "cmake: Add option to fail configuration instead of auto-reconfiguration" 2020-08-13 23:10:06 +03:00
Konstantin Podshumok
d22de6c7e9 cmake: Make sure USE_SIMDJSON default value is in sync with ENABLE_LIBRARIES
Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-09 21:37:46 +03:00
Konstantin Podshumok
fe56697893 cmake: Add option to fail configuration instead of auto-reconfiguration
squashed:
find/msgpack.cmake - merge collision fix, trivial

Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-09 21:37:26 +03:00
robot-clickhouse
a82c0f5371 Auto version update to [20.8.1.1] [54438] 2020-08-07 21:45:49 +03:00
robot-clickhouse
19f3800a55 Auto version update to [20.7.1.4310] [54437] 2020-08-07 21:45:15 +03:00
Alexey Milovidov
cf6ccbc656 Adjustments 2020-08-03 20:44:23 +03:00
Alexey Milovidov
f5f0c788b8 Relax the limit 2020-08-03 00:10:57 +03:00
Alexey Milovidov
cf388cc32f Slightly lower the limit 2020-08-02 04:35:49 +03:00
Ilya Golshtein
ff92812ac5 openssl to link with krb5 2020-07-31 13:49:08 +03:00
Ilya Golshtein
017a433403 some cleanup, disabling sasl really works 2020-07-31 13:49:08 +03:00
Ilya Golshtein
d93f0c7548 initial - kerberized kafka works 2020-07-31 13:49:07 +03:00
Alexey Milovidov
c202691319 Merge branch 'master' into stack-frams-size-limit 2020-07-30 23:53:52 +03:00
alesapin
d52f67d24e Fix abtesting build 2020-07-29 14:00:04 +03:00
Alexey Milovidov
50bbfc848d Merge branch 'master' into stack-frams-size-limit 2020-07-21 14:45:15 +03:00
Vitaly Baranov
0b17595a91
Merge pull request #12423 from vitlibar/update-simdjson
Update simdjson
2020-07-20 10:15:38 +03:00
Vitaly Baranov
4cc101f1d4 Fix build scripts. 2020-07-17 10:30:13 +03:00
Vitaly Baranov
000b197ad1
Merge pull request #11234 from traceon/ldap-per-user-authentication
Add LDAP authentication support
2020-07-16 13:17:21 +03:00
Denis Glazachev
59cb758cf7 Merge branch 'master' into ldap-per-user-authentication 2020-07-16 02:29:24 +04:00
alesapin
e35c49faa2 Merge branch 'master' into fasttest 2020-07-15 22:35:46 +03:00
Alexander Kuzmenkov
3e6b9f297f Some provisions for Android build 2020-07-15 14:16:00 +03:00
alesapin
a31109f023 Better submdoules 2020-07-14 21:44:57 +03:00
alesapin
0740b88d6b Merge branch 'master' into fasttest 2020-07-14 15:19:05 +03:00
robot-clickhouse
0f23642a3d Auto version update to [20.7.1.1] [54437] 2020-07-13 18:26:03 +03:00
robot-clickhouse
0c5be7309c Auto version update to [20.6.1.4066] [54436] 2020-07-13 18:25:23 +03:00
alesapin
a9824fd650 Better cmake and less libraries 2020-07-10 22:08:18 +03:00
alesapin
4ab5edadb7 Less libraries 2020-07-10 12:47:50 +03:00
Denis Glazachev
5a4762bb8a
Merge branch 'master' into ldap-per-user-authentication 2020-07-09 14:50:02 +04:00
alexey-milovidov
f021376874
Merge pull request #12178 from azat/gcc10-build-fixes
gcc10/cmake build fixes
2020-07-09 01:49:07 +03:00
Denis Glazachev
9effacfbc1 Merge branch 'master' into ldap-per-user-authentication
* master: (1102 commits)
  Update README.md
  Update README.md
  Update README.md
  Update index.md
  [docs] add intrdocution for statements page (#12189)
  Revert "Run perf tests with memory sampling (for allocations >1M)"
  Sanitize LINK_LIBRARIES property for the directories (#12160)
  [docs] refactor Domains overview (#12186)
  DOCS-647: toStartOfSecond (#12190)
  [docs] add intrdocution for commercial page (#12187)
  DOCSUP-1348 Russian translation for new functions (#133) (#12194)
  changelog fixes
  Update index.md (#12191)
  Update zh kafka.md title (#12192)
  Added test for #3767
  style fix for #12152
  Tests for fixed issues #10846 and #7347
  changelog fixes
  [docs] introduction for special table engines (#12170)
  [docs] introduction for third-party interfaces (#12175)
  ...

# Conflicts:
#	src/Access/ya.make
#	src/Common/ErrorCodes.cpp
2020-07-08 00:42:09 +04:00
Azat Khuzhin
a5cb0aed12
Sanitize LINK_LIBRARIES property for the directories (#12160)
When you will try to link target with the directory (that exists), cmake will
skip this without an error, only the following warning will be reported:

    target_link_libraries(main /tmp)

    WARNING: Target "main" requests linking to directory "/tmp".  Targets may link only to libraries.  CMake is dropping the item.

And there is no cmake policy that controls this.
(I guess the reason that it is allowed is because of FRAMEWORK for OSX).

So to avoid error-prone cmake rules, this can be sanitized.
There are the following ways:
- overwrite target_link_libraries()/link_libraries() and check *before*
  calling real macro, but this requires duplicate all supported syntax
  -- too complex
- overwrite target_link_libraries() and check LINK_LIBRARIES property, this
  works great
  -- but cannot be used with link_libraries()
- use BUILDSYSTEM_TARGETS property to get list of all targets and sanitize
  -- this will work.

I also tested it with the following patch:

    $ git di
    diff --git a/base/daemon/CMakeLists.txt b/base/daemon/CMakeLists.txt
    index 26d59a57e7..35e6ff6432 100644
    --- a/base/daemon/CMakeLists.txt
    +++ b/base/daemon/CMakeLists.txt
    @@ -9,4 +9,5 @@ target_link_libraries (daemon PUBLIC loggers PRIVATE clickhouse_common_io clickh

     if (USE_SENTRY)
         target_link_libraries (daemon PRIVATE ${SENTRY_LIBRARY})
    +    target_link_libraries (daemon PRIVATE /tmp)
     endif ()

And it works:

    CMake Error at cmake/sanitize_target_link_libraries.cmake:48 (message):
       daemon requested to link with directory: /tmp
    Call Stack (most recent call first):
      cmake/sanitize_target_link_libraries.cmake:55 (sanitize_link_libraries)
      CMakeLists.txt:425 (include)

Refs: #12041
2020-07-07 19:22:41 +03:00
Azat Khuzhin
2d2a34d6cd Use ENABLE_LIBRARIES option for AMQP-CPP
This is tiny fix, there are more problems that just this small little
bit.
2020-07-06 23:34:40 +03:00
alexey-milovidov
1719ab1ec5
Merge pull request #12041 from azat/curl-proper-cmake
Rewrite curl dependency in a more ch compatible way
2020-07-06 06:31:51 +03:00
Azat Khuzhin
e6aacaac5c Do not enable sentry if ENABLE_LIBRARIES is not set 2020-07-04 16:41:22 +03:00
Azat Khuzhin
0700a705bc Rewrite curl dependency in a more ch compatible way
- add support of unbundled curl
- add CURL::libcurl libraries
- avoid explicit linkage of daemon with curl (added with sentry)
- set CACHE variables for non-direct users:
  - mariadb-connector-c
  - aws-s3-cmake
  - sentry-native

Cc: @alexey-milovidov
Cc: @alesapin (requires docker image update)
Cc: @abyss7

Refs: #11300
Refs: #8011
Refs: #8905

v2: replace cmake/find/curl.cmake with proper contrib/curl-cmake (as
pointed by @abyss7, cmake/find/*.cmake is deprecated)
2020-07-03 00:57:08 +03:00
alesapin
64583ceb22 Merge branch 'master' into add-storage-rabbitmq-read-only 2020-07-02 16:08:29 +03:00
kssenii
3d2cc9d4b2 Remove libevent library 2020-06-29 12:11:17 +00:00
alesapin
3fc65b3269 Merge branch 'master' into kssenii-rabbit-mq 2020-06-24 20:14:28 +03:00
Ivan Blinkov
45d1ca2567 merge master 2020-06-24 00:18:16 +03:00
robot-clickhouse
4bd47b8adc Auto version update to [20.6.1.1] [54436] 2020-06-22 20:40:26 +03:00
robot-clickhouse
bc7f98141f Auto version update to [20.5.1.3833] [54435] 2020-06-22 20:39:51 +03:00