Commit Graph

830 Commits

Author SHA1 Message Date
Alexey Milovidov
bf2df558d4 Merge branch 'master' into FawnD2-switch-upstream-for-arrow-submodule 2020-12-20 09:38:46 +03:00
alexey-milovidov
41b2503d1a
Merge pull request #18114 from azat/20.13-gcc10-sanitizers
gcc10 sanitizers support
2020-12-19 12:54:36 +03:00
Alexey Milovidov
42a4e38f08 Merge branch 'master' into harmful 2020-12-17 06:28:55 +03:00
Azat Khuzhin
fd19fa1b70 Disable grpc protocol by default for TSAN and gcc
Since libtsan.a already has them:

    ld.lld: error: duplicate symbol: RunningOnValgrind
    >>> defined at dynamic_annotations.cc:99 (../contrib/abseil-cpp/absl/base/dynamic_annotations.cc:99)
    >>>            dynamic_annotations.cc.o:(RunningOnValgrind) in archive contrib/abseil-cpp/absl/base/libabsl_dynamic_annotations.a
    >>> defined at tsan_interface_ann.o:(.text+0x15A0) in archive /usr/lib/gcc/x86_64-linux-gnu/10/libtsan.a

    ld.lld: error: duplicate symbol: AnnotateMemoryIsInitialized
    >>> defined at dynamic_annotations.cc:75 (../contrib/abseil-cpp/absl/base/dynamic_annotations.cc:75)
    >>>            dynamic_annotations.cc.o:(AnnotateMemoryIsInitialized) in archive contrib/abseil-cpp/absl/base/libabsl_dynamic_annotations.a
    >>> defined at tsan_interface_ann.o:(.text+0x1600) in archive /usr/lib/gcc/x86_64-linux-gnu/10/libtsan.a

    ld.lld: error: duplicate symbol: AnnotateMemoryIsUnt show initialized
    >>> defined at dynamic_annotations.cc:75 (../contrib/abseil-cpp/absl/base/dynamic_annotations.cc:75)
    >>>            dynamic_annotations.cc.o:(AnnotateMemoryIsUninitialized) in archive contrib/abseil-cpp/absl/base/libabsl_dynamic_annotations.a
    >>> defined at tsan_interface_ann.o:(.text+0x1610) in archive /usr/lib/gcc/x86_64-linux-gnu/10/libtsan.a

    ...

And after this patch I successfully managed to build with thread
sanitizer under gcc.
2020-12-16 21:31:03 +03:00
Azat Khuzhin
c862ae0c42 Explicitly link with -llib$SANITIZER for gcc
gcc with -nodefaultlibs does not add sanitizers library during linkage
with -static-libasan and similar, fix this, by add them explicitly.

From the gcc(1) about -nodefaultlibs:

    Do not use the standard system libraries when linking. Only the
    libraries you specify are passed to the linker, and options specifying
    linkage of the system libraries, such as -static-libgcc or
    -shared-libgcc, are ignored. The standard startup files are used
    normally, unless -nostartfiles is used.

Plus checked it manually, w/o -nodefaultlibs everything works without
this explicit linkage.

After this patch UBSAN under gcc compiled successfully.

v2: freebsd and darwin
2020-12-16 21:31:03 +03:00
Azat Khuzhin
1dca6e9269 cmake/linux/default_libs.cmake: fix alignment 2020-12-16 21:31:03 +03:00
Azat Khuzhin
90df89b504 cmake/sanitizers.cmake: tiny refactoring 2020-12-16 21:31:03 +03:00
Alexey Milovidov
dabde573df Adjust CMake 2020-12-16 19:30:23 +03:00
Azat Khuzhin
8de0c0e8fc Fix warning message for unusupported -fsanitize-blacklist 2020-12-15 02:29:09 +03:00
Azat Khuzhin
519d89dc9f Do not pass -fsanitize-blacklist for gcc (it does not support it) for UBSAN 2020-12-15 02:24:28 +03:00
Azat Khuzhin
75153c9663 build: ThinLTO provides its own parallel linking, limit to 2 in case of ThinLTO 2020-12-11 01:32:38 +03:00
Maksim Kita
5b57c59a3b Updated fast_float integration 2020-12-06 23:37:55 +03:00
Maksim Kita
87c50602bc Fast float updated implementation 2020-12-06 23:37:35 +03:00
Maksim Kita
80d66b8868 FastFloat library integration 2020-12-06 23:37:35 +03:00
Ivan
315ff4f0d9
ANTLR4 Grammar for ClickHouse and new parser (#11298) 2020-12-04 05:15:44 +03:00
FawnD2
26488467f8 Merge branch 'master' into switch-upstream-for-arrow-submodule 2020-11-29 15:53:28 +03:00
Denis Glazachev
0e6dd287e7 Fix CMake generation and build for native Xcode and AppleClang 2020-11-27 20:33:16 +04:00
Nikolai Kochetov
9291bbb04b
Merge pull request #16804 from vdimir/ip-dict-no-trie
sorted-array based ip_dict
2020-11-26 19:26:06 +03:00
Alexander Kuzmenkov
394b81ac46 Merge remote-tracking branch 'origin/master' into HEAD 2020-11-25 23:38:55 +03:00
Nikolai Kochetov
729272391f
Merge branch 'master' into ip-dict-no-trie 2020-11-25 23:07:19 +03:00
Vitaly Baranov
8d96a11d8d Update grpc build scripts. 2020-11-24 17:53:34 +03:00
myrrc
fbb0e6e6aa Merge remote-tracking branch 'upstream/master' into improvement/diff-types-in-avg-weighted 2020-11-24 16:04:17 +03:00
robot-clickhouse
dc19fba36f Auto version update to [20.13.1.1] [54444] 2020-11-23 10:29:26 +03:00
robot-clickhouse
67e9901cda Auto version update to [20.12.1.5236] [54443] 2020-11-23 10:26:45 +03:00
vdimir
e2fac19686
Remove libbtrie 2020-11-21 21:31:15 +03:00
myrrc
a8d8dd7d3e Merge remote-tracking branch 'upstream/master' into improvement/diff-types-in-avg-weighted 2020-11-12 14:50:59 +03:00
sundy-li
46c01eb7a2 improve unbundle build 2020-11-11 09:08:53 +08:00
sundy-li
fb7066d73d * fix fasttest and cmake && pipline for all_scan
* unique the keys
* add inputstream && outputstream
2020-11-11 09:08:53 +08:00
sundy-li
c7617173c4 add sync_file_range for glibc-compatibility 2020-11-11 09:08:52 +08:00
sundy-li
1fa4906dd2 update cmake && remove useless codes 2020-11-11 09:08:52 +08:00
sundy-li
20315edae8 update cmake 2020-11-11 09:08:52 +08:00
root
e9de5b6ad4 Add StorageEmbeddedRocksdb Engine 2020-11-11 09:08:52 +08:00
myrrc
e460248624 Merge remote-tracking branch 'upstream/master' into improvement/diff-types-in-avg-weighted 2020-11-10 14:54:24 +03:00
Danila Kutenin
3c6794bf37 Add Floyd-Rivest selection algorithm instead of std::partial_sort 2020-11-09 20:22:05 +03:00
FawnD2
6a89d965fd Merge branch 'master' into switch-upstream-for-arrow-submodule 2020-11-07 22:34:55 +03:00
FawnD2
65e4fa555b Remove build branch with internal parquet native cmake 2020-11-07 19:49:36 +03:00
robot-clickhouse
f7b7854ace Auto version update to [20.12.1.1] [54443] 2020-11-05 21:52:50 +03:00
robot-clickhouse
19ea4f1a95 Auto version update to [20.11.1.5109] [54442] 2020-11-05 21:50:15 +03:00
myrrc
ab1b7267b6 fixed the build, added some comments 2020-11-03 17:56:07 +03:00
alexey-milovidov
2d80c98964
Merge pull request #16509 from kitaisreal/cmake-moved-power-pc-check-in-tools
CMake moved power pc check in tools
2020-10-30 16:03:50 +03:00
Maksim Kita
d3f08b21de CMake moved power pc check in tools 2020-10-29 11:28:40 +03:00
Alexey Milovidov
91e80598cd Add "harmful" library 2020-10-28 02:00:16 +03:00
Mikhail Filimonov
41971e073a
Fix typos reported by codespell 2020-10-27 12:04:03 +01:00
alexey-milovidov
2b00b5391a
Merge pull request #11844 from Enmk/AES_encrypt_decrypt
encrypt and decrypt functions
2020-10-20 22:57:35 +03:00
Alexey Milovidov
19e31578df Better message about build error on Mac #16072 2020-10-16 15:00:26 +03:00
Alexey Milovidov
5e502c52b3 Merge branch 'master' into AES_encrypt_decrypt 2020-10-14 16:30:23 +03:00
Azat Khuzhin
0f5ba33be8 Exclude zero-as-null-pointer-constant for libstdc++ under WEVERYTHING 2020-10-11 22:11:18 +03:00
alexey-milovidov
557002262f
Merge pull request #15828 from azat/unbundled-INTERFACE_COMPILE_DEFINITIONS
Fix INTERFACE_COMPILE_DEFINITIONS for IMPORTED targets
2020-10-11 19:28:18 +03:00
Azat Khuzhin
35335f07db Fix INTERFACE_COMPILE_DEFINITIONS for IMPORTED targets
INTERFACE_COMPILE_DEFINITIONS does not work IMPORTED targets:

From 3.6:

    Specify compile definitions to use when compiling a given <target>.
    The named <target> must have been created by a command such as
    add_executable() or add_library() and must not be an Imported Target.

Since 3.11:

    Specify compile definitions to use when compiling a given <target>.
    The named <target> must have been created by a command such as
    add_executable() or add_library() and must not be an ALIAS target.

And this causes the -Wundef warning:
    src/Functions/FunctionsHashing.h:1326:5: warning: 'USE_XXHASH' is not defined, evaluates to 0 [-Wundef]
2020-10-11 09:06:01 +03:00
Azat Khuzhin
4e49812e32 Enable Wzero-as-null-pointer-constant only for libcxx
Since libstdc++ has some of such compares for 3way compare:

    /src/ch/clickhouse/src/Common/UInt128.h:61:71: warning: zero as null pointer constant [-Wzero-as-null-pointer-constant]
        bool inline operator>= (const UInt128 rhs) const { return tuple() >= rhs.tuple(); }
                                                                          ^~
                                                                          nullptr
    /usr/include/c++/10.2.0/tuple:1426:5: note: while rewriting comparison as call to 'operator<=>' declared here
        operator<=>(const tuple<_Tps...>& __t, const tuple<_Ups...>& __u)
2020-10-10 22:48:36 +03:00
Azat Khuzhin
0813c1073b Add -Wzero-as-null-pointer-constant for gcc 2020-10-10 22:45:39 +03:00
robot-clickhouse
554fa482e6 Auto version update to [20.11.1.1] [54442] 2020-10-10 18:39:58 +03:00
robot-clickhouse
94be292b92 Auto version update to [20.10.1.4881] [54441] 2020-10-10 18:37:28 +03:00
alesapin
f950198fa5 Allow to use c11 with clang pedantic 2020-10-06 19:00:42 +03:00
Vitaly Baranov
dd634aa731
Merge pull request #15491 from vitlibar/update-grpc
Update gRPC to the latest version
2020-10-02 15:54:35 +03:00
alesapin
c5df9433f0
Merge pull request #15239 from ClickHouse/100_percent_lld_11_for_clang_11
Build ClickHouse for release with clang-11
2020-10-02 14:59:29 +03:00
Vitaly Baranov
65a1684ed7 Fix gRPC build scripts. 2020-10-02 01:45:01 +03:00
Vitaly Baranov
8a3b024a6d Fix using external protobuf library for the build. 2020-10-01 06:28:54 +03:00
tavplubix
a4b4895b26
Merge pull request #15435 from ClickHouse/ADQM-160-fix
Try merge #14265 again
2020-09-30 11:21:55 +03:00
Alexey Milovidov
b0e8119744 Merge branch 'master' into clang-tidy-11 2020-09-30 01:17:17 +03:00
alesapin
078f7075ba Fix cmake 2020-09-29 15:15:45 +03:00
Alexander Tokmakov
a7d3a024c7 Revert "Revert "Test and doc for PR12771 krb5 + cyrus-sasl + kerberized kafka""
This reverts commit c298c633a7.
2020-09-29 11:56:37 +03:00
tavplubix
c298c633a7
Revert "Test and doc for PR12771 krb5 + cyrus-sasl + kerberized kafka" 2020-09-24 16:10:24 +03:00
Ilya Yatsishin
7dad84ea6b
Merge pull request #14265 from arenadata/ADQM-160
Test and doc for PR12771 krb5 + cyrus-sasl + kerberized kafka
2020-09-23 19:02:58 +03:00
Ilya Golshtein
e2ee97202b USE_LIBGSASL not required to use internal librdkafka 2020-09-22 23:16:34 +03:00
alesapin
269d225637 Merge branch 'master' into myrrc-feature-cmake-flags-doc-generator 2020-09-22 17:00:02 +03:00
myrrc
eb1bb88fa0 updated the gtest find cmake file 2020-09-19 20:45:11 +03:00
myrrc
c78d9f9f38 moved the generators to docs/ folder 2020-09-19 19:42:36 +03:00
Alexey Milovidov
059646743b Remove OpenCL 2020-09-19 17:20:00 +03:00
Alexey Milovidov
cac65b38f2 Merge branch 'master' into clang-tidy-11 2020-09-19 00:02:44 +03:00
myrrc
f6bce8bd1e reverted USE to ENABLE 2020-09-18 17:18:59 +03:00
myrrc
3953ed485b added sub-tables in report, replaced some USE_ -> ENABLE_ for
consistency
2020-09-18 16:09:22 +03:00
Alexey Milovidov
799a675b5d Merge branch 'master' into AES_encrypt_decrypt 2020-09-17 21:42:51 +03:00
myrrc
7cd5c0d34f Merge remote-tracking branch 'upstream/master' into feature/cmake-flags-doc-generator 2020-09-17 19:17:09 +03:00
myrrc
ac606dca77 updated cmake/ files, added some comments 2020-09-17 18:37:23 +03:00
Vitaly Baranov
3356d75b23
Merge pull request #13156 from azat/cluster-secure
Secure inter-cluster query execution (with initial_user as current query user) [v3]
2020-09-17 17:11:00 +03:00
Alexey Milovidov
9ceb4934c2 Use clang-tidy 11 2020-09-17 13:32:16 +03:00
Simon Liu
c41e230589 Modify the minimum version of the Clang compiler that Clickhouse can support (8 -> 9),
The main reason is that in Clang 8.0 "#pragma clang attribute" cannot be used in the namesapce scope.
2020-09-16 22:14:30 +08:00
alesapin
a75254766e
Merge pull request #14609 from ClickHouse/update_to_gcc_10
Update default build to gcc-10
2020-09-15 09:49:43 +03:00
Azat Khuzhin
85b63ecadf Bump revision manually (to make it compatible with other version from this tag)
As @alexey-milovidov wrote.

P.S. I did not use ./utils/release/release_lib.sh (via ./release), since
it is pretty complex (it fetches tags and so on)

The following macros has been updated:
- VERSION_REVISION -- usually bumped with utils/release/release_lib.sh
  (interesting that even if there were no changes)
- DBMS_MIN_REVISION_WITH_INTERSERVER_SECRET
- DBMS_TCP_PROTOCOL_VERSION -- does not affect anything, some internal
  yandex stuff
2020-09-15 01:36:28 +03:00
alesapin
fb7fc28e6f
Update warnings.cmake 2020-09-14 12:20:43 +03:00
alesapin
8075ce2809
Update warnings.cmake 2020-09-12 15:42:32 +03:00
alesapin
e25b1da29f Disable -Wstringop-overflow for gcc-10 2020-09-11 13:53:26 +03:00
Azat Khuzhin
0f4fdcbf38 Pass -fsanitize-blacklist for TSAN only under clang (gcc does not support this)
And no such check for -fsnaitize=memory, since gcc does not support it
anyway.
2020-09-10 02:07:28 +03:00
alesapin
4ba8f8960b Increase frame-larger-than 2020-09-09 12:53:24 +03:00
robot-clickhouse
8def718661 Auto version update to [20.10.1.1] [54440] 2020-09-08 17:04:41 +03:00
robot-clickhouse
7bc3c43a38 Auto version update to [20.9.1.4585] [54439] 2020-09-08 17:04:06 +03:00
alesapin
b91f37169d Fix cmake 2020-09-08 10:09:40 +03:00
Vasily Nemkov
b147ffcd43 encrypt, aes_encrypt_mysql, decrypt, aes_decrypt_mysql functions
Functions to encrypt/decrypt any input data with OpenSSL's ciphers
with custom key, iv, and add (-gcm mode only).

_mysql versions are 100% compatitable with corresponding MySQL functions

Supported modes depend on OpenSSL version, but generally are:
aes-{128,192,56}-{ecb,cbc,cfb1,cfb8,cfb128,ofb,gcm}

Please note that in a -gcm mode a 16-byte tag is appended to the ciphertext
on encryption and is expected to be found at the end of ciphertext on decryption.

Added tests that verify compatibility with MySQL functions,
and test vectors for GCM mode from OpenSSL.

Added masking rules for aes_X funtions
Rules are installed by default to config.d/query_masking_rules.xml
2020-09-01 12:42:15 +03:00
robot-clickhouse
e2fa0eae2f Auto version update to [20.9.1.1] [54439] 2020-08-31 23:07:41 +03:00
robot-clickhouse
f63e5faac0 Auto version update to [20.8.1.4513] [54438] 2020-08-31 23:07:05 +03:00
alesapin
30cf59fc9a Revert version update 2020-08-31 22:51:56 +03:00
robot-clickhouse
59a3937adb Auto version update to [20.8.1.4513] [54438] 2020-08-31 19:41:50 +03:00
Konstantin Podshumok
a9d2de595d
xcmake: enbale ccache by default if it is installed
Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-27 14:27:00 +03:00
Konstantin Podshumok
8ce844a40d
cmake: when USE_STATIC_LIBRARIES prefer bundled brotli
even if UNBUNDLED=ON because libbrotli*.a and similar are not
available in most of the systems

Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:44 +03:00
Konstantin Podshumok
e296997b6c
cmake: fix potential curl linking problems
Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:44 +03:00
Konstantin Podshumok
68a921cdea
cmake: use bundled rdkafaka for in unbundled tests for now
Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:43 +03:00
Konstantin Podshumok
5097793872
cmake: Fix odbc in unbudled
Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:43 +03:00
Konstantin Podshumok
5d13c1f03f
cmake: unbundled: libxml2 requires liblzma when building static
Squashed:
- fixup hdfs3-libxml2

Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:43 +03:00
Konstantin Podshumok
ef858027d4
unbundled: fix detection of mariadb headers
Squashed:
- unbundled: add warning about mariadbclient dependencies

Signed-off-by: Konstantin Podshumok <kpp.live+signed@gmail.com>
2020-08-25 20:16:42 +03:00