Commit Graph

419 Commits

Author SHA1 Message Date
Anton Popov
7715afa595
Merge pull request #42618 from CurtizJ/refactor-data-part
Try to save `IDataPartStorage` interface
2022-10-27 22:20:44 +02:00
Anton Popov
c8199bc125
Merge branch 'master' into refactor-data-part 2022-10-25 14:31:05 +02:00
Robert Schulze
c119cd2f00
Merge branch 'master' into update-libcxx-to-15 2022-10-24 08:29:37 +02:00
Anton Popov
b40d9200d2 better semantic of constsness of DataPartStorage 2022-10-23 15:24:20 +00:00
Azat Khuzhin
3d400068fd tests/gtest_compressionCodec: fix UBSAN report to avoid test failure
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:43 +02:00
Azat Khuzhin
2474303aef tests/gtest_compressionCodec: Fix UBSAN report about signed integer overflow
UBSAN report:

    $ UBSAN_SYMBOLIZER_PATH=/usr/bin/llvm-symbolizer-14 UBSAN_OPTIONS=print_stacktrace=1 ./unit_tests_dbms --gtest_filter=*Gorilla*
    ../src/Compression/tests/gtest_compressionCodec.cpp:1216:47: runtime error: signed integer overflow: 23 * 100000000 cannot be represented in type 'int'
        #0 0x14f67fd1 in auto (anonymous namespace)::$_6::operator()(int) const::'lambda'(auto)::operator()<int>(auto) const build_docker/../src/Compression/tests/gtest_compressionCodec.cpp:1216:47
        #1 0x14f67fd1 in (anonymous namespace)::CodecTestSequence (anonymous namespace)::generateSeq<long, (anonymous namespace)::$_6::operator()(int) const::'lambda'(auto), int, int>((anonymous namespace)::$_6::operator()(int) const::'lambda'(auto), char const*, int, int) build_docker/../src/Compression/tests/gtest_compressionCodec.cpp:394:36
        #2 0x14f67fd1 in auto (anonymous namespace)::GCompatibilityTestSequence<long>() build_docker/../src/Compression/tests/gtest_compressionCodec.cpp:1224:12
        #3 0x14f3c7f3 in (anonymous namespace)::gtest_GorillaCodecTestCompatibility_EvalGenerator_() build_docker/../src/Compression/tests/gtest_compressionCodec.cpp:1227:1
        #4 0x14f6bdb5 in testing::internal::ParameterizedTestSuiteInfo<(anonymous namespace)::CodecTestCompatibility>::RegisterTests() build_docker/../contrib/googletest/googletest/include/gtest/internal/gtest-param-util.h:553:45
        #5 0x27d87988 in testing::internal::ParameterizedTestSuiteRegistry::RegisterTests() build_docker/../contrib/googletest/googletest/include/gtest/internal/gtest-param-util.h:726:24
        #6 0x27d87988 in testing::internal::UnitTestImpl::RegisterParameterizedTests() build_docker/../contrib/googletest/googletest/src/gtest.cc:2805:34
        #7 0x27d87988 in testing::internal::UnitTestImpl::PostFlagParsingInit() build_docker/../contrib/googletest/googletest/src/gtest.cc:5492:5
        #8 0x27d9d002 in void testing::internal::InitGoogleTestImpl<char>(int*, char**) build_docker/../contrib/googletest/googletest/src/gtest.cc:6499:22
        #9 0x14fd5495 in main build_docker/../src/Coordination/tests/gtest_coordination.cpp:2189:5
        #10 0x7f8c29005209  (/lib/x86_64-linux-gnu/libc.so.6+0x29209) (BuildId: 71a7c7b97bc0b3e349a3d8640252655552082bf5)
        #11 0x7f8c290052bb in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x292bb) (BuildId: 71a7c7b97bc0b3e349a3d8640252655552082bf5)
        #12 0x14ce356d in _start (/work1/azat/tmp/42190/unit_tests_dbms+0x14ce356d) (BuildId: 482550e3f8d45f06e8c7f8147f427ee798c1f645)

    SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../src/Compression/tests/gtest_compressionCodec.cpp:1216:47 in

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:43 +02:00
Azat Khuzhin
b8ab686561 tests/gtest_compressionCodec: fix for darwin
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:43 +02:00
Azat Khuzhin
4e76629aaf Fixes for -Wshorten-64-to-32
- lots of static_cast
- add safe_cast
- types adjustments
  - config
  - IStorage::read/watch
  - ...
- some TODO's (to convert types in future)

P.S. That was quite a journey...

v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:19 +02:00
Robert Schulze
820e6b4276
Build with libcxx(abi) 15 2022-10-20 10:52:43 +00:00
Alexey Milovidov
191158f93b
Merge pull request #42314 from HarryLeeIBM/hlee-s390x-t64-codec
Fix Codec T64 on s390x
2022-10-17 01:13:57 +02:00
Alexey Milovidov
bfd8811a4d
Update CompressionCodecT64.cpp 2022-10-15 00:32:37 +03:00
Alexey Milovidov
4608b70dda
Update CompressionCodecT64.cpp 2022-10-15 00:31:33 +03:00
HarryLeeIBM
6cdb7b5a94 Fix Codec T64 on s390x 2022-10-14 07:16:18 -07:00
Alexander Tokmakov
4175f8cde6 abort instead of __builtin_unreachable in debug builds 2022-10-07 21:49:08 +02:00
Robert Schulze
da5a2e2db0
Merge remote-tracking branch 'origin/master' into generated-file-cleanup
Physical merge conflicts:
- src/Common/ZooKeeper/ZooKeeperImpl.cpp
- src/Core/config_core.h.in
- src/Functions/FunctionsAES.h
- src/Functions/config_functions.h.in
- src/configure_config.cmake

Logical merge conflicts:
- Functions/tryDecrypt.cpp
2022-10-06 08:43:25 +00:00
root
4318ec125e Merge branch 'master' of https://github.com/MeenaRenganathan22/ClickHouse into OpenSSL_InHouse 2022-09-30 15:50:22 -07:00
Robert Schulze
f24fab7747
Fix some #include atrocities 2022-09-28 13:49:28 +00:00
Robert Schulze
fd86829824
Consolidate config_core.h into config.h
Less duplication, less confusion ...
2022-09-28 13:31:57 +00:00
Robert Schulze
78fc36ca49
Generate config.h into ${CONFIG_INCLUDE_PATH}
This makes the target location consistent with other auto-generated
files like config_formats.h, config_core.h, and config_functions.h and
simplifies the build of clickhouse_common.
2022-09-28 12:48:26 +00:00
Alexey Milovidov
730655d4fd Fix 8/9 of trash 2022-09-19 08:53:20 +02:00
Meena Renganathan
51d6611b96 Committing the ClickHouse core changes and other libraries to support OpenSSL. BoringSSL is still set as default 2022-09-12 09:05:38 -07:00
Anton Popov
7c12b448b8 Merge remote-tracking branch 'upstream/master' into HEAD 2022-09-08 01:52:52 +00:00
Nikita Taranov
7c4f42d014
Skip empty literals in lz4 decompression (#40142) 2022-09-06 13:58:26 +02:00
Anton Popov
9dda9658a8 Merge remote-tracking branch 'upstream/master' into HEAD 2022-09-02 12:48:27 +00:00
Antonio Andelic
e64436fef3 Fix typos with new codespell 2022-09-02 08:54:48 +00:00
HarryLeeIBM
d609da182b Fix Endian issue in Codec for s390x 2022-08-08 11:28:35 -07:00
Robert Schulze
a7734672b9
Use std::popcount, ::countl_zero, ::countr_zero functions
- Introduced with the C++20 <bit> header

- The problem with __builtin_c(l|t)z() is that 0 as input has an
  undefined result (*) and the code did not always check. The std::
  versions do not have this issue.

- In some cases, we continue to use buildin_c(l|t)z(), (e.g. in
  src/Common/BitHelpers.h) because the std:: versions only accept
  unsigned inputs (and they also check that) and the casting would be
  ugly.

(*) https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html
2022-07-31 15:16:51 +00:00
Robert Schulze
7d653b3bd8
Merge pull request #39495 from ClickHouse/mark-qpl-deflate-experimental
Mark new codec DEFLATE_QPL as experimental + cosmetics
2022-07-26 19:00:21 +02:00
Robert Schulze
2785783def
Mark DEFLAPT_QPL as 'experimental' codec + cosmetics 2022-07-24 20:10:11 +00:00
Ignat Loskutov
c2954b23cb
LZ4_decompress_faster.cpp: remove endianness-dependent code
Little-endian is little-endian no matter what the native endianness is.
2022-07-24 20:55:17 +08:00
Alexey Milovidov
2d12180f9e
Merge pull request #39195 from HarryLeeIBM/hlee-s390x-LZ4
Fix LZ4 decompression issue for s390x
2022-07-23 22:31:42 +03:00
Robert Schulze
654047dfbf
Fix compiler warning 2022-07-22 11:28:05 +00:00
jinjunzh
f663c1e5e2 fixed condition fault 2022-07-21 17:14:20 -04:00
HarryLeeIBM
d5727dfb44 Removed __BYTE_ORDER__ and fixed style issue 2022-07-21 13:09:45 -07:00
jinjunzh
aa8dae5f72 add nullptr check for flushAsynchronousDecompressRequests 2022-07-21 12:04:11 -04:00
Nikita Taranov
9dbfc354d1
Merge branch 'master' into hlee-s390x-LZ4 2022-07-20 17:15:31 +02:00
jinjunzh
9c440bec28 Merge branch 'dev_intel_iaa_deflate' of https://github.com/jinjunzh/ClickHouse into dev_intel_iaa_deflate 2022-07-19 17:50:23 -04:00
jinjunzh
94332d876a Fixed type&style issue 2022-07-19 17:48:41 -04:00
jinjunzh
b0c6fcf3a0 demoted log to INFO when the job pool is exhausted 2022-07-19 15:30:25 -04:00
jinjunzh
8f6cc6971b add comments for readBig 2022-07-19 14:24:00 -04:00
jinjunzh
3a7fa8887f refine readCompressedDataBlockForAsynchronous 2022-07-19 10:23:25 -04:00
jasperzhu
c044c67745
Merge branch 'master' into dev_intel_iaa_deflate 2022-07-19 15:32:57 +08:00
jinjunzh
a53ed21dba restore the function doDecompressDataSynchronous 2022-07-18 17:44:15 -04:00
jinjunzh
332d3fd1a7 add comments for doDecompressDataAsynchronous 2022-07-18 14:40:49 -04:00
jinjunzh
456ae5ea44 add comments for readCompressedDataBlockForAsynchronous 2022-07-18 10:39:20 -04:00
Anton Popov
5605b60c97
Merge pull request #39039 from CurtizJ/randomize-more-settings
Add more settings for randomization
2022-07-18 15:05:41 +02:00
Alexey Milovidov
f1256c3aef
Merge pull request #37891 from guowangy/lz4-decompress-vbmi
LZ4 decompress: add VBMI optimized copyOverlap32Shuffle
2022-07-18 00:37:59 +03:00
Wangyang Guo
7ef3f56066 lz4 decompress: coding style fix 2022-07-16 22:10:16 +08:00
jinjunzh
f05f3cb69a fixed cosmetic issues 2022-07-15 17:04:27 -04:00
jinjunzh
e3735f7464 fixed cosmetic issues 2022-07-15 16:12:26 -04:00