thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-17 20:02:05 +00:00
Code Issues Packages Projects Releases Wiki Activity
142,586 Commits 9,330 Branches 1,783 Tags 2.1 GiB
da1af2b357
Commit Graph

668 Commits

Author SHA1 Message Date
Anton Popov
c8199bc125
Merge branch 'master' into refactor-data-part 2022-10-25 14:31:05 +02:00
Robert Schulze
c119cd2f00
Merge branch 'master' into update-libcxx-to-15 2022-10-24 08:29:37 +02:00
Anton Popov
b40d9200d2 better semantic of constsness of DataPartStorage 2022-10-23 15:24:20 +00:00
Azat Khuzhin
3d400068fd tests/gtest_compressionCodec: fix UBSAN report to avoid test failure 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-21 13:25:43 +02:00
Azat Khuzhin
2474303aef tests/gtest_compressionCodec: Fix UBSAN report about signed integer overflow 
UBSAN report:

    $ UBSAN_SYMBOLIZER_PATH=/usr/bin/llvm-symbolizer-14 UBSAN_OPTIONS=print_stacktrace=1 ./unit_tests_dbms --gtest_filter=*Gorilla*
    ../src/Compression/tests/gtest_compressionCodec.cpp:1216:47: runtime error: signed integer overflow: 23 * 100000000 cannot be represented in type 'int'
        #0 0x14f67fd1 in auto (anonymous namespace)::$_6::operator()(int) const::'lambda'(auto)::operator()<int>(auto) const build_docker/../src/Compression/tests/gtest_compressionCodec.cpp:1216:47
        #1 0x14f67fd1 in (anonymous namespace)::CodecTestSequence (anonymous namespace)::generateSeq<long, (anonymous namespace)::$_6::operator()(int) const::'lambda'(auto), int, int>((anonymous namespace)::$_6::operator()(int) const::'lambda'(auto), char const*, int, int) build_docker/../src/Compression/tests/gtest_compressionCodec.cpp:394:36
        #2 0x14f67fd1 in auto (anonymous namespace)::GCompatibilityTestSequence<long>() build_docker/../src/Compression/tests/gtest_compressionCodec.cpp:1224:12
        #3 0x14f3c7f3 in (anonymous namespace)::gtest_GorillaCodecTestCompatibility_EvalGenerator_() build_docker/../src/Compression/tests/gtest_compressionCodec.cpp:1227:1
        #4 0x14f6bdb5 in testing::internal::ParameterizedTestSuiteInfo<(anonymous namespace)::CodecTestCompatibility>::RegisterTests() build_docker/../contrib/googletest/googletest/include/gtest/internal/gtest-param-util.h:553:45
        #5 0x27d87988 in testing::internal::ParameterizedTestSuiteRegistry::RegisterTests() build_docker/../contrib/googletest/googletest/include/gtest/internal/gtest-param-util.h:726:24
        #6 0x27d87988 in testing::internal::UnitTestImpl::RegisterParameterizedTests() build_docker/../contrib/googletest/googletest/src/gtest.cc:2805:34
        #7 0x27d87988 in testing::internal::UnitTestImpl::PostFlagParsingInit() build_docker/../contrib/googletest/googletest/src/gtest.cc:5492:5
        #8 0x27d9d002 in void testing::internal::InitGoogleTestImpl<char>(int*, char**) build_docker/../contrib/googletest/googletest/src/gtest.cc:6499:22
        #9 0x14fd5495 in main build_docker/../src/Coordination/tests/gtest_coordination.cpp:2189:5
        #10 0x7f8c29005209  (/lib/x86_64-linux-gnu/libc.so.6+0x29209) (BuildId: 71a7c7b97bc0b3e349a3d8640252655552082bf5)
        #11 0x7f8c290052bb in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x292bb) (BuildId: 71a7c7b97bc0b3e349a3d8640252655552082bf5)
        #12 0x14ce356d in _start (/work1/azat/tmp/42190/unit_tests_dbms+0x14ce356d) (BuildId: 482550e3f8d45f06e8c7f8147f427ee798c1f645)

    SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../src/Compression/tests/gtest_compressionCodec.cpp:1216:47 in

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-21 13:25:43 +02:00
Azat Khuzhin
b8ab686561 tests/gtest_compressionCodec: fix for darwin 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-21 13:25:43 +02:00
Azat Khuzhin
4e76629aaf Fixes for -Wshorten-64-to-32 
- lots of static_cast
- add safe_cast
- types adjustments
  - config
  - IStorage::read/watch
  - ...
- some TODO's (to convert types in future)

P.S. That was quite a journey...

v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-21 13:25:19 +02:00
Robert Schulze
820e6b4276
Build with libcxx(abi) 15 2022-10-20 10:52:43 +00:00
Alexey Milovidov
191158f93b
Merge pull request #42314 from HarryLeeIBM/hlee-s390x-t64-codec 
Fix Codec T64 on s390x
 2022-10-17 01:13:57 +02:00
Alexey Milovidov
bfd8811a4d
Update CompressionCodecT64.cpp 2022-10-15 00:32:37 +03:00
Alexey Milovidov
4608b70dda
Update CompressionCodecT64.cpp 2022-10-15 00:31:33 +03:00
HarryLeeIBM
6cdb7b5a94 Fix Codec T64 on s390x 2022-10-14 07:16:18 -07:00
Alexander Tokmakov
4175f8cde6 abort instead of __builtin_unreachable in debug builds 2022-10-07 21:49:08 +02:00
Robert Schulze
da5a2e2db0
Merge remote-tracking branch 'origin/master' into generated-file-cleanup 
Physical merge conflicts:
- src/Common/ZooKeeper/ZooKeeperImpl.cpp
- src/Core/config_core.h.in
- src/Functions/FunctionsAES.h
- src/Functions/config_functions.h.in
- src/configure_config.cmake

Logical merge conflicts:
- Functions/tryDecrypt.cpp
 2022-10-06 08:43:25 +00:00
root
4318ec125e Merge branch 'master' of https://github.com/MeenaRenganathan22/ClickHouse into OpenSSL_InHouse 2022-09-30 15:50:22 -07:00
Robert Schulze
f24fab7747
Fix some #include atrocities 2022-09-28 13:49:28 +00:00
Robert Schulze
fd86829824
Consolidate config_core.h into config.h 
Less duplication, less confusion ...
 2022-09-28 13:31:57 +00:00
Robert Schulze
78fc36ca49
Generate config.h into ${CONFIG_INCLUDE_PATH} 
This makes the target location consistent with other auto-generated
files like config_formats.h, config_core.h, and config_functions.h and
simplifies the build of clickhouse_common.
 2022-09-28 12:48:26 +00:00
Alexey Milovidov
730655d4fd Fix 8/9 of trash 2022-09-19 08:53:20 +02:00
Meena Renganathan
51d6611b96 Committing the ClickHouse core changes and other libraries to support OpenSSL. BoringSSL is still set as default 2022-09-12 09:05:38 -07:00
Anton Popov
7c12b448b8 Merge remote-tracking branch 'upstream/master' into HEAD 2022-09-08 01:52:52 +00:00
Nikita Taranov
7c4f42d014
Skip empty literals in lz4 decompression (#40142) 2022-09-06 13:58:26 +02:00
Anton Popov
9dda9658a8 Merge remote-tracking branch 'upstream/master' into HEAD 2022-09-02 12:48:27 +00:00
Antonio Andelic
e64436fef3 Fix typos with new codespell 2022-09-02 08:54:48 +00:00
HarryLeeIBM
d609da182b Fix Endian issue in Codec for s390x 2022-08-08 11:28:35 -07:00
Robert Schulze
a7734672b9
Use std::popcount, ::countl_zero, ::countr_zero functions 
- Introduced with the C++20 <bit> header

- The problem with __builtin_c(l|t)z() is that 0 as input has an
  undefined result (*) and the code did not always check. The std::
  versions do not have this issue.

- In some cases, we continue to use buildin_c(l|t)z(), (e.g. in
  src/Common/BitHelpers.h) because the std:: versions only accept
  unsigned inputs (and they also check that) and the casting would be
  ugly.

(*) https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html
 2022-07-31 15:16:51 +00:00
Robert Schulze
7d653b3bd8
Merge pull request #39495 from ClickHouse/mark-qpl-deflate-experimental 
Mark new codec DEFLATE_QPL as experimental + cosmetics
 2022-07-26 19:00:21 +02:00
Robert Schulze
2785783def
Mark DEFLAPT_QPL as 'experimental' codec + cosmetics 2022-07-24 20:10:11 +00:00
Ignat Loskutov
c2954b23cb
LZ4_decompress_faster.cpp: remove endianness-dependent code 
Little-endian is little-endian no matter what the native endianness is.
 2022-07-24 20:55:17 +08:00
Alexey Milovidov
2d12180f9e
Merge pull request #39195 from HarryLeeIBM/hlee-s390x-LZ4 
Fix LZ4 decompression issue for s390x
 2022-07-23 22:31:42 +03:00
Robert Schulze
654047dfbf
Fix compiler warning 2022-07-22 11:28:05 +00:00
jinjunzh
f663c1e5e2 fixed condition fault 2022-07-21 17:14:20 -04:00
HarryLeeIBM
d5727dfb44 Removed __BYTE_ORDER__ and fixed style issue 2022-07-21 13:09:45 -07:00
jinjunzh
aa8dae5f72 add nullptr check for flushAsynchronousDecompressRequests 2022-07-21 12:04:11 -04:00
Nikita Taranov
9dbfc354d1
Merge branch 'master' into hlee-s390x-LZ4 2022-07-20 17:15:31 +02:00
jinjunzh
9c440bec28 Merge branch 'dev_intel_iaa_deflate' of https://github.com/jinjunzh/ClickHouse into dev_intel_iaa_deflate 2022-07-19 17:50:23 -04:00
jinjunzh
94332d876a Fixed type&style issue 2022-07-19 17:48:41 -04:00
jinjunzh
b0c6fcf3a0 demoted log to INFO when the job pool is exhausted 2022-07-19 15:30:25 -04:00
jinjunzh
8f6cc6971b add comments for readBig 2022-07-19 14:24:00 -04:00
jinjunzh
3a7fa8887f refine readCompressedDataBlockForAsynchronous 2022-07-19 10:23:25 -04:00
jasperzhu
c044c67745
Merge branch 'master' into dev_intel_iaa_deflate 2022-07-19 15:32:57 +08:00
jinjunzh
a53ed21dba restore the function doDecompressDataSynchronous 2022-07-18 17:44:15 -04:00
jinjunzh
332d3fd1a7 add comments for doDecompressDataAsynchronous 2022-07-18 14:40:49 -04:00
jinjunzh
456ae5ea44 add comments for readCompressedDataBlockForAsynchronous 2022-07-18 10:39:20 -04:00
Anton Popov
5605b60c97
Merge pull request #39039 from CurtizJ/randomize-more-settings 
Add more settings for randomization
 2022-07-18 15:05:41 +02:00
Alexey Milovidov
f1256c3aef
Merge pull request #37891 from guowangy/lz4-decompress-vbmi 
LZ4 decompress: add VBMI optimized copyOverlap32Shuffle
 2022-07-18 00:37:59 +03:00
Wangyang Guo
7ef3f56066 lz4 decompress: coding style fix 2022-07-16 22:10:16 +08:00
jinjunzh
f05f3cb69a fixed cosmetic issues 2022-07-15 17:04:27 -04:00
jinjunzh
e3735f7464 fixed cosmetic issues 2022-07-15 16:12:26 -04:00
jinjunzh
5104e4200a fixed cosmetic issues 2022-07-15 15:32:10 -04:00
Nikita Taranov
ecdc56bf31
Merge branch 'master' into hlee-s390x-LZ4 2022-07-15 14:23:41 +02:00
Anton Popov
76eec62469
Merge branch 'master' into randomize-more-settings 2022-07-15 14:12:49 +02:00
Wangyang Guo
c9e15017ac lz4 decompress: dynamic dispatch with TargetSpecific 2022-07-15 16:07:19 +08:00
avogar
9291d33080 Pass const std::string_view & by value, not by reference 2022-07-14 16:11:57 +00:00
Anton Popov
d4cb2184c2
Merge branch 'master' into randomize-more-settings 2022-07-14 17:01:07 +02:00
jinjunzh
1c6d70f9b3 fixed sw job buffer missing 2022-07-13 16:12:15 -04:00
HarryLeeIBM
2caa3f8803 Fix LZ4 decompression issue for s390x 2022-07-13 11:45:58 -07:00
Wangyang Guo
f275b35bba Dynamic dispatching for best_variant selection 2022-07-13 15:20:15 +08:00
jinjunzh
b699764bbe unified sync implemetation with async 2022-07-12 14:32:53 -04:00
Anton Popov
3b45e6e347 fix reading of empty arrays in reverse order with uncompressed cache (addition to #36215) 2022-07-12 03:08:12 +00:00
Robert Schulze
1a7727a254
Prefix overridden add_executable() command with "clickhouse_" 
A simple HelloWorld program with zero includes except iostream triggers
a build of ca. 2000 source files. The reason is that ClickHouse's
top-level CMakeLists.txt overrides "add_executable()" to link all
binaries against "clickhouse_new_delete". This links against
"clickhouse_common_io", which in turn has lots of 3rd party library
dependencies ... Without linking "clickhouse_new_delete", the number of
compiled files for "HelloWorld" goes down to ca. 70.

As an example, the self-extracting-executable needs none of its current
dependencies but other programs may also benefit.

In order to restore access to the original "add_executable()", the
overriding version is now prefixed. There is precedence for a
"clickhouse_" prefix (as opposed to "ch_"), for example
"clickhouse_split_debug_symbols". In general prefixing makes sense also
because overriding CMake commands relies on undocumented behavior and is
considered not-so-great practice (*).

(*) https://crascit.com/2018/09/14/do-not-redefine-cmake-commands/
 2022-07-11 19:36:18 +02:00
jinjunzh
bcba581191 fixed cosmetic issues 2022-07-11 12:08:35 -04:00
mergify[bot]
62c62a2df7
Merge branch 'master' into lz4-decompress-vbmi 2022-07-11 14:23:17 +00:00
jinjunzh
bc03e10870 make job pool static member to private 2022-07-09 17:50:57 -04:00
jinjunzh
bc775ecc5b add assert for Illegal path 2022-07-09 17:29:47 -04:00
jinjunzh
044c14745e make job pool name comprehensible 2022-07-09 17:17:18 -04:00
jinjunzh
eb06c66869 cosmetic issue 2022-07-09 14:57:38 -04:00
jinjunzh
816e974ca2 rename deflate to deflate_qpl 2022-07-09 14:42:01 -04:00
jinjunzh
825b7511e4 improve cosmetics 2022-07-09 14:13:12 -04:00
jinjunzh
7cb5b35bef remove qpl from fasttest 2022-07-09 13:54:23 -04:00
jinjunzh
212f342af5 change to std random 2022-07-09 12:03:25 -04:00
jinjunzh
0fdce07cc6 add dedicated value for HW codec fail 2022-07-09 11:12:41 -04:00
jinjunzh
545e6b305c improve log print text 2022-07-09 10:09:23 -04:00
jinjunzh
46d7d3b5c9 constant always goes to the right side 2022-07-08 15:03:10 -04:00
jinjunzh
289c6148e4 member to be LOWERCASE_UNDERSCORE 2022-07-08 12:20:24 -04:00
jinjunzh
7eb71143db add a one-line method for resetting a job lock 2022-07-08 12:09:58 -04:00
jinjunzh
7a3c21174d use C++17 if initializer 2022-07-08 11:55:16 -04:00
jinjunzh
6045681bb4 improve jobPoolReady 2022-07-07 17:21:51 -04:00
jinjunzh
5681e50da8 improve job pool initialization 2022-07-07 17:13:20 -04:00
jinjunzh
2c8f308bbc improve buffer allocation 2022-07-07 16:05:52 -04:00
jinjunzh
29fee729e3 revise the way of get job size 2022-07-07 15:17:23 -04:00
jinjunzh
2765e2a259 remove unused function in deflate job pool 2022-07-07 14:44:42 -04:00
jinjunzh
f80aea9eac Cosmetic issue and improve flush function 2022-07-07 14:14:47 -04:00
jinjunzh
ad82bb80ed Cosmetic: Constants to be UPPERCASE_AND_UNDERSCORE 2022-07-07 12:10:06 -04:00
jinjunzh
85eae40765 remove inline due to negligible benefits 2022-07-07 10:26:57 -04:00
jinjunzh
43c5a68d14 Cosmetic work 2022-07-07 10:04:17 -04:00
jinjunzh
6fa036f6ee remove qpl from clickhouse-keeper 2022-07-06 23:48:09 -04:00
jinjunzh
3422c0479d add more comments for flush 2022-07-06 23:31:22 -04:00
jinjunzh
4cdc660406 change flush name to be more descriptive 2022-07-06 21:37:11 -04:00
jinjunzh
7b262b62c2 restore decompress to be const 2022-07-06 18:34:31 -04:00
jinjunzh
567f90d8a4 add CodecMode for deflate codec 2022-07-06 18:18:55 -04:00
jinjunzh
fbe3d84860 revise typo issue 2022-07-05 15:29:53 -04:00
jinjunzh
ba38969e9c enable qpl for clickhouse-keeper build 2022-07-05 15:28:41 -04:00
jinjunzh
fe451a4317 remove compress sync interface 2022-06-28 22:52:11 -04:00
jasperzhu
b2f98fa73b
Merge branch 'master' into dev_intel_iaa_deflate 2022-06-27 10:21:21 +08:00
alesapin
50801e41c5 Merge branch 'master' into refactor-something-in-part-volumes 2022-06-19 14:05:46 +02:00
Anton Popov
1523c9c9e5 fix filling of empty Nested + small refactoring 2022-06-17 01:10:52 +00:00
Anton Popov
92b7b9789a
try to fix fpc codec 2022-06-16 03:41:09 +02:00
jasperzhu
bca18b2caf
Merge branch 'master' into dev_intel_iaa_deflate 2022-06-16 08:56:27 +08:00
Nikolai Kochetov
2a9a63ac7b Merge branch 'master' into refactor-something-in-part-volumes 2022-06-15 15:35:26 +02:00
Robert Schulze
a0d936cc9f
Small follow-up for FPC codec 
- add paper reference + doxygen

- remove endianness handling (ClickHouse assumes little endian)

- documentation

- other minor stuff
 2022-06-15 14:21:28 +02:00
jinjunzh
1236d74ae7 restructure code design to be compatible both of hardware and software codec 2022-06-08 11:47:44 -04:00
jinjunzh
2b684ff40e remove sw job pool 2022-06-08 09:28:35 -04:00
koloshmet
d5064dd5b5 so many improvements 2022-06-08 02:17:34 +03:00
Wangyang Guo
50ac01c9b8 Enable copyOverlap32Shuffle 2022-06-06 13:45:19 +08:00
Wangyang Guo
c9ff52f2e9 Add VBMI optimized copyOverlap32Shuffle 2022-06-06 13:45:01 +08:00
jinjunzh
c7ca746994 fixed build issues in clickhouse-keeper 2022-06-05 14:46:28 -04:00
jinjunzh
2170a99484 revise ENABLE_QPL_COMPRESSION build option 2022-06-03 21:06:55 -04:00
Nikolai Kochetov
89c5855d20 Merge branch 'master' into refactor-something-in-part-volumes 2022-06-02 12:19:07 +02:00
jinjunzh
e1426c7e9d add build option for iaa compression 2022-06-01 11:47:43 -04:00
jinjunzh
53be924537 fixed invalid case style 2022-05-31 15:40:31 -04:00
jasperzhu
fc03af2259
Merge branch 'master' into dev_intel_iaa_deflate 2022-05-30 17:49:04 +08:00
koloshmet
7e69779575 added fpc codec to float perftest 2022-05-26 22:32:56 +03:00
koloshmet
821100f145 code style fixes 2022-05-26 11:09:36 +03:00
koloshmet
1bfeb982af fixed decoding parameters 2022-05-26 06:58:34 +03:00
koloshmet
cd3e28e3b1 fixed decoding parameters 2022-05-26 06:53:48 +03:00
koloshmet
adf888811c fixed max size computation 2022-05-26 04:53:07 +03:00
koloshmet
0697b9ffcf FPC codec 2022-05-25 22:04:39 +03:00
Maksim Kita
168814ed78
Merge pull request #37442 from kitaisreal/compressed-write-buffer-added-comment 
CompressedWriteBuffer added comment
 2022-05-23 23:44:33 +02:00
Maksim Kita
3a22d8ef2c CompressedWriteBuffer added comment 2022-05-23 12:09:44 +02:00
Robert Schulze
0f6715bd91
Follow-up to PR #37300: semicolon warnings 
In PR #37300, Alexej asked why we the compiler does not warn about
unnecessary semicolons, e.g.

  f()
  {
  }; // <-- here

The answer is surprising: In C++98, above syntax was disallowed but by
most compilers accepted it regardless. C++>11 introduced "empty
declarations" which made the syntax legal.

The previous behavior can be restored using flag
-Wc++98-compat-extra-semi. This finds many useless semicolons which were
removed in this change. Unfortunately, there are also false positives
which would require #pragma-s and HAS_* logic (--> check_flags.cmake) to
suppress. In the end, -Wc++98-compat-extra-semi comes with extra effort
for little benefit. Therefore, this change only fixes some semicolons
but does not enable the flag.
 2022-05-20 15:06:34 +02:00
Maksim Kita
495e04d2c3
Merge pull request #37242 from jinjunzh/insert_improve_jsp 
Improve CompressedWriteBuffer to avoid unnecessary memcpy
 2022-05-19 10:59:35 +02:00
jinjunzh
f413b49364 fixed logger header file path 2022-05-16 14:59:16 -04:00
Maksim Kita
fd28c19c1c
Update CompressedWriteBuffer.cpp 2022-05-16 20:44:22 +02:00
jinjunzh
f334cd371f fixed code style issue 2022-05-16 17:21:20 -04:00
jinjunzh
2ae9e22248 Improve CompressedWriteBuffer to avoid unnecessary memcpy 2022-05-16 16:22:13 -04:00
jinjunzh
9309735016 remove unused job release call 2022-05-10 16:46:50 -04:00
Robert Schulze
1b81bb49b4
Enable clang-tidy modernize-deprecated-headers & hicpp-deprecated-headers 
Official docs:

  Some headers from C library were deprecated in C++ and are no longer
  welcome in C++ codebases. Some have no effect in C++. For more details
  refer to the C++ 14 Standard [depr.c.headers] section. This check
  replaces C standard library headers with their C++ alternatives and
  removes redundant ones.
 2022-05-09 08:23:33 +02:00
Nikolai Kochetov
35095191eb Merge branch 'master' into refactor-something-in-part-volumes 2022-05-03 17:51:47 +02:00
jasperzhu
7f47e9b934
Merge branch 'master' into dev_intel_iaa_deflate 2022-05-03 15:47:57 +08:00
jasperzhu
f4b275c4ac
Improve decompression in readbig to avoid unnecessary memcpy (#36095) 2022-05-02 15:06:52 +02:00
Amos Bird
4a5e4274f0
base should not depend on Common 2022-04-29 10:26:35 +08:00
jinjunzh
8788185e82 fixed typos issue in code 2022-04-27 10:52:44 -04:00
Nikolai Kochetov
e44af67fee Merge branch 'master' into refactor-something-in-part-volumes 2022-04-26 21:08:00 +02:00
jasperzhu
05769b4a9c
Merge branch 'master' into dev_intel_iaa_deflate 2022-04-26 16:30:27 +08:00
jinjunzh
68a7e4b9b5 preliminary patch for iaa deflate solution 2022-04-26 14:14:09 -04:00
Nikolai Kochetov
8c00692844 Part 8 2022-04-22 16:58:09 +00:00
Robert Schulze
b24ca8de52
Fix various clang-tidy warnings 
When I tried to add cool new clang-tidy 14 warnings, I noticed that the
current clang-tidy settings already produce a ton of warnings. This
commit addresses many of these. Almost all of them were non-critical,
i.e. C vs. C++ style casts.
 2022-04-20 10:29:05 +02:00
Anton Popov
0595b5c22b fix reading of empty arrays in reverse order 2022-04-13 21:50:57 +00:00
Nikolai Kochetov
5a1392a8e3 Try refactor something (1) 2022-04-05 19:12:48 +00:00
alesapin
77e700b1cf
Update src/Compression/CompressionFactory.cpp 2022-03-28 11:25:08 +02:00
alesapin
a4ab73619f
Fix UBSan build 2022-03-22 15:05:31 +01:00
Maksim Kita
65c52298b6 Fix clang-tidy warnings in Compression, Coordination, Core folders 2022-03-14 18:17:35 +00:00
kssenii
5260822964 Merge master 2022-03-08 18:21:28 +01:00
kssenii
e231c3a3e0 Fix split build 2022-03-08 18:05:55 +01:00
Maksim Kita
b1a956c5f1 clang-tidy check performance-move-const-arg fix 2022-03-02 18:15:27 +00:00
Amos Bird
ba19c7cf44
Slightly better interface of compressed buffer 2022-02-17 14:31:22 +08:00
alesapin
02a93cb852 Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-08 19:42:27 +03:00
Anton Popov
1b16db72c3 fix consecutive backward seeks in seekable read buffers 2022-02-07 17:20:26 +03:00
alesapin
ba28c2c013 Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-07 12:44:56 +03:00
Anton Popov
10b8684003 fix rare bug in reading of empty arrays 2022-02-06 20:45:59 +00:00
alesapin
18f08ed932
Revert "Revert "Add func tests run with s3"" 2022-02-01 10:20:06 +03:00
alexey-milovidov
cbfcd45be3
Revert "Add func tests run with s3" 2022-02-01 05:46:13 +03:00
alesapin
5be3968526 Fix bug in cache compressed read buffer 2022-01-31 13:40:26 +03:00
alesapin
ea8ec87b34 Fix stupid bug 2022-01-30 21:03:12 +03:00
Azat Khuzhin
a773e7ff01 Remove unbundled libpqxx support 2022-01-20 10:01:59 +03:00
Azat Khuzhin
8485abd52b Remove unbundled lz4 support 2022-01-20 08:47:16 +03:00
Anton Popov
54f51444c0 Merge remote-tracking branch 'upstream/master' into HEAD 2021-12-01 15:49:02 +03:00
Anton Popov
120cb79bac
Merge pull request #31826 from CurtizJ/fix-uncompressed-cache 
Fix reading with uncompressed cache
 2021-11-27 17:36:52 +03:00
alexey-milovidov
83391977f8
Merge pull request #31896 from Algunenano/parser_deps 
Reduce the files that depend on parser headers
 2021-11-27 00:18:31 +03:00
Raúl Marín
91bf938075 Reduce dependencies on ASTLiteral.h 
590 -> 537
 2021-11-26 17:54:57 +01:00
Raúl Marín
051dddd8df Reduce dependencies on ASTIdentifier.h 
Goes from rebuilding 483 objects to 165 when it's modified
 2021-11-26 16:49:40 +01:00
Anton Popov
923159d408 fix typos 2021-11-26 17:47:40 +03:00
Anton Popov
c3f076fd84 fix reading with uncompressed cache 2021-11-25 21:20:03 +03:00
Kruglov Pavel
d9c1a0c8ec
Merge branch 'master' into fix-write-buffers 2021-11-20 17:48:24 +03:00
Vladimir Smirnov
48451182f8 Initial support for risc-v 
Make ClickHouse compilable and runnable on risc-v 64

So far only basic functionality was tested (on real hw),
clickhouse server runs, exceptions works, client works,
simple tests works.

What doesn't work:
 1. traces - they are always empty
 2. system.stack_trace only have first frame
 2021-11-11 19:23:34 +01:00
avogar
51831afff8 Fix tests 2021-11-11 20:27:23 +03:00
avogar
c521a9131a Small refactoring of WriteBiffer-s 2021-11-11 02:11:18 +03:00
Anton Popov
d50137013c Merge remote-tracking branch 'upstream/master' into HEAD 2021-11-01 16:55:53 +03:00
Anton Popov
0099dfd523 refactoring of SerializationInfo 2021-10-29 20:21:02 +03:00
Filatenkov Artur
44b5dd1161
Refactoring in codec encrypted (#30564) 
* refactoring

* remove mistake in docs
 2021-10-29 11:44:28 +03:00
kssenii
7e2ea97e3c Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-28 18:46:21 +03:00
Alexey Milovidov
8b4a6a2416 Remove cruft 2021-10-28 02:10:39 +03:00
kssenii
52296e8b04 Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-25 00:22:32 +03:00
Azat Khuzhin
258b62d375 Remove one unused CompressedReadBufferFromFile ctor 2021-10-24 18:58:22 +03:00
Azat Khuzhin
93c34824c0 Fix explicit 0 size for createReadBufferFromFileBase() 2021-10-24 18:58:22 +03:00
kssenii
39b9e9c258 Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-22 15:40:41 +03:00
kssenii
56f0560c46 Asserts and read till end option 2021-10-21 20:54:03 +03:00
kssenii
31cd71c849 Clean up 2021-10-20 09:37:18 +03:00
kssenii
143e912e4e Fix split build 2021-10-19 22:45:46 +03:00
kssenii
49106f407f Fix some checks 2021-10-19 09:34:27 +03:00
kssenii
d5d4817350 Add disk proxies back, add setReadUntilPosition to ReadBuffer 2021-10-18 18:35:11 +03:00
kssenii
3995506d37 Adjust range reader for remote fs reads 2021-10-15 11:36:26 +03:00
Anton Popov
92413aed68 better interfaces for IDataType and ISerialization 2021-10-14 05:36:49 +03:00
Kseniia Sumarokova
185d58a7cc
Merge pull request #29766 from ClickHouse/remove_redundant_seeks 
Less seeks in compressed buffers
 2021-10-07 17:04:50 +03:00
alesapin
621c9884fc Account offset in readBig call 2021-10-06 12:39:54 +03:00
Filatenkov Artur
dc657a2c5e
Add error for multiple keys without current_key_id (#29546) 
* add error for multiple keys without current_key_id

* add exception if one key with non zero id is used without current_key_id

* refactoring
 2021-10-05 14:37:54 +03:00
alesapin
64158c011b Remove logging 2021-10-05 13:12:31 +03:00
alesapin
c2f34928e0 Fixup 2021-10-05 13:04:44 +03:00
alesapin
3a8126372c Remove redundant header 2021-10-05 12:19:04 +03:00
alesapin
8186339307 Less seeks in compressed buffers 2021-10-05 12:11:25 +03:00
Denis Glazachev
ca7b69b0f3 Manipulate with -Wreserved-identifier only if HAS_RESERVED_IDENTIFIER has been detected 2021-10-03 17:42:36 +04:00
Mike Kot
65e6e211b4 Merge remote-tracking branch 'upstream/master' into improvement/fn-traits 2021-10-02 21:37:50 +02:00
Maksim Kita
395119339f Merge branch 'master' into rename-common 2021-10-02 17:15:10 +03:00
Mike Kot
5d38930a3c Fixing gcc build due to a bug in compiler 2021-10-02 14:27:52 +02:00
Alexey Milovidov
2ad5dbdfe8 Rename "common" to "base" 2021-10-02 11:30:38 +03:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
alexey-milovidov
f21ea8f160
Update CompressedReadBufferBase.cpp 2021-10-02 06:13:46 +03:00
Yatsishin Ilya
2c0071bc6d wip 2021-10-01 09:32:54 +03:00
Alexey Milovidov
cd7f9d981c Remove ya.make 2021-09-25 04:22:54 +03:00
Nikita Mikhaylov
58f2504a56 Update ya.make 2021-09-14 12:00:32 +00:00
Nikita Mikhaylov
4b3c0a95d1
Merge pull request #28046 from syominsergey/fix-build-clang-13-2 
[WIP] Improve support for build with clang-13
 2021-09-13 12:11:38 +03:00
Filatenkov Artur
c23fe5baf6
Improve codec for encr 19896 (#27476) 
* change syntax of encrypted command

* commit all encrypted changes

* correct encryption

* correct config for test

* add tests and correct code style and typos

* correct test

* fix unbundled build

* add log warning messages

* improve code according to review comments

* correct nonce

* correct errors found by fuzzing

* improve codec AES_128_GCM_SIV. Add AES_256_GCM_SIV. Add sections for last in tests. Improve documentation

* Update CompressionCodecEncrypted.h

* Update 01683_codec_encrypted.sql

* correct compression factory after changes in master

* correct behavior with wrong key in data

* correct fuzzer

* add connection for fuzzer with fix for compression_encrypted

* refactor code

* add load from config with throwing errors on server start

* fix typos and check style

* Update Server.cpp

* correct loading and reading

* refactor code. fix uninitialized value

* refactor code

* move defines from server to cpp file

* correct build

* remove repeated code

* correct namespace

* fix code style
 2021-09-13 11:25:36 +03:00
Sergei Semin
12e24a23d3
Merge branch 'master' into fix-build-clang-13-2 2021-09-12 19:30:55 +03:00
alexey-milovidov
60f76d9254
Merge pull request #28759 from nikitamikhaylov/enable-gcc-11 
Switch CI to GCC-11
 2021-09-12 15:56:57 +03:00
alexey-milovidov
ea13a8b562
Merge pull request #28659 from myrrc/improvement/tostring_to_magic_enum 
Improving CH type system with concepts
 2021-09-12 15:26:29 +03:00
Mike Kot
1f6fb3f6ea Fix gtest, add Decimal aliases, Typelist solution for bin arithm 2021-09-12 14:26:07 +02:00
Sergei Semin
f6b613bc6b ignore diagnostic -Wreserved-identifier in CompressionCodecGorilla.cpp and CompressionCodecDoubleDelta.cpp 2021-09-09 23:51:47 +03:00
Nikita Mikhaylov
dcf7cb4a5c Done 2021-09-08 17:33:40 +00:00
alesapin
5c75b93fe8 Revert one warning 2021-09-06 12:17:31 +03:00
alesapin
d1e91a7860 Revert "Revert "Fix tidy"" 
This reverts commit 2e5e017d6d.
 2021-09-06 12:16:52 +03:00
alesapin
2e5e017d6d Revert "Fix tidy" 
This reverts commit 73ef1233ef.
 2021-09-06 10:52:20 +03:00
alesapin
73ef1233ef Fix tidy 
Fix tidy one more time
 2021-09-06 10:22:21 +03:00
Alexey Milovidov
ca6f6306f3 Merge branch 'master' into async-reads 2021-08-31 02:52:58 +03:00
Dmitrii Kovalkov
9871ad70ff Exclude fuzzers 2021-08-30 11:12:25 +03:00
Dmitrii Kovalkov
8acf0fa16a run generate-ya-make 2021-08-30 10:57:40 +03:00
Alexey Milovidov
3cb9cf7242 Merge branch 'master' of github.com:yandex/ClickHouse into async-reads 2021-08-26 01:55:46 +03:00
Nikita Mikhaylov
f99d7d89b8 Added static assert 2021-08-25 21:11:45 +00:00
Nikita Mikhaylov
28d70dae43 Deleted printing exception message 2021-08-25 20:58:53 +00:00
Nikita Mikhaylov
923d6759c0 Fix style 2021-08-25 15:40:56 +00:00
Nikita Mikhaylov
c5ca63c606 Fix performance (probably) 2021-08-25 15:40:56 +00:00
Nikita Mikhaylov
de8aa64e56 Deleted file 2021-08-25 15:40:56 +00:00
Nikita Mikhaylov
b333e7c744 Better 2021-08-25 15:40:56 +00:00
Nikita Mikhaylov
02e097a262 Fix tests 2021-08-25 15:40:56 +00:00
Nikita Mikhaylov
43fb3098ae Deleted some unneded changes 2021-08-25 15:40:56 +00:00
Nikita Mikhaylov
3538f1520b Build fixes after merge 2021-08-25 15:40:56 +00:00
Nikita Mikhaylov
08fbab09b0 Fix some issues 2021-08-25 15:40:56 +00:00
Nikita Mikhaylov
2c09533c83 lower a binary size for compression fuzzers 2021-08-25 15:40:56 +00:00
Nikita Mikhaylov
82a0c9d4df Fix build 2021-08-25 15:40:56 +00:00
Nikita Mikhaylov
ad4966f6d2 better 2021-08-25 15:40:56 +00:00
Nikita Mikhaylov
e63da5969f First blood 2021-08-25 15:40:15 +00:00
Alexey Milovidov
4d9ad3725d Merge branch 'master' into async-reads 2021-08-25 00:45:40 +03:00
Nikita Mikhaylov
80ebc6ccea Done 2021-08-22 15:09:49 +00:00
Nikita Mikhaylov
cfa571cac4
Merge pull request #27895 from nikitamikhaylov/temporary_remove_yaml_fuzzer 
Build fuzzers with clang-tidy
 2021-08-21 16:51:31 +03:00
Nikita Mikhaylov
64821c41d9 Style 2021-08-21 13:00:15 +00:00
Nikita Mikhaylov
3cd689c168 Moved fuzzers to another directory 2021-08-21 02:27:27 +00:00
Nikita Mikhaylov
f72457fa98 Lower the size of the binaries 2021-08-21 02:20:23 +00:00
Nikita Mikhaylov
2272c9896c
Update LZ4_decompress_faster.cpp 2021-08-19 16:53:39 +03:00
Nikita Mikhaylov
1e195cbacf
Update LZ4_decompress_faster.cpp 2021-08-19 14:48:10 +03:00
Nikita Mikhaylov
60dd53784f better 2021-08-16 12:46:59 +00:00
Nikita Mikhaylov
d25d12d7d4 better 2021-08-16 12:30:02 +00:00
Alexey Milovidov
d184b79bba Progress on async reads. 2021-08-16 03:00:32 +03:00
Alexey Milovidov
0cd3f25ba4 Merge remote-tracking branch 'origin/master' into async-reads 2021-08-16 02:41:01 +03:00
Alexey Milovidov
020beb885d Merge branch 'master' into async-reads 2021-08-15 12:08:19 +03:00
alexey-milovidov
86694a2bbb
Update CompressedWriteBuffer.h 2021-08-15 12:04:38 +03:00
Alexey Milovidov
9ae92798d1 Fix stress test in ~CompressedWriteBuffer 2021-08-15 11:44:12 +03:00
Alexey Milovidov
97d921d4da Remove trash 2021-08-15 11:38:16 +03:00
Nikolai Kochetov
a1ec7f75c5 Merge branch 'master' into qoega-fix-access-gtest-in-arcadia 2021-08-10 11:31:47 +03:00
Alexey Milovidov
cea9690327 Merge branch 'master' into async-reads 2021-08-05 23:58:57 +03:00
Yatsishin Ilya
9539ef18d2 fix 2021-08-04 11:55:51 +03:00
Yatsishin Ilya
6e74728714 more sync fixes 2021-08-04 10:58:39 +03:00
Nikita Mikhaylov
6d83eacec4 typo 2021-08-03 12:27:03 +00:00
Nikita Mikhaylov
2aea1c8d4a done 2021-08-03 12:24:16 +00:00
Alexey Milovidov
2a0cba2b9f Merge branch 'master' into async-reads 2021-08-01 01:25:13 +03:00
PHO
6425dd001a
Add a codec AES_128_GCM_SIV for encrypting columns on disk (#19896) 
* Add a codec Encrypted() for encrypting columns on disk

While this is implemented as a compression codec, it does not actually compress data. It instead encrypts data on disk. The key is obtained by executing a user-specified command at the server startup, or if it's not specified the codec refuses to process any data. For now the only supported cipher is 'AES-128-GCM-SIV'.
 2021-07-30 12:12:33 +03:00
Alexey Milovidov
5c3f5cb35a Experiment with asynchronous readers 2021-07-26 03:34:36 +03:00
Alexey Milovidov
21fb6ddea5 Remove AIO for sequential reads 2021-07-05 23:02:24 +03:00
Stas Kelvich
9a84033db5
Fix typo in hardware failure error message 2021-06-22 12:48:15 +03:00
Yuriy Chernyshov
c0cb5387f4 Use zstd/include in ya.make 2021-06-09 23:02:39 +03:00
Alexey Milovidov
8d85145725 Remove experimental compression codecs 2021-06-06 02:26:52 +03:00
Alexey Milovidov
273226de32 Remove string parameter for Density 2021-05-24 06:43:25 +03:00
Alexey Milovidov
4290b48a61 Fix clang-tidy 2021-05-24 06:00:57 +03:00
Alexey Milovidov
5412242908 Fix error 2021-05-24 02:38:46 +03:00
Alexey Milovidov
70d9f5c154 Fix error 2021-05-24 02:36:50 +03:00
Alexey Milovidov
8d2fa6cf50 Allow to build without experimental compression libraries and remove from fasttest 2021-05-24 01:18:12 +03:00
Alexey Milovidov
57a3f132d7 Fix style 2021-05-24 00:48:54 +03:00
Alexey Milovidov
40d4f0678f Remove overload (harmful) 2021-05-23 04:25:06 +03:00
Alexey Milovidov
fdc5bb72f5 Minor modification 2021-05-23 04:14:29 +03:00
Alexey Milovidov
3057bbe831 Remove header files 2021-05-23 04:12:30 +03:00
Alexey Milovidov
deb68b15da Fix bad dependencies in code 2021-05-23 04:03:42 +03:00
Alexey Milovidov
77414b50e6 Remove copy-paste 2021-05-23 03:55:16 +03:00
Alexey Milovidov
1a091c7a17 Fix unbalanced whitespace 2021-05-23 03:32:22 +03:00
Alexey Milovidov
a25edc192c Fix whitespace 2021-05-23 03:31:58 +03:00
Alexey Milovidov
b0df9286f0 Fix whitespace 2021-05-23 03:30:53 +03:00
Alexey Milovidov
ecc2e98265 Fix wrong intent in exception message 2021-05-23 03:30:17 +03:00
Alexey Milovidov
4cfabd2675 Fix style, fix typo and remove extra code 2021-05-23 03:21:48 +03:00
Alexey Milovidov
0da00f6497 Fix style, fix typo and remove extra code 2021-05-23 03:18:24 +03:00
Alexey Milovidov
e330fe4bb5 Fix extra whitespace in exception messages 2021-05-23 03:15:11 +03:00
alexey-milovidov
1ed30f7246
Merge branch 'master' into issue-16775 2021-05-22 17:14:23 +03:00
alesapin
2b62ce9044 Add smoke test for local address 2021-05-18 11:35:51 +03:00
Alexey Milovidov
604daa9581 Fix a bunch of warnings from PVS-Studio 2021-05-08 19:09:17 +03:00
Alexey Milovidov
45dea86395 More fixes for PVS-Studio 2021-05-08 18:35:09 +03:00
Alexey Milovidov
47a4c101b9 Merge branch 'master' into normalize-bigint 2021-05-08 16:41:15 +03:00
mwish
9328ec9c50 [Init] initialize commit 2021-05-08 18:07:39 +08:00
fibersel
f01b53bafb add newline 2021-05-06 16:47:23 +03:00
fibersel
94b159c510 add newlines 2021-05-06 16:34:27 +03:00
fibersel
cb53bbb7b0 add experimental codecs flag, add integration test for experimental codecs 2021-05-06 14:57:22 +03:00
Alexey Milovidov
1fda866881 Merge branch 'master' into normalize-bigint 2021-05-02 21:24:45 +03:00
Maksim Kita
318c4bb80d Add examples folder filter to ya.make.in 2021-04-30 11:25:52 +03:00
Nikita Mikhaylov
9f55424250 move to examples everywhere 2021-04-27 01:51:42 +03:00
Alexey Milovidov
649550c5ab Attempt to normalize big integers 2021-04-25 12:30:43 +03:00
Alexey Milovidov
8f01af62d9 Merge branch 'master' into normalize-bigint 2021-04-25 06:57:44 +03:00
Anton Popov
ea82e7725f
Merge pull request #21562 from CurtizJ/serialization-refactoring-4 
Refactoring of data types serialization
 2021-03-29 16:36:44 +03:00
alexey-milovidov
7ab2890c4d
Merge pull request #22206 from ClickHouse/mmap-cache 
Add cache for mmap IO
 2021-03-29 05:17:55 +03:00
Maksim Kita
459d00f999 Fixed tests 2021-03-28 22:42:34 +03:00
Alexey Milovidov
2a8ac01cdb Rename as suggested by Kita 2021-03-28 22:24:28 +03:00
Maksim Kita
608d37deed CachedCompressedReadBuffer fix cache usage 2021-03-28 21:32:38 +03:00
Alexey Milovidov
50f712e198 Integrate mmap cache to the infrastructure 2021-03-28 04:10:30 +03:00
Alexey Milovidov
0e3571478d Code review changes 2021-03-27 02:42:22 +03:00
Alexey Milovidov
e55f7e6333 Fix error 2021-03-26 02:21:59 +03:00