thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-17 11:52:27 +00:00
Code Issues Packages Projects Releases Wiki Activity
142,586 Commits 9,326 Branches 1,784 Tags 2.1 GiB
da1af2b357
Commit Graph

668 Commits

Author SHA1 Message Date
Александр Нам
5af6a89ee6 Style 2023-08-09 10:51:55 +00:00
Alexey Milovidov
aa757490bd Ditch tons of garbage 2023-08-09 02:19:02 +02:00
Александр Нам
f52d4b5ccf Style 2023-08-08 17:40:16 +00:00
Александр Нам
f533eeaf95 Moved registerCodecGCD to ifndef 2023-08-08 17:26:01 +00:00
Александр Нам
bdc3cdbb67 Style 2023-08-08 17:11:32 +00:00
Александр Нам
0a4c3f1dac Fix 2023-08-08 15:16:31 +00:00
Александр Нам
90d0c48108 Edited codec-files 2023-08-07 16:43:46 +00:00
Александр Нам
181e214a8a Added Codec-file and shit-coded CompressionMethodByte 2023-08-07 16:20:36 +00:00
Yakov Olkhovskiy
8abcc9104e
Merge branch 'master' into hlee-s390x-codec-delta 2023-08-01 19:02:57 -04:00
Alexey Milovidov
caa4590361 Merge branch 'master' into check-for-hiding-cyrillic-characters 2023-07-30 02:16:56 +02:00
Alexey Milovidov
6aab4cc835 Check for unexpected cyrillic 2023-07-27 05:25:40 +02:00
Robert Schulze
aa25ce9e3d
Follow-up to "Implement support of encrypted elements in configuration file" 
Cf. PR #50986

- rename XML attribute "encryption_codec" to "encrypted_by"
 2023-07-26 13:27:13 +00:00
Robert Schulze
d4737ca033
Merge pull request #50986 from arenadata/ADQM-822 
Implement support of encrypted elements in configuration file
 2023-07-26 12:27:04 +02:00
HarryLeeIBM
5ab6b34e11 Fix codec delta endian issue for s390x 2023-07-25 13:27:56 -07:00
Roman Vasin
0aed62ec73 Add codec name into exception message 2023-07-21 13:03:25 +00:00
Roman Vasin
c8c6c31384 Change Method into method in exceptions 2023-07-18 15:33:50 +00:00
Roman Vasin
59570b7e20 Make encryptValue and decryptValue static 2023-07-18 15:07:02 +00:00
jinjunzh
91b1dd6f78 define a function call for zero filling 2023-07-18 11:00:03 -04:00
jinjunzh
e8b3310319 touch buffer in advance to avoid page fault 2023-07-17 14:35:07 -04:00
Nikita Mikhaylov
9a30aaa9aa
Revert "Fixed several issues found by OSS-Fuzz" 2023-07-17 16:14:58 +02:00
Nikita Mikhaylov
d04f39370a
Merge branch 'master' into oss-fuzz-fixes 2023-07-17 13:41:54 +02:00
Roman Vasin
a73dca1c2f Move getEncryptionMethod to CompressionCodecEncrypted.h 2023-07-11 13:27:11 +00:00
flynn
e175be1f42 remove unused code 2023-07-05 12:34:58 +00:00
Nikita Mikhaylov
673c30f717
Merge branch 'master' into oss-fuzz-fixes 2023-07-04 19:52:41 +02:00
Nikita Mikhaylov
af603c2cc6 Fixed stack overflow on Field destruction 2023-07-03 16:40:04 +02:00
Vitaly Baranov
0cccba62cf Support getHexUIntLowercase() with CityHash_v1_0_2::uint128 parameter. 2023-06-29 15:29:37 +02:00
Vitaly Baranov
5570863676
Merge branch 'master' into reorder-part-checksum-halves 2023-06-28 17:25:34 +02:00
Vitaly Baranov
89de7a64eb Fix memory leakage in CompressionCodecDeflateQpl. 2023-06-27 15:06:31 +02:00
Vitaly Baranov
f1f0daa654 Show halves of checksums in "system.parts", "system.projection_parts" and error messages in the correct order. 2023-06-25 17:17:56 +02:00
Vitaly Baranov
3711430d9f Rename member fields of CityHash_v1_0_2::uint128: "first" -> "low64", "second" -> "high64". 2023-06-24 12:25:56 +02:00
jinjunzh
004755914e change to use generic MSan API to bypass uninitialized memory check 2023-06-18 20:55:10 -04:00
jinjunzh
5f8ef27513 initialize buffer with memset 2023-06-16 14:37:01 -04:00
Robert Schulze
c538506f2e
More fixes 2023-06-09 20:50:17 +00:00
Robert Schulze
1aa158909e
enable_qpl_deflate_codec --> enable_deflate_qpl_codec 2023-06-09 12:43:33 +00:00
jinjunzh
f1192d59af
refine patch according to comments 2023-06-09 12:43:15 +00:00
jinjunzh
056ca4f555
Add extensive testing cases for deflate qpl codec 2023-06-09 12:42:59 +00:00
Sergei Trifonov
0d1f2e297b
Unify priorities: lower value means higher priority (#50205) 2023-05-26 15:55:30 +02:00
Robert Schulze
3ab4a46c7b
Merge pull request #49658 from saitama951/clickhouse 
Fix Optimization Issue for T64 library for S390x
 2023-05-09 09:11:09 +02:00
sanjam
fce2e089c6 temp fix for t64 on s390x 2023-05-08 10:49:25 +00:00
Kseniia Sumarokova
336bb41c5d
Merge branch 'master' into remove-dependency-from-context 2023-05-08 12:46:10 +02:00
jinjunzh
576fbb9c25 fixed symbol conflict between contrib/isal and isal in qpl 2023-05-06 10:38:11 -04:00
kssenii
ecfbf1e304 Remove dependency from DB::Context in readers 2023-05-02 21:45:27 +02:00
jinjunzh
4e02ef4650 revise several cosmetic issues 2023-04-26 11:23:07 -04:00
jinjunzh
d4727288f8 fix style issue 2023-04-25 15:24:04 -04:00
jinjunzh
260eb5450b Upgrade IAA-Deflate codec library:qpl from v1.0.0 to v1.1.0 2023-04-25 14:52:28 -04:00
Alexey Milovidov
530b764953 Fix IBM 2023-04-21 12:38:45 +02:00
Robert Schulze
05606a8835
Clean up GCC warning pragmas 2023-04-11 18:21:08 +00:00
Robert Schulze
f72a337074
Remove cruft from build 
No need to check compiler flags, clang >= 15 supports all of them.
 2023-03-17 13:44:04 +00:00
Kruglov Pavel
a3510a2ffe
Merge pull request #47271 from Avogar/codecs-better 
Add optional parameters to some codecs, fix aborts in clickhouse-compressor with some codecs.
 2023-03-16 12:20:23 +01:00
avogar
0005517cee Fix comments 2023-03-15 13:08:01 +00:00
Mike Kot
9920a52c51 use std::lerp, constexpr hex.h 2023-03-07 22:50:17 +00:00
avogar
b5f19ee12c Make better 2023-03-06 19:34:36 +00:00
avogar
8847561fc2 Add optional parameters to some codecs, fix aborts in clickhouse-compressor with some codecs 2023-03-06 16:49:28 +00:00
Yakov Olkhovskiy
fadbeb8ebd T64 codec support for IPv4 2023-02-22 19:25:48 +00:00
Alexey Milovidov
d8cda3dbb8 Remove PVS-Studio 2023-02-19 23:30:05 +01:00
Suzy Wang
e140bed3ea
Merge branch 'master' into z-build-0120 2023-02-13 09:06:48 -05:00
kssenii
3067c1d723 Merge remote-tracking branch 'upstream/master' into resubmit-prefetches 2023-02-11 11:36:23 +01:00
Suzy Wang
64313d7020
Merge branch 'master' into z-build-0120 2023-02-09 14:49:22 -05:00
kssenii
b0b865c32e Resubmit prefetches 2023-02-08 21:26:24 +01:00
Robert Schulze
84b9ff450f
Fix terribly broken, fragile and potentially cyclic linking 
Sorry for the clickbaity title. This is about static method
ConnectionTimeouts::getHTTPTimeouts(). It was be declared in header
IO/ConnectionTimeouts.h, and defined in header
IO/ConnectionTimeoutsContext.h (!). This is weird and caused issues with
linking on s390x (##45520). There was an attempt to fix some
inconsistencies (#45848) but neither did @Algunenano nor me at first
really understand why the definition is in the header.

Turns out that ConnectionTimeoutsContext.h is only #include'd from
source files which are part of the normal server build BUT NOT part of
the keeper standalone build (which must be enabled via CMake
-DBUILD_STANDALONE_KEEPER=1). This dependency was not documented and as
a result, some misguided workarounds were introduced earlier, e.g.
0341c6c54b

The deeper cause was that getHTTPTimeouts() is passed a "Context". This
class is part of the "dbms" libary which is deliberately not linked by
the standalone build of clickhouse-keeper. The context is only used to
read the settings and the "Settings" class is part of the
clickhouse_common library which is linked by clickhouse-keeper already.

To resolve this mess, this PR

- creates source file IO/ConnectionTimeouts.cpp and moves all
  ConnectionTimeouts definitions into it, including getHTTPTimeouts().

- breaks the wrong dependency by passing "Settings" instead of "Context"
  into getHTTPTimeouts().

- resolves the previous hacks
 2023-02-05 20:49:34 +00:00
bkuschel
1f3b1c697f
Add newline 2023-02-04 15:39:28 -05:00
bkuschel
dc995f7c67
Fix size,free ctx,formatting 2023-02-04 15:03:04 -05:00
Boris Kuschel
d9f3698a43
Fix copy/pasta 2023-02-04 14:59:47 -05:00
Boris Kuschel
1d4cf4fe69
Add encryption support with openssl 2023-02-04 14:59:34 -05:00
Suzy Wang
503c64a45d Update as suggested 2023-02-02 20:44:32 +00:00
Suzy Wang
716d2c4ffb Update as suggested 2023-02-02 20:41:06 +00:00
Robert Schulze
3472ed4f0d
Make some classes 'final' 2023-02-02 15:59:39 +00:00
Robert Schulze
7cc1c19227
Fix typo 2023-01-30 10:25:16 +00:00
Robert Schulze
856eba0a4b
Mark delta/doubledelta codec followed by a time series codec as suspicious 2023-01-29 08:51:13 +00:00
Alexey Milovidov
bc2f454522
Merge branch 'master' into block-non-float-gorilla-v2 2023-01-28 03:30:12 +03:00
Robert Schulze
83593810ff
Make check work with Nullable / Array / Tuple 2023-01-27 13:02:54 +00:00
Alexander Tokmakov
c366806c3e
Merge pull request #45527 from ClickHouse/exception_message_patterns4 
Better formatting for exception messages 2
 2023-01-27 15:31:52 +03:00
Robert Schulze
227b8676cd
Tiny improvements around the Gorilla/Delta codecs 2023-01-26 11:11:24 +00:00
Robert Schulze
9c48ac79c8
Fix issue #45195 2023-01-26 10:44:26 +00:00
Robert Schulze
574cab5d7e
Remove transitory parameter 2023-01-24 11:05:29 +00:00
Robert Schulze
aaf7653108
Merge remote-tracking branch 'origin/master' into block-non-float-gorilla-v2 2023-01-24 10:14:10 +00:00
Suzy Wang
1176260b3c
Merge branch 'master' into z-build-0120 2023-01-24 01:14:42 -05:00
Alexander Tokmakov
3f6594f4c6 forbid old ctor of Exception 2023-01-23 22:18:05 +01:00
Alexander Tokmakov
70d1adfe4b
Better formatting for exception messages (#45449) 
* save format string for NetException

* format exceptions

* format exceptions 2

* format exceptions 3

* format exceptions 4

* format exceptions 5

* format exceptions 6

* fix

* format exceptions 7

* format exceptions 8

* Update MergeTreeIndexGin.cpp

* Update AggregateFunctionMap.cpp

* Update AggregateFunctionMap.cpp

* fix
 2023-01-24 00:13:58 +03:00
Robert Schulze
4ece499f19
Fix build 2023-01-22 12:26:03 +00:00
Suzy Wang
19d26828a0 s390x build support 2023-01-20 21:16:55 +00:00
Robert Schulze
e6167d6b36
Deprecate Gorilla compression of non-float columns 
Reasons:

1. The original Gorilla paper proposed a compression schema for pairs of
   time stamps and double-precision FP values. ClickHouse's Gorilla
   codec only implements compression of the latter and it does not
   impose any data type restrictions.
   - Data types != Float* or (U)Int* (e.g. Decimal, Point etc.) are
     definitely not supposed to be used with Gorilla.
   - (U)Int* types are debatable. The paper only considers
     integers-stored-as-FP-values, a practical use case for which
     Gorilla works well. Standalone integers are not considered which
     makes them at least suspicious.

2. Achieve consistency with FPC, another specialized floating-point
   timeseries codec, which rejects non-float data.

3. On practical datasets, ZSTD is often "good enough" (**) so it should
   be okay to disincentive non-ZSTD codecs a little bit. If needed,
   Delta and DoubleDelta codecs are viable alternative for slowly
   changing (time-series-like) integer sequences.

Since on-prem and hosted users may still have Gorilla-compressed
non-float data, this combination is only deprecated for now. No warning
or error will be emitted. Users are encouraged to migrate
Gorilla-compressed non-float data to an alternative codec. It is planned
to treat Gorilla-compressed non-float columns as "suspicious" six months
after this commit (i.e. in v23.6). Even then, it will still be possible
to set "allow_suspicious_codecs = true" and read and write
Gorilla-compressed non-float data.

(*) Sec. 4.1.2, "Gorilla restricts the value element in its tuple to a
    double floating point type.", https://doi.org/10.14778/2824032.2824078

(**) https://clickhouse.com/blog/optimize-clickhouse-codecs-compression-schema
 2023-01-20 17:31:16 +00:00
Alexander Tokmakov
df75c24f01
Revert "Disallow Gorilla codec on non-float columns" 2023-01-16 19:14:28 +03:00
Robert Schulze
b6f12f9edd
Fix unit_tests_dbms 2023-01-15 14:49:04 +00:00
Robert Schulze
a4a6126c9d
Prohibit manual delta compression before floating-point time series compression 2023-01-14 20:09:50 +00:00
Robert Schulze
fbdaca4e2a
Code cleanup 2023-01-14 19:21:30 +00:00
Robert Schulze
5d3f0ec4a0
Disallow Gorilla codec on non-float columns 
Cf. #45195
 2023-01-13 16:53:28 +00:00
alesapin
6c8dbcc040 Fix flaky test 01459_manual_write_to_replicas_quorum_detach_attach and several typos 2023-01-03 16:27:51 +01:00
Alexander Gololobov
f75b203071
Merge pull request #44034 from ClickHouse/evillique-patch-1 
Fix exception message
 2022-12-08 23:30:09 +01:00
Robert Schulze
375f488156
Merge pull request #44024 from jinjunzh/iaadeflate_upgrade_qpl030 
Upgrade Intel®-IAA/QPL-based DEFLATE_QPL Codec
 2022-12-08 12:56:03 +01:00
Nikolay Degterinsky
16b33c204c
Fix exception message 2022-12-08 10:49:25 +01:00
Duc Canh Le
88716b1ec3 Minor fix 2022-12-08 09:39:25 +08:00
jinjunzh
47c3508337 fixed build issues for QPL 0.3.0 2022-12-07 16:41:12 -05:00
Sema Checherinda
f49936b9a0 remove tmp dir from inmemory part when freeze, more logs about old parts 2022-11-23 15:16:09 +00:00
Nikita Taranov
0b4e643c27
Add check to CompressionCodecDelta (#43255) 
* Add check to CompressionCodecDelta

* Apply suggestions from code review

* Update src/Compression/CompressionCodecDelta.cpp
 2022-11-19 14:16:14 +01:00
Sema Checherinda
7e73b187cc
Merge pull request #43159 from Algunenano/over-read 
Fix several buffer over-reads
 2022-11-15 15:04:24 +01:00
Raúl Marín
54db7c6520 Enforce checking read output 2022-11-11 10:56:18 +01:00
Tiaonmmn
331c1cbeb7
Update CompressionCodecDeflateQpl.cpp 
Modify the data type to resolve warning of clang-tidy "implicit conversion".
 2022-11-11 10:00:59 +08:00
Nikita Taranov
2c7708a03e
Add bound check to lz4 decompression (#42868) 
* impl

* add test

* fix bullshit tidy error
 2022-11-04 22:12:36 +01:00
Anton Popov
7715afa595
Merge pull request #42618 from CurtizJ/refactor-data-part 
Try to save `IDataPartStorage` interface
 2022-10-27 22:20:44 +02:00
Anton Popov
c8199bc125
Merge branch 'master' into refactor-data-part 2022-10-25 14:31:05 +02:00
Robert Schulze
c119cd2f00
Merge branch 'master' into update-libcxx-to-15 2022-10-24 08:29:37 +02:00
Anton Popov
b40d9200d2 better semantic of constsness of DataPartStorage 2022-10-23 15:24:20 +00:00
Azat Khuzhin
3d400068fd tests/gtest_compressionCodec: fix UBSAN report to avoid test failure 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-21 13:25:43 +02:00
Azat Khuzhin
2474303aef tests/gtest_compressionCodec: Fix UBSAN report about signed integer overflow 
UBSAN report:

    $ UBSAN_SYMBOLIZER_PATH=/usr/bin/llvm-symbolizer-14 UBSAN_OPTIONS=print_stacktrace=1 ./unit_tests_dbms --gtest_filter=*Gorilla*
    ../src/Compression/tests/gtest_compressionCodec.cpp:1216:47: runtime error: signed integer overflow: 23 * 100000000 cannot be represented in type 'int'
        #0 0x14f67fd1 in auto (anonymous namespace)::$_6::operator()(int) const::'lambda'(auto)::operator()<int>(auto) const build_docker/../src/Compression/tests/gtest_compressionCodec.cpp:1216:47
        #1 0x14f67fd1 in (anonymous namespace)::CodecTestSequence (anonymous namespace)::generateSeq<long, (anonymous namespace)::$_6::operator()(int) const::'lambda'(auto), int, int>((anonymous namespace)::$_6::operator()(int) const::'lambda'(auto), char const*, int, int) build_docker/../src/Compression/tests/gtest_compressionCodec.cpp:394:36
        #2 0x14f67fd1 in auto (anonymous namespace)::GCompatibilityTestSequence<long>() build_docker/../src/Compression/tests/gtest_compressionCodec.cpp:1224:12
        #3 0x14f3c7f3 in (anonymous namespace)::gtest_GorillaCodecTestCompatibility_EvalGenerator_() build_docker/../src/Compression/tests/gtest_compressionCodec.cpp:1227:1
        #4 0x14f6bdb5 in testing::internal::ParameterizedTestSuiteInfo<(anonymous namespace)::CodecTestCompatibility>::RegisterTests() build_docker/../contrib/googletest/googletest/include/gtest/internal/gtest-param-util.h:553:45
        #5 0x27d87988 in testing::internal::ParameterizedTestSuiteRegistry::RegisterTests() build_docker/../contrib/googletest/googletest/include/gtest/internal/gtest-param-util.h:726:24
        #6 0x27d87988 in testing::internal::UnitTestImpl::RegisterParameterizedTests() build_docker/../contrib/googletest/googletest/src/gtest.cc:2805:34
        #7 0x27d87988 in testing::internal::UnitTestImpl::PostFlagParsingInit() build_docker/../contrib/googletest/googletest/src/gtest.cc:5492:5
        #8 0x27d9d002 in void testing::internal::InitGoogleTestImpl<char>(int*, char**) build_docker/../contrib/googletest/googletest/src/gtest.cc:6499:22
        #9 0x14fd5495 in main build_docker/../src/Coordination/tests/gtest_coordination.cpp:2189:5
        #10 0x7f8c29005209  (/lib/x86_64-linux-gnu/libc.so.6+0x29209) (BuildId: 71a7c7b97bc0b3e349a3d8640252655552082bf5)
        #11 0x7f8c290052bb in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x292bb) (BuildId: 71a7c7b97bc0b3e349a3d8640252655552082bf5)
        #12 0x14ce356d in _start (/work1/azat/tmp/42190/unit_tests_dbms+0x14ce356d) (BuildId: 482550e3f8d45f06e8c7f8147f427ee798c1f645)

    SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../src/Compression/tests/gtest_compressionCodec.cpp:1216:47 in

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-21 13:25:43 +02:00
Azat Khuzhin
b8ab686561 tests/gtest_compressionCodec: fix for darwin 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-21 13:25:43 +02:00
Azat Khuzhin
4e76629aaf Fixes for -Wshorten-64-to-32 
- lots of static_cast
- add safe_cast
- types adjustments
  - config
  - IStorage::read/watch
  - ...
- some TODO's (to convert types in future)

P.S. That was quite a journey...

v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-21 13:25:19 +02:00
Robert Schulze
820e6b4276
Build with libcxx(abi) 15 2022-10-20 10:52:43 +00:00
Alexey Milovidov
191158f93b
Merge pull request #42314 from HarryLeeIBM/hlee-s390x-t64-codec 
Fix Codec T64 on s390x
 2022-10-17 01:13:57 +02:00
Alexey Milovidov
bfd8811a4d
Update CompressionCodecT64.cpp 2022-10-15 00:32:37 +03:00
Alexey Milovidov
4608b70dda
Update CompressionCodecT64.cpp 2022-10-15 00:31:33 +03:00
HarryLeeIBM
6cdb7b5a94 Fix Codec T64 on s390x 2022-10-14 07:16:18 -07:00
Alexander Tokmakov
4175f8cde6 abort instead of __builtin_unreachable in debug builds 2022-10-07 21:49:08 +02:00
Robert Schulze
da5a2e2db0
Merge remote-tracking branch 'origin/master' into generated-file-cleanup 
Physical merge conflicts:
- src/Common/ZooKeeper/ZooKeeperImpl.cpp
- src/Core/config_core.h.in
- src/Functions/FunctionsAES.h
- src/Functions/config_functions.h.in
- src/configure_config.cmake

Logical merge conflicts:
- Functions/tryDecrypt.cpp
 2022-10-06 08:43:25 +00:00
root
4318ec125e Merge branch 'master' of https://github.com/MeenaRenganathan22/ClickHouse into OpenSSL_InHouse 2022-09-30 15:50:22 -07:00
Robert Schulze
f24fab7747
Fix some #include atrocities 2022-09-28 13:49:28 +00:00
Robert Schulze
fd86829824
Consolidate config_core.h into config.h 
Less duplication, less confusion ...
 2022-09-28 13:31:57 +00:00
Robert Schulze
78fc36ca49
Generate config.h into ${CONFIG_INCLUDE_PATH} 
This makes the target location consistent with other auto-generated
files like config_formats.h, config_core.h, and config_functions.h and
simplifies the build of clickhouse_common.
 2022-09-28 12:48:26 +00:00
Alexey Milovidov
730655d4fd Fix 8/9 of trash 2022-09-19 08:53:20 +02:00
Meena Renganathan
51d6611b96 Committing the ClickHouse core changes and other libraries to support OpenSSL. BoringSSL is still set as default 2022-09-12 09:05:38 -07:00
Anton Popov
7c12b448b8 Merge remote-tracking branch 'upstream/master' into HEAD 2022-09-08 01:52:52 +00:00
Nikita Taranov
7c4f42d014
Skip empty literals in lz4 decompression (#40142) 2022-09-06 13:58:26 +02:00
Anton Popov
9dda9658a8 Merge remote-tracking branch 'upstream/master' into HEAD 2022-09-02 12:48:27 +00:00
Antonio Andelic
e64436fef3 Fix typos with new codespell 2022-09-02 08:54:48 +00:00
HarryLeeIBM
d609da182b Fix Endian issue in Codec for s390x 2022-08-08 11:28:35 -07:00
Robert Schulze
a7734672b9
Use std::popcount, ::countl_zero, ::countr_zero functions 
- Introduced with the C++20 <bit> header

- The problem with __builtin_c(l|t)z() is that 0 as input has an
  undefined result (*) and the code did not always check. The std::
  versions do not have this issue.

- In some cases, we continue to use buildin_c(l|t)z(), (e.g. in
  src/Common/BitHelpers.h) because the std:: versions only accept
  unsigned inputs (and they also check that) and the casting would be
  ugly.

(*) https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html
 2022-07-31 15:16:51 +00:00
Robert Schulze
7d653b3bd8
Merge pull request #39495 from ClickHouse/mark-qpl-deflate-experimental 
Mark new codec DEFLATE_QPL as experimental + cosmetics
 2022-07-26 19:00:21 +02:00
Robert Schulze
2785783def
Mark DEFLAPT_QPL as 'experimental' codec + cosmetics 2022-07-24 20:10:11 +00:00
Ignat Loskutov
c2954b23cb
LZ4_decompress_faster.cpp: remove endianness-dependent code 
Little-endian is little-endian no matter what the native endianness is.
 2022-07-24 20:55:17 +08:00
Alexey Milovidov
2d12180f9e
Merge pull request #39195 from HarryLeeIBM/hlee-s390x-LZ4 
Fix LZ4 decompression issue for s390x
 2022-07-23 22:31:42 +03:00
Robert Schulze
654047dfbf
Fix compiler warning 2022-07-22 11:28:05 +00:00
jinjunzh
f663c1e5e2 fixed condition fault 2022-07-21 17:14:20 -04:00
HarryLeeIBM
d5727dfb44 Removed __BYTE_ORDER__ and fixed style issue 2022-07-21 13:09:45 -07:00
jinjunzh
aa8dae5f72 add nullptr check for flushAsynchronousDecompressRequests 2022-07-21 12:04:11 -04:00
Nikita Taranov
9dbfc354d1
Merge branch 'master' into hlee-s390x-LZ4 2022-07-20 17:15:31 +02:00
jinjunzh
9c440bec28 Merge branch 'dev_intel_iaa_deflate' of https://github.com/jinjunzh/ClickHouse into dev_intel_iaa_deflate 2022-07-19 17:50:23 -04:00
jinjunzh
94332d876a Fixed type&style issue 2022-07-19 17:48:41 -04:00
jinjunzh
b0c6fcf3a0 demoted log to INFO when the job pool is exhausted 2022-07-19 15:30:25 -04:00
jinjunzh
8f6cc6971b add comments for readBig 2022-07-19 14:24:00 -04:00
jinjunzh
3a7fa8887f refine readCompressedDataBlockForAsynchronous 2022-07-19 10:23:25 -04:00
jasperzhu
c044c67745
Merge branch 'master' into dev_intel_iaa_deflate 2022-07-19 15:32:57 +08:00
jinjunzh
a53ed21dba restore the function doDecompressDataSynchronous 2022-07-18 17:44:15 -04:00
jinjunzh
332d3fd1a7 add comments for doDecompressDataAsynchronous 2022-07-18 14:40:49 -04:00
jinjunzh
456ae5ea44 add comments for readCompressedDataBlockForAsynchronous 2022-07-18 10:39:20 -04:00
Anton Popov
5605b60c97
Merge pull request #39039 from CurtizJ/randomize-more-settings 
Add more settings for randomization
 2022-07-18 15:05:41 +02:00
Alexey Milovidov
f1256c3aef
Merge pull request #37891 from guowangy/lz4-decompress-vbmi 
LZ4 decompress: add VBMI optimized copyOverlap32Shuffle
 2022-07-18 00:37:59 +03:00
Wangyang Guo
7ef3f56066 lz4 decompress: coding style fix 2022-07-16 22:10:16 +08:00
jinjunzh
f05f3cb69a fixed cosmetic issues 2022-07-15 17:04:27 -04:00
jinjunzh
e3735f7464 fixed cosmetic issues 2022-07-15 16:12:26 -04:00
jinjunzh
5104e4200a fixed cosmetic issues 2022-07-15 15:32:10 -04:00
Nikita Taranov
ecdc56bf31
Merge branch 'master' into hlee-s390x-LZ4 2022-07-15 14:23:41 +02:00
Anton Popov
76eec62469
Merge branch 'master' into randomize-more-settings 2022-07-15 14:12:49 +02:00
Wangyang Guo
c9e15017ac lz4 decompress: dynamic dispatch with TargetSpecific 2022-07-15 16:07:19 +08:00
avogar
9291d33080 Pass const std::string_view & by value, not by reference 2022-07-14 16:11:57 +00:00
Anton Popov
d4cb2184c2
Merge branch 'master' into randomize-more-settings 2022-07-14 17:01:07 +02:00
jinjunzh
1c6d70f9b3 fixed sw job buffer missing 2022-07-13 16:12:15 -04:00
HarryLeeIBM
2caa3f8803 Fix LZ4 decompression issue for s390x 2022-07-13 11:45:58 -07:00
Wangyang Guo
f275b35bba Dynamic dispatching for best_variant selection 2022-07-13 15:20:15 +08:00
jinjunzh
b699764bbe unified sync implemetation with async 2022-07-12 14:32:53 -04:00
Anton Popov
3b45e6e347 fix reading of empty arrays in reverse order with uncompressed cache (addition to #36215) 2022-07-12 03:08:12 +00:00
Robert Schulze
1a7727a254
Prefix overridden add_executable() command with "clickhouse_" 
A simple HelloWorld program with zero includes except iostream triggers
a build of ca. 2000 source files. The reason is that ClickHouse's
top-level CMakeLists.txt overrides "add_executable()" to link all
binaries against "clickhouse_new_delete". This links against
"clickhouse_common_io", which in turn has lots of 3rd party library
dependencies ... Without linking "clickhouse_new_delete", the number of
compiled files for "HelloWorld" goes down to ca. 70.

As an example, the self-extracting-executable needs none of its current
dependencies but other programs may also benefit.

In order to restore access to the original "add_executable()", the
overriding version is now prefixed. There is precedence for a
"clickhouse_" prefix (as opposed to "ch_"), for example
"clickhouse_split_debug_symbols". In general prefixing makes sense also
because overriding CMake commands relies on undocumented behavior and is
considered not-so-great practice (*).

(*) https://crascit.com/2018/09/14/do-not-redefine-cmake-commands/
 2022-07-11 19:36:18 +02:00
jinjunzh
bcba581191 fixed cosmetic issues 2022-07-11 12:08:35 -04:00
mergify[bot]
62c62a2df7
Merge branch 'master' into lz4-decompress-vbmi 2022-07-11 14:23:17 +00:00
jinjunzh
bc03e10870 make job pool static member to private 2022-07-09 17:50:57 -04:00
jinjunzh
bc775ecc5b add assert for Illegal path 2022-07-09 17:29:47 -04:00
jinjunzh
044c14745e make job pool name comprehensible 2022-07-09 17:17:18 -04:00
jinjunzh
eb06c66869 cosmetic issue 2022-07-09 14:57:38 -04:00
jinjunzh
816e974ca2 rename deflate to deflate_qpl 2022-07-09 14:42:01 -04:00
jinjunzh
825b7511e4 improve cosmetics 2022-07-09 14:13:12 -04:00
jinjunzh
7cb5b35bef remove qpl from fasttest 2022-07-09 13:54:23 -04:00
jinjunzh
212f342af5 change to std random 2022-07-09 12:03:25 -04:00
jinjunzh
0fdce07cc6 add dedicated value for HW codec fail 2022-07-09 11:12:41 -04:00
jinjunzh
545e6b305c improve log print text 2022-07-09 10:09:23 -04:00
jinjunzh
46d7d3b5c9 constant always goes to the right side 2022-07-08 15:03:10 -04:00
jinjunzh
289c6148e4 member to be LOWERCASE_UNDERSCORE 2022-07-08 12:20:24 -04:00
jinjunzh
7eb71143db add a one-line method for resetting a job lock 2022-07-08 12:09:58 -04:00
jinjunzh
7a3c21174d use C++17 if initializer 2022-07-08 11:55:16 -04:00
jinjunzh
6045681bb4 improve jobPoolReady 2022-07-07 17:21:51 -04:00
jinjunzh
5681e50da8 improve job pool initialization 2022-07-07 17:13:20 -04:00
jinjunzh
2c8f308bbc improve buffer allocation 2022-07-07 16:05:52 -04:00
jinjunzh
29fee729e3 revise the way of get job size 2022-07-07 15:17:23 -04:00
jinjunzh
2765e2a259 remove unused function in deflate job pool 2022-07-07 14:44:42 -04:00
jinjunzh
f80aea9eac Cosmetic issue and improve flush function 2022-07-07 14:14:47 -04:00
jinjunzh
ad82bb80ed Cosmetic: Constants to be UPPERCASE_AND_UNDERSCORE 2022-07-07 12:10:06 -04:00
jinjunzh
85eae40765 remove inline due to negligible benefits 2022-07-07 10:26:57 -04:00
jinjunzh
43c5a68d14 Cosmetic work 2022-07-07 10:04:17 -04:00
jinjunzh
6fa036f6ee remove qpl from clickhouse-keeper 2022-07-06 23:48:09 -04:00
jinjunzh
3422c0479d add more comments for flush 2022-07-06 23:31:22 -04:00
jinjunzh
4cdc660406 change flush name to be more descriptive 2022-07-06 21:37:11 -04:00
jinjunzh
7b262b62c2 restore decompress to be const 2022-07-06 18:34:31 -04:00
jinjunzh
567f90d8a4 add CodecMode for deflate codec 2022-07-06 18:18:55 -04:00
jinjunzh
fbe3d84860 revise typo issue 2022-07-05 15:29:53 -04:00
jinjunzh
ba38969e9c enable qpl for clickhouse-keeper build 2022-07-05 15:28:41 -04:00
jinjunzh
fe451a4317 remove compress sync interface 2022-06-28 22:52:11 -04:00
jasperzhu
b2f98fa73b
Merge branch 'master' into dev_intel_iaa_deflate 2022-06-27 10:21:21 +08:00
alesapin
50801e41c5 Merge branch 'master' into refactor-something-in-part-volumes 2022-06-19 14:05:46 +02:00
Anton Popov
1523c9c9e5 fix filling of empty Nested + small refactoring 2022-06-17 01:10:52 +00:00
Anton Popov
92b7b9789a
try to fix fpc codec 2022-06-16 03:41:09 +02:00
jasperzhu
bca18b2caf
Merge branch 'master' into dev_intel_iaa_deflate 2022-06-16 08:56:27 +08:00
Nikolai Kochetov
2a9a63ac7b Merge branch 'master' into refactor-something-in-part-volumes 2022-06-15 15:35:26 +02:00