thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-17 11:52:27 +00:00
Code Issues Packages Projects Releases Wiki Activity
142,586 Commits 9,326 Branches 1,784 Tags 2.1 GiB
da1af2b357
Commit Graph

668 Commits

Author SHA1 Message Date
Mike Kot
9920a52c51 use std::lerp, constexpr hex.h 2023-03-07 22:50:17 +00:00
avogar
b5f19ee12c Make better 2023-03-06 19:34:36 +00:00
avogar
8847561fc2 Add optional parameters to some codecs, fix aborts in clickhouse-compressor with some codecs 2023-03-06 16:49:28 +00:00
Yakov Olkhovskiy
fadbeb8ebd T64 codec support for IPv4 2023-02-22 19:25:48 +00:00
Alexey Milovidov
d8cda3dbb8 Remove PVS-Studio 2023-02-19 23:30:05 +01:00
Suzy Wang
e140bed3ea
Merge branch 'master' into z-build-0120 2023-02-13 09:06:48 -05:00
kssenii
3067c1d723 Merge remote-tracking branch 'upstream/master' into resubmit-prefetches 2023-02-11 11:36:23 +01:00
Suzy Wang
64313d7020
Merge branch 'master' into z-build-0120 2023-02-09 14:49:22 -05:00
kssenii
b0b865c32e Resubmit prefetches 2023-02-08 21:26:24 +01:00
Robert Schulze
84b9ff450f
Fix terribly broken, fragile and potentially cyclic linking 
Sorry for the clickbaity title. This is about static method
ConnectionTimeouts::getHTTPTimeouts(). It was be declared in header
IO/ConnectionTimeouts.h, and defined in header
IO/ConnectionTimeoutsContext.h (!). This is weird and caused issues with
linking on s390x (##45520). There was an attempt to fix some
inconsistencies (#45848) but neither did @Algunenano nor me at first
really understand why the definition is in the header.

Turns out that ConnectionTimeoutsContext.h is only #include'd from
source files which are part of the normal server build BUT NOT part of
the keeper standalone build (which must be enabled via CMake
-DBUILD_STANDALONE_KEEPER=1). This dependency was not documented and as
a result, some misguided workarounds were introduced earlier, e.g.
0341c6c54b

The deeper cause was that getHTTPTimeouts() is passed a "Context". This
class is part of the "dbms" libary which is deliberately not linked by
the standalone build of clickhouse-keeper. The context is only used to
read the settings and the "Settings" class is part of the
clickhouse_common library which is linked by clickhouse-keeper already.

To resolve this mess, this PR

- creates source file IO/ConnectionTimeouts.cpp and moves all
  ConnectionTimeouts definitions into it, including getHTTPTimeouts().

- breaks the wrong dependency by passing "Settings" instead of "Context"
  into getHTTPTimeouts().

- resolves the previous hacks
 2023-02-05 20:49:34 +00:00
bkuschel
1f3b1c697f
Add newline 2023-02-04 15:39:28 -05:00
bkuschel
dc995f7c67
Fix size,free ctx,formatting 2023-02-04 15:03:04 -05:00
Boris Kuschel
d9f3698a43
Fix copy/pasta 2023-02-04 14:59:47 -05:00
Boris Kuschel
1d4cf4fe69
Add encryption support with openssl 2023-02-04 14:59:34 -05:00
Suzy Wang
503c64a45d Update as suggested 2023-02-02 20:44:32 +00:00
Suzy Wang
716d2c4ffb Update as suggested 2023-02-02 20:41:06 +00:00
Robert Schulze
3472ed4f0d
Make some classes 'final' 2023-02-02 15:59:39 +00:00
Robert Schulze
7cc1c19227
Fix typo 2023-01-30 10:25:16 +00:00
Robert Schulze
856eba0a4b
Mark delta/doubledelta codec followed by a time series codec as suspicious 2023-01-29 08:51:13 +00:00
Alexey Milovidov
bc2f454522
Merge branch 'master' into block-non-float-gorilla-v2 2023-01-28 03:30:12 +03:00
Robert Schulze
83593810ff
Make check work with Nullable / Array / Tuple 2023-01-27 13:02:54 +00:00
Alexander Tokmakov
c366806c3e
Merge pull request #45527 from ClickHouse/exception_message_patterns4 
Better formatting for exception messages 2
 2023-01-27 15:31:52 +03:00
Robert Schulze
227b8676cd
Tiny improvements around the Gorilla/Delta codecs 2023-01-26 11:11:24 +00:00
Robert Schulze
9c48ac79c8
Fix issue #45195 2023-01-26 10:44:26 +00:00
Robert Schulze
574cab5d7e
Remove transitory parameter 2023-01-24 11:05:29 +00:00
Robert Schulze
aaf7653108
Merge remote-tracking branch 'origin/master' into block-non-float-gorilla-v2 2023-01-24 10:14:10 +00:00
Suzy Wang
1176260b3c
Merge branch 'master' into z-build-0120 2023-01-24 01:14:42 -05:00
Alexander Tokmakov
3f6594f4c6 forbid old ctor of Exception 2023-01-23 22:18:05 +01:00
Alexander Tokmakov
70d1adfe4b
Better formatting for exception messages (#45449) 
* save format string for NetException

* format exceptions

* format exceptions 2

* format exceptions 3

* format exceptions 4

* format exceptions 5

* format exceptions 6

* fix

* format exceptions 7

* format exceptions 8

* Update MergeTreeIndexGin.cpp

* Update AggregateFunctionMap.cpp

* Update AggregateFunctionMap.cpp

* fix
 2023-01-24 00:13:58 +03:00
Robert Schulze
4ece499f19
Fix build 2023-01-22 12:26:03 +00:00
Suzy Wang
19d26828a0 s390x build support 2023-01-20 21:16:55 +00:00
Robert Schulze
e6167d6b36
Deprecate Gorilla compression of non-float columns 
Reasons:

1. The original Gorilla paper proposed a compression schema for pairs of
   time stamps and double-precision FP values. ClickHouse's Gorilla
   codec only implements compression of the latter and it does not
   impose any data type restrictions.
   - Data types != Float* or (U)Int* (e.g. Decimal, Point etc.) are
     definitely not supposed to be used with Gorilla.
   - (U)Int* types are debatable. The paper only considers
     integers-stored-as-FP-values, a practical use case for which
     Gorilla works well. Standalone integers are not considered which
     makes them at least suspicious.

2. Achieve consistency with FPC, another specialized floating-point
   timeseries codec, which rejects non-float data.

3. On practical datasets, ZSTD is often "good enough" (**) so it should
   be okay to disincentive non-ZSTD codecs a little bit. If needed,
   Delta and DoubleDelta codecs are viable alternative for slowly
   changing (time-series-like) integer sequences.

Since on-prem and hosted users may still have Gorilla-compressed
non-float data, this combination is only deprecated for now. No warning
or error will be emitted. Users are encouraged to migrate
Gorilla-compressed non-float data to an alternative codec. It is planned
to treat Gorilla-compressed non-float columns as "suspicious" six months
after this commit (i.e. in v23.6). Even then, it will still be possible
to set "allow_suspicious_codecs = true" and read and write
Gorilla-compressed non-float data.

(*) Sec. 4.1.2, "Gorilla restricts the value element in its tuple to a
    double floating point type.", https://doi.org/10.14778/2824032.2824078

(**) https://clickhouse.com/blog/optimize-clickhouse-codecs-compression-schema
 2023-01-20 17:31:16 +00:00
Alexander Tokmakov
df75c24f01
Revert "Disallow Gorilla codec on non-float columns" 2023-01-16 19:14:28 +03:00
Robert Schulze
b6f12f9edd
Fix unit_tests_dbms 2023-01-15 14:49:04 +00:00
Robert Schulze
a4a6126c9d
Prohibit manual delta compression before floating-point time series compression 2023-01-14 20:09:50 +00:00
Robert Schulze
fbdaca4e2a
Code cleanup 2023-01-14 19:21:30 +00:00
Robert Schulze
5d3f0ec4a0
Disallow Gorilla codec on non-float columns 
Cf. #45195
 2023-01-13 16:53:28 +00:00
alesapin
6c8dbcc040 Fix flaky test 01459_manual_write_to_replicas_quorum_detach_attach and several typos 2023-01-03 16:27:51 +01:00
Alexander Gololobov
f75b203071
Merge pull request #44034 from ClickHouse/evillique-patch-1 
Fix exception message
 2022-12-08 23:30:09 +01:00
Robert Schulze
375f488156
Merge pull request #44024 from jinjunzh/iaadeflate_upgrade_qpl030 
Upgrade Intel®-IAA/QPL-based DEFLATE_QPL Codec
 2022-12-08 12:56:03 +01:00
Nikolay Degterinsky
16b33c204c
Fix exception message 2022-12-08 10:49:25 +01:00
Duc Canh Le
88716b1ec3 Minor fix 2022-12-08 09:39:25 +08:00
jinjunzh
47c3508337 fixed build issues for QPL 0.3.0 2022-12-07 16:41:12 -05:00
Sema Checherinda
f49936b9a0 remove tmp dir from inmemory part when freeze, more logs about old parts 2022-11-23 15:16:09 +00:00
Nikita Taranov
0b4e643c27
Add check to CompressionCodecDelta (#43255) 
* Add check to CompressionCodecDelta

* Apply suggestions from code review

* Update src/Compression/CompressionCodecDelta.cpp
 2022-11-19 14:16:14 +01:00
Sema Checherinda
7e73b187cc
Merge pull request #43159 from Algunenano/over-read 
Fix several buffer over-reads
 2022-11-15 15:04:24 +01:00
Raúl Marín
54db7c6520 Enforce checking read output 2022-11-11 10:56:18 +01:00
Tiaonmmn
331c1cbeb7
Update CompressionCodecDeflateQpl.cpp 
Modify the data type to resolve warning of clang-tidy "implicit conversion".
 2022-11-11 10:00:59 +08:00
Nikita Taranov
2c7708a03e
Add bound check to lz4 decompression (#42868) 
* impl

* add test

* fix bullshit tidy error
 2022-11-04 22:12:36 +01:00
Anton Popov
7715afa595
Merge pull request #42618 from CurtizJ/refactor-data-part 
Try to save `IDataPartStorage` interface
 2022-10-27 22:20:44 +02:00
Anton Popov
c8199bc125
Merge branch 'master' into refactor-data-part 2022-10-25 14:31:05 +02:00
Robert Schulze
c119cd2f00
Merge branch 'master' into update-libcxx-to-15 2022-10-24 08:29:37 +02:00
Anton Popov
b40d9200d2 better semantic of constsness of DataPartStorage 2022-10-23 15:24:20 +00:00
Azat Khuzhin
3d400068fd tests/gtest_compressionCodec: fix UBSAN report to avoid test failure 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-21 13:25:43 +02:00
Azat Khuzhin
2474303aef tests/gtest_compressionCodec: Fix UBSAN report about signed integer overflow 
UBSAN report:

    $ UBSAN_SYMBOLIZER_PATH=/usr/bin/llvm-symbolizer-14 UBSAN_OPTIONS=print_stacktrace=1 ./unit_tests_dbms --gtest_filter=*Gorilla*
    ../src/Compression/tests/gtest_compressionCodec.cpp:1216:47: runtime error: signed integer overflow: 23 * 100000000 cannot be represented in type 'int'
        #0 0x14f67fd1 in auto (anonymous namespace)::$_6::operator()(int) const::'lambda'(auto)::operator()<int>(auto) const build_docker/../src/Compression/tests/gtest_compressionCodec.cpp:1216:47
        #1 0x14f67fd1 in (anonymous namespace)::CodecTestSequence (anonymous namespace)::generateSeq<long, (anonymous namespace)::$_6::operator()(int) const::'lambda'(auto), int, int>((anonymous namespace)::$_6::operator()(int) const::'lambda'(auto), char const*, int, int) build_docker/../src/Compression/tests/gtest_compressionCodec.cpp:394:36
        #2 0x14f67fd1 in auto (anonymous namespace)::GCompatibilityTestSequence<long>() build_docker/../src/Compression/tests/gtest_compressionCodec.cpp:1224:12
        #3 0x14f3c7f3 in (anonymous namespace)::gtest_GorillaCodecTestCompatibility_EvalGenerator_() build_docker/../src/Compression/tests/gtest_compressionCodec.cpp:1227:1
        #4 0x14f6bdb5 in testing::internal::ParameterizedTestSuiteInfo<(anonymous namespace)::CodecTestCompatibility>::RegisterTests() build_docker/../contrib/googletest/googletest/include/gtest/internal/gtest-param-util.h:553:45
        #5 0x27d87988 in testing::internal::ParameterizedTestSuiteRegistry::RegisterTests() build_docker/../contrib/googletest/googletest/include/gtest/internal/gtest-param-util.h:726:24
        #6 0x27d87988 in testing::internal::UnitTestImpl::RegisterParameterizedTests() build_docker/../contrib/googletest/googletest/src/gtest.cc:2805:34
        #7 0x27d87988 in testing::internal::UnitTestImpl::PostFlagParsingInit() build_docker/../contrib/googletest/googletest/src/gtest.cc:5492:5
        #8 0x27d9d002 in void testing::internal::InitGoogleTestImpl<char>(int*, char**) build_docker/../contrib/googletest/googletest/src/gtest.cc:6499:22
        #9 0x14fd5495 in main build_docker/../src/Coordination/tests/gtest_coordination.cpp:2189:5
        #10 0x7f8c29005209  (/lib/x86_64-linux-gnu/libc.so.6+0x29209) (BuildId: 71a7c7b97bc0b3e349a3d8640252655552082bf5)
        #11 0x7f8c290052bb in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x292bb) (BuildId: 71a7c7b97bc0b3e349a3d8640252655552082bf5)
        #12 0x14ce356d in _start (/work1/azat/tmp/42190/unit_tests_dbms+0x14ce356d) (BuildId: 482550e3f8d45f06e8c7f8147f427ee798c1f645)

    SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ../src/Compression/tests/gtest_compressionCodec.cpp:1216:47 in

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-21 13:25:43 +02:00
Azat Khuzhin
b8ab686561 tests/gtest_compressionCodec: fix for darwin 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-21 13:25:43 +02:00
Azat Khuzhin
4e76629aaf Fixes for -Wshorten-64-to-32 
- lots of static_cast
- add safe_cast
- types adjustments
  - config
  - IStorage::read/watch
  - ...
- some TODO's (to convert types in future)

P.S. That was quite a journey...

v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-21 13:25:19 +02:00
Robert Schulze
820e6b4276
Build with libcxx(abi) 15 2022-10-20 10:52:43 +00:00
Alexey Milovidov
191158f93b
Merge pull request #42314 from HarryLeeIBM/hlee-s390x-t64-codec 
Fix Codec T64 on s390x
 2022-10-17 01:13:57 +02:00
Alexey Milovidov
bfd8811a4d
Update CompressionCodecT64.cpp 2022-10-15 00:32:37 +03:00
Alexey Milovidov
4608b70dda
Update CompressionCodecT64.cpp 2022-10-15 00:31:33 +03:00
HarryLeeIBM
6cdb7b5a94 Fix Codec T64 on s390x 2022-10-14 07:16:18 -07:00
Alexander Tokmakov
4175f8cde6 abort instead of __builtin_unreachable in debug builds 2022-10-07 21:49:08 +02:00
Robert Schulze
da5a2e2db0
Merge remote-tracking branch 'origin/master' into generated-file-cleanup 
Physical merge conflicts:
- src/Common/ZooKeeper/ZooKeeperImpl.cpp
- src/Core/config_core.h.in
- src/Functions/FunctionsAES.h
- src/Functions/config_functions.h.in
- src/configure_config.cmake

Logical merge conflicts:
- Functions/tryDecrypt.cpp
 2022-10-06 08:43:25 +00:00
root
4318ec125e Merge branch 'master' of https://github.com/MeenaRenganathan22/ClickHouse into OpenSSL_InHouse 2022-09-30 15:50:22 -07:00
Robert Schulze
f24fab7747
Fix some #include atrocities 2022-09-28 13:49:28 +00:00
Robert Schulze
fd86829824
Consolidate config_core.h into config.h 
Less duplication, less confusion ...
 2022-09-28 13:31:57 +00:00
Robert Schulze
78fc36ca49
Generate config.h into ${CONFIG_INCLUDE_PATH} 
This makes the target location consistent with other auto-generated
files like config_formats.h, config_core.h, and config_functions.h and
simplifies the build of clickhouse_common.
 2022-09-28 12:48:26 +00:00
Alexey Milovidov
730655d4fd Fix 8/9 of trash 2022-09-19 08:53:20 +02:00
Meena Renganathan
51d6611b96 Committing the ClickHouse core changes and other libraries to support OpenSSL. BoringSSL is still set as default 2022-09-12 09:05:38 -07:00
Anton Popov
7c12b448b8 Merge remote-tracking branch 'upstream/master' into HEAD 2022-09-08 01:52:52 +00:00
Nikita Taranov
7c4f42d014
Skip empty literals in lz4 decompression (#40142) 2022-09-06 13:58:26 +02:00
Anton Popov
9dda9658a8 Merge remote-tracking branch 'upstream/master' into HEAD 2022-09-02 12:48:27 +00:00
Antonio Andelic
e64436fef3 Fix typos with new codespell 2022-09-02 08:54:48 +00:00
HarryLeeIBM
d609da182b Fix Endian issue in Codec for s390x 2022-08-08 11:28:35 -07:00
Robert Schulze
a7734672b9
Use std::popcount, ::countl_zero, ::countr_zero functions 
- Introduced with the C++20 <bit> header

- The problem with __builtin_c(l|t)z() is that 0 as input has an
  undefined result (*) and the code did not always check. The std::
  versions do not have this issue.

- In some cases, we continue to use buildin_c(l|t)z(), (e.g. in
  src/Common/BitHelpers.h) because the std:: versions only accept
  unsigned inputs (and they also check that) and the casting would be
  ugly.

(*) https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html
 2022-07-31 15:16:51 +00:00
Robert Schulze
7d653b3bd8
Merge pull request #39495 from ClickHouse/mark-qpl-deflate-experimental 
Mark new codec DEFLATE_QPL as experimental + cosmetics
 2022-07-26 19:00:21 +02:00
Robert Schulze
2785783def
Mark DEFLAPT_QPL as 'experimental' codec + cosmetics 2022-07-24 20:10:11 +00:00
Ignat Loskutov
c2954b23cb
LZ4_decompress_faster.cpp: remove endianness-dependent code 
Little-endian is little-endian no matter what the native endianness is.
 2022-07-24 20:55:17 +08:00
Alexey Milovidov
2d12180f9e
Merge pull request #39195 from HarryLeeIBM/hlee-s390x-LZ4 
Fix LZ4 decompression issue for s390x
 2022-07-23 22:31:42 +03:00
Robert Schulze
654047dfbf
Fix compiler warning 2022-07-22 11:28:05 +00:00
jinjunzh
f663c1e5e2 fixed condition fault 2022-07-21 17:14:20 -04:00
HarryLeeIBM
d5727dfb44 Removed __BYTE_ORDER__ and fixed style issue 2022-07-21 13:09:45 -07:00
jinjunzh
aa8dae5f72 add nullptr check for flushAsynchronousDecompressRequests 2022-07-21 12:04:11 -04:00
Nikita Taranov
9dbfc354d1
Merge branch 'master' into hlee-s390x-LZ4 2022-07-20 17:15:31 +02:00
jinjunzh
9c440bec28 Merge branch 'dev_intel_iaa_deflate' of https://github.com/jinjunzh/ClickHouse into dev_intel_iaa_deflate 2022-07-19 17:50:23 -04:00
jinjunzh
94332d876a Fixed type&style issue 2022-07-19 17:48:41 -04:00
jinjunzh
b0c6fcf3a0 demoted log to INFO when the job pool is exhausted 2022-07-19 15:30:25 -04:00
jinjunzh
8f6cc6971b add comments for readBig 2022-07-19 14:24:00 -04:00
jinjunzh
3a7fa8887f refine readCompressedDataBlockForAsynchronous 2022-07-19 10:23:25 -04:00
jasperzhu
c044c67745
Merge branch 'master' into dev_intel_iaa_deflate 2022-07-19 15:32:57 +08:00
jinjunzh
a53ed21dba restore the function doDecompressDataSynchronous 2022-07-18 17:44:15 -04:00
jinjunzh
332d3fd1a7 add comments for doDecompressDataAsynchronous 2022-07-18 14:40:49 -04:00
jinjunzh
456ae5ea44 add comments for readCompressedDataBlockForAsynchronous 2022-07-18 10:39:20 -04:00
Anton Popov
5605b60c97
Merge pull request #39039 from CurtizJ/randomize-more-settings 
Add more settings for randomization
 2022-07-18 15:05:41 +02:00
Alexey Milovidov
f1256c3aef
Merge pull request #37891 from guowangy/lz4-decompress-vbmi 
LZ4 decompress: add VBMI optimized copyOverlap32Shuffle
 2022-07-18 00:37:59 +03:00
Wangyang Guo
7ef3f56066 lz4 decompress: coding style fix 2022-07-16 22:10:16 +08:00
jinjunzh
f05f3cb69a fixed cosmetic issues 2022-07-15 17:04:27 -04:00
jinjunzh
e3735f7464 fixed cosmetic issues 2022-07-15 16:12:26 -04:00
jinjunzh
5104e4200a fixed cosmetic issues 2022-07-15 15:32:10 -04:00
Nikita Taranov
ecdc56bf31
Merge branch 'master' into hlee-s390x-LZ4 2022-07-15 14:23:41 +02:00
Anton Popov
76eec62469
Merge branch 'master' into randomize-more-settings 2022-07-15 14:12:49 +02:00
Wangyang Guo
c9e15017ac lz4 decompress: dynamic dispatch with TargetSpecific 2022-07-15 16:07:19 +08:00
avogar
9291d33080 Pass const std::string_view & by value, not by reference 2022-07-14 16:11:57 +00:00
Anton Popov
d4cb2184c2
Merge branch 'master' into randomize-more-settings 2022-07-14 17:01:07 +02:00
jinjunzh
1c6d70f9b3 fixed sw job buffer missing 2022-07-13 16:12:15 -04:00
HarryLeeIBM
2caa3f8803 Fix LZ4 decompression issue for s390x 2022-07-13 11:45:58 -07:00
Wangyang Guo
f275b35bba Dynamic dispatching for best_variant selection 2022-07-13 15:20:15 +08:00
jinjunzh
b699764bbe unified sync implemetation with async 2022-07-12 14:32:53 -04:00
Anton Popov
3b45e6e347 fix reading of empty arrays in reverse order with uncompressed cache (addition to #36215) 2022-07-12 03:08:12 +00:00
Robert Schulze
1a7727a254
Prefix overridden add_executable() command with "clickhouse_" 
A simple HelloWorld program with zero includes except iostream triggers
a build of ca. 2000 source files. The reason is that ClickHouse's
top-level CMakeLists.txt overrides "add_executable()" to link all
binaries against "clickhouse_new_delete". This links against
"clickhouse_common_io", which in turn has lots of 3rd party library
dependencies ... Without linking "clickhouse_new_delete", the number of
compiled files for "HelloWorld" goes down to ca. 70.

As an example, the self-extracting-executable needs none of its current
dependencies but other programs may also benefit.

In order to restore access to the original "add_executable()", the
overriding version is now prefixed. There is precedence for a
"clickhouse_" prefix (as opposed to "ch_"), for example
"clickhouse_split_debug_symbols". In general prefixing makes sense also
because overriding CMake commands relies on undocumented behavior and is
considered not-so-great practice (*).

(*) https://crascit.com/2018/09/14/do-not-redefine-cmake-commands/
 2022-07-11 19:36:18 +02:00
jinjunzh
bcba581191 fixed cosmetic issues 2022-07-11 12:08:35 -04:00
mergify[bot]
62c62a2df7
Merge branch 'master' into lz4-decompress-vbmi 2022-07-11 14:23:17 +00:00
jinjunzh
bc03e10870 make job pool static member to private 2022-07-09 17:50:57 -04:00
jinjunzh
bc775ecc5b add assert for Illegal path 2022-07-09 17:29:47 -04:00
jinjunzh
044c14745e make job pool name comprehensible 2022-07-09 17:17:18 -04:00
jinjunzh
eb06c66869 cosmetic issue 2022-07-09 14:57:38 -04:00
jinjunzh
816e974ca2 rename deflate to deflate_qpl 2022-07-09 14:42:01 -04:00
jinjunzh
825b7511e4 improve cosmetics 2022-07-09 14:13:12 -04:00
jinjunzh
7cb5b35bef remove qpl from fasttest 2022-07-09 13:54:23 -04:00
jinjunzh
212f342af5 change to std random 2022-07-09 12:03:25 -04:00
jinjunzh
0fdce07cc6 add dedicated value for HW codec fail 2022-07-09 11:12:41 -04:00
jinjunzh
545e6b305c improve log print text 2022-07-09 10:09:23 -04:00
jinjunzh
46d7d3b5c9 constant always goes to the right side 2022-07-08 15:03:10 -04:00
jinjunzh
289c6148e4 member to be LOWERCASE_UNDERSCORE 2022-07-08 12:20:24 -04:00
jinjunzh
7eb71143db add a one-line method for resetting a job lock 2022-07-08 12:09:58 -04:00
jinjunzh
7a3c21174d use C++17 if initializer 2022-07-08 11:55:16 -04:00
jinjunzh
6045681bb4 improve jobPoolReady 2022-07-07 17:21:51 -04:00
jinjunzh
5681e50da8 improve job pool initialization 2022-07-07 17:13:20 -04:00
jinjunzh
2c8f308bbc improve buffer allocation 2022-07-07 16:05:52 -04:00
jinjunzh
29fee729e3 revise the way of get job size 2022-07-07 15:17:23 -04:00
jinjunzh
2765e2a259 remove unused function in deflate job pool 2022-07-07 14:44:42 -04:00
jinjunzh
f80aea9eac Cosmetic issue and improve flush function 2022-07-07 14:14:47 -04:00
jinjunzh
ad82bb80ed Cosmetic: Constants to be UPPERCASE_AND_UNDERSCORE 2022-07-07 12:10:06 -04:00
jinjunzh
85eae40765 remove inline due to negligible benefits 2022-07-07 10:26:57 -04:00
jinjunzh
43c5a68d14 Cosmetic work 2022-07-07 10:04:17 -04:00
jinjunzh
6fa036f6ee remove qpl from clickhouse-keeper 2022-07-06 23:48:09 -04:00
jinjunzh
3422c0479d add more comments for flush 2022-07-06 23:31:22 -04:00
jinjunzh
4cdc660406 change flush name to be more descriptive 2022-07-06 21:37:11 -04:00
jinjunzh
7b262b62c2 restore decompress to be const 2022-07-06 18:34:31 -04:00
jinjunzh
567f90d8a4 add CodecMode for deflate codec 2022-07-06 18:18:55 -04:00
jinjunzh
fbe3d84860 revise typo issue 2022-07-05 15:29:53 -04:00
jinjunzh
ba38969e9c enable qpl for clickhouse-keeper build 2022-07-05 15:28:41 -04:00
jinjunzh
fe451a4317 remove compress sync interface 2022-06-28 22:52:11 -04:00
jasperzhu
b2f98fa73b
Merge branch 'master' into dev_intel_iaa_deflate 2022-06-27 10:21:21 +08:00
alesapin
50801e41c5 Merge branch 'master' into refactor-something-in-part-volumes 2022-06-19 14:05:46 +02:00
Anton Popov
1523c9c9e5 fix filling of empty Nested + small refactoring 2022-06-17 01:10:52 +00:00
Anton Popov
92b7b9789a
try to fix fpc codec 2022-06-16 03:41:09 +02:00
jasperzhu
bca18b2caf
Merge branch 'master' into dev_intel_iaa_deflate 2022-06-16 08:56:27 +08:00
Nikolai Kochetov
2a9a63ac7b Merge branch 'master' into refactor-something-in-part-volumes 2022-06-15 15:35:26 +02:00
Robert Schulze
a0d936cc9f
Small follow-up for FPC codec 
- add paper reference + doxygen

- remove endianness handling (ClickHouse assumes little endian)

- documentation

- other minor stuff
 2022-06-15 14:21:28 +02:00
jinjunzh
1236d74ae7 restructure code design to be compatible both of hardware and software codec 2022-06-08 11:47:44 -04:00
jinjunzh
2b684ff40e remove sw job pool 2022-06-08 09:28:35 -04:00
koloshmet
d5064dd5b5 so many improvements 2022-06-08 02:17:34 +03:00
Wangyang Guo
50ac01c9b8 Enable copyOverlap32Shuffle 2022-06-06 13:45:19 +08:00
Wangyang Guo
c9ff52f2e9 Add VBMI optimized copyOverlap32Shuffle 2022-06-06 13:45:01 +08:00
jinjunzh
c7ca746994 fixed build issues in clickhouse-keeper 2022-06-05 14:46:28 -04:00
jinjunzh
2170a99484 revise ENABLE_QPL_COMPRESSION build option 2022-06-03 21:06:55 -04:00
Nikolai Kochetov
89c5855d20 Merge branch 'master' into refactor-something-in-part-volumes 2022-06-02 12:19:07 +02:00
jinjunzh
e1426c7e9d add build option for iaa compression 2022-06-01 11:47:43 -04:00
jinjunzh
53be924537 fixed invalid case style 2022-05-31 15:40:31 -04:00
jasperzhu
fc03af2259
Merge branch 'master' into dev_intel_iaa_deflate 2022-05-30 17:49:04 +08:00
koloshmet
7e69779575 added fpc codec to float perftest 2022-05-26 22:32:56 +03:00
koloshmet
821100f145 code style fixes 2022-05-26 11:09:36 +03:00
koloshmet
1bfeb982af fixed decoding parameters 2022-05-26 06:58:34 +03:00
koloshmet
cd3e28e3b1 fixed decoding parameters 2022-05-26 06:53:48 +03:00
koloshmet
adf888811c fixed max size computation 2022-05-26 04:53:07 +03:00
koloshmet
0697b9ffcf FPC codec 2022-05-25 22:04:39 +03:00
Maksim Kita
168814ed78
Merge pull request #37442 from kitaisreal/compressed-write-buffer-added-comment 
CompressedWriteBuffer added comment
 2022-05-23 23:44:33 +02:00
Maksim Kita
3a22d8ef2c CompressedWriteBuffer added comment 2022-05-23 12:09:44 +02:00
Robert Schulze
0f6715bd91
Follow-up to PR #37300: semicolon warnings 
In PR #37300, Alexej asked why we the compiler does not warn about
unnecessary semicolons, e.g.

  f()
  {
  }; // <-- here

The answer is surprising: In C++98, above syntax was disallowed but by
most compilers accepted it regardless. C++>11 introduced "empty
declarations" which made the syntax legal.

The previous behavior can be restored using flag
-Wc++98-compat-extra-semi. This finds many useless semicolons which were
removed in this change. Unfortunately, there are also false positives
which would require #pragma-s and HAS_* logic (--> check_flags.cmake) to
suppress. In the end, -Wc++98-compat-extra-semi comes with extra effort
for little benefit. Therefore, this change only fixes some semicolons
but does not enable the flag.
 2022-05-20 15:06:34 +02:00
Maksim Kita
495e04d2c3
Merge pull request #37242 from jinjunzh/insert_improve_jsp 
Improve CompressedWriteBuffer to avoid unnecessary memcpy
 2022-05-19 10:59:35 +02:00
jinjunzh
f413b49364 fixed logger header file path 2022-05-16 14:59:16 -04:00
Maksim Kita
fd28c19c1c
Update CompressedWriteBuffer.cpp 2022-05-16 20:44:22 +02:00
jinjunzh
f334cd371f fixed code style issue 2022-05-16 17:21:20 -04:00
jinjunzh
2ae9e22248 Improve CompressedWriteBuffer to avoid unnecessary memcpy 2022-05-16 16:22:13 -04:00
jinjunzh
9309735016 remove unused job release call 2022-05-10 16:46:50 -04:00
Robert Schulze
1b81bb49b4
Enable clang-tidy modernize-deprecated-headers & hicpp-deprecated-headers 
Official docs:

  Some headers from C library were deprecated in C++ and are no longer
  welcome in C++ codebases. Some have no effect in C++. For more details
  refer to the C++ 14 Standard [depr.c.headers] section. This check
  replaces C standard library headers with their C++ alternatives and
  removes redundant ones.
 2022-05-09 08:23:33 +02:00
Nikolai Kochetov
35095191eb Merge branch 'master' into refactor-something-in-part-volumes 2022-05-03 17:51:47 +02:00
jasperzhu
7f47e9b934
Merge branch 'master' into dev_intel_iaa_deflate 2022-05-03 15:47:57 +08:00
jasperzhu
f4b275c4ac
Improve decompression in readbig to avoid unnecessary memcpy (#36095) 2022-05-02 15:06:52 +02:00
Amos Bird
4a5e4274f0
base should not depend on Common 2022-04-29 10:26:35 +08:00
jinjunzh
8788185e82 fixed typos issue in code 2022-04-27 10:52:44 -04:00
Nikolai Kochetov
e44af67fee Merge branch 'master' into refactor-something-in-part-volumes 2022-04-26 21:08:00 +02:00
jasperzhu
05769b4a9c
Merge branch 'master' into dev_intel_iaa_deflate 2022-04-26 16:30:27 +08:00
jinjunzh
68a7e4b9b5 preliminary patch for iaa deflate solution 2022-04-26 14:14:09 -04:00
Nikolai Kochetov
8c00692844 Part 8 2022-04-22 16:58:09 +00:00
Robert Schulze
b24ca8de52
Fix various clang-tidy warnings 
When I tried to add cool new clang-tidy 14 warnings, I noticed that the
current clang-tidy settings already produce a ton of warnings. This
commit addresses many of these. Almost all of them were non-critical,
i.e. C vs. C++ style casts.
 2022-04-20 10:29:05 +02:00
Anton Popov
0595b5c22b fix reading of empty arrays in reverse order 2022-04-13 21:50:57 +00:00
Nikolai Kochetov
5a1392a8e3 Try refactor something (1) 2022-04-05 19:12:48 +00:00
alesapin
77e700b1cf
Update src/Compression/CompressionFactory.cpp 2022-03-28 11:25:08 +02:00
alesapin
a4ab73619f
Fix UBSan build 2022-03-22 15:05:31 +01:00
Maksim Kita
65c52298b6 Fix clang-tidy warnings in Compression, Coordination, Core folders 2022-03-14 18:17:35 +00:00
kssenii
5260822964 Merge master 2022-03-08 18:21:28 +01:00
kssenii
e231c3a3e0 Fix split build 2022-03-08 18:05:55 +01:00
Maksim Kita
b1a956c5f1 clang-tidy check performance-move-const-arg fix 2022-03-02 18:15:27 +00:00
Amos Bird
ba19c7cf44
Slightly better interface of compressed buffer 2022-02-17 14:31:22 +08:00
alesapin
02a93cb852 Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-08 19:42:27 +03:00
Anton Popov
1b16db72c3 fix consecutive backward seeks in seekable read buffers 2022-02-07 17:20:26 +03:00
alesapin
ba28c2c013 Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-07 12:44:56 +03:00
Anton Popov
10b8684003 fix rare bug in reading of empty arrays 2022-02-06 20:45:59 +00:00
alesapin
18f08ed932
Revert "Revert "Add func tests run with s3"" 2022-02-01 10:20:06 +03:00
alexey-milovidov
cbfcd45be3
Revert "Add func tests run with s3" 2022-02-01 05:46:13 +03:00
alesapin
5be3968526 Fix bug in cache compressed read buffer 2022-01-31 13:40:26 +03:00
alesapin
ea8ec87b34 Fix stupid bug 2022-01-30 21:03:12 +03:00
Azat Khuzhin
a773e7ff01 Remove unbundled libpqxx support 2022-01-20 10:01:59 +03:00
Azat Khuzhin
8485abd52b Remove unbundled lz4 support 2022-01-20 08:47:16 +03:00
Anton Popov
54f51444c0 Merge remote-tracking branch 'upstream/master' into HEAD 2021-12-01 15:49:02 +03:00
Anton Popov
120cb79bac
Merge pull request #31826 from CurtizJ/fix-uncompressed-cache 
Fix reading with uncompressed cache
 2021-11-27 17:36:52 +03:00
alexey-milovidov
83391977f8
Merge pull request #31896 from Algunenano/parser_deps 
Reduce the files that depend on parser headers
 2021-11-27 00:18:31 +03:00
Raúl Marín
91bf938075 Reduce dependencies on ASTLiteral.h 
590 -> 537
 2021-11-26 17:54:57 +01:00
Raúl Marín
051dddd8df Reduce dependencies on ASTIdentifier.h 
Goes from rebuilding 483 objects to 165 when it's modified
 2021-11-26 16:49:40 +01:00
Anton Popov
923159d408 fix typos 2021-11-26 17:47:40 +03:00
Anton Popov
c3f076fd84 fix reading with uncompressed cache 2021-11-25 21:20:03 +03:00
Kruglov Pavel
d9c1a0c8ec
Merge branch 'master' into fix-write-buffers 2021-11-20 17:48:24 +03:00
Vladimir Smirnov
48451182f8 Initial support for risc-v 
Make ClickHouse compilable and runnable on risc-v 64

So far only basic functionality was tested (on real hw),
clickhouse server runs, exceptions works, client works,
simple tests works.

What doesn't work:
 1. traces - they are always empty
 2. system.stack_trace only have first frame
 2021-11-11 19:23:34 +01:00
avogar
51831afff8 Fix tests 2021-11-11 20:27:23 +03:00
avogar
c521a9131a Small refactoring of WriteBiffer-s 2021-11-11 02:11:18 +03:00
Anton Popov
d50137013c Merge remote-tracking branch 'upstream/master' into HEAD 2021-11-01 16:55:53 +03:00
Anton Popov
0099dfd523 refactoring of SerializationInfo 2021-10-29 20:21:02 +03:00
Filatenkov Artur
44b5dd1161
Refactoring in codec encrypted (#30564) 
* refactoring

* remove mistake in docs
 2021-10-29 11:44:28 +03:00
kssenii
7e2ea97e3c Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-28 18:46:21 +03:00
Alexey Milovidov
8b4a6a2416 Remove cruft 2021-10-28 02:10:39 +03:00
kssenii
52296e8b04 Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-25 00:22:32 +03:00
Azat Khuzhin
258b62d375 Remove one unused CompressedReadBufferFromFile ctor 2021-10-24 18:58:22 +03:00
Azat Khuzhin
93c34824c0 Fix explicit 0 size for createReadBufferFromFileBase() 2021-10-24 18:58:22 +03:00
kssenii
39b9e9c258 Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-22 15:40:41 +03:00
kssenii
56f0560c46 Asserts and read till end option 2021-10-21 20:54:03 +03:00
kssenii
31cd71c849 Clean up 2021-10-20 09:37:18 +03:00
kssenii
143e912e4e Fix split build 2021-10-19 22:45:46 +03:00
kssenii
49106f407f Fix some checks 2021-10-19 09:34:27 +03:00
kssenii
d5d4817350 Add disk proxies back, add setReadUntilPosition to ReadBuffer 2021-10-18 18:35:11 +03:00
kssenii
3995506d37 Adjust range reader for remote fs reads 2021-10-15 11:36:26 +03:00
Anton Popov
92413aed68 better interfaces for IDataType and ISerialization 2021-10-14 05:36:49 +03:00
Kseniia Sumarokova
185d58a7cc
Merge pull request #29766 from ClickHouse/remove_redundant_seeks 
Less seeks in compressed buffers
 2021-10-07 17:04:50 +03:00
alesapin
621c9884fc Account offset in readBig call 2021-10-06 12:39:54 +03:00
Filatenkov Artur
dc657a2c5e
Add error for multiple keys without current_key_id (#29546) 
* add error for multiple keys without current_key_id

* add exception if one key with non zero id is used without current_key_id

* refactoring
 2021-10-05 14:37:54 +03:00
alesapin
64158c011b Remove logging 2021-10-05 13:12:31 +03:00
alesapin
c2f34928e0 Fixup 2021-10-05 13:04:44 +03:00
alesapin
3a8126372c Remove redundant header 2021-10-05 12:19:04 +03:00
alesapin
8186339307 Less seeks in compressed buffers 2021-10-05 12:11:25 +03:00
Denis Glazachev
ca7b69b0f3 Manipulate with -Wreserved-identifier only if HAS_RESERVED_IDENTIFIER has been detected 2021-10-03 17:42:36 +04:00
Mike Kot
65e6e211b4 Merge remote-tracking branch 'upstream/master' into improvement/fn-traits 2021-10-02 21:37:50 +02:00
Maksim Kita
395119339f Merge branch 'master' into rename-common 2021-10-02 17:15:10 +03:00
Mike Kot
5d38930a3c Fixing gcc build due to a bug in compiler 2021-10-02 14:27:52 +02:00
Alexey Milovidov
2ad5dbdfe8 Rename "common" to "base" 2021-10-02 11:30:38 +03:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
alexey-milovidov
f21ea8f160
Update CompressedReadBufferBase.cpp 2021-10-02 06:13:46 +03:00
Yatsishin Ilya
2c0071bc6d wip 2021-10-01 09:32:54 +03:00
Alexey Milovidov
cd7f9d981c Remove ya.make 2021-09-25 04:22:54 +03:00