Commit Graph

465 Commits

Author SHA1 Message Date
Yakov Olkhovskiy
fadbeb8ebd T64 codec support for IPv4 2023-02-22 19:25:48 +00:00
Alexey Milovidov
d8cda3dbb8 Remove PVS-Studio 2023-02-19 23:30:05 +01:00
Suzy Wang
e140bed3ea
Merge branch 'master' into z-build-0120 2023-02-13 09:06:48 -05:00
kssenii
3067c1d723 Merge remote-tracking branch 'upstream/master' into resubmit-prefetches 2023-02-11 11:36:23 +01:00
Suzy Wang
64313d7020
Merge branch 'master' into z-build-0120 2023-02-09 14:49:22 -05:00
kssenii
b0b865c32e Resubmit prefetches 2023-02-08 21:26:24 +01:00
Robert Schulze
84b9ff450f
Fix terribly broken, fragile and potentially cyclic linking
Sorry for the clickbaity title. This is about static method
ConnectionTimeouts::getHTTPTimeouts(). It was be declared in header
IO/ConnectionTimeouts.h, and defined in header
IO/ConnectionTimeoutsContext.h (!). This is weird and caused issues with
linking on s390x (##45520). There was an attempt to fix some
inconsistencies (#45848) but neither did @Algunenano nor me at first
really understand why the definition is in the header.

Turns out that ConnectionTimeoutsContext.h is only #include'd from
source files which are part of the normal server build BUT NOT part of
the keeper standalone build (which must be enabled via CMake
-DBUILD_STANDALONE_KEEPER=1). This dependency was not documented and as
a result, some misguided workarounds were introduced earlier, e.g.
0341c6c54b

The deeper cause was that getHTTPTimeouts() is passed a "Context". This
class is part of the "dbms" libary which is deliberately not linked by
the standalone build of clickhouse-keeper. The context is only used to
read the settings and the "Settings" class is part of the
clickhouse_common library which is linked by clickhouse-keeper already.

To resolve this mess, this PR

- creates source file IO/ConnectionTimeouts.cpp and moves all
  ConnectionTimeouts definitions into it, including getHTTPTimeouts().

- breaks the wrong dependency by passing "Settings" instead of "Context"
  into getHTTPTimeouts().

- resolves the previous hacks
2023-02-05 20:49:34 +00:00
bkuschel
1f3b1c697f
Add newline 2023-02-04 15:39:28 -05:00
bkuschel
dc995f7c67
Fix size,free ctx,formatting 2023-02-04 15:03:04 -05:00
Boris Kuschel
d9f3698a43
Fix copy/pasta 2023-02-04 14:59:47 -05:00
Boris Kuschel
1d4cf4fe69
Add encryption support with openssl 2023-02-04 14:59:34 -05:00
Suzy Wang
503c64a45d Update as suggested 2023-02-02 20:44:32 +00:00
Suzy Wang
716d2c4ffb Update as suggested 2023-02-02 20:41:06 +00:00
Robert Schulze
3472ed4f0d
Make some classes 'final' 2023-02-02 15:59:39 +00:00
Robert Schulze
7cc1c19227
Fix typo 2023-01-30 10:25:16 +00:00
Robert Schulze
856eba0a4b
Mark delta/doubledelta codec followed by a time series codec as suspicious 2023-01-29 08:51:13 +00:00
Alexey Milovidov
bc2f454522
Merge branch 'master' into block-non-float-gorilla-v2 2023-01-28 03:30:12 +03:00
Robert Schulze
83593810ff
Make check work with Nullable / Array / Tuple 2023-01-27 13:02:54 +00:00
Alexander Tokmakov
c366806c3e
Merge pull request #45527 from ClickHouse/exception_message_patterns4
Better formatting for exception messages 2
2023-01-27 15:31:52 +03:00
Robert Schulze
227b8676cd
Tiny improvements around the Gorilla/Delta codecs 2023-01-26 11:11:24 +00:00
Robert Schulze
9c48ac79c8
Fix issue #45195 2023-01-26 10:44:26 +00:00
Robert Schulze
574cab5d7e
Remove transitory parameter 2023-01-24 11:05:29 +00:00
Robert Schulze
aaf7653108
Merge remote-tracking branch 'origin/master' into block-non-float-gorilla-v2 2023-01-24 10:14:10 +00:00
Suzy Wang
1176260b3c
Merge branch 'master' into z-build-0120 2023-01-24 01:14:42 -05:00
Alexander Tokmakov
3f6594f4c6 forbid old ctor of Exception 2023-01-23 22:18:05 +01:00
Alexander Tokmakov
70d1adfe4b
Better formatting for exception messages (#45449)
* save format string for NetException

* format exceptions

* format exceptions 2

* format exceptions 3

* format exceptions 4

* format exceptions 5

* format exceptions 6

* fix

* format exceptions 7

* format exceptions 8

* Update MergeTreeIndexGin.cpp

* Update AggregateFunctionMap.cpp

* Update AggregateFunctionMap.cpp

* fix
2023-01-24 00:13:58 +03:00
Robert Schulze
4ece499f19
Fix build 2023-01-22 12:26:03 +00:00
Suzy Wang
19d26828a0 s390x build support 2023-01-20 21:16:55 +00:00
Robert Schulze
e6167d6b36
Deprecate Gorilla compression of non-float columns
Reasons:

1. The original Gorilla paper proposed a compression schema for pairs of
   time stamps and double-precision FP values. ClickHouse's Gorilla
   codec only implements compression of the latter and it does not
   impose any data type restrictions.
   - Data types != Float* or (U)Int* (e.g. Decimal, Point etc.) are
     definitely not supposed to be used with Gorilla.
   - (U)Int* types are debatable. The paper only considers
     integers-stored-as-FP-values, a practical use case for which
     Gorilla works well. Standalone integers are not considered which
     makes them at least suspicious.

2. Achieve consistency with FPC, another specialized floating-point
   timeseries codec, which rejects non-float data.

3. On practical datasets, ZSTD is often "good enough" (**) so it should
   be okay to disincentive non-ZSTD codecs a little bit. If needed,
   Delta and DoubleDelta codecs are viable alternative for slowly
   changing (time-series-like) integer sequences.

Since on-prem and hosted users may still have Gorilla-compressed
non-float data, this combination is only deprecated for now. No warning
or error will be emitted. Users are encouraged to migrate
Gorilla-compressed non-float data to an alternative codec. It is planned
to treat Gorilla-compressed non-float columns as "suspicious" six months
after this commit (i.e. in v23.6). Even then, it will still be possible
to set "allow_suspicious_codecs = true" and read and write
Gorilla-compressed non-float data.

(*) Sec. 4.1.2, "Gorilla restricts the value element in its tuple to a
    double floating point type.", https://doi.org/10.14778/2824032.2824078

(**) https://clickhouse.com/blog/optimize-clickhouse-codecs-compression-schema
2023-01-20 17:31:16 +00:00
Alexander Tokmakov
df75c24f01
Revert "Disallow Gorilla codec on non-float columns" 2023-01-16 19:14:28 +03:00
Robert Schulze
b6f12f9edd
Fix unit_tests_dbms 2023-01-15 14:49:04 +00:00
Robert Schulze
a4a6126c9d
Prohibit manual delta compression before floating-point time series compression 2023-01-14 20:09:50 +00:00
Robert Schulze
fbdaca4e2a
Code cleanup 2023-01-14 19:21:30 +00:00
Robert Schulze
5d3f0ec4a0
Disallow Gorilla codec on non-float columns
Cf. #45195
2023-01-13 16:53:28 +00:00
alesapin
6c8dbcc040 Fix flaky test 01459_manual_write_to_replicas_quorum_detach_attach and several typos 2023-01-03 16:27:51 +01:00
Alexander Gololobov
f75b203071
Merge pull request #44034 from ClickHouse/evillique-patch-1
Fix exception message
2022-12-08 23:30:09 +01:00
Robert Schulze
375f488156
Merge pull request #44024 from jinjunzh/iaadeflate_upgrade_qpl030
Upgrade Intel®-IAA/QPL-based DEFLATE_QPL Codec
2022-12-08 12:56:03 +01:00
Nikolay Degterinsky
16b33c204c
Fix exception message 2022-12-08 10:49:25 +01:00
Duc Canh Le
88716b1ec3 Minor fix 2022-12-08 09:39:25 +08:00
jinjunzh
47c3508337 fixed build issues for QPL 0.3.0 2022-12-07 16:41:12 -05:00
Sema Checherinda
f49936b9a0 remove tmp dir from inmemory part when freeze, more logs about old parts 2022-11-23 15:16:09 +00:00
Nikita Taranov
0b4e643c27
Add check to CompressionCodecDelta (#43255)
* Add check to CompressionCodecDelta

* Apply suggestions from code review

* Update src/Compression/CompressionCodecDelta.cpp
2022-11-19 14:16:14 +01:00
Sema Checherinda
7e73b187cc
Merge pull request #43159 from Algunenano/over-read
Fix several buffer over-reads
2022-11-15 15:04:24 +01:00
Raúl Marín
54db7c6520 Enforce checking read output 2022-11-11 10:56:18 +01:00
Tiaonmmn
331c1cbeb7
Update CompressionCodecDeflateQpl.cpp
Modify the data type to resolve warning of clang-tidy "implicit conversion".
2022-11-11 10:00:59 +08:00
Nikita Taranov
2c7708a03e
Add bound check to lz4 decompression (#42868)
* impl

* add test

* fix bullshit tidy error
2022-11-04 22:12:36 +01:00
Anton Popov
7715afa595
Merge pull request #42618 from CurtizJ/refactor-data-part
Try to save `IDataPartStorage` interface
2022-10-27 22:20:44 +02:00
Anton Popov
c8199bc125
Merge branch 'master' into refactor-data-part 2022-10-25 14:31:05 +02:00
Robert Schulze
c119cd2f00
Merge branch 'master' into update-libcxx-to-15 2022-10-24 08:29:37 +02:00
Anton Popov
b40d9200d2 better semantic of constsness of DataPartStorage 2022-10-23 15:24:20 +00:00