Commit Graph

125 Commits

Author SHA1 Message Date
Alexey Milovidov
6d58d85579 Fixed warnings 2019-12-18 08:12:03 +03:00
kreuzerkrieg
9612bb0e79 More GCC warnings 2019-12-16 23:19:50 +02:00
Amos Bird
ad12099642
ubsan fix 2019-11-02 17:00:03 +08:00
Alexander Kuzmenkov
8a1d57bddb Include PODArray.h into fewer files. 2019-10-11 14:28:16 +03:00
Alexander Kuzmenkov
ec2d9a083c Remove excessive includes. 2019-09-30 15:31:14 +03:00
alexey-milovidov
3270ad439a
Update compressed_buffer_fuzz.cpp 2019-09-23 00:18:14 +03:00
alexey-milovidov
8fd66ac4fd
Update compressed_buffer_fuzz.cpp 2019-09-22 15:52:33 +03:00
Eldar Zaitov
b174833bdf Initial fuzzing support with libfuzzer 2019-09-22 13:34:09 +03:00
Amos Bird
a1310ffbe6 Split libdbms.so using object library
Now the linking time of incremental builds are around 1-2 seconds
2019-09-11 01:56:01 +08:00
chertus
12084270de fix build 2019-08-26 19:58:40 +03:00
chertus
d17a64c28d fix vulnerabilities 2019-08-26 17:39:49 +03:00
proller
d09c5c871b Build fixes (#6491)
* Fix build

* cmake: fix cpuinfo

* Fix includes after processors merge

Conflicts:
	dbms/src/Processors/Formats/Impl/CapnProtoRowInputFormat.cpp
	dbms/src/Processors/Formats/Impl/ParquetBlockOutputFormat.cpp
	dbms/src/Processors/Formats/Impl/ProtobufRowInputFormat.cpp
	dbms/src/Processors/Formats/Impl/ProtobufRowOutputFormat.cpp

* Fix build in gcc8

* fix test link

* fix test link

* Fix test link

* link fix

* Fix includes after processors merge 2

Conflicts:
	dbms/src/Processors/Formats/Impl/ParquetBlockInputFormat.cpp

* Fix includes after processors merge 3

* link fix

* Fix likely/unlikely conflict with cython

* Fix conflict with protobuf/stubs/atomicops.h

* remove unlikely.h

* Fix macos build (do not use timer_t)
2019-08-18 01:53:46 +03:00
Alexey Milovidov
c96fa2c080 Fixed build 2019-08-13 01:53:19 +03:00
Alexey Milovidov
6ebd002926 Fixed build 2019-08-13 01:19:20 +03:00
Vasily Nemkov
95a38b9e19 Fixed builds on GCC-9 and Clang-8 2019-08-12 22:43:03 +03:00
Vasily Nemkov
923c6889e8 Fixed Gorilla encoding error on small sequences.
Added test cases for small sequences;
Refurbished test cases for codecs;
2019-08-12 07:59:20 +03:00
kreuzerkrieg
112fc71276 adding -Wshadow for GCC 2019-08-09 23:58:16 +03:00
Alexey Milovidov
98ea652ad6 Whitespaces 2019-08-09 18:30:39 +03:00
proller
77bd2624ea Build fixes (#6016)
* Fix building without submodules

* Fix more gcc9 warnings

* was wrong!
../dbms/src/IO/WriteBufferAIO.cpp:277:54: error: result of comparison 'ssize_t' (aka 'long') > 9223372036854775807 is always false [-Werror,-Wtautological-type-limit-compare]
    if ((static_cast<ssize_t>(flush_buffer.offset()) > std::numeric_limits<off_t>::max()) ||
         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* mimalloc off MI_OVERRIDE

* Fix freebsd build

* Fix shared build

* Fix build in gcc9

* Fix split build

* fix

* fix

* fix

* fix

* clean

* zstd

* Update CMakeLists.txt

* Update Defines.h

* Try fix -Wsign-compare

* Freebsd fix

* Add missing libs

* Fix double-conversion include for copy-headers

* Fix zlib link

* Fix includes for arcadia

* Fix includes for arcadia

* Fix includes for arcadia

* Freebsd fix

* Arcadia fixes

* Update QueryProfiler.cpp

* Freebsd fix

* clean

* 19.11: Fixes for arcadia porting (#6223)

* fix glibc-compatibility

* M_LIBRARY fix

* wip

* Fix

* Fix arm build

* unwind fix

* Update CMakeLists.txt
2019-08-04 03:19:03 +03:00
Alexey Milovidov
856cc1b9c5 Added thread_local_rng 2019-07-28 18:30:38 +03:00
alexey-milovidov
5215d13437
Merge pull request #5998 from Enmk/double_delta_fix_2
Fixed DoubleDelta encoding cases for random Int32 and Int64.
2019-07-16 00:54:22 +03:00
alexey-milovidov
ea54cdbede
Update CompressionCodecDoubleDelta.cpp 2019-07-16 00:53:23 +03:00
Vasily Nemkov
0f4075478e Fixed DoubleDelta encoding cases for random Int32 and Int64. 2019-07-15 08:17:14 +03:00
Alexey Milovidov
499d798b34 Remove "clickhouse_compression" library because it is not a library (cyclic dependency via IDataType) 2019-07-10 00:37:43 +03:00
Alexey Milovidov
8778b9a1f3 "clickhouse_compression" library is dependent on "typeinfo for DB::IDataType" (at least when building with ubsan) 2019-07-09 04:09:42 +03:00
Alexey Milovidov
52bb292327 Merge branch 'master' into libcxx-as-submodule 2019-07-09 01:58:09 +03:00
Alexey Milovidov
ac20c515ab Better code in unit tests 2019-07-08 04:58:31 +03:00
Alexey Milovidov
6b9df067d8 Merge branch 'master' into libcxx-as-submodule 2019-07-03 22:34:58 +03:00
alexey-milovidov
7ea33203d7
Merge pull request #5742 from 4ertus2/t64
T64 full bit-transpose variant
2019-07-03 01:56:34 +03:00
chertus
d1250e6042 fix build 2019-07-02 15:01:39 +03:00
Vasily Nemkov
c5b2ba2a25 Fixed DoubleDelta codec edge case
Casused by mistreating negative double delta value as HUGE unsigned value, crippling compression ratio.
2019-07-02 07:58:03 +03:00
chertus
13fa9a07e5 Merge branch 'master' into t64 2019-07-01 16:40:32 +03:00
chertus
37b7ff7a0a fix maning, add comment 2019-07-01 16:35:04 +03:00
Alexander Kuzmenkov
0116c10e41 Require explicit type in unalignedStore
This is a follow-up to PR #5786, which fixed a segfault caused by
an unexpected deduced type for unalignedStore. To prevent future errors
of this kind, require a caller to specify the stored type explicitly.
2019-06-28 20:33:14 +03:00
alesapin
cfd8df6568 Fix bug in delta codec caused by genious C++ implicit type conversion and consequent template type deduction 2019-06-28 13:42:36 +03:00
chertus
e681bef71d T64 full transpose variant 2019-06-25 16:59:33 +03:00
Vasily Nemkov
138087d2e1 Performance tests for Gorilla and DoubleDelta
Testing INSERTs and SELECTs on codec combinations:
 * DoubleDelta (or Gorilla)
 * LZ4
 * DD (or G) + LZ4
On 3 types of data:
 * sequential
 * monotonic
 * random.
2019-06-25 13:47:55 +03:00
Vasily Nemkov
e79c3d0e22 Post-PR fixes stage 3
maskLowBits()
building for clang
more test cases for codecs.
2019-06-19 13:38:15 +03:00
Vasily Nemkov
81f9055d81 Post-PR fixes #2
Fixed style issues and build for clang-7;
64-bit buffer for BitReader and BitWriter;
Fixed overflow and writing more bytes on flushing;
Added maskLowBits() and tests for it.
2019-06-17 21:49:01 +03:00
Vasily Nemkov
0ebb145d32 Post-PR fixes:
* BitHelpers.cpp was removed, corresponding code was moved to the header
* BitIO test as GTest-based test binary
* gtest-based unit test for DoubleDelta and Gorilla codecs
* getLeadingZeroBits from SFINAE to consexpr if
* removed couple of unneeded if's
* Fixed sql-test to use one table
2019-06-17 21:39:45 +03:00
Vasily Nemkov
dfd8ff7675 Gorilla column encoding
Added Gorilla column encoding for any fixed-width type;
Added tests for Float32 and Float64 values.
2019-06-17 21:39:42 +03:00
Vasily Nemkov
4ae63072d0 DoubleDelta column encoding.
Added DoubleDelta column encoding + test cases;
Added BitWriter and BitReader that allow to read data from ReadBuffer
and write data to WriteBuffer bit by bit, up to 64 bits at once;
Added test for BitReader and BitWriter.
2019-06-17 21:38:27 +03:00
chertus
08983b0669 T64 codec optimisation: unroll cycles 2019-06-12 16:03:51 +03:00
chertus
f9abc8194a make ubsan happy v2 2019-06-12 03:12:38 +03:00
chertus
0c0a1517fb make ubsan happy & minor perf improvement 2019-06-12 00:32:43 +03:00
chertus
3648889aed T64 codec: add test for signed & related fixes 2019-06-11 21:25:34 +03:00
chertus
10c720f059 T64 codec: fix getMaxCompressedDataSize and test 2019-06-11 17:14:45 +03:00
chertus
f099ee5680 T64 support signed types (not tested) 2019-06-10 22:14:48 +03:00
chertus
a95e0015dc fix build and tests 2019-06-07 21:11:50 +03:00
chertus
3007975199 T64 codec concept 2019-06-07 19:12:45 +03:00