thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-04 21:42:39 +00:00
Code Issues Packages Projects Releases Wiki Activity
70,986 Commits 9,185 Branches 1,782 Tags 2.3 GiB
4d9ad3725d
Commit Graph

186 Commits

Author SHA1 Message Date
Alexey Milovidov
47a4c101b9 Merge branch 'master' into normalize-bigint 2021-05-08 16:41:15 +03:00
mwish
9328ec9c50 [Init] initialize commit 2021-05-08 18:07:39 +08:00
fibersel
f01b53bafb add newline 2021-05-06 16:47:23 +03:00
fibersel
94b159c510 add newlines 2021-05-06 16:34:27 +03:00
fibersel
cb53bbb7b0 add experimental codecs flag, add integration test for experimental codecs 2021-05-06 14:57:22 +03:00
Alexey Milovidov
1fda866881 Merge branch 'master' into normalize-bigint 2021-05-02 21:24:45 +03:00
Maksim Kita
318c4bb80d Add examples folder filter to ya.make.in 2021-04-30 11:25:52 +03:00
Nikita Mikhaylov
9f55424250 move to examples everywhere 2021-04-27 01:51:42 +03:00
Alexey Milovidov
649550c5ab Attempt to normalize big integers 2021-04-25 12:30:43 +03:00
Alexey Milovidov
8f01af62d9 Merge branch 'master' into normalize-bigint 2021-04-25 06:57:44 +03:00
Anton Popov
ea82e7725f
Merge pull request #21562 from CurtizJ/serialization-refactoring-4 
Refactoring of data types serialization
 2021-03-29 16:36:44 +03:00
alexey-milovidov
7ab2890c4d
Merge pull request #22206 from ClickHouse/mmap-cache 
Add cache for mmap IO
 2021-03-29 05:17:55 +03:00
Maksim Kita
459d00f999 Fixed tests 2021-03-28 22:42:34 +03:00
Alexey Milovidov
2a8ac01cdb Rename as suggested by Kita 2021-03-28 22:24:28 +03:00
Maksim Kita
608d37deed CachedCompressedReadBuffer fix cache usage 2021-03-28 21:32:38 +03:00
Alexey Milovidov
50f712e198 Integrate mmap cache to the infrastructure 2021-03-28 04:10:30 +03:00
Alexey Milovidov
0e3571478d Code review changes 2021-03-27 02:42:22 +03:00
Alexey Milovidov
e55f7e6333 Fix error 2021-03-26 02:21:59 +03:00
Alexey Milovidov
ad66c4a916 Fix error 2021-03-26 01:56:24 +03:00
Alexey Milovidov
a8ce138788 Speedup codec NONE 2021-03-25 23:08:47 +03:00
Anton Popov
6a15431be7 Merge remote-tracking branch 'upstream/master' into HEAD 2021-03-25 15:57:35 +03:00
Alexey Milovidov
9ca0566132 Minor modification 2021-03-17 02:08:33 +03:00
Anton Popov
ed42437219 Merge remote-tracking branch 'upstream/master' into HEAD 2021-03-13 01:41:26 +03:00
Yatsishin Ilya
7b69ef92be gcc old cast check failed 2021-03-10 18:04:34 +03:00
Anton Popov
bc417cf54a refactoring of serializations 2021-03-09 17:46:52 +03:00
Yatsishin Ilya
71d745b4ec implicit const conversion 2021-03-09 17:31:54 +03:00
Azat Khuzhin
98e3a99a88 Do not catch exceptions during final flush in writers destructors 
Since this hides real problems, since destructor does final flush and if
it fails, then data will be lost.

One of such examples if MEMORY_LIMIT_EXCEEDED exception, so lock
exceptions from destructors, by using
MemoryTracker::LockExceptionInThread to block these exception, and allow
others (so std::terminate will be called, since this is c++11 with
noexcept for destructors by default).

Here is an example, that leads to empty block in the distributed batch:

    2021.01.21 12:43:18.619739 [ 46468 ] {7bd60d75-ebcb-45d2-874d-260df9a4ddac} <Error> virtual DB::CompressedWriteBuffer::~CompressedWriteBuffer(): Code: 241, e.displayText() = DB::Exception: Memory limit (for user) exceeded: would use 332.07 GiB (attempt to allocate chunk of 4355342 bytes), maximum: 256.00 GiB, Stack trace (when copying this message, always include the lines below):

    0. DB::Exception::Exception<>() @ 0x86f7b88 in /usr/bin/clickhouse
    ...
    4. void DB::PODArrayBase<>::resize<>(unsigned long) @ 0xe9e878d in /usr/bin/clickhouse
    5. DB::CompressedWriteBuffer::nextImpl() @ 0xe9f0296 in /usr/bin/clickhouse
    6. DB::CompressedWriteBuffer::~CompressedWriteBuffer() @ 0xe9f0415 in /usr/bin/clickhouse
    7. DB::DistributedBlockOutputStream::writeToShard() @ 0xf6bed4a in /usr/bin/clickhouse
 2021-02-05 01:31:45 +03:00
Alexey Milovidov
093108bf81 Normalize BigInt implementation 2021-01-27 03:54:57 +03:00
Alexey Milovidov
5f13fdddc0 Remove useless code from BigInt 2021-01-26 21:22:40 +03:00
Alexey Milovidov
cc11e7c2f2 Whitespace 2021-01-25 00:23:06 +03:00
alexey-milovidov
ca825f36f1
Merge pull request #18853 from azat/dist-send-checksums 
Improve checksum checks for async INSERT into Distributed on the sender
 2021-01-15 17:12:54 +03:00
ygrek
8f2a830d83
add zstd long range option (#17184) 
* add zstd long compression option

* tests: add zstd long read-write test

Co-authored-by: Joris Giovannangeli <joris.giovannangeli@ahrefs.com>
Co-authored-by: ip <igor@ahrefs.com>
 2021-01-13 16:22:59 +03:00
Azat Khuzhin
946a126799 Add CheckingCompressedReadBuffer 
Buffer for reading from a compressed file with just checking checksums
of the compressed blocks, without any decompression, so result can be
proxied.
 2021-01-10 21:23:42 +03:00
alexey-milovidov
6be8338584
Update CompressedReadBufferBase.cpp 2021-01-08 22:41:24 +03:00
Azat Khuzhin
f2ff785460 Do not check bit flips for big buffers (since the size can be corrupted) 2021-01-08 02:26:17 +03:00
Alexey Milovidov
1572d2122b Respect network_compression_method in async INSERT into Distributed table 2021-01-06 03:41:34 +03:00
Alexey Milovidov
438f0f971b Fix the issue with async Distributed INSERTs and network_compression_method #18741 2021-01-06 03:24:42 +03:00
Alexey Milovidov
24f4fa6edf Follow Arcadia ya.make rules 2020-11-17 00:16:50 +03:00
Alexey Milovidov
3df04ce0c2 Follow Arcadia ya.make rules 2020-11-16 21:24:58 +03:00
Alexander Tokmakov
b94cc5c4e5 remove more stringstreams 2020-11-10 21:22:26 +03:00
Alexey Milovidov
fd84d16387 Fix "server failed to start" error 2020-11-07 03:14:53 +03:00
Alexey Milovidov
b56486510f Improve Arcadia 2020-11-05 12:55:01 +03:00
Maxim Akhmedov
3627fabfb9 Remove -g0 form Arcadia build settings. 2020-10-29 17:37:23 +03:00
tavplubix
a1f1db753b
Update CompressedReadBufferBase.cpp 2020-10-28 18:23:10 +03:00
Mikhail Filimonov
41971e073a
Fix typos reported by codespell 2020-10-27 12:04:03 +01:00
Ivan
1d170f5745
ASTTableIdentifier Part #1: improve internal representation of ASTIdentifier name (#16149) 
* Use only |name_parts| as primary name source

* Restore legacy logic for table restoration

* Fix build

* Fix tests

* Add pytest server config

* Fix tests

* Fixes due to review
 2020-10-24 21:46:10 +03:00
alesapin
50eba27d1a
Update CompressionFactory.h 2020-09-23 10:24:25 +03:00
alesapin
fd394f699c Better comments 2020-09-22 15:49:55 +03:00
alesapin
bcea1a4207 Remove redundant change 2020-09-21 18:55:21 +03:00
alesapin
8517dda807 First working test 2020-09-21 17:47:10 +03:00
alesapin
4debccb7fe Almost working version 2020-09-21 17:22:13 +03:00
alesapin
a3e999784b Buildable code 2020-09-21 14:24:10 +03:00
alesapin
7fc5a2dae1 Some intermediate stage 2 2020-09-21 10:18:23 +03:00
alesapin
e96a3ac5f3 DataType in enumerate streams 2020-09-18 14:37:58 +03:00
alesapin
5f9e7ed169
Merge pull request #14834 from ClickHouse/fix_compression_codec_perf 
Fix performance degradation during parts selection for merge
 2020-09-17 10:09:41 +03:00
alesapin
18398c1510 Fix style 2020-09-16 12:08:39 +03:00
alesapin
78cb39ff92
Update ICompressionCodec.h 2020-09-16 11:21:39 +03:00
alesapin
af8ed46261
Update CompressionCodecGorilla.cpp 2020-09-16 11:20:39 +03:00
alesapin
cf45eb1545
Update CompressionCodecDoubleDelta.cpp 2020-09-16 11:20:22 +03:00
alesapin
e3299d9249
Update CompressionCodecMultiple.cpp 2020-09-16 11:19:59 +03:00
alesapin
da386990fe
Update ICompressionCodec.cpp 2020-09-16 11:18:42 +03:00
alesapin
74c0b6eb76
Update CompressionCodecGorilla.cpp 2020-09-16 10:00:41 +03:00
Artem Zuikov
51ba12c2c3
Try speedup build (#14809) 2020-09-15 12:55:57 +03:00
alesapin
e96256d367 Trying to improve performance 2020-09-14 22:15:25 +03:00
Anton Popov
32135d96f9
Merge pull request #12183 from CurtizJ/polymorphic-parts-2 
Support codecs in compact parts
 2020-09-09 12:34:13 +03:00
Anton Popov
a419267dc6 minor fixes 2020-09-08 19:28:49 +03:00
Alexey Milovidov
e3924b8057 Fix "Arcadia" 2020-09-08 01:14:13 +03:00
Anton Popov
68913eab62 better reading from compact parts with differents codecs 2020-09-04 15:48:55 +03:00
Anton Popov
d5da58918e create less compressed streams while writing compact parts 2020-09-04 01:04:46 +03:00
Anton Popov
98011f6fc3 Merge remote-tracking branch 'upstream/master' into HEAD 2020-09-03 17:53:05 +03:00
alesapin
6f16c08191 Fix ya.make 2020-08-28 12:29:07 +03:00
alesapin
d8d7c46711 Remove ya.make update 2020-08-28 12:27:59 +03:00
alesapin
77faf9587f Better interface 2020-08-28 12:07:20 +03:00
alesapin
2fc80189af Add default compression codec to merge tree data part 2020-08-26 18:29:46 +03:00
alesapin
4326c9c971 Fix delta 2020-08-26 12:16:32 +03:00
alesapin
249c4b4a94 Less strange code and one method 2020-08-26 11:59:02 +03:00
alesapin
21d78f8513 Better codec description representation 2020-08-26 11:45:13 +03:00
alesapin
e5bc5ea419 Fix compression codec 2020-08-25 22:30:52 +03:00
alesapin
9ca8c5e77a Rename DefaultCompression to simple Default 2020-08-25 18:53:06 +03:00
alesapin
7bf0b74293 Fix style 2020-08-25 18:22:21 +03:00
alesapin
1180bad3a7 Add ability to specify DefaultCompression codec which correspond to settings specified in config.xml 2020-08-25 18:02:32 +03:00
Artem Zuikov
becc186c91
Add support for extended precision integers and decimals (#13097) 2020-08-19 14:52:17 +03:00
Alexey Milovidov
edd89a8610 Fix half of typos 2020-08-08 03:47:03 +03:00
Alexey Milovidov
6f690b7c0d Normalize ya.make files, fix "Arcadia" build 2020-08-02 16:57:38 +03:00
Nikita Mikhaylov
4d49d2c671 another removes 2020-07-30 13:31:14 +03:00
Nikita Mikhaylov
1d6c2ba689 rewrite other test + remove useless includes 2020-07-29 20:37:57 +03:00
Anton Popov
fbec940e0f fix reading and check query for compact parts with different codecs 2020-07-11 02:33:36 +03:00
Anton Popov
24f627e52c fix reading from compact parts with different codecs 2020-07-10 18:57:10 +03:00
Alexey Milovidov
fcdcb3cb1e Remove useless code 2020-07-10 04:58:27 +03:00
Alexey Milovidov
380f748358 Fix issues 2020-07-09 08:08:41 +03:00
Alexey Milovidov
b48d375ec5 Fix bad code 2020-07-09 08:00:53 +03:00
Alexey Milovidov
c82711b013 Fix build 2020-07-09 07:56:38 +03:00
Alexey Milovidov
a01a61c179 Remove another chunk of unneeded code 2020-07-09 04:07:18 +03:00
Alexey Milovidov
3cc9fd5b8f Remove something obviously wrong 2020-07-09 04:02:20 +03:00
Alexey Milovidov
ea970fd57c Remove bad ugliness 2020-07-09 04:00:16 +03:00
Alexey Milovidov
45e706aa8c Fix bad code, once again 2020-07-09 03:48:51 +03:00
Alexey Milovidov
105a35a6ae Fix strange things 2020-07-09 03:47:16 +03:00
Alexey Milovidov
0c8e12affd Remove ridiculous code 2020-07-09 03:46:00 +03:00
Alexey Milovidov
9dc91e3ee4 Fix outstandingly wrong code 2020-07-09 03:35:14 +03:00
Alexey Milovidov
4b8c827f84 Fix wrong exception code in codecs Delta, DoubleDelta #12110 2020-07-09 03:19:02 +03:00