thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-17 13:13:36 +00:00
Code Issues Packages Projects Releases Wiki Activity
77,455 Commits 9,138 Branches 1,774 Tags 2.1 GiB
cb6f99fe6f
Commit Graph

1123 Commits

Author SHA1 Message Date
kssenii
f3f820c6e0 HTTP read buffer retries 2021-10-08 15:41:55 +03:00
kssenii
5e63ec4489 Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-07 19:11:45 +03:00
kssenii
d6e1d0503d Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-10-07 16:21:00 +03:00
Nikita Mikhaylov
b31d11478b
Merge pull request #29680 from azat/defines 
Cleanup common defines
 2021-10-04 20:13:29 +03:00
alexey-milovidov
6abb3d4c6d
Merge pull request #29682 from traceon/xcode-13-fix-build 
Fix Xcode 13 build
 2021-10-04 02:34:43 +03:00
Denis Glazachev
ca7b69b0f3 Manipulate with -Wreserved-identifier only if HAS_RESERVED_IDENTIFIER has been detected 2021-10-03 17:42:36 +04:00
Azat Khuzhin
16c72f2107 Remove unused macros from ReadWriteBufferFromHTTP 2021-10-03 14:34:03 +03:00
Azat Khuzhin
6a9dd9828d Move protocol macros into separate header 
Defines.h is a very common header, so lots of modules will be recompiled
on changes.
Move macros for protocol into separate header, this should significantly
decreases number of units to compile on it's changes.
 2021-10-03 14:34:03 +03:00
Alexey Milovidov
9bc32c95c0 Miscellaneous 2021-10-03 10:01:09 +03:00
Alexey Milovidov
0bf597374f More enhancements for query obfuscator 2021-10-03 05:56:32 +03:00
kssenii
1146035f11 Small fix 2021-10-02 17:28:12 +03:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
kssenii
7ff092712b Better 2021-10-01 18:13:16 +03:00
kssenii
55972db866 Small fix and more comments 2021-10-01 11:38:58 +03:00
kssenii
e4bc81ac78 Better 2021-09-30 21:45:57 +03:00
kssenii
cc9ce13bac Finally all fixed 2021-09-30 15:35:59 +03:00
kssenii
85aae03a26 Fix 2021-09-29 22:54:49 +03:00
kssenii
20192352ad Better 2021-09-29 11:03:36 +03:00
MaxWk
723ff795a8 support obs storage 2021-09-29 14:11:28 +08:00
kssenii
ebca30694d Fix 2021-09-27 18:25:30 +03:00
Alexey Milovidov
cd7f9d981c Remove ya.make 2021-09-25 04:22:54 +03:00
alesapin
3614fb9e37
Merge pull request #29223 from ClickHouse/compressed_logs 
Compressed logs for keeper
 2021-09-24 16:03:26 +03:00
kssenii
6219d541a5 Finally fixed 2021-09-24 15:29:04 +03:00
kssenii
c1ea44b9c2 Fixes 2021-09-24 13:38:08 +03:00
Vladimir Chebotarev
0343106692
Fixed logging level for message in S3Common.cpp. 2021-09-23 21:47:31 +03:00
alesapin
eb3a651bab Fix bug found on review 2021-09-23 16:55:50 +03:00
mergify[bot]
d69fc35dd9
Merge branch 'master' into compressed_logs 2021-09-22 19:43:16 +00:00
kssenii
1b9565dfa8 Merge branch 'master' of github.com:ClickHouse/ClickHouse into disk-async-read 2021-09-22 22:08:44 +03:00
Kseniia Sumarokova
192c5397cc
Merge pull request #29211 from kssenii/better-retries-2 
Fix retries for disk web
 2021-09-22 12:37:25 +03:00
alesapin
043ef03d42 revert method name 2021-09-22 10:30:15 +03:00
alesapin
334a7ab8db Change default extension to zst 2021-09-21 23:43:44 +03:00
alesapin
ac7579f187 Compressed logs for keeper 2021-09-21 17:29:05 +03:00
kssenii
e301457e91 Fix prefetch mistake 2021-09-21 15:40:48 +03:00
kssenii
975033e6e3 Fix 2021-09-21 09:44:54 +00:00
kssenii
40ee75fe97 Common thread pool for disks 2021-09-21 10:00:56 +03:00
kssenii
cacc81ca61 Fix 2021-09-21 02:31:02 +03:00
kssenii
0104cb1c8e Better 2021-09-21 01:51:17 +03:00
kssenii
f312760dca Better 2021-09-20 19:57:15 +03:00
kssenii
11ae94b8d3 Async read for remote fs 2021-09-20 16:16:46 +03:00
Azat Khuzhin
e5f66fd103 Add socket timeout values into SOCKET_TIMEOUT exception 2021-09-20 09:51:42 +03:00
Maksim Kita
14e4d49601
Merge pull request #29063 from ClickHouse/read-integer-overflow-bug-fix 
Merging #27963
 2021-09-17 16:06:58 +03:00
Maksim Kita
73368aa351 Fixed tests 2021-09-17 12:17:19 +03:00
Maksim Kita
49f04700cc Fixed style 2021-09-16 00:29:29 +03:00
Maksim Kita
7a72a765c2 Added tests 2021-09-16 00:01:01 +03:00
Maksim Kita
7c748b6140
Merge pull request #27963 from zhanglistar/overflow_bugfix 
Overflow bugfix
 2021-09-15 23:31:43 +03:00
Anton Popov
fc17936c12 Merge remote-tracking branch 'upstream/master' into HEAD 2021-09-14 23:02:30 +03:00
Nikita Mikhaylov
58f2504a56 Update ya.make 2021-09-14 12:00:32 +00:00
Sergei Semin
12e24a23d3
Merge branch 'master' into fix-build-clang-13-2 2021-09-12 19:30:55 +03:00
alexey-milovidov
ea13a8b562
Merge pull request #28659 from myrrc/improvement/tostring_to_magic_enum 
Improving CH type system with concepts
 2021-09-12 15:26:29 +03:00
Sergei Semin
f74350c148 fix special build 2021-09-12 02:49:31 +03:00
Mike Kot
3401b32e37 Adapted usage of is_decimal concept 2021-09-11 19:35:12 +02:00
Mike Kot
bce011cb33 Replacing IsDecimalNumber 2021-09-11 00:40:18 +02:00
Mike Kot
c4385a2c40 Simplifying exception messages 2021-09-11 00:40:18 +02:00
mergify[bot]
e3b3439535
Merge branch 'master' into fix-build-clang-13-2 2021-09-10 12:05:54 +00:00
Anton Popov
5cff615eca Merge remote-tracking branch 'upstream/master' into HEAD 2021-09-10 13:40:48 +03:00
alexey-milovidov
2236a5df98
Merge pull request #28085 from myrrc/improvement/merge-tree-part-opt-parse 
optional<> semantics for parsing MergeTreePartInfo and DetachedPartInfo
 2021-09-10 00:04:03 +03:00
Sergei Semin
f92e720d18 fix gtest_bit_io.cpp 2021-09-09 23:51:47 +03:00
Sergei Semin
39929e52bd add ignore of -Wreserved-identifier into LineReader.cpp, ThreadFuzzer.cpp, ReadBufferFromFileDescriptor.cpp 2021-09-09 23:51:47 +03:00
Anton Popov
8e5b3b2f6c Merge remote-tracking branch 'upstream/master' into HEAD 2021-09-09 15:12:34 +03:00
alesapin
3b9dae8718
Merge pull request #28616 from ClickHouse/qoega-try-llvm-12 
Merging #20495
 2021-09-08 11:43:46 +03:00
mergify[bot]
e9fbdec13a
Merge branch 'master' into improvement/merge-tree-part-opt-parse 2021-09-07 09:52:24 +00:00
Mike Kot
8e9aacadd1 Initial: replacing hardcoded toString for enums with magic_enum 2021-09-06 16:24:03 +02:00
alesapin
9db10a7164 Fix typo: 2021-09-06 14:34:17 +03:00
alesapin
fcfe77cb57 Fix allocation in buffer 2021-09-06 13:35:37 +03:00
Nikolai Kochetov
a8cad572df
Merge pull request #28561 from FArthur-cmd/remove_recursion_in_zstd 
remove recursion in ZstdInflatingReadBuffer
 2021-09-06 13:03:52 +03:00
mergify[bot]
8eba5f29a1
Merge branch 'master' into improvement/merge-tree-part-opt-parse 2021-09-06 09:53:46 +00:00
alesapin
d1e91a7860 Revert "Revert "Fix tidy"" 
This reverts commit 2e5e017d6d.
 2021-09-06 12:16:52 +03:00
Dmitrii Kovalkov
181bf0dbd6 Fix arcadia build 2021-09-06 11:11:45 +03:00
alesapin
2e5e017d6d Revert "Fix tidy" 
This reverts commit 73ef1233ef.
 2021-09-06 10:52:20 +03:00
alesapin
73ef1233ef Fix tidy 
Fix tidy one more time
 2021-09-06 10:22:21 +03:00
Anton Popov
59128412ea Merge remote-tracking branch 'upstream/master' into HEAD 2021-09-03 22:17:11 +03:00
Anton Popov
c9a0cc5ce9 fix race on table schema in insert-select 2021-09-03 17:23:21 +03:00
Nikita Mikhaylov
7929ee4d9b
Merge pull request #28534 from nikitamikhaylov/fix-race-zlib 
Fix race in zlib
 2021-09-03 15:00:23 +03:00
Artur
04e5e52dbd prevent endless loop 2021-09-03 11:18:19 +00:00
Artur
9c480c0de6 remove recursion in ZstdInflatingReadBuffer 2021-09-03 10:56:18 +00:00
Nikita Mikhaylov
a3f008c626 Enable define from cmake 2021-09-03 08:30:03 +00:00
Azat Khuzhin
4909d3f2e0 Fix endless loop for truncated bzip2 
Stress test (thread) found an issue [1]:

    elapsed:              2094.356407533
    query:                SELECT count(), max(x) FROM file
    thread_ids:           [2927,11494,7502]
    current_database:     test_19

Stacktrace for 7502:

    0  __syscall () at ../base/glibc-compatibility/musl/x86_64/syscall.s:14
    1  0x000000001ba033b8 in clock_nanosleep (clk=1, flags=1, req=0x7fbb8adaed68, rem=0x0) at ../base/glibc-compatibility/musl/clock_nanosleep.c:21
    2  0x0000000018e25b84 in sleepForNanoseconds (nanoseconds=<optimized out>) at ../base/common/sleep.cpp:48
    3  0x0000000009a47143 in DB::injection (yield_probability=0, migrate_probability=0, sleep_probability=0.10000000000000001, sleep_time_us=100000) at ../src/Common/ThreadFuzzer.cpp:193
    4  DB::ThreadFuzzer::signalHandler () at ../src/Common/ThreadFuzzer.cpp:204
    5  0x00000000098e1447 in __tsan::CallUserSignalHandler(__tsan::ThreadState*, bool, bool, bool, int, __sanitizer::__sanitizer_siginfo*, void*) ()
    6  0x00000000098d7f99 in __tsan::ProcessPendingSignals(__tsan::ThreadState*) ()
    7  0x00000000098e57de in pread ()
    8  0x00000000099fab84 in DB::ReadBufferFromFileDescriptor::nextImpl (this=0x7b38010ddb00) at ../src/IO/ReadBufferFromFileDescriptor.cpp:62
    9  0x0000000009ab0cf3 in DB::ReadBuffer::next (this=0x7b38010ddb00) at ../src/IO/ReadBuffer.h:62
    10 DB::ReadBuffer::nextIfAtEnd (this=0x7b38010ddb00) at ../src/IO/ReadBuffer.h:78
    11 DB::Bzip2ReadBuffer::nextImpl (this=0x7b20032f4800) at ../src/IO/Bzip2ReadBuffer.cpp:59

It looks like bz2 is not capable to read/fail partially written archive:

    2021.09.03 00:52:36.744818 [ 3553 ] {90419c0a-c410-42ea-ba76-7672ff5ae5ec} <Debug> executeQuery: (from [::1]:54714) (comment: '/usr/share/clickhouse-test/queries/0_stateless/01059_storage_file_compression.sh') CREATE TABLE file (x UInt64) ENGINE = File(TSV, 'test_19/bz2.tsv.bz2')
    2021.09.03 00:52:41.255409 [ 2927 ] {9c7b4e21-cc37-438d-9c13-f9dc01efd30c} <Debug> executeQuery: (from [::1]:54750) (comment: '/usr/share/clickhouse-test/queries/0_stateless/01059_storage_file_compression.sh') TRUNCATE TABLE file
    2021.09.03 00:52:43.762791 [ 2927 ] {82857e3d-f7cc-453d-82a4-e8b8fb82ae0e} <Debug> executeQuery: (from [::1]:54758) (comment: '/usr/share/clickhouse-test/queries/0_stateless/01059_storage_file_compression.sh') INSERT INTO file SELECT * FROM numbers(1000000)
    2021.09.03 00:52:54.939333 [ 2927 ] {82857e3d-f7cc-453d-82a4-e8b8fb82ae0e} <Error> executeQuery: Code: 241. DB::Exception: Memory limit (total) exceeded: would use 63.07 GiB (attempt to allocate chunk of 4194364 bytes), maximum: 62.94 GiB. (MEMORY_LIMIT_EXCEEDED) (version 21.10.1.7982) (from [::1]:54758) (comment: '/usr/share/clickhouse-test/queries/0_stateless/01059_storage_file_compression.sh') (in query: INSERT INTO file SELECT * FROM numbers(1000000)), Stack trace (when copying this message, always include the lines below):
    2021.09.03 00:53:08.505015 [ 2927 ] {2bc7444e-c0d6-462e-9d6a-567f70b01697} <Debug> executeQuery: (from [::1]:54842) (comment: '/usr/share/clickhouse-test/queries/0_stateless/01059_storage_file_compression.sh') SELECT count(), max(x) FROM file

  [1]: https://clickhouse-test-reports.s3.yandex.net/28474/6d5f01a56bb1715c47de8444bfc85b39228f3081/stress_test_(thread).html#fail1
 2021-09-03 10:49:17 +03:00
Azat Khuzhin
5a4d554c15 FIx Bzip2WriteBuffer ifdef (USE_BZIP2 over USE_BROTLI) 2021-09-03 10:48:52 +03:00
Nikita Mikhaylov
0f284db1fc Done 2021-09-02 22:33:04 +00:00
Anton Popov
5e421ab272 Merge remote-tracking branch 'upstream/master' into HEAD 2021-09-02 02:42:50 +03:00
Nikolay Degterinsky
53b6d918eb
Merge pull request #25310 from bharatnc/ncb/import-export-lz4 
LZ4 compression for import/export
 2021-09-01 21:12:50 +03:00
Alexey Milovidov
19445ac096 Drop unused headers 2021-09-01 02:47:52 +03:00
Mike Kot
4194ed2ec7 Merge remote-tracking branch 'upstream/master' into improvement/merge-tree-part-opt-parse 2021-08-31 14:35:14 +02:00
Nikita Mikhaylov
8abee2480e src/ 2021-08-31 12:14:08 +00:00
Nikolay Degterinsky
ed4d5edd9a Style fix 2021-08-31 13:53:44 +03:00
Nikita Mikhaylov
00394014ff
Merge pull request #28382 from DimasKovas/fix_arcadia_build_31_08_2021 
Try to fix arcadia build (generate ya.make)
 2021-08-31 13:49:32 +03:00
Nikolay Degterinsky
e8713a7d7e Merge branch 'master' into ncb/import-export-lz4 2021-08-31 13:32:39 +03:00
Dmitrii Kovalkov
b1eb48804b Run generate-ya-make.sh 2021-08-31 09:05:33 +03:00
Alexey Milovidov
ca6f6306f3 Merge branch 'master' into async-reads 2021-08-31 02:52:58 +03:00
Kseniia Sumarokova
00bea71610
Merge pull request #25251 from kssenii/disk-over-web-server 
ClickHouse over web server with static files
 2021-08-30 16:18:59 +03:00
Dmitrii Kovalkov
9871ad70ff Exclude fuzzers 2021-08-30 11:12:25 +03:00
Alexey Milovidov
93ebb40130 Fix warnings 2021-08-28 01:59:34 +03:00
Alexey Milovidov
71ed9ad060 Renaming according to the review suggestions 2021-08-28 01:32:35 +03:00
Alexey Milovidov
b1478368cd Fix error 2021-08-27 04:45:14 +03:00
Alexey Milovidov
6f1789e9da Add missing file 2021-08-27 03:47:34 +03:00
Alexey Milovidov
864d01e564 Add pread_fake_async method 2021-08-27 03:35:04 +03:00
Alexey Milovidov
0316f1cc3e Even more instrumentation 2021-08-27 03:08:10 +03:00
Alexey Milovidov
f5ae8ef225 Instrumentation 2021-08-27 03:02:14 +03:00
Alexey Milovidov
91cef77edc Instrumentation 2021-08-27 03:01:07 +03:00
Alexey Milovidov
84a9be0b14 Move code to .cpp 2021-08-27 02:32:11 +03:00
Alexey Milovidov
46732e781f Merge branch 'master' of github.com:yandex/ClickHouse into async-reads 2021-08-27 01:57:48 +03:00
Alexey Milovidov
8fd643e22a Fix build 2021-08-27 01:57:34 +03:00
Filatenkov Artur
eeeef3cc18
Update ZlibInflatingReadBuffer.cpp 2021-08-26 15:43:01 +03:00
Filatenkov Artur
a1240fd575
Update ZlibInflatingReadBuffer.cpp 2021-08-26 15:42:10 +03:00
Filatenkov Artur
1d97ae3f6a
Update ZlibInflatingReadBuffer.cpp 2021-08-26 12:15:10 +03:00
Filatenkov Artur
a7d982198c
Update src/IO/ZlibInflatingReadBuffer.cpp 
Co-authored-by: Ilya Yatsishin <2159081+qoega@users.noreply.github.com>
 2021-08-26 11:33:24 +03:00
Filatenkov Artur
f8eb02cb64
Update src/IO/ZlibInflatingReadBuffer.cpp 
Co-authored-by: Ilya Yatsishin <2159081+qoega@users.noreply.github.com>
 2021-08-26 11:33:10 +03:00
Anton Popov
6d3274c22c Merge remote-tracking branch 'upstream/master' into HEAD 2021-08-26 03:31:46 +03:00
mergify[bot]
69d2a6080b
Merge branch 'master' into fix_compression_26149 2021-08-25 19:01:08 +00:00
Artur
13c2de5fa8 add comments 2021-08-25 18:23:17 +00:00
Artur
239b198a12 fix eof 2021-08-25 16:56:19 +00:00
Mike Kot
407af64280 Merge remote-tracking branch 'upstream/master' into improvement/merge-tree-part-opt-parse 2021-08-25 18:24:43 +02:00
Nikolay Degterinsky
99a26b3a9a Small fix 2021-08-25 15:35:53 +03:00
Nikolay Degterinsky
f722132f23 Merge branch 'master' into ncb/import-export-lz4 2021-08-25 14:59:17 +03:00
Alexey Milovidov
c03dc92dd1 Fix excessive logging in ReadBufferFromS3 2021-08-25 03:28:15 +03:00
Alexey Milovidov
5243315227 Improve performance in case without prefetch by avoiding swapping of memory buffers 2021-08-25 03:13:05 +03:00
Alexey Milovidov
384b4304c0 Minor change 2021-08-25 02:56:14 +03:00
Alexey Milovidov
f7ca674b0c Progress on development 2021-08-25 01:07:06 +03:00
Alexey Milovidov
8f57216180 Progress on development 2021-08-25 00:45:58 +03:00
Alexey Milovidov
4d9ad3725d Merge branch 'master' into async-reads 2021-08-25 00:45:40 +03:00
Alexey Milovidov
a3ecc2b705 Remove unused include 2021-08-24 23:47:13 +03:00
Mike Kot
f85e2e027c Optional semantics for [Detached]MergeTreePartInfo 2021-08-24 19:43:09 +02:00
alexey-milovidov
739caf86d5
Merge pull request #27484 from excitoon/patch-10 
Don't silently ignore errors and don't count delays in `ReadBufferFromS3`
 2021-08-24 15:26:39 +03:00
Nikolay Degterinsky
d60f128a9a Merge branch 'master' into ncb/import-export-lz4 2021-08-24 11:47:10 +03:00
kssenii
b536653851 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into disk-over-web-server 2021-08-24 07:29:50 +00:00
Vladimir C
98acccbdcc
Merge branch 'master' into s3partitionedwrite 2021-08-24 10:03:57 +03:00
kssenii
5307d31924 Throw for alter and silence for drop 2021-08-23 23:48:53 +00:00
Vitaly Baranov
4c9d9aadd4 Add owning ConcatReadBuffer. 2021-08-23 23:41:45 +03:00
listar
a1b13b08b1 1. fix format 2021-08-23 08:28:52 +00:00
vdimir
6fe63a80bc Fix validateKey/Bucket for S3 2021-08-23 09:22:15 +03:00
vdimir
295c8d5686 Remove unused Error Code from IO/S3Common.cpp 2021-08-23 09:22:15 +03:00
vdimir
6ba6577978 Do not allow slashes in bucket formatted from PARTITION BY 2021-08-23 09:22:15 +03:00
vdimir
c4c42b61c4 Validate uft8 in partition key from PARTITION BY for s3 2021-08-23 09:22:15 +03:00
Vladimir Chebotarev
925c496b6c Rolled back changes in Functions. 2021-08-23 09:22:15 +03:00
Vladimir Chebotarev
3c2e090b6b Build and style fix. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
a2bbf98253 Update. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
0479edd47e Improved validation of S3 buckets and keys. 2021-08-23 09:22:14 +03:00
Vladimir Chebotarev
21d4d26f20 Final fix ReadBufferFromS3. 2021-08-23 05:05:49 +00:00
listar
b4a33ce001 1. fix oveflow bug 2021-08-21 19:08:34 +00:00
bharatnc
5de6d19da8 Merge remote-tracking branch 'upstream/master' into ncb/import-export-lz4 2021-08-20 17:19:36 -07:00
kssenii
945e2c4ce5 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into disk-over-web-server 2021-08-20 18:01:23 +00:00
Nikolay Degterinsky
f2852d1a08 Fix algorithm 2021-08-20 19:40:18 +03:00
Nikolay Degterinsky
ecb985c4b5 Refactor WriteBuffer 2021-08-19 13:34:23 +03:00
Nikolay Degterinsky
24046d6ecc Refactor ReadBuffer 2021-08-19 13:29:58 +03:00
Nikolay Degterinsky
7a5c5b4208 Fix encoding/decoding 2021-08-19 11:10:56 +03:00
Vitaly Baranov
fabd7193bd Code cleanups and improvements. 2021-08-18 14:24:52 +03:00
Nikolay Degterinsky
c245af5243 Merge branch 'master' into ncb/import-export-lz4 2021-08-18 10:20:23 +03:00
kssenii
77c350cedc Catch and reformat error with invalid url 2021-08-16 14:30:06 +00:00
Alexey Milovidov
8adaef7c8e Make text format for Decimal tuneable 2021-08-16 11:03:23 +03:00
Alexey Milovidov
bdab932f97 Fix some tests 2021-08-16 08:56:42 +03:00