thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-09-29 13:10:48 +00:00
Code Issues Packages Projects Releases Wiki Activity
72,246 Commits 8,958 Branches 1,767 Tags 1.8 GiB
b467df98d3
Commit Graph

72246 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Nikita Mikhaylov
26ab6ebc0a finalize in destructor 2021-09-03 14:20:34 +00:00
Raúl Marín
09a9ad0dfe Improve [C|T]SV errors 2021-09-03 16:15:56 +02:00
Kseniia Sumarokova
8c5acc371a
Merge pull request #28365 from kssenii/rabbitmq-ssl 
Allow ssl connection for rabbitmq
 2021-09-03 16:53:28 +03:00
Nikita Mikhaylov
b867eabe73 Use Poco::Event 2021-09-03 13:02:49 +00:00
alesapin
dd7493d871
Merge pull request #27038 from excitoon-favorites/altermaterializecolumn 
`ALTER TABLE ... MATERIALIZE COLUMN`
 2021-09-03 15:50:52 +03:00
Nikita Mikhaylov
218f0adca7 Merge upstream/master into global-merge-executor (using imerge) 2021-09-03 12:29:48 +00:00
Nikita Mikhaylov
ca7adb76b9 Owning future 2021-09-03 12:27:49 +00:00
Nikita Mikhaylov
7929ee4d9b
Merge pull request #28534 from nikitamikhaylov/fix-race-zlib 
Fix race in zlib
 2021-09-03 15:00:23 +03:00
alesapin
9345b3c45e Fixup flaky 2021-09-03 14:59:21 +03:00
alesapin
7ddbadeeb3
Merge pull request #28542 from ClickHouse/fix_kafka_test 
Fix broken kafka test
 2021-09-03 14:46:14 +03:00
Mike Kot
1f148719ce Review fixes 2021-09-03 13:33:40 +02:00
alesapin
8b8b0043c1 Fix logger conf 2021-09-03 14:20:09 +03:00
Artur
04e5e52dbd prevent endless loop 2021-09-03 11:18:19 +00:00
alesapin
4fe3909d74
Update Dockerfile 2021-09-03 14:01:47 +03:00
alesapin
69e7e1dff7
Update Dockerfile 2021-09-03 14:00:57 +03:00
Artur
9c480c0de6 remove recursion in ZstdInflatingReadBuffer 2021-09-03 10:56:18 +00:00
Amos Bird
bb87d01579
Fix wrong header of minmax_count projection 2021-09-03 18:49:01 +08:00
Maksim Kita
3445ae3f2c
Merge pull request #28556 from benbiti/fix-keep_alive_timeout-default-value_docs 
fix keep_alive_timeout default value (10s) in docs
 2021-09-03 13:12:43 +03:00
alesapin
497c225203 Test log level for CI 2021-09-03 13:07:40 +03:00
Kruglov Pavel
8f5b3f51b7
Merge pull request #28557 from Avogar/fix-format-names 
Fix format names in docs
 2021-09-03 13:02:34 +03:00
Maksim Kita
af7220b6aa Updated ShellCommandSource 2021-09-03 13:00:40 +03:00
Pavel Kruglov
1d9ad322ae Fix format names in docs 2021-09-03 12:43:56 +03:00
alesapin
10f4d6be62
Merge pull request #28360 from ClickHouse/debug_keeper_lost_logs 
Fix keeper logs disappearance
 2021-09-03 12:38:16 +03:00
benbiti
fa5db1be76 fix keep_alive_timeout default value in docs 2021-09-03 17:32:31 +08:00
Maksim Kita
7071ef80d0 Fixed typos 2021-09-03 12:06:16 +03:00
Maksim Kita
c03bcc7c0f Fixed tests 2021-09-03 11:56:51 +03:00
alesapin
a182e621f0 Maybe it help 2021-09-03 11:53:26 +03:00
Maksim Kita
25a242fa68
Merge pull request #28543 from azat/bzip2-fix 
Fix endless loop for truncated bzip2 archive
 2021-09-03 11:41:42 +03:00
Maksim Kita
ba2b8213d2
Merge pull request #28538 from benbiti/master 
fix max_server_memory_usage_to_ram_ratio default value(0.9) in docs
 2021-09-03 11:41:17 +03:00
Nikita Mikhaylov
a3f008c626 Enable define from cmake 2021-09-03 08:30:03 +00:00
alesapin
36abf9f2f4 Fix broken kafka test 2021-09-03 10:51:37 +03:00
Azat Khuzhin
4909d3f2e0 Fix endless loop for truncated bzip2 
Stress test (thread) found an issue [1]:

    elapsed:              2094.356407533
    query:                SELECT count(), max(x) FROM file
    thread_ids:           [2927,11494,7502]
    current_database:     test_19

Stacktrace for 7502:

    0  __syscall () at ../base/glibc-compatibility/musl/x86_64/syscall.s:14
    1  0x000000001ba033b8 in clock_nanosleep (clk=1, flags=1, req=0x7fbb8adaed68, rem=0x0) at ../base/glibc-compatibility/musl/clock_nanosleep.c:21
    2  0x0000000018e25b84 in sleepForNanoseconds (nanoseconds=<optimized out>) at ../base/common/sleep.cpp:48
    3  0x0000000009a47143 in DB::injection (yield_probability=0, migrate_probability=0, sleep_probability=0.10000000000000001, sleep_time_us=100000) at ../src/Common/ThreadFuzzer.cpp:193
    4  DB::ThreadFuzzer::signalHandler () at ../src/Common/ThreadFuzzer.cpp:204
    5  0x00000000098e1447 in __tsan::CallUserSignalHandler(__tsan::ThreadState*, bool, bool, bool, int, __sanitizer::__sanitizer_siginfo*, void*) ()
    6  0x00000000098d7f99 in __tsan::ProcessPendingSignals(__tsan::ThreadState*) ()
    7  0x00000000098e57de in pread ()
    8  0x00000000099fab84 in DB::ReadBufferFromFileDescriptor::nextImpl (this=0x7b38010ddb00) at ../src/IO/ReadBufferFromFileDescriptor.cpp:62
    9  0x0000000009ab0cf3 in DB::ReadBuffer::next (this=0x7b38010ddb00) at ../src/IO/ReadBuffer.h:62
    10 DB::ReadBuffer::nextIfAtEnd (this=0x7b38010ddb00) at ../src/IO/ReadBuffer.h:78
    11 DB::Bzip2ReadBuffer::nextImpl (this=0x7b20032f4800) at ../src/IO/Bzip2ReadBuffer.cpp:59

It looks like bz2 is not capable to read/fail partially written archive:

    2021.09.03 00:52:36.744818 [ 3553 ] {90419c0a-c410-42ea-ba76-7672ff5ae5ec} <Debug> executeQuery: (from [::1]:54714) (comment: '/usr/share/clickhouse-test/queries/0_stateless/01059_storage_file_compression.sh') CREATE TABLE file (x UInt64) ENGINE = File(TSV, 'test_19/bz2.tsv.bz2')
    2021.09.03 00:52:41.255409 [ 2927 ] {9c7b4e21-cc37-438d-9c13-f9dc01efd30c} <Debug> executeQuery: (from [::1]:54750) (comment: '/usr/share/clickhouse-test/queries/0_stateless/01059_storage_file_compression.sh') TRUNCATE TABLE file
    2021.09.03 00:52:43.762791 [ 2927 ] {82857e3d-f7cc-453d-82a4-e8b8fb82ae0e} <Debug> executeQuery: (from [::1]:54758) (comment: '/usr/share/clickhouse-test/queries/0_stateless/01059_storage_file_compression.sh') INSERT INTO file SELECT * FROM numbers(1000000)
    2021.09.03 00:52:54.939333 [ 2927 ] {82857e3d-f7cc-453d-82a4-e8b8fb82ae0e} <Error> executeQuery: Code: 241. DB::Exception: Memory limit (total) exceeded: would use 63.07 GiB (attempt to allocate chunk of 4194364 bytes), maximum: 62.94 GiB. (MEMORY_LIMIT_EXCEEDED) (version 21.10.1.7982) (from [::1]:54758) (comment: '/usr/share/clickhouse-test/queries/0_stateless/01059_storage_file_compression.sh') (in query: INSERT INTO file SELECT * FROM numbers(1000000)), Stack trace (when copying this message, always include the lines below):
    2021.09.03 00:53:08.505015 [ 2927 ] {2bc7444e-c0d6-462e-9d6a-567f70b01697} <Debug> executeQuery: (from [::1]:54842) (comment: '/usr/share/clickhouse-test/queries/0_stateless/01059_storage_file_compression.sh') SELECT count(), max(x) FROM file

  [1]: https://clickhouse-test-reports.s3.yandex.net/28474/6d5f01a56bb1715c47de8444bfc85b39228f3081/stress_test_(thread).html#fail1
 2021-09-03 10:49:17 +03:00
Azat Khuzhin
5a4d554c15 FIx Bzip2WriteBuffer ifdef (USE_BZIP2 over USE_BROTLI) 2021-09-03 10:48:52 +03:00
alesapin
1894524438
Merge pull request #28528 from ClickHouse/fix_detach_attach_versioned_collapsing 
Fix detach/attach for ReplicatedVersionedCollapsingMergeTree after alter
 2021-09-03 10:13:52 +03:00
alesapin
9281c4786b Fix queue test 2021-09-03 10:10:19 +03:00
benbiti
f8e367757d fix max_server_memory_usage_to_ram_ratio default value(0.9) in docs 2021-09-03 14:24:24 +08:00
Azat Khuzhin
5bab06ed90 Bump poco to remove getpid() calls 
Refs: https://github.com/ClickHouse-Extras/poco/pull/46
 2021-09-03 07:45:58 +03:00
Nikita Mikhaylov
d777c6cd70 Better [2] 2021-09-03 00:51:12 +00:00
Nikita Mikhaylov
60765b8c6e Better 2021-09-03 00:45:19 +00:00
Nikita Mikhaylov
292a24abe8 Merge upstream/master into global-merge-executor (using imerge) 2021-09-03 00:34:24 +00:00
Nikita Mikhaylov
9257188512 Comments 2021-09-03 00:28:29 +00:00
Nikita Mikhaylov
7baad6adde Fix build 2021-09-02 22:56:03 +00:00
Nikita Mikhaylov
f8020a1f07 Fix build 2021-09-02 22:47:51 +00:00
Nikita Mikhaylov
0f284db1fc Done 2021-09-02 22:33:04 +00:00
Nikita Mikhaylov
cc7c221fad Own PriorityQueue + prettifying the code 2021-09-02 21:31:32 +00:00
alesapin
d571c84a56 Better test for flaky check 2021-09-02 23:41:00 +03:00
alesapin
a949329830 Review fixes 2021-09-02 23:37:34 +03:00
alesapin
3a3c3acd18
Update src/Coordination/SessionExpiryQueue.h 
Co-authored-by: tavplubix <avtokmakov@yandex-team.ru>
 2021-09-02 23:19:44 +03:00
Azat Khuzhin
6d5f01a56b Cover MySQL setting connection_wait_timeout 2021-09-02 22:39:07 +03:00
Azat Khuzhin
ed70ed6f71 Introduce connection_no_wait setting for MySQL engine. 
This will allow to avoid superfluous sleep during query execution, since
this not only not desired behavoiur, but also may hang the server, since
if you will execute enough queries that will use MySQL database but will
not allow enough connections (or your MySQL server is too slow) then you
may run out of threads in the global thread pool.

Also note that right now it is possible to get deadlock when the mysql
pool is full, consider the following scenario:

- you have m1 and m2 mysql tables
- you have q1 and q2 queries, bot queries join m1 and m2
- q1 allocated connection for m1 but cannot allocate connection for m2
- q2 allocated connection for m2 but cannot allocate connection for m1
- but to resolve the lock one should give up on the locking while it is not possible right now...

And then you got no free threads and this:

    # grep -h ^202 /proc/$(pgrep clickhouse-serv)/task/*/syscall  | cut -d' ' -f2 | sort | uniq -c | sort -nr | head
       1554 0x7ffb60b92fe8 # mutex in mysqlxx::PoolWithFailover::get
       1375 0x7ffb9f1c4748 # mutex in ::PoolEntryHelper::~PoolEntryHelper from DB::MultiplexedConnections::invalidateReplica
       1160 0x7ffb612918b8 # mutex in mysqlxx::PoolWithFailover::get
         42 0x7ffb9f057984 # mutex in ThreadPoolImpl<std::__1::thread>::worker

    *NOTE: 202 is a `futex` with WAIT*

(Went with `syscall` because debugging 10k+ threads is not easy, and
eventually it may TRAP)
 2021-09-02 22:23:37 +03:00