thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-25 09:02:00 +00:00
Code Issues Packages Projects Releases Wiki Activity
72,186 Commits 9,170 Branches 1,781 Tags 2.1 GiB
c7e5cd7cda
Commit Graph

17477 Commits

Author SHA1 Message Date
alesapin
3b9dae8718
Merge pull request #28616 from ClickHouse/qoega-try-llvm-12 
Merging #20495
 2021-09-08 11:43:46 +03:00
Nikolai Kochetov
0a1c3d701f
Merge pull request #28717 from DimasKovas/fix_arcadia_build_2021_09_08 
Regenerate ya.make
 2021-09-08 11:13:32 +03:00
Vladimir C
70fbc85768
Merge pull request #28686 from ljcui/distributed_coredump 
Fix coredump in creating distributed table
 2021-09-08 10:52:56 +03:00
Dmitrii Kovalkov
5e133a3cc6 Run generate-ya-make.sh 2021-09-08 07:58:22 +03:00
Nikita Mikhaylov
6062dd0021 Better 2021-09-08 00:21:21 +00:00
Nikita Mikhaylov
7c9b1a7db4
Merge pull request #28679 from ianton-ru/MDB-14168 
Reduce default settings for S3 multipart upload part size
 2021-09-08 01:14:18 +03:00
Maksim Kita
1966ffc040
Merge pull request #28685 from orloffv/patch-1 
Add Settings.Names, Settings.Values aliases for system.processes table
 2021-09-08 01:01:56 +03:00
Azat Khuzhin
fe3c53040a Cache thread name to avoid prctl(PR_GET_NAME) for query_log/text_log 2021-09-08 00:57:14 +03:00
Azat Khuzhin
db0767a194 Implement detach_not_byte_identical_parts 
Maybe useful for further analysis of non byte-identical parts.
 2021-09-07 23:29:57 +03:00
Vitaly Baranov
59148fa453
Merge pull request #22415 from Enmk/governance/session_log 
Governance/session log
 2021-09-07 22:09:32 +03:00
mergify[bot]
771ae92c8e
Merge branch 'master' into global-merge-executor 2021-09-07 14:52:33 +00:00
Nikita Mikhaylov
231f64cf37 Fix unit test 2021-09-07 13:51:30 +00:00
Nikita Mikhaylov
9b38248e64 Remove scheduler thread 2021-09-07 13:13:23 +00:00
alesapin
d9ca1e29c3 Increase stack size for coroutines 2021-09-07 15:56:32 +03:00
Nikita Mikhaylov
8a0765c60d Review fixes (partially) 2021-09-07 12:45:39 +00:00
Nikita Mikhaylov
ff59105f28
Merge pull request #28663 from bharatnc/ncb/fix-rect-args 
fix getNumberOfArguments() for s2Rect functions
 2021-09-07 14:30:45 +03:00
Vitaly
69604eab3f
Add Settings.Names, Settings.Values aliases for system.processes table 2021-09-07 14:05:55 +03:00
ZhiYong Wang
978dd19fa2 Fix coredump in creating distributed table 2021-09-07 19:05:26 +08:00
mergify[bot]
e9fbdec13a
Merge branch 'master' into improvement/merge-tree-part-opt-parse 2021-09-07 09:52:24 +00:00
Anton Ivashkin
5ff99314b1 Reduce default settings for S3 multipart upload part size 2021-09-07 12:51:00 +03:00
Maksim Kita
7a4a0b0ede
Merge pull request #28474 from azat/mysql-connection_no_block 
Introduce connection_wait_timeout for MySQL engine.
 2021-09-07 12:28:02 +03:00
Dmitry Novik
99dc685146
Merge pull request #28315 from amosbird/indexfix 
Fix NOT-IN index optimization when not all key columns are used.
 2021-09-07 11:58:30 +03:00
Vitaly Baranov
70c6623036
Merge branch 'master' into governance/session_log 2021-09-07 10:12:54 +03:00
Kseniia Sumarokova
8e8b15edd7
Merge pull request #28665 from kssenii/revert 
Revert  #28082
 2021-09-07 09:43:02 +03:00
Vitaly Baranov
bcc31f1f3e Remove unnecessary changes. 2021-09-07 01:37:28 +03:00
Nikita Mikhaylov
cfe2207be3 Better 2021-09-06 22:07:41 +00:00
bharatnc
503b7a59f0 fix getNumberOfArguments() for s2Rect functions 
This fixes the value returned by the getNumberOfArguments() by the
s2RectAdd and the s2RectContains functions. Only 3 arguments are
used by these functions and not 4:

- low s2Point of rectangle
- high s2Point of rectangle
- given s2Point

The given s2Point is used to groow the size of the bounding rectangle to
include the given S2Point in case of the s2RectAdd function. In case
of s2RectContains, the function determines if the bounded rectangle
contains the given point.

PS: I wonder if it's more apt to call rectAdd as rectGrow since
it's used to grow the size of a given rectangle.
 2021-09-06 14:19:18 -07:00
Kseniia Sumarokova
9e496910a0
Merge pull request #28614 from kssenii/materialized-postgresql-2 
MaterializedPostgreSQL: add solution to survive postgres primary failover
 2021-09-06 22:31:15 +03:00
kssenii
7bea8200d8 Revert "Merge pull request #28082 from zhongyuankai/add-system-of-table_views" 
This reverts commit e5bcfba89e, reversing
changes made to d77f243651.
 2021-09-06 19:20:33 +00:00
Nikita Mikhaylov
74c0b61e53 Better 2021-09-06 18:33:31 +00:00
Nikita Mikhaylov
324ab97f50 Fix gcc build 2021-09-06 16:44:46 +00:00
Nikita Mikhaylov
29a18d8d10 Try to fix unbundled 2021-09-06 15:58:08 +00:00
Nikita Mikhaylov
7c39f84a5e Merge upstream/master into global-merge-executor (using imerge) 2021-09-06 15:37:18 +00:00
Nikita Mikhaylov
17d65b3d7b Allow allocations 2021-09-06 14:04:37 +00:00
Anton Popov
47f46e388d fix optimization of disjunctions chain 2021-09-06 16:54:26 +03:00
alesapin
2a851546c7
Merge pull request #28648 from ClickHouse/add_missed_level_to_text_log 
Add missed log level into TextLog
 2021-09-06 16:16:25 +03:00
Nikita Mikhaylov
ea0fbf81af Renaming 2021-09-06 12:01:16 +00:00
Nikita Mikhaylov
0249015515 added priority queue 2021-09-06 11:37:51 +00:00
alesapin
9db10a7164 Fix typo: 2021-09-06 14:34:17 +03:00
alesapin
fcfe77cb57 Fix allocation in buffer 2021-09-06 13:35:37 +03:00
alesapin
209b748fca Add missed level 2021-09-06 13:11:19 +03:00
Nikolai Kochetov
a8cad572df
Merge pull request #28561 from FArthur-cmd/remove_recursion_in_zstd 
remove recursion in ZstdInflatingReadBuffer
 2021-09-06 13:03:52 +03:00
Nikolai Kochetov
2178ed21df
Merge pull request #28560 from amosbird/projection-fix15 
Fix wrong header of minmax_count projection
 2021-09-06 13:00:44 +03:00
mergify[bot]
8eba5f29a1
Merge branch 'master' into improvement/merge-tree-part-opt-parse 2021-09-06 09:53:46 +00:00
tavplubix
4780008805
Merge pull request #28533 from azat/MaterializeMySQL-drop-fix 
Fix UUID overlap in DROP TABLE for internal DDL from MaterializeMySQL
 2021-09-06 12:34:13 +03:00
alesapin
5c75b93fe8 Revert one warning 2021-09-06 12:17:31 +03:00
alesapin
d1e91a7860 Revert "Revert "Fix tidy"" 
This reverts commit 2e5e017d6d.
 2021-09-06 12:16:52 +03:00
Dmitrii Kovalkov
181bf0dbd6 Fix arcadia build 2021-09-06 11:11:45 +03:00
alesapin
2e5e017d6d Revert "Fix tidy" 
This reverts commit 73ef1233ef.
 2021-09-06 10:52:20 +03:00
alesapin
7b2d4a3ace
Merge pull request #28559 from ClickHouse/add_test_logs_level 
Test log level for CI
 2021-09-06 10:49:59 +03:00
alesapin
d6244bcd7b Merge branch 'master' into qoega-try-llvm-12 2021-09-06 10:25:26 +03:00
alesapin
73ef1233ef Fix tidy 
Fix tidy one more time
 2021-09-06 10:22:21 +03:00
kssenii
7b8101f289 Minor change 2021-09-06 09:57:05 +03:00
Amos Bird
18a7adf0fa
Fix NOT-IN index optimization when not all keys are used. 2021-09-06 12:21:03 +08:00
Azat Khuzhin
e7ed98fa58 Do not allow ThreadFromGlobalPool::join() from the spawned/occupated thread 
v2: wrap with shared_ptr
    https://clickhouse-test-reports.s3.yandex.net/28431/f7b3a3e952038923b1347b3d461061326d0ffc27/functional_stateless_tests_(address).html#fail1
 2021-09-05 21:59:49 +03:00
Maksim Kita
f8e491d648
Merge pull request #28629 from ucasfl/fix 
fix minor typo
 2021-09-05 12:38:14 +03:00
Maksim Kita
337875199a
Merge pull request #28615 from kitaisreal/function-dict-get-small-fix 
Function dictGet small fix
 2021-09-05 12:37:03 +03:00
feng lv
3c166df8ea fix minor typo 2021-09-05 07:25:52 +00:00
Maksim Kita
685d061c4e Fixes tests 2021-09-04 22:13:20 +03:00
Maksim Kita
311110e36b
Merge pull request #28518 from kitaisreal/executable-pool-storage 
Added ExecutablePool storage
 2021-09-04 21:09:07 +03:00
Nikita Mikhaylov
25a879044b Allow allocations to allocate a thread 2021-09-04 17:23:05 +00:00
alesapin
36a11af351 Merge branch 'master' into add_test_logs_level 2021-09-04 15:48:47 +03:00
alesapin
327bd44f23 Fix gtest 2021-09-04 15:46:41 +03:00
alesapin
aa58f22207
Merge pull request #28519 from ClickHouse/better_session_expiration 
Simplier sessions expiration in Keeper and add comments
 2021-09-04 15:44:55 +03:00
Maksim Kita
bcbc830b05 Function dictGet small fix 2021-09-04 15:28:11 +03:00
kssenii
ac2d9a73a8 User managed slots 2021-09-04 14:49:40 +03:00
Maksim Kita
69fa28f332 Fixed tests 2021-09-04 14:29:40 +03:00
Maksim Kita
f076cc69b4
Merge pull request #28530 from kitaisreal/dict-get-default-implementation-for-nulls 
Function dictGet default implementation for nulls
 2021-09-04 13:29:22 +03:00
Kseniia Sumarokova
04b26d26bf
Merge pull request #28428 from kssenii/fix-database-ordinary-test 
Fix 01457_create_as_table_function_structure
 2021-09-04 11:07:42 +03:00
Azat Khuzhin
bf6ba796f8 Fix UUID overlap in DROP TABLE for internal DDL from MaterializeMySQL 
This will fix race with DatabaseCatalog::loadMarkedAsDroppedTables(),
since MaterializeMySQL, and MaterializedMySQLSyncThread in background,
will be started earlier then
DatabaseCatalog::loadMarkedAsDroppedTables() and will move those tables
to metadata_dropped, and after loadMarkedAsDroppedTables() will start
and try to load partially dropped tables and will hit UUID overlap:

    12:02:51.536783 [ 3026034 ] {} <Information> Application: starting up
    12:02:53.019282 [ 3026034 ] {} <Information> DatabaseMaterializeMySQL<Atomic> (mysql): Total 9 tables and 0 dictionaries.
    12:02:53.041699 [ 3026200 ] {} <Debug> mysql.data (7143b65f-6982-4600-b143-b65f6982e600): Loading data parts
    12:02:53.041740 [ 3026200 ] {} <Debug> mysql.data (7143b65f-6982-4600-b143-b65f6982e600): There are no data parts
    12:02:53.620382 [ 3026034 ] {} <Information> DatabaseMaterializeMySQL<Atomic> (mysql): Starting up tables.
    12:03:00.669730 [ 3026183 ] {} <Debug> executeQuery: (internal) /*Materialize MySQL step 1: execute MySQL DDL for dump data*/ DROP TABLE mysql.data
    12:03:00.741894 [ 3026269 ] {} <Information> DatabaseCatalog: Trying load partially dropped table mysql.data (7143b65f-6982-4600-b143-b65f6982e600) from /var/lib/clickhouse/metadata_dropped/mysql.data.7143b65f-6982-4600-b143-b65f6982e600.sql
    12:03:00.742582 [ 3026269 ] {} <Debug> mysql.data (7143b65f-6982-4600-b143-b65f6982e600): Loading data parts
    12:03:00.742650 [ 3026269 ] {} <Debug> mysql.data (7143b65f-6982-4600-b143-b65f6982e600): There are no data parts
    12:03:00.773137 [ 3026034 ] {} <Error> Application: Caught exception while loading metadata: Code: 57, e.displayText() = DB::Exception: Mapping for table with UUID=7143b65f-6982-4600-b143-b65f6982e600 already exists. It happened due to UUID collision, most likely because some not random UUIDs were manually specified in CREATE queries., Stack trace (when copying this message, always include the lines below):
    12:03:01.224557 [ 3026034 ] {} <Error> Application: DB::Exception: Mapping for table with UUID=7143b65f-6982-4600-b143-b65f6982e600 already exists. It happened due to UUID collision, most likely because some not random UUIDs were manually specified in CREATE queries.

Cc: @zhang2014
 2021-09-04 10:41:28 +03:00
Nikita Mikhaylov
3eec8a3c2f Use boost::circular_buffer 2021-09-03 22:15:20 +00:00
Nikita Mikhaylov
b85f98d2f4
Merge pull request #28579 from Algunenano/csv_spelling 
Improve [C|T]SV errors
 2021-09-04 01:10:56 +03:00
Maksim Kita
c329f04b22 Fixed tests 2021-09-03 23:08:55 +03:00
Maksim Kita
838b823f22
Merge pull request #28236 from kitaisreal/jit-custom-memory-manager 
CHJIT custom memory manager
 2021-09-03 21:46:18 +03:00
Alexander Tokmakov
13466a7cc3 minor fix 2021-09-03 20:06:38 +03:00
Kruglov Pavel
f559c34113
Merge pull request #28302 from amosbird/binaryconstantwrap 
Always monotonic for non-zero division
 2021-09-03 20:02:54 +03:00
Kseniia Sumarokova
2486b6d53c
Update StorageTableFunction.h 2021-09-03 19:52:43 +03:00
Kseniia Sumarokova
67ffb59b72
Merge pull request #28531 from CurtizJ/fix-nested 
Fix usage of nested columns with non-array columns with the same prefix
 2021-09-03 18:06:27 +03:00
alesapin
65ff5fd052 Rewrite test 2021-09-03 17:57:29 +03:00
Kseniia Sumarokova
5c8e634584
Update StorageTableFunction.h 2021-09-03 17:46:26 +03:00
alesapin
e1f2fe8c5d
Merge branch 'master' into better_session_expiration 2021-09-03 17:38:38 +03:00
Nikita Mikhaylov
26ab6ebc0a finalize in destructor 2021-09-03 14:20:34 +00:00
Raúl Marín
09a9ad0dfe Improve [C|T]SV errors 2021-09-03 16:15:56 +02:00
Kseniia Sumarokova
8c5acc371a
Merge pull request #28365 from kssenii/rabbitmq-ssl 
Allow ssl connection for rabbitmq
 2021-09-03 16:53:28 +03:00
Nikita Mikhaylov
b867eabe73 Use Poco::Event 2021-09-03 13:02:49 +00:00
alesapin
dd7493d871
Merge pull request #27038 from excitoon-favorites/altermaterializecolumn 
`ALTER TABLE ... MATERIALIZE COLUMN`
 2021-09-03 15:50:52 +03:00
Nikita Mikhaylov
218f0adca7 Merge upstream/master into global-merge-executor (using imerge) 2021-09-03 12:29:48 +00:00
Nikita Mikhaylov
ca7adb76b9 Owning future 2021-09-03 12:27:49 +00:00
Nikita Mikhaylov
7929ee4d9b
Merge pull request #28534 from nikitamikhaylov/fix-race-zlib 
Fix race in zlib
 2021-09-03 15:00:23 +03:00
Mike Kot
1f148719ce Review fixes 2021-09-03 13:33:40 +02:00
Artur
04e5e52dbd prevent endless loop 2021-09-03 11:18:19 +00:00
Artur
9c480c0de6 remove recursion in ZstdInflatingReadBuffer 2021-09-03 10:56:18 +00:00
Amos Bird
bb87d01579
Fix wrong header of minmax_count projection 2021-09-03 18:49:01 +08:00
alesapin
497c225203 Test log level for CI 2021-09-03 13:07:40 +03:00
Maksim Kita
af7220b6aa Updated ShellCommandSource 2021-09-03 13:00:40 +03:00
alesapin
10f4d6be62
Merge pull request #28360 from ClickHouse/debug_keeper_lost_logs 
Fix keeper logs disappearance
 2021-09-03 12:38:16 +03:00
Maksim Kita
7071ef80d0 Fixed typos 2021-09-03 12:06:16 +03:00
Maksim Kita
c03bcc7c0f Fixed tests 2021-09-03 11:56:51 +03:00
Nikita Mikhaylov
a3f008c626 Enable define from cmake 2021-09-03 08:30:03 +00:00
Azat Khuzhin
4909d3f2e0 Fix endless loop for truncated bzip2 
Stress test (thread) found an issue [1]:

    elapsed:              2094.356407533
    query:                SELECT count(), max(x) FROM file
    thread_ids:           [2927,11494,7502]
    current_database:     test_19

Stacktrace for 7502:

    0  __syscall () at ../base/glibc-compatibility/musl/x86_64/syscall.s:14
    1  0x000000001ba033b8 in clock_nanosleep (clk=1, flags=1, req=0x7fbb8adaed68, rem=0x0) at ../base/glibc-compatibility/musl/clock_nanosleep.c:21
    2  0x0000000018e25b84 in sleepForNanoseconds (nanoseconds=<optimized out>) at ../base/common/sleep.cpp:48
    3  0x0000000009a47143 in DB::injection (yield_probability=0, migrate_probability=0, sleep_probability=0.10000000000000001, sleep_time_us=100000) at ../src/Common/ThreadFuzzer.cpp:193
    4  DB::ThreadFuzzer::signalHandler () at ../src/Common/ThreadFuzzer.cpp:204
    5  0x00000000098e1447 in __tsan::CallUserSignalHandler(__tsan::ThreadState*, bool, bool, bool, int, __sanitizer::__sanitizer_siginfo*, void*) ()
    6  0x00000000098d7f99 in __tsan::ProcessPendingSignals(__tsan::ThreadState*) ()
    7  0x00000000098e57de in pread ()
    8  0x00000000099fab84 in DB::ReadBufferFromFileDescriptor::nextImpl (this=0x7b38010ddb00) at ../src/IO/ReadBufferFromFileDescriptor.cpp:62
    9  0x0000000009ab0cf3 in DB::ReadBuffer::next (this=0x7b38010ddb00) at ../src/IO/ReadBuffer.h:62
    10 DB::ReadBuffer::nextIfAtEnd (this=0x7b38010ddb00) at ../src/IO/ReadBuffer.h:78
    11 DB::Bzip2ReadBuffer::nextImpl (this=0x7b20032f4800) at ../src/IO/Bzip2ReadBuffer.cpp:59

It looks like bz2 is not capable to read/fail partially written archive:

    2021.09.03 00:52:36.744818 [ 3553 ] {90419c0a-c410-42ea-ba76-7672ff5ae5ec} <Debug> executeQuery: (from [::1]:54714) (comment: '/usr/share/clickhouse-test/queries/0_stateless/01059_storage_file_compression.sh') CREATE TABLE file (x UInt64) ENGINE = File(TSV, 'test_19/bz2.tsv.bz2')
    2021.09.03 00:52:41.255409 [ 2927 ] {9c7b4e21-cc37-438d-9c13-f9dc01efd30c} <Debug> executeQuery: (from [::1]:54750) (comment: '/usr/share/clickhouse-test/queries/0_stateless/01059_storage_file_compression.sh') TRUNCATE TABLE file
    2021.09.03 00:52:43.762791 [ 2927 ] {82857e3d-f7cc-453d-82a4-e8b8fb82ae0e} <Debug> executeQuery: (from [::1]:54758) (comment: '/usr/share/clickhouse-test/queries/0_stateless/01059_storage_file_compression.sh') INSERT INTO file SELECT * FROM numbers(1000000)
    2021.09.03 00:52:54.939333 [ 2927 ] {82857e3d-f7cc-453d-82a4-e8b8fb82ae0e} <Error> executeQuery: Code: 241. DB::Exception: Memory limit (total) exceeded: would use 63.07 GiB (attempt to allocate chunk of 4194364 bytes), maximum: 62.94 GiB. (MEMORY_LIMIT_EXCEEDED) (version 21.10.1.7982) (from [::1]:54758) (comment: '/usr/share/clickhouse-test/queries/0_stateless/01059_storage_file_compression.sh') (in query: INSERT INTO file SELECT * FROM numbers(1000000)), Stack trace (when copying this message, always include the lines below):
    2021.09.03 00:53:08.505015 [ 2927 ] {2bc7444e-c0d6-462e-9d6a-567f70b01697} <Debug> executeQuery: (from [::1]:54842) (comment: '/usr/share/clickhouse-test/queries/0_stateless/01059_storage_file_compression.sh') SELECT count(), max(x) FROM file

  [1]: https://clickhouse-test-reports.s3.yandex.net/28474/6d5f01a56bb1715c47de8444bfc85b39228f3081/stress_test_(thread).html#fail1
 2021-09-03 10:49:17 +03:00