thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-17 03:42:48 +00:00
Code Issues Packages Projects Releases Wiki Activity
72,247 Commits 9,315 Branches 1,783 Tags 2.1 GiB
10a314dd46
Commit Graph

17521 Commits

Author SHA1 Message Date
Maksim Kita
f076cc69b4
Merge pull request #28530 from kitaisreal/dict-get-default-implementation-for-nulls 
Function dictGet default implementation for nulls
 2021-09-04 13:29:22 +03:00
Kseniia Sumarokova
04b26d26bf
Merge pull request #28428 from kssenii/fix-database-ordinary-test 
Fix 01457_create_as_table_function_structure
 2021-09-04 11:07:42 +03:00
Azat Khuzhin
bf6ba796f8 Fix UUID overlap in DROP TABLE for internal DDL from MaterializeMySQL 
This will fix race with DatabaseCatalog::loadMarkedAsDroppedTables(),
since MaterializeMySQL, and MaterializedMySQLSyncThread in background,
will be started earlier then
DatabaseCatalog::loadMarkedAsDroppedTables() and will move those tables
to metadata_dropped, and after loadMarkedAsDroppedTables() will start
and try to load partially dropped tables and will hit UUID overlap:

    12:02:51.536783 [ 3026034 ] {} <Information> Application: starting up
    12:02:53.019282 [ 3026034 ] {} <Information> DatabaseMaterializeMySQL<Atomic> (mysql): Total 9 tables and 0 dictionaries.
    12:02:53.041699 [ 3026200 ] {} <Debug> mysql.data (7143b65f-6982-4600-b143-b65f6982e600): Loading data parts
    12:02:53.041740 [ 3026200 ] {} <Debug> mysql.data (7143b65f-6982-4600-b143-b65f6982e600): There are no data parts
    12:02:53.620382 [ 3026034 ] {} <Information> DatabaseMaterializeMySQL<Atomic> (mysql): Starting up tables.
    12:03:00.669730 [ 3026183 ] {} <Debug> executeQuery: (internal) /*Materialize MySQL step 1: execute MySQL DDL for dump data*/ DROP TABLE mysql.data
    12:03:00.741894 [ 3026269 ] {} <Information> DatabaseCatalog: Trying load partially dropped table mysql.data (7143b65f-6982-4600-b143-b65f6982e600) from /var/lib/clickhouse/metadata_dropped/mysql.data.7143b65f-6982-4600-b143-b65f6982e600.sql
    12:03:00.742582 [ 3026269 ] {} <Debug> mysql.data (7143b65f-6982-4600-b143-b65f6982e600): Loading data parts
    12:03:00.742650 [ 3026269 ] {} <Debug> mysql.data (7143b65f-6982-4600-b143-b65f6982e600): There are no data parts
    12:03:00.773137 [ 3026034 ] {} <Error> Application: Caught exception while loading metadata: Code: 57, e.displayText() = DB::Exception: Mapping for table with UUID=7143b65f-6982-4600-b143-b65f6982e600 already exists. It happened due to UUID collision, most likely because some not random UUIDs were manually specified in CREATE queries., Stack trace (when copying this message, always include the lines below):
    12:03:01.224557 [ 3026034 ] {} <Error> Application: DB::Exception: Mapping for table with UUID=7143b65f-6982-4600-b143-b65f6982e600 already exists. It happened due to UUID collision, most likely because some not random UUIDs were manually specified in CREATE queries.

Cc: @zhang2014
 2021-09-04 10:41:28 +03:00
Nikita Mikhaylov
3eec8a3c2f Use boost::circular_buffer 2021-09-03 22:15:20 +00:00
Nikita Mikhaylov
b85f98d2f4
Merge pull request #28579 from Algunenano/csv_spelling 
Improve [C|T]SV errors
 2021-09-04 01:10:56 +03:00
Maksim Kita
c329f04b22 Fixed tests 2021-09-03 23:08:55 +03:00
Maksim Kita
838b823f22
Merge pull request #28236 from kitaisreal/jit-custom-memory-manager 
CHJIT custom memory manager
 2021-09-03 21:46:18 +03:00
Alexander Tokmakov
13466a7cc3 minor fix 2021-09-03 20:06:38 +03:00
Kruglov Pavel
f559c34113
Merge pull request #28302 from amosbird/binaryconstantwrap 
Always monotonic for non-zero division
 2021-09-03 20:02:54 +03:00
Kseniia Sumarokova
2486b6d53c
Update StorageTableFunction.h 2021-09-03 19:52:43 +03:00
Kseniia Sumarokova
67ffb59b72
Merge pull request #28531 from CurtizJ/fix-nested 
Fix usage of nested columns with non-array columns with the same prefix
 2021-09-03 18:06:27 +03:00
alesapin
65ff5fd052 Rewrite test 2021-09-03 17:57:29 +03:00
Kseniia Sumarokova
5c8e634584
Update StorageTableFunction.h 2021-09-03 17:46:26 +03:00
alesapin
e1f2fe8c5d
Merge branch 'master' into better_session_expiration 2021-09-03 17:38:38 +03:00
Nikita Mikhaylov
26ab6ebc0a finalize in destructor 2021-09-03 14:20:34 +00:00
Raúl Marín
09a9ad0dfe Improve [C|T]SV errors 2021-09-03 16:15:56 +02:00
Kseniia Sumarokova
8c5acc371a
Merge pull request #28365 from kssenii/rabbitmq-ssl 
Allow ssl connection for rabbitmq
 2021-09-03 16:53:28 +03:00
Nikita Mikhaylov
b867eabe73 Use Poco::Event 2021-09-03 13:02:49 +00:00
alesapin
dd7493d871
Merge pull request #27038 from excitoon-favorites/altermaterializecolumn 
`ALTER TABLE ... MATERIALIZE COLUMN`
 2021-09-03 15:50:52 +03:00
Nikita Mikhaylov
218f0adca7 Merge upstream/master into global-merge-executor (using imerge) 2021-09-03 12:29:48 +00:00
Nikita Mikhaylov
ca7adb76b9 Owning future 2021-09-03 12:27:49 +00:00
Nikita Mikhaylov
7929ee4d9b
Merge pull request #28534 from nikitamikhaylov/fix-race-zlib 
Fix race in zlib
 2021-09-03 15:00:23 +03:00
Mike Kot
1f148719ce Review fixes 2021-09-03 13:33:40 +02:00
Artur
04e5e52dbd prevent endless loop 2021-09-03 11:18:19 +00:00
Artur
9c480c0de6 remove recursion in ZstdInflatingReadBuffer 2021-09-03 10:56:18 +00:00
Amos Bird
bb87d01579
Fix wrong header of minmax_count projection 2021-09-03 18:49:01 +08:00
alesapin
497c225203 Test log level for CI 2021-09-03 13:07:40 +03:00
Maksim Kita
af7220b6aa Updated ShellCommandSource 2021-09-03 13:00:40 +03:00
alesapin
10f4d6be62
Merge pull request #28360 from ClickHouse/debug_keeper_lost_logs 
Fix keeper logs disappearance
 2021-09-03 12:38:16 +03:00
Maksim Kita
7071ef80d0 Fixed typos 2021-09-03 12:06:16 +03:00
Maksim Kita
c03bcc7c0f Fixed tests 2021-09-03 11:56:51 +03:00
Nikita Mikhaylov
a3f008c626 Enable define from cmake 2021-09-03 08:30:03 +00:00
Azat Khuzhin
4909d3f2e0 Fix endless loop for truncated bzip2 
Stress test (thread) found an issue [1]:

    elapsed:              2094.356407533
    query:                SELECT count(), max(x) FROM file
    thread_ids:           [2927,11494,7502]
    current_database:     test_19

Stacktrace for 7502:

    0  __syscall () at ../base/glibc-compatibility/musl/x86_64/syscall.s:14
    1  0x000000001ba033b8 in clock_nanosleep (clk=1, flags=1, req=0x7fbb8adaed68, rem=0x0) at ../base/glibc-compatibility/musl/clock_nanosleep.c:21
    2  0x0000000018e25b84 in sleepForNanoseconds (nanoseconds=<optimized out>) at ../base/common/sleep.cpp:48
    3  0x0000000009a47143 in DB::injection (yield_probability=0, migrate_probability=0, sleep_probability=0.10000000000000001, sleep_time_us=100000) at ../src/Common/ThreadFuzzer.cpp:193
    4  DB::ThreadFuzzer::signalHandler () at ../src/Common/ThreadFuzzer.cpp:204
    5  0x00000000098e1447 in __tsan::CallUserSignalHandler(__tsan::ThreadState*, bool, bool, bool, int, __sanitizer::__sanitizer_siginfo*, void*) ()
    6  0x00000000098d7f99 in __tsan::ProcessPendingSignals(__tsan::ThreadState*) ()
    7  0x00000000098e57de in pread ()
    8  0x00000000099fab84 in DB::ReadBufferFromFileDescriptor::nextImpl (this=0x7b38010ddb00) at ../src/IO/ReadBufferFromFileDescriptor.cpp:62
    9  0x0000000009ab0cf3 in DB::ReadBuffer::next (this=0x7b38010ddb00) at ../src/IO/ReadBuffer.h:62
    10 DB::ReadBuffer::nextIfAtEnd (this=0x7b38010ddb00) at ../src/IO/ReadBuffer.h:78
    11 DB::Bzip2ReadBuffer::nextImpl (this=0x7b20032f4800) at ../src/IO/Bzip2ReadBuffer.cpp:59

It looks like bz2 is not capable to read/fail partially written archive:

    2021.09.03 00:52:36.744818 [ 3553 ] {90419c0a-c410-42ea-ba76-7672ff5ae5ec} <Debug> executeQuery: (from [::1]:54714) (comment: '/usr/share/clickhouse-test/queries/0_stateless/01059_storage_file_compression.sh') CREATE TABLE file (x UInt64) ENGINE = File(TSV, 'test_19/bz2.tsv.bz2')
    2021.09.03 00:52:41.255409 [ 2927 ] {9c7b4e21-cc37-438d-9c13-f9dc01efd30c} <Debug> executeQuery: (from [::1]:54750) (comment: '/usr/share/clickhouse-test/queries/0_stateless/01059_storage_file_compression.sh') TRUNCATE TABLE file
    2021.09.03 00:52:43.762791 [ 2927 ] {82857e3d-f7cc-453d-82a4-e8b8fb82ae0e} <Debug> executeQuery: (from [::1]:54758) (comment: '/usr/share/clickhouse-test/queries/0_stateless/01059_storage_file_compression.sh') INSERT INTO file SELECT * FROM numbers(1000000)
    2021.09.03 00:52:54.939333 [ 2927 ] {82857e3d-f7cc-453d-82a4-e8b8fb82ae0e} <Error> executeQuery: Code: 241. DB::Exception: Memory limit (total) exceeded: would use 63.07 GiB (attempt to allocate chunk of 4194364 bytes), maximum: 62.94 GiB. (MEMORY_LIMIT_EXCEEDED) (version 21.10.1.7982) (from [::1]:54758) (comment: '/usr/share/clickhouse-test/queries/0_stateless/01059_storage_file_compression.sh') (in query: INSERT INTO file SELECT * FROM numbers(1000000)), Stack trace (when copying this message, always include the lines below):
    2021.09.03 00:53:08.505015 [ 2927 ] {2bc7444e-c0d6-462e-9d6a-567f70b01697} <Debug> executeQuery: (from [::1]:54842) (comment: '/usr/share/clickhouse-test/queries/0_stateless/01059_storage_file_compression.sh') SELECT count(), max(x) FROM file

  [1]: https://clickhouse-test-reports.s3.yandex.net/28474/6d5f01a56bb1715c47de8444bfc85b39228f3081/stress_test_(thread).html#fail1
 2021-09-03 10:49:17 +03:00
Azat Khuzhin
5a4d554c15 FIx Bzip2WriteBuffer ifdef (USE_BZIP2 over USE_BROTLI) 2021-09-03 10:48:52 +03:00
alesapin
1894524438
Merge pull request #28528 from ClickHouse/fix_detach_attach_versioned_collapsing 
Fix detach/attach for ReplicatedVersionedCollapsingMergeTree after alter
 2021-09-03 10:13:52 +03:00
alesapin
9281c4786b Fix queue test 2021-09-03 10:10:19 +03:00
Nikita Mikhaylov
d777c6cd70 Better [2] 2021-09-03 00:51:12 +00:00
Nikita Mikhaylov
60765b8c6e Better 2021-09-03 00:45:19 +00:00
Nikita Mikhaylov
292a24abe8 Merge upstream/master into global-merge-executor (using imerge) 2021-09-03 00:34:24 +00:00
Nikita Mikhaylov
9257188512 Comments 2021-09-03 00:28:29 +00:00
Nikita Mikhaylov
7baad6adde Fix build 2021-09-02 22:56:03 +00:00
Nikita Mikhaylov
f8020a1f07 Fix build 2021-09-02 22:47:51 +00:00
Nikita Mikhaylov
0f284db1fc Done 2021-09-02 22:33:04 +00:00
Nikita Mikhaylov
cc7c221fad Own PriorityQueue + prettifying the code 2021-09-02 21:31:32 +00:00
alesapin
a949329830 Review fixes 2021-09-02 23:37:34 +03:00
alesapin
3a3c3acd18
Update src/Coordination/SessionExpiryQueue.h 
Co-authored-by: tavplubix <avtokmakov@yandex-team.ru>
 2021-09-02 23:19:44 +03:00
Azat Khuzhin
ed70ed6f71 Introduce connection_no_wait setting for MySQL engine. 
This will allow to avoid superfluous sleep during query execution, since
this not only not desired behavoiur, but also may hang the server, since
if you will execute enough queries that will use MySQL database but will
not allow enough connections (or your MySQL server is too slow) then you
may run out of threads in the global thread pool.

Also note that right now it is possible to get deadlock when the mysql
pool is full, consider the following scenario:

- you have m1 and m2 mysql tables
- you have q1 and q2 queries, bot queries join m1 and m2
- q1 allocated connection for m1 but cannot allocate connection for m2
- q2 allocated connection for m2 but cannot allocate connection for m1
- but to resolve the lock one should give up on the locking while it is not possible right now...

And then you got no free threads and this:

    # grep -h ^202 /proc/$(pgrep clickhouse-serv)/task/*/syscall  | cut -d' ' -f2 | sort | uniq -c | sort -nr | head
       1554 0x7ffb60b92fe8 # mutex in mysqlxx::PoolWithFailover::get
       1375 0x7ffb9f1c4748 # mutex in ::PoolEntryHelper::~PoolEntryHelper from DB::MultiplexedConnections::invalidateReplica
       1160 0x7ffb612918b8 # mutex in mysqlxx::PoolWithFailover::get
         42 0x7ffb9f057984 # mutex in ThreadPoolImpl<std::__1::thread>::worker

    *NOTE: 202 is a `futex` with WAIT*

(Went with `syscall` because debugging 10k+ threads is not easy, and
eventually it may TRAP)
 2021-09-02 22:23:37 +03:00
Nikita Mikhaylov
6624fa12ba deleted set 2021-09-02 18:49:37 +00:00
Anton Popov
db77349957 add unit test for NestedUtils 2021-09-02 20:43:42 +03:00
Nikita Mikhaylov
a6fe91ca47 get rid of half of allocations 2021-09-02 17:40:29 +00:00
Anton Popov
ee7f50396b fix nested and scalar columns with dot 2021-09-02 20:22:01 +03:00
alesapin
9bad77f806 Fix clang tidy 2021-09-02 19:16:29 +03:00
Maksim Kita
8fbef6c69d Function dictGet default implementation for nulls 2021-09-02 18:30:55 +03:00
alesapin
464b8167d1 Update comment 2021-09-02 18:29:26 +03:00
alesapin
ba78a737dc Fix detach/attach for ReplicatedVersionedCollapsingMergeTree after alter 2021-09-02 17:47:00 +03:00
Nikolai Kochetov
af0da33147
Merge pull request #27827 from ClickHouse/more-checks-for-lv-and-native 
Add more checks for LC in native protocol.
 2021-09-02 17:27:42 +03:00
Maksim Kita
f9ca726bf7 PageArena updated implementation 2021-09-02 16:31:50 +03:00
alexey-milovidov
ab3e005e6a
Merge pull request #28411 from kssenii/fix-web-disk 
Fix disk with static files a little
 2021-09-02 16:23:35 +03:00
alesapin
bc818dea4b Merge branch 'master' into debug_keeper_lost_logs 2021-09-02 15:07:08 +03:00
alesapin
010985ce87 Add comment 2021-09-02 14:54:32 +03:00
Maksim Kita
abda2a636e Added ExecutablePool storage 2021-09-02 14:53:20 +03:00
alesapin
c5470864e8 Fixup 2021-09-02 14:43:34 +03:00
alesapin
4c613f30b3 Simplier sessions expiration in Keeper 2021-09-02 14:40:54 +03:00
Maksim Kita
0a12592b50 UserDefinedFunctionFactory added comments 2021-09-02 14:37:06 +03:00
Amos Bird
163a416366
Refactor 2021-09-02 19:29:14 +08:00
alesapin
b592400e54
Merge pull request #28404 from ClickHouse/remove_outdated_settings 
Remove obsolete settings for replicated fetches.
 2021-09-02 14:03:12 +03:00
alesapin
9b1b3ec916 Fix style check 2021-09-02 14:02:37 +03:00
Nikita Mikhaylov
7f21cd7f3d save 2021-09-02 10:39:27 +00:00
Nikolai Kochetov
5f0b1bc4e3
Merge pull request #26286 from amosbird/projection-improve2 
Virtual projection for min max indices.
 2021-09-02 13:03:30 +03:00
alesapin
d681ffabf8 Comment 2021-09-02 12:22:51 +03:00
alesapin
a8003e444b Review fixes 2021-09-02 12:20:46 +03:00
Maksim Kita
befb82e441
Merge pull request #28433 from kssenii/fix-materialized-postgresql 
Fix cannot use non-ordinary table names in materialized postgresql
 2021-09-02 09:49:33 +03:00
Maksim Kita
be4c9102d8
Merge pull request #28438 from azat/udf-race-fix-v2 
Fix race in UDF (follow up)
 2021-09-01 23:57:05 +03:00
tavplubix
b5b9fd8663
Merge pull request #28446 from ClickHouse/ddlworker_fix_watch_leak 
Fix watch leak in DDLWorker
 2021-09-01 22:44:39 +03:00
Nikolay Degterinsky
53b6d918eb
Merge pull request #25310 from bharatnc/ncb/import-export-lz4 
LZ4 compression for import/export
 2021-09-01 21:12:50 +03:00
Kseniia Sumarokova
a67741b626
Merge pull request #28430 from kssenii/fix-startup 
Do not allow create postgres storage with bad arguments
 2021-09-01 17:29:14 +03:00
mergify[bot]
9536da9319
Merge branch 'master' into debug_keeper_lost_logs 2021-09-01 14:07:53 +00:00
mergify[bot]
5d299fbdee
Merge branch 'master' into remove_outdated_settings 2021-09-01 14:07:48 +00:00
Nikolai Kochetov
cdafa6fb5d
Update Fiber.h 2021-09-01 16:46:23 +03:00
Nikita Mikhaylov
ceab6feb2a Fix test 2021-09-01 12:35:37 +00:00
vdimir
bf9cb8482b
Revert "Add option to getMostSubtype/getLeastSupertype to not throw exception" 
This reverts commit c4654d6b45.
 2021-09-01 15:08:59 +03:00
vdimir
6172cc1676
Fix build, inferJoinKeyCommonType definition 2021-09-01 15:08:58 +03:00
vdimir
ac17c0c817
Minor style src/Interpreters/TableJoin.cpp 2021-09-01 15:08:58 +03:00
vdimir
2a1c7d7336
fix typo in tryInitDictJoin 2021-09-01 15:05:48 +03:00
vdimir
450e49fd28
Throw type mismatch exception from inferJoinKeyCommonType 2021-09-01 15:05:48 +03:00
vdimir
df873866c9
Don't store converting actions in TableJoin 2021-09-01 15:05:48 +03:00
vdimir
ed8c156190
Clean up TableJoin storage join 2021-09-01 15:05:47 +03:00
vdimir
13bf141e1d
Fix storage_join type inference 2021-09-01 15:05:47 +03:00
vdimir
c4ffc2879f
Perform join type inference for USING at ExpressionAnalyzer instead of TreeRewriter 2021-09-01 15:05:47 +03:00
vdimir
33ebdb283c
Split ExpressionAnalyzer::analyzeAggregation 2021-09-01 15:05:47 +03:00
mergify[bot]
143d5a169c
Merge branch 'master' into udf-race-fix-v2 2021-09-01 10:42:08 +00:00
Kseniia Sumarokova
09c9dd489b
Update StorageRabbitMQ.cpp 2021-09-01 13:28:34 +03:00
Nikolai Kochetov
6bd9b82141
Update Fiber.h 2021-09-01 12:43:38 +03:00
Alexander Tokmakov
f8ea865283 fix watch leak in DDLWorker 2021-09-01 12:18:28 +03:00
tavplubix
e55c0bc40a
Merging #27980 (#28413) 
* fix: empty string convert to DecimalOrNumbeOrDateOrDateTime

* fix: ifnull convert to defaut value

* fix variable name

* fix cast args

* modify ifnull args

* use short circuit

Co-authored-by: liyang830 <liyang830@jd.com>
 2021-09-01 12:12:23 +03:00
alesapin
d5c93200f3
Merge pull request #28412 from ClickHouse/more-accurate-exists-check-for-zk-init 
More accurate check that zk root exists.
 2021-09-01 11:16:42 +03:00
alesapin
fd1581aee1 Fix style 2021-09-01 10:52:33 +03:00
alesapin
319f03d66f Fix PVS 2021-09-01 10:51:08 +03:00
Kseniia Sumarokova
bf9abc2888
Update StorageTableFunction.h 2021-09-01 10:46:41 +03:00
Azat Khuzhin
99fe4308b5 Fix race in UDF (follow up) 2021-09-01 10:38:46 +03:00
alesapin
3e834035b8 Merge branch 'master' into debug_keeper_lost_logs 2021-09-01 10:36:42 +03:00
Alexey Milovidov
19445ac096 Drop unused headers 2021-09-01 02:47:52 +03:00
Nikita Mikhaylov
c7727f093d Merge upstream/master into global-merge-executor (using imerge) 2021-08-31 23:34:11 +00:00
Nikita Mikhaylov
ac5316ce98 Stress test + reworker executor 2021-08-31 23:20:23 +00:00
alexey-milovidov
77d085f264
Merge pull request #27527 from evillique/log_queries_probability 
Add log_queries_probability setting
 2021-09-01 00:51:23 +03:00
alexey-milovidov
4cc0b0298c
Merge pull request #28269 from amosbird/fixweirdcode 
Better nullable primary key implementation
 2021-09-01 00:48:45 +03:00
Kseniia Sumarokova
437c87fff3
Update StorageTableFunction.h 2021-09-01 00:06:10 +03:00
kssenii
445b6b34fc Fix 2021-08-31 20:58:00 +00:00
kssenii
f9592cae11 Fix 2021-08-31 22:24:58 +03:00
Kseniia Sumarokova
63a0eb3922
Update DiskWebServer.cpp 2021-08-31 21:51:00 +03:00
Nikita Mikhaylov
f471ed4506
Merge pull request #28037 from nikitamikhaylov/settings-s3-engine 
Support `SETTINGS` clause for `S3` engine
 2021-08-31 21:08:53 +03:00
Nikita Mikhaylov
28eff0e369 save 2021-08-31 18:07:24 +00:00
kssenii
185d29439b Fix 2021-08-31 20:51:42 +03:00
alesapin
2c7a51d816
Merge pull request #28366 from azat/remove-Temporary-part-fix-v2 
Fix removing of parts in a Temporary state (v2 followup)
 2021-08-31 19:13:15 +03:00
Nikita Mikhaylov
dbc950caa4 added a test 2021-08-31 14:54:24 +00:00
Nikolai Kochetov
487e36720c
Update src/Common/ZooKeeper/ZooKeeper.cpp 
Co-authored-by: alesapin <alesapin@gmail.com>
 2021-08-31 17:50:24 +03:00
Nikolai Kochetov
43d5ada6bd More accurate check that zk root exists. 2021-08-31 17:37:36 +03:00
Vladimir C
94d5f3a87b
Merge pull request #28349 from vdimir/issue-27691-qf 
Fix non joined rows from nullable column
 2021-08-31 17:09:24 +03:00
Vladimir C
3768bde572
Remove extra space in src/Interpreters/join_common.cpp 2021-08-31 17:08:35 +03:00
kssenii
248fe1fd09 Fix disk 2021-08-31 16:54:43 +03:00
Nikita Mikhaylov
cf50b247cf update ya.make 2021-08-31 12:55:23 +00:00
Nikita Mikhaylov
e41d1cff1d Fix test 2021-08-31 12:54:45 +00:00
Nikita Mikhaylov
fcfda4b983 Fix tests and update ya.make 2021-08-31 12:54:45 +00:00
Nikita Mikhaylov
e5593f594d After rebase 2021-08-31 12:54:06 +00:00
Nikita Mikhaylov
d7a627e5a9 Added test to skip list 2021-08-31 12:54:06 +00:00
Nikita Mikhaylov
ae241fc129 done 2021-08-31 12:54:06 +00:00
Vasily Nemkov
109d2f63d0 Fixed tests and minor style issues 2021-08-31 15:39:15 +03:00
Mike Kot
4194ed2ec7 Merge remote-tracking branch 'upstream/master' into improvement/merge-tree-part-opt-parse 2021-08-31 14:35:14 +02:00
Nikita Mikhaylov
e39f9aabde delete useless file 2021-08-31 12:29:51 +00:00
alesapin
7223a32aac Followup 2021-08-31 15:29:33 +03:00
alesapin
9a3bbf44e3 Better description 2021-08-31 15:28:17 +03:00
vdimir
e8e26463bf
Change signature of ASTSelectQuery::arrayJoinExpressionList 2021-08-31 15:27:44 +03:00
vdimir
34b9bd7d33
Fix build: missing argument in inferJoinKeyCommonType 2021-08-31 15:27:44 +03:00
vdimir
3443b3b5c4
Separate functoin buildJoinedPlan, minor refactoring 2021-08-31 15:27:44 +03:00
vdimir
30e112d6a6
Do not convert join key columns for storage/dict join 2021-08-31 15:27:43 +03:00
vdimir
c4654d6b45
Add option to getMostSubtype/getLeastSupertype to not throw exception 2021-08-31 15:27:25 +03:00
alesapin
921e51e061 Remove some obsolete settings for replicated fetches 2021-08-31 15:22:56 +03:00
vdimir
becd288cb9
Try to remove useless applyJoinKeyConvert 2021-08-31 15:18:44 +03:00
Nikita Mikhaylov
8abee2480e src/ 2021-08-31 12:14:08 +00:00
Nikita Mikhaylov
3027efe860 after merge with master 2021-08-31 12:09:35 +00:00
Anton Popov
268550cc4b
Merge pull request #28272 from Algunenano/faster_sumIF 
Speed up sumIf/countIf
 2021-08-31 15:00:46 +03:00
Nikita Mikhaylov
f8d4f04294 Merge upstream/master into global-merge-executor (using imerge) 2021-08-31 11:52:11 +00:00
Nikita Mikhaylov
1adb9bfe23 better 2021-08-31 11:02:39 +00:00
alesapin
a1ac4fc7e0 Comments 2021-08-31 14:02:26 +03:00
alesapin
49d247ca40 Fix compaction after snapshot 2021-08-31 13:59:39 +03:00
Nikolay Degterinsky
ed4d5edd9a Style fix 2021-08-31 13:53:44 +03:00
Nikita Mikhaylov
00394014ff
Merge pull request #28382 from DimasKovas/fix_arcadia_build_31_08_2021 
Try to fix arcadia build (generate ya.make)
 2021-08-31 13:49:32 +03:00
Nikolay Degterinsky
e8713a7d7e Merge branch 'master' into ncb/import-export-lz4 2021-08-31 13:32:39 +03:00
alesapin
525999145e
Merge pull request #26791 from ClickHouse/async-reads 
Experiment with asynchronous readers
 2021-08-31 13:17:45 +03:00
alesapin
e8ab58bc87 Remove strange comment 2021-08-31 13:07:05 +03:00
alesapin
167372e48a Add comment 2021-08-31 12:57:22 +03:00
alesapin
decbd99338 Remove debug logs 2021-08-31 12:50:11 +03:00
alesapin
ff0ed1b290 Fix unit test 2021-08-31 12:42:15 +03:00
alesapin
a5400928ab Interm 2021-08-31 12:12:11 +03:00
Maksim Kita
4ebd0ae941
Merge pull request #28102 from ClickHouse/executable-table-function 
Merging #23192
 2021-08-31 12:00:50 +03:00
Nikolai Kochetov
8143c2e2b8
Merge pull request #28353 from ClickHouse/another-try-to-fix-BackgroundPoolTask-decrement 
Another try to fix BackgroundPoolTask decrement.
 2021-08-31 11:34:26 +03:00
tavplubix
0708d51ad1
Merge pull request #28310 from azat/intersecting-parts 
Fix intersecting parts due to new part had been replaced with an empty part
 2021-08-31 10:27:33 +03:00
Dmitrii Kovalkov
b1eb48804b Run generate-ya-make.sh 2021-08-31 09:05:33 +03:00
Alexey Milovidov
3663597fa3 Remove idiotic whitespace 2021-08-31 02:54:11 +03:00
Alexey Milovidov
ca6f6306f3 Merge branch 'master' into async-reads 2021-08-31 02:52:58 +03:00
alexey-milovidov
c6a754289b
Merge pull request #28361 from DimasKovas/block_ouput_streams_compats 
Add compat between SinkToStorage and BlockOutputStream
 2021-08-31 01:41:33 +03:00
Alexey Milovidov
2198548ead Remove unused header 2021-08-31 00:16:19 +03:00
Maksim Kita
c1482ee45b CHJIT added PageArena 2021-08-30 22:42:35 +03:00
Nikita Mikhaylov
c4416906c8 done 2021-08-30 19:37:03 +00:00
Maksim Kita
c9a0b1e28e Updated tests 2021-08-30 21:41:36 +03:00
Azat Khuzhin
2c0390f326 Fix removing of parts in a Temporary state (v2 followup) 
CI reports [1]:

    <Fatal> : Logical error: 'Deleting data part all_0_0_0_3 doesn't exist'.
    ...
    <Fatal> BaseDaemon: 8. ./obj-x86_64-linux-gnu/../src/Storages/MergeTree/MergeTreeData.cpp:1254: DB::MergeTreeData::removePartsFinally() @ 0x1f94dffa in /usr/bin/clickhouse

  [1]: https://clickhouse-test-reports.s3.yandex.net/0/855a53ff8160c4638fe345b0d26e062804ba790a/stress_test_(debug).html#fail1

Follow-up for: #28221
 2021-08-30 21:07:26 +03:00
Azat Khuzhin
0828548b88 Fix intersecting parts due to new part had been replaced with an empty part 
AFAICS the problem is that some parts may be replaced with empty parts
(after #25820), and removed by the cleanup thread, due to it is empty
[1] (while it should not be deleted since it can download source part):

    <details>

    ```
    2021.08.18 20:11:22.687933 [ 341 ] {} <Trace> test_dpefxp.alter_table_1 (0758ca24-90e7-452c-8758-ca2490e7252c): Created log entry for mutation -1_115_115_0_146
    ...
    2021.08.18 20:11:22.707609 [ 22825 ] {} <Trace> test_dpefxp.alter_table_6 (766ae414-e113-4965-b66a-e414e1137965): Executing log entry to mutate part -1_115_115_0 to -1_115_115_0_146
    2021.08.18 20:11:22.707643 [ 22825 ] {} <Debug> test_dpefxp.alter_table_6 (766ae414-e113-4965-b66a-e414e1137965): Source part -1_115_115_0 for -1_115_115_0_146 is not ready; will try to fetch it instead
    ...
    2021.08.18 20:11:22.709397 [ 333 ] {} <Trace> test_dpefxp.alter_table_6 (ReplicatedMergeTreeQueue): Not executing log entry queue-0000001579 for part -1_115_115_0 because it is covered by part -1_115_115_0_146 that is currently executing.
    2021.08.18 20:11:22.718861 [ 22825 ] {} <Information> test_dpefxp.alter_table_6 (766ae414-e113-4965-b66a-e414e1137965): DB::Exception: No active replica has part -1_115_115_0_146 or covering part
    ...
    2021.08.18 20:11:27.936829 [ 295 ] {} <Information> test_dpefxp.alter_table_6 (766ae414-e113-4965-b66a-e414e1137965): Going to replace lost part -1_115_115_0_146 with empty part
    2021.08.18 20:11:27.957839 [ 295 ] {} <Information> test_dpefxp.alter_table_6 (766ae414-e113-4965-b66a-e414e1137965): Created empty part -1_115_115_0_146 instead of lost part
    ...
    2021.08.18 20:11:28.731635 [ 257 ] {} <Trace> test_dpefxp.alter_table_6 (ReplicatedMergeTreeCleanupThread): Cleared 190 old blocks from ZooKeeper
    ...
    2021.08.18 20:11:28.734507 [ 257 ] {} <Trace> test_dpefxp.alter_table_6 (766ae414-e113-4965-b66a-e414e1137965): Will try to insert a log entry to DROP_RANGE for part: -1_115_115_0_146
    2021.08.18 20:11:28.779373 [ 22837 ] {} <Debug> test_dpefxp.alter_table_6 (766ae414-e113-4965-b66a-e414e1137965): Removed 1 parts inside -1_115_115_0_146.
    ...
    2021.08.18 20:11:28.792600 [ 273 ] {} <Trace> test_dpefxp.alter_table_6 (766ae414-e113-4965-b66a-e414e1137965): Created log entry /clickhouse/tables/00993_system_parts_race_condition_drop_zookeeper_test_dpefxp/alter_table/log/log-0000003459 for merge -1_111_118_1
    ...
    2021.08.18 20:11:28.910988 [ 354 ] {} <Error> test_dpefxp.alter_table_7 (ReplicatedMergeTreeQueue): Code: 49. DB::Exception: Part -1_111_118_1 intersects next part -1_115_115_0_146. It is a bug. (LOGICAL_ERROR), Stack trace (when copying this message, always include the lines below):
    2021.08.18 20:11:31.282160 [ 305 ] {} <Error> test_dpefxp.alter_table_2 (ReplicatedMergeTreeQueue): Code: 49. DB::Exception: Part -1_111_118_1 intersects next part -1_115_115_0_146. It is a bug. (LOGICAL_ERROR), Stack trace (when copying this message, always include the lines below):
    ```

    </details>

  [1]: https://clickhouse-test-reports.s3.yandex.net/27752/59e3cb18f4e53c453951267b5599afeb664290d8/functional_stateless_tests_(release,_wide_parts_enabled).html
 2021-08-30 21:05:46 +03:00
kssenii
335b6f12fd Fix 2021-08-30 20:59:03 +03:00
Raúl Marín
d3d8d4eae6 PVS Studio 2021-08-30 19:37:54 +02:00
Raúl Marín
92799f6cff Make gcc happy 2021-08-30 19:09:04 +02:00
Vasily Nemkov
3ca0b0c860 Fixed GCC-9 build 2021-08-30 19:08:02 +03:00
alexey-milovidov
ea84b939c2
Merge pull request #28333 from DimasKovas/try_to_fix_arcadia_2_30_08_2021 
Try to fix arcadia build
 2021-08-30 18:53:14 +03:00
Vasily Nemkov
c902afddde Added system.session_log table 
Which logs all the info about LogIn, LogOut and LogIn Failure events.
Additional info that is logged:
- User name
- event type (LogIn, LogOut, LoginFailure)
- Event date\time\time with microseconds
- authentication type (same as for IDENTIFIED BY of CREATE USER statement)
- array of active settings profiles upon login
- array of active roles upon login
- array of changed settings with corresponding values
- client address and port
- interface (TCP\HTTP\MySQL\PostgreSQL, etc.)
- client info (name, version info)
- optional LoginFailure reason text message.

Added some tests to verify that events are properly saved with all necessary info via following interfaces:
- TCP
- HTTP
- MySQL

Known limitations
- Not tested against named HTTP sessions, PostgreSQL and gRPC, hence those are not guaranteed to work 100% properly.
 2021-08-30 18:28:28 +03:00
Dmitrii Kovalkov
54eeeb7cf2 Add compat between SinkToStorage and BlockOutputStream 2021-08-30 16:58:40 +03:00
vdimir
943b71e4ff
Fix non joined rows from nullable column 2021-08-30 16:33:01 +03:00
vdimir
bb1d22aa19
Log join input and output structure 2021-08-30 16:33:01 +03:00
alesapin
2160a2f8f7 Followup 2021-08-30 16:23:31 +03:00
Kseniia Sumarokova
00bea71610
Merge pull request #25251 from kssenii/disk-over-web-server 
ClickHouse over web server with static files
 2021-08-30 16:18:59 +03:00
alesapin
00935c3bb3 Trying to debug keeper logs disappearance 2021-08-30 16:07:58 +03:00
Nikolai Kochetov
ab45c412d0 Another try to fix BackgroundPoolTask decrement. 2021-08-30 15:45:32 +03:00
Raúl Marín
11d242cf7d If: Check if work is necessary and combine flags 2021-08-30 14:28:09 +02:00
Raúl Marín
7fb57482c0 Fix bug in countBytesInFilterWithNull 2021-08-30 13:35:22 +02:00
Raúl Marín
d0f8e2d063 Remove memcpy and use the heap 2021-08-30 13:31:51 +02:00
Nikita Mikhaylov
fb66ab75be
Merge pull request #25633 from Avogar/json-as-string 
Allow data in square brackets in JSONAsString format
 2021-08-30 14:06:28 +03:00
tavplubix
b47f34aa17
Merge pull request #28313 from amosbird/fastload 
Fast data parts loading by delaying table startup process
 2021-08-30 13:53:55 +03:00
tavplubix
855a53ff81
Merge pull request #28195 from ClickHouse/maybe_fix_livelock_in_zookeeper_client 
Maybe fix livelock in ZooKeeper client
 2021-08-30 13:40:00 +03:00
Kruglov Pavel
95fd6197dc
Merge pull request #27729 from ContentSquare/sequenceMatchQuickCheck 
Sequence match quick check
 2021-08-30 13:04:23 +03:00
Nikolay Degterinsky
b9ea863a88 Remove unnecessary include 2021-08-30 12:58:56 +03:00
Nikolay Degterinsky
2433d88b82 Merge branch 'master' into log_queries_probability 2021-08-30 12:55:33 +03:00
Nikolay Degterinsky
c9adef001d Change settings earlier 2021-08-30 12:52:35 +03:00
alesapin
531079c452
Merge pull request #28221 from azat/remove-Temporary-part-fix 
Fix removing of parts in a Temporary state
 2021-08-30 12:39:08 +03:00
zhangxiao871
0cf6506156 reserve for newPoints. 2021-08-30 17:29:25 +08:00
Dmitrii Kovalkov
2c8e2c6995 Fix arcadia build 2021-08-30 12:19:13 +03:00
Raúl Marín
5ad8e93458 Speedup sumIf with Nullable types 2021-08-30 10:54:49 +02:00
Raúl Marín
a5c8af1e1b Speed up sumIf 2021-08-30 10:54:49 +02:00
Vladimir Chebotarev
de5daa22ab Revert "MATERIALIZE COLUMN xxx FINAL" 
This reverts commit bf1430c1ec.
 2021-08-30 11:50:26 +03:00
Vladimir Chebotarev
40bedd88e7 One more fix. 2021-08-30 11:50:01 +03:00
Vladimir Chebotarev
06476089d9 Fixed PVS report. 2021-08-30 11:50:01 +03:00
Vladimir Chebotarev
0d6e0181d6 MATERIALIZE COLUMN xxx FINAL 2021-08-30 11:49:59 +03:00
Vladimir Chebotarev
428657b14f 2. Draw the rest of that owl. 2021-08-30 11:49:26 +03:00