thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 03:12:43 +00:00
Code Issues Packages Projects Releases Wiki Activity
116,374 Commits 9,326 Branches 1,783 Tags 2.1 GiB
6caf8080b0
Commit Graph

42263 Commits

Author SHA1 Message Date
Alexey Milovidov
8e6100b0f9 Merge branch 'clang-17-2' 2023-06-01 01:33:23 +02:00
Alexey Milovidov
c54abdb4f8 Support clang-17 2023-06-01 01:32:03 +02:00
Maksim Kita
da09823ece Compile aggregate expressions enable by default 2023-05-31 23:14:39 +03:00
Nikita Mikhaylov
3543d95980
Parallel replicas always skip unavailable ones (#50293) 2023-05-31 22:10:33 +02:00
SmitaRKulkarni
123cd505ee
Merge branch 'master' into 43983_Updated_to_hold_lock_fetch_part 2023-05-31 21:59:55 +02:00
alekar
38db3313be
Merge branch 'master' into minor-cgroup-improvements 2023-05-31 10:01:06 -07:00
Alexander Tokmakov
2357c03388
Merge branch 'master' into fix_materialized_mysql_poll_timeout 2023-05-31 19:47:14 +03:00
Sema Checherinda
aedd3afb8a
fix hung in unit tests (#50391) 
* fix hung in unit tests

* Update gtest_writebuffer_s3.cpp

* Update gtest_writebuffer_s3.cpp

---------

Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2023-05-31 19:20:58 +03:00
Kruglov Pavel
0beca0336d
Merge pull request #49112 from ClickHouse/Avogar-patch-3 
Fix possible terminate called for uncaught exception in some places
 2023-05-31 16:55:43 +02:00
Nikita Mikhaylov
a225663b8f
Merge branch 'master' into 46229-repl-clickhouse-keeper 2023-05-31 16:17:34 +02:00
HaiBo Li
52c2e1d037
Merge branch 'master' into add_boost_link 2023-05-31 21:46:42 +08:00
Alexander Tokmakov
f74d7474d2 fix poll timeout in MaterializedMySQL 2023-05-31 15:04:41 +02:00
kssenii
dbae50b6db Better logging 2023-05-31 14:50:59 +02:00
kssenii
a59effcc88 Minor improvements 2023-05-31 14:40:13 +02:00
kssenii
7d077f6130 Add one more assertion 2023-05-31 14:30:27 +02:00
Alexander Tokmakov
4d4112ff53
Revert "less logs in WriteBufferFromS3" (#50390) 2023-05-31 15:26:56 +03:00
kssenii
74dc37cf61 Fix assertion 2023-05-31 14:15:28 +02:00
Kruglov Pavel
3460667cac
Merge pull request #47496 from iiiuwioajdks/allow_empty 
Allow empty column names in CSVWithNames/TSVWithNames
 2023-05-31 13:24:39 +02:00
Kruglov Pavel
ec10415328
Merge branch 'master' into fix-secure-async-read-write 2023-05-31 13:16:52 +02:00
Vitaly Baranov
6d45d0c374
Use fingerprints instead of key IDs in encrypted disks (#49882) 
* Use fingerprints instead of key IDs to find keys in encrypted disks.
Always use little endian in the headers of encryption files.

* Add tests.

* Fix copying binary files to test containers.

* Fix ownership for copied files in test containers.

* Add comments after review.

---------

Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
 2023-05-31 13:11:10 +02:00
vdimir
9e4eafc75f
Merge pull request #49359 from ClickHouse/vdimir/matcher_join_use_nulls 2023-05-31 13:04:14 +02:00
Nikita Mikhaylov
1a00a5fa4a
Merge branch 'master' into add_boost_link 2023-05-31 13:01:50 +02:00
Sema Checherinda
ab027ca1ce
Merge pull request #50275 from CheSema/d-tor-free 
buffers d-tor finalize free
 2023-05-31 12:26:19 +02:00
lihaibo42
8ce56dfe46 Link boost::context library to clickhouse_common_io 2023-05-31 18:20:14 +08:00
kssenii
322d9b68f6 Merge remote-tracking branch 'upstream/master' into add-some-assertions 2023-05-31 12:04:44 +02:00
Kseniia Sumarokova
410eb44f0f
Merge branch 'master' into fix-logging 2023-05-31 11:55:53 +02:00
alekar
64cc0a4a69
Merge branch 'master' into minor-cgroup-improvements 2023-05-31 01:52:38 -07:00
Manas Alekar
65586c50f5 Minor improvements in CGroup awareness. 
1. Support CGroup2 in getMemoryAmountOrZero().
2. Report CFS period and quota in asynchronous metric log.
 2023-05-31 01:52:02 -07:00
Sema Checherinda
fbae970b8e
Merge branch 'master' into less-logs 2023-05-31 10:21:24 +02:00
Robert Schulze
c5343679df
Merge branch 'master' into msan-siphash-keyed 2023-05-31 09:05:24 +02:00
Robert Schulze
f171ad93bc
Merge pull request #49656 from ClickHouse/rs/entropy-learned-hashing 
Add SQL functions for Entropy Learned Hashing
 2023-05-31 08:26:57 +02:00
Alexey Gerasimchuk
44ba35d2c1
Merge branch 'master' into ADQM-830 2023-05-31 15:07:37 +10:00
pufit
dbd3766f5f Specify roles in users.xml 2023-05-30 20:42:49 -04:00
Nikita Mikhaylov
31829f7cfc
Merge branch 'master' into 46229-repl-clickhouse-keeper 2023-05-31 02:29:08 +02:00
Robert Schulze
fc157b3dad
Merge branch 'master' into msan-siphash-keyed 2023-05-30 21:30:33 +02:00
kssenii
f15769ee6c Fix 2023-05-30 21:29:08 +02:00
kssenii
7fcafaabd5 Fix 2023-05-30 20:56:48 +02:00
Kseniia Sumarokova
55b2e42150
Merge pull request #50232 from kssenii/fix-iceberg 
Fix iceberg metadata parsing
 2023-05-30 20:11:11 +02:00
Michael Kolupaev
536c4a99c8 Fix clickhouse-local crashing when writing empty Arrow or Parquet output 2023-05-30 10:45:49 -07:00
Nikolay Degterinsky
1a6517d4a0 Fix priority of the operators after IS NULL 2023-05-30 17:13:39 +00:00
vdimir
783b54624b
add comment to join_use_nulls in QueryAnalyzer::resolveMatcher 2023-05-30 16:49:10 +00:00
vdimir
d727893693
Handle function nodes in getColumnSideFromJoinTree 2023-05-30 16:40:40 +00:00
vdimir
12141bb6cd
Check can become nullable before applying join_use_nulls 2023-05-30 16:40:39 +00:00
vdimir
40d658e467
Fix join_use_nulls in analyzer 2023-05-30 16:40:36 +00:00
Kruglov Pavel
7966fa2da6
Merge pull request #50034 from Avogar/fiber-local-var-2 
Fix assert in SpanHolder::finish() with fibers attempt 2
 2023-05-30 17:50:31 +02:00
Kruglov Pavel
1c904ecc8e
Fix typo 2023-05-30 16:25:17 +02:00
Kruglov Pavel
2dd4701115
Merge branch 'master' into allow_empty 2023-05-30 16:04:12 +02:00
Sema Checherinda
91a3c88126 less logs in WriteBufferFromS3 2023-05-30 15:49:47 +02:00
Sema Checherinda
84d6bfe5f6
Merge pull request #50204 from azat/InsertSelectQueries-pe 
Add new profile events for queries with subqueries
 2023-05-30 15:19:04 +02:00
Kruglov Pavel
092b06f32e
Fix special build 2023-05-30 14:36:47 +02:00
Nikolay Degterinsky
391e61549d Fixes 2023-05-30 12:18:56 +00:00
avogar
c87e8c1f23 Fix bitShift* functions with both constant arguments 2023-05-30 12:12:54 +00:00
Robert Schulze
1cbce21968
Better, pt. II 2023-05-30 11:46:23 +00:00
Smita Kulkarni
fe944a55dc Updated to hold temporary_directory_lock till part is fetched, renamed and committed 2023-05-30 13:30:47 +02:00
Kruglov Pavel
fe82d2bbe2
Fix build 2023-05-30 12:58:37 +02:00
Sema Checherinda
0a128cec61 remove tricky debug trap 2023-05-30 12:28:58 +02:00
Sema Checherinda
2d2b411c26 fix build 2023-05-30 12:26:20 +02:00
Sema Checherinda
0b4ea3e2e1 remove reminder comments 2023-05-30 12:26:20 +02:00
Sema Checherinda
b13990efcc fix build 2023-05-30 12:26:20 +02:00
Sema Checherinda
13dcb62ffb fix logs engins 2023-05-30 12:26:20 +02:00
Sema Checherinda
a5dcd8dabb do not call finalize in d-tor MergedBlockOutputStream::Finalizer 2023-05-30 12:26:20 +02:00
Sema Checherinda
d95e5b51af mark all finalize calls in buffers d-tors with issue id 2023-05-30 12:26:20 +02:00
Sema Checherinda
99ad481505 fix sizes.log writes 2023-05-30 12:26:20 +02:00
Sema Checherinda
fe3939287b add test, fix .gin_sid files 2023-05-30 12:26:20 +02:00
Sema Checherinda
0e019c8e83 turn off some d-tor finalize 2023-05-30 12:26:20 +02:00
Sema Checherinda
100b4d0969 cope with finalize in d-tors 2023-05-30 12:26:20 +02:00
Antonio Andelic
bb2acb50d2
Merge pull request #50186 from LiuYangkuan/support_fqdn_in_cluster_discovery 
support passing fqdn to register cluster node in keeper
 2023-05-30 11:54:09 +02:00
Robert Schulze
2d06947522
Better 2023-05-30 09:06:45 +00:00
Robert Schulze
fb6c4f2802
Fix msan issue, pt. II 2023-05-30 08:51:44 +00:00
Val Doroshchuk
ecea1ac090 Fix crash when Pool::Entry::disconnect() is called 
Many Pool::Entry objects can keep the same pointer to Pool::Connection.
If Pool::Entry::disconnect() is called on one such object,
Pool::removeConnection() is called to remove Pool::Connection from the pool,
where connection->ref_count is cleared and connection->removed_from_pool is set.

Next Pool::Entry::~Entry() calls decrementRefCount() with

1. const auto ref_count = data->ref_count.fetch_sub(1);
   where data->ref_count will be negative, since it was cleared
2. checks removed_from_pool and deletes Pool::Connection
   but there might be multiple Entry objects still keep pointer to this Pool::Connection

Suggesting not to clear ref_count on disconnect()
and delete Pool::Connection only on the last Pool::Entry is being destroyed.

Fixes ea375ef989
 2023-05-30 10:29:49 +02:00
SmitaRKulkarni
f2172ab8dd
Merge pull request #46661 from ClickHouse/42192_Lower_parallel_parsing_threads_with_globs 
When reading from multiple files displace parallel parsing
 2023-05-30 08:53:08 +02:00
Victor Krasnov
6c94632d47 Deprive toStartOfWeek and toLastDayOfWeek functions of in-source documentation 2023-05-29 22:10:34 +00:00
Alexey Milovidov
f97460b453
Merge branch 'master' into nickitat-patch-12 2023-05-30 00:27:56 +03:00
Robert Schulze
aefa9df1ce
Merge branch 'master' into rs/entropy-learned-hashing 2023-05-29 22:20:50 +02:00
Nikolay Degterinsky
e10f951467 Fix IS (NOT) NULL operator priority 2023-05-29 19:11:09 +00:00
Alexey Milovidov
701e2fb6e3
Merge pull request #50242 from ClickHouse/fix-max-parallel-replicas-custom-key 
Fix `02535_max_parallel_replicas_custom_key`
 2023-05-29 21:06:57 +03:00
Alexey Milovidov
3cc9feafc2
Merge pull request #50285 from ucasfl/uniqExactIf 
Parallel merge of uniqExactIf states
 2023-05-29 19:42:44 +03:00
Nikita Taranov
b2921df228
Merge branch 'master' into optimize_reading2 2023-05-29 18:35:53 +02:00
Robert Schulze
516fa1c375
Merge branch 'master' into rs/entropy-learned-hashing 2023-05-29 17:40:14 +02:00
Nikita Taranov
437880d4c1
Enable enable_memory_bound_merging_of_aggregation_results by default 2023-05-29 15:52:50 +02:00
Nikita Taranov
87d6564b9b
Update gtest_async_loader.cpp 2023-05-29 15:50:06 +02:00
Nikita Taranov
be62c8faba fix build 2023-05-29 15:27:39 +02:00
Nikita Taranov
a3652f1727 Merge branch 'master' into optimize_reading2 2023-05-29 15:26:18 +02:00
Sergei Trifonov
688f23b325
Merge pull request #50313 from ClickHouse/fix-flaky-async-loader-test 
Fix flaky `AsyncLoader.StaticPriorities` unit test
 2023-05-29 13:42:15 +02:00
Sergei Trifonov
c26b3b36ae
Merge branch 'master' into fix-flaky-async-loader-test 2023-05-29 13:38:35 +02:00
Sergei Trifonov
355f917daa
Update src/Common/tests/gtest_async_loader.cpp 2023-05-29 13:38:14 +02:00
Sergei Trifonov
df14ec4d58
Merge branch 'master' into clang15_crash 2023-05-29 13:36:08 +02:00
alesapin
6b7e2a0041
Merge pull request #49961 from CheSema/mem-consumption-s3-write 
Introduce a limitation of async calls inside s3 write buffer, make a test that memory is bounded even when s3 is slow
 2023-05-29 13:29:01 +02:00
serxa
bdb974ab3c Fix flaky AsyncLoader.StaticPriorities unit test 2023-05-29 11:21:44 +00:00
Raúl Marín
1ae1f1273a Avoid clang 15 crash 2023-05-29 11:40:45 +02:00
Victor Krasnov
0ad5b9f598 Merge branch 'master' of github.com:ClickHouse/ClickHouse into ADQM-810-dev 2023-05-29 08:26:26 +00:00
Robert Schulze
41d60f0be3
Fix style 2023-05-29 08:08:47 +00:00
Robert Schulze
47daaad172
Merge pull request #49778 from helifu/master2 
Support custom 'order by' expression for system log tables
 2023-05-29 10:05:14 +02:00
flynn
758beda39a
Merge branch 'master' into uniqExactIf 2023-05-29 11:06:08 +08:00
Alexey Milovidov
53de761e0e
Merge pull request #50272 from ClickHouse/inverse-async-loader-priority 
Unify priorities: rework AsyncLoader
 2023-05-29 05:01:44 +03:00
Alexey Milovidov
ef7e9e28b7
Merge pull request #50231 from ClickHouse/inverse-io-scheduler-priority-3 
Unify priorities: rework IO scheduling subsystem
 2023-05-29 04:34:23 +03:00
Alexey Milovidov
1875a93328
Merge pull request #50224 from Avogar/fix-custom-separated-ignore-spaces 
Fix skipping spaces at end of row in CustomSeparatedIgnoreSpaces format
 2023-05-29 02:42:38 +03:00
Alexey Milovidov
9647cfa33d
Merge pull request #50291 from kitaisreal/jit-aggregation-nullable-key-fix 
JIT aggregation nullable key fix
 2023-05-29 02:08:02 +03:00
Alexey Milovidov
e1d535c890
Merge pull request #50287 from kitaisreal/function-equals-nan-fix 
Comparison functions NaN fix
 2023-05-29 01:56:02 +03:00
alesapin
1bf32f70a0
Merge branch 'master' into fix-iceberg 2023-05-28 21:59:13 +02:00
alesapin
3a3cee586a
Merge pull request #50257 from CurtizJ/useless-refactoring 
Yet another refactoring
 2023-05-28 21:42:16 +02:00
alesapin
274fc4640b
Merge pull request #49004 from ClickHouse/keeper-log-improvements 
Small Keeper performance improvements for larger requests
 2023-05-28 21:25:23 +02:00
Maksim Kita
e9840bc6e1 JIT aggregation nullable key fix 2023-05-28 21:05:17 +03:00
Nikita Taranov
42a9072888 Merge branch 'master' into optimize_reading2 2023-05-28 18:04:44 +02:00
Maksim Kita
7ef20bbdcd Function equals NaN fix 2023-05-28 17:02:46 +03:00
Kruglov Pavel
3aa795355f
Fix build 2023-05-28 15:46:32 +02:00
Kruglov Pavel
65c34e73e7
Clean 2023-05-28 15:45:40 +02:00
flynn
ec82c657eb Parallel merge of uniqExactIf states 2023-05-28 06:04:23 +00:00
helifu
6e8136e6f4 Add 'settings' expression for system log tables 2023-05-28 12:59:34 +08:00
helifu
59a815e4e6 Update according to review comments 2023-05-28 12:59:34 +08:00
helifu
57e5a4aa63 Add the 'order_by' description for system log tables 2023-05-28 12:59:34 +08:00
helifu
03e11b6e16 Remove the duplicate pair of parentheses 2023-05-28 12:59:34 +08:00
helifu
90a6437716 Support custom 'order by' expression for system log tables 2023-05-28 12:59:34 +08:00
Vitaly Baranov
0e2156c776
Fix checksum calculation for backup entries (#50264) 
* Fix possible deadlock in calculating backup entries' checksums.

* Fix data type used to store checksums.

* Allow ReadBufferFromMemory::seek() to go to the end of the buffer
(seek(0, SEEK_SET) must be allowed for an empty buffer)

* Add tests.

---------

Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
 2023-05-27 14:35:37 +02:00
Sema Checherinda
86f85d7b3b cope with finalize in d-tors 2023-05-27 01:18:01 +02:00
avogar
ef9bae50b9 Fix bugs in Poco, use true non-blocking IO 2023-05-26 23:11:57 +00:00
Alexey Gerasimchuk
12105c3dfa
Merge branch 'master' into ADQM-830 2023-05-27 07:35:05 +10:00
Sema Checherinda
1588219ffd comments 2023-05-26 19:03:59 +02:00
Sema Checherinda
8523cd54a0 working with review comments 2023-05-26 19:00:22 +02:00
Sema Checherinda
7c9d2ab6d0 working with review comments 2023-05-26 19:00:13 +02:00
Sema Checherinda
b0eff95388 update doc, add profile event WriteBufferFromS3WaitInflightLimitMicroseconds 2023-05-26 18:59:21 +02:00
Sema Checherinda
7031796d3f fixing tests 2023-05-26 18:59:21 +02:00
Sema Checherinda
0b781bd4f1 fix test_merge_canceled_by_s3_errors 2023-05-26 18:59:21 +02:00
Sema Checherinda
206efee9b7 add test with merge tree table 2023-05-26 18:59:21 +02:00
Sema Checherinda
4249bda449 s3_max_inflight_parts_for_one_file with tests 2023-05-26 18:59:11 +02:00
serxa
c4687a8b93 Unify priorities: rework AsyncLoader 2023-05-26 16:46:59 +00:00
Azat Khuzhin
6b5933a3db Add new profile events for queries with subqueries 
Differs from SelectQuery/InsertQuery that it could be > 1 (in case of
MATERIALIZED VIEW attached to the table), since each local SELECT/INSERT
query is accounted, not only initial.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-26 18:28:40 +02:00
Robert Schulze
f49160ef4e
Build partial key positions from entire training data 2023-05-26 15:27:56 +00:00
serxa
edffedbabf fix 2023-05-26 15:10:08 +00:00
Antonio Andelic
4bdce2a669 Fix test for new analyzer 2023-05-26 15:02:50 +00:00
Sergei Trifonov
84203341ca
Merge branch 'master' into async-loader-workloads 2023-05-26 16:17:56 +02:00
Sergei Trifonov
640ada6182
Merge branch 'master' into inverse-io-scheduler-priority-3 2023-05-26 16:16:06 +02:00
Denny Crane
3ef04ad417
Doc. clarify dns_max_consecutive_failures one more time (#50212) 2023-05-26 15:56:24 +02:00
Sergei Trifonov
0d1f2e297b
Unify priorities: lower value means higher priority (#50205) 2023-05-26 15:55:30 +02:00
Sema Checherinda
9603f2b49c
Merge pull request #50190 from ClibMouse/feature/unify-core-id-querying 
Implement a uniform way to query processor core IDs
 2023-05-26 15:46:39 +02:00
Victor Krasnov
050b93363b Add unit tests 2023-05-26 12:25:11 +00:00
Anton Popov
f9c7fcbe0a
Merge branch 'master' into useless-refactoring 2023-05-26 13:45:58 +02:00
Antonio Andelic
a9569478e6
fix typo 2023-05-26 13:09:09 +02:00
Antonio Andelic
3d98e591ba Handle backwards compatibility 2023-05-26 10:46:18 +00:00
Kruglov Pavel
67c8c5c561
Add missing include 2023-05-26 12:44:43 +02:00
Kruglov Pavel
03652efe58
Add missing include 2023-05-26 12:44:28 +02:00
alesapin
bdf1c54f36
Merge branch 'master' into transactions_for_encrypted_disk 2023-05-26 12:25:39 +02:00
Robert Schulze
2298eeb2b2
Merge branch 'master' into rs/entropy-learned-hashing 2023-05-26 12:04:49 +02:00
Robert Schulze
d14dafcd9c
Merge pull request #50103 from rschu1ze/space 
Spark compatibility: Add new function space()
 2023-05-26 11:16:37 +02:00
Alexey Milovidov
27617fb590
Merge pull request #50216 from azat/fix-asts 
Fix UB for INTO OUTFILE extensions (APPEND / AND STDOUT) and WATCH EVENTS
 2023-05-26 12:06:02 +03:00
Robert Schulze
e2a5acbeb9
Merge branch 'master' into space 2023-05-26 10:40:02 +02:00
Antonio Andelic
767193caf5 Merge branch 'master' into keeper-log-improvements 2023-05-26 07:35:36 +00:00
Raúl Marín
b3a96de533
Pure parallel replicas: JOIN support (#49544) 2023-05-26 01:04:24 +02:00
Anton Popov
612173e734 refactoring near alter conversions 2023-05-25 22:54:54 +00:00
Alexey Gerasimchuck
01f3a46cf0 fixed wrong case in removeNullable 2023-05-25 22:49:36 +00:00
Kruglov Pavel
18be731e30
Merge branch 'master' into fix-secure-async-read-write 2023-05-26 00:46:33 +02:00
Kruglov Pavel
f03ca41b08
Fix build 2023-05-26 00:21:46 +02:00
Kruglov Pavel
67b78829fc
Fix build 2023-05-26 00:21:14 +02:00
Kruglov Pavel
a9082b24b4
Fix build 2023-05-26 00:20:20 +02:00
Robert Schulze
bc869eac7b
Merge branch 'master' into msan-siphash-keyed 2023-05-26 00:18:07 +02:00
Alexey Gerasimchuk
8d7cb7fc3b
Merge branch 'master' into ADQM-830 2023-05-26 07:49:51 +10:00
Alexey Gerasimchuk
613568423d
Update src/Processors/Formats/Impl/CSVRowInputFormat.cpp 
Co-authored-by: Kruglov Pavel <48961922+Avogar@users.noreply.github.com>
 2023-05-26 07:49:45 +10:00
serxa
4f0aeee236 fix more conflicts 2023-05-25 20:50:22 +00:00
Kruglov Pavel
1964d1bb7e
Fix comment 2023-05-25 22:30:16 +02:00
avogar
42e1e3ae20 Fix working with secure socket after async connection 2023-05-25 20:24:03 +00:00
Robert Schulze
c4f91a1c45
Merge branch 'master' into space 2023-05-25 19:56:20 +02:00
Robert Schulze
ad4a21034f
Fix msan issue in keyed siphash 
Issue:
https://s3.amazonaws.com/clickhouse-test-reports/0/ffdd91669471f4934704f98f0191524496b4e85b/fuzzer_astfuzzermsan/report.html

Repro:
SELECT hex(sipHash128ReferenceKeyed((toUInt64(2), toUInt64(-9223372036854775807)))) GROUP BY (toUInt64(506097522914230528), toUInt64(now64(2, NULL + NULL), 1084818905618843912)), toUInt64(2), NULL + NULL, char(-2147483649, 1)

Minimal repro:
SELECT sipHash64Keyed((2::UInt64, toUInt64(2)), 4) GROUP BY toUInt64(2)
 2023-05-25 17:52:03 +00:00
Robert Schulze
eca08438f4
Fix macos build 2023-05-25 17:05:18 +00:00
alesapin
e4c1e2f232 Fix build while it's not failing locally 2023-05-25 17:37:09 +02:00
alesapin
e94b0c8e5e Fix bug 2023-05-25 16:38:19 +02:00
Nikita Mikhaylov
0580859e6f Better 2023-05-25 14:05:44 +00:00
kssenii
3fefacbf20 Fix 2023-05-25 15:48:56 +02:00
Nikita Mikhaylov
cf6ff7ab32 Merge branch 'master' of github.com:ClickHouse/ClickHouse into 46229-repl-clickhouse-keeper 2023-05-25 13:41:25 +00:00
serxa
0ca526c603 Unify priorities: rework IO scheduling subsystem 2023-05-25 13:25:41 +00:00
serxa
3ef6cb7bdc git-apply #50205 2023-05-25 13:24:45 +00:00
Kseniia Sumarokova
f1a3c9cfd5
Merge pull request #50109 from kssenii/abstract-async-prefetched-buffer 
Make async prefetched buffer work with arbitrary impl
 2023-05-25 15:06:44 +02:00
serxa
b8d3e495e5 add pool_id out-of-bound checks 2023-05-25 12:42:19 +00:00
Nikita Mikhaylov
1c3b6738f4
Fixes for parallel replicas (#50195) 2023-05-25 14:41:04 +02:00
serxa
e3ce2f834a fix style 2023-05-25 12:35:00 +00:00
Sergei Trifonov
2df22d396d
Merge branch 'master' into async-loader-workloads 2023-05-25 14:32:12 +02:00
Robert Schulze
8804dfd4b0
Fix resizing 2023-05-25 11:57:09 +00:00
Sergei Trifonov
78c89da8bb
Update src/Common/AsyncLoader.cpp 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2023-05-25 13:48:03 +02:00
Sergei Trifonov
32cb9931b6
Update src/Common/AsyncLoader.h 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2023-05-25 13:47:56 +02:00
Han Fei
55c2dbcc2d
Merge pull request #50062 from ZhiguoZh/20230511-toyear 
Optimize predicate with toYear converter
 2023-05-25 13:25:55 +02:00
avogar
ce99825200 Fix skipping spaces at end of row in CustomSeparatedIgnoreSpaces format 2023-05-25 11:19:15 +00:00
vdimir
2df4130d82
Update src/Interpreters/ClusterDiscovery.cpp 2023-05-25 13:06:40 +02:00
alesapin
5b76ab4e03 Fix build 2023-05-25 13:02:40 +02:00
Sema Checherinda
23f894b995
Merge pull request #49777 from helifu/master1 
Add 'initial_query_id' field for system.processors_profile_log
 2023-05-25 12:55:32 +02:00
Sema Checherinda
3329a8428d
Merge pull request #49779 from helifu/master3 
Add 'partitions' field for system.query_log
 2023-05-25 12:51:40 +02:00
Igor Nikonov
1c0b02c3c4
Merge pull request #49503 from ClickHouse/fill_with_by_sorting_prefix_2 
WITH FILL by sorting prefix
 2023-05-25 12:37:40 +02:00
alesapin
b2c9611da6 Fix build 2023-05-25 12:01:24 +02:00
何李夫
e4c8c4cecf
Add zookeeper name in endpoint id (#49780) 
* Add zookeeper name in endpoint id

When we migrate a replicated table from one zookeeper cluster to
another (the reason why we migration is that zookeeper's load is
too high), we will create a new table with the same zpath, but it
will fail and the old table will be in trouble.

Here is some infomation:
1.old table:
  CREATE TABLE a1 (`id` UInt64)
  ENGINE = ReplicatedMergeTree('/clickhouse/tables/default/a1/{shard}', '{replica}')
  ORDER BY (id);
2.new table:
  CREATE TABLE a2 (`id` UInt64)
  ENGINE = ReplicatedMergeTree('aux1:/clickhouse/tables/default/a1/{shard}', '{replica}')
  ORDER BY (id);
3.error info:
  <Error> executeQuery: Code: 220. DB::Exception: Duplicate interserver IO endpoint:
          DataPartsExchange:/clickhouse/tables/default/a1/01/replicas/02.
          (DUPLICATE_INTERSERVER_IO_ENDPOINT)
  <Error> InterserverIOHTTPHandler: Code: 221. DB::Exception: No interserver IO endpoint
          named DataPartsExchange:/clickhouse/tables/default/a1/01/replicas/02.
          (NO_SUCH_INTERSERVER_IO_ENDPOINT)

* Revert "Add zookeeper name in endpoint id"

This reverts commit 9deb75b249619b7abdd38e3949ca8b3a76c9df8e.

* Add zookeeper name in endpoint id

When we migrate a replicated table from one zookeeper cluster to
another (the reason why we migration is that zookeeper's load is
too high), we will create a new table with the same zpath, but it
will fail and the old table will be in trouble.

* Fix incompatible with a new setting

* add a test, fix other issues

* Update 02442_auxiliary_zookeeper_endpoint_id.sql

* Update 02735_system_zookeeper_connection.reference

* Update 02735_system_zookeeper_connection.sql

* Update run.sh

* Remove the 'no-fasttest' tag

* Update 02442_auxiliary_zookeeper_endpoint_id.sql

---------

Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
Co-authored-by: Alexander Tokmakov <tavplubix@gmail.com>
 2023-05-25 12:50:14 +03:00
alesapin
8a7c4bee53 Merge branch 'master' into transactions_for_encrypted_disk 2023-05-25 11:35:17 +02:00
Azat Khuzhin
b680697cce Initialize POD members of ASTs to make it less error-prone 
The cost of initializing members is insignificant in compare to parsing,
while the cost of the error is high.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-25 10:18:55 +02:00
Azat Khuzhin
b30cfe5503 Fix UB in ASTWatchQuery for is_watch_events 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-25 10:15:30 +02:00
Azat Khuzhin
c053d75741 Fix formatting of INTO OUTFILE extensions (APPEND / AND STDOUT) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-25 10:15:30 +02:00
Azat Khuzhin
9582d9e892 Fix UB for INTO OUTFILE extensions (APPEND / AND STDOUT) 
MSAn report:

==38627==WARNING: MemorySanitizer: use-of-uninitialized-value
    0 0x555599f5e114 in std::__1::__unique_if<DB::WriteBufferFromFile>::__unique_single std::__1::make_unique[abi:v15000]<> build_docker/./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:714:32
    1 0x555599f5e114 in DB::ClientBase::initOutputFormat() build_docker/./src/Client/ClientBase.cpp:604:21
    2 0x555599f590a8 in DB::ClientBase::onData() build_docker/./src/Client/ClientBase.cpp:446:5
    3 0x555599f6f36e in DB::ClientBase::receiveAndProcessPacket() build_docker/./src/Client/ClientBase.cpp:1019:17
    4 0x555599f6e863 in DB::ClientBase::receiveResult() build_docker/./src/Client/ClientBase.cpp:987:18
    5 0x555599f6c05b in DB::ClientBase::processOrdinaryQuery() build_docker/./src/Client/ClientBase.cpp:905:13
    6 0x555599f67e05 in DB::ClientBase::processParsedSingleQuery() build_docker/./src/Client/ClientBase.cpp:1711:13
    7 0x555599f86fb6 in DB::ClientBase::executeMultiQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_docker/./src/Client/ClientBase.cpp:1975:21

  Uninitialized value was created by a heap allocation
    8 0x55559bd3e038 in DB::ParserExplainQuery::parseImpl(DB::IParser::Pos&, std::__1::shared_ptr<DB::IAST>&, DB::Expected&) build_docker/./src/Parsers/ParserExplainQuery.cpp:53:26
    9 0x55559bce31f4 in DB::IParserBase::parse(DB::IParser::Pos&, std::__1::shared_ptr<DB::IAST>&, DB::Expected&)::$_0::operator()() const build_docker/./src/Parsers/IParserBase.cpp:13:20
    ..
    21 0x55559be13b5c in DB::parseQueryAndMovePosition(DB::IParser&, char const*&, char const*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, bool, unsigned long, unsigned long) build_docker/./src/Parsers/parseQuery.cpp:357:18
    22 0x555599f5673a in DB::ClientBase::parseQuery(char const*&, char const*, bool) const build_docker/./src/Client/ClientBase.cpp:362:15
    23 0x555599f84a4f in DB::ClientBase::analyzeMultiQueryText() build_docker/./src/Client/ClientBase.cpp:1821:24
    24 0x555599f867b3 in DB::ClientBase::executeMultiQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_docker/./src/Client/ClientBase.cpp:1910:22
    25 0x555599f8a2fd in DB::ClientBase::processQueryText(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) build_docker/./src/Client/ClientBase.cpp:2120:12
    26 0x555599f94aee in DB::ClientBase::runNonInteractive() build_docker/./src/Client/ClientBase.cpp:2403:9

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-05-25 10:15:23 +02:00
Alexey Gerasimchuck
75791d7a63 Added input_format_csv_trim_whitespaces parameter 2023-05-25 07:51:32 +00:00
Zhiguo Zhou
1bc4eb1a6c OptimizeDateFilterVisitor: Revise variable names for clarity 2023-05-25 13:47:03 +08:00
helifu
802b63f2ab Add 'initial_query_id' field for system.processors_profile_log 
Facilitate profile data association and aggregation for the same query
 2023-05-25 09:37:02 +08:00
Zhiguo Zhou
773a5bbbaa Optimize predicate with toYear converter 
The date converters, such as toYear, are widely used in the where
clauses of the SQL queries, however, these conversions are often
expensive due to the complexity of the calendar system.

The function preimage is found an optimization for the predicates
with the converters. Given a predicate, toYear(c) = y, we could
convert it to its equivalent form: c >= b AND c <= e, where b is
"y-01-01" and e is "y-12-31". The similar transformation applies
to other comparisons (<>, <, >, <=, <=).

This commit implemented the above transformation at the AST level
by adding a new pass in the TreeOptimizer and a new AST visitor
for in-place replacing the predicates of toYear with the converted
ones.
 2023-05-25 09:11:51 +08:00
Kseniia Sumarokova
2e17503d36
Merge pull request #50187 from kssenii/fix-pg-source 
Fix PostgreSQLSource reading all unread the data in onFinish
 2023-05-24 22:51:48 +02:00
Alexander Gololobov
8996fcb090
Merge pull request #50193 from ClickHouse/fix_for_replicate_delete 
Don't replicate delete through DDL worker if there is just 1 shard
 2023-05-24 22:45:00 +02:00
Dan Roscigno
026a15d8a7
Update dns_max_consecutive_failures docs (#50196) 
Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
 2023-05-24 21:01:59 +02:00
Alexander Sapin
3e69648268 Fxi moar 2023-05-24 20:02:03 +02:00