thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-17 03:42:48 +00:00
Code Issues Packages Projects Releases Wiki Activity
98,601 Commits 9,315 Branches 1,783 Tags 2.1 GiB
59605a6c7e
Commit Graph

31791 Commits

Author SHA1 Message Date
Kseniia Sumarokova
fb93ca7ca8
Merge pull request #41510 from kssenii/maybe-fix-storage-nats-tests 
Try fix flaky test_storage_nats
 2022-09-20 11:32:10 +02:00
Vladimir C
4935bf60b7
Merge pull request #41130 from ClickHouse/vdimir/decimal-in-41125 2022-09-20 10:07:50 +02:00
Vladimir C
befc0440e3
Merge pull request #41511 from Algunenano/batch_nullable_assert 2022-09-20 09:21:52 +02:00
Filatenkov Artur
be8221fae1
Correct work with negative distance 2022-09-20 09:48:33 +03:00
Alexey Milovidov
1c3696bd38 Merge branch 'fix-three-fourth-of-trash' of github.com:ClickHouse/ClickHouse into fix-three-fourth-of-trash 2022-09-20 05:22:51 +02:00
Alexey Milovidov
cd68c6dc2d Update tests (the new results looks correct) 2022-09-20 05:22:31 +02:00
Alexey Milovidov
03bbf0676f
Merge branch 'master' into revert_prefixes_s3 2022-09-20 06:14:07 +03:00
Alexey Milovidov
133318d8bd
Merge branch 'master' into hlee-s390x-siphash 2022-09-20 06:10:57 +03:00
Alexey Milovidov
37a4be249b
Merge pull request #41483 from amosbird/jbod-fix1 
Fix nullptr dereference in DB::VolumeJBOD::reserve
 2022-09-20 06:08:14 +03:00
Frank Chen
a89140ae98 Fix style 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-09-20 10:37:54 +08:00
Alexey Milovidov
a35c62933f
Update src/Functions/sleep.h 
Co-authored-by: Robert Schulze <robert@clickhouse.com>
 2022-09-20 05:16:21 +03:00
Alexey Milovidov
7c299dc561
Update src/Storages/MergeTree/MergeTreeDataMergerMutator.cpp 
Co-authored-by: Robert Schulze <robert@clickhouse.com>
 2022-09-20 05:16:12 +03:00
Alexey Milovidov
e7525da88f
Update src/Storages/MergeTree/MergeTreePartsMover.cpp 
Co-authored-by: Robert Schulze <robert@clickhouse.com>
 2022-09-20 05:16:02 +03:00
Alexey Milovidov
4fcfcf282a
Update src/Storages/examples/merge_selector2.cpp 
Co-authored-by: Robert Schulze <robert@clickhouse.com>
 2022-09-20 05:15:47 +03:00
Duc Canh Le
e2bd478b98 fix conversion from nullable fixed string to string 2022-09-20 10:05:18 +08:00
Sergei Trifonov
cf2db48c29
Merge pull request #40631 from ClickHouse/readonly-settings-allow 
Allow to modify constrained settings in readonly mode
 2022-09-20 02:18:14 +02:00
Sergei Trifonov
358c5a0139
Merge pull request #41532 from ClickHouse/s3-api-calls-events-for-disk-s3 
Detailed S3 profile events `DiskS3*` per S3 API call split for S3 ObjectStorage
 2022-09-20 02:10:50 +02:00
Vitaly Baranov
b1199878ad
Merge pull request #39977 from vitlibar/improve-recovery-of-replicated-access-storage 
Improve recovery of ReplicatedAccessStorage after errors.
 2022-09-20 00:36:30 +02:00
Filatenkov Artur
3598a7e680
Fix query type information 2022-09-20 01:03:55 +03:00
alesapin
ee24b21aaf Fix revert order 2022-09-19 21:19:36 +02:00
kssenii
e40ced28fd Done 2022-09-19 21:05:30 +02:00
Sergei Trifonov
7902922ae4
Merge branch 'master' into readonly-settings-allow 2022-09-19 21:00:12 +02:00
serxa
2ef696ffe1 fix issues 2022-09-19 18:40:32 +00:00
Igor Nikonov
aca810ba62 Merge remote-tracking branch 'origin/master' into distinct_in_order_wo_order_by 2022-09-19 18:34:38 +00:00
Igor Nikonov
78bb598d64 Handling DISTINCT in order on top of other read in order optimization 2022-09-19 18:33:05 +00:00
alesapin
f67e23183f Fix parts removal on drop + add unrelated test 2022-09-19 20:19:49 +02:00
serxa
7a1d4a404a add DiskS3* profile events per S3 API calls 2022-09-19 18:10:47 +00:00
serxa
f8aa738511 more conventional profile events names 2022-09-19 17:23:22 +00:00
serxa
6ed53585ab rearrange and fix profile event descriptions 2022-09-19 17:03:48 +00:00
Igor Nikonov
5de8298ca1
Merge branch 'master' into fix_sort_desc_when_read_in_order 2022-09-19 18:54:42 +02:00
Azat Khuzhin
f76f14a99c Fix possible hung/deadlock on query cancellation 
Right due to graceful cancellation of the query, it is possible to hung
accepting new queries or even have a deadlock, this is because
cancellation is done while acquiring ProcessListBase::mutex.

So this patch makes query cancellation lock-free, and now the lock will
be acquired only for preparing the query and after cancel is done.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-19 17:31:21 +02:00
Amos Bird
652c7cb519
Add comments to FunctionNameNormalizer. 2022-09-19 22:07:04 +08:00
Amos Bird
1a21f0a0f0
normalize ast for replicated metadata 2022-09-19 22:06:41 +08:00
Amos Bird
0bb70d4378
Make minmax_count projection backward compatible 2022-09-19 22:06:40 +08:00
Vitaly Baranov
052ecaa211
Merge branch 'master' into improve-recovery-of-replicated-access-storage 2022-09-19 15:26:21 +02:00
Vitaly Baranov
f75dd93965 Mask some information in logs. 2022-09-19 15:00:06 +02:00
Sergei Trifonov
ff2e1ff8d8
Merge branch 'master' into support_s3_api_event_metrics 2022-09-19 14:35:45 +02:00
Raúl Marín
e89dffbb95 Fix AggregateFunctionNullVariadic batch with null_is_skipped=false 2022-09-19 14:34:08 +02:00
Kruglov Pavel
712aebc077
Merge pull request #41083 from Avogar/fix-map-combinator 
Fix possible segfaults, use-heap-after-free and memory leak in aggregate function combinators
 2022-09-19 14:29:17 +02:00
kssenii
f8665c83da Add retries for nats 2022-09-19 14:27:11 +02:00
kssenii
30a5477514 Add retries for nats 2022-09-19 14:26:38 +02:00
Alexander Tokmakov
c6efb1773e
Merge branch 'master' into fix_drop_partition_on_cluster 2022-09-19 15:18:05 +03:00
Kruglov Pavel
4c3194eefe
Merge pull request #41286 from azat/utf8-fix 
Do not allow invalid sequences influence other rows in lowerUTF8/upperUTF8
 2022-09-19 14:07:10 +02:00
kssenii
e3cd3686af Rename 2022-09-19 14:02:51 +02:00
Sergei Trifonov
f7a1ed3141
Merge branch 'master' into readonly-settings-allow 2022-09-19 12:52:36 +02:00
serxa
a0bfa801e5 explicit ctor 2022-09-19 10:52:08 +00:00
Kseniia Sumarokova
631aad1671
Merge branch 'master' into fix-logical-error-write-through-cache 2022-09-19 12:29:55 +02:00
Antonio Andelic
1763af37b9
Merge pull request #41403 from ClickHouse/keeper-fix-shutdown-without-server 
Fix Keeper segfault during shutdown when RAFT server failed to start
 2022-09-19 12:03:58 +02:00
Igor Nikonov
81a46b6346 Merge remote-tracking branch 'origin/master' into fix_sort_desc_when_read_in_order 2022-09-19 09:40:23 +00:00
Igor Nikonov
54c2fed9cc Update: use type explicitly 2022-09-19 09:31:45 +00:00
Alexey Milovidov
45b6052b20 Merge branch 'master' into compress-marks 2022-09-19 10:57:05 +02:00
vdimir
a6dc99983d
Fix tidy build 2022-09-19 08:42:23 +00:00
vdimir
5c9bfbc116
Fix incorrect result in case of decimal precision loss in IN operator 2022-09-19 08:42:22 +00:00
vdimir
c2634c9fae
Revert "Throw exception on precision lost during decimal to decimal convertion" 
This reverts commit d4233cf3c658adf2541070b1bbba678ad7c769db.
 2022-09-19 08:42:22 +00:00
vdimir
c33e231ce9
Throw exception on precision lost during decimal to decimal convertion 2022-09-19 08:42:18 +00:00
Igor Nikonov
b4a3ac926f Fix typo 2022-09-19 08:26:44 +00:00
Vladimir C
b6b5932efc
Merge pull request #41223 from ClickHouse/vdimir/exponential_rewrite_21557 2022-09-19 10:07:24 +02:00
Alexey Milovidov
b9bd2c8373
Merge pull request #41458 from ClickHouse/fix-bug-orc 
Fix very strange behavior of Apache ORC
 2022-09-19 10:43:17 +03:00
Alexey Milovidov
000b6ac81c Fix error 2022-09-19 09:30:48 +02:00
Duc Canh Le
61d6fae14b add settings allow_suspicious_fixed_string_types 2022-09-19 15:00:04 +08:00
Alexey Milovidov
b87d12494e Fix build 2022-09-19 08:53:20 +02:00
Alexey Milovidov
5224fc4d7f Fix strange code 2022-09-19 08:53:20 +02:00
Alexey Milovidov
d1275091e8 Fix strange code 2022-09-19 08:53:20 +02:00
Alexey Milovidov
6a473d8998 Merge 2022-09-19 08:53:20 +02:00
Alexey Milovidov
bb6f1bfce2 Fix 9/10 of trash 2022-09-19 08:53:20 +02:00
Alexey Milovidov
730655d4fd Fix 8/9 of trash 2022-09-19 08:53:20 +02:00
Alexey Milovidov
42b0d444da Fix 7/8 of trash 2022-09-19 08:53:20 +02:00
Alexey Milovidov
91baedf03a Fix 6/7 of trash 2022-09-19 08:53:20 +02:00
Alexey Milovidov
ab4db2d0c4 Fix 5/6 of trash 2022-09-19 08:50:53 +02:00
Alexey Milovidov
84f42e0874 Fix 3/4 of trash 2022-09-19 08:50:53 +02:00
Amos Bird
d2601b8192
Add a comment 2022-09-19 14:41:34 +08:00
Alexey Milovidov
3c0961c48b
Merge branch 'master' into fix-map-combinator 2022-09-19 09:32:16 +03:00
Alexey Milovidov
b01f5bdca8
Merge pull request #39354 from ClickHouse/window-order-by 
Respect ORDER BY clause in window functions
 2022-09-19 09:22:37 +03:00
Amos Bird
09b75901f0
Fix nullptr deref 2022-09-19 14:07:31 +08:00
Alexey Milovidov
6a09340f11
Merge pull request #41459 from azat/io/threadpoolreader-elapsed 
Account any IO that is done from ThreadPoolReader
 2022-09-19 09:04:20 +03:00
Frank Chen
b056bc1021 Add span for executeDDLQueryOnCluster 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-09-19 13:43:40 +08:00
Alexey Milovidov
81e8cb4be6
Merge branch 'master' into fix-bug-orc 2022-09-19 06:38:17 +03:00
Yong Wang
ec852b3faa Kusto-phase1 : change the parser in ClientBase from shared_ptr to unique_ptr 2022-09-18 20:38:07 -07:00
Alexey Milovidov
d4b9fe41be
Merge pull request #41457 from ClickHouse/remove-trash-5 
Remove trash from Field
 2022-09-19 06:36:48 +03:00
Alexey Milovidov
57a1f47972
Merge pull request #41441 from ClickHouse/pod-array-overflow 
Fix overflow in `PODArray`
 2022-09-19 06:34:21 +03:00
Alexey Milovidov
ac193facde
Merge branch 'master' into add_missed_throw 2022-09-19 06:31:12 +03:00
Alexey Milovidov
df52df83f9
Merge pull request #41463 from ClickHouse/add-test-38383 
Add a test for #38383
 2022-09-19 06:27:38 +03:00
Yong Wang
37ae7a8cca Kusto-phase1 : apply parser comments to kusto, remove unused variable 2022-09-18 20:25:27 -07:00
Frank Chen
06ae2fb2b5 Remove assertEOF to improve the compability 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-09-19 11:20:58 +08:00
Frank Chen
2ae43bb4e8 Add test case 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-09-19 11:18:35 +08:00
Yong Wang
4e4a04c08b
Merge branch 'ClickHouse:master' into Kusto-phase1 2022-09-18 21:40:30 -04:00
Igor Nikonov
785c33bf7d Updating sort description for ReadFromMergeTree correctly 2022-09-18 23:35:21 +00:00
Alexey Milovidov
790cb730d6 Fix clang-tidy 2022-09-19 00:34:41 +02:00
Alexey Milovidov
cc28120761 Merge branch 'master' into pod-array-overflow 2022-09-19 00:34:38 +02:00
Alexey Milovidov
f3fd5ef845 Fix clang-tidy 2022-09-19 00:32:52 +02:00
Alexey Milovidov
36aaeecc2f Fix clang-tidy 2022-09-19 00:32:11 +02:00
Alexey Milovidov
5d9e125c71 Fix clang-tidy 2022-09-19 00:28:32 +02:00
Igor Nikonov
e888859396 Fix: correct sort description for ReadFromMergeTree with read in order 
optimization
 2022-09-18 22:21:13 +00:00
alesapin
a5a9d89fb9
Merge branch 'master' into enable_projections_and_zero_copy 2022-09-18 17:20:08 +02:00
alesapin
e4c120742e Add missed throw expression 2022-09-18 16:56:19 +02:00
alesapin
560bc2bc22
Merge pull request #41386 from ClickHouse/fix_redundant_fetch 
Fix redundant fetch in zero copy replication
 2022-09-18 15:57:27 +02:00
alesapin
1d0dbfb112
Merge pull request #41444 from ClickHouse/one_wrong_config_assert 
One more validation for broken config in keeper
 2022-09-18 15:56:48 +02:00
Kruglov Pavel
519bcbb3be
Merge pull request #41236 from Avogar/fix-totals-extremes 
Don't output totals/extremes in all row formats, update docs
 2022-09-18 14:55:07 +02:00
Kseniia Sumarokova
d6748b182f
Update CachedOnDiskWriteBufferFromFile.cpp 2022-09-18 14:48:24 +02:00
Vitaly Baranov
e7e51ab2d9 Add comments. 2022-09-18 12:44:05 +02:00
Vitaly Baranov
69996c960c Add retries for the initialization of ReplicatedAccessStorage. 2022-09-18 12:44:05 +02:00
Vitaly Baranov
5365b105cc Add SYSTEM RELOAD USERS command. 2022-09-18 12:44:00 +02:00
Vitaly Baranov
1b40f94c6b Move common code from MemoryAccessStorage and DiskAccessStorage to IAccessStorage. 2022-09-18 12:17:15 +02:00
Alexey Milovidov
09ae1a4bf4
Merge branch 'master' into hlee-s390x-siphash 2022-09-18 11:09:02 +03:00
Azat Khuzhin
64b44be513 Account any IO that is done from ThreadPoolReader 
In case of some issues with the disk it is possible that
preadv2(RWF_NOWAIT)==EAGAIN can take a lot of time, so let's measure
even this case.

I've verified with a simple (almost) loop of preadv2(RWF_NOWAIT), and
got this:

    EAGAIN: 8565670 (out of 17148317)
    EAGAIN: 8565670 (out of 17148317)
    EAGAIN: 8565670 (out of 17148317)
    too long at offset=8691694, ret=18446744073709551615 (14.5016)
    too long at offset=8694766, ret=18446744073709551615 (14.4338)
    too long at offset=8694766, ret=18446744073709551615 (16.2144)

So here you can see that preadv2(RWF_NOWAIT) took 14-16 seconds in case
of failure.

Complete code can be found here [1].

  [1]: https://gist.github.com/azat/f4a6b0750316c42bec021064e0a7627c

I came up with this patch because I saw some stranginess in
ProfileEvents, that does not show all the time that was spend in IO when
pread_threadpool method was used.
Also there was some stranginess with trace_log too.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-18 10:05:21 +02:00
Alexey Milovidov
7e6da853cd Merge branch 'master' into pod-array-overflow 2022-09-18 09:24:23 +02:00
Alexey Milovidov
a6e3ab5515 Fix error 2022-09-18 09:21:57 +02:00
Alexey Milovidov
5c75a7d661 Fix error 2022-09-18 09:20:48 +02:00
Alexey Milovidov
c41e37fa8e Code simplification 2022-09-18 08:58:32 +02:00
Alexey Milovidov
3c43d6516b Fix style 2022-09-18 08:56:52 +02:00
Alexey Milovidov
bafc5f24c7 Merge branch 'master' into remove-trash-5 2022-09-18 08:56:02 +02:00
Alexey Milovidov
8764fa4439 Fix very strange behavior of Apache ORC 2022-09-18 08:25:25 +02:00
Alexey Milovidov
a958751c98
Merge pull request #40219 from den-crane/test/zvonand-monotonic 
test for datetime64 monotonic #39864
 2022-09-18 07:48:49 +03:00
Alexey Milovidov
6b752bff3a
Merge branch 'master' into revert_prefixes_s3 2022-09-18 07:10:57 +03:00
Alexey Milovidov
32efe6952e
Merge pull request #41429 from ClickHouse/forbid-dlopen-2 
Forbid invocations of `dlopen`
 2022-09-18 07:05:56 +03:00
Alexey Milovidov
791de6592b Remove trash from Field 2022-09-18 05:16:08 +02:00
HarryLeeIBM
f3e8738145 Fixed issues in code review 2022-09-17 19:48:08 -07:00
Alexey Milovidov
c8c728e740
Merge pull request #41449 from ClickHouse/fix-categorial-information 
Fix incorrect properties of `categorialInformationValue` function
 2022-09-18 05:26:26 +03:00
Alexey Milovidov
7f68f2c7df Remove trash from LowCardinality 2022-09-18 04:13:21 +02:00
Alexey Milovidov
15e5fc96a1 Add a trap 2022-09-18 02:24:56 +02:00
Alexey Milovidov
dc76677eb2
Merge pull request #41440 from ClickHouse/remove-cruft 
Remove cruft about gcc
 2022-09-18 03:15:26 +03:00
Alexey Milovidov
e3dafdb865 More hardening 2022-09-18 02:10:06 +02:00
Alexey Milovidov
e5e3bca3ae More hardening and tests 2022-09-18 02:05:13 +02:00
Alexey Milovidov
6c74c54cb5 Merge branch 'master' into pod-array-overflow 2022-09-18 01:24:52 +02:00
Alexey Milovidov
6f1878b12a Fix error 2022-09-18 01:17:22 +02:00
Alexey Milovidov
c424ad12aa Simplification 2022-09-18 00:09:21 +02:00
alesapin
b6d61318d0 One more validation for broken config in keeper 2022-09-18 00:04:16 +02:00
Alexey Milovidov
e2d434ea18 Start with code removal 2022-09-17 23:37:57 +02:00
Alexey Milovidov
2f0684b97c Fix trash in schema inference 2022-09-17 23:11:33 +02:00
Alexey Milovidov
bac578b23a
Merge pull request #41428 from ClickHouse/remove-dlopen 
Remove `dlopen`
 2022-09-18 00:09:57 +03:00
Alexey Milovidov
e3d75b0933 Merge branch 'fix-trash-schema-inference' into pod-array-overflow 2022-09-17 22:56:06 +02:00
Alexey Milovidov
47167494d9 Fix trash in schema inference 2022-09-17 22:53:41 +02:00
Alexey Milovidov
9ca4e2ec7c Fix overflow in PODArray 2022-09-17 22:26:39 +02:00
Alexey Milovidov
f4a48cd4d6 Remove cruft 2022-09-17 22:16:31 +02:00
kssenii
51745dcb43 Fix assertion in write-through cache 2022-09-17 20:20:32 +02:00
Alexander Tokmakov
49a9c9e5fa
Merge pull request #41436 from ClickHouse/revert_38200 
Revert "Keeper faster linearizable reads"
 2022-09-17 20:25:43 +03:00
Robert Schulze
38aecc15a2
Merge pull request #41214 from arenadata/ADQM-528-B 
Add enable_extended_results_for_datetime_functions option to return results of type Date32
 2022-09-17 19:25:11 +02:00
Alexander Tokmakov
42dab6dc02 Revert "Merge pull request #38200 from ClickHouse/keeper-linearizable-reads" 
This reverts commit 222b964070, reversing
changes made to da01982652.
 2022-09-17 19:22:58 +02:00
alesapin
33216dedf9
Merge pull request #41323 from ClickHouse/remove_bad_exception 
Fix parts removal in case of connection loss
 2022-09-17 19:14:53 +02:00
alesapin
dcb1586977 Add comment 2022-09-17 19:11:58 +02:00
alesapin
db30a3041e
Update src/Storages/MergeTree/MutateFromLogEntryTask.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2022-09-17 19:10:23 +02:00
alesapin
84828874e0 Change log level 2022-09-17 19:06:54 +02:00
Anton Popov
07cfb0cba4
Merge branch 'master' into dynamic-columns-21 2022-09-17 19:02:58 +02:00
Alexey Milovidov
9ce0c5f738
Merge pull request #41431 from ClickHouse/remove-method-simplification 
Remove `-WithTerminatingZero` methods
 2022-09-17 19:48:35 +03:00
Azat Khuzhin
01b5b9ceba Do not allow invalid sequences influence other rows in lowerUTF8/upperUTF8 
Right now lowerUTF8() and upperUTF8() does not respect row boundaries,
and so one row may break another.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-17 11:16:45 +02:00
Igor Nikonov
475443c10c
Merge pull request #41257 from ClickHouse/igor/mixed_improvements_along_the_way 
Tiny improvements along the code
 2022-09-17 10:43:46 +02:00
Alexey Milovidov
ada7a44ae4 Remove -WithTerminatingZero methods 2022-09-17 05:34:18 +02:00
Alexey Milovidov
0c3835240b Fix error with multidimensional arrays 2022-09-17 04:16:24 +02:00
Alexey Milovidov
b60a5a5bc5 Merge branch 'master' into fix-getdataat-multidimensional-arrays 2022-09-17 04:07:12 +02:00
Alexey Milovidov
35cce03125 Remove dlopen 2022-09-17 03:02:34 +02:00
Alexey Milovidov
31ebd34f5b
Merge pull request #41193 from azat/DWARF-5-parser-v3 
Add support for DWARF-5 (without emitting them in binaries)
 2022-09-17 02:58:02 +03:00
Alexey Milovidov
c09604bef2
Merge pull request #41363 from kitaisreal/small-style-fixes 
Small style fixes
 2022-09-17 00:19:00 +03:00
Denny Crane
6540d55d42
Merge branch 'master' into test/zvonand-monotonic 2022-09-16 17:02:09 -03:00
Nikita Taranov
6f186d3dd2
Do not return empty blocks from ConvertingAggregatedToChunksTransform (#41152) 
* impl

* add test

* update test
 2022-09-16 21:54:36 +02:00
avogar
fa172e21ae Remove unnecessary method implementation 2022-09-16 17:47:05 +00:00
Igor Nikonov
24de5530d6 Remove unnecessary headers 2022-09-16 17:16:32 +00:00
Igor Nikonov
b34ec6453b Fix review comments 
+ one more test
 2022-09-16 17:11:08 +00:00
Vitaly Baranov
391507b4b6 Update src/Access/DiskAccessStorage.cpp 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2022-09-16 19:02:06 +02:00
Vitaly Baranov
2ec6ef497d Make DiskAccessStorage::insertWithID() more consistent with MemoryAccessStorage::insertWithID(). 2022-09-16 19:02:06 +02:00
Vitaly Baranov
7e716d14cd Make ReplicatedAccessStorage::insertWithID() more consistent with MemoryAccessStorage::insertWithID(). 2022-09-16 19:02:06 +02:00
Vitaly Baranov
646cd55690 Improve recovery of ReplicatedAccessStorage after errors. 2022-09-16 19:02:06 +02:00
Vitaly Baranov
8f9f5c69da Use MemoryAccessStorage to simplify ReplicatedAccessStorage. 2022-09-16 19:02:06 +02:00
Dmitry Novik
eacc790050
Merge branch 'master' into fix-odr-vialation 2022-09-16 18:44:53 +02:00
avogar
c9a9ef5b7e Fix comments 2022-09-16 16:12:30 +00:00
Alexander Tokmakov
65474830ac make tryEnqueueReplicatedDDL virtual 2022-09-16 16:25:32 +02:00
Nikolai Kochetov
16f78eb804
Merge pull request #41281 from ClickHouse/fix-key-condition-with-actions-dag-literal-name 
Fix wrong literal name in KeyCondition using ActionsDAG.
 2022-09-16 16:06:38 +02:00
Kruglov Pavel
2d4a6b38af
Merge branch 'master' into fix-totals-extremes 2022-09-16 15:03:59 +02:00
alesapin
d9c3549cc8 Revert s3 prefixes + add retries on write requests unexpected errors 2022-09-16 14:43:43 +02:00
alesapin
456da30c9f Revert "Merge pull request #41249 from ClickHouse/revert-40968-s3-sharding-2" 
This reverts commit ac60300997, reversing
changes made to a9c272283a.
 2022-09-16 14:06:26 +02:00
alesapin
b4d2d217f4
Merge branch 'master' into enable_projections_and_zero_copy 2022-09-16 13:53:17 +02:00
alesapin
06ccdb1ecf Fix nasty bug 2022-09-16 13:49:39 +02:00
Alexander Tokmakov
9597a47016
Merge pull request #41379 from ClickHouse/tavplubix-patch-5 
Fix an assertion in `wasInvolvedInTransaction`
 2022-09-16 14:24:26 +03:00
Robert Schulze
b32b02d844
Merge pull request #40897 from ClickHouse/catboost-bridge-resurrected 
Move CatBoost evaluation into clickhouse-library-bridge
 2022-09-16 13:12:09 +02:00
Anton Popov
5584ef71f6
Merge pull request #41388 from den-crane/bug/optimize_rewrite_sum_if_to_count_if 
Disable optimize_rewrite_sum_if_to_count_if by default
 2022-09-16 13:05:23 +02:00
Igor Nikonov
eeecaf7a31 Merge remote-tracking branch 'origin/master' into distinct_in_order_wo_order_by 2022-09-16 10:30:52 +00:00
Igor Nikonov
ba1f7de243 Fix review comments 2022-09-16 10:30:30 +00:00
Antonio Andelic
acd9cb089e Fix segfault during shutdown when RAFT server failed to start 2022-09-16 10:03:21 +00:00
Roman Vasin
bc97da686a Add const for bool enable_extended_results_for_datetime_functions 2022-09-16 09:54:24 +00:00
alesapin
af259a2a68
Update src/Storages/MergeTree/MergeFromLogEntryTask.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2022-09-16 11:50:07 +02:00
Roman Vasin
d82cd7b007 Add many cosmetic changes to C++ code and MD docs 2022-09-16 09:37:30 +00:00
Nikolay Degterinsky
7dac605829 Merge remote-tracking branch 'master' into new_parser 2022-09-16 08:56:30 +00:00
Nikolay Degterinsky
db85f97a3f Fix 2022-09-16 08:55:58 +00:00
Roman Vasin
7e733887b9 Add narrowing for toStartOfISOYear; improve 02403_enable_extended_results_for_datetime_functions 2022-09-16 08:40:15 +00:00
Alexey Milovidov
701fdd77b4 Fix wrong implementation of getDataAt for multidimensional arrays 2022-09-16 08:12:28 +02:00
Antonio Andelic
222b964070
Merge pull request #38200 from ClickHouse/keeper-linearizable-reads 
Keeper faster linearizable reads
 2022-09-16 07:58:52 +02:00
Alexey Milovidov
da01982652
Merge pull request #41046 from azat/build/llvm-15 
Switch to llvm/clang 15
 2022-09-16 07:31:06 +03:00
Nikolay Degterinsky
f63b2d4b9f Revert some breaking changes 2022-09-16 03:29:34 +00:00
Nikolay Degterinsky
36c3a18035 Fix tests 2022-09-16 02:38:14 +00:00
Nikolay Degterinsky
2e2d804b90 Better 2022-09-15 21:54:37 +00:00
Raúl Marín
c3ff66bd9d
Implement batch processing for aggregate functions with multiple nullable arguments (#41058) 
* Implement batch processing for aggregate functions with multiple nullable arguments

* Fix broken perf test

* Improve filter handling in addBatchSinglePlace with nullable arguments

* Fix detecting the Null filter usage
 2022-09-15 23:51:38 +02:00
Raúl Marín
6dac509739
Speed up reading uniqState (#41089) 
* Speed up reading UniquesHashSet

* Improve uniq serialization tests
 2022-09-15 23:41:15 +02:00
Igor Nikonov
8f500019eb Merge remote-tracking branch 'origin/master' into distinct_in_order_wo_order_by 2022-09-15 21:05:59 +00:00
Alexander Tokmakov
8301f5758b Update IMergeTreeDataPart.cpp 2022-09-15 21:32:57 +02:00
Alexander Tokmakov
03c193ccca fix ON CLUSTER with Replicated database cluster 2022-09-15 21:15:57 +02:00
Denny Crane
7cfdfbe787 disable optimize_rewrite_sum_if_to_count_if by default 2022-09-15 15:30:24 -03:00
Alexander Tokmakov
32391f737a Merge branch 'master' into fix_41209 2022-09-15 20:27:51 +02:00
avogar
ef7f7b30aa Fix tests 2022-09-15 18:20:17 +00:00
avogar
92979af3bf Add missing methods in combinators 2022-09-15 17:45:50 +00:00
alesapin
82611d891b
Merge branch 'master' into fix-odr-vialation 2022-09-15 19:33:09 +02:00
alesapin
be81d653e1 Fix comment 2022-09-15 18:56:16 +02:00
alesapin
b098c1be36 Fix data duplication in zero copy replication 2022-09-15 18:54:27 +02:00
Antonio Andelic
fe78c063f6
Merge branch 'master' into keeper-linearizable-reads 2022-09-15 18:40:23 +02:00
Antonio Andelic
39307f9dba Extract into function 2022-09-15 16:14:53 +00:00
Antonio Andelic
0dbd0c1f72 Merge branch 'master' into keeper-linearizable-reads 2022-09-15 16:03:18 +00:00
Anton Popov
b5e9884355 fix subcolumns tree 2022-09-15 14:17:10 +00:00
Roman Vasin
5c242de43a Merge branch 'master' of github.com:ClickHouse/ClickHouse into ADQM-528-B 2022-09-15 13:47:51 +00:00
alesapin
7570c5cf67 Remove recursive without blobs 2022-09-15 15:31:19 +02:00
HarryLeeIBM
d62ba01e93 Fix SipHash Endian issue for s390x 2022-09-15 06:25:23 -07:00
alesapin
c585ed47fc Forget to remove 2022-09-15 15:15:28 +02:00
serxa
b3b8123f73 fix style 2022-09-15 12:53:56 +00:00
alesapin
e127c8b4a9 Add kringe 2022-09-15 14:49:31 +02:00
alesapin
c04f7afb2a Delete kringe 2022-09-15 14:48:59 +02:00
Roman Vasin
563449779c Replace enable_date32_results by enable_extended_results_for_datetime_functions 2022-09-15 12:48:09 +00:00
Alexander Tokmakov
b26cc5a1ba
Merge pull request #41296 from young-scott/fix_issuse_41096 
fix issue #41096
 2022-09-15 15:44:05 +03:00
avogar
6bb6c9e916 Fix comment 2022-09-15 12:41:39 +00:00
avogar
d41bb7c2a4 Fix comment 2022-09-15 12:40:32 +00:00
Alexander Tokmakov
77c4ea9647 fix 2022-09-15 14:34:31 +02:00
Denny Crane
1b8b2ebed5
Disable optimize_monotonous_functions_in_order_by by default (#41136) 
* #40094 disable optimize_monotonous_functions_in_order_by by default

* fix 01576_alias_column_rewrite test

* fix incorrect 02149_read_in_order_fixed_prefix.sql test

* Update 02149_read_in_order_fixed_prefix.sql
 2022-09-15 13:41:46 +02:00
Azat Khuzhin
5cc5ca22de Fix possible deadlock with async_socket_for_remote/use_hedged_requests and parallel KILL 
Right now it is possible to call QueryStatus::addPipelineExecutor() when
the executors_mutex already acquired, it is possible when the query was
cancelled via KILL QUERY.

Here I will show some traces from debugger from a real example, where
tons of ProcessList::insert() got deadlocked.

Let's look at the lock owner for one of the threads that was deadlocked
in ProcessList::insert():

    (gdb) p *mutex
    $2 = {
      __data = {
        __owner = 46899,
      },
    }

And now let's see the stack trace of the 46899:

    #0  __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:103
    #1  0x00007fb65569b714 in __GI___pthread_mutex_lock (mutex=0x7fb4a9d15298) at ../nptl/pthread_mutex_lock.c:80
    #2  0x000000001b6edd91 in pthread_mutex_lock (arg=0x7fb4a9d15298) at ../src/Common/ThreadFuzzer.cpp:317
    #3  std::__1::__libcpp_mutex_lock (__m=0x7fb4a9d15298) at ../contrib/libcxx/include/__threading_support:303
    #4  std::__1::mutex::lock (this=0x7fb4a9d15298) at ../contrib/libcxx/src/mutex.cpp:33
    #5  0x0000000014c7ae63 in std::__1::lock_guard<std::__1::mutex>::lock_guard (__m=..., this=<optimized out>) at ../contrib/libcxx/include/__mutex_base:91
    #6  DB::QueryStatus::addPipelineExecutor (this=0x7fb4a9d14f90, e=0x80) at ../src/Interpreters/ProcessList.cpp:372
    #7  0x0000000015bee4a7 in DB::PipelineExecutor::PipelineExecutor (this=0x7fb4b1e53618, processors=..., elem=<optimized out>) at ../src/Processors/Executors/PipelineExecutor.cpp:54
    #12 std::__1::make_shared<DB::PipelineExecutor, std::__1::vector<std::__1::shared_ptr<DB::IProcessor>, std::__1::allocator<std::__1::shared_ptr<DB::IProcessor> > >&, DB::QueryStatus*&, void> (__args=@0x7fb63095b9b0: 0x7fb4a9d14f90, __args=@0x7fb63095b9b0: 0x7fb4a9d14f90) at ../contrib/libcxx/include/__memory/shared_ptr.h:963
    #13 DB::QueryPipelineBuilder::execute (this=0x7fb63095b8b0) at ../src/QueryPipeline/QueryPipelineBuilder.cpp:552
    #14 0x00000000158c6c27 in DB::Connection::sendExternalTablesData (this=0x7fb6545e9d98, data=...) at ../src/Client/Connection.cpp:797
    #27 0x0000000014043a81 in DB::RemoteQueryExecutorRoutine::operator() (this=0x7fb63095bf20, sink=...) at ../src/QueryPipeline/RemoteQueryExecutorReadContext.cpp:46
    #32 0x000000000a16dd4f in make_fcontext () at ../contrib/boost/libs/context/src/asm/make_x86_64_sysv_elf_gas.S:71

And also in the logs you can see very strange things for this thread:

    2022.09.13 14:14:51.228979 [ 51145 ] {1712D4E914EC7C99} <Debug> Connection (localhost:9000): Sent data for 1 external tables, total 11 rows in 0.00046389 sec., 23688 rows/sec., 3.84 KiB (8.07 MiB/sec.), compressed 1.1070121092649958 times to 3.47 KiB (7.29 MiB/sec.)
    ...
    2022.09.13 14:14:51.719402 [ 46899 ] {7c90ffa4-1dc8-42fd-938c-4e307c244394} <Debug> executeQuery: (from 10.101.15.181:42478) KILL QUERY WHERE query_id = '1712D4E914EC7C99' (stage: Complete)
    2022.09.13 14:14:51.719488 [ 46899 ] {7c90ffa4-1dc8-42fd-938c-4e307c244394} <Debug> executeQuery: (internal) SELECT query_id, user, query FROM system.processes WHERE query_id = '1712D4E914EC7C99' (stage: Complete)
    2022.09.13 14:14:51.719754 [ 46899 ] {7c90ffa4-1dc8-42fd-938c-4e307c244394} <Trace> ContextAccess (default): Access granted: SELECT(user, query_id, query) ON system.processes
    2022.09.13 14:14:51.720544 [ 46899 ] {7c90ffa4-1dc8-42fd-938c-4e307c244394} <Trace> InterpreterSelectQuery: FetchColumns -> Complete
    2022.09.13 14:14:53.228964 [ 46899 ] {7c90ffa4-1dc8-42fd-938c-4e307c244394} <Debug> Connection (localhost:9000): Sent data for 2 scalars, total 2 rows in 2.6838e-05 sec., 73461 rows/sec., 68.00 B (2.38 MiB/sec.), compressed 0.4594594594594595 times to 148.00 B (5.16 MiB/sec.)

How is this possible? The answer is fibers and query cancellation
routine. During cancellation of async queries it going into fibers again
and try to do this gracefully. However because of this during canceling
query it may call QueryStatus::addPipelineExecutor() from
QueryStatus::cancelQuery().

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-15 14:41:27 +03:00
serxa
4971c1f04d review fixes 2022-09-15 11:34:14 +00:00
Maksim Kita
276d3628e9 Small style fixes 2022-09-15 13:05:18 +02:00
Roman Vasin
3346d8e27b Add many cosmetic changes to C++ code 2022-09-15 10:56:05 +00:00
Igor Nikonov
da75d28019 Merge remote-tracking branch 'origin/master' into distinct_in_order_wo_order_by 2022-09-15 10:54:42 +00:00
Frank Chen
f576d8191c Merge branch 'master' into on_cluster_dll 2022-09-15 18:05:36 +08:00
Kseniia Sumarokova
9e37a6356f
Merge pull request #41297 from kssenii/fix-logical-error 
Fix incorrect logical error in disk object storage
 2022-09-15 12:00:53 +02:00
Kseniia Sumarokova
69f38a6228
Merge pull request #41330 from kssenii/fix-composable-cache-setting-incompatibility 
Fix incompatibility of cache after switching setting `do_no_evict_...` from 1 to 0, 0 to 1
 2022-09-15 12:00:22 +02:00
Frank Chen
ac848727e5 Read tracing context in right pos 2022-09-15 18:00:08 +08:00
Kruglov Pavel
73cf72a5a4
Merge pull request #41309 from Avogar/fix-msgpack 
Add column type check before UUID insertion in MsgPack format
 2022-09-15 11:37:57 +02:00
Kruglov Pavel
982b9baeec
Fix 2022-09-15 11:32:14 +02:00
Nikolay Degterinsky
72d8a04ee6 Add setting 'allow_implicit_no_password' 2022-09-15 08:07:42 +00:00
Robert Schulze
b64af5989e
Merge branch 'master' into opentelemetry_stress 2022-09-15 09:43:11 +02:00
Frank Chen
a4ef0c0281 Set up tracing context for DDLWorker 2022-09-15 14:57:00 +08:00
Frank Chen
490089cc4f Copy tracing context from current thread to DDLLogEntry as parent context 2022-09-15 14:48:24 +08:00
Frank Chen
52224875e2 Serialize tracing context to DDL log entry 2022-09-15 14:47:43 +08:00
Frank Chen
77d0971efa Serialize tracing context 2022-09-15 14:46:55 +08:00
Anton Popov
66667737f0 fix writing of empty columns of type Object 2022-09-15 03:38:22 +00:00
Nikolay Degterinsky
6d01f92641 Fix silly bug & little enhancement 2022-09-15 00:48:14 +00:00
Kseniia Sumarokova
780c0e9b2f
Update CachedOnDiskReadBufferFromFile.cpp 2022-09-15 00:18:23 +02:00
Kseniia Sumarokova
ebf7cb6338
Merge pull request #41308 from ClickHouse/kssenii-patch-5 
Fix after #41042
 2022-09-15 00:15:54 +02:00
Igor Nikonov
f011f72114 Try to apply optimization only if optimizer reaches pre distinct node 
+ more comments
 2022-09-14 20:55:49 +00:00
Kseniia Sumarokova
0b1c2c62fd
Update CachedOnDiskReadBufferFromFile.cpp 2022-09-14 20:59:45 +02:00
kssenii
559c696230 Fix 2022-09-14 20:35:28 +02:00
avogar
f1a0501eb2 Fix memory leaks and segfaults in combinators 2022-09-14 18:01:49 +00:00
Igor Nikonov
bab5744a1d Merge remote-tracking branch 'origin/master' into distinct_in_order_wo_order_by 2022-09-14 17:54:46 +00:00
Igor Nikonov
d91ea0c5ab Fix + tests 2022-09-14 17:54:05 +00:00
Alexander Tokmakov
173df9a73a
Update StorageSystemDDLWorkerQueue.cpp 2022-09-14 20:38:50 +03:00
Alexander Tokmakov
fc73134145 fix race between DROP and short form of ATTACH 2022-09-14 18:03:36 +02:00
alesapin
63213465ef Fix comment 2022-09-14 17:28:12 +02:00
alesapin
ad279940d6 Trying print less scary messages in case No such key for outdated parts 2022-09-14 17:25:58 +02:00
Robert Schulze
fd97058e45
fix: incorporate review comments 2022-09-14 15:21:24 +00:00