thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-17 11:52:27 +00:00
Code Issues Packages Projects Releases Wiki Activity
99,006 Commits 9,326 Branches 1,784 Tags 2.1 GiB
3fdea908ff
Commit Graph

31988 Commits

Author SHA1 Message Date
Frank Chen
a89140ae98 Fix style 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-09-20 10:37:54 +08:00
Alexey Milovidov
a35c62933f
Update src/Functions/sleep.h 
Co-authored-by: Robert Schulze <robert@clickhouse.com>
 2022-09-20 05:16:21 +03:00
Alexey Milovidov
7c299dc561
Update src/Storages/MergeTree/MergeTreeDataMergerMutator.cpp 
Co-authored-by: Robert Schulze <robert@clickhouse.com>
 2022-09-20 05:16:12 +03:00
Alexey Milovidov
e7525da88f
Update src/Storages/MergeTree/MergeTreePartsMover.cpp 
Co-authored-by: Robert Schulze <robert@clickhouse.com>
 2022-09-20 05:16:02 +03:00
Alexey Milovidov
4fcfcf282a
Update src/Storages/examples/merge_selector2.cpp 
Co-authored-by: Robert Schulze <robert@clickhouse.com>
 2022-09-20 05:15:47 +03:00
Duc Canh Le
e2bd478b98 fix conversion from nullable fixed string to string 2022-09-20 10:05:18 +08:00
Sergei Trifonov
cf2db48c29
Merge pull request #40631 from ClickHouse/readonly-settings-allow 
Allow to modify constrained settings in readonly mode
 2022-09-20 02:18:14 +02:00
Sergei Trifonov
358c5a0139
Merge pull request #41532 from ClickHouse/s3-api-calls-events-for-disk-s3 
Detailed S3 profile events `DiskS3*` per S3 API call split for S3 ObjectStorage
 2022-09-20 02:10:50 +02:00
Vitaly Baranov
b1199878ad
Merge pull request #39977 from vitlibar/improve-recovery-of-replicated-access-storage 
Improve recovery of ReplicatedAccessStorage after errors.
 2022-09-20 00:36:30 +02:00
Filatenkov Artur
3598a7e680
Fix query type information 2022-09-20 01:03:55 +03:00
alesapin
ee24b21aaf Fix revert order 2022-09-19 21:19:36 +02:00
kssenii
e40ced28fd Done 2022-09-19 21:05:30 +02:00
Sergei Trifonov
7902922ae4
Merge branch 'master' into readonly-settings-allow 2022-09-19 21:00:12 +02:00
BoloniniD
55fcb98f29 Merge branch 'master' of github.com:ClickHouse/ClickHouse into BLAKE3 2022-09-19 21:53:14 +03:00
serxa
2ef696ffe1 fix issues 2022-09-19 18:40:32 +00:00
Igor Nikonov
aca810ba62 Merge remote-tracking branch 'origin/master' into distinct_in_order_wo_order_by 2022-09-19 18:34:38 +00:00
Igor Nikonov
78bb598d64 Handling DISTINCT in order on top of other read in order optimization 2022-09-19 18:33:05 +00:00
alesapin
f67e23183f Fix parts removal on drop + add unrelated test 2022-09-19 20:19:49 +02:00
serxa
7a1d4a404a add DiskS3* profile events per S3 API calls 2022-09-19 18:10:47 +00:00
serxa
f8aa738511 more conventional profile events names 2022-09-19 17:23:22 +00:00
serxa
6ed53585ab rearrange and fix profile event descriptions 2022-09-19 17:03:48 +00:00
Igor Nikonov
5de8298ca1
Merge branch 'master' into fix_sort_desc_when_read_in_order 2022-09-19 18:54:42 +02:00
Azat Khuzhin
f76f14a99c Fix possible hung/deadlock on query cancellation 
Right due to graceful cancellation of the query, it is possible to hung
accepting new queries or even have a deadlock, this is because
cancellation is done while acquiring ProcessListBase::mutex.

So this patch makes query cancellation lock-free, and now the lock will
be acquired only for preparing the query and after cancel is done.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-19 17:31:21 +02:00
Amos Bird
652c7cb519
Add comments to FunctionNameNormalizer. 2022-09-19 22:07:04 +08:00
Amos Bird
1a21f0a0f0
normalize ast for replicated metadata 2022-09-19 22:06:41 +08:00
Amos Bird
0bb70d4378
Make minmax_count projection backward compatible 2022-09-19 22:06:40 +08:00
Vitaly Baranov
052ecaa211
Merge branch 'master' into improve-recovery-of-replicated-access-storage 2022-09-19 15:26:21 +02:00
Vitaly Baranov
f75dd93965 Mask some information in logs. 2022-09-19 15:00:06 +02:00
Sergei Trifonov
ff2e1ff8d8
Merge branch 'master' into support_s3_api_event_metrics 2022-09-19 14:35:45 +02:00
Raúl Marín
e89dffbb95 Fix AggregateFunctionNullVariadic batch with null_is_skipped=false 2022-09-19 14:34:08 +02:00
Kruglov Pavel
712aebc077
Merge pull request #41083 from Avogar/fix-map-combinator 
Fix possible segfaults, use-heap-after-free and memory leak in aggregate function combinators
 2022-09-19 14:29:17 +02:00
kssenii
f8665c83da Add retries for nats 2022-09-19 14:27:11 +02:00
kssenii
30a5477514 Add retries for nats 2022-09-19 14:26:38 +02:00
Alexander Tokmakov
c6efb1773e
Merge branch 'master' into fix_drop_partition_on_cluster 2022-09-19 15:18:05 +03:00
Kruglov Pavel
4c3194eefe
Merge pull request #41286 from azat/utf8-fix 
Do not allow invalid sequences influence other rows in lowerUTF8/upperUTF8
 2022-09-19 14:07:10 +02:00
kssenii
e3cd3686af Rename 2022-09-19 14:02:51 +02:00
Sergei Trifonov
f7a1ed3141
Merge branch 'master' into readonly-settings-allow 2022-09-19 12:52:36 +02:00
serxa
a0bfa801e5 explicit ctor 2022-09-19 10:52:08 +00:00
Kseniia Sumarokova
631aad1671
Merge branch 'master' into fix-logical-error-write-through-cache 2022-09-19 12:29:55 +02:00
Antonio Andelic
1763af37b9
Merge pull request #41403 from ClickHouse/keeper-fix-shutdown-without-server 
Fix Keeper segfault during shutdown when RAFT server failed to start
 2022-09-19 12:03:58 +02:00
Igor Nikonov
81a46b6346 Merge remote-tracking branch 'origin/master' into fix_sort_desc_when_read_in_order 2022-09-19 09:40:23 +00:00
Igor Nikonov
54c2fed9cc Update: use type explicitly 2022-09-19 09:31:45 +00:00
Alexey Milovidov
45b6052b20 Merge branch 'master' into compress-marks 2022-09-19 10:57:05 +02:00
vdimir
a6dc99983d
Fix tidy build 2022-09-19 08:42:23 +00:00
vdimir
5c9bfbc116
Fix incorrect result in case of decimal precision loss in IN operator 2022-09-19 08:42:22 +00:00
vdimir
c2634c9fae
Revert "Throw exception on precision lost during decimal to decimal convertion" 
This reverts commit d4233cf3c658adf2541070b1bbba678ad7c769db.
 2022-09-19 08:42:22 +00:00
vdimir
c33e231ce9
Throw exception on precision lost during decimal to decimal convertion 2022-09-19 08:42:18 +00:00
Igor Nikonov
b4a3ac926f Fix typo 2022-09-19 08:26:44 +00:00
Vladimir C
b6b5932efc
Merge pull request #41223 from ClickHouse/vdimir/exponential_rewrite_21557 2022-09-19 10:07:24 +02:00
Alexey Milovidov
b9bd2c8373
Merge pull request #41458 from ClickHouse/fix-bug-orc 
Fix very strange behavior of Apache ORC
 2022-09-19 10:43:17 +03:00
Alexey Milovidov
000b6ac81c Fix error 2022-09-19 09:30:48 +02:00
Duc Canh Le
61d6fae14b add settings allow_suspicious_fixed_string_types 2022-09-19 15:00:04 +08:00
Alexey Milovidov
b87d12494e Fix build 2022-09-19 08:53:20 +02:00
Alexey Milovidov
5224fc4d7f Fix strange code 2022-09-19 08:53:20 +02:00
Alexey Milovidov
d1275091e8 Fix strange code 2022-09-19 08:53:20 +02:00
Alexey Milovidov
6a473d8998 Merge 2022-09-19 08:53:20 +02:00
Alexey Milovidov
bb6f1bfce2 Fix 9/10 of trash 2022-09-19 08:53:20 +02:00
Alexey Milovidov
730655d4fd Fix 8/9 of trash 2022-09-19 08:53:20 +02:00
Alexey Milovidov
42b0d444da Fix 7/8 of trash 2022-09-19 08:53:20 +02:00
Alexey Milovidov
91baedf03a Fix 6/7 of trash 2022-09-19 08:53:20 +02:00
Alexey Milovidov
ab4db2d0c4 Fix 5/6 of trash 2022-09-19 08:50:53 +02:00
Alexey Milovidov
84f42e0874 Fix 3/4 of trash 2022-09-19 08:50:53 +02:00
Amos Bird
d2601b8192
Add a comment 2022-09-19 14:41:34 +08:00
Alexey Milovidov
3c0961c48b
Merge branch 'master' into fix-map-combinator 2022-09-19 09:32:16 +03:00
Alexey Milovidov
b01f5bdca8
Merge pull request #39354 from ClickHouse/window-order-by 
Respect ORDER BY clause in window functions
 2022-09-19 09:22:37 +03:00
Amos Bird
09b75901f0
Fix nullptr deref 2022-09-19 14:07:31 +08:00
Alexey Milovidov
6a09340f11
Merge pull request #41459 from azat/io/threadpoolreader-elapsed 
Account any IO that is done from ThreadPoolReader
 2022-09-19 09:04:20 +03:00
Frank Chen
b056bc1021 Add span for executeDDLQueryOnCluster 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-09-19 13:43:40 +08:00
Alexey Milovidov
81e8cb4be6
Merge branch 'master' into fix-bug-orc 2022-09-19 06:38:17 +03:00
Yong Wang
ec852b3faa Kusto-phase1 : change the parser in ClientBase from shared_ptr to unique_ptr 2022-09-18 20:38:07 -07:00
Alexey Milovidov
d4b9fe41be
Merge pull request #41457 from ClickHouse/remove-trash-5 
Remove trash from Field
 2022-09-19 06:36:48 +03:00
Alexey Milovidov
57a1f47972
Merge pull request #41441 from ClickHouse/pod-array-overflow 
Fix overflow in `PODArray`
 2022-09-19 06:34:21 +03:00
Alexey Milovidov
ac193facde
Merge branch 'master' into add_missed_throw 2022-09-19 06:31:12 +03:00
Alexey Milovidov
df52df83f9
Merge pull request #41463 from ClickHouse/add-test-38383 
Add a test for #38383
 2022-09-19 06:27:38 +03:00
Yong Wang
37ae7a8cca Kusto-phase1 : apply parser comments to kusto, remove unused variable 2022-09-18 20:25:27 -07:00
Frank Chen
06ae2fb2b5 Remove assertEOF to improve the compability 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-09-19 11:20:58 +08:00
Frank Chen
2ae43bb4e8 Add test case 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-09-19 11:18:35 +08:00
Yong Wang
4e4a04c08b
Merge branch 'ClickHouse:master' into Kusto-phase1 2022-09-18 21:40:30 -04:00
Igor Nikonov
785c33bf7d Updating sort description for ReadFromMergeTree correctly 2022-09-18 23:35:21 +00:00
Alexey Milovidov
790cb730d6 Fix clang-tidy 2022-09-19 00:34:41 +02:00
Alexey Milovidov
cc28120761 Merge branch 'master' into pod-array-overflow 2022-09-19 00:34:38 +02:00
Alexey Milovidov
f3fd5ef845 Fix clang-tidy 2022-09-19 00:32:52 +02:00
Alexey Milovidov
36aaeecc2f Fix clang-tidy 2022-09-19 00:32:11 +02:00
Alexey Milovidov
5d9e125c71 Fix clang-tidy 2022-09-19 00:28:32 +02:00
Igor Nikonov
e888859396 Fix: correct sort description for ReadFromMergeTree with read in order 
optimization
 2022-09-18 22:21:13 +00:00
BoloniniD
5b4fb4bf66
Merge branch 'master' into BLAKE3 2022-09-18 23:14:37 +03:00
alesapin
a5a9d89fb9
Merge branch 'master' into enable_projections_and_zero_copy 2022-09-18 17:20:08 +02:00
alesapin
e4c120742e Add missed throw expression 2022-09-18 16:56:19 +02:00
alesapin
560bc2bc22
Merge pull request #41386 from ClickHouse/fix_redundant_fetch 
Fix redundant fetch in zero copy replication
 2022-09-18 15:57:27 +02:00
alesapin
1d0dbfb112
Merge pull request #41444 from ClickHouse/one_wrong_config_assert 
One more validation for broken config in keeper
 2022-09-18 15:56:48 +02:00
Kruglov Pavel
519bcbb3be
Merge pull request #41236 from Avogar/fix-totals-extremes 
Don't output totals/extremes in all row formats, update docs
 2022-09-18 14:55:07 +02:00
Kseniia Sumarokova
d6748b182f
Update CachedOnDiskWriteBufferFromFile.cpp 2022-09-18 14:48:24 +02:00
Vitaly Baranov
e7e51ab2d9 Add comments. 2022-09-18 12:44:05 +02:00
Vitaly Baranov
69996c960c Add retries for the initialization of ReplicatedAccessStorage. 2022-09-18 12:44:05 +02:00
Vitaly Baranov
5365b105cc Add SYSTEM RELOAD USERS command. 2022-09-18 12:44:00 +02:00
Vitaly Baranov
1b40f94c6b Move common code from MemoryAccessStorage and DiskAccessStorage to IAccessStorage. 2022-09-18 12:17:15 +02:00
Alexey Milovidov
09ae1a4bf4
Merge branch 'master' into hlee-s390x-siphash 2022-09-18 11:09:02 +03:00
Azat Khuzhin
64b44be513 Account any IO that is done from ThreadPoolReader 
In case of some issues with the disk it is possible that
preadv2(RWF_NOWAIT)==EAGAIN can take a lot of time, so let's measure
even this case.

I've verified with a simple (almost) loop of preadv2(RWF_NOWAIT), and
got this:

    EAGAIN: 8565670 (out of 17148317)
    EAGAIN: 8565670 (out of 17148317)
    EAGAIN: 8565670 (out of 17148317)
    too long at offset=8691694, ret=18446744073709551615 (14.5016)
    too long at offset=8694766, ret=18446744073709551615 (14.4338)
    too long at offset=8694766, ret=18446744073709551615 (16.2144)

So here you can see that preadv2(RWF_NOWAIT) took 14-16 seconds in case
of failure.

Complete code can be found here [1].

  [1]: https://gist.github.com/azat/f4a6b0750316c42bec021064e0a7627c

I came up with this patch because I saw some stranginess in
ProfileEvents, that does not show all the time that was spend in IO when
pread_threadpool method was used.
Also there was some stranginess with trace_log too.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-18 10:05:21 +02:00
Alexey Milovidov
7e6da853cd Merge branch 'master' into pod-array-overflow 2022-09-18 09:24:23 +02:00
Alexey Milovidov
a6e3ab5515 Fix error 2022-09-18 09:21:57 +02:00
Alexey Milovidov
5c75a7d661 Fix error 2022-09-18 09:20:48 +02:00
Alexey Milovidov
c41e37fa8e Code simplification 2022-09-18 08:58:32 +02:00
Alexey Milovidov
3c43d6516b Fix style 2022-09-18 08:56:52 +02:00
Alexey Milovidov
bafc5f24c7 Merge branch 'master' into remove-trash-5 2022-09-18 08:56:02 +02:00
Alexey Milovidov
8764fa4439 Fix very strange behavior of Apache ORC 2022-09-18 08:25:25 +02:00
Alexey Milovidov
a958751c98
Merge pull request #40219 from den-crane/test/zvonand-monotonic 
test for datetime64 monotonic #39864
 2022-09-18 07:48:49 +03:00
Alexey Milovidov
6b752bff3a
Merge branch 'master' into revert_prefixes_s3 2022-09-18 07:10:57 +03:00
Alexey Milovidov
32efe6952e
Merge pull request #41429 from ClickHouse/forbid-dlopen-2 
Forbid invocations of `dlopen`
 2022-09-18 07:05:56 +03:00
Alexey Milovidov
791de6592b Remove trash from Field 2022-09-18 05:16:08 +02:00
HarryLeeIBM
f3e8738145 Fixed issues in code review 2022-09-17 19:48:08 -07:00
Alexey Milovidov
c8c728e740
Merge pull request #41449 from ClickHouse/fix-categorial-information 
Fix incorrect properties of `categorialInformationValue` function
 2022-09-18 05:26:26 +03:00
Alexey Milovidov
7f68f2c7df Remove trash from LowCardinality 2022-09-18 04:13:21 +02:00
Alexey Milovidov
15e5fc96a1 Add a trap 2022-09-18 02:24:56 +02:00
Alexey Milovidov
dc76677eb2
Merge pull request #41440 from ClickHouse/remove-cruft 
Remove cruft about gcc
 2022-09-18 03:15:26 +03:00
Alexey Milovidov
e3dafdb865 More hardening 2022-09-18 02:10:06 +02:00
Alexey Milovidov
e5e3bca3ae More hardening and tests 2022-09-18 02:05:13 +02:00
Alexey Milovidov
6c74c54cb5 Merge branch 'master' into pod-array-overflow 2022-09-18 01:24:52 +02:00
Alexey Milovidov
6f1878b12a Fix error 2022-09-18 01:17:22 +02:00
Alexey Milovidov
c424ad12aa Simplification 2022-09-18 00:09:21 +02:00
alesapin
b6d61318d0 One more validation for broken config in keeper 2022-09-18 00:04:16 +02:00
Alexey Milovidov
e2d434ea18 Start with code removal 2022-09-17 23:37:57 +02:00
Alexey Milovidov
2f0684b97c Fix trash in schema inference 2022-09-17 23:11:33 +02:00
Alexey Milovidov
bac578b23a
Merge pull request #41428 from ClickHouse/remove-dlopen 
Remove `dlopen`
 2022-09-18 00:09:57 +03:00
Alexey Milovidov
e3d75b0933 Merge branch 'fix-trash-schema-inference' into pod-array-overflow 2022-09-17 22:56:06 +02:00
Alexey Milovidov
47167494d9 Fix trash in schema inference 2022-09-17 22:53:41 +02:00
Alexey Milovidov
9ca4e2ec7c Fix overflow in PODArray 2022-09-17 22:26:39 +02:00
Alexey Milovidov
f4a48cd4d6 Remove cruft 2022-09-17 22:16:31 +02:00
BoloniniD
627dea333e Merge branch 'master' into BLAKE3 2022-09-17 22:14:30 +03:00
kssenii
51745dcb43 Fix assertion in write-through cache 2022-09-17 20:20:32 +02:00
Alexander Tokmakov
49a9c9e5fa
Merge pull request #41436 from ClickHouse/revert_38200 
Revert "Keeper faster linearizable reads"
 2022-09-17 20:25:43 +03:00
Robert Schulze
38aecc15a2
Merge pull request #41214 from arenadata/ADQM-528-B 
Add enable_extended_results_for_datetime_functions option to return results of type Date32
 2022-09-17 19:25:11 +02:00
Alexander Tokmakov
42dab6dc02 Revert "Merge pull request #38200 from ClickHouse/keeper-linearizable-reads" 
This reverts commit 222b964070, reversing
changes made to da01982652.
 2022-09-17 19:22:58 +02:00
alesapin
33216dedf9
Merge pull request #41323 from ClickHouse/remove_bad_exception 
Fix parts removal in case of connection loss
 2022-09-17 19:14:53 +02:00
alesapin
dcb1586977 Add comment 2022-09-17 19:11:58 +02:00
alesapin
db30a3041e
Update src/Storages/MergeTree/MutateFromLogEntryTask.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2022-09-17 19:10:23 +02:00
alesapin
84828874e0 Change log level 2022-09-17 19:06:54 +02:00
Anton Popov
07cfb0cba4
Merge branch 'master' into dynamic-columns-21 2022-09-17 19:02:58 +02:00
Alexey Milovidov
9ce0c5f738
Merge pull request #41431 from ClickHouse/remove-method-simplification 
Remove `-WithTerminatingZero` methods
 2022-09-17 19:48:35 +03:00
BoloniniD
1c0047dbff Review fixes 2022-09-17 16:34:19 +03:00
Azat Khuzhin
01b5b9ceba Do not allow invalid sequences influence other rows in lowerUTF8/upperUTF8 
Right now lowerUTF8() and upperUTF8() does not respect row boundaries,
and so one row may break another.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-17 11:16:45 +02:00
Igor Nikonov
475443c10c
Merge pull request #41257 from ClickHouse/igor/mixed_improvements_along_the_way 
Tiny improvements along the code
 2022-09-17 10:43:46 +02:00
Alexey Milovidov
ada7a44ae4 Remove -WithTerminatingZero methods 2022-09-17 05:34:18 +02:00
Alexey Milovidov
0c3835240b Fix error with multidimensional arrays 2022-09-17 04:16:24 +02:00
Alexey Milovidov
b60a5a5bc5 Merge branch 'master' into fix-getdataat-multidimensional-arrays 2022-09-17 04:07:12 +02:00
Alexey Milovidov
35cce03125 Remove dlopen 2022-09-17 03:02:34 +02:00
Alexey Milovidov
31ebd34f5b
Merge pull request #41193 from azat/DWARF-5-parser-v3 
Add support for DWARF-5 (without emitting them in binaries)
 2022-09-17 02:58:02 +03:00
Alexey Milovidov
c09604bef2
Merge pull request #41363 from kitaisreal/small-style-fixes 
Small style fixes
 2022-09-17 00:19:00 +03:00
Denny Crane
6540d55d42
Merge branch 'master' into test/zvonand-monotonic 2022-09-16 17:02:09 -03:00
Nikita Taranov
6f186d3dd2
Do not return empty blocks from ConvertingAggregatedToChunksTransform (#41152) 
* impl

* add test

* update test
 2022-09-16 21:54:36 +02:00
avogar
fa172e21ae Remove unnecessary method implementation 2022-09-16 17:47:05 +00:00
Igor Nikonov
24de5530d6 Remove unnecessary headers 2022-09-16 17:16:32 +00:00
Igor Nikonov
b34ec6453b Fix review comments 
+ one more test
 2022-09-16 17:11:08 +00:00
BoloniniD
452ef4435b Merge branch 'master' of github.com:ClickHouse/ClickHouse into BLAKE3 2022-09-16 20:05:56 +03:00
Vitaly Baranov
391507b4b6 Update src/Access/DiskAccessStorage.cpp 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2022-09-16 19:02:06 +02:00
Vitaly Baranov
2ec6ef497d Make DiskAccessStorage::insertWithID() more consistent with MemoryAccessStorage::insertWithID(). 2022-09-16 19:02:06 +02:00
Vitaly Baranov
7e716d14cd Make ReplicatedAccessStorage::insertWithID() more consistent with MemoryAccessStorage::insertWithID(). 2022-09-16 19:02:06 +02:00
Vitaly Baranov
646cd55690 Improve recovery of ReplicatedAccessStorage after errors. 2022-09-16 19:02:06 +02:00
Vitaly Baranov
8f9f5c69da Use MemoryAccessStorage to simplify ReplicatedAccessStorage. 2022-09-16 19:02:06 +02:00
Dmitry Novik
eacc790050
Merge branch 'master' into fix-odr-vialation 2022-09-16 18:44:53 +02:00
avogar
c9a9ef5b7e Fix comments 2022-09-16 16:12:30 +00:00
Alexander Tokmakov
65474830ac make tryEnqueueReplicatedDDL virtual 2022-09-16 16:25:32 +02:00
Nikolai Kochetov
16f78eb804
Merge pull request #41281 from ClickHouse/fix-key-condition-with-actions-dag-literal-name 
Fix wrong literal name in KeyCondition using ActionsDAG.
 2022-09-16 16:06:38 +02:00
Kruglov Pavel
2d4a6b38af
Merge branch 'master' into fix-totals-extremes 2022-09-16 15:03:59 +02:00
alesapin
d9c3549cc8 Revert s3 prefixes + add retries on write requests unexpected errors 2022-09-16 14:43:43 +02:00
alesapin
456da30c9f Revert "Merge pull request #41249 from ClickHouse/revert-40968-s3-sharding-2" 
This reverts commit ac60300997, reversing
changes made to a9c272283a.
 2022-09-16 14:06:26 +02:00
Antonio Andelic
81f7cf3ca6 Add support for MultiRead in Keeper 2022-09-16 12:00:35 +00:00
alesapin
b4d2d217f4
Merge branch 'master' into enable_projections_and_zero_copy 2022-09-16 13:53:17 +02:00
alesapin
06ccdb1ecf Fix nasty bug 2022-09-16 13:49:39 +02:00
Alexander Tokmakov
9597a47016
Merge pull request #41379 from ClickHouse/tavplubix-patch-5 
Fix an assertion in `wasInvolvedInTransaction`
 2022-09-16 14:24:26 +03:00
Robert Schulze
b32b02d844
Merge pull request #40897 from ClickHouse/catboost-bridge-resurrected 
Move CatBoost evaluation into clickhouse-library-bridge
 2022-09-16 13:12:09 +02:00
Anton Popov
5584ef71f6
Merge pull request #41388 from den-crane/bug/optimize_rewrite_sum_if_to_count_if 
Disable optimize_rewrite_sum_if_to_count_if by default
 2022-09-16 13:05:23 +02:00
Igor Nikonov
eeecaf7a31 Merge remote-tracking branch 'origin/master' into distinct_in_order_wo_order_by 2022-09-16 10:30:52 +00:00
Igor Nikonov
ba1f7de243 Fix review comments 2022-09-16 10:30:30 +00:00
Antonio Andelic
acd9cb089e Fix segfault during shutdown when RAFT server failed to start 2022-09-16 10:03:21 +00:00
Antonio Andelic
5662e0050c Define client for multi read 2022-09-16 09:55:02 +00:00
Roman Vasin
bc97da686a Add const for bool enable_extended_results_for_datetime_functions 2022-09-16 09:54:24 +00:00
alesapin
af259a2a68
Update src/Storages/MergeTree/MergeFromLogEntryTask.cpp 
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2022-09-16 11:50:07 +02:00
Roman Vasin
d82cd7b007 Add many cosmetic changes to C++ code and MD docs 2022-09-16 09:37:30 +00:00
Nikolay Degterinsky
7dac605829 Merge remote-tracking branch 'master' into new_parser 2022-09-16 08:56:30 +00:00
Nikolay Degterinsky
db85f97a3f Fix 2022-09-16 08:55:58 +00:00
Roman Vasin
7e733887b9 Add narrowing for toStartOfISOYear; improve 02403_enable_extended_results_for_datetime_functions 2022-09-16 08:40:15 +00:00
Alexey Milovidov
701fdd77b4 Fix wrong implementation of getDataAt for multidimensional arrays 2022-09-16 08:12:28 +02:00
Antonio Andelic
222b964070
Merge pull request #38200 from ClickHouse/keeper-linearizable-reads 
Keeper faster linearizable reads
 2022-09-16 07:58:52 +02:00
Alexey Milovidov
da01982652
Merge pull request #41046 from azat/build/llvm-15 
Switch to llvm/clang 15
 2022-09-16 07:31:06 +03:00
Nikolay Degterinsky
f63b2d4b9f Revert some breaking changes 2022-09-16 03:29:34 +00:00
Nikolay Degterinsky
36c3a18035 Fix tests 2022-09-16 02:38:14 +00:00
Nikolay Degterinsky
2e2d804b90 Better 2022-09-15 21:54:37 +00:00
Raúl Marín
c3ff66bd9d
Implement batch processing for aggregate functions with multiple nullable arguments (#41058) 
* Implement batch processing for aggregate functions with multiple nullable arguments

* Fix broken perf test

* Improve filter handling in addBatchSinglePlace with nullable arguments

* Fix detecting the Null filter usage
 2022-09-15 23:51:38 +02:00
Raúl Marín
6dac509739
Speed up reading uniqState (#41089) 
* Speed up reading UniquesHashSet

* Improve uniq serialization tests
 2022-09-15 23:41:15 +02:00
Igor Nikonov
8f500019eb Merge remote-tracking branch 'origin/master' into distinct_in_order_wo_order_by 2022-09-15 21:05:59 +00:00
Alexander Tokmakov
8301f5758b Update IMergeTreeDataPart.cpp 2022-09-15 21:32:57 +02:00
Alexander Tokmakov
03c193ccca fix ON CLUSTER with Replicated database cluster 2022-09-15 21:15:57 +02:00
Denny Crane
7cfdfbe787 disable optimize_rewrite_sum_if_to_count_if by default 2022-09-15 15:30:24 -03:00
Alexander Tokmakov
32391f737a Merge branch 'master' into fix_41209 2022-09-15 20:27:51 +02:00
avogar
ef7f7b30aa Fix tests 2022-09-15 18:20:17 +00:00
avogar
92979af3bf Add missing methods in combinators 2022-09-15 17:45:50 +00:00
alesapin
82611d891b
Merge branch 'master' into fix-odr-vialation 2022-09-15 19:33:09 +02:00
alesapin
be81d653e1 Fix comment 2022-09-15 18:56:16 +02:00
alesapin
b098c1be36 Fix data duplication in zero copy replication 2022-09-15 18:54:27 +02:00
Antonio Andelic
fe78c063f6
Merge branch 'master' into keeper-linearizable-reads 2022-09-15 18:40:23 +02:00
Antonio Andelic
39307f9dba Extract into function 2022-09-15 16:14:53 +00:00
Antonio Andelic
0dbd0c1f72 Merge branch 'master' into keeper-linearizable-reads 2022-09-15 16:03:18 +00:00
Anton Popov
b5e9884355 fix subcolumns tree 2022-09-15 14:17:10 +00:00
Roman Vasin
5c242de43a Merge branch 'master' of github.com:ClickHouse/ClickHouse into ADQM-528-B 2022-09-15 13:47:51 +00:00
alesapin
7570c5cf67 Remove recursive without blobs 2022-09-15 15:31:19 +02:00
HarryLeeIBM
d62ba01e93 Fix SipHash Endian issue for s390x 2022-09-15 06:25:23 -07:00
alesapin
c585ed47fc Forget to remove 2022-09-15 15:15:28 +02:00
serxa
b3b8123f73 fix style 2022-09-15 12:53:56 +00:00
alesapin
e127c8b4a9 Add kringe 2022-09-15 14:49:31 +02:00
alesapin
c04f7afb2a Delete kringe 2022-09-15 14:48:59 +02:00
Roman Vasin
563449779c Replace enable_date32_results by enable_extended_results_for_datetime_functions 2022-09-15 12:48:09 +00:00
Alexander Tokmakov
b26cc5a1ba
Merge pull request #41296 from young-scott/fix_issuse_41096 
fix issue #41096
 2022-09-15 15:44:05 +03:00
avogar
6bb6c9e916 Fix comment 2022-09-15 12:41:39 +00:00
avogar
d41bb7c2a4 Fix comment 2022-09-15 12:40:32 +00:00
Alexander Tokmakov
77c4ea9647 fix 2022-09-15 14:34:31 +02:00
Denny Crane
1b8b2ebed5
Disable optimize_monotonous_functions_in_order_by by default (#41136) 
* #40094 disable optimize_monotonous_functions_in_order_by by default

* fix 01576_alias_column_rewrite test

* fix incorrect 02149_read_in_order_fixed_prefix.sql test

* Update 02149_read_in_order_fixed_prefix.sql
 2022-09-15 13:41:46 +02:00
Azat Khuzhin
5cc5ca22de Fix possible deadlock with async_socket_for_remote/use_hedged_requests and parallel KILL 
Right now it is possible to call QueryStatus::addPipelineExecutor() when
the executors_mutex already acquired, it is possible when the query was
cancelled via KILL QUERY.

Here I will show some traces from debugger from a real example, where
tons of ProcessList::insert() got deadlocked.

Let's look at the lock owner for one of the threads that was deadlocked
in ProcessList::insert():

    (gdb) p *mutex
    $2 = {
      __data = {
        __owner = 46899,
      },
    }

And now let's see the stack trace of the 46899:

    #0  __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:103
    #1  0x00007fb65569b714 in __GI___pthread_mutex_lock (mutex=0x7fb4a9d15298) at ../nptl/pthread_mutex_lock.c:80
    #2  0x000000001b6edd91 in pthread_mutex_lock (arg=0x7fb4a9d15298) at ../src/Common/ThreadFuzzer.cpp:317
    #3  std::__1::__libcpp_mutex_lock (__m=0x7fb4a9d15298) at ../contrib/libcxx/include/__threading_support:303
    #4  std::__1::mutex::lock (this=0x7fb4a9d15298) at ../contrib/libcxx/src/mutex.cpp:33
    #5  0x0000000014c7ae63 in std::__1::lock_guard<std::__1::mutex>::lock_guard (__m=..., this=<optimized out>) at ../contrib/libcxx/include/__mutex_base:91
    #6  DB::QueryStatus::addPipelineExecutor (this=0x7fb4a9d14f90, e=0x80) at ../src/Interpreters/ProcessList.cpp:372
    #7  0x0000000015bee4a7 in DB::PipelineExecutor::PipelineExecutor (this=0x7fb4b1e53618, processors=..., elem=<optimized out>) at ../src/Processors/Executors/PipelineExecutor.cpp:54
    #12 std::__1::make_shared<DB::PipelineExecutor, std::__1::vector<std::__1::shared_ptr<DB::IProcessor>, std::__1::allocator<std::__1::shared_ptr<DB::IProcessor> > >&, DB::QueryStatus*&, void> (__args=@0x7fb63095b9b0: 0x7fb4a9d14f90, __args=@0x7fb63095b9b0: 0x7fb4a9d14f90) at ../contrib/libcxx/include/__memory/shared_ptr.h:963
    #13 DB::QueryPipelineBuilder::execute (this=0x7fb63095b8b0) at ../src/QueryPipeline/QueryPipelineBuilder.cpp:552
    #14 0x00000000158c6c27 in DB::Connection::sendExternalTablesData (this=0x7fb6545e9d98, data=...) at ../src/Client/Connection.cpp:797
    #27 0x0000000014043a81 in DB::RemoteQueryExecutorRoutine::operator() (this=0x7fb63095bf20, sink=...) at ../src/QueryPipeline/RemoteQueryExecutorReadContext.cpp:46
    #32 0x000000000a16dd4f in make_fcontext () at ../contrib/boost/libs/context/src/asm/make_x86_64_sysv_elf_gas.S:71

And also in the logs you can see very strange things for this thread:

    2022.09.13 14:14:51.228979 [ 51145 ] {1712D4E914EC7C99} <Debug> Connection (localhost:9000): Sent data for 1 external tables, total 11 rows in 0.00046389 sec., 23688 rows/sec., 3.84 KiB (8.07 MiB/sec.), compressed 1.1070121092649958 times to 3.47 KiB (7.29 MiB/sec.)
    ...
    2022.09.13 14:14:51.719402 [ 46899 ] {7c90ffa4-1dc8-42fd-938c-4e307c244394} <Debug> executeQuery: (from 10.101.15.181:42478) KILL QUERY WHERE query_id = '1712D4E914EC7C99' (stage: Complete)
    2022.09.13 14:14:51.719488 [ 46899 ] {7c90ffa4-1dc8-42fd-938c-4e307c244394} <Debug> executeQuery: (internal) SELECT query_id, user, query FROM system.processes WHERE query_id = '1712D4E914EC7C99' (stage: Complete)
    2022.09.13 14:14:51.719754 [ 46899 ] {7c90ffa4-1dc8-42fd-938c-4e307c244394} <Trace> ContextAccess (default): Access granted: SELECT(user, query_id, query) ON system.processes
    2022.09.13 14:14:51.720544 [ 46899 ] {7c90ffa4-1dc8-42fd-938c-4e307c244394} <Trace> InterpreterSelectQuery: FetchColumns -> Complete
    2022.09.13 14:14:53.228964 [ 46899 ] {7c90ffa4-1dc8-42fd-938c-4e307c244394} <Debug> Connection (localhost:9000): Sent data for 2 scalars, total 2 rows in 2.6838e-05 sec., 73461 rows/sec., 68.00 B (2.38 MiB/sec.), compressed 0.4594594594594595 times to 148.00 B (5.16 MiB/sec.)

How is this possible? The answer is fibers and query cancellation
routine. During cancellation of async queries it going into fibers again
and try to do this gracefully. However because of this during canceling
query it may call QueryStatus::addPipelineExecutor() from
QueryStatus::cancelQuery().

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-15 14:41:27 +03:00
serxa
4971c1f04d review fixes 2022-09-15 11:34:14 +00:00
Maksim Kita
276d3628e9 Small style fixes 2022-09-15 13:05:18 +02:00
Roman Vasin
3346d8e27b Add many cosmetic changes to C++ code 2022-09-15 10:56:05 +00:00
Igor Nikonov
da75d28019 Merge remote-tracking branch 'origin/master' into distinct_in_order_wo_order_by 2022-09-15 10:54:42 +00:00
Frank Chen
f576d8191c Merge branch 'master' into on_cluster_dll 2022-09-15 18:05:36 +08:00
Kseniia Sumarokova
9e37a6356f
Merge pull request #41297 from kssenii/fix-logical-error 
Fix incorrect logical error in disk object storage
 2022-09-15 12:00:53 +02:00
Kseniia Sumarokova
69f38a6228
Merge pull request #41330 from kssenii/fix-composable-cache-setting-incompatibility 
Fix incompatibility of cache after switching setting `do_no_evict_...` from 1 to 0, 0 to 1
 2022-09-15 12:00:22 +02:00
Frank Chen
ac848727e5 Read tracing context in right pos 2022-09-15 18:00:08 +08:00
Kruglov Pavel
73cf72a5a4
Merge pull request #41309 from Avogar/fix-msgpack 
Add column type check before UUID insertion in MsgPack format
 2022-09-15 11:37:57 +02:00
Kruglov Pavel
982b9baeec
Fix 2022-09-15 11:32:14 +02:00
Nikolay Degterinsky
72d8a04ee6 Add setting 'allow_implicit_no_password' 2022-09-15 08:07:42 +00:00
Robert Schulze
b64af5989e
Merge branch 'master' into opentelemetry_stress 2022-09-15 09:43:11 +02:00
Frank Chen
a4ef0c0281 Set up tracing context for DDLWorker 2022-09-15 14:57:00 +08:00
Frank Chen
490089cc4f Copy tracing context from current thread to DDLLogEntry as parent context 2022-09-15 14:48:24 +08:00
Frank Chen
52224875e2 Serialize tracing context to DDL log entry 2022-09-15 14:47:43 +08:00
Frank Chen
77d0971efa Serialize tracing context 2022-09-15 14:46:55 +08:00
Anton Popov
66667737f0 fix writing of empty columns of type Object 2022-09-15 03:38:22 +00:00
JackyWoo
afcbf3a853
Merge branch 'master' into add_function_java_int_hash 2022-09-15 10:08:26 +08:00
Nikolay Degterinsky
6d01f92641 Fix silly bug & little enhancement 2022-09-15 00:48:14 +00:00
Kseniia Sumarokova
780c0e9b2f
Update CachedOnDiskReadBufferFromFile.cpp 2022-09-15 00:18:23 +02:00
Kseniia Sumarokova
ebf7cb6338
Merge pull request #41308 from ClickHouse/kssenii-patch-5 
Fix after #41042
 2022-09-15 00:15:54 +02:00
Igor Nikonov
f011f72114 Try to apply optimization only if optimizer reaches pre distinct node 
+ more comments
 2022-09-14 20:55:49 +00:00
Kseniia Sumarokova
0b1c2c62fd
Update CachedOnDiskReadBufferFromFile.cpp 2022-09-14 20:59:45 +02:00
kssenii
559c696230 Fix 2022-09-14 20:35:28 +02:00
avogar
f1a0501eb2 Fix memory leaks and segfaults in combinators 2022-09-14 18:01:49 +00:00
Igor Nikonov
bab5744a1d Merge remote-tracking branch 'origin/master' into distinct_in_order_wo_order_by 2022-09-14 17:54:46 +00:00
Igor Nikonov
d91ea0c5ab Fix + tests 2022-09-14 17:54:05 +00:00
Alexander Tokmakov
173df9a73a
Update StorageSystemDDLWorkerQueue.cpp 2022-09-14 20:38:50 +03:00
Alexander Tokmakov
fc73134145 fix race between DROP and short form of ATTACH 2022-09-14 18:03:36 +02:00
alesapin
63213465ef Fix comment 2022-09-14 17:28:12 +02:00
alesapin
ad279940d6 Trying print less scary messages in case No such key for outdated parts 2022-09-14 17:25:58 +02:00
Robert Schulze
fd97058e45
fix: incorporate review comments 2022-09-14 15:21:24 +00:00
alesapin
d8c14ebcf6 Fixup 2022-09-14 16:44:40 +02:00
alesapin
9d3ab9a7f7 Merge branch 'master' into enable_projections_and_zero_copy 2022-09-14 15:16:16 +02:00
JackyWoo
9578594439
Merge branch 'master' into add_function_java_int_hash 2022-09-14 20:42:41 +08:00
Anton Popov
3623807527
Merge pull request #41290 from CurtizJ/dynamic-columns-20 
Fix `ALTER ADD COLUMN`  with columns of type `Object`
 2022-09-14 14:15:29 +02:00
Anton Popov
defd393fdc
Update src/DataTypes/ObjectUtils.h 
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
 2022-09-14 14:07:07 +02:00
mateng0915
31a0900756 deleted local file 2022-09-14 19:19:47 +08:00
mateng0915
f7f976e94e remove extra space 2022-09-14 19:19:47 +08:00
mateng0915
f38e262706 fixed the unnecessory changes 2022-09-14 19:19:47 +08:00
mateng0915
5badb1b186 resolve the review comments 2022-09-14 19:19:40 +08:00
teng.ma
a85c917488 fix master conflict 2022-09-14 19:18:31 +08:00
teng.ma
ebea9f9359 Feature: Support Event Tracing when calling Aws S3 API 
Description:
============
In  Computing && Storage Architecture,  using S3 as remote / Shared storage, the Method to access S3 is using AWS S3 API
There is a gap between ClickHouse DB with Ozone Operation
In ClickHouse, operation is SQL and background task
In S3 , the operation should be AWS S3 API
And one sql maybe can mapped to multiple  API

Solution:
=========
Added Calling API as event into system.events table
 2022-09-14 19:18:31 +08:00
avogar
59e7eb084c Add column type check before UUID insertion in MsgPack format 2022-09-14 11:15:10 +00:00
Kseniia Sumarokova
cc639f2e2d
Update CachedOnDiskReadBufferFromFile.cpp 2022-09-14 13:13:02 +02:00
Nikolay Degterinsky
e9c4968211 Merge remote-tracking branch 'master' into new_parser 2022-09-14 11:08:48 +00:00
Kruglov Pavel
3396ff6c3a
Merge pull request #40516 from zjial/record_errors_for_import_by_csv 
Record errors while reading text formats (CSV, TSV).
 2022-09-14 12:52:32 +02:00
alesapin
51a302a70f
Merge branch 'master' into revert-40968-s3-sharding-2 2022-09-14 11:53:14 +02:00
Kseniia Sumarokova
52db0e5c40
Update DiskObjectStorageMetadata.cpp 2022-09-14 11:52:23 +02:00
Nikolay Degterinsky
ab4e3b831e Fix TableFunction expressions 2022-09-14 09:44:51 +00:00
young scott
b6888022cd
Merge branch 'master' into fix_issuse_41096 2022-09-14 17:38:22 +08:00
kssenii
ea31302071 Fix 2022-09-14 11:30:06 +02:00
young scott
bc111b5680 fix issuse#41096 2022-09-14 09:15:40 +00:00
Vladimir C
00d0b37cd0
Merge pull request #41263 from ClickHouse/vdimir/quorum-log-msg-fix 2022-09-14 10:54:21 +02:00
JackyWoo
2c8e5f0156 Merge branch 'master' into add_function_java_int_hash 2022-09-14 10:11:15 +08:00
Anton Popov
ee54451741 fix adding of columns of type Object 2022-09-13 22:43:59 +00:00
Nikolay Degterinsky
48ec96149e Fix style 2022-09-13 21:07:15 +00:00
Nikolay Degterinsky
d42a751c1f Fix table function view & more cool stuff 2022-09-13 20:31:08 +00:00
Alexander Tokmakov
cf7d928554
Update PredicateRewriteVisitor.cpp 2022-09-13 22:38:55 +03:00
Robert Schulze
e8e6dddc76
Merge pull request #41034 from FrankChen021/distributed 
Improve the observability of INSERT on distributed table
 2022-09-13 20:55:11 +02:00
Nikolai Kochetov
3f0bc94d25 Comment. 2022-09-13 18:50:01 +00:00
Nikolai Kochetov
66d9d92de4 Fix wrong literal name in KeyCondition using ActionsDAG. 2022-09-13 18:47:29 +00:00
Alexander Tokmakov
0175f3d0d7 fix WHERE pushdown when subquery has OFFSET 2022-09-13 20:44:45 +02:00
Igor Nikonov
c1dc77cd50 Merge remote-tracking branch 'origin/master' into distinct_in_order_wo_order_by 2022-09-13 18:24:48 +00:00
Igor Nikonov
1be5a79e9f Exclude const columns in distinct when matching with sorting key 
+ tests
 2022-09-13 18:23:55 +00:00
Igor Nikonov
c04d46bc96 Update data stream properties after reading in order applied 
+ tests
 2022-09-13 17:50:43 +00:00
Vladimir C
f0e4088c84
Merge pull request #41154 from azat/fix-sorted-block-writer 2022-09-13 19:14:45 +02:00
Antonio Andelic
36f46891dd
Merge pull request #41215 from ClickHouse/keeper-close-connection-on-shutdown 
Close sessions on Keeper shutdown
 2022-09-13 19:09:34 +02:00
alesapin
2cecf9e55f Fix bug 2022-09-13 18:30:55 +02:00
Dmitry Novik
7e3fb0a681
Merge pull request #40762 from ClickHouse/grouping-comp 
Fix GROUPING function SQL compatibility
 2022-09-13 17:03:06 +02:00
Dmitry Novik
990ce888bf
Merge branch 'master' into fix-odr-vialation 2022-09-13 16:53:57 +02:00
kssenii
c4c044c758 Merge remote-tracking branch 'origin/fix-thread-status' into fix-thread-status 2022-09-13 16:35:08 +02:00
kssenii
52ef3758c4 Merge remote-tracking branch 'upstream/master' into fix-thread-status 2022-09-13 16:34:31 +02:00
Kseniia Sumarokova
2cd6ceb203
Update StorageS3.h 2022-09-13 16:30:28 +02:00
vdimir
f2acb53887
Better log message for replicas number in StorageReplicatedMerge 2022-09-13 13:49:51 +00:00
JackyWoo
db6dcd6ce9 delete a line 2022-09-13 21:28:24 +08:00
JackyWoo
e3caa6f096 some fixes 2022-09-13 21:21:35 +08:00
kssenii
420ac4eb43 s3 header auth in ast 2022-09-13 15:13:28 +02:00
alesapin
7a1ae1f316 Fix for stateless test 2022-09-13 15:10:49 +02:00
Kruglov Pavel
110be0688e
Merge pull request #40909 from ClickHouse/Avogar-patch-1 
Make better exception message in schema inference
 2022-09-13 14:44:29 +02:00
Kruglov Pavel
faa62f96ae
Merge branch 'master' into new-json-formats 2022-09-13 14:41:13 +02:00
Kruglov Pavel
17621b5607
Merge branch 'master' into fix-totals-extremes 2022-09-13 14:36:31 +02:00
Igor Nikonov
f99f57aed8 Remove unrelated changes 2022-09-13 12:24:46 +00:00
Antonio Andelic
baf7255cff
Merge pull request #39976 from ClickHouse/keeper-storage 
KeeperMap storage engine
 2022-09-13 14:15:09 +02:00
Anton Popov
1989380485
Merge pull request #41122 from lthaooo/optimize_compact_part_mutation 
optimize compact part mutation
 2022-09-13 14:10:21 +02:00
Igor Nikonov
d8acb0e7ad Tiny improvements along the code 
- less allocations (potentially)
 2022-09-13 12:06:56 +00:00
zhongyuankai
f36d70e9dd
Merge branch 'master' into compress_marks_and_primary_key 2022-09-13 20:05:39 +08:00
Alexander Tokmakov
4d146b05a9
Merge pull request #38262 from PolyProgrammist/fix-ordinary-system-unfreeze 
Fix SYSTEM UNFREEZE for ordinary database
 2022-09-13 14:55:11 +03:00
Alexander Tokmakov
2845494624
Merge pull request #41120 from azat/async-insert-queue-term 
Terminate AsynchronousInsertQueue w/o waiting for timeout
 2022-09-13 14:52:50 +03:00
Alexander Tokmakov
54001d3871
Revert "Fix uncaught DNS_ERROR on failed connection to replicas" 2022-09-13 13:51:09 +03:00
Vladimir C
cc7292df74
Merge pull request #41105 from ClickHouse/vdimir/upd_storage_rdb_sample_block 2022-09-13 12:37:12 +02:00
alesapin
cc9f6d47be Merge branch 'master' into enable_projections_and_zero_copy 2022-09-13 12:34:22 +02:00
Kseniia Sumarokova
b2c9c04c7b
Merge pull request #40821 from kssenii/improve-marks-cache-loading 
Allow to load marks in threadpool in advance
 2022-09-13 12:31:58 +02:00
alesapin
9af591a328
Revert "Sharding s3 key names (2)" 2022-09-13 12:29:02 +02:00
Frank Chen
d3265150c0 Make sure span is finished in the onFinish callback 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-09-13 18:28:21 +08:00
Frank Chen
7303ae1796 Make sure span holder will be destructed only once 2022-09-13 18:27:29 +08:00
Kseniia Sumarokova
3dd28bb79a
Merge pull request #41042 from kssenii/refactor-cache-for-async-download 
Cache refactoring for async download
 2022-09-13 12:26:35 +02:00
Kseniia Sumarokova
5a76440078
Merge pull request #41232 from kssenii/fix-logical-error-in-write-through-cache 
Fix logical error in write-through cache found by stress test
 2022-09-13 12:25:21 +02:00
Kseniia Sumarokova
651d850652
Merge pull request #41230 from kssenii/minor-change 
minor change in MergeTreeDataPartCompact
 2022-09-13 12:23:18 +02:00
Kseniia Sumarokova
6655f1933a
Merge pull request #41233 from josemimbrero-tinybird/fix-filesystem_cache_log 
Fix String log entry in system.filesystem_cache_log
 2022-09-13 12:21:18 +02:00
alesapin
69d08b1007
Merge pull request #41075 from ClickHouse/keeper-fix-possible-segfault 
Fix possible segfault during Keeper shutdown
 2022-09-13 12:13:24 +02:00
Kruglov Pavel
7d7606a1e3
Merge pull request #41162 from Avogar/fix-http-redirect 
Support relative path in Location header after http redirect
 2022-09-13 12:10:44 +02:00
Antonio Andelic
a6b5ffec5d Polishing 2022-09-13 09:51:46 +00:00
Igor Nikonov
c129b31d7b Test fix 2022-09-13 09:40:23 +00:00
Igor Nikonov
4cd35601d5 Merge remote-tracking branch 'origin/master' into distinct_in_order_wo_order_by 2022-09-13 08:19:59 +00:00
zhenjial
5841d9e9b0 sync before destruct 2022-09-13 15:53:24 +08:00
JackyWoo
ccff846e92 Merge branch 'master' into add_function_java_int_hash 2022-09-13 15:25:11 +08:00
zhenjial
67c08e3e22 sync before destruct 2022-09-13 15:06:22 +08:00
zhenjial
16c8cd0bd3 wait write finish 2022-09-13 14:19:40 +08:00
zhongyuankai
75c0854a78 fix test 2022-09-13 11:18:25 +08:00
zhongyuankai
1477bb132f
Merge branch 'master' into compress_marks_and_primary_key 2022-09-13 11:05:57 +08:00
Igor Nikonov
1c9c303cce Fix typos 2022-09-12 21:11:05 +00:00
Igor Nikonov
7502be3b75 Fixes 2022-09-12 20:11:24 +00:00
BoloniniD
147dfac11e Try using Corrosion 2022-09-12 23:05:41 +03:00
Sergei Trifonov
cad3a6b764 fix style 2022-09-12 22:01:45 +02:00
Robert Schulze
fac1be9700
chore: restore SYSTEM RELOAD MODEL(S) and moniting view SYSTEM.MODELS 
- This commit restores statements "SYSTEM RELOAD MODEL(S)" which provide
  a mechanism to update a model explicitly. It also saves potentially
  unnecessary reloads of a model from disk after it's initial load.

  To keep the complexity low, the semantics of "SYSTEM RELOAD MODEL(S)
  was changed from eager to lazy. This means that both statements
  previously immedately reloaded the specified/all models, whereas now
  the statements only trigger an unload and the first call to
  catboostEvaluate() does the actual load.

- Monitoring view SYSTEM.MODELS is also restored but with some obsolete
  fields removed. The view was not documented in the past and for now it
  remains undocumented. The commit is thus not considered a breach of
  ClickHouse's public interface.
 2022-09-12 19:33:02 +00:00
Sergei Trifonov
c31818260f renames and refactoring 2022-09-12 21:03:06 +02:00
Kseniia Sumarokova
5d1e9822f8
Merge branch 'master' into fix-filesystem_cache_log 2022-09-12 20:32:42 +02:00
Antonio Andelic
583c530d0f
Merge pull request #41177 from azat/http-handlers-reload 
Apply changes to http handlers on fly without server restart
 2022-09-12 20:25:00 +02:00
Alexander Tokmakov
05b4303234
Merge pull request #41146 from ClickHouse/followup_39933 
Stop background operations before converting database from Ordinary
 2022-09-12 21:20:25 +03:00
Antonio Andelic
b2cc6a8cc6 Use promise/future 2022-09-12 18:19:41 +00:00
Kruglov Pavel
702ddff5f6
Fix style 2022-09-12 19:38:34 +02:00
Kruglov Pavel
060adfbe93
Merge branch 'master' into new-json-formats 2022-09-12 19:37:46 +02:00
avogar
8ac2fc7b26 Don't outout totals/extremes in all row formats, update docs 2022-09-12 17:21:40 +00:00
Igor Nikonov
bdd7c7c2c1 Merge remote-tracking branch 'origin/master' into distinct_in_order_wo_order_by 2022-09-12 17:02:23 +00:00
Igor Nikonov
9fc0aaf477 Something working with tests 2022-09-12 17:01:26 +00:00
Jose
1880a745b2 Fix String in log entry 2022-09-12 18:53:54 +02:00
kssenii
a07f3d4bea Fix 2022-09-12 18:45:48 +02:00
Frank Chen
20191932df Fix style 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-09-13 00:41:05 +08:00
kssenii
06f80d78f8 Fix 2022-09-12 18:16:53 +02:00
Frank Chen
7e1f2901da Fix 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-09-13 00:06:17 +08:00
Frank Chen
7d6903bdc0 Fix 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-09-13 00:06:03 +08:00
BoloniniD
fe239e7ee9 Fix style + free err_msg before throwing exception 2022-09-12 18:46:54 +03:00
Azat Khuzhin
b698a4ff65 Apply changes to http handlers on fly without server restart 
This has been implemented by simply restarting http servers in case of
http_handlers directive in configuration xml had been changed.

But, for this I have to change the handlers interface to accept
configuration separatelly, since the configuration that contains in the
server is the configuration with which server had been started.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Co-authored-by: Antonio Andelic
 2022-09-12 17:34:51 +02:00
Nikolay Degterinsky
26bfabbb39 Fix bugs & remove old parsers 2022-09-12 15:26:45 +00:00
vdimir
ba506043ce
Fix exponential query rewrite in case of lots of cross joins with where 2022-09-12 15:05:34 +00:00
Alexander Tokmakov
da4fbb552d
Merge branch 'master' into fix_parts_removal 2022-09-12 17:49:29 +03:00
Antonio Andelic
deee0d639f Address PR comments 2022-09-12 14:43:52 +00:00
Alexey Milovidov
2aedd41023
Remove strange code (#40195) 
* Remove strange code

* Even more code removal

* Fix style

* Remove even more code

* Simplify code by making it slower

* Attempt to do something

* Attempt to do something

* Well do something with this horrible trash

* Add a test
 2022-09-12 16:29:23 +02:00
Sergei Trifonov
fe88a7991d
Merge branch 'master' into readonly-settings-allow 2022-09-12 16:20:40 +02:00
Frank Chen
ebaa24ecae Fix flaky tests 
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
 2022-09-12 22:15:30 +08:00
Azat Khuzhin
5cb37eb1a0 Terminate AsynchronousInsertQueue w/o waiting for timeout 
Previously it simply sleeps for async_insert_busy_timeout_ms delay on
shutdown, which is not great, and produces false-positive hung checks on
CI [1].

  [1]: https://pastila.nl/?0087a100/db80915eac58e67eb05516bf459e5510

Refs: #29390 (cc @tavplubix, @CurtizJ)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-12 16:11:54 +02:00
kssenii
413fbb6507 Self-review fixes 2022-09-12 15:46:11 +02:00
Antonio Andelic
1cd43d8e5c Merge branch 'master' into keeper-storage 2022-09-12 13:42:51 +00:00
Anton Popov
76af7efba9
Merge branch 'master' into optimize_compact_part_mutation 2022-09-12 15:35:47 +02:00
Dmitry Novik
c236706703
Merge branch 'master' into grouping-comp 2022-09-12 15:19:51 +02:00
Antonio Andelic
c2dfabe51c Use shared_ptr for Event 2022-09-12 12:25:39 +00:00
Antonio Andelic
e985ee3354 Close sessions on Keeper shutdown 2022-09-12 12:22:48 +00:00
Alexander Tokmakov
ed3c95c28f review fix 2022-09-12 13:53:43 +02:00
alesapin
e3e2c76606 FIX STYLE KEK 2022-09-12 13:52:57 +02:00
Sergei Trifonov
4810a59866 fix uninitialized variables 2022-09-12 13:27:06 +02:00
Kseniia Sumarokova
df22e8d30c
Merge pull request #41190 from kssenii/fix-clean-up-thread 
Fix cleaning up broken detached part after timeout
 2022-09-12 13:21:45 +02:00
kssenii
88b589917c Update defaults 2022-09-12 12:48:47 +02:00
Kruglov Pavel
6535301888
Merge branch 'master' into Avogar-patch-1 2022-09-12 12:23:28 +02:00
Alexander Tokmakov
7f462a8762
Update src/Storages/StorageReplicatedMergeTree.cpp 2022-09-12 13:22:08 +03:00
Kseniia Sumarokova
d0c058f273
Merge pull request #41132 from Algunenano/views_log_window 
Fix query_views_log with Window views
 2022-09-12 11:56:22 +02:00
Alexey Milovidov
9c5f9f1815
Merge pull request #41158 from den-crane/bug/trivial_count_optimization_with_array_join 
Fix: trivial count optimization with array join. revert revert #39466
 2022-09-12 08:56:10 +03:00
Alexey Milovidov
94d6e9ce7d
Merge pull request #41189 from kssenii/fix-positional-args 
Fix for positional arguments
 2022-09-12 03:21:39 +03:00
Azat Khuzhin
7e130aeb69 Add support for DWARF-5 (without emitting them in binaries) 
ClickHouse changes to the folly parser:
- use camel_case
- add NOLINT
- avoid using folly:: (use std:: instead)
- avoid using boost:: (use std:: instead)

But note, now it has not been enabled by default (like it was
initially), because you may need recent debugger to support DWARF-5
correctly, and to make debugging easier, let's do this later.

A good example is gdb 10, even though it looks like it should support
it, it still produce some errors, like here [1]:

    Dwarf Error: DW_FORM_strx1 found in non-DWO CU [in module /usr/bin/clickhouse]

  [1]: https://github.com/ClickHouse/ClickHouse/pull/40772#issuecomment-1236331323

And not only it complains, apparently it can "activate" SDT probes
(replace "nop" with "int3"), and I believe this is what happens here
[2].

  [2]: https://github.com/ClickHouse/ClickHouse/pull/41063#issuecomment-1242992314

There you got int3 in the case when ClickHouse got SIGTRAP:

<details>

```
    0x7f494705e093 <+1139>: jne    0x7f494705e450            ; <+2096> [inlined] update_tls_slotinfo at dl-open.c:732
    0x7f494705e099 <+1145>: testl  %r13d, %r13d
    0x7f494705e09c <+1148>: je     0x7f494705e09f            ; <+1151> at dl-open.c:744:6
    0x7f494705e09e <+1150>: int3
->  0x7f494705e09f <+1151>: movl   -0x54(%rbp), %eax
    0x7f494705e0a2 <+1154>: testl  %eax, %eax
    0x7f494705e0a4 <+1156>: jne    0x7f494705e410            ; <+2032> at dl-open.c:745:5

But if I repeat the query it does not:

    0x7ffff7fe5093 <+1139>: jne    0x7ffff7fe5450            ; <+2096> [inlined] update_tls_slotinfo at dl-open.c:732
    0x7ffff7fe5099 <+1145>: testl  %r13d, %r13d
    0x7ffff7fe509c <+1148>: je     0x7ffff7fe509f            ; <+1151> at dl-open.c:744:6
    0x7ffff7fe509e <+1150>: nop
->  0x7ffff7fe509f <+1151>: movl   -0x54(%rbp), %eax
    0x7ffff7fe50a2 <+1154>: testl  %eax, %eax
    0x7ffff7fe50a4 <+1156>: jne    0x7ffff7fe5410            ; <+2032> at dl-open.c:745:5
```

</details>

Test command was:

    clickhouse local --stacktrace -q "select * from file('data.capnp', 'CapnProto', 'val1 char') settings format_schema='nonexist:Message'

*P.S. I did this, because I have libraries compiled with DWARF5 (i.e. glibc), and dwarf parser simply fails on my dev env.*

Refs: 490b287ca3
(cherry picked from commit ee5696bb32)
(cherry picked from commit e03870bc8b)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-11 21:06:14 +02:00
Alexey Milovidov
8b5328f7e5 Merge branch 'master' into fix-trash-base58 2022-09-11 20:43:54 +02:00
Alexey Milovidov
3e6c4e3f16 Fix clang-tidy 2022-09-11 20:43:36 +02:00
Alexey Milovidov
9a0892c40c
Merge pull request #41171 from ClickHouse/fix-half-of-trash 
Fix half of trash
 2022-09-11 21:40:22 +03:00
Alexey Milovidov
bd40d0de7d
Merge pull request #41187 from ClickHouse/faster-build 
Faster build
 2022-09-11 21:39:36 +03:00
Denny Crane
0e4e868f29
Merge branch 'master' into bug/trivial_count_optimization_with_array_join 2022-09-11 15:23:10 -03:00
BoloniniD
c0024dff3a Add embedded docs for BLAKE3 2022-09-11 19:42:36 +03:00
kssenii
9c2bbcf4a6 Fix 2022-09-11 17:38:51 +02:00
kssenii
02c3d8b0a0 Fix 2022-09-11 16:42:56 +02:00
alesapin
6eead4ea1e Fix fasttest build 2022-09-11 15:16:54 +02:00
kssenii
2f3bfc5411 Fix 2022-09-11 13:54:25 +02:00
alesapin
21ab379026 Fix comment 2022-09-11 13:47:04 +02:00
alesapin
ac53830000 Fix hung check 2022-09-11 13:37:39 +02:00
kssenii
b7d751b782 Merge remote-tracking branch 'upstream/master' into improve-marks-cache-loading 2022-09-11 13:23:30 +02:00
kssenii
37bdba98b1 Fix 2022-09-11 12:58:04 +02:00
Alexey Milovidov
7f5a1cbe16 Faster build 2022-09-11 09:29:24 +02:00
Alexey Milovidov
5802c2fdd2
Merge pull request #40713 from ClickHouse/remove-useless-line 
Remove one line from XML, because we do not care
 2022-09-11 09:53:30 +03:00
Alexey Milovidov
5592c8e775 Faster build 2022-09-11 08:50:44 +02:00
Alexey Milovidov
88e9178830 Faster build 2022-09-11 08:46:05 +02:00
Alexey Milovidov
0896e6b9b6
Merge pull request #41183 from ClickHouse/fix-parallel-hash-join-for-floats 
Better parallel hash JOIN for floats
 2022-09-11 09:13:57 +03:00
Alexey Milovidov
e0a9ae0496 Fix base58 trash 2022-09-11 08:09:14 +02:00
Alexey Milovidov
3d8a2130cf Continue fixing the trash 2022-09-11 06:31:15 +02:00
Alexey Milovidov
9fbd361a2b Merge branch 'master' into fix-trash-base58 2022-09-11 06:23:01 +02:00
Alexey Milovidov
7f1e7b5967 Merge branch 'master' into fix-half-of-trash 2022-09-11 06:20:47 +02:00
Alexey Milovidov
05d0f48d17
Merge pull request #41173 from ClickHouse/remove-some-methods 
Remove some methods
 2022-09-11 07:19:52 +03:00