Azat Khuzhin
f76f14a99c
Fix possible hung/deadlock on query cancellation
...
Right due to graceful cancellation of the query, it is possible to hung
accepting new queries or even have a deadlock, this is because
cancellation is done while acquiring ProcessListBase::mutex.
So this patch makes query cancellation lock-free, and now the lock will
be acquired only for preparing the query and after cancel is done.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-09-19 17:31:21 +02:00
Vitaly Baranov
052ecaa211
Merge branch 'master' into improve-recovery-of-replicated-access-storage
2022-09-19 15:26:21 +02:00
Vitaly Baranov
f75dd93965
Mask some information in logs.
2022-09-19 15:00:06 +02:00
Sergei Trifonov
ff2e1ff8d8
Merge branch 'master' into support_s3_api_event_metrics
2022-09-19 14:35:45 +02:00
Raúl Marín
e89dffbb95
Fix AggregateFunctionNullVariadic batch with null_is_skipped=false
2022-09-19 14:34:08 +02:00
Kruglov Pavel
712aebc077
Merge pull request #41083 from Avogar/fix-map-combinator
...
Fix possible segfaults, use-heap-after-free and memory leak in aggregate function combinators
2022-09-19 14:29:17 +02:00
kssenii
f8665c83da
Add retries for nats
2022-09-19 14:27:11 +02:00
Alexander Tokmakov
c6efb1773e
Merge branch 'master' into fix_drop_partition_on_cluster
2022-09-19 15:18:05 +03:00
Kruglov Pavel
4c3194eefe
Merge pull request #41286 from azat/utf8-fix
...
Do not allow invalid sequences influence other rows in lowerUTF8/upperUTF8
2022-09-19 14:07:10 +02:00
Sergei Trifonov
f7a1ed3141
Merge branch 'master' into readonly-settings-allow
2022-09-19 12:52:36 +02:00
serxa
a0bfa801e5
explicit ctor
2022-09-19 10:52:08 +00:00
Kseniia Sumarokova
631aad1671
Merge branch 'master' into fix-logical-error-write-through-cache
2022-09-19 12:29:55 +02:00
Antonio Andelic
1763af37b9
Merge pull request #41403 from ClickHouse/keeper-fix-shutdown-without-server
...
Fix Keeper segfault during shutdown when RAFT server failed to start
2022-09-19 12:03:58 +02:00
Igor Nikonov
81a46b6346
Merge remote-tracking branch 'origin/master' into fix_sort_desc_when_read_in_order
2022-09-19 09:40:23 +00:00
Igor Nikonov
54c2fed9cc
Update: use type explicitly
2022-09-19 09:31:45 +00:00
Alexey Milovidov
45b6052b20
Merge branch 'master' into compress-marks
2022-09-19 10:57:05 +02:00
vdimir
a6dc99983d
Fix tidy build
2022-09-19 08:42:23 +00:00
vdimir
5c9bfbc116
Fix incorrect result in case of decimal precision loss in IN operator
2022-09-19 08:42:22 +00:00
vdimir
c2634c9fae
Revert "Throw exception on precision lost during decimal to decimal convertion"
...
This reverts commit d4233cf3c658adf2541070b1bbba678ad7c769db.
2022-09-19 08:42:22 +00:00
vdimir
c33e231ce9
Throw exception on precision lost during decimal to decimal convertion
2022-09-19 08:42:18 +00:00
Vladimir C
b6b5932efc
Merge pull request #41223 from ClickHouse/vdimir/exponential_rewrite_21557
2022-09-19 10:07:24 +02:00
Alexey Milovidov
b9bd2c8373
Merge pull request #41458 from ClickHouse/fix-bug-orc
...
Fix very strange behavior of Apache ORC
2022-09-19 10:43:17 +03:00
Alexey Milovidov
000b6ac81c
Fix error
2022-09-19 09:30:48 +02:00
Duc Canh Le
61d6fae14b
add settings allow_suspicious_fixed_string_types
2022-09-19 15:00:04 +08:00
Alexey Milovidov
b87d12494e
Fix build
2022-09-19 08:53:20 +02:00
Alexey Milovidov
5224fc4d7f
Fix strange code
2022-09-19 08:53:20 +02:00
Alexey Milovidov
d1275091e8
Fix strange code
2022-09-19 08:53:20 +02:00
Alexey Milovidov
6a473d8998
Merge
2022-09-19 08:53:20 +02:00
Alexey Milovidov
bb6f1bfce2
Fix 9/10 of trash
2022-09-19 08:53:20 +02:00
Alexey Milovidov
730655d4fd
Fix 8/9 of trash
2022-09-19 08:53:20 +02:00
Alexey Milovidov
42b0d444da
Fix 7/8 of trash
2022-09-19 08:53:20 +02:00
Alexey Milovidov
91baedf03a
Fix 6/7 of trash
2022-09-19 08:53:20 +02:00
Alexey Milovidov
ab4db2d0c4
Fix 5/6 of trash
2022-09-19 08:50:53 +02:00
Alexey Milovidov
84f42e0874
Fix 3/4 of trash
2022-09-19 08:50:53 +02:00
Amos Bird
d2601b8192
Add a comment
2022-09-19 14:41:34 +08:00
Alexey Milovidov
3c0961c48b
Merge branch 'master' into fix-map-combinator
2022-09-19 09:32:16 +03:00
Alexey Milovidov
b01f5bdca8
Merge pull request #39354 from ClickHouse/window-order-by
...
Respect ORDER BY clause in window functions
2022-09-19 09:22:37 +03:00
Amos Bird
09b75901f0
Fix nullptr deref
2022-09-19 14:07:31 +08:00
Alexey Milovidov
6a09340f11
Merge pull request #41459 from azat/io/threadpoolreader-elapsed
...
Account any IO that is done from ThreadPoolReader
2022-09-19 09:04:20 +03:00
Alexey Milovidov
81e8cb4be6
Merge branch 'master' into fix-bug-orc
2022-09-19 06:38:17 +03:00
Alexey Milovidov
d4b9fe41be
Merge pull request #41457 from ClickHouse/remove-trash-5
...
Remove trash from Field
2022-09-19 06:36:48 +03:00
Alexey Milovidov
57a1f47972
Merge pull request #41441 from ClickHouse/pod-array-overflow
...
Fix overflow in `PODArray`
2022-09-19 06:34:21 +03:00
Alexey Milovidov
ac193facde
Merge branch 'master' into add_missed_throw
2022-09-19 06:31:12 +03:00
Alexey Milovidov
df52df83f9
Merge pull request #41463 from ClickHouse/add-test-38383
...
Add a test for #38383
2022-09-19 06:27:38 +03:00
Alexey Milovidov
790cb730d6
Fix clang-tidy
2022-09-19 00:34:41 +02:00
Alexey Milovidov
cc28120761
Merge branch 'master' into pod-array-overflow
2022-09-19 00:34:38 +02:00
Alexey Milovidov
f3fd5ef845
Fix clang-tidy
2022-09-19 00:32:52 +02:00
Alexey Milovidov
36aaeecc2f
Fix clang-tidy
2022-09-19 00:32:11 +02:00
Alexey Milovidov
5d9e125c71
Fix clang-tidy
2022-09-19 00:28:32 +02:00
Igor Nikonov
e888859396
Fix: correct sort description for ReadFromMergeTree with read in order
...
optimization
2022-09-18 22:21:13 +00:00
alesapin
e4c120742e
Add missed throw expression
2022-09-18 16:56:19 +02:00
alesapin
560bc2bc22
Merge pull request #41386 from ClickHouse/fix_redundant_fetch
...
Fix redundant fetch in zero copy replication
2022-09-18 15:57:27 +02:00
alesapin
1d0dbfb112
Merge pull request #41444 from ClickHouse/one_wrong_config_assert
...
One more validation for broken config in keeper
2022-09-18 15:56:48 +02:00
Kruglov Pavel
519bcbb3be
Merge pull request #41236 from Avogar/fix-totals-extremes
...
Don't output totals/extremes in all row formats, update docs
2022-09-18 14:55:07 +02:00
Kseniia Sumarokova
d6748b182f
Update CachedOnDiskWriteBufferFromFile.cpp
2022-09-18 14:48:24 +02:00
Vitaly Baranov
e7e51ab2d9
Add comments.
2022-09-18 12:44:05 +02:00
Vitaly Baranov
69996c960c
Add retries for the initialization of ReplicatedAccessStorage.
2022-09-18 12:44:05 +02:00
Vitaly Baranov
5365b105cc
Add SYSTEM RELOAD USERS command.
2022-09-18 12:44:00 +02:00
Vitaly Baranov
1b40f94c6b
Move common code from MemoryAccessStorage and DiskAccessStorage to IAccessStorage.
2022-09-18 12:17:15 +02:00
Azat Khuzhin
64b44be513
Account any IO that is done from ThreadPoolReader
...
In case of some issues with the disk it is possible that
preadv2(RWF_NOWAIT)==EAGAIN can take a lot of time, so let's measure
even this case.
I've verified with a simple (almost) loop of preadv2(RWF_NOWAIT), and
got this:
EAGAIN: 8565670 (out of 17148317)
EAGAIN: 8565670 (out of 17148317)
EAGAIN: 8565670 (out of 17148317)
too long at offset=8691694, ret=18446744073709551615 (14.5016)
too long at offset=8694766, ret=18446744073709551615 (14.4338)
too long at offset=8694766, ret=18446744073709551615 (16.2144)
So here you can see that preadv2(RWF_NOWAIT) took 14-16 seconds in case
of failure.
Complete code can be found here [1].
[1]: https://gist.github.com/azat/f4a6b0750316c42bec021064e0a7627c
I came up with this patch because I saw some stranginess in
ProfileEvents, that does not show all the time that was spend in IO when
pread_threadpool method was used.
Also there was some stranginess with trace_log too.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-09-18 10:05:21 +02:00
Alexey Milovidov
7e6da853cd
Merge branch 'master' into pod-array-overflow
2022-09-18 09:24:23 +02:00
Alexey Milovidov
a6e3ab5515
Fix error
2022-09-18 09:21:57 +02:00
Alexey Milovidov
5c75a7d661
Fix error
2022-09-18 09:20:48 +02:00
Alexey Milovidov
c41e37fa8e
Code simplification
2022-09-18 08:58:32 +02:00
Alexey Milovidov
3c43d6516b
Fix style
2022-09-18 08:56:52 +02:00
Alexey Milovidov
bafc5f24c7
Merge branch 'master' into remove-trash-5
2022-09-18 08:56:02 +02:00
Alexey Milovidov
8764fa4439
Fix very strange behavior of Apache ORC
2022-09-18 08:25:25 +02:00
Alexey Milovidov
a958751c98
Merge pull request #40219 from den-crane/test/zvonand-monotonic
...
test for datetime64 monotonic #39864
2022-09-18 07:48:49 +03:00
Alexey Milovidov
6b752bff3a
Merge branch 'master' into revert_prefixes_s3
2022-09-18 07:10:57 +03:00
Alexey Milovidov
32efe6952e
Merge pull request #41429 from ClickHouse/forbid-dlopen-2
...
Forbid invocations of `dlopen`
2022-09-18 07:05:56 +03:00
Alexey Milovidov
791de6592b
Remove trash from Field
2022-09-18 05:16:08 +02:00
Alexey Milovidov
c8c728e740
Merge pull request #41449 from ClickHouse/fix-categorial-information
...
Fix incorrect properties of `categorialInformationValue` function
2022-09-18 05:26:26 +03:00
Alexey Milovidov
7f68f2c7df
Remove trash from LowCardinality
2022-09-18 04:13:21 +02:00
Alexey Milovidov
15e5fc96a1
Add a trap
2022-09-18 02:24:56 +02:00
Alexey Milovidov
dc76677eb2
Merge pull request #41440 from ClickHouse/remove-cruft
...
Remove cruft about gcc
2022-09-18 03:15:26 +03:00
Alexey Milovidov
e3dafdb865
More hardening
2022-09-18 02:10:06 +02:00
Alexey Milovidov
e5e3bca3ae
More hardening and tests
2022-09-18 02:05:13 +02:00
Alexey Milovidov
6c74c54cb5
Merge branch 'master' into pod-array-overflow
2022-09-18 01:24:52 +02:00
Alexey Milovidov
6f1878b12a
Fix error
2022-09-18 01:17:22 +02:00
Alexey Milovidov
c424ad12aa
Simplification
2022-09-18 00:09:21 +02:00
alesapin
b6d61318d0
One more validation for broken config in keeper
2022-09-18 00:04:16 +02:00
Alexey Milovidov
e2d434ea18
Start with code removal
2022-09-17 23:37:57 +02:00
Alexey Milovidov
2f0684b97c
Fix trash in schema inference
2022-09-17 23:11:33 +02:00
Alexey Milovidov
bac578b23a
Merge pull request #41428 from ClickHouse/remove-dlopen
...
Remove `dlopen`
2022-09-18 00:09:57 +03:00
Alexey Milovidov
e3d75b0933
Merge branch 'fix-trash-schema-inference' into pod-array-overflow
2022-09-17 22:56:06 +02:00
Alexey Milovidov
47167494d9
Fix trash in schema inference
2022-09-17 22:53:41 +02:00
Alexey Milovidov
9ca4e2ec7c
Fix overflow in PODArray
2022-09-17 22:26:39 +02:00
Alexey Milovidov
f4a48cd4d6
Remove cruft
2022-09-17 22:16:31 +02:00
kssenii
51745dcb43
Fix assertion in write-through cache
2022-09-17 20:20:32 +02:00
Alexander Tokmakov
49a9c9e5fa
Merge pull request #41436 from ClickHouse/revert_38200
...
Revert "Keeper faster linearizable reads"
2022-09-17 20:25:43 +03:00
Robert Schulze
38aecc15a2
Merge pull request #41214 from arenadata/ADQM-528-B
...
Add enable_extended_results_for_datetime_functions option to return results of type Date32
2022-09-17 19:25:11 +02:00
Alexander Tokmakov
42dab6dc02
Revert "Merge pull request #38200 from ClickHouse/keeper-linearizable-reads"
...
This reverts commit 222b964070da01982652
2022-09-17 19:22:58 +02:00
alesapin
33216dedf9
Merge pull request #41323 from ClickHouse/remove_bad_exception
...
Fix parts removal in case of connection loss
2022-09-17 19:14:53 +02:00
alesapin
dcb1586977
Add comment
2022-09-17 19:11:58 +02:00
alesapin
db30a3041e
Update src/Storages/MergeTree/MutateFromLogEntryTask.cpp
...
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
2022-09-17 19:10:23 +02:00
alesapin
84828874e0
Change log level
2022-09-17 19:06:54 +02:00
Anton Popov
07cfb0cba4
Merge branch 'master' into dynamic-columns-21
2022-09-17 19:02:58 +02:00
Alexey Milovidov
9ce0c5f738
Merge pull request #41431 from ClickHouse/remove-method-simplification
...
Remove `-WithTerminatingZero` methods
2022-09-17 19:48:35 +03:00
Azat Khuzhin
01b5b9ceba
Do not allow invalid sequences influence other rows in lowerUTF8/upperUTF8
...
Right now lowerUTF8() and upperUTF8() does not respect row boundaries,
and so one row may break another.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-09-17 11:16:45 +02:00
Igor Nikonov
475443c10c
Merge pull request #41257 from ClickHouse/igor/mixed_improvements_along_the_way
...
Tiny improvements along the code
2022-09-17 10:43:46 +02:00
Alexey Milovidov
ada7a44ae4
Remove -WithTerminatingZero methods
2022-09-17 05:34:18 +02:00
Alexey Milovidov
0c3835240b
Fix error with multidimensional arrays
2022-09-17 04:16:24 +02:00
Alexey Milovidov
b60a5a5bc5
Merge branch 'master' into fix-getdataat-multidimensional-arrays
2022-09-17 04:07:12 +02:00
Alexey Milovidov
35cce03125
Remove dlopen
2022-09-17 03:02:34 +02:00
Alexey Milovidov
31ebd34f5b
Merge pull request #41193 from azat/DWARF-5-parser-v3
...
Add support for DWARF-5 (without emitting them in binaries)
2022-09-17 02:58:02 +03:00
Alexey Milovidov
c09604bef2
Merge pull request #41363 from kitaisreal/small-style-fixes
...
Small style fixes
2022-09-17 00:19:00 +03:00
Denny Crane
6540d55d42
Merge branch 'master' into test/zvonand-monotonic
2022-09-16 17:02:09 -03:00
Nikita Taranov
6f186d3dd2
Do not return empty blocks from ConvertingAggregatedToChunksTransform ( #41152 )
...
* impl
* add test
* update test
2022-09-16 21:54:36 +02:00
avogar
fa172e21ae
Remove unnecessary method implementation
2022-09-16 17:47:05 +00:00
Vitaly Baranov
391507b4b6
Update src/Access/DiskAccessStorage.cpp
...
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2022-09-16 19:02:06 +02:00
Vitaly Baranov
2ec6ef497d
Make DiskAccessStorage::insertWithID() more consistent with MemoryAccessStorage::insertWithID().
2022-09-16 19:02:06 +02:00
Vitaly Baranov
7e716d14cd
Make ReplicatedAccessStorage::insertWithID() more consistent with MemoryAccessStorage::insertWithID().
2022-09-16 19:02:06 +02:00
Vitaly Baranov
646cd55690
Improve recovery of ReplicatedAccessStorage after errors.
2022-09-16 19:02:06 +02:00
Vitaly Baranov
8f9f5c69da
Use MemoryAccessStorage to simplify ReplicatedAccessStorage.
2022-09-16 19:02:06 +02:00
Dmitry Novik
eacc790050
Merge branch 'master' into fix-odr-vialation
2022-09-16 18:44:53 +02:00
avogar
c9a9ef5b7e
Fix comments
2022-09-16 16:12:30 +00:00
Alexander Tokmakov
65474830ac
make tryEnqueueReplicatedDDL virtual
2022-09-16 16:25:32 +02:00
Nikolai Kochetov
16f78eb804
Merge pull request #41281 from ClickHouse/fix-key-condition-with-actions-dag-literal-name
...
Fix wrong literal name in KeyCondition using ActionsDAG.
2022-09-16 16:06:38 +02:00
Kruglov Pavel
2d4a6b38af
Merge branch 'master' into fix-totals-extremes
2022-09-16 15:03:59 +02:00
alesapin
d9c3549cc8
Revert s3 prefixes + add retries on write requests unexpected errors
2022-09-16 14:43:43 +02:00
alesapin
456da30c9f
Revert "Merge pull request #41249 from ClickHouse/revert-40968-s3-sharding-2"
...
This reverts commit ac60300997a9c272283a
2022-09-16 14:06:26 +02:00
alesapin
06ccdb1ecf
Fix nasty bug
2022-09-16 13:49:39 +02:00
Alexander Tokmakov
9597a47016
Merge pull request #41379 from ClickHouse/tavplubix-patch-5
...
Fix an assertion in `wasInvolvedInTransaction`
2022-09-16 14:24:26 +03:00
Robert Schulze
b32b02d844
Merge pull request #40897 from ClickHouse/catboost-bridge-resurrected
...
Move CatBoost evaluation into clickhouse-library-bridge
2022-09-16 13:12:09 +02:00
Anton Popov
5584ef71f6
Merge pull request #41388 from den-crane/bug/optimize_rewrite_sum_if_to_count_if
...
Disable optimize_rewrite_sum_if_to_count_if by default
2022-09-16 13:05:23 +02:00
Antonio Andelic
acd9cb089e
Fix segfault during shutdown when RAFT server failed to start
2022-09-16 10:03:21 +00:00
Roman Vasin
bc97da686a
Add const for bool enable_extended_results_for_datetime_functions
2022-09-16 09:54:24 +00:00
alesapin
af259a2a68
Update src/Storages/MergeTree/MergeFromLogEntryTask.cpp
...
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
2022-09-16 11:50:07 +02:00
Roman Vasin
d82cd7b007
Add many cosmetic changes to C++ code and MD docs
2022-09-16 09:37:30 +00:00
Roman Vasin
7e733887b9
Add narrowing for toStartOfISOYear; improve 02403_enable_extended_results_for_datetime_functions
2022-09-16 08:40:15 +00:00
Alexey Milovidov
701fdd77b4
Fix wrong implementation of getDataAt for multidimensional arrays
2022-09-16 08:12:28 +02:00
Antonio Andelic
222b964070
Merge pull request #38200 from ClickHouse/keeper-linearizable-reads
...
Keeper faster linearizable reads
2022-09-16 07:58:52 +02:00
Alexey Milovidov
da01982652
Merge pull request #41046 from azat/build/llvm-15
...
Switch to llvm/clang 15
2022-09-16 07:31:06 +03:00
Raúl Marín
c3ff66bd9d
Implement batch processing for aggregate functions with multiple nullable arguments ( #41058 )
...
* Implement batch processing for aggregate functions with multiple nullable arguments
* Fix broken perf test
* Improve filter handling in addBatchSinglePlace with nullable arguments
* Fix detecting the Null filter usage
2022-09-15 23:51:38 +02:00
Raúl Marín
6dac509739
Speed up reading uniqState ( #41089 )
...
* Speed up reading UniquesHashSet
* Improve uniq serialization tests
2022-09-15 23:41:15 +02:00
Alexander Tokmakov
8301f5758b
Update IMergeTreeDataPart.cpp
2022-09-15 21:32:57 +02:00
Alexander Tokmakov
03c193ccca
fix ON CLUSTER with Replicated database cluster
2022-09-15 21:15:57 +02:00
Denny Crane
7cfdfbe787
disable optimize_rewrite_sum_if_to_count_if by default
2022-09-15 15:30:24 -03:00
Alexander Tokmakov
32391f737a
Merge branch 'master' into fix_41209
2022-09-15 20:27:51 +02:00
avogar
ef7f7b30aa
Fix tests
2022-09-15 18:20:17 +00:00
avogar
92979af3bf
Add missing methods in combinators
2022-09-15 17:45:50 +00:00
alesapin
82611d891b
Merge branch 'master' into fix-odr-vialation
2022-09-15 19:33:09 +02:00
alesapin
be81d653e1
Fix comment
2022-09-15 18:56:16 +02:00
alesapin
b098c1be36
Fix data duplication in zero copy replication
2022-09-15 18:54:27 +02:00
Antonio Andelic
fe78c063f6
Merge branch 'master' into keeper-linearizable-reads
2022-09-15 18:40:23 +02:00
Antonio Andelic
39307f9dba
Extract into function
2022-09-15 16:14:53 +00:00
Antonio Andelic
0dbd0c1f72
Merge branch 'master' into keeper-linearizable-reads
2022-09-15 16:03:18 +00:00
Anton Popov
b5e9884355
fix subcolumns tree
2022-09-15 14:17:10 +00:00
Roman Vasin
5c242de43a
Merge branch 'master' of github.com:ClickHouse/ClickHouse into ADQM-528-B
2022-09-15 13:47:51 +00:00
alesapin
7570c5cf67
Remove recursive without blobs
2022-09-15 15:31:19 +02:00
