Yakov Olkhovskiy
7c94f98f6a
Merge pull request #37961 from ClibMouse/Kusto-phase1
...
Kusto phase1
2022-09-27 12:42:15 -04:00
Antonio Andelic
265c8b3d26
Merge branch 'master' into keeper-multiread
2022-09-27 12:37:22 +00:00
Antonio Andelic
7a9afc4bdd
Fix zookeeper_log
2022-09-27 12:36:59 +00:00
Alexander Tokmakov
3d4a5a493b
Merge pull request #41484 from FrankChen021/on_cluster_dll
...
Add OpenTelemetry support to ON CLUSTER DDL
2022-09-27 15:15:52 +03:00
Alexander Gololobov
1719e000ea
Merge pull request #41763 from zhangjmruc/fix/disable_mergetree_table_with_lwd_virtual_column
...
Disable merge tree table with lightweight delete virtual column name _row_exists
2022-09-27 12:20:19 +02:00
Kseniia Sumarokova
caf6a99f41
Merge pull request #41011 from kssenii/fix-thread-status
...
Fix incorrect attach query in threadpool readers, get rid of static threadpools for reads/writes, make threadpool size for reads/writes configurable
2022-09-27 11:35:07 +02:00
Alexander Gololobov
f21f933da7
Merge branch 'master' into fix/disable_mergetree_table_with_lwd_virtual_column
2022-09-26 23:44:45 +02:00
Robert Schulze
3efd42ae2e
Fix tests (hopefully)
...
- LLVM migrates to opaque pointers
(https://llvm.org/docs/OpaquePointers.html ). In LLVM 13, opaque
pointers are WIP and not recommended for use.
- But as preparation, they already changed some API, specifically
they expect that IRBuilder::CreateConstGEP*() no longer get nullptr as
first argument.
- We are now passing explicitly the value which was in LLVM 12 figured
out automatically under the hood.
- Since the passed pointer is still non-opaque, I don't think this will
work in LLVM 15 but it could be okay for a migration.
2022-09-26 19:43:28 +00:00
Nikita Mikhaylov
c993051265
Fix deadlock and flaky test
2022-09-26 15:17:34 +00:00
Sergei Trifonov
060402cab7
Merge pull request #41675 from ZhiguoZh/20220922-lock-opt
...
Optimize the lock contentions for ThreadGroupStatus::mutex
2022-09-26 14:33:47 +02:00
Kruglov Pavel
3dc54272ed
Merge branch 'master' into improve-combinators
2022-09-26 13:03:32 +02:00
Nikita Mikhaylov
4d1d87da2c
Delete garbage
2022-09-26 10:24:39 +00:00
Nikita Mikhaylov
852d084950
Save
2022-09-26 10:24:39 +00:00
Nikita Mikhaylov
fa587ce057
Better
2022-09-26 10:24:39 +00:00
Nikita Mikhaylov
a5eacc2e02
Even better
2022-09-26 10:24:39 +00:00
Nikita Mikhaylov
636cedf488
Better
2022-09-26 10:24:39 +00:00
Nikita Mikhaylov
9b35d54e8f
First try
2022-09-26 10:24:39 +00:00
Antonio Andelic
97385ca861
Format
2022-09-26 09:33:50 +00:00
Antonio Andelic
56cc3c7137
Better TransactionLog with multiread
2022-09-26 09:31:27 +00:00
Antonio Andelic
cc3719e463
Use multiread
2022-09-26 09:07:34 +00:00
jianmei zhang
cf40c57562
Disable mergetree table with lightweight delete column name
2022-09-26 15:33:43 +08:00
Robert Schulze
9c62797d28
Bump LLVM from 12 to 13
2022-09-26 07:13:46 +00:00
Kseniia Sumarokova
50f7ce6107
Merge branch 'master' into fix-thread-status
2022-09-24 17:22:45 +02:00
Kseniia Sumarokova
21e09f3e1f
Merge pull request #41733 from kssenii/cache-logging-level-reduce
...
Change logging levels in cache
2022-09-24 17:20:46 +02:00
Nikolai Kochetov
1c67437aa4
Fix possible crash for SELECT from Merge table with optimize_monotonous_functions_in_order_by eanbled.
2022-09-23 18:37:25 +00:00
kssenii
0a801dad2a
Merge remote-tracking branch 'upstream/master' into fix-thread-status
2022-09-23 19:39:07 +02:00
kssenii
30726721ad
Fix threadpool reader (for local fs)
2022-09-23 19:35:16 +02:00
Kseniia Sumarokova
b205824670
Merge branch 'master' into cache-config-allow-readable-format-settings
2022-09-23 15:40:07 +02:00
kssenii
c122e4dd1f
Refactor log levels
2022-09-23 15:32:05 +02:00
kssenii
ab702e43fc
Merge remote-tracking branch 'upstream/master' into fix-thread-status
2022-09-23 15:21:33 +02:00
Nikita Mikhaylov
dbcba1490c
Try to make MergeTreeRestartingThread more reactive in case of expired sessions ( #41092 )
2022-09-23 14:40:04 +02:00
Frank Chen
34bcb6a82b
Fix style
...
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2022-09-23 11:48:50 +08:00
Frank Chen
45e3d7d788
Update name/comments
2022-09-23 11:36:06 +08:00
Frank Chen
2344e0738e
Keep compatibility during upgrading
2022-09-23 11:32:21 +08:00
Frank Chen
40f9e0b69a
Address review comments
...
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2022-09-23 10:08:42 +08:00
Nikolai Kochetov
00965ce17a
Fixing race.
2022-09-22 17:57:04 +00:00
Nikolay Degterinsky
a7f3c7086c
Merge pull request #41341 from evillique/mandatory-identification
...
Add a setting requiring no_password to be explicitly specified when creating a user
2022-09-22 15:19:43 +02:00
kssenii
f038c7ea00
Better config for cache
2022-09-22 14:50:29 +02:00
Nikolai Kochetov
63e757a659
Merge pull request #36379 from amosbird/projection-compatibility-fix
...
Make minmax_count projection backward compatible
2022-09-22 14:15:35 +02:00
Kseniia Sumarokova
b11aa0dd0e
Merge pull request #41680 from ClickHouse/kssenii-patch-5
...
Fix error message
2022-09-22 13:26:42 +02:00
Kseniia Sumarokova
5caac39328
Merge pull request #41508 from kssenii/rename-some-cache-commands
...
Rename: `show caches` -> `show filesystem caches`, `describe cache` -> `describe filesystem cache`
2022-09-22 13:09:14 +02:00
Alexander Tokmakov
021b882455
Revert "Disable parallel s3 multipart upload for part moves. ( #41268 )"
...
This reverts commit 0414d95878
.
2022-09-22 13:59:25 +03:00
Kseniia Sumarokova
73ccf94778
Update FileCacheSettings.cpp
2022-09-22 12:53:26 +02:00
Zhiguo Zhou
223c1230b6
Optimize the lock contentions for ThreadGroupStatus::mutex
...
The release of ThreadGroupStatus::finished_threads_counters_memory
via the getProfileEventsCountersAndMemoryForThreads method brings
lots of lock contentions for ThreadGroupStatus::mutex and lowers
the overall performance. This commit optimizes this performance
issue by replacing the method call with an equivalent but more
lightweight code block.
2022-09-22 16:23:16 +08:00
Alexey Milovidov
e16d583aaf
Merge pull request #41280 from ClickHouse/another_fix_for_an_ast_optimization
...
Fix WHERE pushdown when subquery has OFFSET
2022-09-22 07:33:44 +03:00
Alexey Milovidov
45afacdae4
Merge pull request #41186 from ClickHouse/fix-three-fourth-of-trash
...
Fix more than half of the trash
2022-09-22 07:28:26 +03:00
Alexey Milovidov
5c508909c0
Merge pull request #41605 from ClickHouse/fix-amusing-error
...
Fix amusing error in conversion from Ordinary to Atomic
2022-09-22 07:25:41 +03:00
Frank Chen
99c74d182c
Merge branch 'master' into on_cluster_dll
2022-09-22 09:58:10 +08:00
Nikolai Kochetov
446453bdf5
Fix a destruction order for views ThreadStatus
2022-09-21 18:37:40 +00:00
Nikolay Degterinsky
49e0a87b8f
Merge pull request #41495 from canhld94/ch_fixedstring_len
...
Add settings allow_suspicious_fixed_string_types
2022-09-21 19:35:34 +02:00
Nikolai Kochetov
0414d95878
Disable parallel s3 multipart upload for part moves. ( #41268 )
...
* Disable parallel s3 multipart upload for part moves.
* Add setting s3_allow_parallel_part_upload
2022-09-21 19:10:32 +02:00
Nikita Taranov
100c055510
Prefetching in aggregation ( #39304 )
...
* impl
* stash
* clean up
* do not apply when HT is small
* make branch static
* also in merge
* do not hardcode look ahead value
* fix
* apply to methods with cheap key calculation
* more tests
* silence tidy
* fix build
* support HashMethodKeysFixed
* apply during merge only for cheap
* stash
* fixes
* rename method
* add feature flag
* cache prefetch threshold value
* fix
* fix
* Update HashMap.h
* fix typo
* 256KB as default l2 size
Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>
2022-09-21 18:59:07 +02:00
Alexander Tokmakov
fa6a4c5b78
Merge branch 'master' into another_fix_for_an_ast_optimization
2022-09-21 19:05:08 +03:00
Kruglov Pavel
22e11aef2d
Merge pull request #40910 from Avogar/new-json-formats
...
Add new JSON formats, add improvements and refactoring
2022-09-21 14:19:08 +02:00
Robert Schulze
186d4e4140
Merge pull request #40818 from Vector-Similarity-Search-for-ClickHouse/annoy
...
Add annoy index
2022-09-21 13:42:43 +02:00
Nikolay Degterinsky
7292d47923
Merge branch 'master' into mandatory-identification
2022-09-21 12:18:17 +02:00
Nikolay Degterinsky
f0ca365fa0
Better
2022-09-21 10:16:49 +00:00
Alexey Milovidov
f5f08f973e
Fix amusing error in conversion from Ordinary to Atomic
2022-09-21 07:10:33 +02:00
Nikolai Kochetov
9e1bb19559
Purge jemalloc arenas in case of high memory usage. ( #40277 )
...
* Purge jemalloc arenas in case of high memory usage.
* Purge jemalloc arenas in case of high memory usage.
* Get RSS before jemalloc counters. Try to avoid negative RSS.
* Try to avoid negative RSS.
* muzzy -> dirty
* Another fix.
* Update MemoryTracker.cpp
* Wait for purged memory.
* Revert "Wait for purged memory."
This reverts commit 53a2621a2d
.
2022-09-20 23:56:10 +02:00
Alexey Milovidov
45bd3cfc30
Merge branch 'master' into fix-three-fourth-of-trash
2022-09-20 21:27:41 +02:00
Alexander Tokmakov
f9e7451d88
Merge pull request #41343 from azat/deadlock-fix
...
Fix possible deadlock with async_socket_for_remote/use_hedged_requests and parallel KILL
2022-09-20 16:59:31 +03:00
Filatenkov Artur
2139d5be17
Merge branch 'master' into annoy
2022-09-20 15:28:08 +03:00
Duc Canh Le
c3970afc45
Merge branch 'master' into ch_fixedstring_len
2022-09-20 20:22:14 +08:00
kssenii
b8079e4577
Merge remote-tracking branch 'upstream/master' into rename-some-cache-commands
2022-09-20 13:02:18 +02:00
Alexander Tokmakov
c8f2820bf1
Merge pull request #41467 from azat/kill-query/lock-fix
...
Fix possible hung/deadlock on query cancellation
2022-09-20 13:59:55 +03:00
Vitaly Baranov
f58f8104b5
Merge pull request #41474 from vitlibar/mask-some-info-in-logs
...
Mask some information in logs.
2022-09-20 12:11:02 +02:00
Kseniia Sumarokova
3c7bbad0b0
Merge pull request #41437 from kssenii/fix-logical-error-write-through-cache
...
Fix assertion in write-through cache
2022-09-20 11:51:19 +02:00
Vladimir C
4935bf60b7
Merge pull request #41130 from ClickHouse/vdimir/decimal-in-41125
2022-09-20 10:07:50 +02:00
Frank Chen
a89140ae98
Fix style
...
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2022-09-20 10:37:54 +08:00
Azat Khuzhin
f76f14a99c
Fix possible hung/deadlock on query cancellation
...
Right due to graceful cancellation of the query, it is possible to hung
accepting new queries or even have a deadlock, this is because
cancellation is done while acquiring ProcessListBase::mutex.
So this patch makes query cancellation lock-free, and now the lock will
be acquired only for preparing the query and after cancel is done.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-09-19 17:31:21 +02:00
Amos Bird
652c7cb519
Add comments to FunctionNameNormalizer.
2022-09-19 22:07:04 +08:00
Vitaly Baranov
052ecaa211
Merge branch 'master' into improve-recovery-of-replicated-access-storage
2022-09-19 15:26:21 +02:00
Vitaly Baranov
f75dd93965
Mask some information in logs.
2022-09-19 15:00:06 +02:00
Kruglov Pavel
47f6f09ce0
Merge branch 'master' into improve-combinators
2022-09-19 14:31:12 +02:00
Kruglov Pavel
712aebc077
Merge pull request #41083 from Avogar/fix-map-combinator
...
Fix possible segfaults, use-heap-after-free and memory leak in aggregate function combinators
2022-09-19 14:29:17 +02:00
Alexander Tokmakov
c6efb1773e
Merge branch 'master' into fix_drop_partition_on_cluster
2022-09-19 15:18:05 +03:00
Yakov Olkhovskiy
2f2080f5a4
Merge branch 'master' into composable-protocol
2022-09-19 08:15:07 -04:00
kssenii
e3cd3686af
Rename
2022-09-19 14:02:51 +02:00
Kseniia Sumarokova
631aad1671
Merge branch 'master' into fix-logical-error-write-through-cache
2022-09-19 12:29:55 +02:00
vdimir
a6dc99983d
Fix tidy build
2022-09-19 08:42:23 +00:00
vdimir
5c9bfbc116
Fix incorrect result in case of decimal precision loss in IN operator
2022-09-19 08:42:22 +00:00
Vladimir C
b6b5932efc
Merge pull request #41223 from ClickHouse/vdimir/exponential_rewrite_21557
2022-09-19 10:07:24 +02:00
Duc Canh Le
61d6fae14b
add settings allow_suspicious_fixed_string_types
2022-09-19 15:00:04 +08:00
Alexey Milovidov
730655d4fd
Fix 8/9 of trash
2022-09-19 08:53:20 +02:00
Alexey Milovidov
ab4db2d0c4
Fix 5/6 of trash
2022-09-19 08:50:53 +02:00
Alexey Milovidov
3c0961c48b
Merge branch 'master' into fix-map-combinator
2022-09-19 09:32:16 +03:00
Alexey Milovidov
b01f5bdca8
Merge pull request #39354 from ClickHouse/window-order-by
...
Respect ORDER BY clause in window functions
2022-09-19 09:22:37 +03:00
Frank Chen
b056bc1021
Add span for executeDDLQueryOnCluster
...
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2022-09-19 13:43:40 +08:00
Yong Wang
37ae7a8cca
Kusto-phase1 : apply parser comments to kusto, remove unused variable
2022-09-18 20:25:27 -07:00
Frank Chen
06ae2fb2b5
Remove assertEOF to improve the compability
...
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2022-09-19 11:20:58 +08:00
Yong Wang
4e4a04c08b
Merge branch 'ClickHouse:master' into Kusto-phase1
2022-09-18 21:40:30 -04:00
Vitaly Baranov
5365b105cc
Add SYSTEM RELOAD USERS command.
2022-09-18 12:44:00 +02:00
Yakov Olkhovskiy
c66f412300
pass session certificate for showCertificate()
2022-09-18 07:11:52 +00:00
Alexey Milovidov
c41e37fa8e
Code simplification
2022-09-18 08:58:32 +02:00
Alexey Milovidov
3c43d6516b
Fix style
2022-09-18 08:56:52 +02:00
Alexey Milovidov
791de6592b
Remove trash from Field
2022-09-18 05:16:08 +02:00
kssenii
51745dcb43
Fix assertion in write-through cache
2022-09-17 20:20:32 +02:00
Anton Popov
07cfb0cba4
Merge branch 'master' into dynamic-columns-21
2022-09-17 19:02:58 +02:00
Igor Nikonov
475443c10c
Merge pull request #41257 from ClickHouse/igor/mixed_improvements_along_the_way
...
Tiny improvements along the code
2022-09-17 10:43:46 +02:00
Alexey Milovidov
c09604bef2
Merge pull request #41363 from kitaisreal/small-style-fixes
...
Small style fixes
2022-09-17 00:19:00 +03:00
avogar
0101cc2e56
Support complex combinators in window transform, arrayReduce*, initializeAggregation and Aggregate functons versionning
2022-09-16 19:07:36 +00:00
avogar
c9a9ef5b7e
Fix comments
2022-09-16 16:12:30 +00:00
Alexander Tokmakov
65474830ac
make tryEnqueueReplicatedDDL virtual
2022-09-16 16:25:32 +02:00
Robert Schulze
b32b02d844
Merge pull request #40897 from ClickHouse/catboost-bridge-resurrected
...
Move CatBoost evaluation into clickhouse-library-bridge
2022-09-16 13:12:09 +02:00
Alexey Milovidov
da01982652
Merge pull request #41046 from azat/build/llvm-15
...
Switch to llvm/clang 15
2022-09-16 07:31:06 +03:00
Alexander Tokmakov
03c193ccca
fix ON CLUSTER with Replicated database cluster
2022-09-15 21:15:57 +02:00
Alexander Tokmakov
32391f737a
Merge branch 'master' into fix_41209
2022-09-15 20:27:51 +02:00
Alexander Tokmakov
b26cc5a1ba
Merge pull request #41296 from young-scott/fix_issuse_41096
...
fix issue #41096
2022-09-15 15:44:05 +03:00
Alexander Tokmakov
77c4ea9647
fix
2022-09-15 14:34:31 +02:00
Azat Khuzhin
5cc5ca22de
Fix possible deadlock with async_socket_for_remote/use_hedged_requests and parallel KILL
...
Right now it is possible to call QueryStatus::addPipelineExecutor() when
the executors_mutex already acquired, it is possible when the query was
cancelled via KILL QUERY.
Here I will show some traces from debugger from a real example, where
tons of ProcessList::insert() got deadlocked.
Let's look at the lock owner for one of the threads that was deadlocked
in ProcessList::insert():
(gdb) p *mutex
$2 = {
__data = {
__owner = 46899,
},
}
And now let's see the stack trace of the 46899:
#0 __lll_lock_wait () at ../sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:103
#1 0x00007fb65569b714 in __GI___pthread_mutex_lock (mutex=0x7fb4a9d15298) at ../nptl/pthread_mutex_lock.c:80
#2 0x000000001b6edd91 in pthread_mutex_lock (arg=0x7fb4a9d15298) at ../src/Common/ThreadFuzzer.cpp:317
#3 std::__1::__libcpp_mutex_lock (__m=0x7fb4a9d15298) at ../contrib/libcxx/include/__threading_support:303
#4 std::__1::mutex::lock (this=0x7fb4a9d15298) at ../contrib/libcxx/src/mutex.cpp:33
#5 0x0000000014c7ae63 in std::__1::lock_guard<std::__1::mutex>::lock_guard (__m=..., this=<optimized out>) at ../contrib/libcxx/include/__mutex_base:91
#6 DB::QueryStatus::addPipelineExecutor (this=0x7fb4a9d14f90, e=0x80) at ../src/Interpreters/ProcessList.cpp:372
#7 0x0000000015bee4a7 in DB::PipelineExecutor::PipelineExecutor (this=0x7fb4b1e53618, processors=..., elem=<optimized out>) at ../src/Processors/Executors/PipelineExecutor.cpp:54
#12 std::__1::make_shared<DB::PipelineExecutor, std::__1::vector<std::__1::shared_ptr<DB::IProcessor>, std::__1::allocator<std::__1::shared_ptr<DB::IProcessor> > >&, DB::QueryStatus*&, void> (__args=@0x7fb63095b9b0: 0x7fb4a9d14f90, __args=@0x7fb63095b9b0: 0x7fb4a9d14f90) at ../contrib/libcxx/include/__memory/shared_ptr.h:963
#13 DB::QueryPipelineBuilder::execute (this=0x7fb63095b8b0) at ../src/QueryPipeline/QueryPipelineBuilder.cpp:552
#14 0x00000000158c6c27 in DB::Connection::sendExternalTablesData (this=0x7fb6545e9d98, data=...) at ../src/Client/Connection.cpp:797
#27 0x0000000014043a81 in DB::RemoteQueryExecutorRoutine::operator() (this=0x7fb63095bf20, sink=...) at ../src/QueryPipeline/RemoteQueryExecutorReadContext.cpp:46
#32 0x000000000a16dd4f in make_fcontext () at ../contrib/boost/libs/context/src/asm/make_x86_64_sysv_elf_gas.S:71
And also in the logs you can see very strange things for this thread:
2022.09.13 14:14:51.228979 [ 51145 ] {1712D4E914EC7C99} <Debug> Connection (localhost:9000): Sent data for 1 external tables, total 11 rows in 0.00046389 sec., 23688 rows/sec., 3.84 KiB (8.07 MiB/sec.), compressed 1.1070121092649958 times to 3.47 KiB (7.29 MiB/sec.)
...
2022.09.13 14:14:51.719402 [ 46899 ] {7c90ffa4-1dc8-42fd-938c-4e307c244394} <Debug> executeQuery: (from 10.101.15.181:42478) KILL QUERY WHERE query_id = '1712D4E914EC7C99' (stage: Complete)
2022.09.13 14:14:51.719488 [ 46899 ] {7c90ffa4-1dc8-42fd-938c-4e307c244394} <Debug> executeQuery: (internal) SELECT query_id, user, query FROM system.processes WHERE query_id = '1712D4E914EC7C99' (stage: Complete)
2022.09.13 14:14:51.719754 [ 46899 ] {7c90ffa4-1dc8-42fd-938c-4e307c244394} <Trace> ContextAccess (default): Access granted: SELECT(user, query_id, query) ON system.processes
2022.09.13 14:14:51.720544 [ 46899 ] {7c90ffa4-1dc8-42fd-938c-4e307c244394} <Trace> InterpreterSelectQuery: FetchColumns -> Complete
2022.09.13 14:14:53.228964 [ 46899 ] {7c90ffa4-1dc8-42fd-938c-4e307c244394} <Debug> Connection (localhost:9000): Sent data for 2 scalars, total 2 rows in 2.6838e-05 sec., 73461 rows/sec., 68.00 B (2.38 MiB/sec.), compressed 0.4594594594594595 times to 148.00 B (5.16 MiB/sec.)
How is this possible? The answer is fibers and query cancellation
routine. During cancellation of async queries it going into fibers again
and try to do this gracefully. However because of this during canceling
query it may call QueryStatus::addPipelineExecutor() from
QueryStatus::cancelQuery().
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-09-15 14:41:27 +03:00
Maksim Kita
276d3628e9
Small style fixes
2022-09-15 13:05:18 +02:00
Frank Chen
f576d8191c
Merge branch 'master' into on_cluster_dll
2022-09-15 18:05:36 +08:00
Frank Chen
ac848727e5
Read tracing context in right pos
2022-09-15 18:00:08 +08:00
Nikolay Degterinsky
72d8a04ee6
Add setting 'allow_implicit_no_password'
2022-09-15 08:07:42 +00:00
Robert Schulze
b64af5989e
Merge branch 'master' into opentelemetry_stress
2022-09-15 09:43:11 +02:00
Frank Chen
a4ef0c0281
Set up tracing context for DDLWorker
2022-09-15 14:57:00 +08:00
Frank Chen
490089cc4f
Copy tracing context from current thread to DDLLogEntry as parent context
2022-09-15 14:48:24 +08:00
Frank Chen
52224875e2
Serialize tracing context to DDL log entry
2022-09-15 14:47:43 +08:00
Anton Popov
66667737f0
fix writing of empty columns of type Object
2022-09-15 03:38:22 +00:00
avogar
f1a0501eb2
Fix memory leaks and segfaults in combinators
2022-09-14 18:01:49 +00:00
Alexander Tokmakov
fc73134145
fix race between DROP and short form of ATTACH
2022-09-14 18:03:36 +02:00
Kruglov Pavel
3396ff6c3a
Merge pull request #40516 from zjial/record_errors_for_import_by_csv
...
Record errors while reading text formats (CSV, TSV).
2022-09-14 12:52:32 +02:00
young scott
b6888022cd
Merge branch 'master' into fix_issuse_41096
2022-09-14 17:38:22 +08:00
young scott
bc111b5680
fix issuse#41096
2022-09-14 09:15:40 +00:00
Alexander Tokmakov
cf7d928554
Update PredicateRewriteVisitor.cpp
2022-09-13 22:38:55 +03:00
Alexander Tokmakov
0175f3d0d7
fix WHERE pushdown when subquery has OFFSET
2022-09-13 20:44:45 +02:00
Vladimir C
f0e4088c84
Merge pull request #41154 from azat/fix-sorted-block-writer
2022-09-13 19:14:45 +02:00
Dmitry Novik
7e3fb0a681
Merge pull request #40762 from ClickHouse/grouping-comp
...
Fix GROUPING function SQL compatibility
2022-09-13 17:03:06 +02:00
kssenii
52ef3758c4
Merge remote-tracking branch 'upstream/master' into fix-thread-status
2022-09-13 16:34:31 +02:00
Kruglov Pavel
faa62f96ae
Merge branch 'master' into new-json-formats
2022-09-13 14:41:13 +02:00
Antonio Andelic
baf7255cff
Merge pull request #39976 from ClickHouse/keeper-storage
...
KeeperMap storage engine
2022-09-13 14:15:09 +02:00
Anton Popov
1989380485
Merge pull request #41122 from lthaooo/optimize_compact_part_mutation
...
optimize compact part mutation
2022-09-13 14:10:21 +02:00
Igor Nikonov
d8acb0e7ad
Tiny improvements along the code
...
- less allocations (potentially)
2022-09-13 12:06:56 +00:00
Alexander Tokmakov
4d146b05a9
Merge pull request #38262 from PolyProgrammist/fix-ordinary-system-unfreeze
...
Fix SYSTEM UNFREEZE for ordinary database
2022-09-13 14:55:11 +03:00
Alexander Tokmakov
2845494624
Merge pull request #41120 from azat/async-insert-queue-term
...
Terminate AsynchronousInsertQueue w/o waiting for timeout
2022-09-13 14:52:50 +03:00
Kseniia Sumarokova
b2c9c04c7b
Merge pull request #40821 from kssenii/improve-marks-cache-loading
...
Allow to load marks in threadpool in advance
2022-09-13 12:31:58 +02:00
Frank Chen
d3265150c0
Make sure span is finished in the onFinish callback
...
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2022-09-13 18:28:21 +08:00
Kseniia Sumarokova
3dd28bb79a
Merge pull request #41042 from kssenii/refactor-cache-for-async-download
...
Cache refactoring for async download
2022-09-13 12:26:35 +02:00
Robert Schulze
fac1be9700
chore: restore SYSTEM RELOAD MODEL(S) and moniting view SYSTEM.MODELS
...
- This commit restores statements "SYSTEM RELOAD MODEL(S)" which provide
a mechanism to update a model explicitly. It also saves potentially
unnecessary reloads of a model from disk after it's initial load.
To keep the complexity low, the semantics of "SYSTEM RELOAD MODEL(S)
was changed from eager to lazy. This means that both statements
previously immedately reloaded the specified/all models, whereas now
the statements only trigger an unload and the first call to
catboostEvaluate() does the actual load.
- Monitoring view SYSTEM.MODELS is also restored but with some obsolete
fields removed. The view was not documented in the past and for now it
remains undocumented. The commit is thus not considered a breach of
ClickHouse's public interface.
2022-09-12 19:33:02 +00:00
Kseniia Sumarokova
5d1e9822f8
Merge branch 'master' into fix-filesystem_cache_log
2022-09-12 20:32:42 +02:00
Alexander Tokmakov
05b4303234
Merge pull request #41146 from ClickHouse/followup_39933
...
Stop background operations before converting database from Ordinary
2022-09-12 21:20:25 +03:00
Kruglov Pavel
060adfbe93
Merge branch 'master' into new-json-formats
2022-09-12 19:37:46 +02:00
Jose
1880a745b2
Fix String in log entry
2022-09-12 18:53:54 +02:00
vdimir
ba506043ce
Fix exponential query rewrite in case of lots of cross joins with where
2022-09-12 15:05:34 +00:00
Azat Khuzhin
5cb37eb1a0
Terminate AsynchronousInsertQueue w/o waiting for timeout
...
Previously it simply sleeps for async_insert_busy_timeout_ms delay on
shutdown, which is not great, and produces false-positive hung checks on
CI [1].
[1]: https://pastila.nl/?0087a100/db80915eac58e67eb05516bf459e5510
Refs: #29390 (cc @tavplubix, @CurtizJ)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-09-12 16:11:54 +02:00
kssenii
413fbb6507
Self-review fixes
2022-09-12 15:46:11 +02:00
Antonio Andelic
1cd43d8e5c
Merge branch 'master' into keeper-storage
2022-09-12 13:42:51 +00:00
Anton Popov
76af7efba9
Merge branch 'master' into optimize_compact_part_mutation
2022-09-12 15:35:47 +02:00
Dmitry Novik
c236706703
Merge branch 'master' into grouping-comp
2022-09-12 15:19:51 +02:00
Alexander Tokmakov
ed3c95c28f
review fix
2022-09-12 13:53:43 +02:00
Kseniia Sumarokova
d0c058f273
Merge pull request #41132 from Algunenano/views_log_window
...
Fix query_views_log with Window views
2022-09-12 11:56:22 +02:00
Alexey Milovidov
9c5f9f1815
Merge pull request #41158 from den-crane/bug/trivial_count_optimization_with_array_join
...
Fix: trivial count optimization with array join. revert revert #39466
2022-09-12 08:56:10 +03:00
Alexey Milovidov
94d6e9ce7d
Merge pull request #41189 from kssenii/fix-positional-args
...
Fix for positional arguments
2022-09-12 03:21:39 +03:00
Alexey Milovidov
8b5328f7e5
Merge branch 'master' into fix-trash-base58
2022-09-11 20:43:54 +02:00
Alexey Milovidov
3e6c4e3f16
Fix clang-tidy
2022-09-11 20:43:36 +02:00
Denny Crane
0e4e868f29
Merge branch 'master' into bug/trivial_count_optimization_with_array_join
2022-09-11 15:23:10 -03:00
kssenii
9c2bbcf4a6
Fix
2022-09-11 17:38:51 +02:00
kssenii
2f3bfc5411
Fix
2022-09-11 13:54:25 +02:00
kssenii
b7d751b782
Merge remote-tracking branch 'upstream/master' into improve-marks-cache-loading
2022-09-11 13:23:30 +02:00
kssenii
37bdba98b1
Fix
2022-09-11 12:58:04 +02:00
Alexey Milovidov
7f1e7b5967
Merge branch 'master' into fix-half-of-trash
2022-09-11 06:20:47 +02:00
Azat Khuzhin
ba4174402a
Temporary disable complation of compileAddIntoAggregateStatesFunctions
...
Fixes the following use-of-uninitialized-value in llvm [1]:
==696==WARNING: MemorySanitizer: use-of-uninitialized-value
0 0x498141d9 in llvm::ilist_traits<llvm::MachineInstr>::removeNodeFromList(llvm::MachineInstr*) build_docker/../contrib/llvm/llvm/lib/CodeGen/MachineBasicBlock.cpp:155:24
1 0x498141d9 in llvm::iplist_impl<llvm::simple_ilist<llvm::MachineInstr, llvm::ilist_sentinel_tracking<true>>, llvm::ilist_traits<llvm::MachineInstr>>::remove(llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::MachineInstr, true, true, void>, false, false>&) build_docker/../contrib/llvm/llvm/include/llvm/ADT/ilist.h:253:11
2 0x498141d9 in llvm::iplist_impl<llvm::simple_ilist<llvm::MachineInstr, llvm::ilist_sentinel_tracking<true>>, llvm::ilist_traits<llvm::MachineInstr>>::erase(llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::MachineInstr, true, true, void>, false, false>) build_docker/../contrib/llvm/llvm/include/llvm/ADT/ilist.h:268:22
3 0x498141d9 in llvm::iplist_impl<llvm::simple_ilist<llvm::MachineInstr, llvm::ilist_sentinel_tracking<true>>, llvm::ilist_traits<llvm::MachineInstr>>::erase(llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::MachineInstr, true, true, void>, false, false>, llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::MachineInstr, true, true, void>, false, false>) build_docker/../contrib/llvm/llvm/include/llvm/ADT/ilist.h:305:15
4 0x498141d9 in llvm::iplist_impl<llvm::simple_ilist<llvm::MachineInstr, llvm::ilist_sentinel_tracking<true>>, llvm::ilist_traits<llvm::MachineInstr>>::clear() build_docker/../contrib/llvm/llvm/include/llvm/ADT/ilist.h:309:18
5 0x498141d9 in llvm::iplist_impl<llvm::simple_ilist<llvm::MachineInstr, llvm::ilist_sentinel_tracking<true>>, llvm::ilist_traits<llvm::MachineInstr>>::~iplist_impl() build_docker/../contrib/llvm/llvm/include/llvm/ADT/ilist.h:210:20
6 0x498145ff in llvm::MachineBasicBlock::~MachineBasicBlock() build_docker/../contrib/llvm/llvm/lib/CodeGen/MachineBasicBlock.cpp:56:1
7 0x49969c1f in llvm::MachineFunction::DeleteMachineBasicBlock(llvm::MachineBasicBlock*) build_docker/../contrib/llvm/llvm/lib/CodeGen/MachineFunction.cpp:426:8
8 0x49969c1f in llvm::ilist_alloc_traits<llvm::MachineBasicBlock>::deleteNode(llvm::MachineBasicBlock*) build_docker/../contrib/llvm/llvm/lib/CodeGen/MachineFunction.cpp:127:21
9 0x4983f4d6 in llvm::iplist_impl<llvm::simple_ilist<llvm::MachineBasicBlock>, llvm::ilist_traits<llvm::MachineBasicBlock>>::erase(llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::MachineBasicBlock, false, false, void>, false, false>) build_docker/../contrib/llvm/llvm/include/llvm/ADT/ilist.h:268:11
10 0x4983f4d6 in llvm::iplist_impl<llvm::simple_ilist<llvm::MachineBasicBlock>, llvm::ilist_traits<llvm::MachineBasicBlock>>::erase(llvm::MachineBasicBlock*) build_docker/../contrib/llvm/llvm/include/llvm/ADT/ilist.h:272:39
11 0x4983f4d6 in llvm::MachineFunction::erase(llvm::MachineBasicBlock*) build_docker/../contrib/llvm/llvm/include/llvm/CodeGen/MachineFunction.h:767:53
12 0x4983f4d6 in llvm::MachineBasicBlock::eraseFromParent() build_docker/../contrib/llvm/llvm/lib/CodeGen/MachineBasicBlock.cpp:1317:16
13 0x4a0c9a27 in llvm::TailDuplicator::removeDeadBlock(llvm::MachineBasicBlock*, llvm::function_ref<void (llvm::MachineBasicBlock*)>*) build_docker/../contrib/llvm/llvm/lib/CodeGen/TailDuplicator.cpp:1051:8
14 0x4a0c1e41 in llvm::TailDuplicator::tailDuplicateAndUpdate(bool, llvm::MachineBasicBlock*, llvm::MachineBasicBlock*, llvm::SmallVectorImpl<llvm::MachineBasicBlock*>*, llvm::function_ref<void (llvm::MachineBasicBlock*)>*, llvm::SmallVectorImpl<llvm::MachineBasicBlock*>*) build_docker/../contrib/llvm/llvm/lib/CodeGen/TailDuplicator.cpp:189:5
15 0x4a0ca16e in llvm::TailDuplicator::tailDuplicateBlocks() build_docker/../contrib/llvm/llvm/lib/CodeGen/TailDuplicator.cpp:288:19
16 0x4a0be9f9 in (anonymous namespace)::TailDuplicateBase::runOnMachineFunction(llvm::MachineFunction&) build_docker/../contrib/llvm/llvm/lib/CodeGen/TailDuplication.cpp:98:21
17 0x499a2777 in llvm::MachineFunctionPass::runOnFunction(llvm::Function&) build_docker/../contrib/llvm/llvm/lib/CodeGen/MachineFunctionPass.cpp:72:13
18 0x4dbba34d in llvm::FPPassManager::runOnFunction(llvm::Function&) build_docker/../contrib/llvm/llvm/lib/IR/LegacyPassManager.cpp:1435:27
19 0x4dbe3761 in llvm::FPPassManager::runOnModule(llvm::Module&) build_docker/../contrib/llvm/llvm/lib/IR/LegacyPassManager.cpp:1481:16
20 0x4dbbebbb in (anonymous namespace)::MPPassManager::runOnModule(llvm::Module&) build_docker/../contrib/llvm/llvm/lib/IR/LegacyPassManager.cpp:1550:27
21 0x4dbbebbb in llvm::legacy::PassManagerImpl::run(llvm::Module&) build_docker/../contrib/llvm/llvm/lib/IR/LegacyPassManager.cpp:541:44
22 0x4dbe454b in llvm::legacy::PassManager::run(llvm::Module&) build_docker/../contrib/llvm/llvm/lib/IR/LegacyPassManager.cpp:1677:14
23 0x414405df in DB::JITCompiler::compile(llvm::Module&) build_docker/../src/Interpreters/JIT/CHJIT.cpp:78:22
24 0x4143bb7d in DB::CHJIT::compileModule(std::__1::unique_ptr<llvm::Module, std::__1::default_delete<llvm::Module>>) build_docker/../src/Interpreters/JIT/CHJIT.cpp:378:29
25 0x4143aded in DB::CHJIT::compileModule(std::__1::function<void (llvm::Module&)>) build_docker/../src/Interpreters/JIT/CHJIT.cpp:359:24
26 0x4147b25e in DB::compileAggregateFunctions(DB::CHJIT&, std::__1::vector<DB::AggregateFunctionWithOffset, std::__1::allocator<DB::AggregateFunctionWithOffset>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>) build_docker/../src/Interpreters/JIT/compileFunction.cpp:738:32
27 0x3de0a23a in DB::Aggregator::compileAggregateFunctionsIfNeeded()::$_0::operator()() const build_docker/../src/Interpreters/Aggregator.cpp:661:49
28 0x3de0a23a in std::__1::pair<std::__1::shared_ptr<DB::CompiledExpressionCacheEntry>, bool> DB::CacheBase<wide::integer<128ul, unsigned int>, DB::CompiledExpressionCacheEntry, UInt128Hash, DB::CompiledFunctionWeightFunction>::getOrSet<DB::Aggregator::compileAggregateFunctionsIfNeeded()::$_0>(wide::integer<128ul, unsigned int> const&, DB::Aggregator::compileAggregateFunctionsIfNeeded()::$_0&&) build_docker/../src/Common/CacheBase.h:125:24
29 0x3de0a23a in DB::Aggregator::compileAggregateFunctionsIfNeeded() build_docker/../src/Interpreters/Aggregator.cpp:657:70
Memory was marked as uninitialized
0 0xb988ded in __sanitizer_dtor_callback (/usr/bin/clickhouse+0xb988ded) (BuildId: c4a880b742797a1f37bc4f5ed869f055cc86486b)
1 0x498145da in llvm::MachineBasicBlock::~MachineBasicBlock() build_docker/../contrib/llvm/llvm/lib/CodeGen/MachineBasicBlock.cpp:56:1
2 0x49969c1f in llvm::MachineFunction::DeleteMachineBasicBlock(llvm::MachineBasicBlock*) build_docker/../contrib/llvm/llvm/lib/CodeGen/MachineFunction.cpp:426:8
3 0x49969c1f in llvm::ilist_alloc_traits<llvm::MachineBasicBlock>::deleteNode(llvm::MachineBasicBlock*) build_docker/../contrib/llvm/llvm/lib/CodeGen/MachineFunction.cpp:127:21
4 0x4983f4d6 in llvm::iplist_impl<llvm::simple_ilist<llvm::MachineBasicBlock>, llvm::ilist_traits<llvm::MachineBasicBlock>>::erase(llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::MachineBasicBlock, false, false, void>, false, false>) build_docker/../contrib/llvm/llvm/include/llvm/ADT/ilist.h:268:11
5 0x4983f4d6 in llvm::iplist_impl<llvm::simple_ilist<llvm::MachineBasicBlock>, llvm::ilist_traits<llvm::MachineBasicBlock>>::erase(llvm::MachineBasicBlock*) build_docker/../contrib/llvm/llvm/include/llvm/ADT/ilist.h:272:39
6 0x4983f4d6 in llvm::MachineFunction::erase(llvm::MachineBasicBlock*) build_docker/../contrib/llvm/llvm/include/llvm/CodeGen/MachineFunction.h:767:53
7 0x4983f4d6 in llvm::MachineBasicBlock::eraseFromParent() build_docker/../contrib/llvm/llvm/lib/CodeGen/MachineBasicBlock.cpp:1317:16
8 0x4a0c9a27 in llvm::TailDuplicator::removeDeadBlock(llvm::MachineBasicBlock*, llvm::function_ref<void (llvm::MachineBasicBlock*)>*) build_docker/../contrib/llvm/llvm/lib/CodeGen/TailDuplicator.cpp:1051:8
9 0x4a0c1e41 in llvm::TailDuplicator::tailDuplicateAndUpdate(bool, llvm::MachineBasicBlock*, llvm::MachineBasicBlock*, llvm::SmallVectorImpl<llvm::MachineBasicBlock*>*, llvm::function_ref<void (llvm::MachineBasicBlock*)>*, llvm::SmallVectorImpl<llvm::MachineBasicBlock*>*) build_docker/../contrib/llvm/llvm/lib/CodeGen/TailDuplicator.cpp:189:5
10 0x4a0ca16e in llvm::TailDuplicator::tailDuplicateBlocks() build_docker/../contrib/llvm/llvm/lib/CodeGen/TailDuplicator.cpp:288:19
11 0x4a0be9f9 in (anonymous namespace)::TailDuplicateBase::runOnMachineFunction(llvm::MachineFunction&) build_docker/../contrib/llvm/llvm/lib/CodeGen/TailDuplication.cpp:98:21
12 0x499a2777 in llvm::MachineFunctionPass::runOnFunction(llvm::Function&) build_docker/../contrib/llvm/llvm/lib/CodeGen/MachineFunctionPass.cpp:72:13
13 0x4dbba34d in llvm::FPPassManager::runOnFunction(llvm::Function&) build_docker/../contrib/llvm/llvm/lib/IR/LegacyPassManager.cpp:1435:27
14 0x4dbe3761 in llvm::FPPassManager::runOnModule(llvm::Module&) build_docker/../contrib/llvm/llvm/lib/IR/LegacyPassManager.cpp:1481:16
15 0x4dbbebbb in (anonymous namespace)::MPPassManager::runOnModule(llvm::Module&) build_docker/../contrib/llvm/llvm/lib/IR/LegacyPassManager.cpp:1550:27
16 0x4dbbebbb in llvm::legacy::PassManagerImpl::run(llvm::Module&) build_docker/../contrib/llvm/llvm/lib/IR/LegacyPassManager.cpp:541:44
17 0x4dbe454b in llvm::legacy::PassManager::run(llvm::Module&) build_docker/../contrib/llvm/llvm/lib/IR/LegacyPassManager.cpp:1677:14
18 0x414405df in DB::JITCompiler::compile(llvm::Module&) build_docker/../src/Interpreters/JIT/CHJIT.cpp:78:22
19 0x4143bb7d in DB::CHJIT::compileModule(std::__1::unique_ptr<llvm::Module, std::__1::default_delete<llvm::Module>>) build_docker/../src/Interpreters/JIT/CHJIT.cpp:378:29
20 0x4143aded in DB::CHJIT::compileModule(std::__1::function<void (llvm::Module&)>) build_docker/../src/Interpreters/JIT/CHJIT.cpp:359:24
21 0x4147b25e in DB::compileAggregateFunctions(DB::CHJIT&, std::__1::vector<DB::AggregateFunctionWithOffset, std::__1::allocator<DB::AggregateFunctionWithOffset>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>) build_docker/../src/Interpreters/JIT/compileFunction.cpp:738:32
22 0x3de0a23a in DB::Aggregator::compileAggregateFunctionsIfNeeded()::$_0::operator()() const build_docker/../src/Interpreters/Aggregator.cpp:661:49
23 0x3de0a23a in std::__1::pair<std::__1::shared_ptr<DB::CompiledExpressionCacheEntry>, bool> DB::CacheBase<wide::integer<128ul, unsigned int>, DB::CompiledExpressionCacheEntry, UInt128Hash, DB::CompiledFunctionWeightFunction>::getOrSet<DB::Aggregator::compileAggregateFunctionsIfNeeded()::$_0>(wide::integer<128ul, unsigned int> const&, DB::Aggregator::compileAggregateFunctionsIfNeeded()::$_0&&) build_docker/../src/Common/CacheBase.h:125:24
24 0x3de0a23a in DB::Aggregator::compileAggregateFunctionsIfNeeded() build_docker/../src/Interpreters/Aggregator.cpp:657:70
[1]: https://s3.amazonaws.com/clickhouse-test-reports/41046/490a2c75610c4bc3191d55226f8454b3c3d3919a/stateful_tests__msan_.html
Note, that it is safe to do, but only for this method, since it had been
disabled anyway, back in #27574 , and I guess this MSan report may be
related.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-09-10 21:38:35 +02:00
Alexey Milovidov
fd235919aa
Remove some methods
2022-09-10 05:04:40 +02:00
Alexey Milovidov
fa62c7e982
Fix half of trash
2022-09-10 04:08:16 +02:00
Igor Nikonov
726639484a
Revert "Query plan optimization setting: read in window order"
2022-09-09 22:15:38 +02:00
Denny Crane
be90cecbd2
Revert "Revert "Fix trivial count optimization with array join""
...
This reverts commit 17de7b2876
.
2022-09-09 14:56:38 -03:00
Azat Khuzhin
763bb18f98
Fix SIGSEGV in SortedBlocksWriter in case of empty block
...
CI found one issue [1].
Here is the stack trace for invalid read:
<details>
<summary>stack trace</summary>
```
0: DB::TemporaryFileLazySource::TemporaryFileLazySource(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Block const&) [inlined] std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::__is_long(this="") const at string:1445:22
1: DB::TemporaryFileLazySource::TemporaryFileLazySource(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Block const&) [inlined] std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::basic_string(this="", __str="") at string:1927
2: DB::TemporaryFileLazySource::TemporaryFileLazySource(this=0x00007f3aec105f58, path_="", header_=0x00007f38ffd93b40) at TemporaryFileLazySource.cpp:11
3: DB::SortedBlocksWriter::streamFromFile(std::__1::unique_ptr<Poco::TemporaryFile, std::__1::default_delete<Poco::TemporaryFile> > const&) const [inlined] DB::TemporaryFileLazySource* std::__1::construct_at<DB::TemporaryFileLazySource, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Block, DB::TemporaryFileLazySource*>(__args=0x00007f38ffd91560) at construct_at.h:38:50
4: DB::SortedBlocksWriter::streamFromFile(std::__1::unique_ptr<Poco::TemporaryFile, std::__1::default_delete<Poco::TemporaryFile> > const&) const [inlined] void std::__1::allocator_traits<std::__1::allocator<DB::TemporaryFileLazySource> >::construct<DB::TemporaryFileLazySource, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Block, void, void>(__args=0x00007f38ffd91560) at allocator_traits.h:298
5: DB::SortedBlocksWriter::streamFromFile(std::__1::unique_ptr<Poco::TemporaryFile, std::__1::default_delete<Poco::TemporaryFile> > const&) const [inlined] std::__1::__shared_ptr_emplace<DB::TemporaryFileLazySource, std::__1::allocator<DB::TemporaryFileLazySource> >::__shared_ptr_emplace<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Block>(this=0x00007f3aec105f40, __args=0x00007f38ffd91560) at shared_ptr.h:293
6: DB::SortedBlocksWriter::streamFromFile(std::__1::unique_ptr<Poco::TemporaryFile, std::__1::default_delete<Poco::TemporaryFile> > const&) const [inlined] std::__1::shared_ptr<DB::TemporaryFileLazySource> std::__1::allocate_shared<DB::TemporaryFileLazySource, std::__1::allocator<DB::TemporaryFileLazySource>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Block, void>(__args=<unavailable>, __args=<unavailable>) at shared_ptr.h:954
7: DB::SortedBlocksWriter::streamFromFile(std::__1::unique_ptr<Poco::TemporaryFile, std::__1::default_delete<Poco::TemporaryFile> > const&) const [inlined] std::__1::shared_ptr<DB::TemporaryFileLazySource> std::__1::make_shared<DB::TemporaryFileLazySource, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, DB::Block, void>(__args=<unavailable>, __args=<unavailable>) at shared_ptr.h:963
8: DB::SortedBlocksWriter::streamFromFile(this=<unavailable>, file=<unavailable>) const at SortedBlocksWriter.cpp:238
9: DB::SortedBlocksWriter::premerge(this=<unavailable>) at SortedBlocksWriter.cpp:209:32
```
</details>
[1]: https://s3.amazonaws.com/clickhouse-test-reports/41046/adea92f847373d1fcfd733d8979c63024f9b80bf/stress_test__asan_.html
So the problem here is that there was empty unique_ptr<> reference to
temporary file, because of empty block that accepted by
SortedBlocksWriter::insert(), but insert() is not a problem the problem
is premerge() that steals blocks from insert() and do not have check
that there are some rows. However this check exists in
SortedBlocksWriter::flush(), and in that case temporary file is not
created.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-09-09 19:52:38 +02:00
Azat Khuzhin
57146c9361
Fix typos in SortedBlocksWriter
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-09-09 19:52:38 +02:00
Anton Popov
d1fa2148de
Merge branch 'master' into dynamic-columns-14
2022-09-09 19:32:07 +02:00
Anton Popov
a1b5582a7c
Merge pull request #37152 from CurtizJ/fix-filling-of-missed-nested
...
Fix filling of missed Nested columns with multiple levels
2022-09-09 19:29:01 +02:00
Alexander Tokmakov
9bcec2d49a
stop background ops before converting db engine
2022-09-09 18:27:19 +02:00
zhenjial
51badf3d0a
fix
2022-09-10 00:08:50 +08:00
Dmitry Novik
64c4023737
Merge branch 'master' into grouping-comp
2022-09-09 18:05:01 +02:00
zhenjial
bd9fabc3f7
code optimization, add test
2022-09-09 23:27:42 +08:00
avogar
ad68b7be0f
Better
2022-09-09 15:01:45 +00:00
Kseniia Sumarokova
5bb40e5856
Change cache setting do_not_evict_index_and_mark_files default to 0. ( #41139 )
2022-09-09 15:04:32 +02:00
Antonio Andelic
32483aeec8
Merge branch 'master' into keeper-storage
2022-09-09 12:24:01 +00:00
kssenii
5cba9b789c
Merge remote-tracking branch 'upstream/master' into refactor-cache-for-async-download
2022-09-09 14:14:04 +02:00
Igor Nikonov
5379f37331
Merge pull request #41015 from ClickHouse/window_func_optimization_setting
...
Query plan optimization setting: read in window order
2022-09-09 14:00:41 +02:00
Raúl Marín
b33c1fa60f
Fix query_views_log with Window views
2022-09-09 13:46:53 +02:00
kssenii
6a885430d7
Add assertion
2022-09-09 13:36:23 +02:00
kssenii
c2c35278ba
Refactor cache for async download
2022-09-09 13:12:47 +02:00
lthaooo
d4dd6b05b2
optimize compact part mutation
2022-09-09 10:22:35 +08:00
zhenjial
469ceaa156
code optimization
2022-09-09 00:47:43 +08:00
Anton Popov
ba41239ecd
Merge remote-tracking branch 'upstream/master' into HEAD
2022-09-08 15:20:29 +00:00
Kruglov Pavel
93d38b9451
Fix style
2022-09-08 13:19:18 +02:00
Robert Schulze
c16707ff00
chore: delete obsolete modelEvaluate() function + SYSTEM.MODELS view
...
- The deleted function modelEvaluate() was superseded by
catboostEvaluate().
- Also delete the external model repository, as modelEvaluate() was it's
last user. Additionally remove the system view SYSTEM.MODELS for
inspecting the repository.
- SYSTEM RELOAD MODELS is also obsolete. HOWEVER, it was retained and
made a no-op instead of deleted.
Why?
The reason is that RBAC in distributed setups works by storing
privileges (granted and revoked) as plain SQL statements in Keeper.
Nodes read these statements at startup and parse them. If a privilege
for SYSTEM RELOAD MODELS exists but parser doesn't recognize it
nodes would fail to come up.
Considered but rejected alternatives:
- Ignore SYSTEM RELOAD MODELS during parsing RBAC privileges and
return an error for regular SYSTEM RELOAD MODELS SQL. Special-case
of no-op behavior, too brittle.
- Remove SYSTEM RELOAD MODELS manually from Keeper via command-line
manipulation of Keeper nodes or via SQL by dropping the privileges.
Needs user intervention during upgrade.
2022-09-08 09:10:11 +00:00
Anton Popov
7c12b448b8
Merge remote-tracking branch 'upstream/master' into HEAD
2022-09-08 01:52:52 +00:00
Anton Popov
5a790b15b4
try to fix filling of missed Nested columns with multiple levels
2022-09-08 00:20:11 +00:00
Igor Nikonov
e74bb00e90
Fix: EXPLAIN PLAN - make settings in SETTING clause effective
2022-09-07 19:11:25 +00:00
avogar
0475eb0b17
Make better
2022-09-07 18:53:22 +00:00
avogar
5c0c19c533
Make generic solution
2022-09-07 18:49:34 +00:00
kssenii
e51313b6b3
Get rid of static threadpools
2022-09-07 17:48:11 +02:00
avogar
9da22f7b7c
Fix possible use-heap-after-free in -Map combinator
2022-09-07 15:06:56 +00:00
Antonio Andelic
d15fdae8dc
Merge pull request #40853 from ClickHouse/embeddedrocksdb-delete-update-support
...
Support for `UPDATE` and `DELETE` in `EmbeddedRocksDB`
2022-09-07 11:03:49 +02:00
Anton Popov
f0a404e2c8
Merge remote-tracking branch 'upstream/master' into HEAD
2022-09-06 15:51:16 +00:00
kssenii
487bc0fba3
Fix heap use after free
2022-09-06 16:50:58 +02:00
Anton Popov
f76c1482bd
try to fix filling of missed Nested columns with multiple levels
2022-09-06 13:56:32 +00:00
Filatenkov Artur
7949b3fb74
Merge branch 'master' into annoy
2022-09-06 11:23:15 +03:00
Yong Wang
d5337fd813
Merge branch 'master' into Kusto-phase1
2022-09-06 00:51:39 -04:00
Alexey Milovidov
940a53e519
Merge pull request #40984 from Lucky-Chang/typo_fix
...
Fix some typos and clang-tidy warnings
2022-09-06 02:37:50 +03:00
Yong Wang
2ca54d12ee
Merge branch 'ClickHouse:master' into Kusto-phase1
2022-09-05 17:42:38 -04:00
kssenii
4c8aa04b73
Revert "Fix"
...
This reverts commit ec55c90f36
.
2022-09-05 20:12:40 +02:00
ianton-ru
39e1fc7a0f
Merge branch 'master' into fix-ordinary-system-unfreeze
2022-09-05 17:10:59 +03:00
FArthur-cmd
26eb6cdaec
refactoring
2022-09-05 13:04:42 +00:00
FArthur-cmd
9096b73084
add setting
2022-09-05 12:41:46 +00:00
Nikita Mikhaylov
33e530392e
Check grants for databases in case of renaming databases ( #40906 )
2022-09-05 12:38:44 +02:00
Antonio Andelic
e07539d1b9
Merge branch 'master' into keeper-storage
2022-09-05 09:54:13 +00:00
Luck-Chang
1ac8e739c9
fix some typos and clang-tidy warnings
2022-09-05 09:50:24 +08:00
Nikita Taranov
16af4aebc8
impl ( #40952 )
2022-09-04 16:26:31 +02:00
Alexey Milovidov
193cd1b3b2
Merge pull request #39138 from nickitat/control_block_size_in_aggregator
...
Control block size in aggregator
2022-09-04 04:51:00 +03:00
Yakov Olkhovskiy
fdcced8962
Merge pull request #40850 from canhld94/ch_canh_fix_set
...
Fix tryGetPreparedSet crash when same set expression built from different column(s)
2022-09-02 14:40:35 -04:00
kssenii
2dcf3fef94
Fix tests
2022-09-02 19:27:52 +02:00
Kseniia Sumarokova
7dea71c83f
Merge pull request #40884 from TKaxe/fix_removeUnneededColumn
...
fix bug when remove unneeded columns in subquery
2022-09-02 18:51:18 +02:00
Kseniia Sumarokova
1370bf0de7
Merge pull request #40820 from kssenii/cache-assert-directory-ownership
...
Create status file for cache directory
2022-09-02 18:42:11 +02:00
Anton Popov
9dda9658a8
Merge remote-tracking branch 'upstream/master' into HEAD
2022-09-02 12:48:27 +00:00
Vladimir C
963c0111bf
Merge pull request #39418 from vdimir/join_and_sets
...
Filter joined streams for `full_sorting_join` by each other before sorting
2022-09-02 13:57:06 +02:00
Antonio Andelic
9ca20b97c2
Merge branch 'master' into keeper-storage
2022-09-02 12:13:07 +02:00
Antonio Andelic
e64436fef3
Fix typos with new codespell
2022-09-02 08:54:48 +00:00
Robert Schulze
319d8b00a7
Merge pull request #39010 from FrankChen021/tracing_context_propagation
...
Improve the opentelemetry tracing context propagation across threads
2022-09-02 07:56:43 +02:00
Duc Canh Le
65b64010a0
move out-of-bound check into set::areTypesEqual
2022-09-02 11:56:13 +08:00
Dmitry Novik
ddadb362cf
Merge pull request #39762 from quickhouse/betterorderbyoptimization
...
Fixed `Unknown identifier (aggregate-function)` exception which appears when a user tries to calculate WINDOW ORDER BY/PARTITION BY expressions over aggregate functions
2022-09-01 18:08:06 +02:00
Frank Chen
9d63cbe811
Merge 'origin/master' into tracing_context_propagation to resolve conflicts
2022-09-01 23:18:59 +08:00
Frank Chen
1da35a486c
Disable query_span for internal queries
2022-09-01 23:16:05 +08:00
kssenii
ec55c90f36
Fix
2022-09-01 17:04:34 +02:00
Antonio Andelic
9231084c57
Style fix
2022-09-01 14:40:50 +00:00
Antonio Andelic
a0c1ce5eb7
Merge branch 'master' into embeddedrocksdb-delete-update-support
2022-09-01 14:40:24 +00:00
Vladimir C
12e6fc4182
Merge branch 'master' into join_and_sets
2022-09-01 14:56:14 +02:00
Vladimir C
f2cf7d7762
Merge pull request #40334 from ClickHouse/vdimir/analyze-stuck-limit
...
Limit number of analyze for one query, att. 2
2022-09-01 14:40:20 +02:00
Kseniia Sumarokova
c6c67a248d
Merge pull request #40792 from canhld94/ch_canh_intersect_distinct
...
Implement intersect + except distinct
2022-09-01 14:35:26 +02:00
Antonio Andelic
62ff61404f
Improve remove recursive
2022-09-01 12:16:13 +00:00
kssenii
82879543d3
Merge remote-tracking branch 'upstream/master' into improve-marks-cache-loading
2022-09-01 13:13:01 +02:00
kssenii
9ae95f8c04
Fix unit test
2022-09-01 13:11:47 +02:00
kssenii
ca4363e2e3
Merge master
2022-09-01 13:09:00 +02:00
Anton Popov
f7bdf07adc
Merge pull request #38715 from CurtizJ/fix-read-in-order-fixed-prefix
...
Better support of `optimize_read_in_order` in case of fixed prefix of sorting key
2022-09-01 12:59:18 +02:00
kssenii
0d1f7afb17
Merge remote-tracking branch 'upstream/master' into fix-removing-directories
2022-09-01 12:28:51 +02:00
Kseniia Sumarokova
849262bd99
Merge pull request #40858 from kssenii/move-cache-files
...
Move cache files from Common
2022-09-01 12:28:00 +02:00
Sema Checherinda
7b59fdc042
Merge pull request #40779 from CheSema/detached-parts-metric
...
Metric for the number of detached parts
2022-09-01 12:24:42 +02:00
luocongkai
56e4179cad
fix bug when remove unneeded columns in subquery
2022-09-01 17:10:32 +08:00
kssenii
10a7e73d52
Fix unit test
2022-08-31 21:07:39 +02:00
Robert Schulze
912663b719
Revert "Move CatBoost evaluation into clickhouse-library-bridge"
2022-08-31 20:54:43 +02:00
kssenii
0f747e7987
Move cache files
2022-08-31 17:50:55 +02:00
Antonio Andelic
f043790e6d
Add support for update
2022-08-31 14:33:53 +00:00
Robert Schulze
ca01286028
Merge pull request #39629 from ClickHouse/catboost-bridge
...
Move CatBoost evaluation into clickhouse-library-bridge
2022-08-31 16:16:11 +02:00
Antonio Andelic
0e6b3b870a
Add support for delete in RocksDB
2022-08-31 13:08:27 +00:00
vdimir
6c15db9b11
Setting max_analyze_depth forbid only second analyze
2022-08-31 11:57:41 +02:00
vdimir
60454bd722
Fix error codes in InterpreterSelectQuery.cpp
2022-08-31 11:57:41 +02:00
vdimir
f38f39ed6f
Use max_analyze_depth
2022-08-31 11:57:41 +02:00
vdimir
928adb2b38
[WIP] Limit number of analyze for one query
2022-08-31 11:57:41 +02:00
Kseniia Sumarokova
9abcfce529
Merge pull request #40759 from kssenii/fix-write-through-cache-logical-error
...
Fix logical error from write-though cache from stress test
2022-08-31 11:55:18 +02:00