Commit Graph

6492 Commits

Author SHA1 Message Date
kssenii
7beddceadd Fix bug that caused awful CPU usage 2020-10-27 10:02:27 +00:00
kssenii
cd0dbcb9d8 Minimize event loop lifetime 2020-10-27 10:01:39 +00:00
alesapin
d6130f13ef Add table to system tables 2020-10-27 10:21:49 +03:00
alexey-milovidov
30325689c4
Merge pull request #16397 from CurtizJ/fix-group-by-modifiers
Fix group by with modifiers.
2020-10-27 02:10:44 +03:00
Alexey Milovidov
59910991b6 Fix "Darwin" 2020-10-27 00:33:46 +03:00
alexey-milovidov
a34b0880c3
Update ReservoirSamplerDeterministic.h 2020-10-27 00:21:56 +03:00
Alexey Milovidov
73f3aac663 Fix assert error in quantileDeterministic #15683 2020-10-27 00:07:24 +03:00
Alexey Milovidov
cd597bf494 Merge branch 'master' into better-rethrow 2020-10-26 22:29:49 +03:00
Alexey Milovidov
740cf8ce82 Some aggregate functions were erroneously marked as case-insensitive 2020-10-26 22:20:40 +03:00
Alexey Milovidov
32ed8c9681 Fix trailing whitespace 2020-10-26 22:12:40 +03:00
Alexey Milovidov
e449266a16 Fix TSan report in lgamma 2020-10-26 22:06:30 +03:00
alesapin
880f4bbd05 System fetches 2020-10-26 19:38:35 +03:00
Alexander Kuzmenkov
5a70e26c70 Merge remote-tracking branch 'origin/master' into HEAD 2020-10-26 19:21:13 +03:00
Alexander Kuzmenkov
6552613118
Merge pull request #16280 from filimonov/clickhouse-local-tmp-folder
clickhouse-local can work without tmp directory
2020-10-26 19:10:15 +03:00
Alexander Kuzmenkov
79a6be08d1
Update src/Interpreters/Context.cpp 2020-10-26 19:09:53 +03:00
Nikolai Kochetov
42836c86e2
Merge pull request #16386 from ClickHouse/more-introspection-for-exception-double-free
Check exception message is poisoned in ExpressionActions.
2020-10-26 18:08:54 +03:00
Anton Popov
ab4c43cb81 fix group by with totals/rollup/cube modifers and min/max functions over group by keys 2020-10-26 16:48:11 +03:00
Anton Popov
5fbca7d62d
Merge pull request #16337 from kitaisreal/uuid-safe-cast-functions-added
UUID safe cast functions added
2020-10-26 16:03:27 +03:00
alesapin
9ed4668dbb Refactor common part of background list 2020-10-26 15:40:55 +03:00
Nikolai Kochetov
4e8bea397b
Update ExpressionActions.cpp 2020-10-26 15:10:16 +03:00
Nikolai Kochetov
3151de1b4e Update ColumnConst include. 2020-10-26 13:26:52 +03:00
Nikolai Kochetov
fe9db9866c Check exception message is poisoned in ExpressionActions. 2020-10-26 12:56:50 +03:00
Nikolai Kochetov
02bd35924e
Merge pull request #16242 from ClickHouse/another-one-untuple
Another one untuple implementation
2020-10-26 12:26:43 +03:00
alexey-milovidov
3aa49204d6
Update MergeTreeSettings.cpp 2020-10-26 07:39:20 +03:00
Alexey Milovidov
d5e277f080 Better exceptions rethrow 2020-10-26 06:35:57 +03:00
Alexey Milovidov
f8876fe989 Merge branch 'master' into fix-queue-processing-of-very-large-entries 2020-10-26 05:24:59 +03:00
alexey-milovidov
bfe17d7dca
Merge pull request #16358 from azat/dist-insert-fix
Fix async Distributed INSERT w/ prefer_localhost_replica=0 and internal_replication
2020-10-26 05:07:35 +03:00
tavplubix
934f64a2fd
Update DatabaseCatalog.cpp 2020-10-26 00:51:30 +03:00
Alexander Tokmakov
f8f2d62d70 minor improvements in DatabaseCatalog 2020-10-25 20:27:21 +03:00
alesapin
f97e6beb70 Remove message broker pool from context 2020-10-25 15:34:18 +03:00
Azat Khuzhin
edc8d6e5e7 Fix async Distributed INSERT w/ prefer_localhost_replica=0 and internal_replication 2020-10-25 14:27:06 +03:00
tavplubix
37de8727ea
Merge branch 'master' into fix/ISSUES-15883 2020-10-25 12:14:16 +03:00
Maksim Kita
440ae2bc57 UUID safe cast functions added
1. Added readUUIDTextImpl, readUUIDText, tryReadUUIDText functions in
ReadHelpers.
2. Added toUUIDOrNull, toUUIDOrZero functions based on ReadHelpers read
implementations.
3. Updated documentation.
2020-10-25 11:45:29 +03:00
Alexey Milovidov
22a89fca01 Fix processing very large entries in queue 2020-10-25 05:57:03 +03:00
Alexey Milovidov
a2d888f19f Fix processing very large entries in queue 2020-10-25 05:57:03 +03:00
Alexey Milovidov
2613012fd1 Improve performance of FunctionsLogical a little by adding "restrict" 2020-10-25 04:43:06 +03:00
alexey-milovidov
abb9b25fa0
Merge pull request #16343 from ClickHouse/fix-perf-test-functions-logical
Fix performance test "functions logical" after move to clang
2020-10-25 04:04:24 +03:00
alexey-milovidov
c13ab94f91
Merge pull request #16344 from ClickHouse/exclamation-marks
Remove excessive exclamation marks
2020-10-25 03:13:57 +03:00
alexey-milovidov
74b37f37fb
Merge pull request #16329 from ClickHouse/better-diagnostics-when-client-dropped-connection
Better diagnostics when client has dropped connection
2020-10-25 01:01:12 +03:00
Alexey Milovidov
8b21ef5d4f Remove excessive exclamation marks 2020-10-25 00:50:52 +03:00
Alexey Milovidov
9d50921e52 Fix performance test "functions logical" after move to clang 2020-10-24 23:55:17 +03:00
Ivan
1d170f5745
ASTTableIdentifier Part #1: improve internal representation of ASTIdentifier name (#16149)
* Use only |name_parts| as primary name source

* Restore legacy logic for table restoration

* Fix build

* Fix tests

* Add pytest server config

* Fix tests

* Fixes due to review
2020-10-24 21:46:10 +03:00
alexey-milovidov
70427bfa63
Merge pull request #16335 from kexianda/fix
minor fix.
2020-10-24 21:41:35 +03:00
alexey-milovidov
34b9d15b66
Update ThreadStatusExt.cpp 2020-10-24 21:34:54 +03:00
alexey-milovidov
e00f6c4a0d
Merge branch 'master' into total_memory_tracker-by-default 2020-10-24 21:17:53 +03:00
Nikita Mikhaylov
58b4342998
Merge pull request #15874 from nikitamikhaylov/welch-t-test
Student and Welch t-test
2020-10-24 15:13:12 +03:00
Xianda Ke
97d84dd855 minor fix.
redundant variable, duplicated defination of the variable aad_data
compiler error: variable ‘aad_value’ set but not used [-Werror=unused-but-set-variable]
2020-10-24 17:37:51 +08:00
alexey-milovidov
1a1a6350aa
Merge pull request #16285 from sundy-li/cmake-roaring
try use cmake version for croaring instead of amalgamation.sh
2020-10-24 11:07:32 +03:00
alexey-milovidov
32e0de5121
Merge pull request #16071 from hczhcz/patch-1
Change error message in function if & ifNull
2020-10-24 07:40:22 +03:00
alexey-milovidov
7e4079dca1
Merge pull request #16286 from ClickHouse/aku/error-context
Add more context to error messages
2020-10-24 07:28:36 +03:00
alexey-milovidov
c579611420
Merge pull request #16306 from azat/Set-Join-total-rows-bytes
Add total_rows/total_bytes support for Set/Join
2020-10-24 07:26:32 +03:00
Alexey Milovidov
74558a4e27 Better diagnostics when client has dropped connection 2020-10-24 06:41:47 +03:00
alexey-milovidov
6037982343
Merge pull request #16284 from ClickHouse/primary_key_without_order_by
Add ability to specify PRIMARY KEY without ORDER BY
2020-10-24 06:05:04 +03:00
sundy-li
21315127a0 remove unused codes in AggregateFunctionGroupBitmapData 2020-10-24 10:05:54 +08:00
alexey-milovidov
91dc8d4cb3
Merge pull request #16264 from amosbird/ssmfix
Fix TwoLevelStringHashTable bug
2020-10-24 01:00:27 +03:00
Alexey Milovidov
13b8dce792 Update contributors 2020-10-24 00:26:22 +03:00
tavplubix
790950ea56
Merge pull request #15934 from ClickHouse/ddl_queries_fixes
Fixes for DDL queries
2020-10-23 23:52:13 +03:00
nikitamikhaylov
a949293f95 better 2020-10-23 23:27:38 +03:00
nikitamikhaylov
0064a75916 Merge branch 'master' of github.com:ClickHouse/ClickHouse into welch-t-test 2020-10-23 23:06:15 +03:00
tavplubix
587f83b2ef
Merge branch 'master' into atomic-drop-database-sync 2020-10-23 21:54:26 +03:00
Azat Khuzhin
4d1385ef19 Add total_rows/total_bytes support for Set/Join 2020-10-23 21:11:55 +03:00
Azat Khuzhin
0cccf3049a Fix parent memory tracker during query detaching 2020-10-23 21:07:53 +03:00
Azat Khuzhin
114ea9b1eb Fix accounting for new/delete from different threads for VariableContext::Thread
MemoryTracker assumes that for VariableContext::Thread new/delete may be
called from different threads, hence the amount of memory can go
negative.

However the MemoryTracker is nested, so even if the negative amount is
allowed for VariableContext::Thread it does not allowed for anything
upper, and hence the MemoryTracking will not be decremented properly.

Fix this, by passing initial size to the parent free.

This should fix memory drift for HTTP queries.
2020-10-23 21:07:52 +03:00
Azat Khuzhin
72d7b6117e Use total_memory_tracker when there is no other MemoryTracker object.
This should significantly reduce the MemoryTracking drift, test shows
that there is 0 drift after query storm (100 queries, via http/tcp/tcp
in one session).

TL;DR;

To track memory, clickhouse creates memory tracker object for each
thread **explicitly**, but until it is not created the memory
allocations are not under account.
There should not be lot of allocations w/o memory tracker, since most of
the time it is created early enough, but even this maybe enough to
trigger some problems.

Plus sometimes it is not possible to create it, for example some 3d
party library does not allow to do this explicitly:
- for example before #15740 allocations from librdkafka threads,
- or even worse, poco threads, they don't have any routines to do this.
This won't be a problem for `MemoryTracking` metric if the deallocation
will be done from the same thread w/o memory tracker (or vise versa),
but this is not always true.

NOTE, that this will slow down per-thread allocations w/o memory
tracker, since before this patch there were no memory tracking for them
while now they will be accounted in total_memory_tracker, and for
total_memory_tracker max_untracked_memory is always reached.
But this should not be significant.
2020-10-23 21:07:52 +03:00
alexey-milovidov
840d96255e
Merge pull request #16199 from azat/clang10-build-fix
Link dbms w/ atomic to fix undefined reference in unbundled build
2020-10-23 20:17:37 +03:00
Nikolai Kochetov
c808ab878b Review fixed. 2020-10-23 20:15:23 +03:00
Nikolai Kochetov
a2014378fe Remove has_untuple flag 2020-10-23 19:57:22 +03:00
Nikolai Kochetov
5c791edd43 Remove ColumnNameCache 2020-10-23 19:56:11 +03:00
alesapin
bc331a8029 Better set 2020-10-23 19:34:41 +03:00
alesapin
b28894c43b More compatible way 2020-10-23 19:31:51 +03:00
Mikhail Filimonov
f3a18864d5
After CR fixes - better & simpler. Issue in Context::shutdown fixed 2020-10-23 16:57:06 +02:00
Alexander Kuzmenkov
77654eeee8 fixup 2020-10-23 17:35:03 +03:00
Alexander Kuzmenkov
b0a14a41c3 fixup 2020-10-23 17:28:55 +03:00
tavplubix
84b453f4be
Update InterpreterDropQuery.cpp 2020-10-23 17:19:02 +03:00
Alexander Tokmakov
337fe7b81f add comment 2020-10-23 15:53:58 +03:00
Nikita Mikhaylov
cb4945d8b0
Merge pull request #16205 from azat/do-not-cache-dictGet
Do not cache dictionary for dictGet*/dictHas*
2020-10-23 15:47:11 +03:00
Alexander Kuzmenkov
d17a49f127 Add more context to error messages 2020-10-23 15:20:07 +03:00
Alexander Kuzmenkov
28c9e66dc1
Merge pull request #15978 from ucasFL/interval-op
INTERVAL operator support string literal
2020-10-23 14:44:05 +03:00
sundy-li
f407504a7a try use cmake version for croaring instead of amalgamation.sh 2020-10-23 19:42:21 +08:00
alesapin
59ca0d071c Fix comment 2020-10-23 14:17:37 +03:00
alesapin
69f2f9838e Add ability to specify PRIMARY KEY without ORDER BY 2020-10-23 14:16:27 +03:00
alesapin
033525e27e
Merge pull request #16267 from den-crane/patch-6
Fix for incorrect error message
2020-10-23 10:43:34 +03:00
alexey-milovidov
121ebfdc81
Merge pull request #16249 from traceon/access-storage-added-log
Add a log message after an access storage is added
2020-10-23 09:17:10 +03:00
alexey-milovidov
6cd7d46f93
Update AccessControlManager.cpp 2020-10-23 09:13:26 +03:00
Denny Crane
b310d0abb4
Fix for incorrect error message
max_[table/partition]_size_to_drop can be changed without restart since https://github.com/ClickHouse/ClickHouse/pull/7779
2020-10-22 15:22:17 -03:00
Alexander Kuzmenkov
5cbf645de4 Merge remote-tracking branch 'origin/master' into tmp 2020-10-22 19:48:02 +03:00
Denis Glazachev
a6439aba44 More specific log messages for each access storage type 2020-10-22 20:47:52 +04:00
Alexander Kuzmenkov
145e2b012f cleanup 2020-10-22 19:47:20 +03:00
Amos Bird
ccf59c6412
Fix definitely wrong bug 2020-10-22 22:56:45 +08:00
alesapin
1d07ece5d1 Fix clang tidy warning 2020-10-22 17:15:56 +03:00
alesapin
4b65d5469f Fix some unrelated performance issues in select parts for merge 2020-10-22 15:41:01 +03:00
Nikolai Kochetov
0902bc96cf Fix build 2020-10-22 14:01:26 +03:00
Nikolai Kochetov
625af670da Fix tests. 2020-10-22 12:47:24 +03:00
Nikolai Kochetov
48142df26c Fix tests. 2020-10-22 11:25:24 +03:00
alesapin
945dd3aa59 Merge branch 'master' into blind_fix_of_read_in_order 2020-10-22 10:37:37 +03:00
Nikolai Kochetov
4a250ef9c6 Fix tests. 2020-10-22 09:58:20 +03:00
alexey-milovidov
f351b52851
Update AccessControlManager.cpp 2020-10-22 07:03:17 +03:00
alexey-milovidov
0433bc2034
Merge pull request #16248 from azat/nullptr_t-build-fix
Use std::nullptr_t over nullptr_t (build fix)
2020-10-22 07:02:38 +03:00
alexey-milovidov
adeba6bdd8
Merge pull request #15074 from amosbird/btc
Extend trivial count optimization.
2020-10-22 02:50:57 +03:00
Denis Glazachev
691b28e98b Add a log message after a storage is added 2020-10-22 02:43:02 +04:00
Azat Khuzhin
6f78243e32 Use std::nullptr_t over nullptr_t 2020-10-22 01:21:06 +03:00
alexey-milovidov
cfaf82ba0c
Merge pull request #16135 from hexiaoting/dev_rankCorr
Fix inconsistent behaviour of rankCorr function
2020-10-22 01:15:50 +03:00
alexey-milovidov
222a4d2e9b
Update AggregateFunctionRankCorrelation.h 2020-10-22 01:15:24 +03:00
alexey-milovidov
2ae32e3d5c
Merge pull request #16101 from azat/http-memory-tracking-fixes
Fix memory tracker settings (for HTTP queries)
2020-10-22 00:00:47 +03:00
Nikolai Kochetov
69279e6d76 Fix tests. 2020-10-21 23:05:10 +03:00
tavplubix
3c53d47840
Merge pull request #16189 from ClickHouse/fix_assertion_table_function_file
Fix assertion in table function file()
2020-10-21 22:38:24 +03:00
Azat Khuzhin
60aae56266 Link dbms w/ atomic to fix undefined reference in unbundled build
This will fix undefined reference for __atomic_load under clang10 and
unbundled build (libstdc++ shared linkage):

    clickhouse-server: symbol lookup error: /src/ch/clickhouse/.cmake/src/libclickhouse_disksd.so: undefined symbol: __atomic_load

From [1]:

  "libatomic (GNU)

   libgcc_s does not provide an implementation of an atomics library.
   Instead, GCC’s libatomic library can be used to supply these when using
   libgcc_s.

   Note

   Clang does not currently automatically link against libatomic when using
   libgcc_s. You may need to manually add -latomic to support this
   configuration when using non-native atomic operations (if you see link
   errors referring to __atomic_* functions)."

  [1]: https://clang.llvm.org/docs/Toolchain.html
2020-10-21 22:33:53 +03:00
Alexander Tokmakov
ac42199ae5 fix race on StorageReplicatedMergeTree::data_parts_exchange_endpoint 2020-10-21 22:25:22 +03:00
Nikolai Kochetov
fd77ca8251 Fix tests. 2020-10-21 22:20:52 +03:00
Nikolai Kochetov
69ae6ace47 Add untuple. 2020-10-21 21:17:27 +03:00
Nikolai Kochetov
a0da588f06 Merge branch 'master' into new-block-for-functions-4 2020-10-21 16:16:58 +03:00
Nikita Mikhaylov
ef5dd73ae0
Merge pull request #16155 from nikitamikhaylov/vladimir-golovchenko-fix-IPvXCIDRToRange
Merging #15856
2020-10-21 15:01:30 +03:00
alexey-milovidov
0b7430dda1
Merge pull request #16206 from ClickHouse/fix-memory-tracking
Fix multiple issues with memory tracking
2020-10-21 14:34:22 +03:00
alesapin
c53f59dece
Merge pull request #15984 from ClickHouse/fix_insert_mv_dangling_ref
Fix possibly dangling reference to target table of MV
2020-10-21 12:13:51 +03:00
alesapin
b45d42bb9c Maybe devirtualization can help 2020-10-21 11:35:36 +03:00
Nikolai Kochetov
42a90cf533 Fix special build. 2020-10-21 11:32:57 +03:00
Nikolai Kochetov
c4c1446111 Fix build. 2020-10-21 09:22:20 +03:00
Nikolai Kochetov
68c9a57788 Merge branch 'master' into new-block-for-functions-4 2020-10-21 09:17:25 +03:00
alexey-milovidov
a7098c2557
Merge pull request #16154 from nvartolomei/nv/max_concurrent_queries_for_all_users
Add max_concurrent_queries_for_all_users setting
2020-10-21 04:35:57 +03:00
alexey-milovidov
2913a10fd3
Merge pull request #16198 from azat/fix-comment-in-query_log
Fix query comments in query_log and server log (w/ enable_global_with_statement)
2020-10-21 03:40:48 +03:00
alexey-milovidov
db481a33bd
Merge pull request #14135 from ClickHouse/simplify-init-script
Simplify init script
2020-10-21 03:37:50 +03:00
Alexey Milovidov
2619efadc8 Fix multiple issues with memory tracking 2020-10-21 03:31:34 +03:00
Nikolai Kochetov
6177996d5b Fix gcc build. 2020-10-21 00:27:01 +03:00
Azat Khuzhin
3559e3355c Do not cache dictionary for dictGet*/dictHas*
There are places where ExpressionActionsPtr is cached
(StorageDistributed caching it for sharding_key_expr and
optimize_skip_unused_shards), and if the dictionary will be cached
within "query" then cached ExpressionActionsPtr will always have first
version of the query and the dictionary will not be updated after
reload.

For example this will fix dictGet in sharding_key (and similar places,
i.e. when the function context is stored permanently)

Fixes: 01527_dist_sharding_key_dictGet_reload
2020-10-21 00:22:40 +03:00
alexey-milovidov
8084ce75cb
Merge pull request #16105 from azat/allow_nondeterministic_optimize_skip_unused_shards
Add allow_nondeterministic_optimize_skip_unused_shards
2020-10-21 00:16:26 +03:00
alesapin
8097c696de Trying another fix 2020-10-20 23:05:20 +03:00
alexey-milovidov
2b00b5391a
Merge pull request #11844 from Enmk/AES_encrypt_decrypt
encrypt and decrypt functions
2020-10-20 22:57:35 +03:00
alexey-milovidov
8998829f66
Merge pull request #15685 from vivarum/enable-parsing-of-input-enum-values-by-id-10682
Enable parsing enum values by their ids for CSV, TSV and JSON input formats
2020-10-20 22:52:57 +03:00
Azat Khuzhin
e76066718d Fix query comments in query_log and server log (w/ enable_global_with_statement)
Plus, I guess that enable_global_with_statement will be enabled by
default someday, so it may become significant.

The enable_global_with_statement had been introduced in #15451

Cc: @amosbird
2020-10-20 21:10:36 +03:00
Alexey Milovidov
70bf613a28 Merge branch 'simplify-init-script' of github.com:yandex/ClickHouse into simplify-init-script 2020-10-20 21:10:01 +03:00
Alexey Milovidov
1ab9c7fbb7 Merge branch 'master' into simplify-init-script 2020-10-20 21:08:42 +03:00
alexey-milovidov
75a02112bc
Merge pull request #15592 from sundy-li/readonly-metrics
sub ReadonlyReplica metric when detach readonly tables
2020-10-20 19:17:52 +03:00
Nikolai Kochetov
19ccce0305 Fix gcc build. 2020-10-20 18:56:05 +03:00
alesapin
439bbd9861 Blind performance fix 2020-10-20 18:30:46 +03:00
Vladimir Chebotarev
aa5f207fd4
Added disable_merges option for volumes in multi-disk configuration (#13956)
Co-authored-by: Alexander Kazakov <Akazz@users.noreply.github.com>
2020-10-20 18:10:24 +03:00
Nikolai Kochetov
395b9293d5 Remove double space. 2020-10-20 17:03:54 +03:00
nikitamikhaylov
f33ae7f167 style 2020-10-20 16:34:57 +03:00
Nikolai Kochetov
a1d5adf8d5 Merge branch 'master' into new-block-for-functions-4 2020-10-20 16:16:08 +03:00
Nikolai Kochetov
295e612343 Fix build and tests. 2020-10-20 16:11:57 +03:00
nikitamikhaylov
4ce2670ab1 Merge branch 'master' of github.com:ClickHouse/ClickHouse into welch-t-test 2020-10-20 15:59:24 +03:00
Alexander Tokmakov
62a0ee8255 fix assertion in table function file() 2020-10-20 15:29:46 +03:00
vladimir golovchenko
a64bc6c586 Fixed IPv4CIDRToRange/IPv6CIDRToRange functions to pass Const IP address. 2020-10-20 15:06:22 +03:00
Alexander Kuzmenkov
84908df6d8 fix setting parent from HTTP traceparent 2020-10-20 14:35:13 +03:00
Nicolae Vartolomei
65013fcbd1 Explain why max_concurrent_queries_for_all_users exists 2020-10-20 12:00:04 +01:00
alexey-milovidov
b4f0e08369
Merge pull request #16158 from ClickHouse/minimal-ui
Added minimal web UI
2020-10-20 11:57:56 +03:00
Alexander Kuzmenkov
f3abb76e57 disable traceparent header in Arcadia
it interferes with the test_clickhouse.TestTracing.test_tracing_via_http_proxy[traceparent] test
2020-10-20 11:13:21 +03:00
Alexey Milovidov
448c87363c Style 2020-10-20 04:29:34 +03:00
Alexey Milovidov
94592819ad Add comment 2020-10-20 04:29:13 +03:00
alexey-milovidov
5e46e7c670
Merge pull request #15723 from Jokser/idempotent-part-move
Possibility to move part to another disk/volume if first attempt was failed
2020-10-20 04:26:50 +03:00
alexey-milovidov
124379cccc
Update IMergeTreeDataPart.h 2020-10-20 04:24:30 +03:00