thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-18 13:42:02 +00:00
Code Issues Packages Projects Releases Wiki Activity
49,481 Commits 9,138 Branches 1,774 Tags 2 GiB
0de364ec94
Commit Graph

6034 Commits

Author SHA1 Message Date
Alexey Milovidov
7541be86cd Fix "Arcadia" 2020-10-10 23:26:36 +03:00
Nikolai Kochetov
3f446b9713 Fix build. 2020-10-10 22:19:46 +03:00
Nikolai Kochetov
780bd424ad Fix build. 2020-10-10 22:11:39 +03:00
Nikolai Kochetov
3099d44a96 Merge branch 'master' into new-block-for-functions-2 2020-10-10 22:09:50 +03:00
Nikolai Kochetov
6d8dc1f8ed Fix style. 2020-10-10 21:44:51 +03:00
Alexey Milovidov
269b6383f5 Check for #pragma once in headers 2020-10-10 21:37:02 +03:00
Nikolai Kochetov
d28325a353 Replace getByPosition to [] 2020-10-10 21:24:57 +03:00
Alexey Milovidov
f42b471de8 Moved functions to separate files; remove copy-paste 2020-10-10 21:23:54 +03:00
Alexey Milovidov
9ca9855f96 Merge branch 'master' into filipecaixeta-master 2020-10-10 20:52:34 +03:00
Alexey Milovidov
7502fad77c Change level of some log messages 2020-10-10 20:47:34 +03:00
alexey-milovidov
1f6c2c8efc
Merge pull request #15803 from ucasFL/tid-function 
Add `tid` and `logTrace` function
 2020-10-10 19:37:04 +03:00
Alexey Milovidov
5b482f4191 Cleanups 2020-10-10 19:31:10 +03:00
Alexey Milovidov
06fcf437fe Addition to prev. revision 2020-10-10 19:02:13 +03:00
Alexey Milovidov
09a31e01e4 Addition to prev. revision 2020-10-10 18:55:42 +03:00
Alexey Milovidov
9f15daf45c Remove useless metrics to avoid confusion 2020-10-10 18:52:44 +03:00
Alexey Milovidov
a2ef0f6cba Minor changes in BigInt hash 2020-10-10 18:46:41 +03:00
robot-clickhouse
554fa482e6 Auto version update to [20.11.1.1] [54442] 2020-10-10 18:39:58 +03:00
Amos Bird
8e0862a0c3
BlockIO can be empty. 2020-10-10 23:25:50 +08:00
feng lv
9438f2f4d2 fix 
fix

fix
 2020-10-10 19:21:11 +08:00
Alexey Milovidov
4f2d4988c6 Fix build on Mac OS 2020-10-10 13:51:57 +03:00
Amos Bird
dbc14655fc
Code refactoring. 2020-10-10 18:22:33 +08:00
Alexey Milovidov
2c71327ff5 Fix build on Mac OS 2020-10-10 11:02:02 +03:00
alexey-milovidov
59cfb87a96
Merge pull request #15799 from ClickHouse/fix-links-in-docs 
Fix ugliness in docs
 2020-10-10 10:48:11 +03:00
Nikolai Kochetov
76a04fb4b4
Merge pull request #15762 from ClickHouse/new-block-for-functions 
Use `ColumnsWithTypeAndName` instead of `Block` for function calls
 2020-10-10 08:50:38 +03:00
feng lv
45925f60c7 fix 2020-10-10 11:20:45 +08:00
feng lv
d39868b853 fix 2020-10-10 10:56:21 +08:00
feng lv
ec87254abe add test 2020-10-10 10:48:15 +08:00
Nikita Mikhaylov
6ac5431804
Merge pull request #15783 from nikitamikhaylov/copier-backquotes 
Merging  #14939
 2020-10-10 02:32:27 +03:00
alexey-milovidov
9d02347e2e
Update FunctionsFormatting.h 2020-10-09 22:55:02 +03:00
alexey-milovidov
0fd007ad9e
Merge pull request #15601 from alexelex/alexelex-master 
Improvement of Quorum Inserts in ClickHouse
 2020-10-09 22:43:53 +03:00
Alexey Milovidov
bd805300bb Whitespace 2020-10-09 22:33:23 +03:00
Azat Khuzhin
064f901ea8 Add ability to preallocate hashtables for hashed/sparsehashed dictionaries 
preallocation can be used only when we know number of rows, and for this
we need:
- source clickhouse
- no filtering (i.e. lack of <where>), since filtering can filter
  too much rows and eventually it may allocate memory that will
  never be used.

For sparse_hash the difference is quite significant, preallocated
sparse_hash hashtable allocates ~33% faster (7.5 seconds vs 5 seconds
for insert, and the difference is more significant for higher number of
elements):

    $ ninja bench-sparse_hash-run
    [1/1] cd /src/ch/hashtable-bench/.cmake && ...ch/hashtable-bench/.cmake/bench-sparse_hash
    sparse_hash/insert: 7.574 <!--
    sparse_hash/find  : 2.14426
    sparse_hash/maxrss: 174MiB
    sparse_hash/time:   9710.51 msec (user+sys)

    $ time ninja bench-sparse_hash-preallocate-run
    [1/1] cd /src/ch/hashtable-bench/.cmake && ...-bench/.cmake/bench-sparse_hash-preallocate
    sparse_hash/insert: 5.0522 <!--
    sparse_hash/find  : 2.14024
    sparse_hash/maxrss: 174MiB
    sparse_hash/time:   7192.06 msec (user+sys)

P.S. the difference for sparse_hashed dictionary with 4e9 elements
(uint64, uint16) is ~18% (4975.905 vs 4103.569 sec)

v2: do not reallocate the dictionary from the progress callback
    Since this will access hashtable in parallel.
v3: drop PREALLOCATE() and do this only for source=clickhouse and empty
    <where>
 2020-10-09 22:28:14 +03:00
Nikolai Kochetov
ef3640c3f7
Merge pull request #15451 from amosbird/scopewith 
Global WITH and WITH propagation.
 2020-10-09 22:16:45 +03:00
Azat Khuzhin
f4f79aa84a Add HashTable::reserve() (for preallocated hashed/sparse_hashed dictionaries) 2020-10-09 21:53:57 +03:00
filipe
09b5e1dbc3 fix FunctionsFormatting error for ARM 2020-10-09 13:46:59 -03:00
Nikolai Kochetov
3898a8ac5f Try fix tests. 2020-10-09 19:05:11 +03:00
feng lv
8a1a41f77b add tid function 2020-10-09 23:49:14 +08:00
alesapin
35165c8ea4
Merge pull request #15756 from CurtizJ/fix-prewhere-with-virtuals 
Fix prewhere with virtual columns
 2020-10-09 18:07:41 +03:00
nikitamikhaylov
88a5302759 Merge branch 'master' of github.com:ClickHouse/ClickHouse into copier-backquotes 2020-10-09 16:26:31 +03:00
Alexandra Latysheva
f6f33168aa fix compilation error 2020-10-09 12:11:21 +00:00
Nikolai Kochetov
9328cea4f1 Supress pvs 2020-10-09 14:35:58 +03:00
Alexandra Latysheva
0594a77b57 fix thread restart for parallel quorum inserts 2020-10-09 11:20:20 +00:00
Nikolai Kochetov
a7fb2e38a5 Use ColumnWithTypeAndName as function argument instead of Block. 2020-10-09 10:41:28 +03:00
filipe
3861a03927 fix pvs check 2020-10-08 20:14:54 -03:00
Alexander Tokmakov
a6178cc876 try fix select count() for MaterializeMySQL 2020-10-08 23:39:24 +03:00
tavplubix
7b5f2b2bb0
Merge pull request #15701 from ClickHouse/better_clone_replica 
Improvements in lost replica recovery
 2020-10-08 23:19:23 +03:00
alexey-milovidov
c693c653a5
Merge pull request #15639 from GrigoryPervakov/tmp-s3-merge 
Use tmp disk for vertical merge files
 2020-10-08 20:58:07 +03:00
Alexandra Latysheva
6877f4aa9c for parallel and non-parallel quorum insertion at the same time 2020-10-08 15:35:41 +00:00
tavplubix
6065a5b65c
Update StorageReplicatedMergeTree.cpp 2020-10-08 16:37:32 +03:00
alexey-milovidov
8ae2bc06a1
Merge pull request #15378 from amosbird/realfix 
Fix multiple column transformers.
 2020-10-08 16:26:48 +03:00
Anton Popov
4bc40421e2
Merge pull request #15731 from CurtizJ/fix-tranform-null-in 
Fix assertion with setting transform_null_in
 2020-10-08 16:12:56 +03:00
Anton Popov
1a6d2bc538 fix prewhere with virtuals columns 2020-10-08 15:59:47 +03:00
Pervakov Grigorii
3a2068f19d Use tmp disk for vertical merge files 2020-10-08 13:49:41 +03:00
alexey-milovidov
9b9a5d9dc8
Merge pull request #15643 from azat/remove-obsolete-settings 
Remove obsolete settings
 2020-10-08 13:30:36 +03:00
alexey-milovidov
342f3b07dc
Merge pull request #15737 from azat/profile-events-fixes 
Fix RealTimeMicroseconds ProfileEvents
 2020-10-08 13:14:15 +03:00
alexey-milovidov
c0cb18b445
Merge pull request #15740 from azat/kafka-memory-accounting 
Take memory allocations from librdkafka threads into account
 2020-10-08 13:12:54 +03:00
Amos Bird
c9ab87ed17
Fix explain syntax 2020-10-08 17:06:04 +08:00
Amos Bird
ac102540d0
Global WITH and WITH propagation. 2020-10-08 15:03:34 +08:00
Azat Khuzhin
1f51de362f Take memory allocations from librdkafka threads into account 2020-10-07 23:18:20 +03:00
alexey-milovidov
e465ce3d49
Merge pull request #15642 from azat/force_data_skipping_indices 
Implement force_data_skipping_indices setting
 2020-10-07 23:14:51 +03:00
Alexander Tokmakov
ffb19fa4bc fix 2020-10-07 22:47:31 +03:00
Azat Khuzhin
3cd71f3357 Cleanup interceptors for librdkafka in StorageKafka 
Wrap them into StorageKafkaInterceptors to allow access to private
fields and add logging inside interceptors if something fails.

This is also preparation for ThreadStatus interceptor.
 2020-10-07 21:42:13 +03:00
Azat Khuzhin
c0e15ba348 Fix RealTimeMicroseconds ProfileEvents 
RUsageCounters::real_time (RealTimeMicroseconds) was constructed from
CLOCK_REALTIME (or similar) in the ThreadStatus::initPerformanceCounters()
(ThreadStatusExt.cpp), while the intention is to store CLOCK_MONOTONIC
(time from boot, i.e. /proc/uptime) values there (in ThreadProfileEvents.h)
 2020-10-07 21:32:36 +03:00
Azat Khuzhin
553f6aa4cd Remove unused functions 2020-10-07 21:30:53 +03:00
alesapin
d093d4c535
Merge pull request #15675 from ClickHouse/fix_races_in_hdfs 
Fix races in libhdfs
 2020-10-07 20:39:40 +03:00
Artem Zuikov
eb1cb506c9
minor code improvement (#15668) 2020-10-07 20:20:07 +03:00
Nikolai Kochetov
54e77e906e
Merge pull request #15717 from ClickHouse/fix-array-join-optimisation-for-mv 
Fix array join optimisation for MV
 2020-10-07 20:19:35 +03:00
Anton Popov
cc451e41f5 fix assertion with transform_null_in 2020-10-07 18:36:34 +03:00
Latysheva Alexandra
f43ad338ed
Merge pull request #12 from alexelex/quorum_inserts_2 
style check
 2020-10-07 18:56:14 +06:00
Alexandra Latysheva
8f8872dc42 style check 2020-10-07 12:24:46 +00:00
Latysheva Alexandra
6c26f77380
Merge pull request #11 from alexelex/quorum_inserts_2 
changing the condition for updateQuorum, specialized it and reduced t…
 2020-10-07 17:49:20 +06:00
Artem Zuikov
403a5320f5
enable bigint float division (#15691) 2020-10-07 14:32:57 +03:00
Alexandra Latysheva
a43cac2c1a changing the condition for updateQuorum, specialized it and reduced the number of calls to the zk 2020-10-07 11:28:48 +00:00
tavplubix
041015545c
Merge pull request #15537 from ClickHouse/mutation_hangs_after_replace_partition 
Fix mutation may hang after REPLACE/DROP PARTITION
 2020-10-07 14:25:29 +03:00
Nikolai Kochetov
df02573c92 Fix ARRAY JOIN optimisation when reading from MV. 2020-10-07 13:11:58 +03:00
Nikolai Kochetov
5f7aeddfe5
Merge pull request #15682 from ClickHouse/akz/storage_memory_less_locks 
Less locking in StorageMemory: totalRows + totalBytes
 2020-10-07 12:59:20 +03:00
alexey-milovidov
e00961f882
Merge pull request #15657 from Jokser/disk-s3-write-error-handling 
Proper error handling during insert into MergeTree with S3
 2020-10-07 11:17:32 +03:00
Alexander Tokmakov
47e9fe9ff1 fix 2020-10-07 03:05:48 +03:00
Alexander Tokmakov
ea16370024 improvements in lost replica recovery 2020-10-07 02:29:33 +03:00
Azat Khuzhin
75e612fc16 Use full featured parser for force_data_skipping_indices 2020-10-07 01:44:14 +03:00
Azat Khuzhin
ef6d12967f Implement force_data_skipping_indices setting 2020-10-07 01:42:31 +03:00
Latysheva Alexandra
991e51a80e
Merge pull request #10 from alexelex/quorum_inserts_2 
style check fix
 2020-10-07 04:38:30 +06:00
Alexandra Latysheva
9077c92862 style check fix 2020-10-06 22:36:42 +00:00
Latysheva Alexandra
85bcc07c0b
Merge pull request #9 from alexelex/quorum_inserts_2 
Quorum inserts [version 3]
 2020-10-07 04:04:41 +06:00
Alexandra Latysheva
644091459f style fixes and remove small changes 2020-10-06 22:01:30 +00:00
Azat Khuzhin
675c70acbc Remove obsolete settings 
Came to this when looking at allow_experimental_data_skipping_indices,
for implementing force_data_skipping_indices
 2020-10-07 00:56:22 +03:00
Alexandra Latysheva
f549ecf9d1 insert_quorum_parallel: status nodes in /quorum/parallel and allow merges 2020-10-06 21:49:48 +00:00
Nikolai Kochetov
1f2e867ce6
Merge pull request #15645 from ClickHouse/read-from-storage-step-field-order 
Ensure resource destruction order in ReadFromStorageStep.
 2020-10-06 19:17:38 +03:00
Alexander Kazakov
cabab90ab2 Get back to moving args 2020-10-06 17:54:22 +03:00
Alexander Kazakov
ae2c106f94 Minor fixes 2020-10-06 17:04:08 +03:00
Alexander Kazakov
e54ff6e60a Lock-less totalRows/totalBytes + more clear postponed init 2020-10-06 16:45:17 +03:00
alesapin
00f29e400f Fix race condition in hdfs 2020-10-06 16:29:08 +03:00
Azat Khuzhin
9594e463b4
Cancel other processors on query exception (#15578) 
* Add a test for canceling distributed queries on error

* Cancel other processors on exception in PipelineExecutor

Fixes: 01514_distributed_cancel_query_on_error
 2020-10-06 14:24:55 +03:00
Nikolai Kochetov
3564ba1c63 Remove moves. 2020-10-06 13:02:07 +03:00
Pavel Kovalenko
69b4bc6f33 Proper error handling during insert into MergeTree with S3. 2020-10-06 12:38:00 +03:00
Nikolai Kochetov
8320efc189 Fix read from Buffer table with empty nested table with different types. 2020-10-06 11:24:12 +03:00
Vladimir Chebotarev
d98366ed79 Better initialization of S3 storage. 2020-10-06 11:20:47 +03:00
Artem Zuikov
bad8171f24
Fix distributed IN without database on initiator (#15538) 2020-10-06 11:18:25 +03:00
Nikolai Kochetov
28e12c559c Ensure resource desrtuction order in ReadFromStorageStep. 2020-10-06 09:35:35 +03:00
Latysheva Alexandra
d495a2fb2d
Merge pull request #7 from alexelex/quorum_inserts_2 
build checker fix (style) and change comments
 2020-10-06 02:33:29 +06:00
Alexandra Latysheva
41e0148d72 build checker fix (style) and change comments 2020-10-05 20:30:50 +00:00
Alexander Tokmakov
418766e92b fix another bug in mutations 2020-10-05 22:16:28 +03:00
Latysheva Alexandra
3b952aff7d
Merge pull request #6 from alexelex/quorum_inserts_2 
fix test_00814
 2020-10-05 23:01:18 +06:00
Alexandra Latysheva
2be62bbd1b fix test_00814 2020-10-05 16:58:26 +00:00
alesapin
8ec58c17f3 More consistent metadata usage 2020-10-05 19:41:46 +03:00
Alexander Tokmakov
30a18f7567 make it better 2020-10-05 16:52:03 +03:00
Anton Popov
a49591b250
Merge pull request #15543 from azat/optimize_aggregation_in_order-fix-memory-usage 
[RFC] Significantly reduce memory usage in AggregatingInOrderTransform
 2020-10-05 16:34:12 +03:00
Latysheva Alexandra
18007622a1
Merge pull request #5 from alexelex/quorum_inserts_2 
style check fix
 2020-10-05 19:00:39 +06:00
Alexandra Latysheva
aa391a47eb style check fix 2020-10-05 12:59:07 +00:00
Latysheva Alexandra
7e7cedd73f
Merge pull request #4 from alexelex/quorum_inserts_2 
style check fix
 2020-10-05 16:43:14 +06:00
Alexandra Latysheva
0662432c5c style check 2020-10-05 10:41:49 +00:00
alexey-milovidov
2fcf2e0fbe
Merge pull request #15443 from ucasFL/pretty_row_numbers 
Add setting `output_format_pretty_row_numbers`
 2020-10-05 11:49:55 +03:00
Latysheva Alexandra
a63084adaf
Merge pull request #3 from alexelex/quorum_inserts_2 
Quorum inserts 2
 2020-10-05 03:28:58 +06:00
Alexandra Latysheva
d54b6c9542 add question in comments 2020-10-04 20:32:08 +00:00
Alexandra Latysheva
ff8cbe6745 remove debug things 2020-10-04 20:19:05 +00:00
Alexandra Latysheva
8263e62298 working copy (with some debug info) 2020-10-04 19:55:39 +00:00
alexey-milovidov
a005ec4c89
Merge pull request #15589 from ClickHouse/tavplubix-patch-1 
Use default implemetation for constants in JSON* functions
 2020-10-04 16:02:08 +03:00
alexey-milovidov
6b39d248a0
Merge pull request #15542 from Avogar/quantile_t_digest 
Improve quantileTDigest performance
 2020-10-04 15:53:36 +03:00
alexey-milovidov
53ce5e3892
Merge pull request #15551 from azat/for-user-limits-over-distributed 
Pass through *_for_user settings via Distributed with cluster-secure
 2020-10-04 04:18:20 +03:00
filipe
0436e0f8f1 fix style and pvs check 2020-10-03 20:50:44 -03:00
filipe
bab7c762bc fix style and pvs check 2020-10-03 20:38:21 -03:00
filipe
146f973437 add maximum_unit arg to formatReadableTimeDelta function 2020-10-03 19:38:51 -03:00
tavplubix
e219162b82
Update FunctionsJSON.h 2020-10-03 21:09:22 +03:00
alexey-milovidov
6f3aab8f09
Merge pull request #15319 from bharatnc/ncb/fix-14509 
Fix bug in queries like SELECT toStartOfDay(today())
 2020-10-03 17:02:57 +03:00
alexey-milovidov
f7b390465b
Merge branch 'master' into realfix 2020-10-03 16:54:23 +03:00
alexey-milovidov
97d97f6b2e
Merge pull request #15513 from ClickHouse/aku/error-context 
Add some context to an error message
 2020-10-03 16:24:19 +03:00
alexey-milovidov
ca05425a87
Merge pull request #15535 from nvartolomei/patch-2 
Add missing entry to client packet stringify
 2020-10-03 16:22:04 +03:00
alexey-milovidov
46fa5ff53e
Merge pull request #15557 from ClickHouse/statvfs-eintr 
Handle EINTR in statvfs
 2020-10-03 15:57:56 +03:00
alexey-milovidov
ae61f8f7a6
Update filesystemHelpers.cpp 2020-10-03 15:57:35 +03:00
alexey-milovidov
4d6402525a
Update filesystemHelpers.h 2020-10-03 15:57:12 +03:00
alexey-milovidov
9551562889
Merge pull request #15442 from ClickHouse/add_alter_version 
Add ability to alter version column in VersionedCollapsingMergeTree.
 2020-10-03 05:40:15 +03:00
alexey-milovidov
75fb73956e
Merge pull request #15238 from ianton-ru/multi_delete_in_s3 
Use S3 multi object delete
 2020-10-03 03:48:06 +03:00
alexey-milovidov
5e37ba4335
Update DiskS3.cpp 2020-10-03 03:47:45 +03:00
alexey-milovidov
3ca7fca48a
Update DiskS3.cpp 2020-10-03 03:42:33 +03:00
Alexey Milovidov
d426ba3074 Step 4: cleanup 2020-10-03 03:07:55 +03:00
Alexey Milovidov
b6ca8a0676 Step 3: check other usages, add a comment 2020-10-03 03:05:54 +03:00
Alexey Milovidov
2441be4b66 Step 2: move code to .cpp 2020-10-03 03:02:47 +03:00
Alexey Milovidov
6608de470d Step 1: handle EINTR in statvfs 2020-10-03 03:01:32 +03:00
alexey-milovidov
cb708212a8
Merge pull request #15536 from ClickHouse/fix_ilike 
Separate cache for ilike after like
 2020-10-03 02:07:40 +03:00
Azat Khuzhin
f25c1742b8 Pass through *_for_user settings via Distributed with cluster-secure 
In cluster-secure case the user on shards (remote) is equal to the user
on the initiator, so those settings can be safely applied.
 2020-10-03 02:04:47 +03:00
Azat Khuzhin
2a2f858365 Significantly reduce memory usage in AggregatingInOrderTransform 
Clean the aggregates pools (Arena's objects) between flushes, this will
reduce memory usage significantly (since Arena is not intended for
memory reuse in the already full Chunk's)

Before this patch you cannot run SELECT FROM huge_table GROUP BY
primary_key SETTINGS optimize_aggregation_in_order=1 (and the whole
point of optimize_aggregation_in_order got lost), while after, this
should be possible.
 2020-10-03 00:56:58 +03:00
Pavel Kruglov
7acd85d2b1 Fix style error 2020-10-02 22:20:35 +03:00
alesapin
77c9f7af79
Merge pull request #15532 from ClickHouse/substitute_defaults_recursively 
Fix defaults substitution for absent columns which depend on other defaults
 2020-10-02 21:42:01 +03:00
Alexander Tokmakov
6b2da861dc Merge branch 'master' into write_structure_of_table_functions 2020-10-02 20:46:58 +03:00
Pavel Kruglov
f6f4285348 Improve quantileTDigest performance 2020-10-02 20:13:55 +03:00
Azat Khuzhin
9cb3c743bd
Convert to python3 (#15007) 2020-10-02 19:54:07 +03:00
feng lv
d2f75f8508 fix 2020-10-02 23:13:28 +08:00
Nikolai Kochetov
404c199448
Merge pull request #15509 from markpapadakis/patch-6 
Update ActionsVisitor.cpp
 2020-10-02 18:07:46 +03:00
alesapin
8c0581c503 Fix ilike cache 2020-10-02 17:27:47 +03:00
nvartolomei
d35ac3c3fb
Add missing entry to client packet stringify 
(noticed while reading the code)
 2020-10-02 15:10:19 +01:00