thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 10:52:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
63,939 Commits 9,328 Branches 1,783 Tags 2.1 GiB
29336a4a34
Commit Graph

3888 Commits

Author SHA1 Message Date
kssenii
8453c60200 Better check 2021-04-21 14:54:52 +00:00
kssenii
c1151cea23 Add failover 2021-04-21 14:39:51 +00:00
Nikita Mikhaylov
d4e8f9676b
Merge pull request #21304 from amosbird/indexhint 
Resurrect indexHint function.
 2021-04-21 16:14:43 +03:00
mergify[bot]
7da7279390
Merge branch 'master' into fix_create_drop_replica_race 2021-04-21 13:03:29 +00:00
alexey-milovidov
34b30d80d4
Update StorageReplicatedMergeTree.cpp 2021-04-21 16:01:54 +03:00
tavplubix
1a776b47a7
Update MergeTreeData.cpp 2021-04-21 15:34:42 +03:00
kssenii
4c4f765aba Globs for url table function 2021-04-21 12:32:57 +00:00
alexey-milovidov
9c649932bc
Merge pull request #23375 from ClickHouse/fix-read-in-order-with-collation 
Fix read in order optimization of COLLATE is present #22379
 2021-04-21 15:00:02 +03:00
Amos Bird
096d76627e
Skip unavaiable shards when writing to distributed tables 2021-04-21 10:30:40 +08:00
Amos Bird
50f2e488bd
Fix invalid virtual column expr 2021-04-21 10:29:03 +08:00
Anton Popov
a83a07246e allow to move more conditions to PREWHERE 2021-04-21 01:03:58 +03:00
alexey-milovidov
4c1022ac03
Merge pull request #23318 from kssenii/fix-hdfs-with-spaces 
Fix hdfs reading from files with spaces
 2021-04-20 21:48:41 +03:00
Alexey Milovidov
9bc0706d2f Fix read in order optimization of COLLATE is present #22379 2021-04-20 21:20:45 +03:00
tavplubix
70c89eeff6
Merge pull request #23200 from ClickHouse/fix_prewhere_in_materialize_mysql 
Fix prewhere in MaterializeMySQL
 2021-04-20 20:05:07 +03:00
alexey-milovidov
cbbff1bc4b
Merge pull request #23319 from ClickHouse/mysql-insert-without-database 
Allow to INSERT into remote MySQL storage with empty database #19281
 2021-04-20 19:30:44 +03:00
tavplubix
bef0ab182e
Merge branch 'master' into fix_prewhere_in_materialize_mysql 2021-04-20 15:38:48 +03:00
tavplubix
d9ef1a9368
Update src/Storages/StorageMaterializeMySQL.cpp 
Co-authored-by: Alexander Kuzmenkov <36882414+akuzm@users.noreply.github.com>
 2021-04-20 15:38:02 +03:00
Nikita Mikhaylov
f55eacee7d
Merge pull request #22988 from excitoon-favorites/s3connectionresetretries 
Retries on HTTP connection drops during reads from S3
 2021-04-20 14:59:21 +03:00
Anton Popov
8f57141fad
Merge pull request #23195 from CurtizJ/nested-update 
Merging #22503
 2021-04-20 14:57:47 +03:00
Amos Bird
32c84f77c3
Resurrect indexHint function. 2021-04-20 19:27:23 +08:00
kssenii
da7a4ac1ea Better 2021-04-20 07:53:55 +00:00
Kseniia Sumarokova
b43656bf89
Merge pull request #22697 from TCeason/feature/add_MaterializeMySQL_sync_bytes_judgment 
Add MySQL read history data bytes judgment
 2021-04-20 10:26:42 +03:00
Amos Bird
d5f606c544
Another fix 2021-04-20 14:15:28 +08:00
Anton Popov
0a0c510aa2 minor fixes 2021-04-20 05:51:07 +03:00
robot-clickhouse
db82e9e3d5 Auto version update to [21.6.1.1] [54451] 2021-04-20 01:48:20 +03:00
Alexey Milovidov
f492495507 Allow to INSERT into remote MySQL storage with empty database #19281 2021-04-19 23:36:13 +03:00
alexey-milovidov
4cef8c68f7
Merge pull request #23258 from azat/wundef 
Add -Wundef for gcc builds
 2021-04-19 22:45:22 +03:00
kssenii
cc5f49fbee Fix 2021-04-19 19:43:22 +00:00
Nikolai Kochetov
c59628846c
Merge pull request #22352 from ClickHouse/add-read-from-mt-step 
Special query plan step for read from MergeTree
 2021-04-19 17:38:40 +03:00
Amos Bird
aeff06d67d
Don't relax NOT conditions during partition pruning. 2021-04-19 22:15:53 +08:00
Nikita Mikhaylov
adc103e338
Merge pull request #21852 from excitoon-favorites/s3stscredentials 
Added insecure IMDS credentials provider for S3
 2021-04-19 14:47:44 +03:00
Alexander Tokmakov
ecc625692e fix build 2021-04-19 13:40:20 +03:00
tavplubix
c9ebc95faf
Update ReplicatedMergeTreeCleanupThread.cpp 2021-04-19 13:27:32 +03:00
Alexander Tokmakov
539356b441 check block version on removal 2021-04-19 12:52:37 +03:00
Alexander Tokmakov
9bc98771f7 fix race on replica creation 2021-04-19 11:21:42 +03:00
Vladimir Chebotarev
9d4295f261 Removed infinite retries. 2021-04-19 11:02:36 +03:00
TCeason
b4bf53dfc9 add some comment and modify a parameter type 2021-04-19 15:34:27 +08:00
Nikolai Kochetov
4d86f51eff Merge branch 'master' into add-read-from-mt-step 2021-04-19 10:17:21 +03:00
TCeason
63403c709c modify settings name to external_xxx and rewrite Storage MySQL max_block_size 2021-04-19 10:51:50 +08:00
TCeason
472c131420 Add MySQL read history data bytes judgment 2021-04-19 10:51:50 +08:00
Azat Khuzhin
2561a67fd8 Replace !__clang__ with !defined(__clang) to fix gcc builds 
$ gg 'if !__clang__' | cut -d: -f1 | sort -u | xargs sed -i 's/#if !__clang__/#if !defined(__clang__)/g'
 2021-04-18 23:37:50 +03:00
alexey-milovidov
7f3a40e1b0
Merge pull request #22237 from ClickHouse/protocol-compression-auto 
Autodetect compression #22234
 2021-04-18 03:13:54 +03:00
Alexey Milovidov
77e64b3ebd Merge branch 'master' into protocol-compression-auto 2021-04-17 16:46:51 +03:00
kssenii
5dc2dfa437 Fix 2021-04-17 09:00:58 +00:00
alexey-milovidov
74a0ac6917
Merge pull request #23160 from azat/logging-v2 
[RFC] Change logging from trace to debug for messages with rows/bytes
 2021-04-16 21:18:32 +03:00
Alexander Tokmakov
3b4bd18809 fix prewhere in materialize mysql 2021-04-16 18:51:27 +03:00
tavplubix
3d1885f08c
Merge pull request #23177 from marsno1/fix-exception-log 
Fix exception message for "parts_to_throw_insert"
 2021-04-16 18:15:07 +03:00
Kruglov Pavel
deb86ee520
Merge pull request #22986 from azat/StorageBuffer-flush 
[RFC] Add ability to flush buffer only in background for StorageBuffer
 2021-04-16 15:14:24 +03:00
Nikolai Kochetov
8d8e57615c A little bit better index description. 2021-04-16 12:42:23 +03:00
Anton Ivashkin
09379b2b8a Fix Zero-Copy replication with several S3 volumes (issue 22679) 2021-04-16 12:34:48 +03:00
tavplubix
b5cbbce314
Merge pull request #22706 from songenjie/fetch_part 
[clickhouse][server][ddl]support fetch part
 2021-04-16 12:26:51 +03:00
tavplubix
ae3dc3dfc0
Merge pull request #23161 from ClickHouse/logs_for_debuging_test_failures 
More verbose logs for debuging test failures with Replicated and Keeper
 2021-04-16 12:01:58 +03:00
madianjun
9238d8e54a Fix exception message for parts_to_throw_insert 2021-04-16 14:28:52 +08:00
Alexander Tokmakov
0ad6205fa6 logs for debuging test failures with Replicated and Keeper 2021-04-15 21:34:53 +03:00
Azat Khuzhin
19e0439629 Add ability to flush buffer only in background for StorageBuffer 
Add 3 new engine arguments:
- flush_time
- flush_rows
- flush_bytes

That will be checked only for background flush, this maybe useful if
INSERT latency is "crucial".
 2021-04-15 21:22:13 +03:00
Azat Khuzhin
d2cf03ea41 Change logging from trace to debug for messages with rows/bytes 2021-04-15 21:00:16 +03:00
Nikolai Kochetov
be52b2889a Better description for key condition. 2021-04-15 20:30:04 +03:00
alexey-milovidov
b097140bb4
Merge pull request #23083 from azat/logging 
Tiny logging improvements
 2021-04-15 14:39:47 +03:00
Maksim Kita
e361f5943f
Merge pull request #22999 from azat/no-optimize_skip_unused_shards-single-node 
Do not perform optimize_skip_unused_shards for cluster with one node
 2021-04-15 14:36:56 +03:00
Vladimir Chebotarev
95866e6e8d Rebase fix. 2021-04-15 10:57:08 +03:00
Vladimir Chebotarev
f59d35efd8 Rebase fix. 2021-04-15 10:53:00 +03:00
Vladimir Chebotarev
c0ea3ce08a Removed STS client. 2021-04-15 10:01:43 +03:00
Vladimir Chebotarev
3a467ffc50 Attempt to make credentials provider which uses internal credentials provider chain. 2021-04-15 10:01:13 +03:00
Vladimir Chebotarev
037a4de854 Added STS GetCallerIdentity credentials provider. 2021-04-15 10:00:18 +03:00
Vladimir Chebotarev
10bf56e540 Attempt to add retries to S3 reads. 2021-04-15 09:47:19 +03:00
Alexey Milovidov
17d04cbca9 Merge branch 'master' into protocol-compression-auto 2021-04-15 00:24:07 +03:00
tavplubix
2479c80fb7
Merge branch 'master' into fix_attach_mv 2021-04-15 00:07:48 +03:00
Azat Khuzhin
4b69684cd2 Add logging about pushing to underlying in StorageKafka 2021-04-14 23:15:31 +03:00
songenjie
c06c624fc7 fix build 2021-04-14 18:55:42 +08:00
songenjie
1fc040ac58 fix some Suggestfix some Suggest 2021-04-14 17:54:56 +08:00
songenjie
8566df9b7d fix some Suggest 2021-04-14 17:11:59 +08:00
songenjie
f731739fab fix suggests 2021-04-14 10:05:41 +08:00
Alexey Milovidov
6f56c3280f Uncompress data in Distributed sends if needed 2021-04-14 00:53:39 +03:00
Nikita Mikhaylov
2a86d76ccd style 2021-04-13 23:19:04 +03:00
Nikita Mikhaylov
ec35a878d3 simplify storages3cluster 2021-04-13 23:17:25 +03:00
tavplubix
39d55556b8
Update StorageMaterializedView.cpp 2021-04-13 23:14:05 +03:00
Nikita Mikhaylov
3954eff276 do not use cluster from config on remote replicas 2021-04-13 22:39:43 +03:00
Nikita Mikhaylov
b05718feac style 2021-04-13 22:39:43 +03:00
Nikita Mikhaylov
c9ae4cb467 multiplem pipes on remote 2021-04-13 22:39:43 +03:00
Nikita Mikhaylov
024374a2ec review fixes 2021-04-13 22:39:43 +03:00
Nikita Mikhaylov
a15757a9c9 better renaming 2021-04-13 22:39:43 +03:00
Nikita Mikhaylov
09a62e713a rename to s3Cluster 2021-04-13 22:39:43 +03:00
Nikita Mikhaylov
a743442a17 build fixes 2021-04-13 22:39:43 +03:00
Nikita Mikhaylov
507cb8514a review fixes 2021-04-13 22:39:43 +03:00
Nikita Mikhaylov
75230e3701 better 2021-04-13 22:39:43 +03:00
Nikita Mikhaylov
7a68820342 style 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
7b95ff579d fix S3 test 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
4465a0627f better 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
c333c3dedb review fixes 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
efef179b89 remove prints 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
36a8419f60 style 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
8a4b5a586e fixed uuid 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
7276b40556 better 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
2555ae5d3f better processing stage 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
587fbdd10d better 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
fea3aaafe6 simplify storages3dist 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
4843f86329 use only one connection between initiator and worker 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
3ab17233cf cleanup 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
b3094412b1 better 2021-04-13 22:39:42 +03:00
Nikita Mikhaylov
cef9e19eb2 better 2021-04-13 22:39:41 +03:00
Nikita Mikhaylov
a5d55781b1 remote debug prints 2021-04-13 22:39:41 +03:00
Nikita Mikhaylov
9b7a6e66d0 add ifded 2021-04-13 22:39:41 +03:00
Nikita Mikhaylov
4e4b383214 added hash of itiator address 2021-04-13 22:39:41 +03:00
Nikita Mikhaylov
2549468c14 better 2021-04-13 22:39:41 +03:00
Nikita Mikhaylov
0be3fa178b save 2021-04-13 22:39:41 +03:00
Nikita Mikhaylov
64b4cd0e63 save 2021-04-13 22:39:41 +03:00
Nikita Mikhaylov
31b4f9b17f save 2021-04-13 22:39:41 +03:00
Nikita Mikhaylov
44ca65a9a4 save 2021-04-13 22:39:41 +03:00
Nikita Mikhaylov
081ea84a41 save 2021-04-13 22:39:41 +03:00
Alexander Tokmakov
05e4c8d3ef fix attach mv in atomic db 2021-04-13 22:13:26 +03:00
Alexey Milovidov
97611faad0 Whitespace 2021-04-13 22:06:24 +03:00
tavplubix
1525e38a3c
Merge pull request #22990 from ClickHouse/tavplubix-patch-1 
Fix excessive warning in StorageDistributed with cross-replication
 2021-04-13 18:58:12 +03:00
tavplubix
da681d9907
Merge pull request #22998 from ClickHouse/fix_22989 
Fix assertion when filtering tables in StorageMerge
 2021-04-13 18:57:24 +03:00
alexey-milovidov
46e1da03fb
Update KeyCondition.cpp 2021-04-13 18:47:11 +03:00
songenjie
7c32cc1e18 fix case style for local variable 2021-04-13 17:34:04 +08:00
Maksim Kita
305d3d6b16
Merge pull request #22978 from ClickHouse/filter-by-virtual-columns-fix 
Fix trivial mistake in filtering by virtual columns
 2021-04-13 11:42:16 +03:00
songenjie
564136ec46 [clickhouse][server][dll][alter]support fetch part 2021-04-13 12:40:33 +08:00
alexey-milovidov
d239c66fd3
Merge pull request #21511 from azat/optimize_skip_unused_shards-improvement 
Rewrite IN in query for remote shards to exclude values that do not belong to the shard
 2021-04-13 01:18:11 +03:00
Alexey Milovidov
1b8a054880 Fix error 2021-04-13 00:11:30 +03:00
Azat Khuzhin
a497d4d462 Do not perform optimize_skip_unused_shards for cluster with one node 2021-04-12 22:18:31 +03:00
Alexander Tokmakov
ad414bab05 fix assertion when filtering tables in StorageMerge 2021-04-12 22:04:26 +03:00
Denny Crane
eaa092cae2
Update MergeTreeData.cpp 
Better error message.
 2021-04-12 10:58:06 -03:00
Nikolai Kochetov
f6d86d6032 Merge branch 'master' into add-read-from-mt-step 2021-04-12 15:23:32 +03:00
tavplubix
a995962e6a
Update StorageDistributed.cpp 2021-04-12 14:58:24 +03:00
Nikolai Kochetov
28d6e3f56c
Merge pull request #22977 from azat/combinator-fix 
Fix combinators with common prefix name (State and SimpleState) with libstdc++
 2021-04-12 12:27:01 +03:00
Maksim Kita
3078fd7d79
Merge pull request #22983 from borodark/master 
Error message reads better
 2021-04-12 10:56:10 +03:00
Azat Khuzhin
79bd8d4d3f Respect optimize_skip_unused_shards_rewrite_in with optimize_skip_unused_shards_limit 2021-04-12 10:37:28 +03:00
Azat Khuzhin
e439914d38 Fix optimized cluster logic for optimize_skip_unused_shards 2021-04-12 10:37:28 +03:00
Azat Khuzhin
fbb386dca5 Rewrite IN in query for remote shards to exclude values that does not belongs to shard 
v2: fix optimize_skip_unused_shards_rewrite_in for sharding_key wrapped into function
v3: fix column name for optimize_skip_unused_shards_rewrite_in
v4: fix optimize_skip_unused_shards_rewrite_in with Null
v5:
- squash with Remove query argument for IStreamFactory::createForShard()
- use proper column after function execution (using sharding_key_column_name)
- update the test reference since (X) now is tuple(X)
 2021-04-12 10:37:28 +03:00
Egor O'Sten
c8b572ed59 Readability of error message for ORDER BY 2021-04-11 20:42:05 -04:00
Egor O'Sten
c77dd36ca9 Readability of error message for ORDER BY 2021-04-11 20:40:00 -04:00
kssenii
df84e4e93a Fix 2021-04-11 23:39:18 +00:00
Alexey Milovidov
22720dd7a4 Fix trivial mistake in filtering by virtual columns 2021-04-11 22:39:22 +03:00
Azat Khuzhin
0bf6b61b59 Fix combinators with common prefix name (State and SimpleState) with libstdc++ 
Previously sort order of the std::unordered_map in libstdc++ was
different and any *SimpleState() reports an error that function does not
exists.

Fix this by using proper order in container, and use std::vector over
std::unordered_map, since there linear traversing anyway in the single
method -- tryFindSuffix()

Note that test is not required, since it either fail with unknown
function or not.
 2021-04-11 22:10:02 +03:00
Maksim Kita
aa6183af86
Merge pull request #22971 from azat/max_part_removal_threads 
Respect max_part_removal_threads for ReplicatedMergeTree
 2021-04-11 21:14:47 +03:00
Azat Khuzhin
b24d1251de Introduce helper for removing one part from fs for ReplicatedMergeTree 2021-04-11 14:38:46 +03:00
alexey-milovidov
0f6fc5bd72
Merge pull request #22962 from kitaisreal/storage-external-distributed-arcadia-fix 
StorageExternalDistributed arcadia fix
 2021-04-11 13:34:03 +03:00
Azat Khuzhin
940ff6cb76 Do not use c-style cast for std::min(max_part_removal_threads) 2021-04-11 12:22:39 +03:00
Azat Khuzhin
6e71757814 Fix thread accounting for max_part_removal_threads 2021-04-11 11:43:30 +03:00
Azat Khuzhin
dfe3699e77 Respect max_part_removal_threads for ReplicatedMergeTree 2021-04-11 11:33:45 +03:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr (#22297) 
* Replace all Context references with std::weak_ptr

* Fix shared context captured by value

* Fix build

* Fix Context with named sessions

* Fix copy context

* Fix gcc build

* Merge with master and fix build

* Fix gcc-9 build
 2021-04-11 02:33:54 +03:00
Maksim Kita
685da4e234 StorageExternalDistributed arcadia fix 2021-04-10 21:52:17 +03:00
Azat Khuzhin
a6703c5ac2 Fix inactive_parts_to_throw_insert=0 with inactive_parts_to_delay_insert>0 2021-04-10 09:57:08 +03:00
Alexander Kuzmenkov
e44b3822e3
Merge pull request #21850 from fastio/handle_errors_for_kafka_engine 
Handle errors for Kafka engine
 2021-04-09 22:59:40 +03:00
Alexander Kuzmenkov
0264124146
Merge pull request #21942 from ucasFL/distributed_depth 
Add settings max_distributed_depth
 2021-04-09 15:52:58 +03:00
Nikolai Kochetov
28ca191102
Merge pull request #22087 from ClickHouse/better-filter-push-down 
Better filter push down
 2021-04-09 10:22:17 +03:00
alesapin
4b0f973f93
Merge pull request #18215 from filimonov/quorum-dedup-issue 
Fix issue with quorum retries behaviour
 2021-04-09 10:19:50 +03:00
alexey-milovidov
42b865c9c1
Merge pull request #22829 from azat/dist-send-log 
Slightly improve logging messages for Distributed async sends
 2021-04-09 09:51:53 +03:00
alexey-milovidov
b02a82250b
Merge pull request #22858 from vitlibar/fix-hanging-in-temporary-live-view-cleaner 
Fix hanging in TemporaryLiveViewCleaner
 2021-04-09 09:25:38 +03:00
Anton Popov
1c28878f5d
Merge pull request #22726 from azat/mt-reverse-total-rows-approx-fix 
Fix approx total rows accounting for reverse reading from MergeTree
 2021-04-09 00:16:49 +03:00
Nikolai Kochetov
794d7c89b6 Remove ReverseRowsStep 2021-04-08 16:18:17 +03:00
Vitaly Baranov
5bcb7a0fed Fix hanging in TemporaryLiveViewCleaner 2021-04-08 16:14:49 +03:00
Nikolai Kochetov
febb187da0 Added test, 2021-04-08 15:54:27 +03:00
Nikolai Kochetov
7ffbeac9df Add info about indexes to ReadFromMergeTree step. 2021-04-08 14:48:54 +03:00
alesapin
e762d02e37 More const 2021-04-08 13:38:40 +03:00
alesapin
e17444757b Fix rare case when quorum insert is not really quorum because of deduplication 2021-04-08 13:35:38 +03:00
Nikolai Kochetov
7c5a9133df Add index info to ReadFromStorageStep. 2021-04-08 11:19:04 +03:00
alesapin
1533f9b9aa
Merge pull request #14113 from johnskopis/dynamic-interserver-creds-v20 
Support interserver credential rotation
 2021-04-08 11:01:40 +03:00
Azat Khuzhin
c27b931f6a Slightly improve logging messages for Distributed async sends 
- add took time (in ms)
- add rows/bytes
 2021-04-08 08:10:39 +03:00
alesapin
b2a370ad49 Fix PVS 2021-04-07 19:06:20 +03:00
alesapin
2987bbc948 Small improvements 2021-04-07 16:52:11 +03:00
Nikita Mikhaylov
48af7a898c
Merge pull request #22299 from nikitamikhaylov/keen-wolf-storagefile-column-oriented 
Merging #21302
 2021-04-07 16:01:12 +03:00
Kruglov Pavel
5b52657428
Merge pull request #22725 from azat/buffer-lock-metrics 
Add metric to track how much time is spend during waiting for Buffer layer lock
 2021-04-07 14:35:57 +03:00
Vladimir
864ab4a7ab
Merge pull request #21370 from vdimir/fix-merge-engine-join-group-by 2021-04-07 11:52:27 +03:00
Vladimir
db0550013c
Merge pull request #22325 from azat/dist-async-insert-header-check 
Check only column name and type for async distributed blocks
 2021-04-07 11:51:12 +03:00
alesapin
9fd251eaaf
Merge pull request #22514 from ClickHouse/merge_tree_deduplication 
Non replicated merge tree deduplication
 2021-04-07 10:19:43 +03:00
alexey-milovidov
a7e216969a
Merge pull request #22592 from azat/uncaught-exceptions 
Fix some uncaught exceptions (in SCOPE_EXIT) under memory pressure
 2021-04-07 07:17:26 +03:00
Nikita Mikhailov
71a2c85d9a better 2021-04-06 22:23:16 +03:00
Nikita Mikhailov
af85ce2fd1 better 2021-04-06 22:23:16 +03:00
Nikita Mikhailov
37f48d13b4 add test 2021-04-06 22:23:16 +03:00
Nikita Mikhailov
282775b10c save 2021-04-06 22:23:16 +03:00
keenwolf
1a4961499b the Native format is not supported after test 2021-04-06 22:23:16 +03:00
keenwolf
94c8e7af1a remove const for value-copy-return 2021-04-06 22:23:16 +03:00
keenwolf
3d3923b518 update comments 2021-04-06 22:23:16 +03:00
keenwolf
b95f54edce Small fix 2021-04-06 22:23:16 +03:00
keenwolf
139bb678a9 Branch with Format isColumnOriented() or not 2021-04-06 22:23:15 +03:00
keenwolf
4eaf718fd2 fix the getColumsForNames() to bring the whole column info from metadata 2021-04-06 22:23:15 +03:00
keenwolf
9d6e2b4bad update comments 2021-04-06 22:23:15 +03:00
keenwolf
de365a5852 fix 2021-04-06 22:23:15 +03:00
keenwolf
6f9bad682f Only read needed columns for formats as parquet etc 2021-04-06 22:23:15 +03:00
Maksim Kita
e38206ee02
Merge pull request #22721 from kitaisreal/memory-storage-sync-comments-and-code 
MemoryStorage sync comments and code
 2021-04-06 21:56:41 +03:00
Azat Khuzhin
60b000ed51 Fix approx total rows accounting for reverse reading from MergeTree 2021-04-06 21:13:51 +03:00
Azat Khuzhin
c4a7e81287 Add metric to track how much time is spend during waiting for Buffer layer lock 
It uses very fast CLOCK_MONOTONIC_COARSE, so this should not be a
problem.
Also note that there is no sense in using microseconds/nanoseconds since
accuracy of CLOCK_MONOTONIC_COARSE usually milliseconds.
 2021-04-06 21:13:24 +03:00
Azat Khuzhin
f157278b72 Safer SCOPE_EXIT 
It executes the code in the dtor, that should never throw.
 2021-04-06 21:10:37 +03:00
Nikolai Kochetov
9f39f5d52d Add more counters to MergeTreeDataSelectExecutor 2021-04-06 15:39:55 +03:00
Maksim Kita
af19f89273 MemoryStorage sync comments and code 2021-04-06 13:41:48 +03:00
Nikolai Kochetov
6102652c99 Merge branch 'master' into better-filter-push-down 2021-04-06 13:38:03 +03:00
alesapin
53a7836a9b Fix typos 2021-04-06 13:16:34 +03:00
alesapin
6456a1507c Support alter setting 2021-04-06 13:14:44 +03:00
vdimir
735154c81a
Handle not plain where tree in StorageMerge modifySelect 2021-04-06 12:29:29 +03:00
alesapin
86a843bb51
Merge pull request #21509 from kssenii/library-bridge 
clickhouse-library-bridge for library dictionary source
 2021-04-06 12:26:08 +03:00
tavplubix
8b1d320633
Merge pull request #22217 from kssenii/replicas-shards-for-mysql-and-postgres 
Replicas and shards for mysql and postgres storages
 2021-04-06 11:05:24 +03:00
alesapin
3cd571f92b
Merge pull request #22669 from ClickHouse/fix_two_bugs_with_mutation_wait 
Fix two stupid bugs in mutation wait
 2021-04-06 11:04:16 +03:00
Azat Khuzhin
27d4fbd13b Compare Block itself for distributed async INSERT batches 
INSERT into Distributed with insert_distributed_sync=1 stores the
distributed batches on the disk for sending in background.

But types may be a little bit different for the Distributed and it's
underlying table, so the initiator need to know whether conversion is
required or not.

Before this patch those on disk distributed batches contains header,
which includes dumpStructure() for the block in that batch, however it
checks not only names and types and plus dumpStructure() is a debug
method.

So instead of storing string representation for the block header we
should store empty block in the file header (note, that we cannot store
the empty block not in header, since this will require reading all
blocks from file, due to some trickery of the readers interface).

Note, that this patch also contains tiny refactoring:
- s/header/distributed_header/

v1: dumpNamesAndTypes()
v2: dump empty block into the batch itself
v3: move empty block into the header
 2021-04-06 10:05:21 +03:00
Maksim Kita
de71553c9d
Merge pull request #22593 from zlobober/introduce_distributed_write 
Introduce IStorage::distributedWrite method for distributed INSERT SELECTS
 2021-04-05 22:59:57 +03:00
kssenii
dc42d5189d Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into replicas-shards-for-mysql-and-postgres 2021-04-05 15:36:35 +00:00
alesapin
c830397317 Fix two stupid bugs in mutation wait 2021-04-05 18:08:43 +03:00
feng lv
f6e4e13f89 fix conflict 2021-04-05 14:08:30 +00:00
feng lv
56073db22d max distributed depth 
Add settings max_distributed_depth

fix style

fix

fix

fix

fix test

fix

fix
 2021-04-05 14:00:54 +00:00
kssenii
9dba529494 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into library-bridge 2021-04-05 13:14:51 +00:00
alesapin
156ba8a7fc Disable setting by default and ban for old-syntax MergeTree 2021-04-05 12:16:38 +03:00
alesapin
8d5a787f6b Merge branch 'master' into merge_tree_deduplication 2021-04-05 10:40:03 +03:00
alexey-milovidov
6cefb8e620
Merge pull request #22583 from azat/log-rwlock-ub-fix 
Avoid UB in *Log engines for rwlock unlock due to unlock from another thread
 2021-04-05 05:20:16 +03:00
Maxim Akhmedov
725fa17961 Introduce IStorage::distributedWrite method for distributed INSERT SELECT. 2021-04-05 02:14:27 +03:00
alexey-milovidov
b1e7624b3f
Merge pull request #22589 from ClickHouse/fix-comments 
Fix comments
 2021-04-05 00:22:27 +03:00
alesapin
0ceb95aa55 Merge branch 'master' into merge_tree_deduplication 2021-04-04 18:32:27 +03:00
Azat Khuzhin
44abb7588a StorageStripeLog: avoid UB for rwlock unlock due to unlock from another thread 2021-04-04 13:25:44 +03:00
Azat Khuzhin
89a96cf14d StorageLog: avoid UB for rwlock unlock due to unlock from another thread 2021-04-04 13:25:44 +03:00
Azat Khuzhin
5d795661ed StorageTinyLog: add regression test for UB due to unlock from different thread 
Refs: #22560
 2021-04-04 13:25:44 +03:00
Azat Khuzhin
5ced6e2e9a StorageStripeLog: drop mutable qualifier for rwlock 2021-04-04 13:25:43 +03:00
Azat Khuzhin
5ef845af40 StorageLog: drop mutable qualifier for rwlock 2021-04-04 13:25:43 +03:00
Alexey Milovidov
54caa61af5 Fix comments 2021-04-04 12:33:06 +03:00
Alexey Milovidov
8b5d0a598f Minor improvement in index deserialization 2021-04-04 12:17:54 +03:00
alesapin
548ec6d2cb Trying enabled setting 2021-04-04 11:02:09 +03:00
alexey-milovidov
67fdfd4b89
Merge pull request #22560 from azat/tiny-log-rwlock-ub-fix 
Fix UB by unlocking the rwlock of the TinyLog from the same thread
 2021-04-03 21:59:01 +03:00
alesapin
1906c77884 Fix tidy one more time 2021-04-03 17:57:03 +03:00
Azat Khuzhin
6cebd3ee1f Fix UB by unlocking the rwlock of the TinyLog from the same thread 
Before this patch the build with libstdc++ hangs for
00967_insert_into_distributed_different_types test (and I guess some
others), this is because the rwlock is acquired from different thread as
it was unlocked which causes UB [1], fix this by moving unlock into
writeSuffix().

  [1]: The pthread_rwlock_unlock() function shall release a lock held on
       the read-write lock object referenced by rwlock. Results are undefined
       if the read-write lock rwlock is not held by the **calling thread**.
 2021-04-03 16:04:39 +03:00
Azat Khuzhin
10d35867e3 Drop mutable qualifier for StorageTinyLog::rwlock 2021-04-03 16:03:17 +03:00
alesapin
0204f5dd35 Merge branch 'master' into merge_tree_deduplication 2021-04-03 15:24:26 +03:00
alesapin
b127e31451 CI fixes 2021-04-03 14:56:56 +03:00
kssenii
13d1f17d3e Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into replicas-shards-for-mysql-and-postgres 2021-04-03 08:00:09 +00:00
alexey-milovidov
c3a9cbe01c
Merge pull request #22421 from ClickHouse/fix-buffer-overflow-in-token-extractor 
Fix buffer overflow in token extractor
 2021-04-02 22:08:28 +03:00
alesapin
b884593186 Followup fix 2021-04-02 20:55:05 +03:00
alesapin
ca30c91868 More comments 2021-04-02 20:54:24 +03:00
alesapin
fc3afe1087 Fix typos 2021-04-02 20:51:57 +03:00
alesapin
b105153d2c Useful comment 2021-04-02 20:48:58 +03:00
alesapin
ab34873344 Add some comments 2021-04-02 20:46:01 +03:00
alesapin
a555d078a2 Add exception handling 2021-04-02 19:56:02 +03:00
alesapin
759dd79c76 Add tests 2021-04-02 19:45:18 +03:00
kssenii
2de6b99f84 Better names 2021-04-02 16:12:14 +00:00
robot-clickhouse
39b5674bef Auto version update to [21.5.1.1] [54450] 2021-04-02 18:34:29 +03:00
Alexander Kuzmenkov
e7df8893df
Merge pull request #22300 from ClickHouse/trying_parallel_func_tests 
Trying parallel func tests
 2021-04-02 18:09:58 +03:00
tavplubix
93bd2c3b67
Merge pull request #18978 from myrrc/feature/attach-partition-local 
Lookup parts/partitions in replica's own detached/ folder before downloading them from other replicas
 2021-04-02 17:46:03 +03:00
alesapin
6f36661575 Buildable code 2021-04-02 16:34:36 +03:00
tavplubix
c1ab9de720
Update MergeTreeDataPartChecksum.cpp 2021-04-02 15:57:30 +03:00
alesapin
2ab87d61eb Some build fixes 2021-04-02 15:37:42 +03:00
alesapin
14634689e5 Idea code 2021-04-02 14:46:42 +03:00
Maksim Kita
d06fb1197a
Merge pull request #22096 from kitaisreal/hierarchy-dictionaries-updated 
Refactored hierarchy dictionaries interface
 2021-04-02 11:38:01 +03:00
alesapin
4eaf585fe5 Almost nothing added 2021-04-01 22:49:51 +03:00
Mike Kot
c947280dfc Merge remote-tracking branch 'upstream/master' into feature/attach-partition-local 2021-04-01 21:38:51 +03:00
Mike Kot
60e5946449 Removed needless logging, fixed data part initialization while checking 
in the detached folder
 2021-04-01 20:11:14 +03:00
kssenii
99f0b08ac3 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into replicas-shards-for-mysql-and-postgres 2021-04-01 13:48:33 +00:00
Alexey Milovidov
39c3efdb00 Fix build 2021-04-01 16:38:04 +03:00
Alexander Kuzmenkov
c7bf01fa33 typo 2021-04-01 16:08:18 +03:00
Alexander Kuzmenkov
ba64bc8557 quick fix for inaccessible DatabaseReplicated 2021-04-01 15:21:29 +03:00
kssenii
8300a74f5a Fix 2021-04-01 12:09:44 +00:00
vdimir
3f464595eb
Accurate removing 'join' part for queries to 'Merge' engine 2021-04-01 14:31:57 +03:00
vdimir
fff8043e55
Revert "Fix joined table access with Merge engine and aggregation" 
This reverts commit 5476e68d6c.
 2021-04-01 14:31:46 +03:00
kssenii
522d170588 Add default port 2021-04-01 11:30:20 +00:00
vdimir
5476e68d6c
Fix joined table access with Merge engine and aggregation 2021-04-01 14:26:28 +03:00