thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 10:52:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
63,059 Commits 9,328 Branches 1,783 Tags 2.1 GiB
524113f497
Commit Graph

12931 Commits

Author SHA1 Message Date
mergify[bot]
71bca99bbf
Merge branch 'master' into explain-output 2021-04-22 07:02:47 +00:00
hexiaoting
d24946f475 Fix bug when modify column without type 2021-04-22 14:48:57 +08:00
alexey-milovidov
3df3acc970
Merge pull request #23458 from kitaisreal/system-dictionaries-virtual-key-column 
System dictionaries virtual key column
 2021-04-22 08:42:15 +03:00
alexey-milovidov
8134c270a2
Merge pull request #23456 from kitaisreal/cast-to-array-from-empty-array 
Cast to array from empty array
 2021-04-22 08:41:35 +03:00
alexey-milovidov
1fcf198cec
Merge pull request #23415 from godliness/fix-kafka-with-arrow 
Fix kafka storage does not work with arrow and arrowstream format messages
 2021-04-22 04:59:31 +03:00
Anton Popov
3548e31afd fix corner cases in vertical merges with ReplacingMergeTree 2021-04-22 01:27:18 +03:00
Maksim Kita
7168b3a04c Table system.dictionaries added virtual key column 2021-04-22 01:16:03 +03:00
Maksim Kita
ddcffcb723 Fixed tests 2021-04-22 00:19:01 +03:00
kssenii
bb6bc61f22 Fix checks 2021-04-21 19:53:38 +00:00
Russ Frank
53f6a8f9a0 Merge remote-tracking branch 'origin/master' into rf/deltasum-with-timestamp 2021-04-21 13:52:50 -04:00
Maksim Kita
631d870ae3 Cast to array from empty array 2021-04-21 20:29:49 +03:00
kssenii
473903b396 Fix typos check 2021-04-21 17:09:01 +00:00
kssenii
b0bdb6b323 Fix build 2021-04-21 16:17:45 +00:00
kssenii
2e6c753add Allow separate vhost for each table 2021-04-21 15:51:05 +00:00
kssenii
8453c60200 Better check 2021-04-21 14:54:52 +00:00
kssenii
c1151cea23 Add failover 2021-04-21 14:39:51 +00:00
Russ Frank
822fc9b110 deltasum counter reset bugfix & test 2021-04-21 09:52:53 -04:00
Nikita Mikhaylov
d4e8f9676b
Merge pull request #21304 from amosbird/indexhint 
Resurrect indexHint function.
 2021-04-21 16:14:43 +03:00
mergify[bot]
7da7279390
Merge branch 'master' into fix_create_drop_replica_race 2021-04-21 13:03:29 +00:00
alexey-milovidov
34b30d80d4
Update StorageReplicatedMergeTree.cpp 2021-04-21 16:01:54 +03:00
tavplubix
1a776b47a7
Update MergeTreeData.cpp 2021-04-21 15:34:42 +03:00
kssenii
4c4f765aba Globs for url table function 2021-04-21 12:32:57 +00:00
alexey-milovidov
20a5fed53e
Merge pull request #23392 from ClickHouse/fix-4854 
Keep default timezone on DateTime operations if it was not provided explicitly #4854
 2021-04-21 15:00:59 +03:00
alexey-milovidov
9c649932bc
Merge pull request #23375 from ClickHouse/fix-read-in-order-with-collation 
Fix read in order optimization of COLLATE is present #22379
 2021-04-21 15:00:02 +03:00
Chao Ma
713eb9486c Fix kafka storage does not work with arrow and arrowstream format messages 2021-04-21 19:44:09 +08:00
Pavel Kovalenko
37c637cc34 Minor fixes. 2021-04-21 13:34:07 +03:00
Pavel Kovalenko
2ab8013455 DiskRestartProxy improvements and tests. 2021-04-21 12:53:46 +03:00
Nikolai Kochetov
9e943f4fad Fix ya.make 2021-04-21 08:50:42 +03:00
Nikolai Kochetov
44112587d4
Merge pull request #23309 from ClickHouse/try-fix-crash-with-unknown-packet 
Fix crash in case of unknown packet
 2021-04-21 08:47:17 +03:00
Nikolai Kochetov
9ec6fd1943
Merge pull request #23359 from ClickHouse/try-fix-23029 
Fix #23029
 2021-04-21 08:46:21 +03:00
Amos Bird
096d76627e
Skip unavaiable shards when writing to distributed tables 2021-04-21 10:30:40 +08:00
Amos Bird
50f2e488bd
Fix invalid virtual column expr 2021-04-21 10:29:03 +08:00
TCeason
51fe9fc771 modify MaterializeMySQL check MySQL var logical 
Supprot for MySQL 5.7.9: https://github.com/ClickHouse/ClickHouse/issues/4006#issuecomment-817622685
 2021-04-21 10:16:04 +08:00
Anton Popov
a83a07246e allow to move more conditions to PREWHERE 2021-04-21 01:03:58 +03:00
Maksim Kita
02515ff0a4
Merge pull request #23191 from kitaisreal/use-local-shard-if-no-remote-connections 
Updated QueryAliasVisitor to prefer alias for aliases like _subquery
 2021-04-21 00:58:47 +03:00
Alexey Milovidov
054da0ea34 Revert #17359 2021-04-21 00:23:47 +03:00
Alexey Milovidov
502bc3575c Revert #17359 2021-04-21 00:22:29 +03:00
Alexey Milovidov
416efbacbb Keep default timezone on DateTime operations if it was not provided explicitly #4854 2021-04-21 00:08:06 +03:00
Pavel Kovalenko
9272237c7e Update DiskRestartProxy.cpp 2021-04-20 23:47:21 +03:00
Pavel Kovalenko
10cf881a08 Add SYSTEM RESTART DISK command. 2021-04-20 22:23:54 +03:00
Pavel Kovalenko
016a9c674b DiskRestartProxy refactoring. 2021-04-20 21:58:34 +03:00
Pavel Kovalenko
98f8cbbdb2 DiskRestartProxy refactoring. 2021-04-20 21:58:34 +03:00
Pavel Kovalenko
e8b4dd1e4b Introduce DiskRestartProxy. 2021-04-20 21:55:57 +03:00
alexey-milovidov
4c1022ac03
Merge pull request #23318 from kssenii/fix-hdfs-with-spaces 
Fix hdfs reading from files with spaces
 2021-04-20 21:48:41 +03:00
Alexey Milovidov
9bc0706d2f Fix read in order optimization of COLLATE is present #22379 2021-04-20 21:20:45 +03:00
tavplubix
70c89eeff6
Merge pull request #23200 from ClickHouse/fix_prewhere_in_materialize_mysql 
Fix prewhere in MaterializeMySQL
 2021-04-20 20:05:07 +03:00
tavplubix
738d8a757b
Merge pull request #23189 from ClickHouse/minor_fixes_in_attach_query 
Minor fixes in ATTACH query
 2021-04-20 20:04:26 +03:00
Nikita Mikhaylov
16e7715039
Merge pull request #23202 from nikitamikhaylov/with_names_use_header_bug 
`CSVWithNames` with `input_format_with_names_use_header=0`
 2021-04-20 20:00:57 +03:00
alexey-milovidov
632a5288fa
Merge pull request #23348 from azat/zero-streams 
Fix LOGICAL_ERROR for max_distributed_connections=0
 2021-04-20 19:48:21 +03:00
alexey-milovidov
cbbff1bc4b
Merge pull request #23319 from ClickHouse/mysql-insert-without-database 
Allow to INSERT into remote MySQL storage with empty database #19281
 2021-04-20 19:30:44 +03:00
alexey-milovidov
a311a5c0b5
Merge pull request #23322 from azat/disable-min_bytes_to_use_mmap_io 
Disable min_bytes_to_use_mmap_io by default
 2021-04-20 19:27:47 +03:00
Anton Popov
3d390b095a
regenerate ya.make 2021-04-20 18:58:53 +03:00
Nikolai Kochetov
6b264933b2 Fix style. 2021-04-20 17:51:48 +03:00
Nikolai Kochetov
c808ffb04c
Update Pipe.cpp 2021-04-20 17:36:45 +03:00
Nikolai Kochetov
ecb2a96af8 Revert "Enable async_socket_for_remote" 
This reverts commit ff908baa06.
 2021-04-20 17:14:54 +03:00
Nikolai Kochetov
bc5b34fb6b
Update Pipe.cpp 
Add comment.
 2021-04-20 16:53:28 +03:00
Nikolai Kochetov
942cd54f23 Fix copy paste thanks to PVS studio. 2021-04-20 16:41:37 +03:00
Nikolai Kochetov
9632b83bdf Support explain json = 1, indexes = 1. Added test. 2021-04-20 16:39:23 +03:00
Nikita Mikhaylov
08839bc203
Merge pull request #23197 from nikitamikhaylov/read-task-response 
Fix logical error in stress tests
 2021-04-20 16:09:25 +03:00
tavplubix
bef0ab182e
Merge branch 'master' into fix_prewhere_in_materialize_mysql 2021-04-20 15:38:48 +03:00
tavplubix
d9ef1a9368
Update src/Storages/StorageMaterializeMySQL.cpp 
Co-authored-by: Alexander Kuzmenkov <36882414+akuzm@users.noreply.github.com>
 2021-04-20 15:38:02 +03:00
Nikita Mikhaylov
f55eacee7d
Merge pull request #22988 from excitoon-favorites/s3connectionresetretries 
Retries on HTTP connection drops during reads from S3
 2021-04-20 14:59:21 +03:00
Anton Popov
8f57141fad
Merge pull request #23195 from CurtizJ/nested-update 
Merging #22503
 2021-04-20 14:57:47 +03:00
Nikolai Kochetov
5e589ca6b2 Try fix #23029 2021-04-20 14:55:23 +03:00
Amos Bird
32c84f77c3
Resurrect indexHint function. 2021-04-20 19:27:23 +08:00
Alexander Tokmakov
4108a235f5 make separate functions 2021-04-20 14:22:02 +03:00
Maksim Kita
457b75a196 Updated tests 2021-04-20 13:31:01 +03:00
Maksim Kita
cbc34afd55 Updated QueryAliasVisitor to set prefer_alias_to_column_name for aliases like _subquery 2021-04-20 13:16:29 +03:00
vdimir
3266bbb54c
TableJoin: forceHashJoin for dictionary_reader, add tests 2021-04-20 12:52:52 +03:00
Nikolai Kochetov
c0514f439c Fix spelling 2021-04-20 11:26:54 +03:00
kssenii
da7a4ac1ea Better 2021-04-20 07:53:55 +00:00
Kseniia Sumarokova
b43656bf89
Merge pull request #22697 from TCeason/feature/add_MaterializeMySQL_sync_bytes_judgment 
Add MySQL read history data bytes judgment
 2021-04-20 10:26:42 +03:00
Nikolai Kochetov
0c4e789900 Remove debug output 2021-04-20 10:08:35 +03:00
Amos Bird
d5f606c544
Another fix 2021-04-20 14:15:28 +08:00
Azat Khuzhin
86069ffb40 Fix LOGICAL_ERROR for max_distributed_connections=0 2021-04-20 09:05:44 +03:00
Nikolai Kochetov
17f7631878 Fix build. 2021-04-20 08:55:20 +03:00
Nikolai Kochetov
548fb52d24 Fix build. 2021-04-20 08:51:51 +03:00
Anton Popov
0a0c510aa2 minor fixes 2021-04-20 05:51:07 +03:00
alexey-milovidov
62899436db
Revert "add uniqThetaSketch" 2021-04-20 03:34:21 +03:00
robot-clickhouse
db82e9e3d5 Auto version update to [21.6.1.1] [54451] 2021-04-20 01:48:20 +03:00
Russ Frank
4a742a17be feedback on deltasumtimestamp 2021-04-19 18:28:31 -04:00
Alexey Milovidov
ac68dd8e97 Fix bad code; allow to INSERT into MySQL without database name 2021-04-20 00:29:37 +03:00
Azat Khuzhin
6fba6f7ddd Disable min_bytes_to_use_mmap_io by default 
Reading files using mmap() does not have any significant benefits over
plain read() [1].

  [1]: https://gist.github.com/azat/3d6c8d82bdd91e7a38d997fd6bcfd574

And not only it does not have significant benefits, it also has some
issues, due to max_server_memory_usage (default to 90% of available
RAM), since when you read files with mmap() eventually process RSS may
exceed max_server_memory_usage, and in this case any allocation will
fail (with "Memory limit exceeded (total)") error (yes kernel will
unload pages, but likely it will happens after queries will starting to
fail), like in this test [2].

  [2]: https://gist.github.com/azat/4813489828162e6c2ce131963c6a1acb

TL;DR;

Note that there was also an idea to take those mmap()'ed regions in
memory tracking (#23211), but there are some drawbacks (since accounting
mmap() is tricky, first of all you need to account only once per inode
for file and plus kernel can unload some pages and those memory will not
be used by the server anymore).

And as an adddition to #23211 there was #23212, that adds
max_bytes_to_use_mmap_io, but since mmap is not a subject for memory
accounting there is no need in it.
 2021-04-20 00:25:16 +03:00
Alexey Milovidov
feae382273 Fix mistake 2021-04-19 23:39:25 +03:00
Alexey Milovidov
f492495507 Allow to INSERT into remote MySQL storage with empty database #19281 2021-04-19 23:36:13 +03:00
Alexey Milovidov
4ad4dbbc09 Regenerate Arcadia 2021-04-19 23:26:23 +03:00
alexey-milovidov
c87f846816
Merge pull request #21938 from azat/connection-settings 
Apply idle_connnection_timeout/poll_interval after each query
 2021-04-19 22:58:21 +03:00
alexey-milovidov
5b66086bc9
Merge pull request #23219 from azat/optimize_skip_unused_shards_rewrite_in-fix 
Add type conversion for optimize_skip_unused_shards_rewrite_in
 2021-04-19 22:45:35 +03:00
alexey-milovidov
4cef8c68f7
Merge pull request #23258 from azat/wundef 
Add -Wundef for gcc builds
 2021-04-19 22:45:22 +03:00
kssenii
cc5f49fbee Fix 2021-04-19 19:43:22 +00:00
Nikolai Kochetov
08d1896950 Fix build. 2021-04-19 22:06:05 +03:00
Nikolai Kochetov
394cca2434 Fix build. 2021-04-19 21:36:45 +03:00
Nikolai Kochetov
f6bdd35943 Try fix tests. 2021-04-19 20:55:18 +03:00
vdimir
a95c4dcfff
Correct sample_block in HashJoin for dict join with join_algorithm = auto 2021-04-19 18:02:56 +03:00
Nikolai Kochetov
8c2d58333a Merge branch 'master' into explain-output 2021-04-19 17:40:27 +03:00
Nikolai Kochetov
c59628846c
Merge pull request #22352 from ClickHouse/add-read-from-mt-step 
Special query plan step for read from MergeTree
 2021-04-19 17:38:40 +03:00
Nikolai Kochetov
ff908baa06 Enable async_socket_for_remote 2021-04-19 17:31:46 +03:00
Nikolai Kochetov
0420b7bb33 Reset replica state in MultiplexedConnections in case of exception. 2021-04-19 17:25:43 +03:00
Amos Bird
aeff06d67d
Don't relax NOT conditions during partition pruning. 2021-04-19 22:15:53 +08:00
Nikolai Kochetov
0c23d44b99 Add test. 2021-04-19 17:12:08 +03:00
Alexander Kuzmenkov
dd55fd8954
Merge pull request #23201 from ClickHouse/aku/negate-stable-formatting 
more stable formatting for negate()
 2021-04-19 16:48:43 +03:00
Alexander Kuzmenkov
2a4bcb6e3f
Merge pull request #23159 from ClickHouse/aku/merge-fusecount 
merging sumCount fusion PR #21337
 2021-04-19 16:47:13 +03:00
Nikita Mikhaylov
648b941922 done 2021-04-19 13:37:02 +00:00
Nikita Mikhaylov
adc103e338
Merge pull request #21852 from excitoon-favorites/s3stscredentials 
Added insecure IMDS credentials provider for S3
 2021-04-19 14:47:44 +03:00
Nikita Mikhaylov
6b928c4b6a
Merge pull request #23102 from godliness/read-orc-stripe 
Read ORC file by stripe to reduce memory cost
 2021-04-19 14:45:40 +03:00
Alexander Tokmakov
ecc625692e fix build 2021-04-19 13:40:20 +03:00
tavplubix
c9ebc95faf
Update ReplicatedMergeTreeCleanupThread.cpp 2021-04-19 13:27:32 +03:00
Alexander Tokmakov
539356b441 check block version on removal 2021-04-19 12:52:37 +03:00
Alexander Tokmakov
9bc98771f7 fix race on replica creation 2021-04-19 11:21:42 +03:00
Vladimir Chebotarev
9d4295f261 Removed infinite retries. 2021-04-19 11:02:36 +03:00
alesapin
14800f0c52
Merge pull request #23038 from ClickHouse/keeper_bench_mark 
Add tool for [Zoo]Keeper benchmark
 2021-04-19 10:47:29 +03:00
TCeason
b4bf53dfc9 add some comment and modify a parameter type 2021-04-19 15:34:27 +08:00
Kruglov Pavel
995973bf1f
Merge pull request #22609 from pingyu/datasketches-uniq 
add uniqThetaSketch
 2021-04-19 10:32:29 +03:00
Vladimir
e4e2b04c53
Merge pull request #23112 from Jokser/disk-s3-restore-to-detached 2021-04-19 10:29:00 +03:00
Nikolai Kochetov
4d86f51eff Merge branch 'master' into add-read-from-mt-step 2021-04-19 10:17:21 +03:00
Maksim Kita
2f56f8d040
Merge pull request #23278 from kitaisreal/to-date-time-decimal-overflow-ubsan-fix 
Function toDateTime decimal overflow ubsan fix
 2021-04-19 10:12:24 +03:00
TCeason
63403c709c modify settings name to external_xxx and rewrite Storage MySQL max_block_size 2021-04-19 10:51:50 +08:00
TCeason
87aa904440 Modify according to review opinion 2021-04-19 10:51:50 +08:00
TCeason
472c131420 Add MySQL read history data bytes judgment 2021-04-19 10:51:50 +08:00
Chao Ma
6238028225 Fix integration test test_storage_kafka failed error 2021-04-19 01:38:29 +00:00
Chao Ma
8d7a53efda Fix test 00163_column_oriented_formats failed error 2021-04-19 01:38:29 +00:00
Chao Ma
2776772ac6 Read ORC file by stripe to reduce memory cost 2021-04-19 01:38:29 +00:00
Maksim Kita
1bfd4f7568 Function toDateTime decimal overflow ubsan fix 2021-04-19 02:00:33 +03:00
alexey-milovidov
1d5a4cfa32
Revert "[RFC] Fix memory tracking with min_bytes_to_use_mmap_io" 2021-04-19 01:30:53 +03:00
Maksim Kita
ca93987ae9
Merge pull request #23182 from kitaisreal/system-query-reload-model 
Added system query reload model
 2021-04-19 00:39:31 +03:00
Maksim Kita
1f67362843
Merge pull request #23262 from kitaisreal/simple-key-dictionary-primary-key-wrong-order-fix 
Simple key dictionary primary key wrong order fix
 2021-04-19 00:37:52 +03:00
Azat Khuzhin
d476a4a9bd Add type conversion for optimize_skip_unused_shards_rewrite_in 
MSan reports [1]:

    2021.04.17 15:20:06.665152 [ 56 ] {2336bf92-0269-4acd-8b3f-f09623223d18} <Debug> executeQuery: (from [::1]:44744, using production parser) SELECT * FROM dist_01757 WHERE dummy IN ('255', 0) FORMAT Null
    ...
    0 0x305af885 in (anonymous namespace)::shardContains() obj-x86_64-linux-gnu/../src/Interpreters/OptimizeShardingKeyRewriteInVisitor.cpp:50:28
    ...
    Uninitialized value was created by an allocation of 'sharding_value' in the stack frame of function '_ZN12_GLOBAL__N_113shardContainsERKN2DB5FieldERKNSt3__112basic_stringIcNS4_11char_traitsIcEENS4_9allocatorIcEEEERKNS4_10shared_ptrINS0_17ExpressionActionsEEESC_RKNS0_7Cluster9ShardInfoERKNS4_6vectorImNS8_ImEEEE'
    0 0x305ae260 in (anonymous namespace)::shardContains() obj-x86_64-linux-gnu/../src/Interpreters/OptimizeShardingKeyRewriteInVisitor.cpp:42

  [1]: https://clickhouse-test-reports.s3.yandex.net/23212/d02eb3f6b7a8dc07bbed5605e1afcf7634436b3a/fuzzer_msan/report.html#fail1

P.S. fuzzers are great!
 2021-04-19 00:03:38 +03:00
Azat Khuzhin
2561a67fd8 Replace !__clang__ with !defined(__clang) to fix gcc builds 
$ gg 'if !__clang__' | cut -d: -f1 | sort -u | xargs sed -i 's/#if !__clang__/#if !defined(__clang__)/g'
 2021-04-18 23:37:50 +03:00
alesapin
912db13d75 Merge branch 'master' into keeper_bench_mark 2021-04-18 21:46:20 +03:00
Maksim Kita
b26dfcd514 Simple key dictionary primary key wrong order fix 2021-04-18 15:19:44 +03:00
Alexey Milovidov
21ff5d85d9 Disable hedged requests 2021-04-18 15:13:57 +03:00
alesapin
2035fb3edc Merge branch 'master' into keeper_bench_mark 2021-04-18 12:29:16 +03:00
Azat Khuzhin
3f156b83c9 Add some comments in OptimizeShardingKeyRewriteInVisitor 2021-04-18 09:07:51 +03:00
benbiti
0b0093fa4d updated: process libuv as cassandra 2021-04-18 09:50:56 +08:00
benbiti
a5b332b536 fix build with AMQP 2021-04-18 09:50:50 +08:00
alexey-milovidov
83038f84af
Revert "Function arrayFold for folding over array with accumulator" 2021-04-18 03:34:05 +03:00
alexey-milovidov
96fced4c3c
Merge pull request #23211 from azat/mmap-accounting 
[RFC] Fix memory tracking with min_bytes_to_use_mmap_io
 2021-04-18 03:17:30 +03:00
alexey-milovidov
7f3a40e1b0
Merge pull request #22237 from ClickHouse/protocol-compression-auto 
Autodetect compression #22234
 2021-04-18 03:13:54 +03:00
alexey-milovidov
75c4728787
Merge pull request #23229 from ClickHouse/unhex-case-insensitive1 
Make function `unhex` case insensitive for compatibility
 2021-04-17 23:32:03 +03:00
Alexander Tokmakov
0354312294 minor fixes in attach query 2021-04-17 22:43:02 +03:00
vdimir
826296c799
Convert columns to LC(Nullable(T)) in join 2021-04-17 22:03:32 +03:00
alexey-milovidov
212f68e279
Merge branch 'master' into fix-odbc 2021-04-17 21:29:03 +03:00
Maksim Kita
abf2a87ce7 Added system query reload model 2021-04-17 20:59:08 +03:00
Alexey Milovidov
8ac953f459 Make function unhex case insensitive for compatibility 2021-04-17 19:31:35 +03:00
alesapin
ce848ff2f0 Fix tidy 2021-04-17 17:06:49 +03:00
Alexey Milovidov
77e64b3ebd Merge branch 'master' into protocol-compression-auto 2021-04-17 16:46:51 +03:00
alexey-milovidov
138c6ae228
Merge pull request #23185 from ClickHouse/fix_22573 
Fix data_type_default_nullable on ATTACH
 2021-04-17 16:13:42 +03:00
alexey-milovidov
2634bccfc2
Merge pull request #23208 from ClickHouse/missing-semicolon 
Fixed missing semicolon in exception message #14978
 2021-04-17 16:03:07 +03:00
alexey-milovidov
a1ce6632d8
Merge pull request #23207 from ClickHouse/fix-bug-in-lowcardinality 
Fix fairly terrible bug in LowCardinality #16171
 2021-04-17 15:58:50 +03:00
alesapin
9fb3784f19 Merge branch 'master' into keeper_bench_mark 2021-04-17 15:58:33 +03:00
kssenii
5dc2dfa437 Fix 2021-04-17 09:00:58 +00:00
Azat Khuzhin
6064f3ef1d Fix memory tracking with min_bytes_to_use_mmap_io 
With min_bytes_to_use_mmap_io=1 memory regions for mmap() will not be
accounted. But this creates issues, since if you will run simple
"SELECT * FROM big_enough_table" it will mmap lot's of files and the
process RSS will grow at some point, and before the kernel will
understand that it needs to unmap physical pages the clickhouse-server
will sync total_memory_tracker with RSS of the process and the
allocations will fail by per-server memory limits.

Also note, that this does not address all the issues with mmap()
regions.

v2: move track/untrack into the MMapReadBufferFromFileWithCache to avoid
data-race [1]

  [1]: https://clickhouse-test-reports.s3.yandex.net/23211/6acf6cb9b22cee7db8d1aa523a6208f63e792f7a/functional_stateful_tests_(thread).html#fail1
 2021-04-17 10:37:38 +03:00
Nikita Mikhaylov
7a23d8292f done 2021-04-16 23:53:26 +00:00
Maksim Kita
3f3f928c1f
Merge pull request #21639 from changvvb/support-apple-m1 
Support Apple m1
 2021-04-17 02:44:15 +03:00
Alexey Milovidov
ec473a90d2 Fixed missing semicolon in exception message 2021-04-17 02:14:48 +03:00
Alexey Milovidov
36fe378a17 Fix fairly terrible bug in LowCardinality #16171 2021-04-17 02:11:21 +03:00
alexey-milovidov
786f340256
Merge pull request #22755 from kssenii/postgresql-protocol-with-row-policy 
Allow row policy in postgresql protocol
 2021-04-17 01:55:30 +03:00
Maksim Kita
d381e59791
Merge pull request #23114 from kitaisreal/fix-arcadia-build-s3 
Fix arcadia build S3
 2021-04-16 23:35:10 +03:00
Maksim Kita
4be441c6d5
Merge branch 'master' into support-apple-m1 2021-04-16 23:33:12 +03:00
alesapin
1b4e3ea1d4 Fix unit tests with new API 2021-04-16 23:07:40 +03:00
alesapin
5c7fa239e2 One more typo 2021-04-16 22:08:52 +03:00
Alexander Kuzmenkov
2782b2a6b0 fix 2021-04-16 21:50:12 +03:00
alesapin
8ff272cf7c Fix typo 2021-04-16 21:35:03 +03:00
alesapin
15cd9b116c Remove debug 2021-04-16 21:32:37 +03:00
alesapin
690045c67f Fix nasty bug 2021-04-16 21:31:23 +03:00
alexey-milovidov
74a0ac6917
Merge pull request #23160 from azat/logging-v2 
[RFC] Change logging from trace to debug for messages with rows/bytes
 2021-04-16 21:18:32 +03:00
Nikolai Kochetov
1bf420cbcf Update json description for other steps. 2021-04-16 19:36:59 +03:00
Alexander Kuzmenkov
b991cbf9d6 more stable formatting for negate() 2021-04-16 19:28:33 +03:00
Alexander Tokmakov
3b4bd18809 fix prewhere in materialize mysql 2021-04-16 18:51:27 +03:00
tavplubix
3d1885f08c
Merge pull request #23177 from marsno1/fix-exception-log 
Fix exception message for "parts_to_throw_insert"
 2021-04-16 18:15:07 +03:00
Pavel Kovalenko
760a2ccedf DiskS3 restore file format change and minor improvements. 2021-04-16 17:43:16 +03:00
alesapin
7ed32dddf0 More checks 2021-04-16 17:00:12 +03:00
alesapin
7f69910e97 Followup fix 2021-04-16 16:56:57 +03:00
alesapin
539f6cda8b Followup fix 2021-04-16 16:56:05 +03:00
alesapin
aecab0a1f6 Followup fix 2021-04-16 16:55:39 +03:00
alesapin
e724952c03 Fix obvious bug 2021-04-16 16:53:48 +03:00
alesapin
f168bfae6d More optimal 2021-04-16 16:50:09 +03:00
Maksim Kita
d88e0b1b44 Use local shard if there are no remote connections 2021-04-16 16:01:53 +03:00
Nikita Mikhaylov
0ddb396338 add Y_IGNORE 2021-04-16 15:52:48 +03:00
Nikolai Kochetov
8f969ec063 Revert "Ignore output format for explain query on client." 
This reverts commit 3f7cee1149.
 2021-04-16 15:44:27 +03:00
Nikolai Kochetov
a5f8b0f269 Revert "Omit EXPLAIN FORMAT for old TCP clients." 
This reverts commit 4c37d461de.
 2021-04-16 15:44:18 +03:00
Kruglov Pavel
deb86ee520
Merge pull request #22986 from azat/StorageBuffer-flush 
[RFC] Add ability to flush buffer only in background for StorageBuffer
 2021-04-16 15:14:24 +03:00
Anton Popov
b14798af2a Merge branch 'dev_nested_upadte' of git://github.com/hexiaoting/ClickHouse into nested-update 2021-04-16 14:41:19 +03:00
Nikolai Kochetov
d5882620b2 Fix condition 2021-04-16 14:33:20 +03:00
Alexander Kuzmenkov
74b4c56ba4
Merge pull request #21589 from dmalkr/dmalkr/arrayFold 
Function `arrayFold` for folding over array with accumulator
 2021-04-16 14:23:53 +03:00
Alexander Kuzmenkov
440efb6fb6
Update arrayFold.cpp 2021-04-16 14:22:23 +03:00
Nikolai Kochetov
23089a2fec Add special setting to show indexes. 2021-04-16 14:11:45 +03:00
Alexander Kuzmenkov
5e0fb440a5
Update TreeRewriter.cpp 2021-04-16 13:35:02 +03:00
Alexander Tokmakov
ebf0a3119e fix data_type_default_nullable on attach 2021-04-16 13:29:33 +03:00
Nikolai Kochetov
8d8e57615c A little bit better index description. 2021-04-16 12:42:23 +03:00
tavplubix
b5cbbce314
Merge pull request #22706 from songenjie/fetch_part 
[clickhouse][server][ddl]support fetch part
 2021-04-16 12:26:51 +03:00
tavplubix
ae3dc3dfc0
Merge pull request #23161 from ClickHouse/logs_for_debuging_test_failures 
More verbose logs for debuging test failures with Replicated and Keeper
 2021-04-16 12:01:58 +03:00
Maksim Kita
1797a67cf8
Merge pull request #22925 from kitaisreal/flat_dictionary_performance_test_fix 
Dictionaries updated performance tests
 2021-04-16 10:46:41 +03:00
madianjun
9238d8e54a Fix exception message for parts_to_throw_insert 2021-04-16 14:28:52 +08:00
Alexey Milovidov
9110a76d00 Reordered settings to avoid confusion 2021-04-16 00:14:37 +03:00
alexey-milovidov
dcb5d1b395
Merge pull request #23096 from ClickHouse/markdown-bad-alignment 
Fix unpleasant behaviour of `Markdown` format
 2021-04-16 00:06:58 +03:00
alexey-milovidov
081bb6e275
Merge pull request #23000 from ClickHouse/speedup-intdiv 
Dynamic dispatch for `intDiv`
 2021-04-16 00:05:43 +03:00
Pavel Kovalenko
84a420b806 Replace dynamic log getting with private log field in Disk classes. 2021-04-15 23:38:44 +03:00
Maksim Kita
94b228acc9
Update PocoHTTPClient.h 2021-04-15 21:51:15 +03:00
Alexander Tokmakov
0ad6205fa6 logs for debuging test failures with Replicated and Keeper 2021-04-15 21:34:53 +03:00
Azat Khuzhin
19e0439629 Add ability to flush buffer only in background for StorageBuffer 
Add 3 new engine arguments:
- flush_time
- flush_rows
- flush_bytes

That will be checked only for background flush, this maybe useful if
INSERT latency is "crucial".
 2021-04-15 21:22:13 +03:00
Azat Khuzhin
d2cf03ea41 Change logging from trace to debug for messages with rows/bytes 2021-04-15 21:00:16 +03:00
Nikolai Kochetov
be52b2889a Better description for key condition. 2021-04-15 20:30:04 +03:00
Alexander Kuzmenkov
2489b6af96 cleanup 2021-04-15 19:40:49 +03:00
Alexander Kuzmenkov
3b95b637a5 Merge remote-tracking branch 'origin/master' into HEAD 2021-04-15 18:19:53 +03:00
changvvb
0be5976967 Merge branch 'support-apple-m1' of github.com:changvvb/ClickHouse into support-apple-m1 2021-04-15 22:57:23 +08:00
changvvb
4eee2d0977 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into support-apple-m1 2021-04-15 22:54:26 +08:00
Alexey Milovidov
bd49e696c4 Minor change 2021-04-15 16:59:00 +03:00
Pavel Kruglov
847cd54c6b Add Y_IGNORE to includes 2021-04-15 15:06:40 +03:00
Pavel Kovalenko
6a067d705a Missed include. 2021-04-15 14:59:01 +03:00
Dmitry Krylov
bf51f94f37 Fix constness 2021-04-15 21:46:15 +10:00
Alexey Milovidov
d19b2cb948 Fix build 2021-04-15 14:41:55 +03:00
alexey-milovidov
b097140bb4
Merge pull request #23083 from azat/logging 
Tiny logging improvements
 2021-04-15 14:39:47 +03:00
Maksim Kita
e361f5943f
Merge pull request #22999 from azat/no-optimize_skip_unused_shards-single-node 
Do not perform optimize_skip_unused_shards for cluster with one node
 2021-04-15 14:36:56 +03:00
alexey-milovidov
69eac517b6
Merge pull request #23086 from azat/memory-blocking 
Block all memory tracking limits in dtors/SCOPE_EXIT_*SAFE/tryLogCurrentException
 2021-04-15 14:36:10 +03:00
Nikita Mikhaylov
a6b43f6765
Merge pull request #22594 from excitoon-favorites/betters3timeouts 
Consistent AWS timeouts
 2021-04-15 14:29:21 +03:00
Vladimir Chebotarev
f3e152739b Rebase fix. 2021-04-15 14:00:59 +03:00
Vladimir
8c7ffda676
Merge pull request #22753 from ClickHouse/revert-19685-dev_joinon 2021-04-15 12:42:50 +03:00
Maksim Kita
eceed68d62 Fix arcadia build S3 2021-04-15 12:31:09 +03:00
Dmitry Krylov
75036debf4 Check type match of lambda and accumulator 2021-04-15 19:25:30 +10:00
alesapin
50103ff0bc Merge branch 'master' into keeper_bench_mark 2021-04-15 11:30:33 +03:00
Pavel Kovalenko
e53fc635f6 Disk S3 restore test improvements. 2021-04-15 11:26:44 +03:00
Vladimir Chebotarev
95866e6e8d Rebase fix. 2021-04-15 10:57:08 +03:00
Vladimir Chebotarev
f59d35efd8 Rebase fix. 2021-04-15 10:53:00 +03:00
Vladimir Chebotarev
6e379fe778 Final style fix. 2021-04-15 10:01:47 +03:00
Vladimir Chebotarev
71621eb676 Style fix. 2021-04-15 10:01:47 +03:00
Vladimir Chebotarev
f85e521e47 Style fix. 2021-04-15 10:01:47 +03:00
Vladimir Chebotarev
342d907b65 Review fixes. 2021-04-15 10:01:47 +03:00
Vladimir Chebotarev
116423b30c Some refactoring. 2021-04-15 10:01:47 +03:00
Vladimir Chebotarev
c0ea3ce08a Removed STS client. 2021-04-15 10:01:43 +03:00
Vladimir Chebotarev
c61cfd0d97 Minor build fixes. 2021-04-15 10:01:16 +03:00
Vladimir Chebotarev
aae8a1f9d9 Moved AWSSTSAssumeRoleCredentialsProvider to the top of the provider chain. 2021-04-15 10:01:16 +03:00
Vladimir Chebotarev
3a467ffc50 Attempt to make credentials provider which uses internal credentials provider chain. 2021-04-15 10:01:13 +03:00
Vladimir Chebotarev
037a4de854 Added STS GetCallerIdentity credentials provider. 2021-04-15 10:00:18 +03:00
Vladimir Chebotarev
817c5cec25 Correct S3 read retries. 2021-04-15 09:47:22 +03:00
Vladimir Chebotarev
cf903d526c Better logging. 2021-04-15 09:47:22 +03:00
Vladimir Chebotarev
10bf56e540 Attempt to add retries to S3 reads. 2021-04-15 09:47:19 +03:00
Pavel Kovalenko
a2623a2d97 Rework Disk S3 revision number storing / searching. 2021-04-15 02:09:20 +03:00
Alexey Milovidov
15153e504a Fix unpleasant behaviour of Markdown format 2021-04-15 02:08:43 +03:00
alexey-milovidov
9b546f3b89
Merge pull request #22657 from amosbird/ctefix3 
Fix CTE usage in view definition
 2021-04-15 02:01:10 +03:00
Alexey Milovidov
2ae8839e3d Style 2021-04-15 01:28:40 +03:00
Alexey Milovidov
7f7e04117d Arcadia, ARM, PowerPC 2021-04-15 01:27:53 +03:00
Alexey Milovidov
f6f0d001cd Merge branch 'master' into speedup-intdiv 2021-04-15 01:27:15 +03:00
Pavel Kovalenko
cad81e3742 Disk S3 possibility to restore parts to 'detached' directory. 2021-04-15 00:55:48 +03:00
Alexey Milovidov
17d04cbca9 Merge branch 'master' into protocol-compression-auto 2021-04-15 00:24:07 +03:00
tavplubix
2479c80fb7
Merge branch 'master' into fix_attach_mv 2021-04-15 00:07:48 +03:00
Azat Khuzhin
9b9e0a9bbc Block all memory tracking limits in destructors 2021-04-14 23:38:42 +03:00
Azat Khuzhin
22d394c4a3 Block all memory tracking limits in tryLogCurrentException() 2021-04-14 23:38:41 +03:00
Azat Khuzhin
b6d7562cc2 Update comments for memoryTrackerCanThrow() 2021-04-14 23:38:41 +03:00
tavplubix
021cafff49
Merge pull request #23041 from ClickHouse/fix_testkeeper_multi_response 
Fix multi response in TestKeeper
 2021-04-14 23:25:40 +03:00