Commit Graph

11382 Commits

Author SHA1 Message Date
Anton Ivashkin
109cb634e1 Remove debug lines 2021-03-11 12:08:19 +03:00
Yatsishin Ilya
abbf4b01c2 Merge remote-tracking branch 'origin/master' into fixes-clang-12-cmake-3.19 2021-03-11 09:44:30 +03:00
alexey-milovidov
647246006a
Merge pull request #21533 from ClickHouse/podarray-cleanup
Cleanup PODArray
2021-03-11 04:34:25 +03:00
Maksim Kita
583b25b68e
Merge pull request #21513 from kitaisreal/direct-dictionary-fix
DirectDictionary updated
2021-03-11 00:01:01 +03:00
tavplubix
26ebd54eaf
Merge pull request #20961 from havardk/reconnect
MaterializeMySQL: Attempt to reconnect to MySQL if the connection is lost
2021-03-10 20:07:11 +03:00
Yatsishin Ilya
7b69ef92be gcc old cast check failed 2021-03-10 18:04:34 +03:00
Anton Ivashkin
6f7800ecb4 Fix PVS detected errors 2021-03-10 16:27:08 +03:00
Nikolai Kochetov
7eb6b87415 Added comments. 2021-03-10 14:50:51 +03:00
Anton Ivashkin
cef0dda933 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into s3_zero_copy_replication 2021-03-10 14:10:29 +03:00
Anton Ivashkin
deb0b83a12 Fix refreshState logic 2021-03-10 14:08:49 +03:00
Nikolai Kochetov
4c8c45b163 Refactor 2021-03-10 14:00:24 +03:00
Alexander Kuzmenkov
b399f80b7e
Merge pull request #21572 from ClickHouse/aku/frame-typo
fix a typo in window functions frame
2021-03-10 13:38:39 +03:00
Pervakov Grigorii
eb87afc4d4 recreate S3 client if credentials changed 2021-03-10 13:36:57 +03:00
Vladimir
33dce1296b
Merge pull request #20392 from vdimir/cross-to-inner-join-rewrite 2021-03-10 09:53:55 +03:00
alexey-milovidov
f4db00bdb2
Merge pull request #21546 from kitaisreal/shell-command-wait-pid-signal-eintr-fix
ShellCommand waitpid eintr signal fix
2021-03-10 09:12:53 +03:00
Azat Khuzhin
61d40c3600 Fix optimize_skip_unused_shards for zero shards case
v2: move check to the beginning of the StorageDistributed::read()
2021-03-10 09:05:14 +03:00
alexey-milovidov
2159fb8481
Merge branch 'master' into governance/query_log 2021-03-10 05:29:56 +03:00
fuwhu
69f516cd47 Refine MergeTreeData::loadDataParts to not parse format version file and detached directory 2021-03-10 10:10:25 +08:00
Maksim Kita
9509516e3c Fixed tests 2021-03-10 01:26:06 +03:00
Alexander Kuzmenkov
8c5240e3a0 fix a typo in window functions frame 2021-03-10 01:24:46 +03:00
Maksim Kita
5e03418651 Added performance tests 2021-03-09 23:50:36 +03:00
Maksim Kita
ed295a934c Fixed tests 2021-03-09 23:50:36 +03:00
Maksim Kita
f9b0fdad14 Fixed tests 2021-03-09 23:50:36 +03:00
Maksim Kita
b5f51e0ca6 Fixed style check 2021-03-09 23:50:06 +03:00
Maksim Kita
45879472d3 DirectDictionary updated 2021-03-09 23:49:34 +03:00
Amos Bird
e52b2fb8ae
update 2021-03-10 02:56:46 +08:00
Anton Ivashkin
aff13c0c52 Make method StorageReplicatedMergeTree::fetchExistsPart 2021-03-09 20:49:50 +03:00
Anton Ivashkin
265d293934 Use 'merge on single replica' option instead of zookeeper lock 2021-03-09 20:39:55 +03:00
tavplubix
5a6cc876e4
Merge pull request #20626 from ClickHouse/fix_force_drop_materialized_view
Make force_drop_table work with MaterializedView
2021-03-09 20:05:51 +03:00
Maksim Kita
43ba554191
Merge pull request #20595 from kitaisreal/cache-dictionaries-lru-cache
Cache dictionaries lru cache
2021-03-09 19:14:30 +03:00
alesapin
bcd073278d
Merge pull request #21425 from ClickHouse/persistent_nukeeper_snapshot_storage
Persistent nukeeper snapshot storage
2021-03-09 18:37:23 +03:00
alesapin
bbfb6c385d
Merge pull request #21544 from nicelulu/fix_fd_ready
Fix incorrect fd_ready assignment in NuKeeperTCPHandler.
2021-03-09 18:23:24 +03:00
Yatsishin Ilya
71d745b4ec implicit const conversion 2021-03-09 17:31:54 +03:00
Alexander Kuzmenkov
0de89e2925
Merge pull request #19401 from CurtizJ/aggregating-in-order
Try improve performance of aggregation in order of sorting key
2021-03-09 16:53:32 +03:00
Maksim Kita
289712f19d
Merge pull request #21532 from kitaisreal/pod-array-left-pad-not-multiple-of-element-crash-fix
Pod array left pad not multiple of element crash fix
2021-03-09 14:14:52 +03:00
Maksim Kita
8dd52f087e Fixed PODArray 2021-03-09 14:14:31 +03:00
Alexander Kuzmenkov
4b0cbb6ed7
Update FinishAggregatingInOrderAlgorithm.h 2021-03-09 13:43:06 +03:00
Maksim Kita
882dde56ea ShellCommand waitpid eintr signal fix 2021-03-09 13:13:19 +03:00
zhangxiao871
846f5a93a9 Fix incorrect fd_ready assignment in NuKeeperTCPHandler. 2021-03-09 17:42:27 +08:00
alesapin
d80c2cef06 Slightly better 2021-03-09 11:45:41 +03:00
alesapin
c00c8c36b7 Merge branch 'master' into ianton-ru-s3_zero_copy_replication 2021-03-09 10:59:11 +03:00
Azat Khuzhin
3474ea044e Avoid processing optimize_skip_unused_shards twice 2021-03-09 10:05:56 +03:00
vdimir
36006a7e96
Merge branch 'master' into cross-to-inner-join-rewrite 2021-03-09 09:36:15 +03:00
Vladimir
6ec19b2632
Merge pull request #19885 from ClickHouse/vdimir/join-cast-types-v2 2021-03-09 09:30:50 +03:00
alexey-milovidov
0a3275e169
Merge pull request #21496 from Felixoid/rollup_grow
Add check for growing age AND precision in rollup config
2021-03-09 07:31:07 +03:00
Kruglov Pavel
c464e3c57b
Merge pull request #21311 from ucasFL/fix-format
improve clickhouse-format
2021-03-09 02:16:05 +03:00
Alexey Milovidov
1b3043a583 Fix error (found by @kitaisreal) 2021-03-09 02:15:33 +03:00
Alexey Milovidov
d3aeb30cc0 Remove unused method 2021-03-09 02:10:26 +03:00
Alexey Milovidov
9e23d16739 Cleanup PODArray 2021-03-09 02:03:46 +03:00
Maksim Kita
9ed791fd30 Updated style check 2021-03-09 01:33:16 +03:00
Maksim Kita
642c45d769 PODArray left pad not multiple of element crash fix 2021-03-09 01:32:31 +03:00
Maksim Kita
edb673af56 Fixed tests 2021-03-09 01:03:25 +03:00
alesapin
f555e65cc8 Merge branch 'master' into persistent_nukeeper_snapshot_storage 2021-03-08 22:30:17 +03:00
vdimir
cc770ad441
Fixes for join key inferring
Fix error with wrong right column type in aggregate with nulls

Add logging inferred type for join keys

Add test for join_use_nulls
2021-03-08 17:21:50 +03:00
Amos Bird
2ec20c5d23
update and add tests 2021-03-08 17:38:07 +08:00
alesapin
94248dd205 Fix one more nasty bug 2021-03-08 11:53:52 +03:00
Vasily Nemkov
1711761459
Merge branch 'master' into governance/query_log 2021-03-08 10:52:27 +03:00
vdimir
354757d94d
Add logging for need_analyze_again in InterpreterSelectQuery 2021-03-08 09:33:30 +03:00
Amos Bird
2b11a2a855
Fix build 2021-03-08 11:10:35 +08:00
Amos Bird
e6522e1ebe
JBOD data balancer 2021-03-08 11:10:35 +08:00
Slach
5791cf5f2b fix review comments https://github.com/ClickHouse/ClickHouse/pull/21078#discussion_r580413875 and https://github.com/ClickHouse/ClickHouse/pull/21078#discussion_r580410204
Signed-off-by: Slach <bloodjazman@gmail.com>
2021-03-08 03:39:24 +03:00
Alexey Milovidov
460658aeab Update tests 2021-03-08 03:18:55 +03:00
Alexey Milovidov
019ed517bc Fix gcc warning 2021-03-08 03:05:06 +03:00
Alexey Milovidov
355411dfa8 Merge branch 'master' into DateTime64_extended_range 2021-03-08 02:55:14 +03:00
Maksim Kita
492c3d0899
Merge pull request #21321 from kitaisreal/dictionary-source-executable-pool
Added ExecutablePool dictionary source
2021-03-08 01:36:33 +03:00
alesapin
fb264c12d3 Blind fix 2021-03-08 00:40:32 +03:00
Maksim Kita
b789f80838 Fixed style check 2021-03-07 14:45:12 +03:00
Maksim Kita
92107a6ded Fixed documentation message 2021-03-07 14:43:08 +03:00
Maksim Kita
9512bfd6d7 Fixed broken pipe exception handling 2021-03-07 14:41:53 +03:00
alesapin
ef192ec310 Merge branch 'master' into persistent_nukeeper_snapshot_storage 2021-03-07 10:31:06 +03:00
alesapin
0ee7425b16 Remove redundant methods 2021-03-07 10:30:50 +03:00
Alexey Milovidov
1bb62f578b Unit tests passing 2021-03-07 04:10:38 +03:00
Maksim Kita
cb5a93bcec Updated exception handling 2021-03-07 02:39:19 +03:00
Alexey Milovidov
8271cec093 Fix error 2021-03-07 02:36:24 +03:00
Alexey Milovidov
e33fae7625 Improve test 2021-03-07 01:35:49 +03:00
Alexey Milovidov
db63668d03 Merge branch 'master' into DateTime64_extended_range 2021-03-06 23:51:27 +03:00
Maksim Kita
6414da73bf Fixed code review issues 2021-03-06 21:45:51 +03:00
Mikhail f. Shiryaev
efc04b29a6
Add check for growing age AND precision in rollup config 2021-03-06 18:35:53 +01:00
alesapin
a8b7469857 Fix session timeout update 2021-03-06 17:14:38 +03:00
robot-clickhouse
4e61f45dbb Auto version update to [21.4.1.1] [54449] 2021-03-06 14:43:30 +03:00
Maksim Kita
4db713103b Fixed documentation 2021-03-06 14:38:27 +03:00
Nikita Mikhaylov
ca3610f66f Update ya.make 2021-03-06 14:38:27 +03:00
Nikita Mikhaylov
2fe8fe17d9 Update ya.make.in 2021-03-06 14:38:27 +03:00
Maksim Kita
10c5518988 Fix tests 2021-03-06 14:38:27 +03:00
Maksim Kita
64e8f09649 Fix tests 2021-03-06 14:37:56 +03:00
Maksim Kita
c314de7388 Fixed documentation 2021-03-06 14:37:56 +03:00
Maksim Kita
aa85194bb0 Fix race 2021-03-06 14:37:56 +03:00
Maksim Kita
252f7604ee Fixed minor issues 2021-03-06 14:37:56 +03:00
Maksim Kita
8ea6c4b852 Updated Dictionaries default value handling 2021-03-06 14:37:56 +03:00
Maksim Kita
717a9b048a Fixed tests 2021-03-06 14:37:36 +03:00
Maksim Kita
a6534c3b42 Fix update position of duplicated keys in memory partition buffer 2021-03-06 14:36:37 +03:00
Maksim Kita
7602c6c6cb Fixed sanitizers errors 2021-03-06 14:36:37 +03:00
Maksim Kita
6ab451d41d Updated tests 2021-03-06 14:36:37 +03:00
Maksim Kita
92b79420a4 Fixed ubsan test 2021-03-06 14:36:37 +03:00
Maksim Kita
67edeadcc3 Fix dictionaries cmake test 2021-03-06 14:36:37 +03:00
Maksim Kita
ae88bbda5a Updated CacheDictionary 2021-03-06 14:36:37 +03:00
Maksim Kita
ef1645b546 Fix test 2021-03-06 14:36:37 +03:00
Maksim Kita
2a79d77c1b Fixed style check 2021-03-06 14:36:37 +03:00
Maksim Kita
e64b18fbac Added dictGet with multiple requested attributes 2021-03-06 14:36:37 +03:00
Maksim Kita
ccb61064e1 Fix build issues 2021-03-06 14:36:37 +03:00
Maksim Kita
caea9ef734 Fix test 2021-03-06 14:36:37 +03:00
Maksim Kita
e830eb41b1 Fixed build 2021-03-06 14:36:37 +03:00
Maksim Kita
8ee89b59ec Fixed build 2021-03-06 14:36:37 +03:00
Maksim Kita
d04e707567 Fixed special build 2021-03-06 14:36:37 +03:00
Maksim Kita
480beeda37 Fix gcc build 2021-03-06 14:36:37 +03:00
Maksim Kita
a22ec65e8a Fixed minor issues 2021-03-06 14:36:37 +03:00
Maksim Kita
8138ced862 Fixed tests 2021-03-06 14:36:37 +03:00
Maksim Kita
2a5a9c6ca4 Fixed hierarchy tests 2021-03-06 14:36:37 +03:00
Maksim Kita
980d171207 Fixed typos 2021-03-06 14:36:37 +03:00
Maksim Kita
168741c67c SSDCacheDictionaryFileBuffer create directory for file 2021-03-06 14:36:37 +03:00
Maksim Kita
1460d57f97 Fixed segfault because of member order of destruction 2021-03-06 14:36:37 +03:00
Maksim Kita
0edb9d8e3d Added ICacheDictionaryStorage returnsFetchedColumnsInOrderOfRequestedKeys 2021-03-06 14:36:37 +03:00
Maksim Kita
c5fbe8793b Added documentation 2021-03-06 14:36:37 +03:00
Maksim Kita
dc0bb7485d Updated CacheDictionary 2021-03-06 14:36:37 +03:00
Maksim Kita
b6e72334ef Fixed DefaultHash for StringRef 2021-03-06 14:36:37 +03:00
Maksim Kita
9763c4a4fb IColumn added skipSerializedInArena method 2021-03-06 14:36:37 +03:00
Pavel Kruglov
dbae58f30b Add slowdown_count and show actual information in system.clusters 2021-03-06 02:45:17 +03:00
Alexey Milovidov
2b1ab0f927 Merge branch 'master' into DateTime64_extended_range 2021-03-06 02:25:43 +03:00
alexey-milovidov
af2135ef9d
Merge pull request #21477 from ClickHouse/fix_mutation_deadlock
Fix deadlock in delete mutations when predicate contains table itself
2021-03-06 02:08:00 +03:00
alexey-milovidov
35b304bdb1
Merge pull request #21393 from ClickHouse/dict-name-dot
Add a test for #15469
2021-03-06 01:47:17 +03:00
alexey-milovidov
2d8632e5a9
Update ExecutablePoolDictionarySource.cpp 2021-03-06 01:14:21 +03:00
alexey-milovidov
19e86f243e
Update src/Dictionaries/ExecutableDictionarySource.cpp
Co-authored-by: Nikita Mikhaylov <mikhaylovnikitka@gmail.com>
2021-03-06 01:06:40 +03:00
alesapin
5b3161e0b5 Get rid of const_cast 2021-03-05 20:24:06 +03:00
tavplubix
9f04683923
Merge pull request #21461 from ClickHouse/just_another_fix_for_ddl_worker
Just another fix for DDLWorker
2021-03-05 19:44:59 +03:00
Anton Ivashkin
d08b481660 Fixes by review responces 2021-03-05 19:20:38 +03:00
vdimir
4a8708fbb2
Clean subqueries_for_sets before second analyze in InterpreterSelectQuery 2021-03-05 17:43:26 +03:00
vdimir
5c18bbeb43
Clean converting-related fields in TableJoin::resetCollected 2021-03-05 17:41:39 +03:00
vdimir
b0f6f30298
Revert "Refactor join: make IJoin implementations independent from TableJoin"
This reverts commit e6406c3f4c.
2021-03-05 17:34:43 +03:00
feng lv
1438fd20cb Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-format 2021-03-05 14:33:26 +00:00
Anton Popov
562065732d Merge remote-tracking branch 'upstream/master' into HEAD 2021-03-05 17:30:05 +03:00
vdimir
e6406c3f4c
Refactor join: make IJoin implementations independent from TableJoin 2021-03-05 16:38:49 +03:00
alesapin
1bfc27ab6f Fix shutdown 2021-03-05 16:06:47 +03:00
tavplubix
35a770a7f6
add comment 2021-03-05 15:50:10 +03:00
alesapin
aff86b1da2 Fix deadlock in delete mutations when predicate contains table itself 2021-03-05 15:32:55 +03:00
alesapin
ce047b4fe1 Remove unused method 2021-03-05 14:06:37 +03:00
alesapin
15ea9a9c0c Async version 2021-03-05 13:40:24 +03:00
Maksim Kita
cdee265378 Fix typos 2021-03-05 13:27:24 +03:00
Maksim Kita
175c485fcb Fix style check 2021-03-05 13:17:46 +03:00
Maksim Kita
9173544003 Added documentation 2021-03-05 13:17:46 +03:00
Maksim Kita
aa0778933c Added BorrowedObjectPool 2021-03-05 13:17:46 +03:00
Maksim Kita
91e43a6b3b Updated ExecutablePoolDictionarySource 2021-03-05 13:17:46 +03:00
Maksim Kita
fd4331083a Removed debug log 2021-03-05 13:17:46 +03:00
Maksim Kita
d3615aca67 Added tests 2021-03-05 13:17:46 +03:00
Maksim Kita
c773afb659 Added documentation 2021-03-05 13:17:46 +03:00
Maksim Kita
6b69d859c1 Added ExecutablePool dictionary source 2021-03-05 13:17:45 +03:00
Anton Popov
87da7f0589
Update FinishAggregatingInOrderAlgorithm.h 2021-03-05 12:50:28 +03:00
alesapin
c9c301e10c Merge branch 'master' into persistent_nukeeper_snapshot_storage 2021-03-05 10:12:09 +03:00
alesapin
9a9d5a74ba
Merge pull request #21439 from ClickHouse/fix_nukeeper_default_path
Use path as default prefix for coordination logs
2021-03-05 10:11:44 +03:00
alexey-milovidov
777731562d
Merge pull request #21458 from amosbird/trivialfix
Tiny fix
2021-03-05 05:21:52 +03:00
Alexey Milovidov
dce79d928f Merge branch 'dict-name-dot' of github.com:yandex/ClickHouse into dict-name-dot 2021-03-05 05:07:36 +03:00
Alexander Tokmakov
614e0d9ba0 just another fix for ddl worker 2021-03-05 04:17:51 +03:00
Alexey Milovidov
3e77c9b1c5 Merge branch 'master' into DateTime64_extended_range 2021-03-05 03:07:19 +03:00
Nikita Mikhaylov
37d4c69d3a
Merge pull request #21433 from azat/parsing-exception-message-fix
Fix ParsingException::displayText()
2021-03-05 01:59:06 +03:00
Amos Bird
de94f33b05
Tiny fix 2021-03-05 00:21:52 +08:00
Kseniia Sumarokova
08cb02c872
Merge pull request #21416 from excitoon-favorites/caseinsensitivecompression
Case-insensitive compression methods for table functions
2021-03-04 17:37:28 +03:00
Maksim Kita
1eb1f3028c FunctionsExternalDictionaries safe case to ColumnTuple 2021-03-04 16:55:05 +03:00
alesapin
aa754a3ca8 Test restore from snapshot 2021-03-04 16:40:43 +03:00
alesapin
77dbe3fee8 Get rid of network order 2021-03-04 16:02:30 +03:00
alesapin
27011f086e More style 2021-03-04 15:31:05 +03:00
alesapin
42c1a65b0f Style 2021-03-04 15:30:26 +03:00
alesapin
c8423249db Less strict checks 2021-03-04 15:01:56 +03:00
alesapin
aa1312ac79 Merge with master 2021-03-04 14:43:12 +03:00
alesapin
ff67860770 Merge branch 'master' into persistent_nukeeper_snapshot_storage 2021-03-04 14:30:51 +03:00
alesapin
5565c37b4f tests and fixes for off by one error 2021-03-04 14:22:59 +03:00
Anton Ivashkin
bec9de7161 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into s3_zero_copy_replication 2021-03-04 13:29:45 +03:00
Ivan
218542589a
Merge pull request #19673 from azat/distributed-bytes_to_throw_insert
Add ability to throttle INSERT into Distributed
2021-03-04 13:28:23 +03:00
Anton Ivashkin
e69124a0a6 Merge master 2021-03-04 13:26:40 +03:00
alesapin
3a0b9102d8 Merge branch 'fix_nukeeper_default_path' into persistent_nukeeper_snapshot_storage 2021-03-04 11:30:54 +03:00
alesapin
d28463d70b Use path as default prefix for coordination logs 2021-03-04 11:29:24 +03:00
alesapin
7556a3d723 Enable snapshots 2021-03-04 11:00:26 +03:00
Haavard Kvaalen
641b1b249e Attempt to reconnect to MySQL
For MaterializeMySQL databases, attempt to reconnect if the connection
to MySQL is lost.  The existing setting
`max_wait_time_when_mysql_unavailable` is used to control how often we
attempt to reconnect.  This setting can now be set to a negative value
to disable reconnects.
2021-03-04 08:49:06 +01:00
Haavard Kvaalen
11c9c8cb10 Reset sequence number on disconnect
Reestablishing the connection for an existing MySQLClient object would
never work because the sequence number would be wrong on the new
connection.
2021-03-04 08:49:06 +01:00
Haavard Kvaalen
8e95d6e174 Add method to metadata class to start replication
Previously this was done by the constructor.  This makes it
possible to restart replication with an existing metadata object.
2021-03-04 08:49:06 +01:00
Haavard Kvaalen
e6711675a1 Make handling of unavailable MySQL consistent
If MySQL was unavailable when loading an existing database on startup,
we would previously load the database, fail to start synchronization,
but allow queries to the database.  Change this to only allow queries
if the `allows_queries_when_mysql_lost` setting is on, so that the
behavior is consistent with what happens if the connection to MySQL is
lost while ClickHouse is running.

Also retry connection to MySQL if MySQL is unavailable when ClickHouse
is started (we would previously reconnect only if the connection was
lost during the initial dump of existing data).
2021-03-04 08:49:06 +01:00
Azat Khuzhin
45ca803973 Fix logging for optimize_aggregation_in_order=1 (with small max_block_size) 2021-03-04 10:34:41 +03:00
Alexey Milovidov
9416f59014 Some progress on tests 2021-03-04 06:48:24 +03:00
Alexey Milovidov
b9a8509f79 Adjustments 2021-03-04 05:18:17 +03:00
Alexey Milovidov
6dc21cd098 Remove discrepances 2021-03-04 03:48:37 +03:00
Alexey Milovidov
9f3760e1e3 Bad filename 2021-03-04 03:18:48 +03:00
Alexey Milovidov
7f1ae506fc The code is too complicated 2021-03-04 03:11:40 +03:00
Alexey Milovidov
3eb3830dd8 Remove trash 2021-03-04 02:13:49 +03:00
Alexey Milovidov
1096fc328a Make some tests automated 2021-03-04 02:11:27 +03:00
Alexey Milovidov
329074bd02 Fix test 2021-03-04 01:55:56 +03:00
Alexey Milovidov
2632b568ae Move tests to appropriate place 2021-03-04 01:34:30 +03:00
Azat Khuzhin
e858dae187 Fix SIGSEGV for distributed queries on failures
Since after pull(), cancel() will be called, and this will lead to
SIGSEGV, since there is no exception, but has_exception was not reseted
in pull()
2021-03-04 00:42:49 +03:00
Azat Khuzhin
e7948819f9 Fix ParsingException::displayText()
Before it was silently try-catched for messages with additional {}, and
it is very easy to trigger, i.e.:

    SELECT toDateTime(format('{}-{}-01 00:00:00', '2021', '1'))

Will print:

    Code: 41. DB::Exception: Received from localhost:9000. DB::Exception: Cannot parse datetime 2021-1-01 00:00:00{}: Cannot parse DateTime from String: while executing 'FUNCTION toDateTime(format('{}-{}-01 00:00:00', '2021', '1') :: 3) -> toDateTime(format('{}-{}-01 00:00:00', '2021', '1')) DateTime : 2'.
2021-03-03 23:44:51 +03:00
Alexey Milovidov
4e8239e098 Merge branch 'master' into DateTime64_extended_range 2021-03-03 23:43:20 +03:00
Azat Khuzhin
79ed35876e DirectoryMonitor: Remove const qualifier and lots of mutable qualifiers 2021-03-03 23:30:24 +03:00
Azat Khuzhin
45ee650e26 Distributed: check for bytes_to_throw/delay_insert only before INSERT
Before it was checked for each block.
2021-03-03 23:30:24 +03:00
Azat Khuzhin
6965ac26c3 Distributed: Add ability to delay/throttle INSERT until pending data will be reduced
Add two new settings for the Distributed engine:
- bytes_to_delay_insert
- max_delay_to_insert

If at the beginning of INSERT there will be too much pending data, more
then bytes_to_delay_insert, then the INSERT will wait until it will be
shrinked, and not more then max_delay_to_insert seconds.

If after this there will be still too much pending, it will throw an
exception.

Also new profile events were added (by analogy to the MergeTree):
- DistributedDelayedInserts (although you can use system.errors instead
  of this, but still)
- DistributedRejectedInserts
- DistributedDelayedInsertsMilliseconds
2021-03-03 23:30:23 +03:00
Azat Khuzhin
15f7459cae Distributed/DirectoryMonitor: protect metric_pending_files with metrics_lock
Since there is local value, that is not atomic, anyway we already have
lock for metrics, so it is fine.
2021-03-03 23:30:03 +03:00
Azat Khuzhin
70049db143 CurrentMetrics/Increment: Introduce add() 2021-03-03 23:30:03 +03:00
Azat Khuzhin
017c054a35 Distributed/DirectoryMonitor: Use std::lock_guard over std::unique_lock
It is more natural, since we do not need lazy locking.
2021-03-03 23:30:03 +03:00
Azat Khuzhin
fcf49a4914 Distributed: Calculate counters for async INSERT at INSERT time
Previous patch fixes the inaccuracy, but it's done using iterating over
directory on each request (to system.distribution_queue or to check
bytes_to_throw_insert), and like previous patch alredy stated, it may
have pretty huge overhead (especially when you have lots of distributed
files pending).

This patch remove that recalculation (but it will still be done, and
if there is different, there will be a log message), and replace it with
proper account at INSERT time (and after file has been sent, or marked
as broken).
2021-03-03 23:30:03 +03:00
Azat Khuzhin
b43046ba06 Distributed: More accurate distribution_queue counters
So now system.distribution_queue will show accurate statistics, so tests
does not requires sleep anymore.

But note that with too much distributed pending this will iterate over
all directories.
2021-03-03 23:30:03 +03:00
Azat Khuzhin
b5a5778589 Distributed: Add ability to limit amount of pending bytes for async INSERT
Right now with distributed_directory_monitor_batch_inserts=1 and
insert_distributed_sync=0 INSERT into Distributed table will store
blocks that should be sent to remote (and in case of
prefer_localhost_replica=0 to the localhost too) on the local
filesystem, and sent it in background.

However there is no limit for this storage, and if the remote is
unavailable (or some other error), these pending blocks may take
significant space, and this is not always desired behaviour.

Add new Distributed setting - bytes_to_throw_insert, that will set the
limit for how much pending bytes is allowed, if the limit will be
reached an exception will be throw.

By default was set to 0, to avoid surprises.
2021-03-03 23:30:00 +03:00
Azat Khuzhin
ce09b7ff89 Distributed: Implement totalBytes() (system.tables.total_bytes) 2021-03-03 23:29:11 +03:00
Azat Khuzhin
456cbaf747 Distributed: Hide private part of the interface 2021-03-03 23:29:11 +03:00
Azat Khuzhin
8531431bbc getClusterName: Use std::string over String (since it includes only <string>) 2021-03-03 23:29:11 +03:00
alesapin
4e175bd988 Useful comments 2021-03-03 19:27:08 +03:00
alesapin
e6083c4dc4 Fix style 2021-03-03 19:25:47 +03:00
alesapin
eb5e416fd5 Merge branch 'master' into persistent_nukeeper_snapshot_storage 2021-03-03 19:19:02 +03:00
alesapin
7e28bfeb9e Fix log store start 2021-03-03 18:37:31 +03:00
Pavel Kruglov
dc9dd8ccdc Fix typo 2021-03-03 16:16:17 +03:00
alesapin
186b39f0d5 Followup fix 2021-03-03 15:31:21 +03:00
alesapin
0136d7d51a Get rid of unused field 2021-03-03 15:29:00 +03:00
alesapin
37fc5faa6f Fix nasty serialization bug 2021-03-03 15:21:21 +03:00
Vladimir Chebotarev
4022418ff3 Case-insensitive compression methods. 2021-03-03 14:51:09 +03:00
alesapin
1707e7f1c3 Rename file 2021-03-03 14:22:38 +03:00
alesapin
e615299ae6 Fix some bugs and add snapshots validation 2021-03-03 14:10:24 +03:00
Nikolai Kochetov
cb122160ae
Merge pull request #19291 from Avogar/hedged-requests
Implementation of HedgedRequests
2021-03-03 12:58:37 +03:00
feng lv
742e5a05ff fix conflict 2021-03-03 09:38:30 +00:00
alesapin
07e39ef47a Merge branch 'master' into persistent_nukeeper_snapshot_storage 2021-03-03 11:12:01 +03:00
alesapin
6b1005aea5
Merge pull request #21334 from ClickHouse/fix_alter_partition_key
Fix alter modify query for partition key and other metadata fields
2021-03-03 10:15:11 +03:00
alexey-milovidov
bd7b540b82
Merge pull request #21307 from amosbird/moduleopt
Specialize intDiv/module
2021-03-03 04:49:29 +03:00
Anton Popov
a4c00ab5dc
Merge pull request #21303 from ucasFL/forbid
Forbid to drop a column if it's referenced by materialized view
2021-03-03 02:55:06 +03:00
alexey-milovidov
33bb77f777
Merge pull request #21367 from amosbird/mysqlfix
mysql client compatibility
2021-03-03 02:13:58 +03:00
alexey-milovidov
6e5638cee8
Merge pull request #21366 from songenjie/shutdown_timeout_log
[ClickHouse][LOG]correct shutdown timeout log
2021-03-03 01:55:15 +03:00
alexey-milovidov
c78256ad65
Merge pull request #21327 from amosbird/hasequalvalue
IColunm::hasEqualValues()
2021-03-03 01:19:26 +03:00
Pavel Kruglov
82a6be44cf Disable changing replica when we have made progress 2021-03-03 01:03:03 +03:00
Kruglov Pavel
fbe0a14a2f
Merge pull request #21035 from anmolarora/system-parts-requested-columns-fill
Fill only requested columns when querying system.parts & system.parts…
2021-03-03 00:39:45 +03:00
Nikolai Kochetov
8e6fa404c4
Merge pull request #21246 from ucasFL/union-distinct-improve
Improve Normalization of ASTSelectWithUnionQuery
2021-03-02 21:37:07 +03:00
alesapin
c29d7c7f49 Shutup clang tidy 2021-03-02 19:13:36 +03:00
alesapin
ee185bc536 Start from committed plus 1 2021-03-02 18:58:02 +03:00
alesapin
346d2b1d27 Deepbugging 2021-03-02 18:19:05 +03:00
Amos Bird
9411d496d9
Fix build 2021-03-02 23:17:02 +08:00
alesapin
8e6252b25f Use snapshot object in serialization 2021-03-02 17:30:56 +03:00
Amos Bird
134216dc4a Specialize intDiv/module 2021-03-02 16:27:28 +03:00
alesapin
10e16e39ea Written synchronous snapshots 2021-03-02 16:18:04 +03:00
Pavel Kruglov
4166ae1f45 Restart timeout when we make progress 2021-03-02 15:40:24 +03:00
alesapin
48bf55aef9 Even better tests 2021-03-02 15:37:00 +03:00
alesapin
08ccae32f3 Fix nasty bug 2021-03-02 15:34:18 +03:00
Amos Bird
494c2dba6c
Fix error 2021-03-02 19:12:51 +08:00
alesapin
e7399bf66f Merge branch 'master' into persistent_nukeeper_snapshot_storage 2021-03-02 14:03:42 +03:00
alesapin
4e33587043 Comments 2021-03-02 13:57:09 +03:00
Amos Bird
ad4fd75fb4
mycli compatibility 2021-03-02 18:53:06 +08:00
Nikolai Kochetov
ffbea93b10
Merge pull request #19576 from traceon/row-policy-with-prewhere
Allow row policies with PREWHERE
2021-03-02 13:51:37 +03:00
alesapin
06678d650d Merge branch 'master' into fix_alter_partition_key 2021-03-02 13:43:41 +03:00
alesapin
9ebf1b4fad Get rid of separate minmax index fields 2021-03-02 13:33:54 +03:00
Vasily Nemkov
6af4d06dff Added Grant, Revoke and System query_kind for system.query_log
Also some tests to ensure that all queries are logged
2021-03-02 12:27:39 +02:00
tavplubix
ee22eeea6c
Merge pull request #21357 from ClickHouse/fix-21140
Fix inconsistent column type for Values input format.
2021-03-02 13:12:12 +03:00
Nikolai Kochetov
59a2c45555
Update executeQuery.cpp 2021-03-02 13:09:29 +03:00
Amos Bird
3df57da2d8
mysql client compatibility 2021-03-02 17:06:40 +08:00
Pavel Kruglov
153bfbfc28 Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-03-02 11:59:32 +03:00
tavplubix
d305b23338
Merge pull request #21079 from azat/distributed_ddl_pool_size-zk-fix
Fix various issues in DDLWorker (SIGSEGV and others)
2021-03-02 11:59:30 +03:00
Pavel Kruglov
780a65e3a0 Merge branch 'master' of github.com:ClickHouse/ClickHouse into system-parts-requested-columns-fill 2021-03-02 10:29:26 +03:00
turbo jason
3a6307a990 [ClickHouse][LOG]correct shutdown timeout log 2021-03-02 15:13:54 +08:00
Amos Bird
c41dedc922
optimize for ColumnLowCardinality 2021-03-02 11:55:58 +08:00
alexey-milovidov
32c011052f
Update IColumnImpl.h 2021-03-02 11:55:58 +08:00
alexey-milovidov
220d3cae7f
Update IColumn.h 2021-03-02 11:55:58 +08:00