thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-24 00:22:29 +00:00
Code Issues Packages Projects Releases Wiki Activity
67,710 Commits 9,167 Branches 1,781 Tags 2.1 GiB
350f2c0455
Commit Graph

15303 Commits

Author SHA1 Message Date
Kseniia Sumarokova
ed687f45a1
Merge branch 'master' into truncate 2021-06-30 20:08:42 +03:00
alesapin
89a401a84f Merge branch 'better_remove_empty_parts' into fix_drop_part_in_queue 2021-06-30 18:53:17 +03:00
alesapin
6a73c8b49e Review fixes 2021-06-30 18:24:51 +03:00
tavplubix
c958f876b8
Update MergeList.h 2021-06-30 15:55:26 +03:00
alesapin
21f15be990 Merge branch 'better_remove_empty_parts' into fix_drop_part_in_queue 2021-06-30 15:30:32 +03:00
alesapin
2eb27540b2 Some test version 2021-06-30 15:29:09 +03:00
Maksim Kita
b47bb086e7
Merge pull request #25817 from ianton-ru/fix_start_client_in_wrong_dir 
Fix start clickhouse-client with unreadable working directory
 2021-06-30 14:14:33 +03:00
Maksim Kita
53dc831a60
Update configReadClient.cpp 2021-06-30 12:42:44 +03:00
alesapin
94d4d32e4c
Merge pull request #25783 from ClickHouse/followup_drop_part_fix 
Relax check during DROP PART
 2021-06-30 12:25:42 +03:00
mergify[bot]
1799804243
Merge branch 'master' into fix_aggregation_ttl 2021-06-30 09:15:46 +00:00
Anton Ivashkin
8aae56f24a Minor fixes based on code review result 2021-06-30 12:06:11 +03:00
mergify[bot]
8182b6e890
Merge branch 'master' into better_remove_empty_parts 2021-06-30 08:42:26 +00:00
Maksim Kita
1e9e073b0a
Merge pull request #25618 from vdimir/slow-dict-join-fix 
Fix unnecessary dictionary re-reads in KeyGetterForDict
 2021-06-30 10:54:48 +03:00
kssenii
1a5c30e249 Merge branch 'pg-ch-replica' of github.com:kssenii/ClickHouse into pg-ch-replica 2021-06-29 23:20:33 +00:00
kssenii
da70f85d4e More tests 2021-06-29 23:11:46 +00:00
Maksim Kita
5b3fc9b49e Added comment 2021-06-30 01:46:57 +03:00
kssenii
9d5847a1a8 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-06-29 21:30:31 +00:00
Maksim Kita
b2177ab723 ExpressionCache destruction fix 2021-06-29 23:40:15 +03:00
alesapin
0af53ace55 Fix comment 2021-06-29 22:49:41 +03:00
alesapin
ca1d21b663
Update src/Storages/MergeTree/MergeTreeData.cpp 
Co-authored-by: tavplubix <avtokmakov@yandex-team.ru>
 2021-06-29 22:48:46 +03:00
alesapin
a6834213a1 Add tests 2021-06-29 22:47:54 +03:00
Maksim Kita
567f6a2b6f
Merge pull request #25819 from egatov/master 
METR-41529
 2021-06-29 21:35:16 +03:00
alexey-milovidov
48348e6375
Update configReadClient.cpp 2021-06-29 21:12:10 +03:00
Maksim Kita
4bb3fd53f7
Merge pull request #24129 from kitaisreal/date-time-timezone-fix 
Date time timezone fix
 2021-06-29 20:13:37 +03:00
Nikolai Kochetov
d5bd2b1fa9
Merge pull request #25719 from ClickHouse/remove-PrewhereDAGInfo 
Remove PrewhereDAGInfo.
 2021-06-29 18:58:03 +03:00
alesapin
5822f0ba29 Replace lost parts with empty parts instead of hacking replication queue 2021-06-29 18:14:44 +03:00
kssenii
6816e8c5aa Revert change, fix injectiveness 2021-06-29 14:55:29 +00:00
Evgeniy Gatov
739966dc99 METR-41529 2021-06-29 17:51:53 +03:00
Anton Ivashkin
955c031474 Fix start clickhouse-client with unreadable working directory 2021-06-29 17:37:12 +03:00
mergify[bot]
c6cda5ebce
Merge branch 'master' into followup_drop_part_fix 2021-06-29 12:43:54 +00:00
Nikolai Kochetov
bf827936b7 Rename PrewhereActions 2021-06-29 14:53:34 +03:00
alesapin
7680eab0e4 one more fix 2021-06-29 13:03:00 +03:00
alesapin
cdc95fa98a
Merge pull request #25428 from ClickHouse/zookeeper_snapshots 
Tool for conversion of ZooKeeper data into clickhouse-keeper snapshot
 2021-06-29 12:58:45 +03:00
kssenii
e49930fde3 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into inconsisteny 2021-06-29 09:14:12 +00:00
Maksim Kita
41620a5661 Fixed tests 2021-06-29 11:05:07 +03:00
alesapin
941a6e539f Fix 2021-06-29 10:36:20 +03:00
Kostiantyn Storozhuk
a9510f25f9 Style fix 2021-06-29 15:22:08 +08:00
alesapin
2b3d50f0e6
Merge pull request #25771 from azat/version-revision-comment 
Add comments for VERSION_REVISION vs DBMS_TCP_PROTOCOL_VERSION
 2021-06-29 10:07:13 +03:00
Kostiantyn Storozhuk
296593716d Reverted constructor change 2021-06-29 14:37:53 +08:00
alexey-milovidov
9a33dd8f9f
Merge pull request #25796 from Algunenano/storages_typos 
Fix some typos in Storage classes
 2021-06-29 04:25:02 +03:00
alexey-milovidov
5b21ca72ac
Merge pull request #25801 from azat/storage-merge-no-col-fix 
Fix "No available columns" for Merge() storage
 2021-06-29 04:23:52 +03:00
alexey-milovidov
878a159538
Merge pull request #25773 from ClickHouse/system-query-log-map-type-merge 
Merging #23934
 2021-06-29 04:19:53 +03:00
Alexey Milovidov
a7ee0b91b1 Fix strange whitespaces @abyss7 2021-06-29 01:52:22 +03:00
Ildus Kurbangaliev
5d89907ae9 Fix style errors 2021-06-29 00:34:45 +02:00
alexey-milovidov
2243aed4c5
Merge pull request #25787 from ClickHouse/tavplubix-patch-2 
Remove dead code from VirtualColumnUtils.cpp
 2021-06-29 01:25:58 +03:00
Azat Khuzhin
15a14d6da2 Fix "No available columns" for Merge() storage 2021-06-29 00:39:38 +03:00
Azat Khuzhin
e590d9c8bb Add comments for VERSION_REVISION vs DBMS_TCP_PROTOCOL_VERSION 2021-06-28 22:01:14 +03:00
Kseniia Sumarokova
d9c46003cb
Merge pull request #25781 from sand6255/MaterializeMySQL-Engine-column-comments-improvement 
MaterializeMySQL: Improved column comments support
 2021-06-28 21:57:24 +03:00
Raúl Marín
bfc122df64 Fix some typos in Storage classes 2021-06-28 19:03:56 +02:00
Maksim Kita
764380f701 Updated tests 2021-06-28 19:56:15 +03:00
mergify[bot]
da09856a94
Merge branch 'master' into followup_drop_part_fix 2021-06-28 16:30:55 +00:00
Kseniia Sumarokova
9d02af7d8b
Merge pull request #25676 from sand6255/MaterializeMySQL-Support-Enum-Data-Type 
MaterializeMySQL: support ENUM data type
 2021-06-28 19:15:00 +03:00
Kostiantyn Storozhuk
c2c78929cb Implemented MySQL column comments support 2021-06-28 22:44:38 +08:00
Kseniia Sumarokova
ff8c44179c
Update DatabaseAtomic.cpp 2021-06-28 17:37:22 +03:00
Kseniia Sumarokova
a96c7a3a70
Update DatabaseAtomic.cpp 2021-06-28 17:32:03 +03:00
mergify[bot]
04814114e7
Merge branch 'master' into zookeeper_snapshots 2021-06-28 14:17:32 +00:00
alesapin
72b281987e Add more debug 2021-06-28 17:15:51 +03:00
Kseniia Sumarokova
54a7e2158c
Update ActionsDAG.cpp 2021-06-28 17:09:26 +03:00
tavplubix
3afb7aa97e
Update VirtualColumnUtils.cpp 2021-06-28 16:47:47 +03:00
kssenii
48d21bb03b Fix inconsistency 2021-06-28 13:20:23 +00:00
Anton Popov
03507f548e
Merge branch 'master' into storage-merge-aliases 2021-06-28 16:03:10 +03:00
alesapin
d20feeb42c Relax check during DROP PART 2021-06-28 15:05:22 +03:00
Maksim Kita
352e1f27ff Update using Map datatyle in system log tables before merge 2021-06-28 14:42:21 +03:00
alesapin
b11254d191
Merge branch 'master' into fix_aggregation_ttl 2021-06-28 13:31:29 +03:00
alesapin
71603a7d13
Merge pull request #25772 from ClickHouse/fix_mutation_test 
Fix flaky test and wrong message
 2021-06-28 13:30:36 +03:00
alesapin
4c213b639f
Merge pull request #25548 from nikitamikhaylov/background-processing 
A little improvement in BackgroundJobsExecutor
 2021-06-28 13:24:21 +03:00
Zijie Lu
f20a1f708b Merge branch 'master' into support-distinct-on 2021-06-28 16:38:29 +08:00
alesapin
7e73762b48 Fix flaky test and wrong message 2021-06-28 11:28:45 +03:00
Maksim Kita
294b937a39
Merge branch 'system-query-log-map-type-merge' into system-querylog-map-continue 2021-06-28 11:27:59 +03:00
alesapin
1b56b0a020 Fix flaky test 2021-06-28 11:10:38 +03:00
Kostiantyn Storozhuk
ca4783d854 Fixed typo and casting 2021-06-28 15:22:13 +08:00
Kseniia Sumarokova
b6b7924413
Update DatabaseAtomic.cpp 2021-06-28 10:20:33 +03:00
Zijie Lu
b408c7d7e6 Merge branch 'master' into support-distinct-on 
Signed-off-by: Zijie Lu <wslzj40@gmail.com>
 2021-06-28 14:53:55 +08:00
robot-clickhouse
648801760e Auto version update to [21.8.1.1] [54453] 2021-06-28 00:50:20 +03:00
alexey-milovidov
a11351a473
Merge pull request #25484 from vdimir/issue-19589 
Fix assert with non uint8 in prewhere
 2021-06-28 00:03:12 +03:00
alexey-milovidov
16f88fd985
Merge pull request #25693 from novikd/master 
Introduce system.data_skipping_indices table
 2021-06-27 23:24:53 +03:00
kssenii
aa7133c239 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-06-27 20:08:00 +00:00
kssenii
c3a2fc0584 Review fixes 2021-06-27 19:17:17 +00:00
Maksim Kita
61b116332d Fixed tests 2021-06-27 20:53:02 +03:00
Maksim Kita
e6adc405d8 DateTime timezone fix 2021-06-27 20:53:02 +03:00
alexey-milovidov
fb89505656
Merge pull request #25538 from kssenii/pg-dix 
Fix postrges array conversion
 2021-06-27 19:35:24 +03:00
alexey-milovidov
0a26687115
Merge pull request #23864 from azat/dist-split-batch-and-retry 
Add ability to split distributed batch on failures (i.e. due to memory limits)
 2021-06-27 19:28:26 +03:00
kssenii
db998c3f6c More robust 2021-06-27 16:28:10 +00:00
alesapin
c977c33d6d Fix bug in execution of TTL GROUP BY 2021-06-27 19:18:15 +03:00
Alexey Milovidov
b11f9223de Merge branch 'master' into fix-receive-time-metric 2021-06-27 19:00:39 +03:00
alexey-milovidov
1b644b9a31
Merge pull request #25663 from azat/dist-startup 
Improve startup time of Distributed engine.
 2021-06-27 18:22:45 +03:00
alexey-milovidov
f6e67d3dc1
Update StorageDistributed.cpp 2021-06-27 18:22:34 +03:00
alexey-milovidov
6e25d11e1f
Merge pull request #25582 from azat/atomic-unlink-symlink 
Remove only symlinks during force_restore_data of Atomic engine
 2021-06-27 18:19:59 +03:00
alexey-milovidov
6e0c8f0122
Merge pull request #25737 from amosbird/dataracereloadcluster 
Fix data race in getClusters()
 2021-06-27 17:55:28 +03:00
Alexey Milovidov
9ab463efdd utils/generate-ya-make/generate-ya-make.sh 2021-06-27 17:49:26 +03:00
Alexey Milovidov
4dbd659d64 Change error code in LIVE VIEW 2021-06-27 15:40:07 +03:00
Alexey Milovidov
4b994fc3c7 Change error code in LIVE VIEW 2021-06-27 15:38:08 +03:00
alexey-milovidov
40095a9bfe
Update StorageSystemDataSkippingIndices.cpp 2021-06-27 15:03:56 +03:00
feng lv
06242f85e6 fix 2021-06-27 06:09:23 +00:00
alexey-milovidov
ec738c6170
Update StorageSystemDataSkippingIndices.cpp 2021-06-27 03:25:04 +03:00
alexey-milovidov
80d9346080
Update StorageSystemDataSkippingIndices.h 2021-06-27 03:20:28 +03:00
mergify[bot]
8ca52ea2df
Merge branch 'master' into master 2021-06-27 00:17:31 +00:00
kssenii
3d90f0710c Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-06-26 22:04:47 +00:00
Amos Bird
4da1b8154a
Fix data race in getClusters() 2021-06-26 22:15:57 +08:00
Kseniia Sumarokova
28c5a14922
Fix clang-tidy 2021-06-26 15:50:15 +03:00
Maksim Kita
44122118de Fixed ASTRenameQuery for EXCHANGE DICTIONARIES 2021-06-26 14:28:13 +03:00
Maksim Kita
3c9ae7b5ba Added more tests 2021-06-26 14:28:13 +03:00
Maksim Kita
7b03150b90 Fixed tests 2021-06-26 14:28:13 +03:00
Maksim Kita
008adabec2 Support REPLACE DICTIONARY, CREATE OR REPLACE DICTIONARY queries 2021-06-26 14:28:13 +03:00
alexey-milovidov
6662e7ab0f
Merge pull request #25732 from matwey/missed_include_optional_2 
Fix some more missed includes
 2021-06-26 14:04:06 +03:00
Matwey V. Kornilov
a29bd2c4b3 Add missed #include <limits> 
Signed-off-by: Matwey V. Kornilov <matwey.kornilov@gmail.com>
 2021-06-26 11:25:54 +03:00
Matwey V. Kornilov
9306b598cf Merge branch 'missed_include_optional' into missed_include_optional_2 2021-06-26 11:07:59 +03:00
Matwey V. Kornilov
2f73469629 Merge branch 'missed_include_condition_variable' into missed_include_optional_2 2021-06-26 11:07:50 +03:00
alexey-milovidov
420839a4c0
Merge pull request #25697 from sundy-li/mysql-fix 
[mysql] fix mysql select user() return empty
 2021-06-26 03:31:46 +03:00
l1tsolaiki
7892e44467 Fix style yet again 😎 2021-06-25 21:14:08 +03:00
l1tsolaiki
6981eb64ac Fixes 2021-06-25 19:24:22 +03:00
l1tsolaiki
f8b1a6d185 WriteBuffer is canceled 😳 2021-06-25 18:33:31 +03:00
Ivan
14e43cfafa
Add CI check for darwin-aarch64 (#25560) 
* Add support for darwin-aarch64

* Fix PVS warnings

* Add build configuration

* Fix packager args
 2021-06-25 18:21:56 +03:00
feng lv
61550f91f2 fix and update test 2021-06-25 15:01:57 +00:00
Nikolai Kochetov
3544b059ce Merge branch 'master' into remove-PrewhereDAGInfo 2021-06-25 17:52:03 +03:00
Nikolai Kochetov
6bc0a628cd Remove PrewhereDAGInfo. 2021-06-25 17:49:28 +03:00
Kseniia Sumarokova
1eb005850c
Merge branch 'master' into storage-merge-aliases 2021-06-25 17:33:31 +03:00
kssenii
c2ac9b6027 Better 2021-06-25 14:31:56 +00:00
Kostiantyn Storozhuk
ca04b077ef Improved column comments support 2021-06-25 22:30:11 +08:00
Nikita Mikhaylov
20d20a5389
Merge pull request #25666 from georgthegreat/murmurhash 
Enable MurmurHash in ArcadiaBuild
 2021-06-25 17:05:02 +03:00
feng lv
07a47b4bd3 fix 2021-06-25 13:51:17 +00:00
Nikita Mikhaylov
930a2aa0b7
Merge pull request #24095 from vdimir/view-join-settings 
Use global settings for query to VIEW
 2021-06-25 16:44:28 +03:00
Anton Popov
4a250dc14e
Merge pull request #25371 from CurtizJ/add-compatibility-setting 
Add compatibility setting for name of tuple literals
 2021-06-25 15:08:13 +03:00
vdimir
2949cd1e6f
Support ALIASed columns in JOIN ON expression 2021-06-25 14:50:02 +03:00
Kostiantyn Storozhuk
3c71e067fa Update docs and style 2021-06-25 19:27:03 +08:00
Kostiantyn Storozhuk
7516d39091 Style fixes 2021-06-25 16:44:28 +08:00
Storozhuk Kostiantyn
4a3145f586 Materialize my sql support enum data type 
* Implemented Enum for MaterializeMySQL
 2021-06-25 16:42:25 +08:00
alesapin
f6afc381be
Merge pull request #25665 from ClickHouse/fix_24869 
Fix REPLACE_RANGE entry removal
 2021-06-25 11:11:25 +03:00
feng lv
228e06628a fix conflict 2021-06-25 08:06:06 +00:00
alesapin
d138ba79c7
Merge pull request #25549 from ClickHouse/add_test_for_concurrent_merges 
Fix bug in TTL merges assignment
 2021-06-25 11:02:29 +03:00
feng lv
b54287c8da fix 2021-06-25 08:00:30 +00:00
kssenii
ded9007ca0 Fix 2021-06-25 07:49:38 +00:00
kssenii
ca1f336c46 Aliases for storageMerge fix 2021-06-25 05:28:44 +00:00
sundy-li
d423d07a87 [mysql] fix mysql select user() return empty 2021-06-25 10:20:22 +08:00
Dmitry Novik
960d0de73d Introduce system.data_skipping_indices table 2021-06-25 02:05:45 +03:00
tavplubix
a14724ca7b
Merge pull request #25667 from ClickHouse/fix_rename_distributed2 
Fix rename Distributed table
 2021-06-25 00:05:06 +03:00
alexey-milovidov
89007a8b6a
Merge pull request #25654 from azat/altstack-fix-osx 
Fix alternative stack (MINSIGSTKSZ) and stack size check under osx
 2021-06-24 19:22:14 +03:00
vdimir
241b64d02c
Support ALIASed columns for right joined table 2021-06-24 17:57:21 +03:00
vdimir
af7776554b
Fix space in ExpressionAnalyzer.cpp 2021-06-24 17:16:57 +03:00
vdimir
84e02911cf
Materialized columns for joined table, don't add to asterisk select 2021-06-24 17:14:36 +03:00
Alexander Tokmakov
76156af5cc cancel merges on drop partition 2021-06-24 17:07:43 +03:00
alesapin
230286776d Merge branch 'master' into add_test_for_concurrent_merges 2021-06-24 16:37:25 +03:00
Ivan
e9e49a9085
Fix ANTLR parser and enable it back in CI (#25638) 
* Fix

* Skip more tests
 2021-06-24 14:45:29 +03:00
Alexander Tokmakov
3a25b05765 fix rename Distributed table 2021-06-24 13:00:33 +03:00
Yuriy Chernyshov
9dcd37b94d Enable MurmurHash in ArcadiaBuild 2021-06-24 12:04:40 +03:00
Alexander Tokmakov
7e6963bb39 fix REPLACE_RANGE entry removal 2021-06-24 11:16:54 +03:00
Azat Khuzhin
8262e251c4 Fix getStackSize() stack address on osx 
Should fix the following exception under osx:

  Stack size too large. Stack address: 0x700001a58000, frame address: 0x700001a53f40, stack size: 540864, maximum stack size: 524288 (version 21.7.1.1)
 2021-06-24 10:48:43 +03:00
tavplubix
b1263c18ee
Fix pcg deserialization (#24538) 
* fix pcg deserialization

* Update 01156_pcg_deserialization.sh

* Update 01156_pcg_deserialization.sh

* Update 01156_pcg_deserialization.sh

* fix another bug

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: alexey-milovidov <milovidov@yandex-team.ru>
 2021-06-24 10:40:00 +03:00
Azat Khuzhin
a616ae8861 Improve startup time of Distributed engine. 
- create directory monitors in parallel (this also includes rmdir in
  case of directory is empty, since even if the directory is empty it
  may take some time to remove it, due to waiting for journal or if the
  directory is large, i.e. it had lots of files before, since remember
  ext4 does not truncate the directory size on each unlink [1])
- initialize increment in parallel too (since it does readdir())

  [1]: https://lore.kernel.org/linux-ext4/930A5754-5CE6-4567-8CF0-62447C97825C@dilger.ca/
 2021-06-24 10:27:51 +03:00
Azat Khuzhin
3c76986311 Fix alternative stack under osx (MINSIGSTKSZ is 32K) 
Fixes: #25632
 2021-06-24 00:33:33 +03:00
alexey-milovidov
bbb216f4c8
Merge pull request #25604 from ClickHouse/roaring-bitmap-small-change 
Small change in Roaring Bitmaps
 2021-06-24 00:29:38 +03:00
alesapin
e6da53dc46 better 2021-06-24 00:09:06 +03:00
alesapin
24876291d1 Followup fix 2021-06-24 00:01:07 +03:00
alesapin
4be4bc21e2 Fix for fix 2021-06-23 23:57:49 +03:00
alexey-milovidov
99f60a7c1d
Merge pull request #25513 from azat/use_compact_format_in_distributed_parts_names-2 
Drop replicas from dirname for internal_replication=true
 2021-06-23 22:51:07 +03:00
alexey-milovidov
6adf0955c2
Merge pull request #25400 from georgthegreat/murmurhash 
Rename & reimport murmurhash sources from smhasher repo
 2021-06-23 22:48:29 +03:00
alesapin
d744136c0d Merge branch 'master' into add_test_for_concurrent_merges 2021-06-23 22:26:15 +03:00
alesapin
81c74435a3 Fix drop part bug 2021-06-23 22:25:30 +03:00
Nikolai Kochetov
8bfa26102b
Merge pull request #25631 from ClickHouse/fix-explain-ast-no-query-crash 
Fix crash in EXPLAIN AST without query.
 2021-06-23 19:56:59 +03:00
Kruglov Pavel
4913b18532
Merge pull request #25531 from Avogar/fix-arrow-decimal 
Fix segfault in Arrow format, add arrow Decimal256 support in Arrow and Parquet.
 2021-06-23 17:28:16 +03:00
vdimir
b00efaf3d1
Add materialized columns to joined columns 2021-06-23 17:03:39 +03:00
Ildus Kurbangaliev
541c09d7b3 Fix tests errors 2021-06-23 15:03:58 +02:00
Nikolai Kochetov
c92c352d9a Fix crash in EXPLAIN AST without query. 2021-06-23 15:29:10 +03:00
l1tsolaiki
abe7e4195e . 2021-06-23 15:02:47 +03:00
vdimir
4ce829d7c1
Create KeyGetter outside of joinRightColumns 2021-06-23 14:27:39 +03:00
vdimir
8f9166df4e
Add read flag to KeyGetterForDict 2021-06-23 14:15:31 +03:00
Anton Popov
a15ee72d0f Merge remote-tracking branch 'upstream/master' into HEAD 2021-06-23 13:53:01 +03:00
Anton Popov
93f15e443d
Merge pull request #25539 from CurtizJ/with-fill-totals 
Fix usage of `WITH FILL` together with `WITH TOTALS`
 2021-06-23 13:42:27 +03:00
l1tsolaiki
ec7ec63a40 Fix style 2021-06-23 13:22:38 +03:00
l1tsolaiki
a5d3600f20 Fix compile errors with WriteBuffer 2021-06-23 12:41:38 +03:00
Kseniia Sumarokova
3e238c1767
Merge pull request #25601 from ianton-ru/fix_s3_restore 
Fix DiskS3 restore
 2021-06-23 12:34:50 +03:00
mergify[bot]
fddf47a900
Merge branch 'master' into zookeeper_snapshots 2021-06-23 08:36:14 +00:00
Azat Khuzhin
50d9d3efd5 Remove only symlinks during force_restore_data of Atomic engine 2021-06-23 10:17:31 +03:00
Azat Khuzhin
3bd53c68f9 Try to split the batch in case of broken batch too 
Broken batches may be because of abnormal server shutdown (and lack of
fsync), and ignoring the whole batch is not great in this case, so apply
the same split logic here too.

v2: rename exception
v3: catch missing exception
v4: fix marking the file as broken multiple times (fixes
test_insert_distributed_async_send with setting enabled)
 2021-06-23 02:48:47 +03:00
Azat Khuzhin
a0209178cc Add ability to split distributed batch on failures 
Add distributed_directory_monitor_split_batch_on_failure setting (OFF by
default), that will split the batch and send files one by one in case of
retriable errors.

v2: more error codes
 2021-06-23 02:48:47 +03:00
Azat Khuzhin
e148ef739d Drop replicas from dirname for internal_replication=true 
Under use_compact_format_in_distributed_parts_names=1 and
internal_replication=true the server encodes all replicas for the
directory name for async INSERT into Distributed, and the directory name
looks like:

    shard1_replica1,shard1_replica2,shard3_replica3

This is required for creating connections (to specific replicas only),
but in case of internal_replication=true, this can be avoided, since
this path will always includes all replicas.

This patch replaces all replicas with "_all_replicas" marker.

Note, that initial problem was that this path may overflow the NAME_MAX
if you will have more then 15 replicas, and the server will fail to
create the directory.

Also note, that changed directory name should not be a problem, since:
- empty directories will be removed since #16729
- and replicas encoded in the directory name is also supported anyway.
 2021-06-23 02:47:38 +03:00
Nikita Mikhaylov
000c937ebe fix pvs check 2021-06-22 23:24:47 +00:00
Nikita Mikhaylov
be4aff0c19 delete test 2021-06-22 23:24:47 +00:00
Nikita Mikhaylov
f4d6a6e128 support dynamic reloading of server config 2021-06-22 23:24:47 +00:00
Nikita Mikhaylov
a39bca0acf better 2021-06-22 23:24:47 +00:00
Nikita Mikhaylov
c66a3b22b5 done 2021-06-22 23:24:47 +00:00
Alexey Milovidov
afeb562744 Small change in Roaring Bitmaps 2021-06-23 00:29:20 +03:00
alesapin
a92bd49229 Supress PVS 2021-06-22 23:23:26 +03:00
Alexey Milovidov
99e08b7406 Make network receive time metric to correctly include the time spent waiting for data from the client to INSERT #9958 2021-06-22 23:22:13 +03:00
Anton Ivashkin
e0199d8006 Fix DiskS3 restore 2021-06-22 20:05:20 +03:00
Anton Popov
18fad6d322 Merge remote-tracking branch 'upstream/master' into HEAD 2021-06-22 19:20:59 +03:00
mergify[bot]
c15493f0df
Merge branch 'master' into add_test_for_concurrent_merges 2021-06-22 15:49:35 +00:00
alexey-milovidov
d6b93f3af8
Merge pull request #25584 from kelvich/patch-1 
Fix typo in hardware failure error message
 2021-06-22 18:38:00 +03:00
Ildus Kurbangaliev
0a937e7377 Support Map type in mapAdd and mapSubtract 2021-06-22 17:02:42 +02:00
vdimir
80a001ab0c
Use settings from global context in StorageView 2021-06-22 17:39:09 +03:00
vdimir
85bcf8a1dc
Merge branch 'master' into view-join-settings 2021-06-22 17:36:44 +03:00
mergify[bot]
f3512f15d5
Merge branch 'master' into zookeeper_snapshots 2021-06-22 14:29:36 +00:00
tavplubix
a6a43ce3ee
Revert "Implement h3ToGeo function" 2021-06-22 17:25:21 +03:00
alesapin
47b29092b4 Fix bug in MergerMutator parts selector 2021-06-22 16:47:42 +03:00
Anton Popov
ac25b55e87
Merge pull request #25546 from CurtizJ/fix-index-array-join 
Fix usage of index with array columns and ARRAY JOIN
 2021-06-22 16:23:00 +03:00
Nikita Mikhaylov
2ec3bc2d96
Merge pull request #25520 from azat/not-parser-fix 
Fix NOT parsing
 2021-06-22 16:19:39 +03:00
Anton Popov
447fef702d fix fuzzer in query with 'WITH FILL' and 'WITH TOTALS' 2021-06-22 16:08:12 +03:00
Nikita Mikhaylov
19b3659517
Merge pull request #24867 from bharatnc/ncb/h3-indexing-funcs 
Implement h3ToGeo function
 2021-06-22 15:59:09 +03:00
Zijie Lu
63db58710d Support for DISTINCT ON (columns) 
Signed-off-by: Zijie Lu <wslzj40@gmail.com>
 2021-06-22 19:25:14 +08:00
alesapin
6b815b4f17 Merge branch 'master' into zookeeper_snapshots 2021-06-22 13:50:27 +03:00
alesapin
bf0a4864ac Add support for set/get ACL commands 2021-06-22 13:49:35 +03:00
Stas Kelvich
9a84033db5
Fix typo in hardware failure error message 2021-06-22 12:48:15 +03:00
alesapin
a8ce3bed2b
Merge pull request #25421 from ClickHouse/update_buffer_size_in_nuraft 
Update NuRaft buffer to 64 bit size
 2021-06-22 10:55:31 +03:00
Kseniia Sumarokova
155bb03239
Merge branch 'master' into pg-dix 2021-06-22 10:14:52 +03:00
Kseniia Sumarokova
f3b0f11b59
Update StorageHDFS.cpp 2021-06-21 23:04:58 +03:00
alesapin
71e9689ba6 Fix PVS warning 2021-06-21 22:59:19 +03:00
Kseniia Sumarokova
41c65a9965
Merge pull request #25528 from kssenii/something 
Fix convertion of datetime with timezone for mysql, odbc, etc..
 2021-06-21 22:55:22 +03:00
alesapin
731edc9a6d Fixes in ACLs 2021-06-21 18:45:45 +03:00
kssenii
ac0f86cdbf Truncate for s3 2021-06-21 15:44:36 +00:00
Anton Popov
880ed24565 move comment 2021-06-21 18:31:18 +03:00
Anton Popov
662d5b8495 fix ALTER MODIFY COLUMN of columns, that participates in TTL expressions 2021-06-21 18:20:18 +03:00
Kseniia Sumarokova
77a3a1416d
Update fetchPostgreSQLTableStructure.cpp 2021-06-21 17:26:14 +03:00
kssenii
4688f9e038 hdfs truncate table 2021-06-21 13:59:57 +00:00
alesapin
76cee4e3cf Debugging 2021-06-21 16:58:39 +03:00
Amos Bird
0adad2425a
json extract string or raw 2021-06-21 21:36:42 +08:00
vdimir
46a5dd6701
Fix MergeTreeBaseSelectProcessor::executePrewhereActions 2021-06-21 16:01:02 +03:00
Anton Popov
ffa56bde24 fix usage of index with array columns and ARRAY JOIN 2021-06-21 15:34:05 +03:00
vdimir
eb85c5a8e7
Fix assert with non uint8 in prewhere 2021-06-21 15:32:29 +03:00
kssenii
7d3dee172c FIx progress 2021-06-21 12:19:25 +00:00
Anton Popov
82f0a5f2dd fix usage of WITH FILL together with WITH TOTALS 2021-06-21 14:43:38 +03:00
kssenii
0f9fc33a4e Fix postgres arrays 2021-06-21 11:27:10 +00:00
alesapin
743600a359 Merge branch 'update_buffer_size_in_nuraft' into zookeeper_snapshots 2021-06-21 13:46:22 +03:00
kssenii
fb9c92cb6c Fix 2021-06-21 10:32:40 +00:00
alesapin
2f24ad6165 Merge branch 'fix_some_tests' into update_buffer_size_in_nuraft 2021-06-21 13:20:27 +03:00
Anton Popov
c2754c34f4 Merge remote-tracking branch 'upstream/master' into HEAD 2021-06-21 12:42:48 +03:00
alexey-milovidov
8ba6a5393f
Merge pull request #25509 from azat/fix-altstack 
Fix alternative stack for SIGSEGV handling
 2021-06-21 12:10:00 +03:00
alexey-milovidov
d2c86e52e6
Update checkStackSize.cpp 2021-06-21 12:09:14 +03:00
alexey-milovidov
f37faf02fa
Update checkStackSize.cpp 2021-06-21 12:08:30 +03:00
alexey-milovidov
9a4125dd7c
Update ThreadStatus.cpp 2021-06-21 12:06:41 +03:00
Pavel Kruglov
556ec9e533 Fix segfault in arrow and support Decimal256 2021-06-21 12:06:28 +03:00
kssenii
c0732ddc12 Fix datetime with timezone 2021-06-21 08:22:12 +00:00
Kruglov Pavel
a54cbef5b7
Merge pull request #24341 from Avogar/arrow 
Support complex types in Arrow/Parquet/ORC
 2021-06-21 11:05:57 +03:00
Azat Khuzhin
fabff86788 Fix NOT parsing 2021-06-20 23:27:24 +03:00
Anton Popov
e409ead64a fix tests 2021-06-20 23:18:05 +03:00
Anton Popov
d8b6f15ef4
Merge pull request #23027 from azat/distributed-push-down-limit 
Add ability to push down LIMIT for distributed queries
 2021-06-20 23:08:50 +03:00
Azat Khuzhin
12eadc2b5e Add a guard page for alternative signal handling stack 2021-06-20 20:08:50 +03:00
Azat Khuzhin
4dcbf3a5c5 Replace __pthread_get_minstack() with a const 2021-06-20 14:03:04 +03:00
Mike Kot
4c391f8e99
SYSTEM RESTORE REPLICA replica [ON CLUSTER cluster] (#13652) 
* initial commit: add setting and stub

* typo

* added test stub

* fix

* wip merging new integration test and code proto

* adding steps interpreters

* adding firstly proposed solution (moving parts etc)

* added checking zookeeper path existence

* fixing the include

* fixing and sorting includes

* fixing outdated struct

* fix the name

* added ast ptr as level of indirection

* fix ref

* updating the changes

* working on test stub

* fix iterator -> reference

* revert rocksdb submodule update

* fixed show privileges test

* updated the test stub

* replaced rand() with thread_local_rng(), updated the tests

updated the test

fixed test config path

test fix

removed error messages

fixed the test

updated the test

fixed string literal

fixed literal

typo: =

* fixed the empty replica error message

* updated the test and the code with logs

* updated the possible test cases, updated

* added the code/test milestone comments

* updated the test (added more testcases)

* replaced native assert with CH one

* individual replicas recursive delete fix

* updated the AS db.name AST

* two small logging fixes

* manually generated AST fixes

* Updated the test, added the possible algo change

* Some thoughts about optimizing the solution:

ALTER MOVE PARTITION .. TO TABLE -> move to detached/ + ALTER ... ATTACH

* fix

* Removed the replica sync in test as it's invalid

* Some test tweaks

* tmp

* Rewrote the algo by using the executeQuery instead of

hand-crafting the ASTPtr.

Two questions still active.

* tr: logging active parts

* Extracted the parts moving algo into a separate helper function

* Fixed the test data and the queries slightly

* Replaced query to system.parts to direct invocation,

started building the test that breaks on various parts.

* Added the case for tables when at least one replica is alive

* Updated the test to test replicas restoration by detaching/attaching

* Altered the test to check restoration without replica restart

* Added the tables swap in the start if the server failed last time

* Hotfix when only /replicas/replica... path was deleted

* Restore ZK paths while creating a replicated MergeTree table

* Updated the docs, fixed the algo for individual replicas restoration case

* Initial parts table storage fix, tests sync fix

* Reverted individual replica restoration to general algo

* Slightly optimised getDataParts

* Trying another solution with parts detaching

* Rewrote algo without any steps, added ON CLUSTER support

* Attaching parts from other replica on restoration

* Getting part checksums from ZK

* Removed ON CLUSTER, finished working solution

* Multiple small changes after review

* Fixing parallel test

* Supporting rewritten form on cluster

* Test fix

* Moar logging

* Using source replica as checksum provider

* improve test, remove some code from parser

* Trying solution with move to detached + forget

* Moving all parts (not only Committed) to detached

* Edited docs for RESTORE REPLICA

* Re-merging

* minor fixes

Co-authored-by: Alexander Tokmakov <avtokmakov@yandex-team.ru>
 2021-06-20 11:24:43 +03:00
Azat Khuzhin
9a3a1397e6 Fix alternative stack for SIGSEGV handling 
Previous stack size of 4096 wasn't enough, and this lead to that on
SIGSEGV signal handler may overwrite some TLS data (like in the
following example [1]) and in real binary "current_thread" was
overwritten, and this leads to lack of query-id on SIGSEGV, like here
[2].

  [1]: https://gist.github.com/azat/2ee360fd4f2828d363b0926431afacc6
  [2]: https://clickhouse-test-reports.s3.yandex.net/24411/79563953201ce6249a8fd49e22be3902ca4ee43a/fuzzer_ubsan/report.html#fail1

Fix this by using __pthread_get_minstack() that takes TLS block into
account. Yes it is private, but let's try (like rust tried).

This patch will also decrease TLS, since now the per-thread stack will
be allocated in the heap, not on the stack of each thread.

Refs: https://sourceware.org/legacy-ml/libc-alpha/2012-06/msg00335.html
Refs: https://maskray.me/blog/2021-02-14-all-about-thread-local-storage
Refs: https://sourceware.org/bugzilla/show_bug.cgi?id=11787
 2021-06-19 22:47:01 +03:00
Azat Khuzhin
57960746e2 Add getStackSize() helper 2021-06-19 20:09:01 +03:00
Azat Khuzhin
c6339b4c08 Stable NOT chain formatting 2021-06-19 08:03:36 +03:00
Vladimir
1c91a8d2c2
Merge pull request #25454 from vdimir/issue-23351-v2 2021-06-18 21:56:46 +03:00
alesapin
b5dae909dd Add some tests 2021-06-18 21:36:19 +03:00
Kseniia Sumarokova
4fe722d1a4
Merge pull request #25431 from vitlibar/no-underscore_before-preprocessed_configs 
Fix underscore before preprocessed configs.
 2021-06-18 17:53:03 +03:00
Kseniia Sumarokova
5974f96f0c
Merge pull request #23656 from kssenii/progress-bar 
Progress indication for File table engine in clickhouse-local and insert from file in clickhouse-client
 2021-06-18 17:46:54 +03:00
mergify[bot]
c723dd7d40
Merge branch 'master' into arrow 2021-06-18 12:18:57 +00:00
vdimir
0ebca58e86
Check column in removeColumnNullability 2021-06-18 14:06:55 +03:00
Vladimir
4de0d1a593
Merge pull request #24885 from Jokser/disk-s3-read-error-fix 2021-06-18 13:38:59 +03:00
alesapin
2d8f45a098 Add some initialization 2021-06-18 11:55:59 +03:00
Vladimir
6af272891a
Merge pull request #24870 from azat/mv-dist-join-fix 2021-06-18 11:18:43 +03:00
vdimir
b25f6ba5b6
Remove assumeMutable from removeColumnNullability 2021-06-18 11:13:53 +03:00
Vladimir
fab7c9c7f6
Merge pull request #24061 from vdimir/issue-23351 2021-06-18 11:07:28 +03:00
Vladimir
51e5a019ed
Merge pull request #25410 from vdimir/issue-21730 2021-06-18 11:00:26 +03:00
kssenii
f7029b168b Generate ya make 2021-06-18 10:56:24 +03:00
kssenii
bdc8fe1ca0 Merge branch 'master' of github.com:ClickHouse/ClickHouse into progress-bar 2021-06-18 09:33:29 +03:00
Nikolai Kochetov
b2620e1def
Merge pull request #24405 from amosbird/replacemorealias 
Rewrite more alias columns
 2021-06-18 09:10:31 +03:00
Vitaly Baranov
fc3d77bd66 Fix underscore before preprocessed configs. 2021-06-18 06:14:11 +03:00
l1tsolaiki
1863a9beb0 Change stringstream to WriteBuffer 2021-06-18 02:15:15 +03:00
l1tsolaiki
e2765991b0 Separate star from ranges for better code 2021-06-18 01:53:10 +03:00
Alexey Milovidov
3e71181451 Whitespaces 2021-06-18 01:42:33 +03:00
Maksim Kita
b34b66c55d
Merge pull request #25394 from ClickHouse/function-bit-position-to-array-merge 
Merging #23843
 2021-06-18 01:31:33 +03:00
Maksim Kita
2c4c2680f7
Merge pull request #25430 from kitaisreal/database-mysql-rename 
DatabaseMySQL rename
 2021-06-18 01:30:52 +03:00
Kseniia Sumarokova
47cb288834
Update src/Common/ProgressIndication.h 
Co-authored-by: Alexander Kuzmenkov <36882414+akuzm@users.noreply.github.com>
 2021-06-18 00:24:28 +03:00
Nikolai Kochetov
669b8a8b96
Merge pull request #25370 from ClickHouse/disable-query_plan_filter_push_down-because-of-25368 
Fix lost WHERE condition in expression-push-down optimization of query plan
 2021-06-17 23:07:32 +03:00
Nikolai Kochetov
5227a76e3a Add comment. 2021-06-17 23:05:39 +03:00
Maksim Kita
ca672b0a0d Function bitPositionsToArray added support for big integers 2021-06-17 22:32:39 +03:00
Pavel Kovalenko
ce8e2f7f28 Avoid double initializing ReadBufferFromS3 2021-06-17 22:31:50 +03:00
Maksim Kita
cf77e9f2c6 DatabaseMySQL rename 2021-06-17 22:26:34 +03:00
Vladimir
9e5ef95677
Merge pull request #25288 from amosbird/fixjoingetornull 2021-06-17 21:54:21 +03:00
alesapin
1747c254dc Remove unused flag 2021-06-17 21:36:50 +03:00
alexey-milovidov
35411d1085
Merge pull request #24427 from azat/formatting-fixes 
Fix formatting of negative values
 2021-06-17 21:21:47 +03:00
alexey-milovidov
496aff2118
Merge pull request #25300 from ClickHouse/simpla-agg-func-for-summing-mt 
Support simple aggregate functions for SummingMergeTree.
 2021-06-17 20:18:51 +03:00
alesapin
dbe4ba8c2c Merge branch 'update_buffer_size_in_nuraft' into zookeeper_snapshots 2021-06-17 19:51:11 +03:00
alexey-milovidov
c48402d997
Merge pull request #23434 from amosbird/preferglobal 
Add prefer_global_in_and_join setting
 2021-06-17 19:48:28 +03:00
alesapin
dc893f9644 Update NuRaft buffer to 64 bit size 2021-06-17 19:37:08 +03:00
alesapin
1a6abb4db4 Better 2021-06-17 19:32:50 +03:00
l1tsolaiki
0c2d0b55a4 Merge branch 'jsonpath' of github.com:l1tsolaiki/ClickHouse into jsonpath 2021-06-17 18:26:14 +03:00
l1tsolaiki
f2d451e0ec
Merge branch 'ClickHouse:master' into jsonpath 2021-06-17 18:25:26 +03:00
Amos Bird
0f7728cd5e
Fix RBAC test again 2021-06-17 23:00:03 +08:00
Kseniia Sumarokova
f9c0bfb769
Merge pull request #25199 from sand6255/MySQL-column-comments-support 
MaterializeMySQL Engine column comments support
 2021-06-17 17:08:53 +03:00
alesapin
d513d14b2c Add some functions for data conversion 2021-06-17 16:29:11 +03:00
vdimir
23fb229dec
Merge branch 'update_01049_join_low_card_bug' into issue-23351 2021-06-17 15:38:40 +03:00
vdimir
e34a6525ef
Support Interval for LowCardinality 2021-06-17 15:28:08 +03:00
l1tsolaiki
f44d6792b0 Fix ext::range 2021-06-17 14:48:05 +03:00
Matwey V. Kornilov
aa1ac2a506 Add missed #include <optional> 
This is required for std::optional<std::string> language;

Signed-off-by: Matwey V. Kornilov <matwey.kornilov@gmail.com>
 2021-06-17 14:41:28 +03:00
Maksim Kita
6d44aa3034 Fixed style check 2021-06-17 13:26:40 +03:00
vdimir
0cee5ad8e8
Use IColumn::mutate in JoinCommon::convertColumnToNullable 2021-06-17 13:24:17 +03:00
l1tsolaiki
8bac279cb8 Merge branch 'jsonpath' of github.com:l1tsolaiki/ClickHouse into jsonpath 2021-06-17 13:21:13 +03:00
l1tsolaiki
ff3857fbe7 Add root parsing 2021-06-17 13:18:43 +03:00
vdimir
ebdce8e1a6
Fix JoinCommon::convertColumnToNullable for LC(Nullable) 2021-06-17 13:17:28 +03:00
Matwey V. Kornilov
37180e0842 Add missed #include <optional> 
std::optional<> is used multiple times here.

Signed-off-by: Matwey V. Kornilov <matwey.kornilov@gmail.com>
 2021-06-17 13:14:56 +03:00
Nikolai Kochetov
4a496f1374 Fix lost considion in predicate push down. 2021-06-17 12:47:12 +03:00
Maksim Kita
bc86b2b738 Updated function bitPositionsToArray before merge 2021-06-17 12:20:53 +03:00
Vladimir
7a56ab8b1d
Merge pull request #25299 from mga-chka/equals_op_in_sequence_functions 2021-06-17 11:45:04 +03:00
mergify[bot]
13fdefebf5
Merge branch 'master' into fixjoingetornull 2021-06-17 08:26:13 +00:00
Yuriy Chernyshov
2bb1c75fb5 Rename & reimport murmurhash sources from smhasher repo 2021-06-17 10:58:42 +03:00
Maksim Kita
5b2380d541
Merge branch 'function-bit-position-to-array-merge' into feature-function-bitpositionToArray 2021-06-17 10:23:41 +03:00
Maksim Kita
37770f20fc
Merge pull request #25372 from ClickHouse/add-date-name-function-merge 
Merging #23085
 2021-06-17 10:16:32 +03:00
Azat Khuzhin
9242654231 negate: replace special case with parens 
v0: negate: add parens for negative values
    Before this patch -(-1) was formatted to --1, but this is not valid
    syntax.
v2: replace special case with parens, otherwise formatting of -(-(-1))
    will be unstable, in v0 it was formatted to -(1) but -(1) is
    formatted to -1 [1].

  [1]: https://clickhouse-test-reports.s3.yandex.net/24427/4512460aeb02fec557c558eee3f8291a6297efb2/fuzzer_asan/fuzzer.log
 2021-06-17 10:07:03 +03:00
Amos Bird
31815f4a68
Proper documentation of the setting 2021-06-17 14:27:33 +08:00