thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-03 04:52:10 +00:00
Code Issues Packages Projects Releases Wiki Activity
64,546 Commits 9,181 Branches 1,782 Tags 2.2 GiB
67044d69f3
Commit Graph

476 Commits

Author SHA1 Message Date
Alexey Milovidov
0a62f95bbe Merge branch 'master' into distributed_aggregation_memory_efficient_by_default 2021-02-21 23:06:49 +03:00
Kruglov Pavel
815b4ff0d9
Merge branch 'master' into hedged-requests 2021-02-21 17:10:24 +03:00
alexey-milovidov
20a7ccf5b1
Merge pull request #20174 from amosbird/alias 
Normalized function names (lower-case case-insensitive names)
 2021-02-21 16:23:51 +03:00
tavplubix
a697b578bc
Merge pull request #16193 from ClickHouse/database_replicated 
DatabaseReplicated: continuation of #10485
 2021-02-20 19:39:34 +03:00
vdimir
e052a5a05e
Revert "Support old cross to inner join rewrite behaviour" 
This reverts commit 527210b5e4.
 2021-02-19 15:14:24 +03:00
Amos Bird
2c4bc43014
Backward compatible 2021-02-18 20:20:29 +08:00
Pavel Kruglov
1b78de2142 Use fibers in HedgedRequests 2021-02-17 20:34:52 +03:00
vdimir
527210b5e4
Support old cross to inner join rewrite behaviour 2021-02-17 12:44:53 +03:00
Alexey Milovidov
fa20016091 Enable distributed_aggregation_memory_efficient by default 2021-02-17 09:53:18 +03:00
M0r64n
75117389ec Add a couple of QOL file engine settings 2021-02-16 18:50:11 +04:00
Alexander Tokmakov
bf6f64a3fb Merge branch 'master' into database_replicated 2021-02-16 01:28:19 +03:00
vdimir
9afa6b5b1b
Add option cross_to_inner_join_rewrite 2021-02-15 22:40:32 +03:00
Kruglov Pavel
598576ce70
Merge branch 'master' into hedged-requests 2021-02-15 16:51:45 +03:00
Alexander Tokmakov
9c7cf9e92e remove some debug code 2021-02-15 13:26:34 +03:00
Alexey Milovidov
4d650a2a56 Adjust config 2021-02-11 16:41:21 +03:00
Amos Bird
fc6c57f8e0
Fix 2021-02-10 12:48:41 +08:00
Amos Bird
565a148da9
Normalize count() variants 2021-02-10 12:48:41 +08:00
Anton Popov
f1beeb7c25
Merge pull request #20099 from azat/remove-not-existing-setting 
Remove never existing insert_in_memory_parts_timeout
 2021-02-08 16:09:03 +03:00
Pavel Kruglov
f946aab759 Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-02-06 17:38:56 +03:00
Kruglov Pavel
d94e8624d7
Merge branch 'master' into shard-id 2021-02-06 16:48:17 +03:00
Azat Khuzhin
48b415d3e4 Remove never existing insert_in_memory_parts_timeout 
Fixes: a43cb93be5 ("remove questionable functionality")
Cc: @CurtizJ
 2021-02-05 01:26:40 +03:00
Aleksei Semiglazov
921518db0a CLICKHOUSE-606: query deduplication based on parts' UUID 
* add the query data deduplication excluding duplicated parts in MergeTree family engines.

query deduplication is based on parts' UUID which should be enabled first with merge_tree setting
assign_part_uuids=1

allow_experimental_query_deduplication setting is to enable part deduplication, default ot false.

data part UUID is a mechanism of giving a data part a unique identifier.
Having UUID and deduplication mechanism provides a potential of moving parts
between shards preserving data consistency on a read path:
duplicated UUIDs will cause root executor to retry query against on of the replica explicitly
asking to exclude encountered duplicated fingerprints during a distributed query execution.

NOTE: this implementation don't provide any knobs to lock part and hence its UUID. Any mutations/merge will
update part's UUID.

* add _part_uuid virtual column, allowing to use UUIDs in predicates.

Signed-off-by: Aleksei Semiglazov <asemiglazov@cloudflare.com>

address comments
 2021-02-02 16:53:39 +00:00
Pavel Kruglov
60a92e9a99 Fix build, add comments, update tests 2021-02-02 15:14:31 +03:00
feng lv
0edf65c094 fix test 
fix

update

fix spell
 2021-02-02 09:22:30 +00:00
feng lv
4279c7da41 add setting insert_shard_id 
add test

fix style

fix
 2021-02-02 04:26:59 +00:00
Vitaliy Zakaznikov
3de4690450 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into live_view_periodic_refresh 2021-01-29 19:57:13 -05:00
Pavel Kruglov
25e85d71ee Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-01-29 21:08:47 +03:00
Alexey Milovidov
065d22cf2b Merge branch 'master' into disable-checksums-on-read 2021-01-28 03:07:31 +03:00
alexey-milovidov
2028228f4b
Merge pull request #18684 from amosbird/betterwith 
better CTE and with global
 2021-01-27 04:33:17 +03:00
Alexey Milovidov
a6f5f40a65 Merge branch 'master' into disable-checksums-on-read 2021-01-27 04:09:00 +03:00
Amos Bird
3afcccaad2
Fix global with after union refactoring 2021-01-26 10:29:12 +08:00
Alexey Milovidov
9ee5c1535e Allow to disable checksums on read 2021-01-25 23:29:04 +03:00
Alexey Milovidov
cfd47b2980 Merge branch 'master' into issue18494 2021-01-25 22:18:23 +03:00
flynn
e75b116466
Rewrite sum(if()) and sumIf to countIf in special cases (#17041) 
Co-authored-by: vdimir <vdimir@yandex-team.ru>
 2021-01-21 12:01:35 +03:00
Pavel Kruglov
97b5179e55 Implement HedgedRequests 2021-01-19 22:41:05 +03:00
Alexander Kuzmenkov
d22c04568d
Merge pull request #13405 from excitoon-favorites/s3keepalive 
Connection pools for S3
 2021-01-18 12:52:37 +03:00
alexey-milovidov
78fff6bc39
Merge branch 'master' into multiple-nested 2021-01-15 14:54:27 +03:00
alexey-milovidov
2ca185c9f9
Merge pull request #16995 from sundy-li/alias_column_partition_prune 
Add optimize_respect_aliases
 2021-01-14 11:30:45 +03:00
Alexey Milovidov
c51df126f1 Disable "optimize_move_functions_out_of_any" by default #18973 2021-01-12 19:46:18 +03:00
sundy-li
ad01534bd5 Merge branch 'master' into alias_column_partition_prune 2021-01-12 18:28:30 +08:00
Anton Popov
36ae0e4d35 Merge remote-tracking branch 'upstream/master' into HEAD 2021-01-11 13:51:12 +03:00
Alexey Milovidov
7e9120b34f Merge branch 'master' into chenziliang/feature/wildcard-dynamic-columns 2021-01-10 04:04:47 +03:00
Amos Bird
a3d19fa64d
Correctly override default settings remotely 2021-01-08 12:28:09 +08:00
Amos Bird
0260953a47
better 2021-01-06 17:18:48 +08:00
Vladimir Chebotarev
5cbc25c647 Added global setting s3_max_connections. 2021-01-05 13:34:55 +03:00
Amos Bird
a157a5b3b3
add max_partitions_to_read setting 2021-01-04 12:40:48 +08:00
sundyli
f8815f233c
Merge branch 'master' into alias_column_partition_prune 2021-01-02 11:23:55 +08:00
Alexey Milovidov
4b3ae495d6 Merge branch 'master' into CurtizJ-multiple-nested 2021-01-02 00:25:16 +03:00
alexey-milovidov
a7c350adda
Merge branch 'master' into feature/wildcard-dynamic-columns 2020-12-31 17:47:21 +03:00
Nikita Mikhailov
8e31ce4a5c Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel-parsing-input-format 2020-12-30 18:22:11 +03:00
Nikita Mikhailov
b94a654715 build fix 2020-12-30 16:55:31 +03:00
Alexey Milovidov
68fac02d35 Change union_default_mode to throw exception 2020-12-29 20:27:00 +03:00
TszkitLo40
f7d7880ad5 fix build 2020-12-29 10:13:27 +08:00
Nikita Mikhailov
c3288c3fbf Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel-parsing-input-format 2020-12-28 15:09:37 +03:00
alfredlu
7ec8a73cc3 fix build 2020-12-28 10:38:16 +08:00
alfredlu
51cfbe8c2e Add log_comment setting 2020-12-27 18:35:46 +08:00
Anton Popov
11283e3d81 Merge remote-tracking branch 'upstream/master' into HEAD 2020-12-25 21:25:59 +03:00
Amos Bird
6fc225e676
Distributed insertion to one random shard (#18294) 
* Distributed insertion to one random shard

* add some tests

* add some documentation

* Respect shards' weights

* fine locking

Co-authored-by: Ivan Lezhankin <ilezhankin@yandex-team.ru>
 2020-12-23 19:04:05 +03:00
Anton Popov
57857dda63 Merge remote-tracking branch 'upstream/master' into HEAD 2020-12-23 15:16:43 +03:00
Nikolai Kochetov
af7f5c9518
Merge pull request #17868 from ClickHouse/async-read-from-socket 
Async read from socket
 2020-12-23 12:20:42 +03:00
Alexander Kuzmenkov
51b2329295
Merge branch 'master' into aku/window-prototype 2020-12-23 09:26:11 +03:00
Nikita Mikhailov
dcfbe782c6 Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel-parsing-input-format 2020-12-23 05:20:22 +03:00
Anton Popov
b6ff6300b2 Merge remote-tracking branch 'upstream/master' into HEAD 2020-12-22 18:06:21 +03:00
Alexander Kuzmenkov
85c1bc1253 Merge remote-tracking branch 'origin/master' into tmp 2020-12-21 10:46:21 +03:00
Alexey Milovidov
ae9b67fefa Support to parse Arrays in CSV as nested CSV in a String 2020-12-20 13:26:08 +03:00
alexey-milovidov
34e4ace029
Merge pull request #17525 from ClickHouse/null-as-default-default 
Attempt to make NULL as default by default
 2020-12-18 18:00:34 +03:00
Nikolai Kochetov
c7ef57c6fd Add setting async_socket_for_remote 2020-12-18 16:15:03 +03:00
Anton Popov
009efeff40 Merge remote-tracking branch 'upstream/master' into HEAD 2020-12-18 16:14:09 +03:00
Alexey Milovidov
a0f7a12c4f Merge branch 'master' into null-as-default-default 2020-12-18 08:09:35 +03:00
Alexander Kuzmenkov
4bfcb2fc16 a setting 2020-12-18 04:25:21 +03:00
Nikita Mikhailov
6a55e018e4 Merge branch 'master' of github.com:ClickHouse/ClickHouse into parallel-parsing-input-format 2020-12-18 02:25:32 +03:00
Alexey Milovidov
e4433157e7 Merge branch 'master' into insert_quorum_parallel-default 2020-12-17 22:42:32 +03:00
Anton Popov
a42b00c9aa Merge remote-tracking branch 'upstream/master' into HEAD 2020-12-17 20:43:23 +03:00
hexiaoting
cb91d64fe7
Add new feature: limit length and offset setting (#17633) 
Co-authored-by: Alexander Kazakov <Akazz@users.noreply.github.com>
 2020-12-17 17:16:14 +03:00
sundy-li
84807e9791 rename settings 2020-12-17 02:17:59 +00:00
Alexey Milovidov
6a7c73a7bd Merge branch 'master' into insert_quorum_parallel-default 2020-12-17 04:54:17 +03:00
sundy-li
24a190a461 update setting description 2020-12-16 10:46:47 +00:00
sundy-li
5c9f84b2f4 Merge branch 'master' into alias_column_partition_prune 2020-12-16 18:44:00 +08:00
Anton Popov
e51cf03f21 Merge remote-tracking branch 'upstream/master' into HEAD 2020-12-15 21:08:01 +03:00
Nikolai Kochetov
6defcbb662
Merge branch 'master' into optimize-data-on-insert 2020-12-15 16:50:42 +03:00
sundy-li
cf087002db Merge branch 'master' into alias_column_partition_prune 2020-12-15 16:37:00 +08:00
Anton Popov
c78861e86e Merge remote-tracking branch 'upstream/master' into HEAD 2020-12-15 01:45:01 +03:00
Anton Popov
afc2d7099b add conversion from array to map 2020-12-15 01:16:04 +03:00
nikitamikhaylov
8ff072c702 better 2020-12-15 00:56:47 +03:00
nikitamikhaylov
57705f5b73 delete and fix strange code 2020-12-15 00:56:47 +03:00
Nikita Mikhaylov
9922324787 it works 2020-12-15 00:56:47 +03:00
alexey-milovidov
363ff79cca
Merge pull request #17934 from Jokser/s3-adaptive-single-part-upload 
Adaptive choose of single/multi part upload in WriteBufferFromS3
 2020-12-14 23:00:43 +03:00
Pavel Kovalenko
346c241c20 Merge remote-tracking branch 'origin/master' into s3-adaptive-single-part-upload 
# Conflicts:
#	src/Disks/S3/DiskS3.cpp
 2020-12-14 11:00:30 +03:00
Alexey Milovidov
6341521b10 Remove obsolete settings 2020-12-14 02:40:11 +03:00
alexey-milovidov
bce02de691
Merge pull request #17539 from azat/remerge_sort_lowered_memory_bytes_ratio 
Add remerge_sort_lowered_memory_bytes_ratio setting
 2020-12-13 04:38:50 +03:00
sundyli
36ca21c24a
Merge branch 'master' into alias_column_partition_prune 2020-12-13 00:45:14 +08:00
sundy-li
9407028540 update 2020-12-13 00:42:15 +08:00
Pavel Kovalenko
db7a87089b Adaptive choose of single/multi part upload in WriteBufferFromS3. 2020-12-09 17:09:04 +03:00
Kruglov Pavel
e19eb6f17a
Merge branch 'master' into optimize-data-on-insert 2020-12-08 15:57:46 +03:00
Anton Popov
d7aad3bf79 Merge remote-tracking branch 'upstream/master' into HEAD 2020-12-07 19:13:01 +03:00
Anton Popov
60b0cbb1c1
Merge pull request #15939 from Avogar/optimize_final_optimization 
Optimize final optimization
 2020-12-05 02:26:27 +03:00
Pavel Kruglov
905ba78adc Merge branch 'master' of github.com:ClickHouse/ClickHouse into optimize-data-on-insert 2020-12-04 18:56:46 +03:00
Ivan
315ff4f0d9
ANTLR4 Grammar for ClickHouse and new parser (#11298) 2020-12-04 05:15:44 +03:00
Anton Popov
cd1917c7a6
Merge branch 'master' into optimize_final_optimization 2020-12-03 16:52:51 +03:00
Azat Khuzhin
ccd794a676 Add remerge_sort_lowered_memory_bytes_ratio setting 
Sometimes default ratio is not good enough (2), since it depends on lots
of factors:
- size of LIMIT
- max_memory_usage
- ...

So add a separate setting for it.

But note that it make sense to set
remerge_sort_lowered_memory_bytes_ratio only in range (1, 2]

Since <= 1 will just use more CPU, and > 2 will use more RAM and will
unlikely be faster.
 2020-11-29 21:16:42 +03:00
Alexey Milovidov
7d79a3cc35 Make insert_quorum_parallel by default 2020-11-29 18:13:57 +03:00
Alexey Milovidov
20e56da8ac Attempt to make NULL as default by default 2020-11-28 10:46:08 +03:00
Ken Chen
349571a458 Add asterisk_include_materialized_columns and asterisk_include_materialized_columns switches to include materialized columns and alias columns respectively for wildcard query 2020-11-28 00:06:53 +00:00
Alexander Kuzmenkov
9b0368ffa5
Merge pull request #16338 from ucasFL/union-distinct 
Add UNION DISTINCT and setting `union_default_mode`
 2020-11-24 17:30:24 +03:00
alesapin
45f09fe293
Merge pull request #17220 from ianton-ru/s3_max_redirects 
S3 max redirects
 2020-11-24 12:13:10 +03:00
Alexander Kuzmenkov
412c2fe079 Merge remote-tracking branch 'origin/master' into HEAD 2020-11-23 19:51:18 +03:00
Anton Popov
b384beb564 Merge remote-tracking branch 'upstream/master' into HEAD 2020-11-23 17:46:51 +03:00
Anton Popov
1f65478b49 implement nested with multiple nesting 2020-11-21 20:27:42 +03:00
Pavel Kruglov
ca3fe49a2a Make setting global 2020-11-20 17:29:13 +03:00
Pavel Kruglov
3ee4972586 Merge branch 'master' of github.com:ClickHouse/ClickHouse into optimize-data-on-insert 2020-11-20 01:23:51 +03:00
Alexander Kuzmenkov
24293ccb30 Merge remote-tracking branch 'origin/master' into HEAD 2020-11-19 15:28:37 +03:00
Alexander Kuzmenkov
f2b3f5f8b6 Allow formatting named tuples as JSON objects 2020-11-18 13:38:30 +03:00
Pavel Kruglov
547ec19fb3 Merge branch 'master' of github.com:ClickHouse/ClickHouse into optimize-data-on-insert 2020-11-18 12:01:59 +03:00
Alexander Kuzmenkov
8cde88440b Write rows as JSON array in JSONEachRow output format 2020-11-17 22:50:47 +03:00
Anton Ivashkin
15b5d3a55a Merge master 2020-11-13 19:32:35 +03:00
Anton Ivashkin
026f7e0a27 Add 's3_max_redirects' setting 2020-11-13 19:31:51 +03:00
Pavel Kruglov
8d5e0784d3 Add setting optimize_on_insert 2020-11-12 23:37:23 +03:00
alesapin
a8d466a645 Merge branch 'master' into add_separate_pool_for_fetches 2020-11-12 12:51:02 +03:00
feng lv
fec4080d42 Merge branch 'master' of github.com:ClickHouse/ClickHouse into union-distinct 2020-11-11 03:10:10 +00:00
alesapin
2e357516a6 Merge branch 'master' into add_separate_pool_for_fetches 2020-11-10 13:53:08 +03:00
feng lv
9194985d48 Merge branch 'master' of github.com:ClickHouse/ClickHouse into union-distinct 2020-11-10 06:09:39 +00:00
Nikolai Kochetov
fdce810237 Add setting back. 2020-11-09 22:32:18 +03:00
tavplubix
67099f28ac
Merge pull request #16591 from ClickHouse/aku/create-file 
Support `SETTINGS` clause for File engine
 2020-11-09 14:15:42 +03:00
alesapin
4e846d2197 Merge branch 'master' into add_separate_pool_for_fetches 2020-11-09 10:00:21 +03:00
feng lv
1ca7dcb0a8 fix conflict 2020-11-07 11:44:29 +00:00
alexey-milovidov
7acfa7d407
Merge pull request #16728 from azat/enable-use_compact_format_in_distributed_parts_names-by-default 
Enable use_compact_format_in_distributed_parts_names by default
 2020-11-07 13:40:54 +03:00
Alexander Kuzmenkov
79f759fb6c Merge remote-tracking branch 'origin/master' into HEAD 2020-11-07 11:54:11 +03:00
Alexander Kuzmenkov
3c60f6cec2 make a separate settings collection + some cleanup 2020-11-07 11:53:39 +03:00
Azat Khuzhin
ee46993359 Enable use_compact_format_in_distributed_parts_names by default 2020-11-06 21:02:43 +03:00
Alexander Kuzmenkov
faf1f1e0cd Merge remote-tracking branch 'origin/master' into HEAD 2020-11-05 11:09:11 +03:00
feng lv
56b34d511c Merge branch 'master' of github.com:ClickHouse/ClickHouse into agg-func-setting-null-for-empty 2020-11-04 10:59:29 +00:00
alexey-milovidov
6f004e4abd
Merge branch 'master' into agg-func-setting-null-for-empty 2020-11-04 13:24:39 +03:00
hcz
4dccc51332
Fix typo 2020-11-04 18:21:59 +08:00
alexey-milovidov
973c1d7983
Merge pull request #16529 from azat/log_queries_min_time 
Add log_queries_min_query_duration_ms
 2020-11-04 01:47:18 +03:00
Anton Popov
a3a8e18637
Merge branch 'master' into select_final 2020-11-03 00:00:43 +03:00
Azat Khuzhin
3be8a56f5c Add log_queries_min_query_duration_ms 
Only queries slower then the value of this setting will go to system.query_log,
i.e. something like slow_query_log in mysql.

v2: log_queries_min_time renamed to log_queries_min_query_duration_ms
v3: add current_database into system.query_thread_log
v4: rewrite test using current_database
v5: fix query_duration_ms in system.query_thread_log
 2020-11-02 21:34:54 +03:00
feng lv
4d722bc52c fix conflict 2020-11-02 12:52:12 +00:00
Ivan
281bf351d2
Remove ANALYZE and AST queries (#16536) 2020-11-02 15:47:12 +03:00
alesapin
8b0d9fbc07 Merge branch 'no_background_pool_no_more' into add_separate_pool_for_fetches 2020-11-02 10:54:09 +03:00
Nikolai Kochetov
1c106691b5
Merge pull request #16423 from amosbird/jbodread 
Balanced reading from JBOD
 2020-10-29 19:22:45 +03:00
Amos Bird
f995ef9797
Balanced reading from JBOD 2020-10-29 04:05:07 +08:00
alesapin
0931658c2f Merge branch 'master' into add_separate_pool_for_fetches 2020-10-28 14:20:56 +03:00
Alexander Kuzmenkov
6509f1eb39
Merge branch 'master' into minumum 2020-10-27 21:38:23 +03:00
Mikhail Filimonov
41971e073a
Fix typos reported by codespell 2020-10-27 12:04:03 +01:00
feng lv
9b95ed9ead fix 2020-10-26 12:21:04 +00:00
alesapin
77d350aaeb Bigger setting value 2020-10-26 14:23:59 +03:00
feng lv
a9e4c7144c fix 2020-10-26 09:33:34 +00:00
feng lv
15e4e03469 add union distinct and setting union_default_mode 2020-10-24 13:18:04 +00:00
Alexander Kuzmenkov
5cbf645de4 Merge remote-tracking branch 'origin/master' into tmp 2020-10-22 19:48:02 +03:00
Kruglov Pavel
0b75885c8a
Merge branch 'master' into select_final 2020-10-21 20:42:39 +03:00