thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 19:02:04 +00:00
Code Issues Packages Projects Releases Wiki Activity
64,572 Commits 9,326 Branches 1,783 Tags 2.1 GiB
b26783b308
Commit Graph

1781 Commits

Author SHA1 Message Date
changvvb
5e07551d17 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into support-apple-m1 2021-03-26 16:57:26 +08:00
PHO
9ece398c35 Changed the return type of DB::getCIDRMaskIPv6() 
It now returns std::array instead of uint8_t *. Also added DB::getCIDRMaskIPv4().
 2021-03-26 12:11:34 +09:00
PHO
6606d8ac0b Correct comments about what parseIPv4() and parseIPv6() actually do 
They were wrong as to what endian their results were in.
 2021-03-26 12:11:34 +09:00
Maksim Kita
bbe1960eea
Merge pull request #21573 from kitaisreal/updated-cache-dictionary-storage 
Added specialized CacheDictionaryStorage
 2021-03-25 21:29:01 +03:00
kssenii
bc36baef26 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into library-bridge 2021-03-25 07:48:07 +00:00
Nikolai Kochetov
977ebe8b44
Merge pull request #22058 from azat/system.errors-trace 
Convert system.errors.stack_trace from String into Array(UInt64)
 2021-03-25 10:23:05 +03:00
kssenii
d9326cc229 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into library-bridge 2021-03-24 18:28:55 +00:00
Yuriy Chernyshov
e2760101c1
Update ya.make.in 2021-03-24 13:13:04 +03:00
Yuriy Chernyshov
c89861a2ac
Better 2021-03-24 13:11:52 +03:00
Nikolai Kochetov
f164c2462f
Update Exception.cpp 
Fix build
 2021-03-24 09:41:57 +03:00
Azat Khuzhin
7c07b43597 Convert system.errors.stack_trace from String into Array(UInt64) 
This should decrease overhead for the errors collecting.
 2021-03-24 06:52:48 +03:00
Azat Khuzhin
912144307d Fix type of the ErrorCodes 2021-03-23 23:40:46 +03:00
Alexey Milovidov
d8088190f9 Fix documentation for CurrentMetrics 2021-03-23 00:44:25 +03:00
kssenii
8a650c4408 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into library-bridge 2021-03-22 13:38:25 +00:00
changvvb
b947fcac75 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into support-apple-m1 2021-03-22 10:42:35 +08:00
Maksim Kita
c703ea137b
Merge pull request #21891 from kitaisreal/lru-cache-fix-exception-unsafe-element-insertion 
LRUCache fix exception unsafe element insertion
 2021-03-19 16:37:26 +03:00
Alexander Kuzmenkov
78c56b8913
Merge pull request #21886 from Avogar/hedged 
Add profile event in hedged requests and change timeout from sec to ms.
 2021-03-19 15:50:33 +03:00
changvvb
b4e92f6b12 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into support-apple-m1 2021-03-19 13:52:09 +08:00
Maksim Kita
31b7734407 LRUCache fix exception unsafe element insertion 2021-03-19 00:26:02 +03:00
Pavel Kruglov
d380454044 Change read_data_timeout from sec to ms, add profile event for hedged requests and better process receiving packets 2021-03-18 20:16:17 +03:00
Alexander Kuzmenkov
f85b089292
Merge pull request #21466 from ClickHouse/aku/frame-formatting 
add query formatting idempotence check to fuzzer
 2021-03-18 17:57:11 +03:00
Alexander Tokmakov
343b8c906d Merge branch 'master' into test_multiple_nodes 2021-03-18 16:01:04 +03:00
Maksim Kita
18ed9c5c67 SSDCacheDictionary remove max_stored_keys option 2021-03-18 12:55:17 +03:00
Maksim Kita
f140204279 Improved performance of SSDCache dictionary 2021-03-17 22:01:45 +03:00
Maksim Kita
bc22f4f6eb Updated implementation 2021-03-17 20:14:36 +03:00
kssenii
34dea63248 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into library-bridge 2021-03-17 13:32:43 +00:00
Nikita Mikhaylov
f51b41b7a4
Merge pull request #21710 from kssenii/replica-priorities 
Support replicas priority for postgres dictionary source
 2021-03-17 16:02:00 +03:00
Nikolai Kochetov
4f1f344d4d
Merge pull request #21529 from azat/system.errors-improvements 
Add last_error_time/last_error_message/last_error_stacktrace/remote columns for system.errors
 2021-03-17 11:51:29 +03:00
Alexander Tokmakov
ff79be8af9 fix 2021-03-16 23:01:20 +03:00
Azat Khuzhin
9dee842b60 Distinguish remote and local error info 2021-03-16 22:31:49 +03:00
Azat Khuzhin
cc87bcfb63 Fix errorCodeToName() for signed integers 
- https://clickhouse-test-reports.s3.yandex.net/21529/2ce2772d35eb3d81628f4d294d5799e9f05333fd/functional_stateless_tests_(address).html#fail1
- https://clickhouse-test-reports.s3.yandex.net/21529/2ce2772d35eb3d81628f4d294d5799e9f05333fd/functional_stateless_tests_(ubsan).html#fail1
- https://clickhouse-test-reports.s3.yandex.net/21529/2ce2772d35eb3d81628f4d294d5799e9f05333fd/stress_test_(address).html#fail1
 2021-03-16 22:31:49 +03:00
Azat Khuzhin
efdd04c958 Drop last_ prefix for ErrorCodes::ValuePair::error_time_ms 2021-03-16 22:31:49 +03:00
Azat Khuzhin
44c9dc753d Add system.errors.last_error_stacktrace column 2021-03-16 22:31:49 +03:00
Azat Khuzhin
775f8f7682 Add system.errors.last_error_message column 2021-03-16 22:31:49 +03:00
Azat Khuzhin
c8852331a2 Add system.errors.last_error_time column 2021-03-16 22:31:49 +03:00
Azat Khuzhin
0d01eaf94f Guard ErrorCodes with mutex over atomic 2021-03-16 22:31:49 +03:00
Azat Khuzhin
7f73ac2b7a Fix ErrorCodes::Value/ErrorCode types (sigh) 
Note, that system.errors already uses correct types
 2021-03-16 22:31:49 +03:00
Azat Khuzhin
259e5ba88e Separate accounting of remote exceptions in system.errors 2021-03-16 22:31:46 +03:00
Azat Khuzhin
b39f8cc6ac Move ErrorCodes::increment() into module part 2021-03-16 22:31:41 +03:00
Alexey Milovidov
ea049b2cc0 Merge branch 'master' into DateTime64_extended_range 2021-03-16 14:08:02 +03:00
changvvb
bba600d558 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into support-apple-m1 2021-03-16 13:26:00 +08:00
Maksim Kita
3d1c42827b Added FixedDeadlineHashMap 2021-03-16 01:59:04 +03:00
Alexander Kuzmenkov
e485a27dc1 some fixes 2021-03-15 22:45:45 +03:00
Anton Popov
840417c957
Merge pull request #21685 from Avogar/reload-cluster-config 
Update clusters only if their configs were updated
 2021-03-15 20:09:58 +03:00
Alexey Milovidov
eaeb4a3bf0 Merge branch 'master' into DateTime64_extended_range 2021-03-15 16:59:32 +03:00
kssenii
19de1ec457 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into library-bridge 2021-03-15 06:54:02 +00:00
alexey-milovidov
afa24938bc
Merge pull request #21686 from kitaisreal/library-dictionary-source-fix-possible-leak 
LibraryDictionarySource fix possible leak
 2021-03-15 00:06:15 +03:00
kssenii
8688ae9346 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into library-bridge 2021-03-14 19:32:21 +00:00
Maksim Kita
dfd2068350 Updated SharedLibrary string_view interface 2021-03-14 17:19:48 +03:00
Maksim Kita
ce20eae2a3 Added specialized CacheDictionaryStorage 2021-03-14 14:33:21 +03:00
Maksim Kita
1ac62970c4 Modified SharedLibrary interface 2021-03-14 14:29:02 +03:00
alexey-milovidov
53e21443eb
Merge branch 'master' into DateTime64_extended_range 2021-03-14 12:27:17 +03:00
kssenii
8eea6a87e9 Support replicas priority for postgres dictionary source 2021-03-14 08:51:24 +00:00
alesapin
06eb2d8dfd
Merge branch 'master' into s3_zero_copy_replication 2021-03-13 22:32:54 +03:00
alexey-milovidov
c9265b64f5
Merge pull request #21678 from ClickHouse/podarray-cleanup 
PODArray swap fix
 2021-03-13 22:28:48 +03:00
Maksim Kita
468ab258bf Remove protect 2021-03-13 14:55:56 +03:00
Pavel Kruglov
58e415c7d4 Update clusters only if their configs were updated 2021-03-13 00:17:19 +03:00
kssenii
323fb54a8e Fix split build finally 2021-03-12 21:12:34 +00:00
Maksim Kita
12e411f8b9 PODArray swap fix 2021-03-12 21:38:52 +03:00
kssenii
db57520369 Update ya.make 2021-03-12 14:39:24 +00:00
Alexander Kuzmenkov
122790efd1
Merge pull request #21629 from ucasFL/errorcode 
redefine some errorcode
 2021-03-12 16:25:13 +03:00
feng lv
1eba096f2a fix conflict 2021-03-12 10:28:04 +00:00
Anton Ivashkin
ace628d565 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into s3_zero_copy_replication 2021-03-12 12:58:32 +03:00
alexey-milovidov
9e0d53c0e6
Merge pull request #20058 from excitoon-favorites/remotehostfilterdisallowall 
Fixed open behavior of remote host filter in case when there is `remote_url_allow_hosts` section in configuration but no entries there
 2021-03-12 11:14:15 +03:00
alesapin
b98e27ba82
Merge branch 'master' into s3_zero_copy_replication 2021-03-12 11:07:20 +03:00
kssenii
70f8f90511 Fix split build 2021-03-12 08:04:20 +00:00
kssenii
9e0101e5fc Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into library-bridge 2021-03-12 07:47:22 +00:00
Denis Glazachev
290a6d273e
Add Kerberos support for authenticating existing users when accessing over HTTP 2021-03-11 23:41:10 +03:00
changvvb
fcb238715b Support apple m1 2021-03-12 01:12:52 +08:00
Nikolai Kochetov
cc60059b50
Merge pull request #21196 from ClickHouse/improve-performance-group-by-multiple-fixed-keys-2 
Save packed keys for GROUP BY with multiple fixed size keys
 2021-03-11 18:28:14 +03:00
Alexander Kuzmenkov
3e5447ccec
Merge pull request #21480 from Avogar/slowdown_count 
Add slowdown_count and show actual errors_count in system.clusters
 2021-03-11 17:18:05 +03:00
feng lv
083299e549 redefine some errorcode 2021-03-11 13:54:11 +00:00
alexey-milovidov
647246006a
Merge pull request #21533 from ClickHouse/podarray-cleanup 
Cleanup PODArray
 2021-03-11 04:34:25 +03:00
kssenii
4d29241d5a Try fix build 2021-03-10 21:14:09 +00:00
kssenii
e77ff68582 Better 2021-03-10 18:26:15 +00:00
kssenii
38f7f37468 Add loadKeys method 2021-03-10 13:23:07 +00:00
Anton Ivashkin
cef0dda933 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into s3_zero_copy_replication 2021-03-10 14:10:29 +03:00
Nikolai Kochetov
4c8c45b163 Refactor 2021-03-10 14:00:24 +03:00
kssenii
f6610ceaad Try fix build 2021-03-10 08:09:05 +00:00
alexey-milovidov
f4db00bdb2
Merge pull request #21546 from kitaisreal/shell-command-wait-pid-signal-eintr-fix 
ShellCommand waitpid eintr signal fix
 2021-03-10 09:12:53 +03:00
tavplubix
5a6cc876e4
Merge pull request #20626 from ClickHouse/fix_force_drop_materialized_view 
Make force_drop_table work with MaterializedView
 2021-03-09 20:05:51 +03:00
Maksim Kita
43ba554191
Merge pull request #20595 from kitaisreal/cache-dictionaries-lru-cache 
Cache dictionaries lru cache
 2021-03-09 19:14:30 +03:00
alesapin
bcd073278d
Merge pull request #21425 from ClickHouse/persistent_nukeeper_snapshot_storage 
Persistent nukeeper snapshot storage
 2021-03-09 18:37:23 +03:00
Maksim Kita
289712f19d
Merge pull request #21532 from kitaisreal/pod-array-left-pad-not-multiple-of-element-crash-fix 
Pod array left pad not multiple of element crash fix
 2021-03-09 14:14:52 +03:00
Maksim Kita
8dd52f087e Fixed PODArray 2021-03-09 14:14:31 +03:00
Maksim Kita
882dde56ea ShellCommand waitpid eintr signal fix 2021-03-09 13:13:19 +03:00
alesapin
c00c8c36b7 Merge branch 'master' into ianton-ru-s3_zero_copy_replication 2021-03-09 10:59:11 +03:00
Kruglov Pavel
c464e3c57b
Merge pull request #21311 from ucasFL/fix-format 
improve clickhouse-format
 2021-03-09 02:16:05 +03:00
Alexey Milovidov
1b3043a583 Fix error (found by @kitaisreal) 2021-03-09 02:15:33 +03:00
Alexey Milovidov
d3aeb30cc0 Remove unused method 2021-03-09 02:10:26 +03:00
Alexey Milovidov
9e23d16739 Cleanup PODArray 2021-03-09 02:03:46 +03:00
Maksim Kita
9ed791fd30 Updated style check 2021-03-09 01:33:16 +03:00
Maksim Kita
642c45d769 PODArray left pad not multiple of element crash fix 2021-03-09 01:32:31 +03:00
kssenii
5ee2925123 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into library-bridge 2021-03-08 19:53:48 +00:00
alesapin
f555e65cc8 Merge branch 'master' into persistent_nukeeper_snapshot_storage 2021-03-08 22:30:17 +03:00
Alexey Milovidov
355411dfa8 Merge branch 'master' into DateTime64_extended_range 2021-03-08 02:55:14 +03:00
kssenii
8a9fa561c7 Fix style check, build, ya check 2021-03-07 20:07:42 +00:00
kssenii
b7b15fe920 Remove unused 2021-03-07 15:23:20 +00:00
kssenii
9e9bf2bb75 Common base for bridges part 3 2021-03-07 13:55:40 +00:00
Maksim Kita
92107a6ded Fixed documentation message 2021-03-07 14:43:08 +03:00
kssenii
1c4d4c8e54 Better handlers 2021-03-07 11:31:55 +00:00
kssenii
61d8e27ea7 Common base for bridges part 2 2021-03-07 11:21:49 +00:00
alesapin
0ee7425b16 Remove redundant methods 2021-03-07 10:30:50 +03:00
Alexey Milovidov
1bb62f578b Unit tests passing 2021-03-07 04:10:38 +03:00
Alexey Milovidov
8271cec093 Fix error 2021-03-07 02:36:24 +03:00
kssenii
25fc6a29e5 Make common BridgeHelper base class for all bridges 2021-03-06 23:22:22 +00:00
Alexey Milovidov
e33fae7625 Improve test 2021-03-07 01:35:49 +03:00
Maksim Kita
6414da73bf Fixed code review issues 2021-03-06 21:45:51 +03:00
kssenii
f83a5d83a2 Better 2021-03-06 18:44:40 +00:00
Maksim Kita
c314de7388 Fixed documentation 2021-03-06 14:37:56 +03:00
Maksim Kita
717a9b048a Fixed tests 2021-03-06 14:37:36 +03:00
Maksim Kita
7602c6c6cb Fixed sanitizers errors 2021-03-06 14:36:37 +03:00
Maksim Kita
d04e707567 Fixed special build 2021-03-06 14:36:37 +03:00
Maksim Kita
a22ec65e8a Fixed minor issues 2021-03-06 14:36:37 +03:00
Maksim Kita
b6e72334ef Fixed DefaultHash for StringRef 2021-03-06 14:36:37 +03:00
Pavel Kruglov
dbae58f30b Add slowdown_count and show actual information in system.clusters 2021-03-06 02:45:17 +03:00
kssenii
2c080da51b Better 2021-03-05 15:37:43 +00:00
feng lv
1438fd20cb Merge branch 'master' of github.com:ClickHouse/ClickHouse into fix-format 2021-03-05 14:33:26 +00:00
kssenii
e0cda1033a More methods 2021-03-05 10:43:47 +00:00
Maksim Kita
cdee265378 Fix typos 2021-03-05 13:27:24 +03:00
kssenii
dd4a7b6e3d First version 2021-03-05 10:19:01 +00:00
Maksim Kita
175c485fcb Fix style check 2021-03-05 13:17:46 +03:00
Maksim Kita
9173544003 Added documentation 2021-03-05 13:17:46 +03:00
Maksim Kita
aa0778933c Added BorrowedObjectPool 2021-03-05 13:17:46 +03:00
Maksim Kita
91e43a6b3b Updated ExecutablePoolDictionarySource 2021-03-05 13:17:46 +03:00
Maksim Kita
6b69d859c1 Added ExecutablePool dictionary source 2021-03-05 13:17:45 +03:00
alesapin
c9c301e10c Merge branch 'master' into persistent_nukeeper_snapshot_storage 2021-03-05 10:12:09 +03:00
Alexey Milovidov
3e77c9b1c5 Merge branch 'master' into DateTime64_extended_range 2021-03-05 03:07:19 +03:00
Nikita Mikhaylov
37d4c69d3a
Merge pull request #21433 from azat/parsing-exception-message-fix 
Fix ParsingException::displayText()
 2021-03-05 01:59:06 +03:00
alesapin
aa1312ac79 Merge with master 2021-03-04 14:43:12 +03:00
alesapin
ff67860770 Merge branch 'master' into persistent_nukeeper_snapshot_storage 2021-03-04 14:30:51 +03:00
Anton Ivashkin
bec9de7161 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into s3_zero_copy_replication 2021-03-04 13:29:45 +03:00
Anton Ivashkin
e69124a0a6 Merge master 2021-03-04 13:26:40 +03:00
Alexey Milovidov
9416f59014 Some progress on tests 2021-03-04 06:48:24 +03:00
Alexey Milovidov
b9a8509f79 Adjustments 2021-03-04 05:18:17 +03:00
Alexey Milovidov
6dc21cd098 Remove discrepances 2021-03-04 03:48:37 +03:00
Alexey Milovidov
9f3760e1e3 Bad filename 2021-03-04 03:18:48 +03:00
Alexey Milovidov
3eb3830dd8 Remove trash 2021-03-04 02:13:49 +03:00
Alexey Milovidov
1096fc328a Make some tests automated 2021-03-04 02:11:27 +03:00
Alexey Milovidov
329074bd02 Fix test 2021-03-04 01:55:56 +03:00
Alexey Milovidov
2632b568ae Move tests to appropriate place 2021-03-04 01:34:30 +03:00
Azat Khuzhin
e7948819f9 Fix ParsingException::displayText() 
Before it was silently try-catched for messages with additional {}, and
it is very easy to trigger, i.e.:

    SELECT toDateTime(format('{}-{}-01 00:00:00', '2021', '1'))

Will print:

    Code: 41. DB::Exception: Received from localhost:9000. DB::Exception: Cannot parse datetime 2021-1-01 00:00:00{}: Cannot parse DateTime from String: while executing 'FUNCTION toDateTime(format('{}-{}-01 00:00:00', '2021', '1') :: 3) -> toDateTime(format('{}-{}-01 00:00:00', '2021', '1')) DateTime : 2'.
 2021-03-03 23:44:51 +03:00
Azat Khuzhin
6965ac26c3 Distributed: Add ability to delay/throttle INSERT until pending data will be reduced 
Add two new settings for the Distributed engine:
- bytes_to_delay_insert
- max_delay_to_insert

If at the beginning of INSERT there will be too much pending data, more
then bytes_to_delay_insert, then the INSERT will wait until it will be
shrinked, and not more then max_delay_to_insert seconds.

If after this there will be still too much pending, it will throw an
exception.

Also new profile events were added (by analogy to the MergeTree):
- DistributedDelayedInserts (although you can use system.errors instead
  of this, but still)
- DistributedRejectedInserts
- DistributedDelayedInsertsMilliseconds
 2021-03-03 23:30:23 +03:00
Azat Khuzhin
70049db143 CurrentMetrics/Increment: Introduce add() 2021-03-03 23:30:03 +03:00
Azat Khuzhin
b5a5778589 Distributed: Add ability to limit amount of pending bytes for async INSERT 
Right now with distributed_directory_monitor_batch_inserts=1 and
insert_distributed_sync=0 INSERT into Distributed table will store
blocks that should be sent to remote (and in case of
prefer_localhost_replica=0 to the localhost too) on the local
filesystem, and sent it in background.

However there is no limit for this storage, and if the remote is
unavailable (or some other error), these pending blocks may take
significant space, and this is not always desired behaviour.

Add new Distributed setting - bytes_to_throw_insert, that will set the
limit for how much pending bytes is allowed, if the limit will be
reached an exception will be throw.

By default was set to 0, to avoid surprises.
 2021-03-03 23:30:00 +03:00
alesapin
e6083c4dc4 Fix style 2021-03-03 19:25:47 +03:00
alesapin
eb5e416fd5 Merge branch 'master' into persistent_nukeeper_snapshot_storage 2021-03-03 19:19:02 +03:00
alesapin
e615299ae6 Fix some bugs and add snapshots validation 2021-03-03 14:10:24 +03:00
feng lv
742e5a05ff fix conflict 2021-03-03 09:38:30 +00:00
alesapin
8e6252b25f Use snapshot object in serialization 2021-03-02 17:30:56 +03:00
alesapin
e7399bf66f Merge branch 'master' into persistent_nukeeper_snapshot_storage 2021-03-02 14:03:42 +03:00
Pavel Kruglov
153bfbfc28 Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-03-02 11:59:32 +03:00
alesapin
03960b1eed Some compileable code 2021-03-01 16:33:34 +03:00
Anton Ivashkin
98065ec56e Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into s3_zero_copy_replication 2021-03-01 13:33:40 +03:00
alesapin
91bc4478d7 Merge branch 'master' into persistent_nukeeper_snapshot_storage 2021-03-01 13:18:35 +03:00
Nikolai Kochetov
976dbe8077
Merge pull request #20341 from ClickHouse/filter-push-down 
Filter push down
 2021-03-01 12:35:06 +03:00
feng lv
cbeb2853fb improve clickhouse-format 
fix

fix error word
 2021-02-28 11:34:25 +00:00
alexey-milovidov
b8fba768e5
Merge pull request #21264 from ClickHouse/fix_zookeeper_update 
Fix several bugs with ZooKeeper client
 2021-02-28 01:57:04 +03:00
alesapin
9dc3952ff3 Fix typo 2021-02-27 16:30:43 +03:00
alesapin
9e93d7f507 Fix tidy and add comments 2021-02-27 11:07:14 +03:00
Pavel Kruglov
197058dc12 Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-02-27 00:36:33 +03:00
alesapin
a48bb7e3a0 Fix redundant zookeeper reconnect and possibility of multiple zookeeper session for a single ClickHouse server 2021-02-26 22:05:25 +03:00
Nikolai Kochetov
d91b8a3acb
Merge branch 'master' into filter-push-down 2021-02-26 19:33:27 +03:00
Nikolai Kochetov
d328bfa41f Review fixes. Add setting max_optimizations_to_apply. 2021-02-26 19:29:56 +03:00
Nikolai Kochetov
545528917f Fix tests. 2021-02-26 13:17:00 +03:00
Anton Ivashkin
5b267b7eec Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into s3_zero_copy_replication 2021-02-26 13:15:18 +03:00
Anton Ivashkin
c891cf4557 Fixes by review response 2021-02-26 12:48:57 +03:00
Nikolai Kochetov
64ea1f8298 Save packet keys. 2021-02-25 18:51:01 +03:00
Maksim Kita
63173eebdb
Merge pull request #21088 from azat/__has_feature-gcc-fix 
Fix __has_feature macros under gcc
 2021-02-25 18:34:14 +03:00
alesapin
cf42824756 Experimenting 2021-02-25 17:22:51 +03:00
Azat Khuzhin
810023a65c Do not undefine __has_feature (may break c++ headers) 
If the order of <common/defines.h> and c++ header will be wrong the
compilation will be broken.

v2: rename __ch_has_feature to ch_has_feature to fix -Wreserved-id-macro
v3: do not fallback to 0
 2021-02-23 19:46:47 +03:00
Azat Khuzhin
2ced21c837 Decrease number of stored stack frame pointers to fit into 512 PIPE_BUF 2021-02-23 07:52:01 +03:00
Azat Khuzhin
6b39e61d09 Decrease QUERY_ID_MAX_LEN for freebsd/osx (PIPE_BUF=512) 2021-02-23 07:47:58 +03:00
Azat Khuzhin
89c311981d Fix buffer size for trace collection 
[1]: https://clickhouse-test-reports.s3.yandex.net/0/a860e3e7582f489a640c42db3bd4bd320f97a5bc/stress_test_(debug).html#fail1
 2021-02-22 22:45:04 +03:00
Kruglov Pavel
815b4ff0d9
Merge branch 'master' into hedged-requests 2021-02-21 17:10:24 +03:00
Pavel Kruglov
dd5185c779 Refactor 4.0 2021-02-21 17:03:24 +03:00
alexey-milovidov
20a7ccf5b1
Merge pull request #20174 from amosbird/alias 
Normalized function names (lower-case case-insensitive names)
 2021-02-21 16:23:51 +03:00
Alexey Milovidov
00e6b6232f Revert "optimize aggfunc column data copy (#19407)" 
This reverts commit 7e3186c826.
 2021-02-21 12:49:50 +03:00
tavplubix
103d569885
Merge branch 'master' into fix_force_drop_materialized_view 2021-02-20 20:41:03 +03:00
tavplubix
a697b578bc
Merge pull request #16193 from ClickHouse/database_replicated 
DatabaseReplicated: continuation of #10485
 2021-02-20 19:39:34 +03:00
Alexander Tokmakov
2a36d6cb55 review suggestions 2021-02-20 02:41:58 +03:00
Alexander Tokmakov
5cfd687dfb fix 2021-02-19 19:22:47 +03:00
Ivan
414f470c79
Make Poco HTTP Server zero-copy again (#19516) 
* Refactoring: part 1

* Refactoring: part 2

* Handle request using ReadBuffer interface

* Struggles with ReadBuffer's

* Fix URI parsing

* Implement parsing of multipart/form-data

* Check HTTP_LENGTH_REQUIRED before eof() or will hang

* Fix HTTPChunkedReadBuffer

* Fix build and style

* Fix test

* Resist double-eof

* Fix arcadian build
 2021-02-19 15:51:26 +03:00
Alexander Tokmakov
8097532cb7 Merge branch 'master' into database_replicated 2021-02-18 17:42:06 +03:00
Amos Bird
77fd060665
Normalize function names 2021-02-18 20:20:28 +08:00
Kruglov Pavel
142a541361
Merge branch 'master' into hedged-requests 2021-02-18 14:23:00 +03:00
Vitaly Baranov
3cbb325913
Merge pull request #20506 from vitlibar/refactor-protobuf-format-io 
Improved serialization in Protobuf format.
 2021-02-18 11:31:37 +03:00
Vitaly Baranov
18e036d19b Improved serialization for data types combined of Arrays and Tuples. 
Improved matching enum data types to protobuf enum type.
Fixed serialization of the Map data type.
Omitted values are now set by default.
 2021-02-17 20:50:09 +03:00
Kruglov Pavel
794c3449ca
Merge branch 'master' into hedged-requests 2021-02-17 20:36:36 +03:00
Pavel Kruglov
1b78de2142 Use fibers in HedgedRequests 2021-02-17 20:34:52 +03:00
Alexander Tokmakov
1aac7b3471 Merge branch 'master' into database_replicated 2021-02-17 00:39:56 +03:00
tavplubix
68b427a99d
Merge pull request #20448 from ClickHouse/better_ddl_queue_cleanup 
Better distributed DDL queue cleanup
 2021-02-16 19:29:50 +03:00
Alexander Tokmakov
bf6f64a3fb Merge branch 'master' into database_replicated 2021-02-16 01:28:19 +03:00
Pavel Kruglov
937a3192eb Fix data race 2021-02-16 00:06:34 +03:00
Alexander Tokmakov
cf57c3b4a2 update comments 2021-02-15 23:01:21 +03:00
Maksim Kita
e3003add57 HashTable fix bug during resize with nonstandard grower 2021-02-15 22:23:10 +03:00
Pavel Kruglov
454b1e0d60 Merge branch 'hedged-requests' of github.com:Avogar/ClickHouse into hedged-requests 2021-02-15 17:44:29 +03:00
Pavel Kruglov
eb0847e2b6 Style 2021-02-15 17:44:05 +03:00
Kruglov Pavel
598576ce70
Merge branch 'master' into hedged-requests 2021-02-15 16:51:45 +03:00
Pavel Kruglov
df09a5cac9 Refactor 3.0 and fix tests 2021-02-15 16:21:36 +03:00
Alexey Milovidov
8b7063acf4 Merge branch 'master' into improve-performance-group-by-multiple-fixed-keys 2021-02-15 04:11:01 +03:00
Alexey Milovidov
5bdc570046 Improve performance of GROUP BY multiple fixed size keys 2021-02-14 01:56:04 +03:00
Alexander Tokmakov
96cc01926c Merge branch 'master' into better_ddl_queue_cleanup 2021-02-14 00:22:09 +03:00
Azat Khuzhin
06e8065ee6 Add missing sync of underlying files 2021-02-13 13:19:51 +03:00
alesapin
f80137626a
Merge pull request #19580 from ClickHouse/in_memory_raft 
In memory coordination inside ClickHouse
 2021-02-13 10:19:23 +03:00
Alexander Tokmakov
184ec67dac better ddl queue cleanup 2021-02-12 22:23:50 +03:00
alexey-milovidov
f669a9f97a
Merge pull request #20348 from kitaisreal/common-types-template-instantiations 
Common types template instantiations
 2021-02-12 15:53:00 +03:00
Maksim Kita
f9527738c9 Added comments 2021-02-12 12:12:04 +03:00
alexey-milovidov
4e32dbd024
Merge pull request #19317 from abyss7/dwarf-folly 
Print inline frames for fatal stacktraces
 2021-02-12 02:38:50 +03:00
Maksim Kita
b276eac197 Common types template instantiations 2021-02-12 00:54:50 +03:00
alesapin
5f32f65211 Merge branch 'master' into in_memory_raft 2021-02-11 23:09:29 +03:00
alexey-milovidov
dc3ffd3fe2
Merge pull request #19451 from azat/safe-writes 
Do not silently ignore write errors
 2021-02-11 21:19:11 +03:00
Alexander Tokmakov
15256d86e5 better replica recovery and queue cleanup 2021-02-10 23:30:40 +03:00
Anton Popov
ece7e87cfb
Merge pull request #20079 from ClickHouse/vdimir/fix-join-used-flag 
Split use flags and RowRef in HashJoin
 2021-02-10 20:34:22 +03:00
alesapin
b4d53886a3 Add sync request/response 2021-02-10 16:01:05 +03:00
alesapin
0d179e021b Add sync cmd 2021-02-10 13:15:42 +03:00
alesapin
9667bdcbd2 Merge branch 'master' into in_memory_raft 2021-02-09 16:06:31 +03:00
Alexander Tokmakov
5e0cbfe972 Merge branch 'master' into database_replicated 2021-02-08 23:01:12 +03:00
Alexander Tokmakov
78c1d69b8c better code 2021-02-08 22:36:17 +03:00
tavplubix
ac477d9850
Merge pull request #19771 from ClickHouse/thread_state_improvements 
Minor code improvements around ThreadStatus
 2021-02-08 22:34:55 +03:00
Maksim Kita
ebf5868652
Merge pull request #20164 from kitaisreal/cache-dictionaries-lru-cache 
Added LRUHashMap
 2021-02-08 21:00:27 +03:00
Ivan Lezhankin
a287443438 Merge remote-tracking branch 'upstream/master' into dwarf-folly 2021-02-08 18:36:41 +03:00
templarzq
7e3186c826
optimize aggfunc column data copy (#19407) 
* optimize agg column data copy

* add aggfunc col data copy test

* fix null warning

* fix style

* fix style

* fix style

* fix style

* fix copy same dat to same pos bug

* fix style

* fix style

* Update ColumnAggregateFunction.cpp

fix type check

* control record buf size

* fix performance

* fix performance

* fix destroy data speed

* decrease run time.

* fix warning

* decrease test run time

* Update aggfunc_col_data_copy.xml

* replace unordered_map with HashMap

* fix compile err

* fix compile err

* fix compile err

* fix compile err

* fix compile err

* fix compile err

* fix compile err

* fix compile err

* fix compile err

* fix hash map compile err

* fix compile err

* fix compile err

* fix format

* faster test

Co-authored-by: centos7 <templarzq@localhost.localdomain>
 2021-02-08 16:18:48 +03:00
vdimir
776b682f28
Add comment for used_flags in hash join, fix build 2021-02-08 14:38:31 +03:00
Kruglov Pavel
b256bd455c
Merge pull request #19961 from ucasFL/shard-id 
add setting insert_shard_id
 2021-02-08 14:28:13 +03:00
Maksim Kita
7ee1ad6833 Fixed unit tests 2021-02-08 11:44:32 +03:00
alesapin
d5ee7d33e9 Merge branch 'master' into in_memory_raft 2021-02-08 10:30:31 +03:00
Maksim Kita
285d46bc56 Fix gcc build issues 2021-02-07 23:54:14 +03:00
Maksim Kita
1201c4c55f Fixed build issues 2021-02-07 16:45:29 +03:00
Maksim Kita
eadd23aa38 Added LRUHashMap 2021-02-07 00:06:27 +03:00
Pavel Kruglov
9048dc43d4 Fix style and build 2021-02-06 22:13:50 +03:00
Pavel Kruglov
f946aab759 Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-02-06 17:38:56 +03:00
Pavel Kruglov
794f185442 Fix 2021-02-06 17:23:48 +03:00
Kruglov Pavel
d94e8624d7
Merge branch 'master' into shard-id 2021-02-06 16:48:17 +03:00
Alexey Milovidov
4092916db6 Useless changes 2021-02-06 14:56:10 +03:00
Pavel Kruglov
0704d3cf27 Refactor 2021-02-06 03:54:27 +03:00
alesapin
011109c82a
Merge pull request #17348 from xjewer/alex/CLICKHOUSE-606_deduplication_UUID 
CLICKHOUSE-606: query deduplication based on parts' UUID
 2021-02-05 22:47:34 +03:00
vdimir
01a6e01ad7
Store usage info flags separate from row refs in hash join 2021-02-05 16:20:13 +03:00
Azat Khuzhin
98e3a99a88 Do not catch exceptions during final flush in writers destructors 
Since this hides real problems, since destructor does final flush and if
it fails, then data will be lost.

One of such examples if MEMORY_LIMIT_EXCEEDED exception, so lock
exceptions from destructors, by using
MemoryTracker::LockExceptionInThread to block these exception, and allow
others (so std::terminate will be called, since this is c++11 with
noexcept for destructors by default).

Here is an example, that leads to empty block in the distributed batch:

    2021.01.21 12:43:18.619739 [ 46468 ] {7bd60d75-ebcb-45d2-874d-260df9a4ddac} <Error> virtual DB::CompressedWriteBuffer::~CompressedWriteBuffer(): Code: 241, e.displayText() = DB::Exception: Memory limit (for user) exceeded: would use 332.07 GiB (attempt to allocate chunk of 4355342 bytes), maximum: 256.00 GiB, Stack trace (when copying this message, always include the lines below):

    0. DB::Exception::Exception<>() @ 0x86f7b88 in /usr/bin/clickhouse
    ...
    4. void DB::PODArrayBase<>::resize<>(unsigned long) @ 0xe9e878d in /usr/bin/clickhouse
    5. DB::CompressedWriteBuffer::nextImpl() @ 0xe9f0296 in /usr/bin/clickhouse
    6. DB::CompressedWriteBuffer::~CompressedWriteBuffer() @ 0xe9f0415 in /usr/bin/clickhouse
    7. DB::DistributedBlockOutputStream::writeToShard() @ 0xf6bed4a in /usr/bin/clickhouse
 2021-02-05 01:31:45 +03:00
Azat Khuzhin
a46d65f99d Fix typo in comment for memoryTrackerCanThrow() 2021-02-05 01:29:46 +03:00
alesapin
da51ea1794 Simplify shutdown and requests processing 2021-02-04 15:07:41 +03:00
Vladimir Chebotarev
a597ed0ff0 Fixed open behavior of remote host filter in case when there is remote_url_allow_hosts section in configuration but no entries there. 2021-02-04 09:34:31 +03:00
alesapin
1ff87ac6f9 Add background session lifetime control 2021-02-03 23:32:15 +03:00
Alexander Tokmakov
87502d0220 Merge branch 'thread_state_improvements' into database_replicated 2021-02-03 20:19:35 +03:00
Alexander Tokmakov
d010f97db0 Merge branch 'master' into database_replicated 2021-02-03 20:13:25 +03:00
Alexander Kuzmenkov
bb45c5a8ff Merge remote-tracking branch 'origin/master' into HEAD 2021-02-03 12:13:23 +03:00
Alexander Kuzmenkov
7e945bab03 fix the calculation for moving frame start 2021-02-03 08:53:21 +03:00
Aleksei Semiglazov
921518db0a CLICKHOUSE-606: query deduplication based on parts' UUID 
* add the query data deduplication excluding duplicated parts in MergeTree family engines.

query deduplication is based on parts' UUID which should be enabled first with merge_tree setting
assign_part_uuids=1

allow_experimental_query_deduplication setting is to enable part deduplication, default ot false.

data part UUID is a mechanism of giving a data part a unique identifier.
Having UUID and deduplication mechanism provides a potential of moving parts
between shards preserving data consistency on a read path:
duplicated UUIDs will cause root executor to retry query against on of the replica explicitly
asking to exclude encountered duplicated fingerprints during a distributed query execution.

NOTE: this implementation don't provide any knobs to lock part and hence its UUID. Any mutations/merge will
update part's UUID.

* add _part_uuid virtual column, allowing to use UUIDs in predicates.

Signed-off-by: Aleksei Semiglazov <asemiglazov@cloudflare.com>

address comments
 2021-02-02 16:53:39 +00:00
Kruglov Pavel
0b4a9ed87a
Fix gcc-9 build 2021-02-02 17:38:42 +03:00
alesapin
9979181c02 Merge branch 'master' into in_memory_raft 2021-02-02 17:05:20 +03:00
feng lv
4279c7da41 add setting insert_shard_id 
add test

fix style

fix
 2021-02-02 04:26:59 +00:00
Alexey Milovidov
656cd583f7 Add MSan annotation for system.stack_trace 2021-02-02 06:46:54 +03:00
Pavel Kruglov
7d9eb966f0 Fix 2021-02-01 20:09:55 +03:00
alesapin
0aca40d8cb Merge branch 'master' into in_memory_raft 2021-02-01 14:30:11 +03:00
alexey-milovidov
342baa4223
Merge pull request #19720 from ClickHouse/unicode-asan 
Fix unsufficient args check (trash code) in StringSearcher
 2021-01-31 15:43:05 +03:00
alexey-milovidov
842c2c840c
Merge pull request #19773 from ClickHouse/tavplubix-patch-3 
Fix stack overflow in FieldVisitorAccurateEquals
 2021-01-31 11:09:41 +03:00
Alexey Milovidov
40a356db74 Fix warning 2021-01-31 07:40:51 +03:00
Alexey Milovidov
0fa5142715 Remove tons of garbage 2021-01-31 05:36:52 +03:00
Alexey Milovidov
5c718c8754 Merge branch 'master' into unicode-asan 2021-01-31 02:40:24 +03:00
Azat Khuzhin
89c4055202 Lock MEMORY_LIMIT_EXCEEDED error from tryLogCurrentException() 
This will avoid hiding some exceptions in logs, when the server is under
high memory pressure (i.e. when any new allocation will lead to
MEMORY_LIMIT_EXCEEDED error).

This became more relevent after all memory allocations was tracked with
MemoryTracker, by falling back to total_memory_tracking, in #16121
 2021-01-29 21:38:14 +03:00
Pavel Kruglov
25e85d71ee Merge branch 'master' of github.com:ClickHouse/ClickHouse into hedged-requests 2021-01-29 21:08:47 +03:00
Pavel Kruglov
01a0cb649a Fix build, style, tests 2021-01-29 18:46:28 +03:00
Alexander Tokmakov
004517009e fix 2021-01-29 18:13:27 +03:00
Anton Popov
031132038b fix filtering by uint8 greater than 127 2021-01-29 10:39:18 +03:00
Alexey Milovidov
1e2669fd3c Fix error 2021-01-29 07:54:46 +03:00
tavplubix
37455304e7
Update FieldVisitorsAccurateComparison.h 2021-01-28 18:51:01 +03:00
tavplubix
817df99d11
Update FieldVisitorsAccurateComparison.h 2021-01-28 18:29:21 +03:00
Alexander Tokmakov
ffaa8e34a6 minor code improvements around ThreadStatus 2021-01-28 16:57:36 +03:00
Alexander Tokmakov
52e5c0aad7 fix thread status 2021-01-28 16:48:17 +03:00
alesapin
c3db529543 Merge branch 'master' into in_memory_raft 2021-01-28 12:01:54 +03:00
Alexey Milovidov
355c99568e Fix error 2021-01-28 10:16:36 +03:00
Alexey Milovidov
95e15131a8 Fix unsufficient args check (trash code) in StringSearcher 2021-01-27 20:32:59 +03:00
alesapin
7af28e758a Process read requests without raft 2021-01-27 16:37:58 +03:00
alesapin
241d3ec8c2 Merge with master 2021-01-27 15:40:16 +03:00
alesapin
dd54e5afe8 Merge branch 'master' into in_memory_raft 2021-01-27 15:32:16 +03:00
alexey-milovidov
c3828a1fb2
Merge pull request #19660 from CurtizJ/sync-directory 
Better abstractions in disk interface
 2021-01-27 13:47:53 +03:00
Maksim Kita
bd81f43ecb
Merge pull request #19672 from ClickHouse/bigint-remove-old-code 
Remove old code for BigInt.
 2021-01-27 11:28:45 +03:00
Alexey Milovidov
093108bf81 Normalize BigInt implementation 2021-01-27 03:54:57 +03:00
Alexey Milovidov
d296424dfa Remove even more code 2021-01-26 22:39:03 +03:00
Alexey Milovidov
f3929b151a Remove even more code 2021-01-26 22:04:03 +03:00
Alexey Milovidov
5f13fdddc0 Remove useless code from BigInt 2021-01-26 21:22:40 +03:00
Anton Popov
c7070da85a better abstractions in disk interface 2021-01-26 17:49:35 +03:00
alesapin
0ff05d2403 Fix deadlock in testkeeper 2021-01-26 17:30:29 +03:00
alesapin
dea4b5009b Some server initialization 2021-01-25 15:29:12 +03:00
Alexander Tokmakov
3bd4d97353 Merge branch 'master' into database_replicated 2021-01-25 14:19:04 +03:00
alesapin
173b6fefdf Merge branch 'master' into in_memory_raft 2021-01-25 12:03:28 +03:00
alexey-milovidov
ba3e064a63
Merge pull request #19371 from kssenii/test-coverage-with-factories 
test coverage with factories
 2021-01-25 07:11:09 +03:00
alexey-milovidov
636d4c41bb
Merge pull request #19520 from amosbird/fixbuildid 
Support new .debug location
 2021-01-24 20:06:13 +03:00
Amos Bird
366c75ed29
Ensure buildid length 2021-01-24 10:28:43 +08:00
Amos Bird
554663a533
Support new .debug location 2021-01-24 03:03:54 +08:00
alexey-milovidov
9818bd319a
Merge pull request #19381 from azat/parts-types-metrics 
Add metrics for MergeTree parts (Wide/Compact/InMemory) types
 2021-01-22 17:24:50 +03:00
alesapin
4a06bf1d79 Merge branch 'fix_one_more_tiny_race' into in_memory_raft 2021-01-22 15:56:43 +03:00
Alexey Milovidov
dcd03eb155 Remove useless file 2021-01-22 00:57:00 +03:00
alesapin
5037d3befd
Merge pull request #19355 from ClickHouse/fix_race_test_keeper 
Fix race condition in TestKeeperHandler on session finish
 2021-01-21 23:53:18 +03:00
Azat Khuzhin
cb951c2116 Add metrics for MergeTree parts types 
- PartsWide
- PartsCompact
- PartsInMemory
 2021-01-21 21:17:00 +03:00
kssenii
c1702f34ee Add factories info into system.query_log 2021-01-21 15:46:37 +00:00
alexey-milovidov
062f00aa5d
Merge pull request #19290 from azat/dist-broken-on-EOF-fix 
Do not mark file for distributed send as broken on EOF
 2021-01-21 17:00:36 +03:00
alesapin
7f32926a21 Fix race condition in TestKeeperHandler on session finish 2021-01-21 14:37:20 +03:00
alesapin
f7175819d5 Add storage simpliest serialization 2021-01-21 14:07:55 +03:00
Ivan
0cbbb84f24
Add missing header 2021-01-21 02:20:11 +03:00