Commit Graph

5074 Commits

Author SHA1 Message Date
Robert Schulze
f2cad33f1b
Merge pull request #43123 from arenadata/ADQM-623
Add function 'age'
2022-12-05 15:53:28 +01:00
Robert Schulze
3c1a721da4
Merge branch 'master' into hlee-s390x-wide-int-order 2022-12-04 18:47:00 +01:00
Roman Vasin
bcb65eb420
Merge branch 'master' into ADQM-623 2022-12-04 20:21:27 +03:00
kssenii
dca3444283 Merge remote-tracking branch 'upstream/master' into named-collections-sql-commands 2022-12-04 15:56:02 +01:00
Kseniia Sumarokova
ec7649cd90
Merge pull request #43639 from CurtizJ/trace-profile-events
Allow to send profile events to `system.trace_log`
2022-12-04 15:50:07 +01:00
Vladimir C
7d6950d397
Revert "Temporary files evict fs cache" 2022-12-02 14:50:56 +01:00
Roman Vasin
59523864b9
Merge branch 'master' into ADQM-623 2022-12-02 15:00:09 +03:00
Suzy Wang
e34af8602c
Merge branch 'master' into s390x_crc32 2022-12-01 11:02:43 -05:00
Suzy Wang
be16e0bce7 Include crc32_s390x as a submodule library 2022-12-01 07:43:32 -08:00
Vladimir C
117d01f2ac
Merge pull request #43457 from ClickHouse/vdimir/tmp-data-in-fs-cache 2022-12-01 13:41:40 +01:00
Alexander Gololobov
6388bc5000
Followup fixes for XML helpers (#43845) 2022-12-01 13:36:40 +01:00
Roman Vasin
620df1ec6f
Merge branch 'master' into ADQM-623 2022-12-01 15:30:36 +03:00
vdimir
7a814d2854
wip: temporary files evict fs cache 2022-12-01 11:49:24 +00:00
vdimir
98fe3c6c02
Temporary files evict fs cache 2022-12-01 11:49:17 +00:00
Antonio Andelic
1f93183dc7
Merge pull request #43087 from ClickHouse/keeper-prometheus
Add support for embedded Prometheus endpoint in Keeper
2022-12-01 12:46:47 +01:00
zhanglistar
c3f6ba6b68 A little improve performance of PODArray 2022-12-01 19:26:58 +08:00
Nikolai Kochetov
7649a3c4d7 Merge branch 'master' into add-allocation-ptr-to-trace-log 2022-12-01 12:12:23 +01:00
alesapin
5b4a16cabb
Merge pull request #43835 from ClickHouse/increase-logging-level-for-keeper
Increase some logging level for keeper client.
2022-12-01 12:01:54 +01:00
Roman Vasin
96d7cf319b
Merge branch 'master' into ADQM-623 2022-11-30 19:25:53 +03:00
Nikolai Kochetov
ecc45e5b1f Review fixes. 2022-11-30 16:15:40 +00:00
Vladimir C
53dc70a2d0
Merge pull request #38191 from BigRedEye/grace_hash_join
Closes https://github.com/ClickHouse/ClickHouse/issues/11596
2022-11-30 17:01:00 +01:00
Vitaly Baranov
f966bf61e0
Merge pull request #43627 from ClickHouse/faster_backup_config_loading
Speedup backup config loading
2022-11-30 16:33:54 +01:00
Nikolai Kochetov
5a67ba8598 Increase some logging level for keeper client. 2022-11-30 15:21:14 +00:00
Antonio Andelic
2f501d5327 Merge branch 'master' into keeper-prometheus 2022-11-30 13:24:37 +01:00
Antonio Andelic
4a145b6c96
Merge pull request #43686 from ClickHouse/more-aggresive-batching-keeper
Try batching requests in Keeper more aggressively
2022-11-30 12:56:25 +01:00
Yakov Olkhovskiy
77266ea754 cleanup 2022-11-29 17:34:16 +00:00
Alexander Gololobov
6f6114993f Enable test with 1M entries after the fix 2022-11-29 15:43:00 +01:00
Alexander Gololobov
06d04a81f5
Merge branch 'master' into faster_backup_config_loading 2022-11-29 14:29:58 +01:00
Antonio Andelic
db46842c8a Merge branch 'master' into more-aggresive-batching-keeper 2022-11-29 12:58:33 +00:00
Nikolai Kochetov
51439e2c19
Merge pull request #43260 from ClickHouse/read-from-mt-in-io-pool
Read from MergeTree in I/O pool
2022-11-29 12:09:03 +01:00
Alexander Gololobov
0a5491c1aa
Merge branch 'master' into faster_backup_config_loading 2022-11-29 09:43:47 +01:00
Alexander Gololobov
a0bd99c8a2
Merge pull request #43584 from ClickHouse/fix_config_child_list_iteration
Iterate list without index-based access
2022-11-29 09:39:46 +01:00
Yakov Olkhovskiy
4d7d9cbd0b some comments and beautification 2022-11-29 01:07:25 +00:00
Alexander Gololobov
8f49c1ea16 Moved helpers to Common/XMLUtils.* 2022-11-28 22:14:18 +01:00
Yakov Olkhovskiy
770b520ded
Merge branch 'master' into refactoring-ip-types 2022-11-28 08:50:19 -05:00
Alexander Gololobov
351c9f7bd2 Descrease element count to 50K until XML key enumeration is fixed in Poco 2022-11-28 13:36:12 +01:00
Alexander Gololobov
d88e3dfe33 Check time elapsed by config loading and key enumeration 2022-11-28 13:24:25 +01:00
Antonio Andelic
2b6661f87c Use lock_guard 2022-11-28 09:17:33 +00:00
Antonio Andelic
678958603b
Merge branch 'master' into keeper-prometheus 2022-11-28 09:48:57 +01:00
Robert Schulze
143637404e
Merge pull request #43078 from ClibMouse/s390x_ip_coding
s390x ip coding functions fixes
2022-11-28 09:11:42 +01:00
Alexander Gololobov
603ccf147b Test ConfigProcessor on a config with many elements 2022-11-28 00:36:42 +01:00
Antonio Andelic
cade21b76e Fix 4LW test 2022-11-26 19:24:10 +00:00
Antonio Andelic
76eca8bdfb Merge branch 'master' into more-aggresive-batching-keeper 2022-11-26 18:28:11 +00:00
Antonio Andelic
cd471e9c61 Add setting for quick batch size 2022-11-26 17:33:40 +00:00
Yakov Olkhovskiy
0854825151 fix DefaultHash64 for IPv6 2022-11-26 17:21:06 +00:00
Suzy Wang
d57a8b502d s390x crc32 fix 2022-11-25 13:12:11 -08:00
Alexey Milovidov
f9c3de5aa5
Update ConfigProcessor.cpp 2022-11-26 00:09:33 +03:00
Alexey Milovidov
69d1be50bc
Merge pull request #43446 from ClickHouse/fix-typo-3
Fix typo
2022-11-25 15:22:28 +01:00
Nikolai Kochetov
660c942258 Merge branch 'master' into read-from-mt-in-io-pool 2022-11-25 13:56:48 +00:00
Anton Popov
396ecf826b fix tests 2022-11-25 13:05:38 +00:00
Alexander Gololobov
38ecdc0627 Save next pointer before doing changes to the current element 2022-11-25 13:42:43 +01:00
Anton Popov
312dd725b4 allow to send profile events to trace_log 2022-11-24 20:12:16 +00:00
Alexander Gololobov
70c821ea4b Fix iteration in other similar places 2022-11-24 19:08:45 +01:00
Ilya Yatsishin
6fcee9df85
Merge pull request #43496 from qoega/progress-add-writtten-rows 2022-11-24 17:31:02 +01:00
Nikolai Kochetov
d53439390f Review fixes. 2022-11-24 16:04:35 +00:00
Igor Nikonov
20e67b7140 Merge remote-tracking branch 'origin/master' into HEAD 2022-11-24 13:10:37 +00:00
Vladimir C
a2e1269ef2
Revert "Fix endian issue in integer hex string conversion" 2022-11-24 12:22:22 +01:00
Antonio Andelic
a6f38cb1cc
Merge branch 'master' into keeper-prometheus 2022-11-24 12:12:08 +01:00
Vladimir C
b784128c88
Merge pull request #41245 from HarryLeeIBM/hlee-s390x-lru-file-cache 2022-11-24 12:08:36 +01:00
Sergei Trifonov
8186a06c28
Merge branch 'master' into io-scheduler 2022-11-24 12:02:52 +01:00
Raúl Marín
4d3217cb18
Merge branch 'master' into perf_experiment 2022-11-24 10:32:12 +01:00
Alexander Gololobov
85b1fb96a5 Iterate list without index-based access 2022-11-23 17:49:17 +01:00
Nikolai Kochetov
f7d02fbe75 Fixing special build. 2022-11-23 14:05:26 +00:00
kssenii
6044a9257c Merge remote-tracking branch 'upstream/master' into named-collections-sql-commands 2022-11-23 12:00:55 +01:00
Alexey Milovidov
8cb34913a1
Merge pull request #42461 from Enmk/ldap_fix_search_limit
fix LDAP in case of many roles on user
2022-11-22 20:05:14 +01:00
Alexey Milovidov
c23b8c69db
Merge pull request #42791 from azat/fix-pe-overflow
Fix incorrect UserTimeMicroseconds/SystemTimeMicroseconds accounting
2022-11-22 19:22:51 +01:00
Raúl Marín
4aa29b6a63 Merge remote-tracking branch 'blessed/master' into perf_experiment 2022-11-22 19:09:00 +01:00
Alexey Milovidov
b7e4547f53
Merge branch 'master' into fix-typo-3 2022-11-22 18:30:06 +01:00
Alexey Milovidov
c8b068b4ff
Merge pull request #43447 from ClickHouse/netlink-remove-noise
Remove noise from logs about NetLink in Docker
2022-11-22 18:29:43 +01:00
Yatsishin Ilya
37bebcbe20 Add information about written rows in progress indicator 2022-11-22 14:29:22 +00:00
Sergei Trifonov
59f422f10c
Merge branch 'master' into storage-system-moves 2022-11-22 14:46:00 +01:00
Sergei Trifonov
3177ed050f
Merge pull request #43307 from ClickHouse/cpu-progress-protocol-fix
fix CPU load indication on client
2022-11-22 14:09:55 +01:00
Dmitry Novik
e67bda65a0
Merge pull request #43453 from ClickHouse/fix-hard-limit-in-test
Do not reuse jemalloc memory in test_global_overcommit
2022-11-22 13:24:34 +01:00
Nikolai Kochetov
9e84a351c7 Fixing special build. Disable setting by default. 2022-11-22 11:24:24 +00:00
Raúl Marín
8776b2f566 Merge remote-tracking branch 'blessed/master' into perf_experiment 2022-11-22 11:45:36 +01:00
Vitaly Baranov
6c2b53f0ae
Merge pull request #42940 from filimonov/exceptions_with_masker
Add SensitiveDataMasker to exceptions messages
2022-11-21 23:20:15 +01:00
kssenii
155ddd5521 Fix review comments 2022-11-21 21:50:24 +01:00
kssenii
a279bac767 Merge remote-tracking branch 'upstream/master' into named-collections-sql-commands 2022-11-21 21:49:34 +01:00
Dmitry Novik
b17dc24a94 Do not reuse jemalloc memory in test 2022-11-21 16:10:47 +00:00
Alexey Milovidov
4ab6896ece Remove noise from logs about NetLink in Docker 2022-11-21 15:16:27 +01:00
Alexey Milovidov
56f8c0b84b Fix typo 2022-11-21 15:10:15 +01:00
Raúl Marín
ed0c174c0c Merge remote-tracking branch 'blessed/master' into perf_experiment 2022-11-21 11:02:31 +01:00
Antonio Andelic
e78761b66c
Merge branch 'master' into keeper-prometheus 2022-11-21 09:10:10 +01:00
Yakov Olkhovskiy
ee3ce9de70 bugs fixed, some optimization, cleanup, test fixed 2022-11-21 07:45:42 +00:00
Nikolai Kochetov
5da1d893fd
Merge branch 'master' into read-from-mt-in-io-pool 2022-11-18 21:10:45 +01:00
Azat Khuzhin
ccde92f855 Fix incorrect UserTimeMicroseconds/SystemTimeMicroseconds accounting
After #40732 it became possible that getrusage() (from detachQuery(),
from buildPushingToViewsChain()) will be called for incorrect thread,
and so when the difference will be calculated it will be simply garbage.

But actually the root of this problem is #25714, after which it became
possible to have multiple ThreadStatus for one thread, and this is very
tricky (sigh).

Here are some other thoughts about it:
- Make ThreadStatus nested - decided that complexity does not worth it,
  at least only for this case
- Move some members into ThreadGroupStatus - will break per-thread
  statistics (and hence query_thread_log, BTW does somebody uses it?)
- Move some members into a separate structure

But decided to fix the issue w/o any refactoring, to make easy for
backport.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-18 21:10:43 +01:00
Azat Khuzhin
2b936a59e4 Add sanity checks RUsageCounters::incrementProfileEvents()
Can be triggered with the 02380_insert_mv_race test now in
debug/sanitizers build.
2022-11-18 21:10:43 +01:00
Yakov Olkhovskiy
dbaeabcf38 fixed some bugs, some functions corrected, some tests corrected 2022-11-18 20:10:27 +00:00
Sergei Trifonov
77100896e3
Merge branch 'master' into storage-system-moves 2022-11-18 16:33:43 +01:00
Sergei Trifonov
35d37bc198
Merge branch 'master' into cpu-progress-protocol-fix 2022-11-18 16:30:40 +01:00
Sergei Trifonov
d05223e70b
Merge pull request #43335 from ClickHouse/revert-43306-revert-43014-disk-s3-throttler
Revert "Revert "S3 request per second rate throttling""
2022-11-18 16:22:30 +01:00
Vitaly Baranov
a348332eab
Merge pull request #43227 from vitlibar/improve-masking-sensitive-info
Improve masking sensitive info
2022-11-18 15:37:50 +01:00
Suzy Wang
8dc99e8d64
Merge branch 'master' into s390x_ip_coding 2022-11-17 23:35:06 -05:00
Alexey Milovidov
2c292e3866
Merge branch 'master' into ldap_fix_search_limit 2022-11-17 22:20:53 +01:00
Sergei Trifonov
f2f0676bcc
Revert "Revert "S3 request per second rate throttling"" 2022-11-17 17:35:04 +01:00
serxa
1920c6267d typo 2022-11-17 16:20:21 +00:00
serxa
c0d98aa749 do not reorder, ignore 4 readings, fix period 2022-11-17 15:53:47 +00:00
Nikita Taranov
7beb58b0cf
Optimize merge of uniqExact without_key (#43072)
* impl for uniqExact

* rm unused (read|write)Text methods

* fix style

* small fixes

* impl for variadic uniqExact

* refactor

* fix style

* more agressive inlining

* disable if max_threads=1

* small improvements

* review fixes

* Revert "rm unused (read|write)Text methods"

This reverts commit a7e7480584.

* encapsulate is_able_to_parallelize_merge in Data

* encapsulate is_exact & argument_is_tuple in Data
2022-11-17 13:19:02 +01:00
Vladimir C
a29c5b9e3a
Merge pull request #43232 from ClickHouse/vdimir/max_map_null_totals_bug 2022-11-17 13:10:05 +01:00
Sema Checherinda
1ad48d091d
Merge pull request #43249 from Algunenano/i43247
Fix ubsan in AggregateFunctionMinMaxAny::read with high sizes
2022-11-17 12:21:05 +01:00
Raúl Marín
97d6fc3071 Merge remote-tracking branch 'blessed/master' into perf_experiment 2022-11-17 11:48:46 +01:00
Alexander Tokmakov
9011a18234
Revert "S3 request per second rate throttling" 2022-11-16 22:33:48 +03:00
Sergei Trifonov
159743edd6
Merge pull request #43014 from ClickHouse/disk-s3-throttler
S3 request per second rate throttling
2022-11-16 18:51:06 +01:00
Alexey Milovidov
923f2dd562
Merge branch 'master' into ldap_fix_search_limit 2022-11-16 06:13:07 +01:00
Alexey Milovidov
b4165f95a9
Merge branch 'master' into build/poco-leftovers 2022-11-16 06:09:58 +01:00
Alexey Milovidov
04687c0d69
Merge pull request #43206 from ClickHouse/miscellaneous-changes-parallel-replicas
Miscellaneous changes
2022-11-16 05:30:03 +01:00
Vitaly Baranov
ce81166c7e Fix style. 2022-11-16 01:35:11 +01:00
Vitaly Baranov
8d72f75556 Make the password wiping an option of IAST::format(). 2022-11-16 01:35:06 +01:00
Yakov Olkhovskiy
813cb7fb0d merge master 2022-11-15 22:46:05 +00:00
Yakov Olkhovskiy
273abdb5b0 bug fixes, optimizations 2022-11-15 21:49:35 +00:00
Suzy Wang
97d76771a4 update format 2022-11-15 08:53:39 -08:00
kssenii
234f6ee54d Add commands CREATE/ALTER/DROP NAMED COLLECTION 2022-11-15 15:51:56 +01:00
Raúl Marín
143b67d0be Fix ubsan in AggregateFunctionMinMaxAny::read with high sizes 2022-11-15 15:40:06 +01:00
Antonio Andelic
96838f5be7 Merge branch 'master' into keeper-prometheus 2022-11-15 12:00:02 +00:00
Kseniia Sumarokova
59cf5def67
Merge branch 'master' into disk-s3-throttler 2022-11-15 12:13:37 +01:00
Vladimir C
0017416069
Update src/Common/FieldVisitorSum.cpp 2022-11-15 11:55:54 +01:00
filimonov
8369998286
Style 2022-11-14 22:06:22 +01:00
vdimir
28fec44ffc
Fix merging null values in AggregateFunctionSumMap 2022-11-14 20:56:36 +00:00
HarryLeeIBM
0133444433 Fix byte order issue of wide integer for s390x 2022-11-14 10:47:32 -08:00
Mikhail Filimonov
19d39d7881 Add SensitiveDataMasker to exceptions messages 2022-11-14 19:06:17 +01:00
Vitaly Baranov
8e99f5fea3 Move maskSensitiveInfoInQueryForLogging() to src/Parsers/ 2022-11-14 18:55:19 +01:00
Yakov Olkhovskiy
9aeebf3bdf
Merge branch 'master' into refactoring-ip-types 2022-11-14 09:21:54 -05:00
Yakov Olkhovskiy
4d144be39c replace domain IP types (IPv4, IPv6) with native 2022-11-14 14:17:17 +00:00
Raúl Marín
4988e387f2 Merge remote-tracking branch 'blessed/master' into over-read 2022-11-14 12:28:31 +01:00
Kseniia Sumarokova
e58a7140be
Merge pull request #43194 from ClickHouse/async-metrics-documentation
Add documentation for `system.asynchronous_metrics`.
2022-11-14 12:21:34 +01:00
Kseniia Sumarokova
79206c6eb0
Merge pull request #43147 from kssenii/named-collections-refactoring
Add generic implementation for arbitrary structured named collections, access type and system.named_collections
2022-11-14 11:57:17 +01:00
Roman Vasin
ee0baa679b Merge branch 'master' of github.com:ClickHouse/ClickHouse into ADQM-623 2022-11-14 09:45:32 +00:00
Antonio Andelic
ee182776da
Merge branch 'master' into keeper-prometheus 2022-11-14 09:14:13 +01:00
Alexey Milovidov
8e87e141a4 Miscellaneous changes 2022-11-14 06:57:53 +01:00
Alexey Milovidov
a77032fafa Miscellaneous changes 2022-11-14 06:54:58 +01:00
Alexey Milovidov
b8c877db29 Fix annoying messages in AsynchronousMetrics 2022-11-12 19:07:28 +01:00
Alexey Milovidov
127631ee47
Merge branch 'master' into perf_experiment 2022-11-12 18:58:25 +01:00
Nikolai Kochetov
9106910de8 Try read from MergeTree in background pool. 2022-11-11 20:47:11 +00:00
Igor Nikonov
ee5756b5b4
Merge pull request #43122 from ClickHouse/revert-43116-revert-42607-igor/insert_zk_retries_retry
Revert "Revert " Keeper retries during insert (clean)""
2022-11-11 18:22:38 +01:00
kssenii
616da1f695 Review fixed, style check fix 2022-11-11 16:50:02 +01:00
Antonio Andelic
cbe92bf8c8
Merge branch 'master' into keeper-prometheus 2022-11-11 13:56:27 +01:00
Raúl Marín
54db7c6520 Enforce checking read output 2022-11-11 10:56:18 +01:00
Roman Vasin
35a1c406ad Merge branch 'master' of github.com:ClickHouse/ClickHouse into ADQM-623 2022-11-11 07:35:44 +00:00
Azat Khuzhin
953457de28 Remove POCO_CLICKHOUSE_PATCH
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-10 22:41:26 +01:00
Maksim Kita
a6d2069c2b Added compound types typo corrections 2022-11-10 19:21:51 +01:00
Maksim Kita
e220906c9e Analyzer added identifier typo corrections 2022-11-10 19:21:51 +01:00
kssenii
c4f522b414 Support arbintrary structure collections 2022-11-10 16:32:15 +01:00
Igor Nikonov
f8c907c463
Revert "Revert " Keeper retries during insert (clean)"" 2022-11-10 13:14:04 +01:00
Antonio Andelic
94aab4276c Bring back heavy metrics 2022-11-10 12:56:27 +01:00
Alexander Tokmakov
fb8c527a2f
Revert " Keeper retries during insert (clean)" 2022-11-10 14:12:24 +03:00
Igor Nikonov
1b9be27fdb
Merge pull request #42607 from ClickHouse/igor/insert_zk_retries_retry
Keeper retries during insert (clean)
2022-11-10 12:01:00 +01:00
Roman Vasin
8f3a7b5024 Change algorithm into more accurate 2022-11-10 10:27:18 +00:00
Antonio Andelic
f2a9b5933d Merge branch 'master' into keeper-prometheus 2022-11-10 09:28:06 +01:00
Suzy Wang
6ae4843b6d Update fixes as suggested 2022-11-09 13:54:32 -08:00
Antonio Andelic
1ed3930809 Add more metrics for Keeper 2022-11-09 15:51:41 +01:00
Yakov Olkhovskiy
a930ad25e1
fix ub type punning 2022-11-09 09:44:59 -05:00
Yakov Olkhovskiy
7f28a89273
fix ub type punning 2022-11-09 09:39:27 -05:00
kssenii
e94e087187 Add generic named collections class and factory 2022-11-09 14:08:16 +01:00
Igor Nikonov
379113f8f1 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-11-09 13:03:32 +00:00
Antonio Andelic
c92ae5a385 Add suport for AsyncMetrics in standalone Keeper 2022-11-09 13:46:03 +01:00
Alexander Tokmakov
1c6a617207
Merge branch 'master' into fix_intersecting_parts2 2022-11-09 14:38:18 +03:00
Suzy Wang
90250ea6ec s390x ip coding functions fixes 2022-11-08 21:12:47 -08:00
Igor Nikonov
345304abe0 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-11-08 20:26:50 +00:00
Sergei Trifonov
8eedd1e046
Merge branch 'master' into disk-s3-throttler 2022-11-08 15:00:56 +01:00
Nikolay Degterinsky
2b286222f7
Merge branch 'master' into tuple-of-intervals 2022-11-08 00:15:35 +01:00
Nikolay Degterinsky
48cc1d8492
Update DateLUTImpl.h 2022-11-08 00:15:19 +01:00
Sergei Trifonov
6d3b96ee42
Merge branch 'master' into storage-system-moves 2022-11-07 21:31:27 +01:00
Alexander Tokmakov
9210e586d2 fix 2022-11-07 20:27:18 +01:00
Alexander Tokmakov
b8174a63a8 Merge branch 'master' into fix_intersecting_parts2 2022-11-07 20:26:36 +01:00
serxa
2daec0b45e S3 request per second rate throttling + refactoring 2022-11-07 18:05:40 +00:00
Vasily Nemkov
68dce47e72
Merge branch 'master' into ldap_fix_search_limit 2022-11-07 12:14:10 +04:00
Igor Nikonov
503649a638
Merge branch 'master' into igor/insert_zk_retries_retry 2022-11-06 22:55:13 +01:00
Alexey Milovidov
641f8576c6
Merge pull request #42003 from ClickHouse/write-progress-to-tty
Write progress directly to terminal
2022-11-05 16:51:37 +01:00
Igor Nikonov
859022660e Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-11-04 23:31:51 +00:00
Krzysztof Góralski
98cc4535f0 Failed queries metrics for Async Inserts 2022-11-03 22:26:24 +01:00
Nikolay Degterinsky
30ad1a6826
Merge branch 'master' into perf_experiment 2022-11-03 02:18:21 +03:00
Alexander Tokmakov
e67d05604c Merge branch 'master' into fix_intersecting_parts2 2022-11-02 19:30:27 +01:00
Vasily Nemkov
2be86ef5a7
Merge branch 'master' into ldap_fix_search_limit 2022-11-02 20:46:33 +04:00
Igor Nikonov
9a315b7899 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-11-02 15:32:54 +00:00
Alexander Tokmakov
00c9e50ee3 fix race between INSERT and ALTER PARTITION 2022-11-02 16:25:26 +01:00
Yakov Olkhovskiy
16159cc7a9
Merge pull request #42412 from ClibMouse/formatIpv6_s390x_support
format ipv6 in s390x, endianess support
2022-11-01 12:02:44 -04:00
Alexey Milovidov
1b3ef58b51
Merge branch 'master' into formatIpv6_s390x_support 2022-10-31 23:20:53 +01:00
Igor Nikonov
9cc64a0fcc Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-10-31 18:21:21 +00:00
Márcio Martins
c33bd7794f
Merge branch 'master' into coercing-json-extract 2022-10-31 15:04:48 +01:00
Sergei Trifonov
8c67257a1c
Merge pull request #42665 from ClickHouse/token-bucket-throttler
Replace throttler algorithm by token bucket
2022-10-31 12:28:18 +01:00
Nikolai Kochetov
742d9af397
Merge pull request #42582 from azat/memory-tracker/fixes
Fix frequent memory drift message and clarify things in comments
2022-10-31 11:43:20 +01:00
vdimir
6a4247ca32
Merge branch 'master' into grace_hash_join 2022-10-31 09:54:37 +00:00
Vasily Nemkov
f805538105
Merge branch 'master' into ldap_fix_search_limit 2022-10-29 19:53:49 +04:00
Igor Nikonov
937dd44399 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-10-28 15:09:23 +00:00
Sergei Trifonov
c948303fa0
Merge branch 'master' into token-bucket-throttler 2022-10-28 16:21:55 +02:00
Raúl Marín
ad91b625f7
Merge branch 'master' into perf_experiment 2022-10-28 16:00:53 +02:00
Sergei Trifonov
836595c7ae
Merge branch 'master' into storage-system-moves 2022-10-28 11:39:38 +02:00
Sergei Trifonov
eda264b492
Merge branch 'master' into token-bucket-throttler 2022-10-28 11:38:13 +02:00
Suzy Wang
6da80f4dcb
Merge branch 'master' into formatIpv6_s390x_support 2022-10-28 00:32:37 -04:00
Raúl Marín
59686c3796
Merge branch 'master' into perf_experiment 2022-10-28 00:51:56 +02:00
Igor Nikonov
2b56bc4394 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-10-27 22:02:17 +00:00
Igor Nikonov
3682913172 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-10-27 22:01:53 +00:00
Igor Nikonov
a4ce0e344a Fix build
src/Common/CaresPTRResolver.cpp:126:27
error: implicit conversion loses integer precision: 'int64_t' (aka
'long') to 'int' [-Werror,-Wshorten-64-to-32]
2022-10-27 21:11:41 +00:00
Sergei Trifonov
8892da3a4c
Merge branch 'master' into token-bucket-throttler 2022-10-27 22:40:59 +02:00
Jordi Villar
073d81c31f Use nfds_t instead 2022-10-27 20:36:42 +02:00
serxa
c4f25228f7 smaller burst to be closer to old behaviour 2022-10-27 17:18:50 +00:00
Jordi Villar
1fa24161e7 Minor fix implicit cast CaresPTRResolver 2022-10-27 18:34:37 +02:00
Alexander Tokmakov
86b81a2b6f
Merge pull request #42234 from arthurpassos/fix_ares_fd_process_race_condition
Fix c-ares crash
2022-10-27 17:17:22 +03:00
Raúl Marín
891484b462 Merge remote-tracking branch 'blessed/master' into perf_experiment 2022-10-27 13:17:07 +02:00
serxa
accf78f1ff fix the first add() call 2022-10-27 08:39:08 +00:00
Sergei Trifonov
ec1389cbe7
Update src/Common/Throttler.cpp
Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>
2022-10-27 10:30:00 +02:00
Márcio Martins
1ca230ff6d Allow coercing in JSONExtract* functions 2022-10-27 10:26:55 +02:00
Márcio Martins
40b572c23b Remove redundant default case in switch statement 2022-10-27 10:26:55 +02:00
Márcio Martins
0fc0e81c0c Add Element::type() to JSONParser 2022-10-27 10:26:55 +02:00
Suzy Wang
fd1281a34d
Merge branch 'master' into formatIpv6_s390x_support 2022-10-26 19:04:55 -04:00
Igor Nikonov
8435cba352 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-10-26 17:35:47 +00:00
Azat Khuzhin
7b69a70e82 Fix frequent memory drift message and clarify things in comments
Somethine like:

    2022.09.28 06:33:34.001433 [ 3133669 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 562.20 MiB, peak 562.21 MiB, will set to 562.20 MiB (RSS), difference: -70.46 MiB
    2022.09.28 06:33:35.001639 [ 3133669 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 562.20 MiB, peak 562.21 MiB, will set to 562.20 MiB (RSS), difference: -70.45 MiB

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-26 17:38:37 +02:00
Azat Khuzhin
f5f6f1b593 Add profile events for jemalloc purge
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-26 17:38:36 +02:00
vdimir
506bf2d225
Merge branch 'master' into grace_hash_join 2022-10-26 12:25:50 +00:00
SmitaRKulkarni
5df138da82
Merge pull request #42376 from ClickHouse/36316_Support_glob_for_recursive_directory_traversal
Added ** glob support for recursive directory traversal to filesystem…
2022-10-26 11:49:20 +02:00
serxa
471b391ab2 fix 2022-10-26 00:00:40 +00:00
serxa
27599ab70c fix comment 2022-10-25 23:58:25 +00:00
serxa
fc223aa673 replace throttler algorithm by token bucket 2022-10-25 23:52:31 +00:00
Sergei Trifonov
34b43f8b4b
Merge branch 'master' into storage-system-moves 2022-10-25 22:56:45 +02:00
serxa
55a86bc2de Add active moves system table 2022-10-25 13:56:46 +00:00
Raúl Marín
6e0a9452e7 Merge remote-tracking branch 'blessed/master' into perf_experiment 2022-10-25 15:25:06 +02:00
Nikita Taranov
49f6692a2e
Adapt internal data structures to 512-bit era (#42564)
* impl

* update tests

* fix tests
2022-10-25 13:56:28 +02:00
Raúl Marín
0016bc2b73
Improve the time to recover keeper connections (#42541)
Co-authored-by: Nikita Mikhaylov <nikitamikhaylov@clickhouse.com>
Co-authored-by: robot-clickhouse <robot-clickhouse@clickhouse.com>
Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>
2022-10-25 13:26:29 +02:00
Alexey Milovidov
079ffa9827
Merge pull request #42353 from azat/build/musl-static
Make getResource() independent from the order of the sections
2022-10-25 05:57:22 +02:00
Guo Wangyang
4e294b9bfc
ColumnVector: re-enable AVX512_VBMI/AVX512_VBMI2 optimized filter and index (#41765)
* Revert "Merge pull request #41752 from ClickHouse/revert-revert-revert"

This reverts commit 7bb245720a, reversing
changes made to 78002dc248.

* Revert "Merge pull request #41797 from ClickHouse/revert-41247"

This reverts commit 69b13d9f31, reversing
changes made to 1719e000ea.

* AVX512VBMI::vectorIndexImpl: Fix heap buffer overflow when limit == 0

* use SIMD_ELEMENTS instead of SIMD_BYTES to avoid confusing
2022-10-24 23:11:46 +02:00
SmitaRKulkarni
96c8260230
Merge branch 'master' into 36316_Support_glob_for_recursive_directory_traversal 2022-10-24 18:34:19 +02:00
Alexey Milovidov
9bce072ec1
Merge branch 'master' into ldap_fix_search_limit 2022-10-24 17:19:05 +02:00
Nikolai Kochetov
aee7bb0d91
Merge pull request #42577 from ClickHouse/clean-thread-id
Clear thread::id when ThreadFromGlobalPool exits.
2022-10-24 12:03:28 +02:00
Alexey Milovidov
98f5f27947
Merge branch 'master' into ldap_fix_search_limit 2022-10-24 10:44:45 +02:00
Igor Nikonov
54013009d3 Code migration from #41734 to align diff and git blame.
Initial implementation was different and it changed the entire
ReplicatedMergeTreeSink::commitPart() which change history provided by git blame.
Then RetriesControl.retryLoop() was introduced later which significantly reduces
the diff since it's like while() used before.
So, check outing the current version will keep more original history in
git blame, which is useful here
2022-10-24 08:20:11 +00:00
Azat Khuzhin
27494aea63 Make getResource() independent from the order of the sections
It is possible for end section goes before begin section for some
resource, and this case it will not find it.

This is what happens here [1]:

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/41988/053fec3f451e97ac41b6c223d95013b758a9a330/fast_test.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-24 07:10:45 +02:00
Alexey Milovidov
409fb28d0c Fix build 2022-10-23 01:52:30 +02:00
Alexey Milovidov
cd43595ea9 Merge branch 'master' into write-progress-to-tty 2022-10-23 01:41:17 +02:00
Azat Khuzhin
56bc85746f Merge remote-tracking branch 'upstream/master' into build/shorten-64-to-32
Conflicts:
- src/Interpreters/ProcessList.cpp
2022-10-22 16:49:08 +02:00
Nikolai Kochetov
06d46ca51b Clear thread::id when ThreadFromGlobalPool exits. 2022-10-22 13:28:22 +00:00
Nikolai Kochetov
5c32cd157d
Merge pull request #42362 from ClickHouse/fix-race-condition-finish-cancel
Fix data race in query finish/cancel
2022-10-22 14:58:42 +02:00
Azat Khuzhin
4eeb7337ed Merge remote-tracking branch 'upstream/master' into build/shorten-64-to-32
Conflicts:
- src/Common/intExp.h
- src/Functions/DateTimeTransforms.h
- src/Functions/FunctionsConversion.h
2022-10-21 22:51:56 +02:00
Azat Khuzhin
bbcdedb945 Revert changes of hash functions signatures and fix callers
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 22:40:17 +02:00
Azat Khuzhin
905a95e166 Review fixes
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 22:40:13 +02:00
Alexey Milovidov
3beace607b
Merge pull request #42465 from ClickHouse/decimal-asan
Fix buffer overflow in Decimal scale
2022-10-21 21:55:20 +02:00
Arthur Passos
615b73773a removing unwanted files 2022-10-21 14:46:10 -03:00
Arthur Passos
e2fcd51c93 what 2022-10-21 14:43:08 -03:00
Arthur Passos
2c41c0357d Make CaresPTRResolver completely thread-safe 2022-10-21 14:32:48 -03:00
Alexander Tokmakov
7d11054134
Merge branch 'master' into fix_ares_fd_process_race_condition 2022-10-21 18:27:22 +03:00
Azat Khuzhin
784f7d988c Fix -Wshorten-64-to-32 in examples
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:43 +02:00
Azat Khuzhin
089bf39132 Fix -Wshorten-64-to-32 for musl
On musl uint_fast32_t is uint32_t, while for glibc it is unsigned long
long.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:42 +02:00
Azat Khuzhin
4e76629aaf Fixes for -Wshorten-64-to-32
- lots of static_cast
- add safe_cast
- types adjustments
  - config
  - IStorage::read/watch
  - ...
- some TODO's (to convert types in future)

P.S. That was quite a journey...

v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:19 +02:00
Robert Schulze
2d448c139c
Merge pull request #42308 from arenadata/ADQM-584
Add support of Date32 arguments for date_diff function
2022-10-21 13:18:44 +02:00
Nikita Taranov
abd585a38a
Increase threshold for using physical cores (#42503) 2022-10-21 12:52:46 +02:00
Nikolai Kochetov
d09a5e8fd7 Revert "Attempt to fix abort from parallel parsing (#42496)"
This reverts commit 4d703b792c.
2022-10-20 21:46:26 +02:00
Arthur Passos
50f9acb221
Merge branch 'master' into fix_ares_fd_process_race_condition 2022-10-20 13:35:58 -03:00
Nikita Mikhaylov
4d703b792c
Attempt to fix abort from parallel parsing (#42496) 2022-10-20 17:13:18 +02:00
Vasily Nemkov
05cde705b0 Addressed issues mentioned in PR
Stronger hashing for LDAP parameters
Fixed SipHash double finalize()
Simplified checks on search_limit
2022-10-20 16:38:41 +04:00
Vasily Nemkov
1ed7ad57d9 fix LDAP in case of many roles on user
- Raised default value of search_limit to 256
- Added option to change that to arbitrary value
- using SipHash for computing hash of LDAP server parameters
- other minor changes
2022-10-20 16:38:38 +04:00
Roman Vasin
4db28d0bd6 Add toStableRelativeHourNum to gtest_DateLUTImpl.cpp 2022-10-19 14:01:26 +00:00
Alexey Milovidov
148275ed64 Fix buffer overflow in Decimal scale 2022-10-19 04:15:26 +02:00
Roman Vasin
d9fcd91875 Add toStableRelativeHourNum 2022-10-18 09:07:12 +00:00
Suzy Wang
c8642514d0 format ipv6 in s390x, endianess support 2022-10-17 12:02:28 -07:00
vdimir
adb63a5583
Merge branch 'master' into grace_hash_join 2022-10-17 12:32:56 +00:00
Raúl Marín
be932f49d2 Merge remote-tracking branch 'blessed/master' into perf_experiment 2022-10-17 12:07:31 +02:00
Smita Kulkarni
91433e5b9c Added ** glob support for recursive directory traversal to filesystem and S3.
Implementation:
* Updated parseGlob to not add ‘/‘ restriction when ** is used.
* Updated S3 & filesystem to fetch files and not use regex match if glob is **.
Testing:
* Added a test for filesystem tests/queries/0_stateless/02459_glob_for_recursive_directory_traversal.sh
2022-10-17 09:04:25 +02:00
Alexey Milovidov
f88ed8195b Fix trash 2022-10-17 04:21:08 +02:00
Vitaly Baranov
91c438eebe
Merge pull request #42263 from vitlibar/refactor-udf
Refactor the implementation of user-defined functions
2022-10-15 12:33:10 +02:00
Arthur Passos
78202f9660
Merge branch 'master' into fix_ares_fd_process_race_condition 2022-10-14 12:56:39 -03:00
Raúl Marín
227af9a074 Optimizations around ThreadStatus 2022-10-13 21:49:21 +02:00
Arthur Passos
91560dd607 working on style 2022-10-12 21:28:32 -03:00
Arthur Passos
f588830e24 working on style 2022-10-12 20:35:17 -03:00
Arthur Passos
72ba210b24 working on style 2022-10-12 20:23:29 -03:00
Arthur Passos
d98eac11eb remove mutex declaration 2022-10-12 17:47:56 -03:00
Arthur Passos
9abf13bf78 Use poll instead of select for c-ares 2022-10-12 13:56:00 -03:00
Vitaly Baranov
9cb2052c7f Consider Poco::Net::HostNotFoundException as ZCONNECTIONLOSS. 2022-10-12 15:52:55 +02:00
vdimir
0178307c27 Followup for TemporaryDataOnDisk 2022-10-12 15:25:23 +02:00
Arthur Passos
6dae76ff88 add mutex include for release builds 2022-10-11 12:47:27 -03:00
Arthur Passos
0a26afd1c8 Add mutex around ares fd processing 2022-10-10 18:54:35 -03:00
Alexey Milovidov
a9cbdeec24
Merge pull request #42180 from ClickHouse/unique-thread-ids
Make thread ids in the process list and query_log unique to avoid waste
2022-10-08 20:06:02 +03:00
Alexey Milovidov
ec5a32f534 Make thread_ids unique 2022-10-08 02:05:36 +02:00
Alexander Tokmakov
4175f8cde6 abort instead of __builtin_unreachable in debug builds 2022-10-07 21:49:08 +02:00
vdimir
5a7c493559
Merge branch 'master' into pr/BigRedEye/38191 2022-10-06 14:43:35 +00:00
Robert Schulze
78be400ac0
Merge remote-tracking branch 'origin/master' into generated-file-cleanup 2022-10-06 12:22:43 +00:00
Anton Popov
aabec50988
Merge pull request #42040 from CurtizJ/async-inserts-log
Add system table `asynchronous_insert_log`
2022-10-06 13:14:41 +02:00
Robert Schulze
da5a2e2db0
Merge remote-tracking branch 'origin/master' into generated-file-cleanup
Physical merge conflicts:
- src/Common/ZooKeeper/ZooKeeperImpl.cpp
- src/Core/config_core.h.in
- src/Functions/FunctionsAES.h
- src/Functions/config_functions.h.in
- src/configure_config.cmake

Logical merge conflicts:
- Functions/tryDecrypt.cpp
2022-10-06 08:43:25 +00:00
vdimir
ff55c369bc
Merge branch 'tmp-data-followup' 2022-10-05 18:10:05 +00:00
vdimir
ccd29e172f
Followup for TemporaryDataOnDisk 2022-10-05 16:42:27 +00:00
Roman Vasin
b54fb83cc5
Merge branch 'master' into ADQM-599-B 2022-10-05 15:50:38 +03:00
Alexey Milovidov
eef40ce884 Some attempt 2022-10-05 02:29:52 +02:00
Anton Popov
121ef74b17 slightly better 2022-10-04 14:19:05 +00:00
Roman Vasin
4a5c3e5a6e Fix toLastDayOfMonth and other functions 2022-10-04 09:09:59 +00:00
Anton Popov
148995894f add system table asynchronous_inserts_log 2022-10-03 21:35:55 +00:00
Alexander Tokmakov
00914a174d
Merge pull request #41410 from ClickHouse/keeper-multiread
Add MultiRead support in Keeper and internal ZK client
2022-10-03 14:17:57 +03:00
Robert Schulze
db5ef7b3cb
Merge branch 'master' into generated-file-cleanup 2022-10-02 23:13:18 +02:00
Robert Schulze
8e727d4fbc
Merge pull request #41910 from arenadata/ADQM-583
Improve enable_extended_results_for_datetime_functions option to return results of type DateTime64
2022-10-02 20:46:51 +02:00
Alexey Milovidov
90b9afab21 Remove useless code 2022-10-02 01:50:36 +02:00
Alexey Milovidov
5d710e21f1 Render progress directly in tty 2022-10-01 23:19:36 +02:00
vdimir
b120ad1452
Use TemporaryDataOnDisk for grace hash join 2022-09-30 11:07:49 +00:00
vdimir
7ebc297f4c
Merge branch 'master' into pr/BigRedEye/38191 2022-09-30 09:40:47 +00:00
Robert Schulze
cc92a2d174
Merge branch 'master' into generated-file-cleanup 2022-09-30 09:56:31 +02:00
Roman Vasin
45414b251d Merge branch 'master' of github.com:ClickHouse/ClickHouse into ADQM-583 2022-09-29 17:02:49 +00:00
Sergei Trifonov
61e12fc54a
Merge branch 'master' into io-scheduler 2022-09-29 13:48:14 +02:00
Antonio Andelic
94f1fe39bb Add support for exists in multiread 2022-09-29 10:39:33 +00:00
Robert Schulze
6381631836
Merge pull request #41824 from ClickHouse/tryBase58
Add function tryBase58Decode()
2022-09-29 12:38:58 +02:00
Antonio Andelic
0056eebb08 Cleanup 2022-09-29 10:10:49 +00:00
Antonio Andelic
fcc54104a4 Define methods for multi read requests 2022-09-29 09:19:47 +00:00
Antonio Andelic
bcefa6e62a Use chassert 2022-09-29 07:06:18 +00:00
Antonio Andelic
3109ce51c6 Merge branch 'master' into keeper-multiread 2022-09-29 06:50:49 +00:00
Sergei Trifonov
8edac7eba1
Merge branch 'master' into io-scheduler 2022-09-28 22:17:17 +02:00