Commit Graph

3717 Commits

Author SHA1 Message Date
Mikhail Artemenko
0f1119dd2a fix impl after merge 2022-04-07 11:56:30 +03:00
mergify[bot]
585cff1e95
Merge branch 'master' into limit_dns_retries 2022-04-06 18:23:17 +00:00
tavplubix
58db8cd9b3
Merge pull request #35946 from ClickHouse/faster_replicated_ddl
Make some replicated DDL queries faster
2022-04-06 19:25:25 +03:00
Mikhail Artemenko
151eeb1a27
Merge branch 'master' into MeiliSearch 2022-04-06 17:07:55 +03:00
Kseniia Sumarokova
a383d9ee6a
Merge pull request #35916 from filimonov/kafka_metrics
Add some metrics to engine Kafka
2022-04-06 11:58:17 +02:00
Raúl Marín
060f5118f0 Improvements based on PR review 2022-04-06 10:48:12 +02:00
Mikhail Filimonov
3412be9d4d
Change KafkaDirectReads to KafkaConsumersInUse 2022-04-05 20:25:05 +02:00
Raúl Marín
39cc3baf5f Keep the previous exception code 2022-04-05 16:37:53 +02:00
Antonio Andelic
d296eeee2d Small changes for Keeper 2022-04-05 13:56:28 +00:00
Alexander Tokmakov
d9e3e9b69f Merge branch 'master' into faster_replicated_ddl 2022-04-05 14:38:38 +02:00
Alexander Tokmakov
1fe50ad201 Merge branch 'master' into mvcc_prototype 2022-04-05 14:38:02 +02:00
Maksim Kita
c3c284e6e6
Merge pull request #35917 from kitaisreal/executable-user-defined-function-fix
Executable user defined functions prevent executing during analysis
2022-04-05 14:05:52 +02:00
Raúl Marín
8a05cf3927 DNS cache: Add option to drop elements after several consecutive failures 2022-04-05 13:00:27 +02:00
Maksim Kita
6e1d837439 Fixed style check 2022-04-05 11:24:33 +02:00
Alexander Tokmakov
da00beaf7f Merge branch 'master' into mvcc_prototype 2022-04-05 11:14:42 +02:00
Maksim Kita
b160ffd726
Merge pull request #35723 from ClickHouse/array-has-all-sse-avx2-optimizations
Merging #27653
2022-04-05 11:09:14 +02:00
Alexander Tokmakov
4e9ec5dc2f make some replicated DDL faster 2022-04-05 00:51:48 +02:00
larryluogit
808d9afd0f
Fix optin.cplusplus.UninitializedObject issue (#35626)
* Fix optin.cplusplus.UninitializedObject issue

* Enable optin.cplusplus.UninitializedObject
2022-04-04 23:47:14 +03:00
Mikhail Filimonov
53c7376e37
Add some metrics to engine Kafka 2022-04-04 20:25:53 +02:00
Maksim Kita
2a8e479277 ExecutableUserDefinedFunction prevent function execution during query analysis 2022-04-04 15:56:01 +02:00
Alexander Tokmakov
a2167f12b8 Merge branch 'master' into mvcc_prototype 2022-04-04 14:24:23 +02:00
Maksim Kita
47528de78b Fix build 2022-04-04 14:07:05 +02:00
Azat Khuzhin
14538f6456 Add system.processors_profile_log
This is the system table that will contain Processors level profiling.

v2: one entry per Processor, not 3 (PortFull/NeedData/work())
v3: us over ms
v4: Enable processors_profile_log table by default

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-04-04 13:19:41 +03:00
kssenii
36c583d0de Better version of cache on insert 2022-04-04 10:58:59 +02:00
Antonio Andelic
a926bc19ea Address PR comments 2022-04-04 07:24:42 +00:00
Antonio Andelic
d96b682a55 Refactor 2022-04-04 07:24:42 +00:00
taiyang-li
d6247338de fix failed stateless tests 2022-04-04 07:24:42 +00:00
taiyang-li
eda299b48b fix building 2022-04-04 07:24:42 +00:00
taiyang-li
4547ed370a add hints for column description 2022-04-04 07:24:42 +00:00
Alexey Milovidov
d9e5ca2119
Merge pull request #34394 from holadepo/last_day
Add toLastDayOfMonth function
2022-04-04 07:02:08 +03:00
Alexey Milovidov
5a47958744
Merge pull request #35736 from CurtizJ/quota-written-bytes
Add quota for written bytes
2022-04-03 05:26:49 +03:00
Anton Popov
687942ce70 more strict quota for written bytes 2022-04-01 15:02:49 +00:00
kssenii
6c8e073a61 Merge master 2022-04-01 16:48:29 +02:00
tcoyvwac
cb698c4cc1
Replace throw in debug mode in destructor
FileSegmentsHolder:
- Replaced throw in destructor with assert for all #ifdef modes.
2022-04-01 15:42:34 +02:00
Alexander Tokmakov
6591d1ceb7 Merge branch 'master' into mvcc_prototype 2022-04-01 15:38:46 +02:00
alesapin
f2c6387a8d
Merge pull request #35381 from kssenii/remote-filesystem-cache-fixes
Cache fixes for high concurrency on corner cases
2022-04-01 10:48:31 +02:00
filimonov
24d72a18d0
Update getNumberOfPhysicalCPUCores.cpp
In Kubernetes shares are calculated from resources.requests.cpu, quota & period - from limits.

When you configure only requests without limits - the container still can use all the cores. So you see all the cores the system has.
(because of requests the pod just can't be scheduled on the node which have less cpus)
So the effective CPU count is all the available cores if you don't have limits, or calculated from quota & period if you have the limits.
2022-03-31 17:21:54 +02:00
mergify[bot]
3183b61c74
Merge branch 'master' into mvcc_prototype 2022-03-31 12:27:09 +00:00
Alexander Tokmakov
5a50ad9de3 Merge branch 'master' into mvcc_prototype 2022-03-31 11:35:04 +02:00
alesapin
2365bbf0d3
Merge pull request #34566 from ClickHouse/startup-sanity-checks
add simple sanity checks for server startup
2022-03-31 11:33:59 +02:00
alesapin
81ce991494 Merge branch 'master' into startup-sanity-checks 2022-03-31 00:08:43 +02:00
Nikita Taranov
30f2a942c5
Predict size of hash table for GROUP BY (#33439)
* use AggregationMethod ctor with reserve

* add new settings

* add HashTablesStatistics

* support queries with limit

* support distributed and with external aggregation

* add new profile events

* add some tests

* add perf test

* export cache stats through AsynchronousMetrics

* rm redundant trace

* fix style

* fix 02122_parallel_formatting test

* review fixes

* fix 02122_parallel_formatting test

* apply also to two-level HTs

* try simpler strategy

* increase max_size_to_preallocate_for_aggregation for experiment

* fixes

* Revert "increase max_size_to_preallocate_for_aggregation for experiment"

This reverts commit 6cf6f75704.

* fix test

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-03-30 22:47:51 +02:00
kssenii
0fc92fe2aa Get rid of all "remote" mentions 2022-03-30 17:15:29 +02:00
kssenii
b30948dd53 Relax test 2022-03-30 17:12:23 +02:00
Vladimir C
31c367d3cd
Merge pull request #35651 from amosbird/columntransformerrename 2022-03-30 12:37:30 +02:00
Antonio Andelic
03a09dc914
Merge pull request #35571 from ClickHouse/parallel-s3-downloading
Parallel S3 downloading
2022-03-30 08:14:49 +02:00
kssenii
ce9131f905 Fix unit test 2022-03-29 19:57:55 +02:00
kssenii
f0e0d97752 Fix lask of query id in merges 2022-03-29 19:49:42 +02:00
kssenii
421b1e5a81 Update 2022-03-29 17:33:02 +02:00
Alexander Tokmakov
287d858fda Merge branch 'master' into mvcc_prototype 2022-03-29 16:24:12 +02:00
Antonio Andelic
9990abb76a Use compile-time check for Exception messages, fix wrong messages 2022-03-29 13:16:11 +00:00
alesapin
842c0b8ff0 Merge remote-tracking branch 'origin/startup-sanity-checks' into startup-sanity-checks 2022-03-29 14:29:33 +02:00
alesapin
ed2c461fe1 Merge branch 'master' into startup-sanity-checks 2022-03-29 14:09:20 +02:00
kssenii
7a70e3a6da Fix 2022-03-29 14:08:24 +02:00
kssenii
eab04674fa Fix review comments 2022-03-29 12:38:17 +02:00
taiyang-li
834ac3a1b9 Merge remote-tracking branch 'ck/master' into rocksdb_metacache 2022-03-29 18:32:06 +08:00
Antonio Andelic
362e877529 Merge branch 'master' into parallel-s3-downloading 2022-03-29 07:05:26 +00:00
Alexey Milovidov
5e262fba85
Merge pull request #35204 from azat/build-gcc
Add build with GCC
2022-03-29 04:55:15 +03:00
kssenii
44157e5c80 Merge master 2022-03-29 02:19:32 +02:00
mergify[bot]
9efc8bda78
Merge branch 'master' into remote-filesystem-cache-fixes 2022-03-28 20:53:29 +00:00
Alexander Tokmakov
208b242188 Merge branch 'master' into mvcc_prototype 2022-03-28 19:58:06 +02:00
Dmitry Novik
a767605f0e Remove redundant mutable 2022-03-28 15:36:25 +00:00
Dmitry Novik
b34c1c02b7 Fix typo 2022-03-28 15:05:37 +00:00
Dmitry Novik
2017361e00 Do not cancel query if memory is free 2022-03-28 14:54:13 +00:00
Dmitry Novik
31566cdd78 Fix typo 2022-03-28 13:49:07 +00:00
taiyang-li
38702be40c Merge remote-tracking branch 'ck/master' into rocksdb_metacache 2022-03-28 21:03:22 +08:00
Amos Bird
6003a49a87
Fix style 2022-03-28 17:48:47 +08:00
Antonio Andelic
d9d826c813 Address PR review 2022-03-28 08:19:23 +00:00
Amos Bird
58a78fba12
formatString bug fix and refactoring 2022-03-28 15:47:48 +08:00
Antonio Andelic
cbe8e747b4 Merge branch 'master' into parallel-s3-downloading 2022-03-28 07:04:45 +00:00
Alexey Milovidov
6fbf5bd9ad More correct 2022-03-28 02:11:27 +02:00
Alexey Milovidov
bb35184da1 Add metric about size of async INSERTs 2022-03-28 02:04:19 +02:00
Dmitry Novik
95c9a03b9c Handle corner case 2022-03-27 16:56:41 +00:00
Azat Khuzhin
6da2c48b7a Add missing __builtin_unreachable for switch/case for gcc
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-26 23:42:21 +03:00
Dmitry Novik
0d75e773ab Continue query execution if memory is freed 2022-03-26 18:25:58 +00:00
alesapin
14e84ff6ef
Merge pull request #34802 from ClickHouse/overcommit-doc
Add documentation for memory overcommit
2022-03-25 20:50:44 +01:00
kssenii
141c290344 Fix build 2022-03-25 18:31:15 +01:00
Antonio Andelic
394682b983 Merge branch 'master' into parallel-s3-downloading 2022-03-25 13:00:44 +00:00
Kruglov Pavel
2f2d3cdbca
Merge pull request #35578 from ClickHouse/Avogar-patch-3
Increase fiber stack size a bit in attempt to fix stack overflow in tests with address sanitizer
2022-03-25 00:33:05 +01:00
Maksim Kita
7b03617341
Merge pull request #35539 from kitaisreal/parse-ipv6-long-fix
Fix parsing of IPv6 addresses longer than 39 characters
2022-03-24 22:28:51 +01:00
kssenii
0c6577eee0 Merge master 2022-03-24 20:49:02 +01:00
kssenii
6769b84638 Merge master 2022-03-24 20:47:43 +01:00
Dmitry Novik
56d20e00cc
Update src/Common/OvercommitTracker.cpp
Co-authored-by: alesapin <alesapin@clickhouse.com>
2022-03-24 17:47:48 +01:00
Kruglov Pavel
f07918c659
Increase stack size 2022-03-24 15:50:32 +01:00
kssenii
d6ab6de30a Minor changes 2022-03-24 15:32:08 +01:00
Sergei Trifonov
2e2a1c06a2 Merge branch 'startup-sanity-checks' of github.com:ClickHouse/ClickHouse into startup-sanity-checks 2022-03-24 15:30:59 +01:00
Sergei Trifonov
041e348076 cosmetic bugfix 2022-03-24 15:30:10 +01:00
mergify[bot]
394fdb4fb8
Merge branch 'master' into startup-sanity-checks 2022-03-24 14:23:38 +00:00
Kruglov Pavel
98be162d20
Update comment 2022-03-24 13:49:26 +01:00
Kruglov Pavel
37286c6141
Increase fiber stack size a bit in attempt to fix stack overflow in tests with address sanitizer 2022-03-24 12:55:56 +01:00
tavplubix
7b96d61057
Merge pull request #35478 from ClickHouse/merging_30325
Merging #30325
2022-03-24 14:53:20 +03:00
kssenii
142a5938a1 Merge master 2022-03-24 12:10:26 +01:00
alesapin
d3fbd9b2ad Fix style 2022-03-24 11:25:28 +01:00
alesapin
93d1b1d198 Merge branch 'master' into startup-sanity-checks 2022-03-24 11:17:06 +01:00
Antonio Andelic
c52cee2219 Merge branch 'master' into parallel-s3-downloading 2022-03-24 08:11:52 +00:00
kssenii
d4161b5925 Add optin read_from_cache_if_exists_otherwise_bypass_cache (for merges) 2022-03-23 20:24:00 +01:00
mergify[bot]
b745b67026
Merge branch 'master' into parse-ipv6-long-fix 2022-03-23 18:56:40 +00:00
mergify[bot]
abbed251c1
Merge branch 'master' into asof-join-improve-performance 2022-03-23 18:53:17 +00:00
Sergei Trifonov
c7cf14e743 fix style and darwin build 2022-03-23 19:15:01 +01:00
kssenii
d2a3cfe5dc Cache on all write operations 2022-03-23 19:14:33 +01:00
Yakov Olkhovskiy
1631656803
Merge pull request #34353 from zvonand/issue_33147
Extended usage of Milliseconds, Microseconds, Nanoseconds
2022-03-23 13:33:07 -04:00
kssenii
e39aba37a2 Minor changes 2022-03-23 18:11:52 +01:00
kssenii
c2c7f365ce Merge master 2022-03-23 16:58:35 +01:00
Maksim Kita
df0d3c9304 Fix parsing of IPv6 addresses longer than 39 characters 2022-03-23 16:11:46 +01:00
kssenii
34c0690a69 Better 2022-03-23 15:43:14 +01:00
kssenii
f3e1ca44a9 Add system.remote_data_paths table 2022-03-23 13:01:18 +01:00
Maksim Kita
3c5c267c17 Improve performance of ASOF join 2022-03-23 12:19:38 +01:00
Alexander Tokmakov
94d32c6b80 Merge branch 'master' into merging_30325 2022-03-23 11:58:57 +01:00
Alexander Tokmakov
bcec53aadc Merge branch 'master' into mvcc_prototype 2022-03-23 11:28:43 +01:00
Antonio Andelic
bb6b9f3799 Merge branch 'parallel-downloading-url-engine' into parallel-s3-downloading 2022-03-23 07:46:14 +00:00
Antonio Andelic
9829f2d051 Merge branch 'master' into parallel-downloading-url-engine 2022-03-23 07:41:14 +00:00
taiyang-li
8dbf1c60e7 merge master and fix conflict 2022-03-23 11:36:50 +08:00
Alexander Tokmakov
7e904ac789 more logs 2022-03-22 19:05:37 +01:00
Alexander Tokmakov
bc5a689899 add debug logging 2022-03-22 15:34:20 +01:00
Antonio Andelic
df99a9cc3c Merge branch 'parallel-downloading-url-engine' into parallel-s3-downloading 2022-03-22 12:49:35 +00:00
Antonio Andelic
5da3058368 Merge branch 'master' into parallel-downloading-url-engine 2022-03-22 12:20:36 +00:00
kssenii
d8739318ea Fix bug 2022-03-22 11:22:57 +01:00
alesapin
bad78f0daa
Don't unwind inlines if we unable to do it 2022-03-22 11:11:36 +01:00
kssenii
a9ba14e386 Fix checks 2022-03-21 21:53:46 +01:00
mergify[bot]
9a505476a5
Merge branch 'master' into mvcc_prototype 2022-03-21 20:33:07 +00:00
Maksim Kita
ce2883a13d
Merge pull request #35462 from kitaisreal/mysqlxx-pool-with-failover-style-fix
mysqlxx PoolWithFailover style fix
2022-03-21 20:30:55 +01:00
Alexander Tokmakov
3c762f566d Merge branch 'master' into mvcc_prototype 2022-03-21 20:16:29 +01:00
kssenii
e5c870add2 Fix bug, fix unit tests 2022-03-21 20:00:52 +01:00
Alexander Tokmakov
0ab258be01 fix_build 2022-03-21 19:59:47 +01:00
Maksim Kita
1cb8faf36c Fixed style 2022-03-21 16:35:18 +01:00
Alexander Tokmakov
3cca5fb181 minor fixes 2022-03-21 15:55:01 +01:00
Antonio Andelic
04e6e6ab5e Add ParallelReadBuffer for S3 2022-03-21 14:52:26 +00:00
kssenii
afd0c64a1a Better 2022-03-21 15:04:16 +01:00
mergify[bot]
e11ef05c2b
Merge branch 'master' into issue_33147 2022-03-21 13:40:17 +00:00
Alexander Tokmakov
94f616b48e Merge branch 'master' into merging_30325 2022-03-21 13:23:45 +01:00
kssenii
eabbce69a7 Add system drop remote filesystem cache query, allow to write cache on insert 2022-03-21 12:32:44 +01:00
Vitaly Baranov
7a46e7a0d1
Merge branch 'master' into improve-backup-and-restore 2022-03-21 11:49:26 +01:00
Vitaly Baranov
51fc556fcd Implement BACKUP and RESTORE for MatViews. 2022-03-21 11:45:21 +01:00
Maksim Kita
5dcbc07d22 Fixed style check 2022-03-21 10:49:46 +01:00
Maksim Kita
695b1300f9
Merge pull request #35383 from rschu1ze/enable-if-to-concepts-pt2
Replace more uses of std::enable_if for SFINAE by concepts
2022-03-21 10:47:28 +01:00
Maksim Kita
15271c9b13 mysqlxx PoolWithFailover style fix 2022-03-21 10:45:53 +01:00
Antonio Andelic
b8c43ff2f2 Merge branch 'master' into parallel-downloading-url-engine 2022-03-21 09:11:01 +00:00
kssenii
1d80262a2a Add write settings, file cache settings 2022-03-21 09:50:50 +01:00
Alexey Milovidov
0ce4696c49
Revert "[WIP] New row policies" 2022-03-21 08:41:33 +03:00
Alexander Tokmakov
9e05b12d2c Merge branch 'master' into mvcc_prototype 2022-03-20 22:42:26 +01:00
Vitaly Baranov
e72a343994 Implement BACKUP & RESTORE for the Memory table engine. 2022-03-20 20:02:15 +01:00
Vitaly Baranov
e8fc9cf476 Move renameInCreateQuery() to a better place. 2022-03-20 20:02:12 +01:00
Vitaly Baranov
c160494f43 Make BACKUP more consistent with RESTORE. 2022-03-20 20:01:52 +01:00
Vitaly Baranov
7a63feb3f7 Make restore tasks explicit. 2022-03-20 20:01:31 +01:00
Vitaly Baranov
9fb50422ce
Merge pull request #35345 from vitlibar/new-row-policies
[WIP] New row policies
2022-03-20 20:00:25 +01:00
Robert Schulze
7617053c58
Revert changes in HashTable.h 2022-03-19 15:44:21 +01:00
mergify[bot]
d2509dadc5
Merge branch 'master' into issue/#35128_add_mysql_error__detail 2022-03-18 13:42:25 +00:00
Alexander Tokmakov
0719d01d39 more introspection 2022-03-18 14:33:59 +01:00
kssenii
43ed2beb3c Merge master 2022-03-18 12:33:08 +01:00
Alexander Tokmakov
c28008a2df more comments, minor fixes 2022-03-18 12:01:26 +01:00
kssenii
b756af7a14 Uncomment code, update test 2022-03-18 10:57:32 +01:00
kssenii
44fb6fb86f Revert some changes 2022-03-18 10:16:06 +01:00
zzsmdfj
e5540fe4c4 to #35128_add_mysql_error_detail 2022-03-18 09:49:33 +08:00
Robert Schulze
df73c75456
Replace more uses of std::enable_if for SFINAE by concepts
- enable_if is usually regarded as fragile and unreadable
- C++20 concepts are much easier to read and produce more expressive error messages

- this is follow-up to PR #35347 but this time most of the remaining and
  more complex usages of enable_if in the codebase were replaced.
2022-03-17 20:44:55 +01:00
kssenii
a7c9fe5c78 Finally-finally fixed 2022-03-17 20:29:07 +01:00
kssenii
2157e8820f Fix 2022-03-17 18:30:32 +01:00
kssenii
c698fb9539 Debug 2022-03-17 18:24:00 +01:00
kssenii
1b6e7fea2f Finally fixed 2022-03-17 16:40:10 +01:00
Vitaly Baranov
3e950d79b3 Add new setting rbac_version. 2022-03-17 15:28:51 +01:00
Alexander Tokmakov
d04dc03fa4 Merge branch 'master' into mvcc_prototype 2022-03-17 15:24:32 +01:00
kssenii
59c0fb0244 Revert changes with downloaded size 2022-03-17 11:53:49 +01:00
kssenii
f354b704e0 Add more checks, different counting for download size 2022-03-17 11:52:59 +01:00
Antonio Andelic
103a3fa140 Merge branch 'master' into parallel-downloading-url-engine 2022-03-17 10:11:58 +00:00
alesapin
457fa0d1ba
Merge pull request #23932 from CurtizJ/dynamic-columns
Support of dynamic subcolumns (JSON data type)
2022-03-17 10:45:46 +01:00
Maksim Kita
4d8c2b2009
Merge pull request #35347 from rschu1ze/enable-if-to-concepts
Replace a few uses of enable_if for SFINAE by concepts
2022-03-17 10:34:08 +01:00
Alexander Tokmakov
4b3e13a4fe Merge branch 'master' into mvcc_prototype 2022-03-16 21:06:19 +01:00
Alexander Tokmakov
c2ac8d4a5c review fixes 2022-03-16 21:05:34 +01:00
Robert Schulze
b56c28d841
Replace a few uses of enable_if for SFINAE by concepts
- enable_if is usually regarded as fragile and unreadable

- C++20 concepts are much easier to read and produce more expressive
  error messages
2022-03-16 19:51:38 +01:00
Maksim Kita
9ba53ae037
Merge pull request #34888 from kitaisreal/performance-tests-fix
Performance tests fix
2022-03-16 17:19:16 +01:00
Maksim Kita
dc31a4175f Fixed style check 2022-03-16 16:16:08 +00:00
Anton Popov
0ba78c3c3a Merge remote-tracking branch 'upstream/master' into HEAD 2022-03-16 15:28:09 +00:00
Kseniia Sumarokova
663c8e9e4a
Merge pull request #35275 from kssenii/fix-logical-error-in-cache
Fix logical error in remote fs cache
2022-03-16 15:01:41 +01:00
Maksim Kita
ed74a464bd Fixed tests 2022-03-16 11:21:18 +00:00
Antonio Andelic
2291b7c1e1 Merge branch 'master' into parallel-downloading-url-engine 2022-03-16 09:59:52 +00:00
Nikolai Kochetov
a380aa6b8a
Merge pull request #35294 from ClickHouse/reload-remote_url_allow_hosts
Reload remote_url_allow_hosts after config update.
2022-03-15 22:07:16 +01:00
Maksim Kita
8bdb906acb Fixed tests 2022-03-15 15:43:39 +00:00
Maksim Kita
08bb39d869 Fixed performance tests 2022-03-15 15:43:39 +00:00
Nikolai Kochetov
d6710b3354 Always clear hosts. 2022-03-15 14:08:56 +00:00
Nikolai Kochetov
97aa6c82ce Reload remote_url_allow_hosts after config update. 2022-03-15 13:00:31 +00:00
mergify[bot]
a7298a98b9
Merge branch 'master' into fix-logical-error-in-cache 2022-03-15 08:29:16 +00:00
Antonio Andelic
067b79b00b Merge branch 'master' into parallel-downloading-url-engine 2022-03-15 07:55:41 +00:00
Alexander Tokmakov
9702b5177d Merge branch 'master' into mvcc_prototype 2022-03-14 21:45:38 +01:00
Alexander Tokmakov
278d779a01 log cleanup, more comments 2022-03-14 21:43:34 +01:00
Maksim Kita
282dd98779 Fix build 2022-03-14 18:17:36 +00:00
Maksim Kita
e7772ed434 Fix clang-tidy warnings in Common folder 2022-03-14 18:17:35 +00:00
kssenii
53d23d901e Fix 2022-03-14 18:11:59 +01:00
Anton Popov
36ec379aeb Merge remote-tracking branch 'upstream/master' into HEAD 2022-03-14 16:28:35 +00:00
Antonio Andelic
d3353f3f0c Merge branch 'master' into parallel-downloading-url-engine 2022-03-14 12:47:23 +00:00
alesapin
b149f28cd3
Merge pull request #35209 from kssenii/cache-fix
Cache fix
2022-03-14 11:07:40 +01:00
Alexey Milovidov
978877a9c0
Merge pull request #35212 from rschu1ze/cpp14-trait-aliases
Use C++14 aliases for some type traits
2022-03-12 22:20:14 +03:00
zzsmdfj
88560c3917 to #35128_add_mysql_error__detail 2022-03-12 11:10:26 +08:00
alexey-milovidov
a3a8a1055d
Merge pull request #34919 from azat/fix-glibc2.35-build
Fix alignment for aligned_alloc() to fix build for glibc2.35+
2022-03-11 17:59:49 +03:00
alexey-milovidov
003619b18c
Merge pull request #34771 from azat/fix-glibc2.34-build
Fix undefined __pthread_mutex_lock/unlock for glibc 2.34+/DISABLE_HERMETIC_BUILD
2022-03-11 17:58:37 +03:00
alexey-milovidov
b6f511506b
Update ThreadFuzzer.cpp 2022-03-11 17:58:27 +03:00
Robert Schulze
85ce4a9bea
Use C++14 aliases for some type traits 2022-03-11 15:54:29 +01:00
kssenii
a18bb606a2 Update 2022-03-11 14:24:02 +01:00
kssenii
2b478e2501 Fix 2022-03-11 12:17:17 +01:00
Kseniia Sumarokova
2bd99a4c70
Update FileSegment.cpp 2022-03-11 09:52:50 +01:00
Antonio Andelic
df0f5e20d0 Refactor 2022-03-11 08:33:34 +00:00
Kseniia Sumarokova
818459b9f0
Merge pull request #33717 from kssenii/local-cache-for-remote-fs
Local cache for remote filesystem
2022-03-11 07:23:10 +01:00
Nikita Mikhaylov
909a7309ca
Delete nodes with attributes remove or replace if they didn't merge (#35141) 2022-03-10 23:40:55 +01:00
alexey-milovidov
40dd7da309
Update gtest_DateLUTImpl.cpp 2022-03-10 22:29:56 +03:00
Maksim Kita
83de2f66d1
Merge pull request #35032 from ClickHouse/fix-query-profiler-on-bad-linux-kernels
Avoid division by zero in Query Profiler if Linux kernel has a bug
2022-03-10 17:49:27 +01:00
Alexander Tokmakov
061fa6a6f2 Merge branch 'master' into mvcc_prototype 2022-03-10 13:13:04 +01:00
Miel Donkers
4a95e6d602
Parsing YAML config to XML leads to incorrect structures (#35135) 2022-03-10 13:09:48 +01:00
kssenii
787a0805a5 Merge master 2022-03-10 11:42:19 +01:00
kssenii
bd68d1d6d0 Update test 2022-03-10 11:02:12 +01:00
kssenii
d089b0ac86 Fix race 2022-03-09 18:14:28 +01:00
Nikita Mikhaylov
d749295222
Fix hardcoded page size (#35129) 2022-03-09 12:35:23 +01:00
kssenii
6571c7faa0 Clean up code 2022-03-09 10:36:52 +01:00
kssenii
5260822964 Merge master 2022-03-08 18:21:28 +01:00
kssenii
28a3aece60 Fix uncaught exception from destructor 2022-03-08 10:58:37 +01:00
taiyang-li
b4174b0bef merge master and fix conflicts 2022-03-08 11:39:25 +08:00
kssenii
4f53080921 Fix 2022-03-07 22:03:12 +01:00
Alexander Tokmakov
8acfb8d27f Merge branch 'master' into mvcc_prototype 2022-03-07 17:40:15 +01:00
Anton Popov
0bc57da238 Merge remote-tracking branch 'upstream/master' into HEAD 2022-03-07 14:46:08 +00:00
kssenii
692b247379 Fix some corner cases 2022-03-07 14:31:21 +01:00
kssenii
2dcfe66236 Fix race 2022-03-06 20:33:07 +01:00
alexey-milovidov
01f8eab2da
Merge pull request #35020 from ClickHouse/preprocessed_config_in_yaml
Always write preprocessed config in XML format
2022-03-06 02:31:37 +03:00
Anton Popov
df3b07fe7c Merge remote-tracking branch 'upstream/master' into HEAD 2022-03-03 22:25:28 +00:00
Alexey Milovidov
dda6632932 Avoid division by zero in Query Profiler if Linux kernel has a bug 2022-03-03 21:28:30 +01:00
Nikita Mikhaylov
33baefb12b Done 2022-03-03 14:35:52 +01:00
Maksim Kita
b1a956c5f1 clang-tidy check performance-move-const-arg fix 2022-03-02 18:15:27 +00:00
Maksim Kita
1f5837359e clang-tidy check performance-noexcept-move-constructor fix 2022-03-02 18:15:27 +00:00
kssenii
d5952109fb Merge master 2022-03-02 18:15:25 +01:00
alesapin
b1f5805647
Merge pull request #34609 from ClickHouse/unrestricted-zk-reads
allow unrestricted reads from zookeeper
2022-03-02 14:53:12 +01:00
kssenii
2ce4d34964 Fix checks 2022-03-02 14:17:19 +01:00
HaiBo Li
2ea18b3085
Fix the bug that the toDatetime function overflows (#32898)
* Fixed overflow bug of the function toDatetime/toDate/toDate32/parseDateTimeBestEffort
2022-03-02 10:06:38 +03:00
kssenii
a594f388a4 Merge master 2022-03-01 19:43:45 +01:00
Anton Popov
c1fdcf7a64 Merge remote-tracking branch 'upstream/master' into HEAD 2022-03-01 20:21:39 +03:00
kssenii
b2cab429a7 Move initialization out of constructor 2022-03-01 18:12:34 +01:00
alesapin
cba5fe44a9 Merge branch 'master' into unrestricted-zk-reads 2022-03-01 18:09:21 +01:00
kssenii
438c63f3d8 Style 2022-03-01 17:52:38 +01:00
kssenii
2f4f76341c Fix deadlock 2022-03-01 17:07:50 +01:00
Danila Kutenin
3ce71f211a First portion of force inlines 2022-02-27 10:43:17 +00:00
Azat Khuzhin
311f884d3d Fix alignment for aligned_alloc() to fix build for glibc2.35+
In glibc2.35+ alloc_align added for aligned_alloc() alignment argument
8a9a59311551e833ca064de44ac23b193e1b704d ("Add alloc_align attribute to
memalign et al"), and this will cause an error:

    /src/ch/clickhouse/src/Common/malloc.cpp:41:26: error: requested alignment is not a power of 2 [-Werror,-Wnon-power-of-two-alignment]
        ignore(aligned_alloc(0, 0)); // -V575 NOLINT

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-26 13:04:38 +03:00
Alexandre Snarskii
4a14b25ca6 use getPageSize, correct typo 2022-02-25 17:02:49 +03:00
Vladimir C
45b157f909
Use SYSTEM_ERROR in MemoryStatisticsOS for FreeBSD 2022-02-25 12:37:12 +01:00
Alexandre Snarskii
93de1b1c99 Implement MemoryStatisticsOS for FreeBSD 2022-02-25 14:05:00 +03:00
Anton Popov
fcdebea925 Merge remote-tracking branch 'upstream/master' into HEAD 2022-02-25 13:41:30 +03:00
kssenii
bb27852739 Fix deadlock 2022-02-24 15:20:51 +01:00
Alexander Tokmakov
aa6b9a2abc Merge branch 'master' into mvcc_prototype 2022-02-23 23:22:03 +03:00
kssenii
8d6f1982df Tiny refactoring 2022-02-23 14:43:40 +01:00
kssenii
3524ae3cfc Fix race 2022-02-23 11:12:14 +01:00
Sergei Trifonov
7fe3bef866 add test for unrestricted zk reads 2022-02-22 16:51:30 +03:00
kssenii
c637385dd0 Merge master 2022-02-22 13:17:51 +01:00
Alexey Milovidov
8d05ecc4c0 Fix error 2022-02-22 01:32:23 +01:00
Dmitry Novik
be9082f1c4 Update comments for OvercommitTracker 2022-02-21 15:45:17 +00:00
kssenii
4543513a5d Add comments 2022-02-21 13:54:03 +01:00
kssenii
009e71e273 Fix tests 2022-02-21 11:43:03 +01:00
Alexey Milovidov
829bd066c4 Fix progress bar 2022-02-21 00:25:07 +01:00
Azat Khuzhin
9b749ae90b Fix undefined __pthread_mutex_lock/unlock for glibc 2.34+/DISABLE_HERMETIC_BUILD
Right now it fails with:

    ld.lld: error: undefined symbol: __pthread_mutex_lock
    >>> referenced by ThreadFuzzer.cpp:300 (./src/Common/ThreadFuzzer.cpp:300)
    >>> src/CMakeFiles/clickhouse_common_io.dir/Common/ThreadFuzzer.cpp.o:(pthread_mutex_lock)
    >>> did you mean: __pthread_mutex_lock@GLIBC_2.2.5
    >>> defined in: /usr/lib/libc.so.6

Here is the list of matched symbols for 2.35:

    $ nm -D /lib/libc.so.6 | fgrep pthread_mutex_lock
    00000000000908a0 T __pthread_mutex_lock@GLIBC_2.2.5
    00000000000908a0 T pthread_mutex_lock@@GLIBC_2.2.5

    $ nm -D /lib/libpthread.so.0 | fgrep -c pthread_mutex_lock
    0

And this is for 2.33:

    $ nm -D /lib/x86_64-linux-gnu/libc.so.6 | fgrep pthread_mutex_lock
    0000000000083eb0 T pthread_mutex_lock@@GLIBC_2.2.5

    $ nm -D /lib/x86_64-linux-gnu/libpthread.so.0 | fgrep pthread_mutex_lock
    000000000000af00 T __pthread_mutex_lock@@GLIBC_2.2.5
    000000000000af00 W pthread_mutex_lock@@GLIBC_2.2.5

Because "likely" starting from 27a448223cb2d3bab191c61303db48cee66f871c
("nptl: Move core mutex functions into libc") [1], __pthread_mutex_lock
is not exported anymore.

  [1]: https://sourceware.org/git/?p=glibc.git;a=commit;h=27a448223cb2d3bab191c61303db48cee66f871c

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-20 22:51:47 +03:00
Alexey Milovidov
1d5e99362f Fix progress bar jumping near 50% 2022-02-20 19:29:09 +01:00
Vladimir C
5284083647
Merge pull request #34683 from vdimir/fix_aarch64/position_utf_8
Fix `positionUTF8` on aarch64
2022-02-20 11:35:40 +01:00
kssenii
6b60f7c57f Add limit for file segment size and split it if exceeds the limit 2022-02-19 18:59:22 +01:00
alexey-milovidov
73fe35e552
Update Volnitsky.h 2022-02-19 19:53:36 +03:00
alexey-milovidov
82728e03b5
Update Volnitsky.h 2022-02-19 19:52:38 +03:00
kssenii
acf9a092a9 Review fixes 2022-02-18 19:09:54 +01:00
Azat Khuzhin
c204ac7744 Export MemoryTracker::getHardLimit()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-18 12:05:12 +03:00
Anton Popov
0a7895ebb9 add comments and small refactoring 2022-02-17 22:00:25 +03:00
zvonand
90c857c5e3 merge 2022-02-17 18:23:37 +03:00
vdimir
b474dc87ac
check len of char with upper and lower case in putNGramUTF8CaseInsensitive 2022-02-17 12:39:29 +00:00
Alexander Tokmakov
dae044f86b Merge branch 'master' into mvcc_prototype 2022-02-17 13:49:37 +03:00
Dmitry Novik
c32313b045
Merge branch 'master' into memory-overcommit-test 2022-02-17 00:50:15 -08:00
Dmitry Novik
371d5e0e2d Disable memory overcommit if timeout is not set 2022-02-16 22:24:09 +00:00
Dmitry Novik
07d0e3c823 cleanup 2022-02-16 20:19:10 +00:00
Dmitry Novik
44d5ddc939 Fix deadlock in OvercommitTracker 2022-02-16 20:02:14 +00:00
kssenii
47f94120da Merge master 2022-02-16 14:43:28 +01:00
Kruglov Pavel
adf58ea1ce
Merge pull request #34490 from Avogar/fix-client
Refactor client fault tolerant connection
2022-02-16 12:39:46 +03:00
kssenii
886b300b8d Less seeks 2022-02-15 16:12:25 +01:00
Dmitry Novik
bb6dad7d0e Fix lock order 2022-02-15 15:04:13 +00:00
serxa
7ea3c003d5 fix style 2022-02-15 10:39:45 +00:00
Sergei Trifonov
a819cb7afe avoid unhandled exceptions on sanity checks at server startup 2022-02-15 12:56:21 +03:00
kssenii
690dcea8be Style 2022-02-15 10:11:33 +01:00
李扬
f52b67b939
Merge branch 'master' into rocksdb_metacache 2022-02-15 02:16:29 -06:00
Alexander Tokmakov
1e4e569151 Merge branch 'master' into mvcc_prototype 2022-02-15 02:26:47 +03:00
alexey-milovidov
bc206ec423
Merge pull request #31182 from ClickHouse/memory-overcommit
Memory overcommit
2022-02-14 23:55:02 +03:00
kssenii
bdbbe85f95 Fixes 2022-02-14 19:28:48 +01:00
Sergei Trifonov
2b0b03276b add simple sanity checks for server startup 2022-02-13 19:58:38 +03:00
Maksim Kita
9ae2271922
Merge pull request #34506 from freedomDR/fix_lowcardinality_group_by
fix lowcardinality on group by of any model
2022-02-13 15:07:49 +01:00
Maksim Kita
9cb440e8d9
Update ColumnsHashing.h 2022-02-13 14:43:02 +01:00
freedomDR
8e5ba9e778 Avoid crash in case of GROUP BY LowCardinality(Nullable(String)) column and group_by_overflow_mode='any' 2022-02-13 10:19:58 +00:00
李扬
daa27d0bda
Merge branch 'master' into rocksdb_metacache 2022-02-12 07:50:12 -06:00
Alexander Tokmakov
07e66e690d Merge branch 'master' into mvcc_prototype 2022-02-11 15:53:32 +03:00
Maksim Kita
13cbf79ecb Improve performance of insert into table functions URL, S3, File, HDFS 2022-02-10 20:06:23 +00:00
mergify[bot]
cb3e5f8538
Merge branch 'master' into memory-overcommit 2022-02-10 11:01:43 +00:00
avogar
75c56163a0 Refactor client fault tolerant connection 2022-02-10 12:43:08 +03:00
Anton Popov
18940b8637 Merge remote-tracking branch 'upstream/master' into HEAD 2022-02-09 23:38:38 +03:00
Nikolai Kochetov
38fb50f736
Merge pull request #33958 from Algunenano/mv_cacheable_scalars
Scalar cache improvements
2022-02-09 16:46:53 +03:00
taiyang-li
d04ccc0489 Merge branch 'master' into rocksdb_metacache 2022-02-09 11:54:10 +08:00
Habibullah Oladepo
57ff05b6e6 Add new function toLastDayOfMonth 2022-02-08 23:48:44 +01:00
DF5HSE
cfefa57485 Merge branch 'master' of github.com:ClickHouse/ClickHouse into client-fault-tolerant-connection 2022-02-08 19:53:55 +03:00
Vitaly Baranov
9a8e45805a
Merge pull request #34226 from vitlibar/add-submodule-minizip
Add submodule minizip
2022-02-08 22:07:34 +07:00
zvonand
0633174b10 Merge branch 'master' of github.com:ClickHouse/ClickHouse into issue_33147 2022-02-08 11:42:31 +03:00
DF5HSE
c08bc7ad65 Merge branch 'master' of github.com:ClickHouse/ClickHouse into client-fault-tolerant-connection 2022-02-08 11:37:31 +03:00
taiyang-li
b6132d490f merge master and solve conflict 2022-02-08 15:24:59 +08:00
DF5HSE
c1df291b18 Fix error with IPv6 in host param 2022-02-08 02:03:41 +03:00
zvonand
9456d89fad Move current work from old laptop
skip-checks: true
2022-02-07 21:44:14 +03:00
kssenii
2e58733750 Merge master 2022-02-07 15:23:10 +01:00
mergify[bot]
dd947f964c
Merge branch 'master' into mv_cacheable_scalars 2022-02-07 10:07:26 +00:00
Alexander Tokmakov
45be75b4db Merge branch 'master' into mvcc_prototype 2022-02-06 23:36:08 +03:00
zvonand
be081d997d basic added fractional intervals types 2022-02-06 15:14:18 +03:00
Vitaly Baranov
23fac284ea Add utility classes ZipArchiveReader and ZipArchiveWriter. 2022-02-05 23:45:20 +07:00
Vitaly Baranov
f8ef1cd23d Add submodule minizip-ng 2022-02-05 23:45:16 +07:00
alexey-milovidov
a089f9918c
Update ThreadPool.cpp 2022-02-04 03:54:33 +03:00
Alexey Milovidov
78eebd5c7c Fix parallel loading of data parts 2022-02-04 02:29:46 +03:00
Alexander Tokmakov
ca5f951558 Merge branch 'master' into mvcc_prototype 2022-02-03 18:56:44 +03:00
Maksim Kita
f4423937bd
Merge pull request #34275 from ClickHouse/fix-progress-bar
Fix progress bar width
2022-02-03 11:21:08 +01:00
Alexey Milovidov
f047a28ed8 Fix progress bar width 2022-02-03 04:48:52 +03:00
Anton Popov
5bb1b3ce77 support async inserts in clickhouse-client for queries with inlined data 2022-02-02 20:59:37 +03:00
kssenii
aeec749173 Fix predownload case part 1 2022-02-02 17:25:25 +03:00
kssenii
5cba11428a Fixes 2022-02-02 00:25:16 +03:00
Dmitry Novik
80694006f2
Update src/Common/MemoryTracker.cpp
Co-authored-by: alesapin <alesapin@clickhouse.com>
2022-02-01 16:15:00 +03:00
Dmitry Novik
1310a89fb2
Update src/Common/MemoryTracker.cpp
Co-authored-by: alesapin <alesapin@clickhouse.com>
2022-02-01 16:14:48 +03:00
Anton Popov
836a348a9c Merge remote-tracking branch 'upstream/master' into HEAD 2022-02-01 15:23:07 +03:00
Azat Khuzhin
d0d71180dd Add fmt::runtime in gtest_log
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-01 14:44:47 +03:00
Azat Khuzhin
3b3635c6d5 Fix formatting error in logging messages
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-01 14:30:04 +03:00
Azat Khuzhin
bedf208cbd Use fmt::runtime() for LOG_* for non constexpr
Here is oneliner:

    $ gg 'LOG_\(DEBUG\|TRACE\|INFO\|TEST\|WARNING\|ERROR\|FATAL\)([^,]*, [a-zA-Z]' -- :*.cpp :*.h | cut -d: -f1 | sort -u | xargs -r sed -E -i 's#(LOG_[A-Z]*)\(([^,]*), ([A-Za-z][^,)]*)#\1(\2, fmt::runtime(\3)#'

Note, that I tried to do this with coccinelle (tool for semantic
patchin), but it cannot parse C++:

    $ cat fmt.cocci
    @@
    expression log;
    expression var;
    @@

    -LOG_DEBUG(log, var)
    +LOG_DEBUG(log, fmt::runtime(var))

I've also tried to use some macros/templates magic to do this implicitly
in logger_useful.h, but I failed to do so, and apparently it is not
possible for now.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

v2: manual fixes
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-01 14:30:03 +03:00
Azat Khuzhin
a09bc1d72e Add fmt::runtime() in Exception ctor
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-01 14:30:03 +03:00
Alexander Tokmakov
2e4ae37d98 Merge branch 'master' into mvcc_prototype 2022-02-01 13:20:03 +03:00
kssenii
218a633fec Some fixes, more assertions and logging 2022-01-31 10:51:34 +03:00
Maksim Kita
5ef83deaa6 Update sort to pdqsort 2022-01-30 19:49:48 +00:00
Anton Popov
78b9f15abb Merge remote-tracking branch 'upstream/master' into HEAD 2022-01-30 03:24:37 +03:00
kssenii
6b03c577d6 Make downloader's term resticted to one nextImpl 2022-01-29 23:58:53 +03:00
Maksim Kita
76067efbef
Merge pull request #34076 from bigo-sg/poolbase
Add expire() in PoolBase
2022-01-29 13:51:55 +01:00
lgbo-ustc
5f1969c6c5 fixed code styles 2022-01-29 09:46:00 +08:00
Alexander Tokmakov
fb9b2d5326 Merge branch 'master' into mvcc_prototype 2022-01-28 21:18:36 +03:00
lgbo-ustc
c576393143 fixed code styles 2022-01-28 18:34:54 +08:00
lgbo-ustc
f5ffdaac21 fixed code styles 2022-01-28 16:55:17 +08:00
lgbo-ustc
7a3324b29a add expire() in PoolBase 2022-01-28 14:03:34 +08:00
taiyang-li
3de8bde7ce Merge remote-tracking branch 'origin/master' into rocksdb_metacache 2022-01-28 09:58:52 +08:00
alexey-milovidov
4060290eb6
Merge pull request #33925 from benbiti/USE_DEBUG_HELPERS-OFF
when not DEBUG set USE_DEBUG_HELPERS OFF
2022-01-27 22:59:55 +03:00
tavplubix
8c9266b24c
Merge pull request #33978 from amosbird/separatesystemlog
Separate base parts out of SystemLog
2022-01-27 20:07:55 +03:00
Mikhail Artemenko
f349f0e006 add support for some new types 2022-01-27 19:22:05 +03:00
mergify[bot]
260942e0f0
Merge branch 'master' into hlee-issue-7334 2022-01-27 14:30:13 +00:00
Nikolay Degterinsky
c5ca5b608e
Merge pull request #33314 from evillique/classification
Merge functions for text classification
2022-01-27 17:15:08 +03:00
kssenii
c91b86e220 Better 2022-01-26 21:56:08 +03:00
Raúl Marín
4b5ab80e3b Better scalar cache handling
- Fixes global CTE scalar cache.
- Adds MVs back (views dependent on the source are cached locally and others globally
2022-01-26 17:36:45 +01:00
Kruglov Pavel
662ea9d024
Merge pull request #33927 from kitaisreal/range-hashed-dictionary-refactoring
RangeHashedDictionary refactoring
2022-01-26 15:26:57 +03:00
alexey-milovidov
0a112bcf61
Update StringSearcher.h 2022-01-26 13:45:26 +03:00
tavplubix
9e91a9dfd1
Merge pull request #33429 from zzsmdfj/MaterializeMySQL_support_set_and_other_dataType
MaterializeMySQL support SET YEAR TIME GEOMETRY dataType
2022-01-26 11:29:46 +03:00
HarryLeeIBM
8b24688afb Issue 7334: Fixed utf8 string case-insensitive searching issue 2022-01-25 13:56:05 -05:00
tavplubix
35534a4b92
Update Types.h 2022-01-25 16:42:55 +03:00
Maksim Kita
bcbd956b83 RangeHashedDictionary change layout structure 2022-01-25 11:13:37 +00:00
Amos Bird
385caa3e0a
Separate base parts out of SystemLog 2022-01-25 18:48:43 +08:00
Nikolay Degterinsky
5edbae6734 Merge branch 'master' into classification 2022-01-25 10:22:47 +00:00
Kruglov Pavel
9f12f4af13
Merge pull request #33302 from Avogar/formats-with-suffixes
Allow to create new files on insert for File/S3/HDFS engines
2022-01-25 10:56:15 +03:00
kssenii
aef99dedba Fixes 2022-01-25 01:46:44 +03:00
kssenii
898139acae Fix tests 2022-01-24 12:32:39 +03:00
benbiti
d97c43034c fix build on USE_DEBUG_HELPERS OFF 2022-01-24 14:29:38 +08:00
taiyang-li
73def8b483 merge master and solve conflict 2022-01-24 11:01:43 +08:00
kssenii
710bba895e Fix partially downloaded no continuation case 2022-01-23 23:49:13 +03:00
kssenii
cf49f41f75 Better 2022-01-23 19:51:18 +03:00
kssenii
690db142b9 Better 2022-01-23 02:12:33 +03:00
Kseniia Sumarokova
2c43e4eb57
Merge branch 'master' into local-cache-for-remote-fs 2022-01-22 22:53:55 +03:00
kssenii
36a41ac146 Better 2022-01-22 20:42:39 +03:00
Mikhail f. Shiryaev
13f7f56c5b
Merge pull request #33342 from JaySon-Huang/get_cpu_cores_under_cgroup
Fix cpu count with cgroup limits
2022-01-22 11:32:08 +01:00
alexey-milovidov
f3053d3727
Merge pull request #33706 from hanqf-git/avx512_memcmpSmall
Avx512 memcmp small
2022-01-22 13:02:47 +03:00
alexey-milovidov
eb6849f7c7
Merge pull request #33842 from azat/cmake-contrib-fixes
More cmake external modules cleanups
2022-01-22 10:34:54 +03:00
Nikolai Kochetov
b4311cf3c9 Fix cmake for mac. 2022-01-21 22:08:42 +03:00
Anton Popov
e8ce091e68 Merge remote-tracking branch 'upstream/master' into HEAD 2022-01-21 20:11:18 +03:00
Alexander Tokmakov
e3b1397001 Merge branch 'master' into mvcc_prototype 2022-01-21 18:59:01 +03:00
kssenii
1858f69f5a Make space reservation incremental 2022-01-21 18:39:34 +03:00
tavplubix
71d69d813c
Merge pull request #33860 from azat/fix-zk-lib
Fix zookeeper library dependency from interpreters (by marking library STATIC)
2022-01-21 16:21:48 +03:00
JaySon-Huang
e155057cc5 Fix cpu count with cgroup limits
Signed-off-by: JaySon-Huang <tshent@qq.com>
2022-01-21 21:03:23 +08:00
Kruglov Pavel
daba57a933
Merge pull request #33737 from Avogar/bool-type
Some improvements and fixes for Bool data type
2022-01-21 15:24:58 +03:00
Azat Khuzhin
d463031105 Fix zookeeper library dependency from interpreters (by marking library STATIC)
This had been introduced in #33534, and an attempt to fix this had been
made in #33844 but it was not strictly correct, since there is undefined
reference to `DB::SystemLog<DB::ZooKeeperLogElement>::add`:

    $ nm -D ./src/Common/ZooKeeper/libclickhouse_common_zookeeperd.so | fgrep SystemLog | c++filt
          U DB::SystemLog<DB::ZooKeeperLogElement>::add(DB::ZooKeeperLogElement const&)

That patch works, because default is not
`-Wl,--unresolved-symbols=report-all`, so I guess shared build on OSX
may after #33844.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-01-21 11:10:03 +03:00
Azat Khuzhin
d25b59803e contrib/abseil: add cmake ALIAS library
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-01-21 10:11:06 +03:00
zzsmdfj
411f43ec4d to MaterializeMySQL_support_set_and_other_dataType 2022-01-21 12:24:12 +08:00
Maksim Kita
8567c3a20e
Merge pull request #33848 from kitaisreal/fix-build-with-examples
Fix build with ENABLE_EXAMPLES
2022-01-20 23:18:47 +01:00
Maksim Kita
e2fe8dc3a7 Fix build with ENABLE_EXAMPLES 2022-01-20 19:29:21 +00:00
tavplubix
d26d139974
Update CMakeLists.txt 2022-01-20 20:27:14 +03:00
Maksim Kita
502c1637d5
Merge pull request #33827 from kitaisreal/range-hashed-dictionary-handle-invalid-intervals
RangeHashedDictionary handle invalid intervals
2022-01-20 17:16:49 +01:00
Alexander Tokmakov
0ea0a8ccc7 Merge branch 'master' into mvcc_prototype 2022-01-20 17:05:55 +03:00
Kruglov Pavel
7bfb1231b9
Merge branch 'master' into formats-with-suffixes 2022-01-20 14:47:17 +03:00
Maksim Kita
844eb4ccdc RangeHashedDictionary handle invalid intervals 2022-01-20 11:16:18 +00:00
avogar
ba54a5ff80 Some improvements and fixes for Bool data type 2022-01-20 14:16:11 +03:00
Azat Khuzhin
a6d482d3e1 Cleanup jemalloc contrib 2022-01-20 10:02:03 +03:00
Azat Khuzhin
9926f336e3 Cleanup cpuid contrib 2022-01-20 10:02:03 +03:00
Azat Khuzhin
3e58094bcb Fix build w/o replxx 2022-01-20 10:02:02 +03:00
Azat Khuzhin
e0e81b340d Fix w/o ODBC build 2022-01-20 10:02:02 +03:00
Azat Khuzhin
e160a20769 Remove USE_OPENCL (removed in #31744) 2022-01-20 10:02:02 +03:00
Azat Khuzhin
a75b748fee Remove unbundled mysql support 2022-01-20 10:02:01 +03:00
Azat Khuzhin
7420bd9b18 Move pdqsort 2022-01-20 10:01:59 +03:00
Azat Khuzhin
a773e7ff01 Remove unbundled libpqxx support 2022-01-20 10:01:59 +03:00
Azat Khuzhin
592bc1e97d Remove unbundled yaml-cpp support 2022-01-20 10:01:59 +03:00
Azat Khuzhin
f1cc63d900 Remove unbundled cityhash support 2022-01-20 10:01:12 +03:00
Azat Khuzhin
8ede97925e Remove unbundled sparsehash support 2022-01-20 10:01:11 +03:00
Azat Khuzhin
789dfd9f3b Remove unbundled re2 support
v2: preserve re2_st name to make PVS check pass (since docker image
update fails)
2022-01-20 10:00:49 +03:00
Azat Khuzhin
aef6668cb4 Remove unbundled hdfs3 support 2022-01-20 08:47:16 +03:00
Azat Khuzhin
cf30669cc9 Remove unbundled openssl/boringssl (crypo/ssl) support 2022-01-20 08:47:16 +03:00
Azat Khuzhin
313716be94 Remove unbundled thrift support 2022-01-20 08:47:16 +03:00
Azat Khuzhin
ca8525a733 Remove unbundled zlib support 2022-01-20 08:47:16 +03:00
Alexander Tokmakov
e9a5a64a71 Merge branch 'master' into mvcc_prototype 2022-01-19 21:41:23 +03:00
mergify[bot]
cbb87e762c
Merge branch 'master' into formats-with-suffixes 2022-01-19 17:21:18 +00:00
Maksim Kita
41a6cd54aa
Merge pull request #33516 from kitaisreal/range-hashed-dictionary-interval-tree
RangeHashedDictionary use interval tree
2022-01-19 16:30:31 +01:00
mergify[bot]
e2f92cf18f
Merge branch 'master' into avx512_memcmpSmall 2022-01-19 12:25:04 +00:00
Maksim Kita
1f72bde52f Fix performance tests 2022-01-19 10:41:58 +00:00
Nikolay Degterinsky
8d03099c91 Merge branch 'master' into classification 2022-01-19 09:21:31 +00:00
alexey-milovidov
d222cb9771
Merge pull request #33695 from amosbird/buildimprove
Some build improvement
2022-01-19 03:19:22 +03:00
kssenii
a566099759 Local cache for remote filesystem 2022-01-19 00:42:11 +03:00
Nikolay Degterinsky
9d45025c7d Merge branch 'master' into classification 2022-01-18 21:41:03 +00:00
Dmitry Novik
4e612d0e1a
Merge branch 'master' into memory-overcommit 2022-01-18 23:33:29 +03:00
alesapin
e5ec1aa446
Merge pull request #33642 from ClickHouse/add_integration_test_for_zero_duplication
Fix bug with background moves in zero copy replication
2022-01-18 18:32:16 +03:00
Maksim Kita
821c5dd989 Fix build 2022-01-18 14:46:19 +00:00
Kruglov Pavel
2295a07066
Merge pull request #33534 from azat/fwd-decl
RFC: Split headers, move SystemLog into module, more forward declarations
2022-01-18 17:22:49 +03:00
Maksim Kita
562098779f Fixed style check 2022-01-18 13:26:23 +00:00
Maksim Kita
60bcf88228 Added IntervalTree documentation 2022-01-18 13:20:43 +00:00
Dmitry Novik
c0970b75ee
Merge branch 'master' into memory-overcommit 2022-01-18 15:30:24 +03:00
Dmitry Novik
83c663e2d6 Cleanup after code review 2022-01-18 12:21:59 +00:00
mergify[bot]
b318f9b5db
Merge branch 'master' into formats-with-suffixes 2022-01-18 12:17:07 +00:00
alesapin
eff85d3352 Review fixes 2022-01-18 11:27:01 +03:00
Amos Bird
6d62060e16
Build improvement 2022-01-17 22:36:27 +08:00
Maksim Kita
f311960b00 Added tests 2022-01-17 14:22:01 +00:00
alesapin
4caef03e6a Add ephemeral lock in zookeeper 2022-01-17 14:52:51 +03:00
alesapin
b2271cc2d9
Merge pull request #33288 from JackyWoo/add_lower_bound_session_timeout_to_keeper
Add lower bound session timeout to keeper
2022-01-17 14:06:15 +03:00
Maksim Kita
ea78c0b33c Fixed style 2022-01-16 16:45:33 +00:00
Maksim Kita
c3d0e8e6f1 Fix style check 2022-01-16 12:23:40 +00:00
Maksim Kita
1076ec5539 Added IntervalTree example 2022-01-16 12:23:40 +00:00
Maksim Kita
0626720893 Added IntervalTree 2022-01-16 12:23:40 +00:00
Alexander Tokmakov
4dd216ce8f Merge branch 'master' into mvcc_prototype 2022-01-14 17:11:32 +03:00
Kruglov Pavel
5a908e8edd
Merge branch 'master' into formats-with-suffixes 2022-01-14 16:45:20 +03:00
avogar
eeec8c6c11 Fix error codes order 2022-01-14 16:36:12 +03:00
hanqf-git
46f3a699df Add x86 feature avx512 support for memcmpSmall 2022-01-14 20:21:22 +08:00
alesapin
d3a93f5d7e
Merge pull request #33602 from ClickHouse/interpret_keeper_error_in_our_client
Make ZooKeeper client better interpret keeper server connection reject
2022-01-14 11:32:27 +03:00
alesapin
30180ddc19 Better text 2022-01-13 19:12:01 +03:00
alesapin
ec6840f49c Make ZooKeeper client better interpret keeper server connection reject 2022-01-13 17:12:10 +03:00
mergify[bot]
085492614b
Merge branch 'master' into add_lower_bound_session_timeout_to_keeper 2022-01-13 12:21:46 +00:00
taiyang-li
964f92f2c1 Merge remote-tracking branch 'origin/master' into rocksdb_metacache 2022-01-13 18:29:38 +08:00
taiyang-li
82a3024712 Merge remote-tracking branch 'ck/master' into bigo_hive_table 2022-01-13 18:11:23 +08:00
taiyang-li
6567cd7abc fix all the stateless test 2022-01-13 15:27:41 +08:00
lgbo-ustc
5c71d3687a fixed some bugs
1. interagtion test for test_hive_query failed
2. nullptr reference in arrowSchemaToCHHeader
2022-01-12 17:01:05 +08:00
taiyang-li
66813a3aa9 merge master 2022-01-12 16:56:29 +08:00
Azat Khuzhin
003cdbcf60 Introduce clickhouse_common_config_no_zookeeper_log
Otherwise incorrect zookeeper (w/o log) will be used in clickhouse
server itself.
2022-01-12 07:17:56 +03:00
Azat Khuzhin
4a2bc64989 Introduce clickhouse_common_zookeeper_no_log (w/o extra deps for examples)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-01-11 23:51:41 +03:00
Azat Khuzhin
248b879fd5 Split TraceCollector between Common and Interpreters
This is required to leave TraceCollector::collect() inside main for
MemoryTrackier.
2022-01-11 22:30:55 +03:00
Maksim Kita
457a38a1ad HashTable constant fix 2022-01-11 18:45:41 +03:00
taiyang-li
6751c8552a merge master and resolve conflict 2022-01-11 15:21:56 +08:00
taiyang-li
d31871d253 commit again 2022-01-11 15:17:00 +08:00
Nikolay Degterinsky
85b8985df2 Better 2022-01-10 20:30:26 +00:00
Mikhail f. Shiryaev
1cf46f378f
Merge pull request #33494 from msaf1980/rollup_rules_type
GraphiteMergeTree Rollup rules type
2022-01-10 20:53:20 +01:00
Azat Khuzhin
cb70544dfe Move LockMemoryExceptionInThread and MemoryTrackerBlockerInThread 2022-01-10 22:39:10 +03:00
Azat Khuzhin
c1dea66907 Move TraceCollector into Interpreters
Since now it relies on SystemLog that is in Interpreters, and it cannot
be moved into Common, since it has lots of dependencies.
2022-01-10 22:35:42 +03:00
Azat Khuzhin
aee034a597 Use explicit template instantiation for SystemLog
- Move some code into module part to avoid dependency from IStorage in SystemLog
- Remove extra headers from SystemLog.h
- Rewrite some code that was relying on headers that was included by SystemLog.h

v2: rebase
v3: squash move into module part with explicit template instantiation
    (to make each commit self compilable after rebase)
2022-01-10 22:01:41 +03:00
Alexander Tokmakov
bf2b6c2c37 Merge branch 'master' into mvcc_prototype 2022-01-10 19:53:50 +03:00
msaf1980
780a1b2abe graphite: split tagged/plain rollup rules (for merges perfomance) 2022-01-10 16:34:16 +05:00
Kseniia Sumarokova
e6ea7c361d
Merge pull request #32836 from stigsb/explain-table-override
Add `EXPLAIN TABLE OVERRIDE` for pre-validating overrides.
2022-01-10 09:58:16 +03:00
taiyang-li
2bb7ec8f72 Merge remote-tracking branch 'origin/master' into rocksdb_metacache 2022-01-06 10:44:17 +08:00
lgbo-ustc
f78564491c code style 2022-01-05 14:40:02 +08:00
lgbo-ustc
624da11ee2 fix a bug : cache files are released in wrong way 2022-01-05 14:29:34 +08:00
alexey-milovidov
48eec3d8f8
Merge pull request #33366 from ClickHouse/aggregate-dependencies
Aggregate functions to calculate dependencies between columns
2022-01-05 02:50:25 +03:00
taiyang-li
14bce62cc4 fix building 2022-01-04 15:34:34 +08:00
李扬
2df2442ad0
Merge branch 'master' into hive_table 2022-01-04 01:26:16 -06:00
lgbo-ustc
45c86757bf change LRUCache to LRUResourceCache 2022-01-04 15:19:09 +08:00
taiyang-li
3f6d830536 Merge remote-tracking branch 'origin/master' into rocksdb_metacache 2022-01-04 10:00:34 +08:00
Alexey Milovidov
4ebc8918a4 Fix style 2022-01-03 02:11:47 +03:00
Amos Bird
0b7a94c2be Saturate date/datetime to zero 2022-01-03 02:07:08 +03:00
Alexey Milovidov
cf66716cd6 Make it less wrong 2022-01-02 22:42:56 +03:00
Alexey Milovidov
4a094c2efd Merging contingency coefficients 2022-01-02 21:50:41 +03:00
taiyang-li
1e102bc1b2 merge master 2022-01-01 09:01:06 +08:00
alexey-milovidov
3f1810c049
Merge pull request #33327 from ClickHouse/rename_committed_to_active
Rename Committed data part state to Active
2021-12-31 14:40:35 +03:00
alexey-milovidov
6941b072f4
Merge pull request #33330 from cmsxbc/support-self-closing-tag-for-cluster-secure
support `<secure/>` in cluster configuration.
2021-12-31 14:36:19 +03:00
Kseniia Sumarokova
a3b30dce32
Merge pull request #33334 from kssenii/follow-up-for-33098
Tiny follow-up to pr 33098
2021-12-31 11:11:02 +03:00
SuperDJY
4322fd4bdb
avoid underscore in gtest name 2021-12-31 10:19:03 +08:00
cmsxbc
fd9f1193d8
use .empty() instead of .size() == 0 2021-12-31 08:30:21 +08:00
Alexander Tokmakov
187c43eba8 rename Committed state to Active 2021-12-30 23:45:38 +03:00
alexey-milovidov
42b8678785
Merge pull request #33271 from ClickHouse/always-display-resource-usage-in-client
Always display resource usage in client
2021-12-30 23:36:49 +03:00
kssenii
ae2813a2d5 Fix 2021-12-30 23:31:12 +03:00
Kseniia Sumarokova
a703bcb0c5
Merge pull request #33098 from bigo-sg/lrucache
Add alternative LRUCache version
2021-12-30 23:10:28 +03:00
cmsxbc
f0e82e2cb6
add comment for ConfigHelper::getBool 2021-12-31 00:47:01 +08:00
cmsxbc
c28fcd5a2f
support <secure/> for cluster 2021-12-30 23:51:05 +08:00
lgbo-ustc
9c32723dcf fix a bug in LRUResourceCache::getImpl() 2021-12-30 21:48:54 +08:00
Alexander Tokmakov
0d91887cdc save versions of parts 2021-12-30 16:15:28 +03:00
lgbo-ustc
e6528ca0ef fix a bug in LRUCache::remove() 2021-12-30 20:38:58 +08:00
lgbo-ustc
77084f5349 add token holder 2021-12-30 18:43:12 +08:00
JackyWoo
569ce62e8d use session_timeout as session timeout uper bound 2021-12-30 17:18:51 +08:00
Stig Bakken
6fbb9f5e87 Implement EXPLAIN TABLE OVERRIDE for pre-validating overrides.
Example:

```sql
EXPLAIN TABLE OVERRIDE mysql('127.0.0.1:3306', 'db', 'table', 'user', 'pw')
    PARTITION BY tuple(toYYYYMM(created), id % 8)
```

Validations done:

 * check that storage overrides do not reference unknown or nullable columns
 * check that default specifier is not modified for columns
2021-12-30 09:02:27 +01:00
lgbo-ustc
e8c9079bb8 rename test cases 2021-12-30 14:34:08 +08:00
lgbo-ustc
38605d6be3 rename test cases 2021-12-30 12:20:14 +08:00
Nikolay Degterinsky
b9a75bc39c Fix build 2021-12-30 03:15:21 +00:00
Nikolay Degterinsky
2bdee0b0b8 Merge branch 'master' into classification 2021-12-30 02:17:47 +00:00
lgbo-ustc
54d78bde0d update comments 2021-12-30 10:15:56 +08:00
Nikolay Degterinsky
ba0083736e Change DetectLanguageMixed output type 2021-12-30 02:14:57 +00:00
lgbo-ustc
88383b6715 some change 2021-12-30 10:08:33 +08:00
avogar
97788b9c21 Allow to create new files on insert for File/S3/HDFS engines 2021-12-29 21:19:13 +03:00
alexey-milovidov
3655965b39
Merge pull request #33264 from azat/symbols-from-resources-fix
Fix parsing symbols from resources (for shared builds)
2021-12-29 21:06:51 +03:00
JackyWoo
ad7f206840 change max_session_timeout_ms default value from 30000 to 100000 2021-12-29 21:05:27 +08:00
JackyWoo
d35e5f8319 add lower bound session timeout to keeper 2021-12-29 20:59:01 +08:00
mergify[bot]
5070784282
Merge branch 'master' into memory-overcommit 2021-12-29 10:35:32 +00:00
lgbo-ustc
a412738264 format code style 2021-12-29 18:27:52 +08:00
lgbo-ustc
6b6a82f3b9 add MappedHolder to get cache values 2021-12-29 18:18:38 +08:00
Kruglov Pavel
dd7f61b337
Merge branch 'master' into schema-inference 2021-12-29 12:59:05 +03:00
avogar
8112a71233 Implement schema inference for most input formats 2021-12-29 12:18:56 +03:00
lgbo-ustc
5a3b215f24 fixed code style 2021-12-29 16:00:57 +08:00
lgbo-ustc
6aaeb285b0 update destructor 2021-12-29 15:44:24 +08:00
lgbo-ustc
0d474069b5 add a new test case 2021-12-29 15:38:58 +08:00
lgbo-ustc
5f85f77266 add a new test case 2021-12-29 15:34:20 +08:00
lgbo-ustc
5a419a356b fix a bug 2021-12-29 15:29:59 +08:00
lgbo-ustc
59cbd76880 Add LRUResourceCache
1. add LRUResourceCache for managing resource cache in lru policy
2. rollback LRUCache to the original version
3. add remove() in LRUCache
4. add unit tests for LRUResourceCache and LRUCache
2021-12-29 15:25:33 +08:00
Azat Khuzhin
a23bb79479 Fix parsing symbols from resources (for shared builds)
In case of shared builds the symbol can be represented in multiple
shared objects, and so if one SO contains _end symbol firstly, then it
it's address can subtracted from the symbol from another SO, and this
give overflow.

Here is an example of such symbol:

    $ llvm-readelf -a programs/server/libclickhouse-server-libd.so | fgrep schemata_sql
       552: 000000000010b869     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_size
      1271: 000000000010b869     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_end
      3627: 000000000010b480     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_start
      5822: 000000000010b869     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_end
      5828: 000000000010b869     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_size
      5831: 000000000010b480     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_start

    $ llvm-readelf -W -a src/TableFunctions/libclickhouse_table_functionsd.* | fgrep schemata_sql
      1709: 00000000001fc5a1     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_end
      5107: 00000000001fc5a1     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_size
      6090: 00000000001fc1b8     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_start
     10975: 00000000001fc1b8     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_start
     10976: 00000000001fc5a1     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_end
     10977: 00000000001fc5a1     0 NOTYPE  GLOBAL DEFAULT    22 _binary_schemata_sql_size

To address this, add base address of the Elf into the resources to
compare with.

Follow-up for: #33134 (cc @alexey-milovidov)
2021-12-29 09:56:59 +03:00