thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 03:12:43 +00:00
Code Issues Packages Projects Releases Wiki Activity
92,468 Commits 9,326 Branches 1,783 Tags 2.1 GiB
c2ae762a5a
Commit Graph

6805 Commits

Author SHA1 Message Date
Dmitry Novik
5ba7a55c18
Merge pull request #36650 from bigo-sg/hive_text_parallel_parsing 
Parallel parsing of hive text format
 2022-05-03 15:56:28 +02:00
kssenii
000f2c9f7a Better 2022-05-03 15:55:36 +02:00
Maksim Kita
36d1c8238a
Merge pull request #36766 from kitaisreal/evaluate-constant-expression-fix 
Fix evaluateConstantExpression for subqueries
 2022-05-03 12:32:14 +02:00
Vitaly Baranov
5257ce31f8 Improved using ThreadPool for making backup or restoring, changed columns in system.backups. 2022-05-03 11:03:13 +02:00
Vitaly Baranov
409edfd3fa Rework RestoreCoordination: make restore deterministic. 2022-05-03 11:01:44 +02:00
Vitaly Baranov
2c754f44fc Make calculation of shard_num & replica_num not dependant on match of the cluster's definitions on nodes. 2022-05-03 11:01:44 +02:00
Vitaly Baranov
2a645bb187 Fix sending 'create_table' and 'create_database' restore settings to cluster. 2022-05-03 11:01:44 +02:00
Vitaly Baranov
bddec55d35 Added ASTBackupQuery::setDatabase(). 2022-05-03 11:01:44 +02:00
Robert Schulze
0a4eccb73e
Activated a bunch of LLVM 12/13/14 clang-tidy warnings 
Omitted new checks which produce too many matches or which are
controversial (e.g. readability-identifier-length).

New checks:

- misc-misleading-bidirectional + misc-misleading-identifier

  Detects potential attack as described in the Trojan Source attack

- modernize-macro-to-enum

  Replaces groups of adjacent macros with an unscoped anonymous enum

- modernize-shrink-to-fit

  Replace copy and swap tricks on shrinkable containers with the
  shrink_to_fit() method call

- modernize-use-transparent-functors

  Prefer transparent functors to non-transparent ones

- modernize-use-uncaught-exceptions

  This check will warn on calls to std::uncaught_exception and replace
  them with calls to std::uncaught_exceptions (uncaught_exception was
  deprecated with C++17)

- performance-no-int-to-ptr

  Diagnoses every integer to pointer cast

- readability-duplicate-include

  Looks for duplicate includes and removes them

- readability-redundant-preprocessor

  Finds potentially redundant preprocessor directives

- bugprone-lambda-function-name

  Checks for attempts to get the name of a function from within a lambda
  expression

- bugprone-redundant-branch-condition

  Finds condition variables in nested if statements that were also
  checked in the outer if statement and were not changed

- bugprone-shared-ptr-array-mismatch

  Finds initializations of C++ shared pointers to non-array type that
  are initialized with an array

- bugprone-stringview-nullptr

  Checks for various ways that the const CharT* constructor of
  std::basic_string_view can be passed a null argument and replaces them
  with the default constructor in most cases

- bugprone-suspicious-memory-comparison

  Finds potentially incorrect calls to memcmp() based on properties of
  the arguments
 2022-05-03 09:22:11 +02:00
Dmitry Novik
2ed5a4013a
Revert "Revert "Memory overcommit: continue query execution if memory is available"" 2022-05-03 00:45:13 +02:00
alesapin
f0b7af0aa2
Revert "Memory overcommit: continue query execution if memory is available" 2022-05-03 00:36:50 +02:00
Dmitry Novik
71b6f89166
Merge pull request #35637 from ClickHouse/memory-overcommit-free 
Memory overcommit: continue query execution if memory is available
 2022-05-02 19:00:18 +02:00
mergify[bot]
93f5aa7488
Merge branch 'master' into aggregator-jit-lock-fix 2022-05-02 16:16:36 +00:00
Dmitry Novik
9be17ef50c
Merge pull request #35111 from azat/optimize_aggregation_in_order-prefix 
Implement partial GROUP BY key for optimize_aggregation_in_order
 2022-05-02 17:49:48 +02:00
Ilya Yatsishin
6c3bbb4933 Add Other Time Microseconds Profile Event 2022-05-02 17:13:57 +02:00
Maksim Kita
c059629f8d Aggregator JIT compilation lock fix 2022-05-02 16:21:10 +02:00
Maksim Kita
2377106370 Fix build 2022-05-02 14:27:00 +02:00
Kruglov Pavel
b98ac57090
Merge pull request #36803 from CurtizJ/fix-unflatten-nested 
Fix creation of tables with `flatten_nested = 0`
 2022-05-02 13:58:16 +02:00
Kruglov Pavel
d613f7eab0
Merge branch 'master' into mysqldump-format 2022-05-02 13:31:57 +02:00
Robert Schulze
330212e0f4
Remove inherited create() method + disallow copying 
The original motivation for this commit was that shared_ptr_helper used
std::shared_ptr<>() which does two heap allocations instead of
make_shared<>() which does a single allocation. Turned out that
1. the affected code (--> Storages/) is not on a hot path (rendering the
performance argument moot ...)
2. yet copying Storage objects is potentially dangerous and was
   previously allowed.

Hence, this change

- removes shared_ptr_helper and as a result all inherited create() methods,

- instead, Storage objects are now created using make_shared<>() by the
  caller (for that to work, many constructors had to be made public), and

- all Storage classes were marked as noncopyable using boost::noncopyable.

In sum, we are (likely) not making things faster but the code becomes
cleaner and harder to misuse.
 2022-05-02 08:46:52 +02:00
Kseniia Sumarokova
e3d2d72b39
Merge pull request #36802 from KinderRiven/local_cache_trace 
system table used for statistical query of remote file access
 2022-05-02 00:16:04 +02:00
Azat Khuzhin
3982638e9a Validate that function had been passed in CREATE FUNCTION 
Otherwise right now it leads to LOGICAL_ERROR, due to bad cast.
 2022-05-01 19:01:03 +03:00
Nikita Mikhaylov
93a65463d0
Added SYSTEM SYNC DATABASE query (#35944) 2022-05-01 15:40:18 +02:00
KinderRiven
9413aa423b fix 2022-05-01 19:45:16 +08:00
KinderRiven
7d0789472c fix 2022-05-01 19:45:16 +08:00
KinderRiven
7db67160fe fix 2022-05-01 19:45:16 +08:00
KinderRiven
e1acacf831 fix 2022-05-01 19:45:16 +08:00
KinderRiven
01210c8d38 fix header 2022-05-01 19:45:16 +08:00
KinderRiven
d80aa0fd17 impl system.filesystem_cache_log 2022-05-01 19:45:16 +08:00
KinderRiven
d595df1604 fix 2022-05-01 19:45:16 +08:00
KinderRiven
6a2e4559c8 impl system.cache_log 2022-05-01 19:45:16 +08:00
KinderRiven
ce9a6965d0 impl system.log_table 2022-05-01 19:45:16 +08:00
KinderRiven
a4b8f9e9cf impl system.cache_log 2022-05-01 19:45:16 +08:00
KinderRiven
9c9870f021 build cache_log table 2022-05-01 19:45:16 +08:00
KinderRiven
463b42b4c3 remove unused var 2022-05-01 19:45:16 +08:00
KinderRiven
5a361a0db5 impl cache_log in filecache 2022-05-01 19:45:16 +08:00
KinderRiven
4db54ff6bc add log record 2022-05-01 19:45:16 +08:00
KinderRiven
29f0471caf add cache log 2022-05-01 19:45:16 +08:00
KinderRiven
0059a660c7 impl create cache_log table 2022-05-01 19:45:16 +08:00
KinderRiven
02080aad1c impl create cache_log table 2022-05-01 19:45:16 +08:00
KinderRiven
44c9996a1e impl create cache_log table 2022-05-01 19:45:16 +08:00
KinderRiven
d5f287369e add include 2022-05-01 19:45:16 +08:00
KinderRiven
ee6ef29538 add include 2022-05-01 19:45:16 +08:00
KinderRiven
ef863a7942 add include 2022-05-01 19:45:16 +08:00
KinderRiven
021a3a788b add cache log table in system database 2022-05-01 19:45:16 +08:00
KinderRiven
6f930be28b impl cache trace 2022-05-01 19:45:16 +08:00
Alexey Milovidov
b034146ba4
Merge pull request #36799 from azat/cleanup 
Tiny cleanup
 2022-05-01 14:23:13 +03:00
Alexey Milovidov
321514a7f3
Merge pull request #36816 from azat/is_all_data_sent-on-exception 
Set is_all_data_sent on exceptions too
 2022-05-01 14:22:06 +03:00
Nikita Taranov
0fd9740c72
Log hash table's cache messages with TRACE level (#36830) 2022-05-01 12:54:54 +02:00
Alexey Milovidov
189938e935
Merge pull request #36431 from rschu1ze/bugprone-branch-clone 
Fixed warnings of clang-tidy check "bugprone-branch-clone"
 2022-05-01 13:49:37 +03:00
Alexey Milovidov
91d1ac913c
Merge pull request #36820 from azat/fix-asynchronous_metric_log 
Avoid recreation of system.asynchronous_metric_log (due to difference in codec)
 2022-05-01 13:42:17 +03:00
Robert Schulze
89aa9ae00f
Fixed clang-tidy check "bugprone-branch-clone" 
The check is currently *not* part of .clang-tidy. It complains about:
(1) "switch has multiple consecutive identical branches"
(2) "repeated branch in conditional chain"

About (1): Lots of findings in switches were about redundant
"[[fallthrough]]" in places where the compiler would not warn anyways. I
have cleaned these up.

About (2): In if-else_if-else chains, fixing the warning would usually
mean concatenating multiple if-conditions. As this would reduce
readability in most cases, I did not fix these places.

Because of (2), I also refrained from adding "bugprone-branch-clone" to
.clang-tidy.
 2022-04-30 19:40:28 +02:00
mergify[bot]
cc08ccb420
Merge branch 'master' into remove-useless-code-2 2022-04-30 12:48:15 +00:00
Nikita Mikhaylov
117ffa90cf
Fix ungrammatical error message (#36760) 2022-04-30 13:57:13 +02:00
Azat Khuzhin
d82f2ff162 Set is_all_data_sent on exceptions too 
This will avoid clickhouse-test complains about left queries after the
test, like in [1].

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/36258/9646487c093a75dc31e3e818417cfad83580b40f/stateful_tests__memory__actions_.html

Follow-up for: #36649
 2022-04-30 13:00:44 +03:00
Azat Khuzhin
a75559aa36 Avoid recreation of system.asynchronous_metric_log (due to difference in codec) 
From the log:

    2022.04.29 12:33:40.397528 [ 486 ] {} <Debug> SystemLog (system.asynchronous_metric_log): Existing table system.asynchronous_metric_log for system log has obsolete or different structure. Renaming it to asynchronous_metric_log_352.
    Old: CREATE TABLE system.asynchronous_metric_log (`event_date` Date CODEC(Delta(2), ZSTD(1)), `event_time` DateTime CODEC(Delta(4), ZSTD(1)), `metric` LowCardinality(String) CODEC(ZSTD(1)), `value` Float64 CODEC(Gorilla, ZSTD(3))) ENGINE = MergeTree PARTITION BY toYYYYMM(event_date) ORDER BY (metric, event_date, event_time) SETTINGS index_granularity = 8192
    New: CREATE TABLE system.asynchronous_metric_log (`event_date` Date CODEC(Delta(2), ZSTD), `event_time` DateTime CODEC(Delta(4), ZSTD), `metric` LowCardinality(String) CODEC(ZSTD), `value` Float64 CODEC(Gorilla, ZSTD(3))) ENGINE = MergeTree PARTITION BY toYYYYMM(event_date) ORDER BY (metric, event_date, event_time) SETTINGS index_granularity = 8192

As you can see the difference in default ZSTD level.

P.S. I decided not to fix the printer, since this may introduce changes
in metadata, that may lead to changes in metadata between ZooKeeper and
clickhouse server.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-30 12:19:29 +03:00
Alexey Milovidov
1ddb04b992
Merge pull request #36715 from amosbird/refactorbase 
Reorganize source files so that base won't depend on Common
 2022-04-30 09:40:58 +03:00
mergify[bot]
e9fde5d067
Merge branch 'master' into memory-overcommit-free 2022-04-29 19:31:59 +00:00
Nikolai Kochetov
5807ca47b0
Merge pull request #36425 from ClickHouse/pool-size-hot-reload 
Allow to increase number of threads and tasks for background executors
 2022-04-29 17:24:12 +02:00
Anton Popov
cef462f83d fix creation of tables with flatten_nested = 0 2022-04-29 14:53:08 +00:00
Azat Khuzhin
16b1d2bee4 Remove outdated comment from ThreadStatusExt 
Refs: https://github.com/ClickHouse/ClickHouse/pull/16121#discussion_r511500850

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-29 17:04:56 +03:00
Azat Khuzhin
b7b7d91bd1 Remove memory reservation for SquashingTransform 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-29 17:04:56 +03:00
Vladimir C
e1e7a67670
Merge pull request #36201 from vdimir/join_dup_columns_issue36199 2022-04-29 12:57:10 +02:00
lgbo-ustc
5021cb707c update 2022-04-29 17:41:26 +08:00
vdimir
81c24fc521
Fix assertion in join 2022-04-29 09:15:00 +00:00
Maksim Kita
5b9bb36f80
Merge pull request #36753 from kitaisreal/added-user-defined-path-config-setting 
Added user_defined_path config setting
 2022-04-29 11:10:51 +02:00
Alexey Milovidov
b81da324de
Merge pull request #36765 from ClickHouse/fix_flaky_test_non_leader_replica 
Fix flaky test `test_ddl_worker_non_leader`
 2022-04-29 11:18:49 +03:00
Amos Bird
9d30be2c59
Fix build again 2022-04-29 14:45:27 +08:00
Azat Khuzhin
0ce44f3021 Optimize optimize_aggregation_in_order with a prefix key 
Before it does lots of extra work, now, it will be significantly more
optimal (thousands of rows -> 1-2 million of rows).

v2: s/executeOnBlockSimple/executeOnBlockSmall/
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-29 06:58:28 +03:00
Azat Khuzhin
190ce217bb Disable GROUP BY statistics for optimize_aggregation_in_order 
This statistics significantly decrease performance of
optimize_aggregation_in_order with a prefix key.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-29 06:58:27 +03:00
Azat Khuzhin
767acd53fb Add ability to pass range of rows to Aggregator 
v2: fix compiled aggregate functions (seek result to row_start)
v3: fix compiled aggregate functions (seek args to row_start)
v4: change signatures for JIT
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-29 06:57:55 +03:00
Amos Bird
4a5e4274f0
base should not depend on Common 2022-04-29 10:26:35 +08:00
Maksim Kita
04429d85db Added user_defined_path config setting 2022-04-28 22:13:33 +02:00
Maksim Kita
55994afe0c Fix evaluateConstantExpression for subqueries 2022-04-28 22:09:29 +02:00
Alexander Tokmakov
6797e7dc76 fix flaky test 2022-04-28 21:39:45 +02:00
Anton Popov
1fc51e09ff fix insertion to column of type Object from multiple files via table function 2022-04-28 18:51:13 +00:00
Alexey Milovidov
d8fa806fca
Merge pull request #36360 from ClickHouse/optimize-asynchronous-metric-log 
Optimize `system.asynchronous_metric_log`
 2022-04-28 20:16:17 +03:00
Vadim Volodin
233f0b4b52 Handle cluster macro in s3Cluster table function 2022-04-28 18:38:17 +03:00
Kruglov Pavel
4d08587559
Merge branch 'master' into mysqldump-format 2022-04-28 15:58:18 +02:00
zhanglistar
7d798b3655
Count distinct optimization by using subquery of group by (#35993) 2022-04-28 14:55:37 +02:00
Maksim Kita
ec47abe9af
Merge pull request #36482 from Enmk/fix_session_log_error_on_local_and_tcp_interserver_session 
Fixed missing enum values for ClientInfo::Interface
 2022-04-28 13:06:19 +02:00
Vitaly Baranov
641a5f5e35
Merge pull request #36198 from vitlibar/backup-improvements-3 
Backup for replicated tables and other improvements
 2022-04-28 09:23:53 +02:00
Alexey Milovidov
60e512ac4b Fix for table structure comparison 2022-04-28 05:10:57 +02:00
mergify[bot]
b7e63154df
Merge branch 'master' into optimize-asynchronous-metric-log 2022-04-27 18:27:27 +00:00
Kseniia Sumarokova
0d09bb9897
Merge pull request #36699 from amosbird/bettermysqlhandler 
Slightly better mysql handler
 2022-04-27 17:07:35 +02:00
Amos Bird
1cba00478d
Slightly better mysql handler 2022-04-27 18:28:09 +08:00
lgbo-ustc
10b7e08319 update 2022-04-27 16:41:34 +08:00
lgbo-ustc
7505ce426a update 2022-04-27 13:11:24 +08:00
taiyang-li
0341880250 Merge remote-tracking branch 'origin/master' into hive_text_parallel_parsing 2022-04-27 11:15:16 +08:00
Alexey Milovidov
b8a9f1411a
Merge pull request #36649 from azat/system.processes-is_all_data_sent 
clickhouse-test: improve left queries after the test hardening
 2022-04-27 05:21:22 +03:00
lgbo-ustc
520b05b9f1 update test case tests/queries/0_stateless/02236_explain_pipeline_join.sql 2022-04-27 10:08:22 +08:00
Nikolai Kochetov
e44af67fee Merge branch 'master' into refactor-something-in-part-volumes 2022-04-26 21:08:00 +02:00
Vitaly Baranov
a8e924caf6 Make BACKUP & RESTORE synchronous by default. 2022-04-26 18:45:39 +02:00
avogar
33d845dade Add MySQLDump input format 2022-04-26 10:42:56 +00:00
lgbo-ustc
20fc676bff fixed bug: resize on left pipeline cause the order by result wrong 2022-04-26 18:27:42 +08:00
mergify[bot]
7571387a6e
Merge branch 'master' into memory-overcommit-free 2022-04-26 10:15:18 +00:00
lgbo-ustc
d96c29810a fixed bug: resize on left pipeline cause the order by result wrong 2022-04-26 18:12:14 +08:00
lgbo-ustc
0b0fa8453b fixed bug: resize on left pipeline cause the order by result wrong 2022-04-26 18:06:16 +08:00
Kseniia Sumarokova
81db081b88
Merge pull request #36639 from kssenii/better-drop-cache 
Improve drop cache query
 2022-04-26 11:25:46 +02:00
Azat Khuzhin
6339a48923 Add is_all_data_sent into system.processes 
v2: fix SHOW PROCESSLIST (does not have process list entry)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-26 12:15:00 +03:00
Vitaly Baranov
78bcb96098 Rename backup & restore setting 'async' -> 'sync', and make backup & restore async by default. 2022-04-26 09:51:19 +02:00
taiyang-li
99dee35b6e parallel parsing of hive text format 2022-04-26 14:33:10 +08:00
mergify[bot]
4e82b56a8d
Merge branch 'master' into optimize-asynchronous-metric-log 2022-04-26 05:40:49 +00:00
lgbo-ustc
c7841d226b udpate 2022-04-26 11:29:18 +08:00
lgbo-ustc
74ccc233d2 Merge remote-tracking branch 'ck/master' into concurrent_join 2022-04-26 09:21:02 +08:00
alesapin
92296484e7
Merge pull request #36348 from rschu1ze/erase_if3 
Replace remove-erase idiom by C++20 erase()/erase_if()
 2022-04-25 23:34:18 +02:00
Vitaly Baranov
000b184691 Fix style & compilation. 2022-04-25 23:05:35 +02:00
alesapin
04afe552f8 Trying to avoid segfaults 2022-04-25 22:51:09 +02:00
alesapin
1fffa56f63 Merge branch 'master' into memory-overcommit-free 2022-04-25 20:21:08 +02:00
mergify[bot]
e79de5a577
Merge branch 'master' into optimize-asynchronous-metric-log 2022-04-25 17:29:11 +00:00
kssenii
a76a35d3ce Better drop cache 2022-04-25 19:20:25 +02:00
Vitaly Baranov
ed0af9d4a0 Improve shutdown with async backup/restore. 2022-04-25 16:34:34 +02:00
Vitaly Baranov
042dc4abb2 Rename settings shard -> shard_num, replica -> replica_num. 2022-04-25 16:34:34 +02:00
Vitaly Baranov
129eca7307 CREATE queries generated by RESTORE command is now considered as internal. 2022-04-25 16:34:34 +02:00
Vitaly Baranov
68a020ecea Implement BACKUP/RESTORE ON CLUSTER. 2022-04-25 16:34:33 +02:00
Vitaly Baranov
030f3e488c Add shard_index and replica_index to params of executeDDLQueryOnCluster(). 2022-04-25 16:34:33 +02:00
Vitaly Baranov
3477665659 Add shard_index and replica_index to params of getRewrittenASTWithoutOnCluster(). 2022-04-25 16:34:33 +02:00
Vitaly Baranov
144d3aefeb Add system table system.backups; implement async mode for commands BACKUP & RESTORE 2022-04-25 16:34:33 +02:00
Kseniia Sumarokova
2106a7b895
Merge pull request #36489 from awakeljw/fork_chmaster 
fix ConvertDecimalType unexpected behavior
 2022-04-25 13:23:54 +02:00
lgbo-ustc
c68806fbff fixed typos 2022-04-25 16:16:50 +08:00
lgbo-ustc
981d560553 Merge remote-tracking branch 'ck/master' into concurrent_join 2022-04-25 13:00:04 +08:00
awakeljw
d58a21678a fix ConvertDecimalType unexpected behavior 2022-04-24 23:35:58 +08:00
lgbo-ustc
221bc9c970 update 2022-04-24 17:14:13 +08:00
lgbo-ustc
a38b580c6f update 2022-04-24 15:19:16 +08:00
Alexey Milovidov
90297e225c
Merge pull request #36430 from amosbird/fixcolumnmatcher 
Fix column matcher and column transformer
 2022-04-24 09:11:21 +03:00
Alexey Milovidov
fe004c3486
Merge pull request #36486 from kitaisreal/executable-user-defined-functions-fix-group-by 
ExecutableUserDefinedFunction fix usage in GROUP BY
 2022-04-24 06:11:57 +03:00
Alexey Milovidov
909646247d
Update TranslateQualifiedNamesVisitor.cpp 2022-04-24 05:04:13 +03:00
lgbo-ustc
056b4d1b61 update 2022-04-24 09:26:57 +08:00
Nikolai Kochetov
8c00692844 Part 8 2022-04-22 16:58:09 +00:00
Nikita Mikhaylov
9baf7b14dc Fix build 2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
aaa7112405 Even better 2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
36bdee0499 Deleted settings 2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
4818b63979 Better 2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
8c325c2cdd Better 2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
31ccb9c1c3 Done 2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
224f4dc620
Made parallel_reading_from_replicas work with localhost replica (#36281) 2022-04-22 15:52:38 +02:00
yuuch
d126de6dda
Apply suggestions from code review 
Co-authored-by: Vladimir C <vdimir@clickhouse.com>
 2022-04-22 20:00:12 +08:00
lgbo-ustc
681d7d6799 update dispatchBlock() 2022-04-22 11:00:00 +08:00
Dmitry Novik
6ee62007d3 Handle GROUPING SETS with size < 2 2022-04-21 23:52:43 +00:00
Maksim Kita
583cbcb834
Merge branch 'master' into fix_session_log_error_on_local_and_tcp_interserver_session 2022-04-21 21:15:11 +02:00
Maksim Kita
9713441650
Merge pull request #36480 from Joeywzr/session_log 
add missing enum values in session_log
 2022-04-21 21:14:12 +02:00
Maksim Kita
485c1d4072 Fixed tests 2022-04-21 19:10:11 +02:00
Dmitry Novik
b5e2b38529 Optimize pipeline in case of single input 2022-04-21 14:11:58 +00:00
Maksim Kita
57444fc7d3
Merge pull request #36444 from rschu1ze/clang-tidy-fixes 
Clang tidy fixes
 2022-04-21 16:11:27 +02:00
Dmitry Novik
df63a9b205 Depricate TOTALS with GROUPING SETS 2022-04-21 13:41:59 +00:00
Dmitry Novik
77a82cc090
Merge pull request #35631 from amosbird/projection-fix1 
Fix broken SET reuse during projection analysis.
 2022-04-21 15:32:52 +02:00
mergify[bot]
be6914f6f9
Merge branch 'master' into session_log 2022-04-21 11:52:20 +00:00
Maksim Kita
397603e9be ExecutableUserDefinedFunction fix usage in GROUP BY 2022-04-21 13:44:56 +02:00
Dmitry Novik
ea6a25b8e7 Fix nullptr dereference 2022-04-21 11:14:22 +00:00
Vasily Nemkov
e2866b0541 Fixed missing enum values for ClientInfo::Interface 2022-04-21 12:16:12 +03:00
Memo
ce91a261e8 add missing enum values in session_log 2022-04-21 17:01:44 +08:00
lgbo-ustc
a9d9abb7a1 use column::scatter() 2022-04-21 16:59:30 +08:00
lgbo-ustc
e27d67ce8a update type name 2022-04-21 14:28:22 +08:00
lgbo-ustc
eac810ed85 use concatenateBlocks 2022-04-21 14:10:09 +08:00
lgbo-ustc
60964b207d update addJoinedBlock() 2022-04-21 13:19:33 +08:00
lgbo-ustc
bfd1a0e33a update codes 2022-04-21 12:14:36 +08:00
Dmitry Novik
61deae7105
Merge branch 'master' into grouping-sets-fix 2022-04-21 03:34:42 +02:00
Dmitry Novik
6e73cd6929 Implement parallel grouping sets processing 2022-04-21 01:18:40 +00:00
Alexey Milovidov
ec9b86fd03
Merge branch 'master' into fixcolumnmatcher 2022-04-21 02:15:25 +03:00
Azat Khuzhin
c6ffec1fec Fix "Cannot find column" error for distributed queries with LIMIT BY 
Consider the following example:

    SELECT k FROM (SELECT materialize('foo') AS k, -1 AS v) ORDER BY abs(v) AS _v ASC LIMIT 1 BY k

The problem here is that in case of query has LIMIT BY for
WithMergeableState* (which is the final state on the remote shard for
distributd queries) it returns the following columns:
- k
- v

While it should return:
- k
- abs(v)

So as query w/o LIMIT BY, so that initiator will be able to do the sorting

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-20 15:23:24 +03:00
lgbo-ustc
f461d18de5 update method name 2022-04-20 19:51:35 +08:00
lgbo-ustc
c844415578 add new join_algorithm parallel_hash 2022-04-20 19:47:16 +08:00
Robert Schulze
7550d9b281
Fix build 2022-04-20 13:34:17 +02:00
Alexander Tokmakov
1d30a97fd2 Merge branch 'master' into remove-useless-code-2 2022-04-20 11:45:56 +02:00
Robert Schulze
b24ca8de52
Fix various clang-tidy warnings 
When I tried to add cool new clang-tidy 14 warnings, I noticed that the
current clang-tidy settings already produce a ton of warnings. This
commit addresses many of these. Almost all of them were non-critical,
i.e. C vs. C++ style casts.
 2022-04-20 10:29:05 +02:00
Kseniia Sumarokova
766a84e64f
Merge pull request #35915 from kssenii/better-s3-settings 
Better settings configuration for s3
 2022-04-20 10:22:06 +02:00
Robert Schulze
b6d7367538
Merge remote-tracking branch 'origin/master' into erase_if3 
Conflicts:
- Interpreters/ActionsDAG.cpp
 2022-04-20 10:02:59 +02:00
Antonio Andelic
bbb0be6a44
Merge pull request #36402 from rschu1ze/clang-tidy-contains 
Activate clang-tidy warning "readability-container-contains"
 2022-04-20 08:15:46 +02:00
Kruglov Pavel
894b1b163e
Merge pull request #36055 from helifu/master2 
Don't materialize external tables if it's explain statement
 2022-04-19 20:26:27 +02:00
Kruglov Pavel
7fb7fc9daa
Merge pull request #36275 from Avogar/fix-insert-select 
Place new logic of schema inference in insert select from table function under setting
 2022-04-19 20:19:28 +02:00
Amos Bird
158a25d5fa
Fix column matcher and column transformer 2022-04-20 01:22:04 +08:00
Robert Schulze
72a331189e
Fix glitch 2022-04-19 14:59:47 +02:00
mergify[bot]
3c1dd44b80
Merge branch 'master' into fix-insert-select 2022-04-19 12:17:31 +00:00
lgbo-ustc
26ff749a32 remove unused header 2022-04-19 19:12:08 +08:00
lgbo-ustc
3f2fe3ff66 fixed code style 2022-04-19 18:47:29 +08:00
kssenii
1f9a597b77 Merge master 2022-04-19 11:31:13 +02:00
lgbo-ustc
9dfe2a6682 fixed header style 2022-04-19 16:53:24 +08:00
lgbo-ustc
3d7338581b Improve join 
now adding joined blocks from right table can be run parallelly, speedup the join process
 2022-04-19 16:07:30 +08:00
Robert Schulze
118e94523c
Activate clang-tidy warning "readability-container-contains" 
This check suggests replacing <Container>.count() by
<Container>.contains() which is more speaking and in case of
multimaps/multisets also faster.
 2022-04-18 23:53:11 +02:00
Alexey Milovidov
1e1ffbbfe2
Update SessionLog.cpp 2022-04-18 23:07:34 +03:00
mergify[bot]
daca157111
Merge branch 'master' into fix_storage_distributed_ttl 2022-04-18 17:31:59 +00:00
Alexey Milovidov
0af183826d
Merge pull request #36358 from ClickHouse/don-t-randomize-priority 
Do not randomize "priority" setting
 2022-04-18 07:04:36 +03:00
Alexey Milovidov
f6ab2bd523
Merge pull request #36312 from ClickHouse/remove-arcadia 
Remove remaining parts of Arcadia
 2022-04-18 07:02:54 +03:00
Alexey Milovidov
36595e4206
Merge pull request #36320 from ClickHouse/fix-clang-tidy-14 
Fix clang-tidy-14 (part 1)
 2022-04-18 07:02:10 +03:00
Alexey Milovidov
70d7ba5d61 Fix clang-tidy 2022-04-18 02:13:28 +02:00
Alexey Milovidov
6de5d8a554 Merge branch 'master' into fix-clang-tidy-14 2022-04-18 02:12:45 +02:00
Alexey Milovidov
0051e651c6
Update AsynchronousMetricLog.h 2022-04-18 03:01:57 +03:00
Alexey Milovidov
242919eddd Remove abbreviation 2022-04-18 01:02:49 +02:00
Alexey Milovidov
8ea14d2664 Custom ORDER BY for system.asynchronous_metric_log 2022-04-17 23:59:05 +02:00
Alexey Milovidov
e36e22812a Custom column list for system.asynchronous_metric_log 2022-04-17 23:53:28 +02:00
Alexey Milovidov
300ef741cc Custom column list for system.asynchronous_metric_log 2022-04-17 23:51:56 +02:00
Alexey Milovidov
4c09e2d30a Custom column list for system.asynchronous_metric_log 2022-04-17 23:49:39 +02:00
Alexey Milovidov
37c61d9f0a Round values for the log 2022-04-17 22:07:50 +02:00
Alexey Milovidov
1f5df287c0 Remove useless code 2022-04-17 22:02:55 +02:00
Alexey Milovidov
6807f8428d Remove microseconds from system.asynchronous_metric_log 2022-04-17 22:01:58 +02:00
Alexey Milovidov
250ca83058 Allow custom default ORDER BY for system logs 2022-04-17 21:57:44 +02:00
Alexey Milovidov
a64765bc6f Allow custom default ORDER BY for system logs 2022-04-17 21:56:36 +02:00
Alexey Milovidov
aa93f4a074 Do not randomize "priority" setting 2022-04-17 20:48:02 +02:00
Alexey Milovidov
89deaf0af1
Merge pull request #36056 from azat/client-trailing-space 
Remove trailing whitespace in default client_name
 2022-04-17 17:37:43 +03:00
Robert Schulze
1e1df8e101
Replace remove-erase idiom by C++20 erase()/erase_if() 
- makes the code less verbose while being as efficient
 2022-04-17 12:04:47 +02:00
Alexey Milovidov
294efeccfe Fix clang-tidy-14 (part 1) 2022-04-16 04:54:04 +02:00
Alexey Milovidov
cbeeb7ec4f Remove Arcadia 2022-04-16 00:20:47 +02:00
Alexey Milovidov
a0fe29e243 Remove Arcadia 2022-04-15 23:59:49 +02:00
yuuch
6919ca9a6e remove parse about the not null in ParserCollation 2022-04-15 21:31:25 +08:00
avogar
ae01646136 Place new logic of schema inference in insert select from table function under setting 2022-04-14 19:05:20 +00:00
Dmitry Novik
a16710c750 Merge remote-tracking branch 'origin/master' into grouping-sets-fix 2022-04-14 17:29:51 +00:00
yuuch
29e0b4c3bb support collation in create query 2022-04-15 00:12:31 +08:00
Alexander Tokmakov
4ce0b280e6 wait for node to disappear instead of removing 2022-04-14 15:31:05 +02:00
Anton Popov
305dd57262
Merge branch 'master' into fix_storage_distributed_ttl 2022-04-14 14:51:15 +02:00
mergify[bot]
95901466e7
Merge branch 'master' into master2 2022-04-14 10:05:03 +00:00
Yakov Olkhovskiy
c1a06ac63a
Merge pull request #24896 from hexiaoting/dev_materialized_view 
Add type check when create materialized view with To clause
 2022-04-13 22:33:54 -04:00
Maksim Kita
ca89af89a1
Merge pull request #36088 from israelg99/fix_executable 
Fix CH silently failing when it cannot execute a file
 2022-04-13 16:53:00 +02:00
kssenii
0912b038fa Merge master 2022-04-13 15:30:01 +02:00
tavplubix
43de4827d7
Merge pull request #36074 from ClickHouse/zookeeper_log_thread_query_id 
Add thread_id and query_id to zookeeper_log
 2022-04-13 15:34:04 +03:00
mergify[bot]
ba22cb76bb
Merge branch 'master' into master2 2022-04-13 11:51:43 +00:00
Kruglov Pavel
03fc08cec3
Merge pull request #36137 from azat/client-profile-events 
ProfileEvents fixes
 2022-04-13 12:27:39 +02:00
Yakov Olkhovskiy
3d52afed71 only check columns sets intersection, some refactoring 2022-04-12 15:10:11 -04:00
tavplubix
303ecc6f7d
Merge pull request #36136 from ClickHouse/transactions_fix_1 
Minor fix for transactions
 2022-04-12 21:38:47 +03:00
Azat Khuzhin
e38057ddb0 Fix code alignment in ProfileEventsExt 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-12 20:54:56 +03:00
mergify[bot]
d5b2355944
Merge branch 'master' into memory-overcommit-free 2022-04-12 16:00:12 +00:00
Julian Gilyadov
a4f56f3330
Throw exception when file cant be executed instead of displaying success 2022-04-12 17:52:44 +02:00
Yakov Olkhovskiy
5868539605 check only if TO table exists 2022-04-12 10:17:39 -04:00
mergify[bot]
0b3c15c07a
Merge branch 'master' into projection-fix1 2022-04-12 13:49:28 +00:00
Yakov Olkhovskiy
25101de7e2 check only selected columns against destination table 2022-04-12 09:29:32 -04:00
tavplubix
2e95e0db3b
Merge pull request #36133 from Algunenano/osx_atomic_exchange 
Add support for atomic exchange in OSX
 2022-04-12 14:10:36 +03:00
Alexander Tokmakov
6e4787560b fix another issue 2022-04-12 11:44:25 +02:00
Yakov Olkhovskiy
dc314d53b3 do not create view before check, use MatchColumnsMode::Name mode 2022-04-11 20:28:11 -04:00
mergify[bot]
46b9a1ef80
Merge branch 'master' into client-trailing-space 2022-04-11 22:07:37 +00:00
Yakov Olkhovskiy
783dbb2e63
Merge branch 'master' into dev_materialized_view 2022-04-11 17:47:35 -04:00
Yakov Olkhovskiy
155a2a0d42
Merge pull request #35349 from yakov-olkhovskiy/interpolate-feature 
Interpolate feature
 2022-04-11 11:15:50 -04:00
Yakov Olkhovskiy
2588f8011f comment fix 2022-04-11 11:13:29 -04:00
Alexander Tokmakov
38c80940fd slightly beter fix 2022-04-11 16:25:59 +02:00
Yakov Olkhovskiy
7293e01647 some comments added 2022-04-11 09:47:27 -04:00
Raúl Marín
6e346a7fc3 Support atomic replace under OSX 2022-04-11 14:32:55 +02:00
Vladimir C
1a604cd279
Merge pull request #36058 from xiedeyantu/master 2022-04-11 10:45:50 +02:00
Vitaly Baranov
8dcb1358b7
Merge pull request #36072 from vitlibar/backups-improve-arguments-handling-and-file-removing 
Backups: improve arguments handling and file removing
 2022-04-10 16:51:04 +02:00
xiedeyantu
5efd8a4ec0 If data type has DEFAULT NULL in table definition, make data type Nullable 2022-04-09 15:00:41 +08:00
Vitaly Baranov
758f8235c2 Collect backup entries before opening an archive for writing. 2022-04-09 01:44:46 +02:00
Alexander Tokmakov
50a32a74b7 add thread_id and query_id to zookeeper_log 2022-04-08 17:10:08 +02:00
Yakov Olkhovskiy
87c2b3e2cf fixed Nullable, tests added 2022-04-08 10:52:10 -04:00
Alexander Tokmakov
141fbc6980 Merge branch 'master' into mvcc_prototype 2022-04-08 13:38:11 +02:00
Alexander Tokmakov
49c35f3261 Merge branch 'master' into mvcc_prototype 2022-04-08 13:34:40 +02:00
Vitaly Baranov
45dea9933b
Merge pull request #35883 from azat/format-settings 
Do not allow SETTINGS after FORMAT for INSERT queries
 2022-04-08 12:38:14 +02:00
kssenii
87855c4ae7 Merge master 2022-04-08 12:32:48 +02:00
alesapin
8ec802bc62
Merge pull request #35475 from kssenii/remote-fs-cache-improvements 
Allow to write remote fs cache on all write operations. Add `system.remote_filesystem_cache` table. Add `drop remote filesystem cache (<path>)` query. Add `system.remote_data_paths` table.
 2022-04-08 12:06:26 +02:00
Vladimir C
265fa90689
Merge pull request #36033 from vdimir/cross-join-empty-list 2022-04-08 11:59:21 +02:00
Azat Khuzhin
44bf8ba568 Remove trailing whitespace in default client_name 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-08 08:09:47 +03:00
helifu
1196dc3dbd Don't materialize external tables if it's explain statement 
The explain statement of GLOBAL JOIN two distributed tables can speed up 100x:
  explain plan select ... from t1_dist global join t2_dist on ...
  explain pipeline select ... from t1_dist global join t2_dist on ...
 2022-04-08 09:00:55 +08:00
Yakov Olkhovskiy
64dcddc6e3 fixed ASTInterpolateElement::clone, fixed QueryNormalizer to exclude ASTInterpolateElement::children 2022-04-07 17:41:05 -04:00
Alexander Tokmakov
6a46da93ae Merge branch 'master' into mvcc_prototype 2022-04-07 23:22:19 +02:00
kssenii
5dce2f18b5 Better 2022-04-07 18:48:35 +02:00
Alexander Tokmakov
2e2e2b9190 fix a race condition 2022-04-07 18:17:43 +02:00
Azat Khuzhin
33d99c8ffb Introduce compatiblity mode for SETTINGS after FORMAT 
Add allow_settings_after_format_in_insert setting, OFF by default.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
v2: s/parser_settings_after_format_compact/allow_settings_after_format_in_insert/ (suggested by vitlibar)
v3: replace ParserSettings with a flag (requested by vitlibar)
 2022-04-07 16:36:34 +03:00
Alexander Tokmakov
7f54e7b422 Merge branch 'master' into mvcc_prototype 2022-04-07 15:14:06 +02:00
Kruglov Pavel
73adbb4c15
Merge pull request #35986 from amosbird/better-scalar1 
Fix performance regression of scalar query
 2022-04-07 14:07:59 +02:00
Alexander Tokmakov
8290ffa88d Merge branch 'master' into mvcc_prototype 2022-04-07 13:50:42 +02:00
vdimir
eb7ac4c3a6
Handle empty expression list in cross join 2022-04-07 11:32:32 +00:00
vdimir
69160fa3e3
remove dead code in needRewrite function 2022-04-07 10:46:39 +00:00
tavplubix
6e9c028bbb
Merge pull request #35956 from Algunenano/limit_dns_retries 
DNS cache: Add option to drop elements after several consecutive failures
 2022-04-07 13:20:36 +03:00
Nikolai Kochetov
3e1b3f14c0
Merge pull request #34355 from azat/processors-profiling 
Profiling on Processors level
 2022-04-07 12:13:14 +02:00
Amos Bird
1238babb6f
Make SelectQueryInfo pseudo-copyable 2022-04-07 17:46:50 +08:00
Amos Bird
69022850b9
Better 2022-04-07 17:46:50 +08:00
Amos Bird
9cf5935604
Fix broken SET reuse during projection analysis. 2022-04-07 17:46:49 +08:00
Yakov Olkhovskiy
7dbe8bc2dc major bugs fixed, tests added, docs updated 2022-04-07 01:21:24 -04:00
mergify[bot]
585cff1e95
Merge branch 'master' into limit_dns_retries 2022-04-06 18:23:17 +00:00
tavplubix
58db8cd9b3
Merge pull request #35946 from ClickHouse/faster_replicated_ddl 
Make some replicated DDL queries faster
 2022-04-06 19:25:25 +03:00
Amos Bird
df06f9f974
Fix performance regression of scalar query 2022-04-06 17:50:22 +08:00
Yakov Olkhovskiy
ac441b9dcf compiler suggestions 2022-04-06 00:05:06 -04:00
Yakov Olkhovskiy
90c4cd3de7
Merge branch 'master' into interpolate-feature 2022-04-05 14:39:07 -04:00
Yakov Olkhovskiy
e0d6033c39 all columns can participate in interpolate expression despite if they are selected or not, some optimization on expressionless INTERPOLATE 2022-04-05 14:26:49 -04:00
Alexander Tokmakov
37a06eec1a fixes 2022-04-05 17:36:53 +02:00
Alexander Tokmakov
d9e3e9b69f Merge branch 'master' into faster_replicated_ddl 2022-04-05 14:38:38 +02:00
Alexander Tokmakov
1fe50ad201 Merge branch 'master' into mvcc_prototype 2022-04-05 14:38:02 +02:00
Maksim Kita
c3c284e6e6
Merge pull request #35917 from kitaisreal/executable-user-defined-function-fix 
Executable user defined functions prevent executing during analysis
 2022-04-05 14:05:52 +02:00
Raúl Marín
8a05cf3927 DNS cache: Add option to drop elements after several consecutive failures 2022-04-05 13:00:27 +02:00
Nikolai Kochetov
4479b68980
Merge pull request #35623 from nickitat/function_calculation_after_sorting_and_limit 
Functions calculation after sorting
 2022-04-05 12:09:56 +02:00
alesapin
eaa72ef002
Merge pull request #35904 from ClickHouse/more_logs_on_unsuccessful_part_removal 
More logs on unsuccessful part removal
 2022-04-05 11:29:08 +02:00
Alexander Tokmakov
da00beaf7f Merge branch 'master' into mvcc_prototype 2022-04-05 11:14:42 +02:00
Alexander Tokmakov
4e9ec5dc2f make some replicated DDL faster 2022-04-05 00:51:48 +02:00
Alexey Milovidov
4d6c030d23
Revert "clang-tidy report issues with Medium priority" 2022-04-04 23:41:42 +03:00
Vladimir C
b80f24957e
Merge pull request #35674 from vdimir/asof-key-null-issue-35565 2022-04-04 16:56:04 +02:00
kssenii
4e50da722b Better s3 settings 2022-04-04 16:14:56 +02:00
Maksim Kita
2a8e479277 ExecutableUserDefinedFunction prevent function execution during query analysis 2022-04-04 15:56:01 +02:00
Nickita Taranov
698a984c07 throw if sorting column not found 2022-04-04 14:33:58 +02:00
Nickita Taranov
b095838444 stash 2022-04-04 14:33:57 +02:00
Nickita Taranov
4c51329ad6 stash 2022-04-04 14:33:57 +02:00
Alexander Tokmakov
a2167f12b8 Merge branch 'master' into mvcc_prototype 2022-04-04 14:24:23 +02:00
Nikita Taranov
bd89fcafdb
Make SortDescription::column_name always non-empty (#35805) 2022-04-04 14:17:15 +02:00
alesapin
885447342c More logs on unsuccessful part removal 2022-04-04 13:17:33 +02:00
Azat Khuzhin
e5a837e574 Add id/parent_ids into system.processors_profile_log 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-04 13:19:43 +03:00
Azat Khuzhin
99528e296c Rename need_data_elapsed_us/port_full_elapsed_us to input_wait_us/output_wait_us 
$ gg -e need_data_ -e port_full_  | cut -d: -f1 | sort -u | xargs sed -i -e s/port_full_/output_wait_/g -e s/need_data_/input_wait_/g -e s/getPortFull/getOutputWait/g -e s/getNeedData/getInputWait/g

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-04 13:19:43 +03:00
Azat Khuzhin
765b4bc45a Implement log_processors_profiles (write to processors_profile_log, OFF by default) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-04 13:19:43 +03:00
Azat Khuzhin
14538f6456 Add system.processors_profile_log 
This is the system table that will contain Processors level profiling.

v2: one entry per Processor, not 3 (PortFull/NeedData/work())
v3: us over ms
v4: Enable processors_profile_log table by default

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-04 13:19:41 +03:00
kssenii
36c583d0de Better version of cache on insert 2022-04-04 10:58:59 +02:00
Yakov Olkhovskiy
ff4d295265 style fix 2022-04-03 22:19:35 -04:00
Yakov Olkhovskiy
95ad1bf6e1 use aliases if exist for original_select_set 2022-04-03 22:10:36 -04:00
Alexey Milovidov
5a47958744
Merge pull request #35736 from CurtizJ/quota-written-bytes 
Add quota for written bytes
 2022-04-03 05:26:49 +03:00
Kruglov Pavel
687e9e7aeb
Merge pull request #35760 from Avogar/insert-from-input 
Improve schema inference for insert select from table functions
 2022-04-02 13:50:48 +02:00
Yakov Olkhovskiy
ec0ad8804a style fix 2022-04-01 21:45:58 -04:00
Yakov Olkhovskiy
0116233d36 allow INTERPOLATE to reference optimized out columns 2022-04-01 16:18:19 -04:00
Anton Popov
a10bf550c4
Update AsynchronousInsertQueue.cpp 2022-04-01 20:57:41 +02:00
Anton Popov
687942ce70 more strict quota for written bytes 2022-04-01 15:02:49 +00:00
Yakov Olkhovskiy
38993f215f
Merge pull request #35706 from yakov-olkhovskiy/ephemeral-35641 
Allow EPHEMERAL without explicit default expression
 2022-04-01 10:49:29 -04:00
kssenii
6c8e073a61 Merge master 2022-04-01 16:48:29 +02:00
Alexander Tokmakov
6591d1ceb7 Merge branch 'master' into mvcc_prototype 2022-04-01 15:38:46 +02:00
Yakov Olkhovskiy
a15996315e bugfix - columns order tracking 2022-03-31 11:51:13 -04:00
Yakov Olkhovskiy
b5682c1f02 minor refactoring 2022-03-31 08:33:50 -04:00
Kruglov Pavel
a217e9e9d0
Remove unused header 2022-03-31 14:27:43 +02:00
Alexander Tokmakov
461ecd56a4 fix race when database is Ordinary 2022-03-31 13:47:38 +02:00
Alexander Tokmakov
5a50ad9de3 Merge branch 'master' into mvcc_prototype 2022-03-31 11:35:04 +02:00
Antonio Andelic
e336302650 Merge branch 'master' into is-secure-client-info 2022-03-31 06:58:16 +00:00
Antonio Andelic
049559d510 Remove extra newline 2022-03-31 06:57:59 +00:00
Antonio Andelic
cdedcee0ce
Merge pull request #35746 from ClickHouse/use-common-io-thread-pool 
Use common IOThreadPool for S3 and URL
 2022-03-31 08:41:54 +02:00
Yakov Olkhovskiy
3b40b56e56 refactored to remove parsers rependency on datatypes 2022-03-30 23:47:16 -04:00
Nikita Taranov
30f2a942c5
Predict size of hash table for GROUP BY (#33439) 
* use AggregationMethod ctor with reserve

* add new settings

* add HashTablesStatistics

* support queries with limit

* support distributed and with external aggregation

* add new profile events

* add some tests

* add perf test

* export cache stats through AsynchronousMetrics

* rm redundant trace

* fix style

* fix 02122_parallel_formatting test

* review fixes

* fix 02122_parallel_formatting test

* apply also to two-level HTs

* try simpler strategy

* increase max_size_to_preallocate_for_aggregation for experiment

* fixes

* Revert "increase max_size_to_preallocate_for_aggregation for experiment"

This reverts commit 6cf6f75704.

* fix test

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2022-03-30 22:47:51 +02:00
Vasily Nemkov
dc88d8d571 Explicitly passing a user object to create a LogIn event 2022-03-30 23:37:24 +03:00
Yakov Olkhovskiy
6a1e116c46 refactoring 2022-03-30 16:34:19 -04:00
Alexander Tokmakov
996d57b918 try fix build with gcc 2022-03-30 17:42:47 +02:00
kssenii
0fc92fe2aa Get rid of all "remote" mentions 2022-03-30 17:15:29 +02:00
Vasily Nemkov
31e07fe078 Fixed crash when reporting successful login. 2022-03-30 17:14:55 +03:00
avogar
849596c7a2 Improve schema inference for insert select queries 2022-03-30 12:44:53 +00:00
Antonio Andelic
de2c6684a3 Remove setup and cleanup 2022-03-30 09:00:06 +00:00
Antonio Andelic
1173eab15e Use same IOThreadPool for URL and S3 2022-03-30 08:15:20 +00:00
Antonio Andelic
d14ad227b1 Polish TCP is_secure flag 2022-03-30 06:39:40 +00:00
Antonio Andelic
b982959c4c Fix writting is_secure 2022-03-30 06:39:40 +00:00
Antonio Andelic
893b8a9513 Initial implementation of is_secure 2022-03-30 06:39:40 +00:00
Anton Popov
caacc7d385 add quota for written bytes 2022-03-29 18:21:29 +00:00
Alexander Tokmakov
287d858fda Merge branch 'master' into mvcc_prototype 2022-03-29 16:24:12 +02:00
Antonio Andelic
9990abb76a Use compile-time check for Exception messages, fix wrong messages 2022-03-29 13:16:11 +00:00
Anton Popov
9610139477
Merge pull request #35629 from CurtizJ/dynamic-columns-5 
Support schema inference for type `Object` in format `JSONEachRow`
 2022-03-29 14:17:09 +02:00
Anton Popov
d42632dc13
Merge pull request #32928 from bigo-sg/rocksdb_metacache 
Speed up parts loading process of MergeTree by metadata cache
 2022-03-29 14:05:22 +02:00
Maksim Kita
a1a4552740
Merge pull request #35184 from DevTeamBK/clang-tidy-issues 
clang-tidy report issues with Medium priority
 2022-03-29 13:19:54 +02:00
taiyang-li
834ac3a1b9 Merge remote-tracking branch 'ck/master' into rocksdb_metacache 2022-03-29 18:32:06 +08:00
Anton Popov
5d8988ad4e
Merge pull request #35645 from CurtizJ/fix-quotas-async-inserts 
Fix usage of quota with async inserts
 2022-03-29 11:54:24 +02:00
Alexey Milovidov
5e262fba85
Merge pull request #35204 from azat/build-gcc 
Add build with GCC
 2022-03-29 04:55:15 +03:00
kssenii
44157e5c80 Merge master 2022-03-29 02:19:32 +02:00
Yakov Olkhovskiy
615efa1381 aliases processing fixed 2022-03-28 19:15:53 -04:00
Azat Khuzhin
bf4df5c6bb Fix SIGSEGV for build under gcc-11 (due to auto deduction) 
During building with gcc-11 you will got SIGSEGV for building
InterpretersMySQLDDLQuery (and some others), and it is due to endless
recursion:

    (gdb) bt 5
    0  0x00000000010978f2 in structural_comptypes (t1=0x7fde028c7dc8, t2=0x7fde028d1e70, strict=0) at ../../src/gcc/cp/typeck.c:1274
    1  0x00000000011c3f9d in comp_template_parms (parms1=<optimized out>, parms2=<optimized out>) at ../../src/gcc/cp/pt.c:3369
    2  0x0000000001097bd9 in structural_comptypes (t1=0x7fde028c7dc8, t2=0x7fde028d1e70, strict=<optimized out>) at ../../src/gcc/cp/typeck.c:1361
    3  0x00000000011c3f9d in comp_template_parms (parms1=<optimized out>, parms2=<optimized out>) at ../../src/gcc/cp/pt.c:3369
    4  0x0000000001097bd9 in structural_comptypes (t1=0x7fde028c7dc8, t2=0x7fde028d1e70, strict=<optimized out>) at ../../src/gcc/cp/typeck.c:1361
    (gdb) bt -X
    1397454 0x0000000001097bd9 in structural_comptypes (t1=0x7fde028d1540, t2=0x7fde028d27e0, strict=<optimized out>) at ../../src/gcc/cp/typeck.c:1361
    1397455 0x0000000000f2d8b5 in cp_tree_equal (t1=<optimized out>, t2=<optimized out>) at ../../src/gcc/cp/tree.c:4144
    1397456 0x00000000010909cb in template_args_equal (ot=0x7fde028cf578, nt=0x7fde028cfc58, partial_order=<optimized out>) at ../../src/gcc/cp/pt.c:9256
    1397457 0x0000000001090422 in template_args_equal (partial_order=false, nt=0x7fde028cfc58, ot=0x7fde028cf578) at ../../src/gcc/cp/pt.c:9295
    1397458 comp_template_args (oldargs=0x7fde028cf550, newargs=0x7fde028cfc30, oldarg_ptr=0x0, newarg_ptr=0x0, partial_order=false) at ../../src/gcc/cp/pt.c:9285
    1397459 0x00000000010a08f4 in spec_hasher::equal (e1=0x7fde028c95d0, e2=0x7ffd1194e8c0) at ../../src/gcc/cp/pt.c:1726
    1397460 0x0000000001085965 in hash_table<spec_hasher, false, xcallocator>::find_with_hash (this=0x7fde36b7f450, comparable=@0x7ffd1194e8b8: 0x7ffd1194e8c0, hash=<optimized out>) at ../../src/gcc/hash-table.h:936
    1397461 0x0000000001079698 in lookup_template_class_1 (d1=<optimized out>, arglist=0x7fde028cfc30, in_decl=0x0, context=<optimized out>, entering_scope=<optimized out>, complain=3) at ../../src/gcc/cp/pt.c:9896
    1397462 0x000000000109f8ef in lookup_template_class (complain=3, entering_scope=1, context=0x7fde27558e40, in_decl=0x0, arglist=0x7fde028cfc08, d1=0x7fde269bcd20) at ../../src/gcc/cp/pt.c:10251
    1397463 tsubst_aggr_type (t=0x7fde269bcd20, args=<optimized out>, complain=3, in_decl=0x0, entering_scope=1) at ../../src/gcc/cp/pt.c:13646
    1397464 0x000000000108f797 in tsubst (t=0x7fde269bcdc8, args=0x7fde028cf7a8, complain=3, in_decl=<optimized out>) at ../../src/gcc/cp/pt.c:16108
    1397465 0x0000000000e61bf2 in rewrite_template_parm (level=1, complain=3, tsubst_args=0x7fde028cf7a8, index=5, olddecl=0x7fde269b5600) at ../../src/gcc/cp/pt.c:28556
    1397466 rewrite_tparm_list(tree_node*, unsigned int, unsigned int, tree_node*, unsigned int, int) [clone .constprop.0] (oldelt=0x7fde269bd190, index=5, targs=0x7fde028cf7a8, targs_index=4, complain=3, level=1) at ../../src/gcc/cp/pt.c:28640
    1397467 0x00000000009f3748 in build_deduction_guide (type=type@entry=0x7fde26e13dc8, ctor=0x7fde269ac300, outer_args=outer_args@entry=0x0, complain=complain@entry=3) at ../../src/gcc/cp/pt.c:28769
    1397468 0x00000000009f444f in ctor_deduction_guides_for (complain=3, tmpl=<optimized out>) at ../../src/gcc/cp/cp-tree.h:842
    1397469 deduction_guides_for (tmpl=<optimized out>, any_dguides_p=<optimized out>, complain=3) at ../../src/gcc/cp/pt.c:29282
    1397470 0x00000000008507a8 in do_class_deduction (complain=3, flags=1, init=<optimized out>, tmpl=0x7fde26e0f980, ptype=0x7fde028c7b28) at ../../src/gcc/cp/pt.c:29402
    1397471 do_auto_deduction (type=0x7fde028c7b28, init=<optimized out>, auto_node=<optimized out>, complain=3, context=<optimized out>, outer_targs=<optimized out>, flags=1) at ../../src/gcc/cp/pt.c:29572
    1397472 0x00000000007c9569 in finish_compound_literal (type=<optimized out>, compound_literal=0x7fde028c95b8, complain=3, fcl_context=fcl_functional) at ../../src/gcc/cp/semantics.c:3060
    1397473 0x0000000001123a79 in cp_parser_functional_cast (parser=0x7fde27558da8, type=0x7fde028c7b28) at ../../src/gcc/cp/parser.c:30670
    1397474 0x0000000000fd7873 in cp_parser_postfix_expression (parser=0x7fde27558da8, address_p=<optimized out>, cast_p=<optimized out>, member_access_only_p=<optimized out>, decltype_p=false, pidk_return=0x0) at ../../src/gcc/cp/parser.c:7437
    1397475 0x0000000000fd4ddf in cp_parser_binary_expression (parser=0x7fde27558da8, cast_p=<optimized out>, no_toplevel_fold_p=false, decltype_p=<optimized out>, prec=PREC_NOT_OPERATOR, pidk=<optimized out>) at ../../src/gcc/cp/parser.c:9842
    1397476 0x0000000000fd4595 in cp_parser_assignment_expression (parser=0x7fde27558da8, pidk=<optimized out>, cast_p=<optimized out>, decltype_p=<optimized out>) at ../../src/gcc/cp/parser.c:10146
    1397477 0x0000000000fd3b90 in cp_parser_constant_expression (parser=0x7fde27558da8, allow_non_constant_p=2, non_constant_p=0x7ffd1194f1d7, strict_p=<optimized out>) at ../../src/gcc/cp/parser.c:10449
    1397478 0x0000000000fcfdd5 in cp_parser_initializer_clause (non_constant_p=<optimized out>, parser=0x7fde27558da8) at ../../src/gcc/cp/parser.c:24253
    1397479 cp_parser_initializer (parser=0x7fde27558da8, is_direct_init=<optimized out>, non_constant_p=<optimized out>, subexpression_p=<optimized out>) at ../../src/gcc/cp/parser.c:24193
    1397480 0x000000000062e5d8 in cp_parser_decomposition_declaration (init_loc=0x7ffd1194f1d8, maybe_range_for_decl=0x7ffd1194f498, decl_specifiers=0x7ffd1194f1f0, parser=0x7fde27558da8) at ../../src/gcc/cp/parser.c:14734
    1397481 cp_parser_simple_declaration (parser=0x7fde27558da8, function_definition_allowed_p=<optimized out>, maybe_range_for_decl=0x7ffd1194f498) at ../../src/gcc/cp/parser.c:14393
    1397482 0x000000000109b870 in cp_parser_init_statement (parser=0x7fde27558da8, decl=0x7ffd1194f498) at ../../src/gcc/cp/parser.c:13420
    1397483 0x00000000010996f0 in cp_parser_for (unroll=0, ivdep=false, parser=0x7fde27558da8) at ../../src/gcc/cp/parser.c:12708
    1397484 cp_parser_iteration_statement (parser=0x7fde27558da8, if_p=0x0, ivdep=<optimized out>, unroll=<optimized out>) at ../../src/gcc/cp/parser.c:13343
    1397485 0x0000000000fe5c46 in cp_parser_statement (parser=0x7fde27558da8, in_statement_expr=0x0, in_compound=<optimized out>, if_p=0x0, chain=0x0, loc_after_labels=0x0) at ../../src/gcc/cp/parser.c:11718
    1397486 0x0000000000fe15ac in cp_parser_statement_seq_opt (in_statement_expr=<optimized out>, parser=<optimized out>) at ../../src/gcc/cp/parser.c:12201
    1397487 cp_parser_compound_statement (parser=0x7fde27558da8, in_statement_expr=0x0, bcs_flags=<optimized out>, function_body=<optimized out>) at ../../src/gcc/cp/parser.c:12150

Interesting frame is 1397471, from which we can extract location:

    (gdb) p line_table[0].info_ordinary.maps[line_table[0].info_ordinary.cache]
    $54 = {
      <line_map> = {
        start_location = 1396581280
      },
      members of line_map_ordinary:
      reason = LC_RENAME,
      sysp = 0 '\000',
      m_column_and_range_bits = 8,
      m_range_bits = 0,
      to_file = 0x3eb4bb0 "/ch/src/Interpreters/MySQL/InterpretersMySQLDDLQuery.cpp",
      to_line = 46,
      included_from = 0
    }

By replicating SOURCE_LINE() macro from gcc-11 (libcpp/include/line-map.h):

    /* Converts a map and a location_t to source line.  */
    inline linenum_type
    SOURCE_LINE (const line_map_ordinary *ord_map, location_t loc)
    {
      return ((loc - ord_map->start_location)
          >> ord_map->m_column_and_range_bits) + ord_map->to_line;
    }

We got line 154:

    (gdb) p ((input_location-1396581280) >> 8) + 46
    $61 = 154

And this is:

    auto [column_name_and_type, declare_column_ast] = std::tuple{columns_name_and_type.begin(), columns_definition->children.begin()};

After rewriting it, everything works correctly.

Also here is a reproducer for gcc-11 (and no failures for gcc-12, but
gcc-12 has other issues, like [1] and one more for hash table):

    # cat /tmp/test.cpp
    #include <tuple>

    auto multi()
    {
            return std::tuple{1, 1};
    }
    double foo()
    {
            auto [a, b] = multi();
            return a - b;
    }

    # g++-11 -std=gnu++20 -c -o /dev/null -isystem /ch/contrib/libcxx/include -nostdinc++ /tmp/test.cpp
    g++-11: internal compiler error: Segmentation fault signal terminated program cc1plus
    Please submit a full bug report,
    with preprocessed source if appropriate.
    See <file:///usr/share/doc/gcc-11/README.Bugs> for instructions.
    # g++-12 -std=gnu++20 -c -o /dev/null -isystem /ch/contrib/libcxx/include -nostdinc++ /tmp/test.cpp

  [1]: https://reviews.llvm.org/D122598

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-28 22:21:45 +03:00
Vladimir C
ecdd89e811
Merge pull request #35666 from vdimir/fix-fuse 2022-03-28 20:40:43 +02:00
Alexander Tokmakov
208b242188 Merge branch 'master' into mvcc_prototype 2022-03-28 19:58:06 +02:00
vdimir
a216bc26c1
Correct check asof join key nullability 2022-03-28 13:29:34 +00:00
taiyang-li
38702be40c Merge remote-tracking branch 'ck/master' into rocksdb_metacache 2022-03-28 21:03:22 +08:00
mergify[bot]
3924609ab2
Merge branch 'master' into metric-async-insert-bytes 2022-03-28 11:37:49 +00:00
vdimir
1122db89db
Use float devision for avg after optimize_fuse_sum_count_avg 2022-03-28 11:26:42 +00:00
Kseniia Sumarokova
482a5e362b
Merge pull request #35620 from kssenii/fix-positional-arguments-with-aliases 
Fix positional arguments
 2022-03-28 09:18:49 +02:00
Anton Popov
4ddfff3084 fix usage of quota with async inserts 2022-03-28 00:46:35 +00:00
Alexey Milovidov
bb35184da1 Add metric about size of async INSERTs 2022-03-28 02:04:19 +02:00
Yakov Olkhovskiy
5a4694f340 major refactoring, simplified, optimized, bugs fixed 2022-03-27 14:32:09 -04:00
Azat Khuzhin
13f2b2c0a3 Use proper type for RadixSortTraits in RowRefs.cpp 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-27 11:32:51 +03:00
Azat Khuzhin
d518614524 Fix multi-line comment in threadPoolCallbackRunner.cpp 2022-03-26 23:42:46 +03:00
Dmitry Novik
0d75e773ab Continue query execution if memory is freed 2022-03-26 18:25:58 +00:00
mergify[bot]
93b3b46caf
Merge branch 'master' into fix-lowcardinality-hash-join 2022-03-25 22:56:59 +00:00
Anton Popov
67195bfdd5 support schema inference for type Object in format JSONEachRow 2022-03-25 21:51:53 +00:00
alesapin
14e84ff6ef
Merge pull request #34802 from ClickHouse/overcommit-doc 
Add documentation for memory overcommit
 2022-03-25 20:50:44 +01:00
kssenii
d8960cde62 Fix positional order/etc by literal 2022-03-25 20:03:12 +01:00
kssenii
5103aefd34 Fix 2022-03-25 14:33:54 +01:00
Antonio Andelic
d7fb51269a Merge branch 'master' into fix-lowcardinality-hash-join 2022-03-25 11:12:31 +00:00
Antonio Andelic
9e11e611c7 Match cardinality when adding columns 2022-03-25 11:12:02 +00:00
mergify[bot]
6267ba4445
Merge branch 'master' into fix_stupid_bug 2022-03-25 01:30:48 +00:00
Nikita Mikhaylov
64f79f0c69
Added an ability to specify cluster secret in replicated database (#35333) 2022-03-25 00:14:26 +01:00
kssenii
6769b84638 Merge master 2022-03-24 20:47:43 +01:00
Yakov Olkhovskiy
adefcfd299
Merge branch 'master' into interpolate-feature 2022-03-24 15:33:09 -04:00
Yakov Olkhovskiy
83f406b722 optimization, INTERPOLATE without expr. list, any column is allowed except WITH FILL 2022-03-24 15:29:29 -04:00
Alexander Tokmakov
7c0bdbfa93 fix stupid bug 2022-03-24 20:24:05 +01:00
Sergei Trifonov
a90e83665d
Merge pull request #35343 from ClickHouse/parallel-multipart-upload-for-s3storage 
allow parallelization of multipart S3 storage upload; fixes #34350
 2022-03-24 15:58:29 +01:00
Antonio Andelic
34b2307259 Convert columns to full when multiple disjunct used 2022-03-24 13:56:00 +00:00
tavplubix
ee611a9af4
Merge pull request #35503 from ClickHouse/fix_assertion_ddl_worker 
Fix too strict assertion in DDLWorker
 2022-03-24 13:57:07 +03:00
Nikolai Kochetov
283e20a9a5
Merge pull request #35395 from amosbird/distributedmultiplejoin 
Validate some thoughts over making sets
 2022-03-24 10:30:26 +01:00
Maksim Kita
1df1721648
Merge pull request #35546 from kitaisreal/refactor-external-models-loader 
ExternalModelsLoader refactoring
 2022-03-24 01:24:45 +01:00
Maksim Kita
3df6a86387 ExternalModelsLoader refactoring 2022-03-23 20:05:17 +01:00
mergify[bot]
abbed251c1
Merge branch 'master' into asof-join-improve-performance 2022-03-23 18:53:17 +00:00
kssenii
d2a3cfe5dc Cache on all write operations 2022-03-23 19:14:33 +01:00
Alexander Tokmakov
9aed0507b7 Merge branch 'master' into mvcc_prototype 2022-03-23 18:07:22 +01:00
mergify[bot]
13bc93c171
Merge branch 'master' into fix_assertion_ddl_worker 2022-03-23 17:06:33 +00:00
Sergei Trifonov
bf9c6be6c6 move threadPoolCallbackRunner from Common into Interpreters folder to fix split build issue 2022-03-23 17:57:19 +01:00
kssenii
c2c7f365ce Merge master 2022-03-23 16:58:35 +01:00
Amos Bird
ab7923a26c
Remove comments 2022-03-23 23:21:02 +08:00
Maksim Kita
da235f9cda
Merge pull request #34536 from kitaisreal/system-log-tables-comment-fix 
System log tables allow to specify COMMENT in ENGINE declaration
 2022-03-23 15:06:05 +01:00
Maksim Kita
561ce88ff3 Fixed tests 2022-03-23 14:25:21 +01:00
Maksim Kita
3c5c267c17 Improve performance of ASOF join 2022-03-23 12:19:38 +01:00
Alexander Tokmakov
e8b7f963b7 fix too strict assertion 2022-03-23 11:55:28 +01:00
Maksim Kita
21735db126 Fixed style check 2022-03-23 11:52:37 +01:00
Maksim Kita
1f940f9e3b
Merge pull request #35512 from CurtizJ/fix-fuzzer 
Fix crash with enabled `optimize_functions_to_subcolumns`
 2022-03-23 11:31:32 +01:00
Maksim Kita
a03bfa0fae Fixed style check 2022-03-23 10:46:27 +01:00
taiyang-li
8dbf1c60e7 merge master and fix conflict 2022-03-23 11:36:50 +08:00
Anton Popov
4ff9627f60 fix crash with enabled optimize_functions_to_subcolumns 2022-03-23 01:27:52 +00:00
Maksim Kita
1a096a87da Added support for SystemLog to specify COMMENT in ENGINE section 2022-03-22 22:44:46 +01:00
Alexander Tokmakov
3c762f566d Merge branch 'master' into mvcc_prototype 2022-03-21 20:16:29 +01:00
Vitaly Baranov
72a2f31984
Merge pull request #33985 from vitlibar/improve-backup-and-restore 
Improve backup and restore (experimental)
 2022-03-21 17:59:51 +01:00
kssenii
afd0c64a1a Better 2022-03-21 15:04:16 +01:00
Alexey Milovidov
8a04ed72af
Merge pull request #34772 from ClickHouse/cluster-dont-require-port 
Allow to use default port in cluster configuration
 2022-03-21 16:45:24 +03:00
kssenii
eabbce69a7 Add system drop remote filesystem cache query, allow to write cache on insert 2022-03-21 12:32:44 +01:00
Vitaly Baranov
7a46e7a0d1
Merge branch 'master' into improve-backup-and-restore 2022-03-21 11:49:26 +01:00
mergify[bot]
e5a5ab2a40
Merge branch 'master' into distributedmultiplejoin 2022-03-21 10:00:51 +00:00
kssenii
59b7394caf WriteSettings 2022-03-21 09:52:48 +01:00
Alexey Milovidov
0ce4696c49
Revert "[WIP] New row policies" 2022-03-21 08:41:33 +03:00
Amos Bird
243de091bb
Validate some thoughts over making sets 2022-03-21 10:58:44 +08:00
Alexander Tokmakov
9e05b12d2c Merge branch 'master' into mvcc_prototype 2022-03-20 22:42:26 +01:00
李扬
d28981e0e3
fix null pointer of TreeRewriterResult::storage_snapshot (#35439) 2022-03-20 21:07:58 +01:00
Vitaly Baranov
c884cf901f Use BackupEntryFromAppendOnlyFile for Log engine, 
and use a BackupEntriesBatch for Memort engine to improve performance.
A lot of minor corrections.
 2022-03-20 20:02:15 +01:00
Vitaly Baranov
5f195d6ead Implement writing backups as zip archive. 2022-03-20 20:02:15 +01:00
Vitaly Baranov
7f89b98308 Rework BackupSettings and RestoreSettings a little, pass StorageRestoreSettings to storages. 2022-03-20 20:02:15 +01:00
Vitaly Baranov
258a472001 Shorter names: rename IRestoreFromBackupTask -> IRestoreTask. 2022-03-20 20:02:15 +01:00
Vitaly Baranov
7a63feb3f7 Make restore tasks explicit. 2022-03-20 20:01:31 +01:00
Vitaly Baranov
9fb50422ce
Merge pull request #35345 from vitlibar/new-row-policies 
[WIP] New row policies
 2022-03-20 20:00:25 +01:00
Vitaly Baranov
afe6a1003a Add new row policy kind: simple 2022-03-20 17:34:59 +01:00
Yakov Olkhovskiy
c4daf514d6
Update InterpreterSelectQuery.cpp 
bugfix: check column existence for INTERPOLATE expression target
 2022-03-19 14:12:29 -04:00
Yakov Olkhovskiy
eb7474e73a
Merge branch 'master' into interpolate-feature 2022-03-19 03:11:14 -04:00
Yakov Olkhovskiy
a8e1671a76 type match check for INTERPOLATE expressions added, bugfix, printout fixed 2022-03-18 16:44:27 -04:00
Alexander Tokmakov
0719d01d39 more introspection 2022-03-18 14:33:59 +01:00
Alexander Tokmakov
c28008a2df more comments, minor fixes 2022-03-18 12:01:26 +01:00
Raúl Marín
fe42895745 Address clang-tidy warnings 2022-03-18 11:15:12 +01:00
Raúl Marín
aeacd42e08 ASOF: Add big integer types support 2022-03-17 19:08:33 +01:00
Alexander Tokmakov
07d952b728 use snapshots for semistructured data, durability fixes 2022-03-17 18:26:18 +01:00
Raúl Marín
e22fe541b2 Improvements based on PR comments 2022-03-17 17:02:13 +01:00
Raúl Marín
e2cec4e65b Merge remote-tracking branch 'blessed/master' into asof_ftw 2022-03-17 16:33:29 +01:00
Vitaly Baranov
98e48d7b54 SHOW CREATE ROW POLICY now always shows AS permissive/restrictive if there is TO section. 2022-03-17 15:28:53 +01:00
Vitaly Baranov
3e950d79b3 Add new setting rbac_version. 2022-03-17 15:28:51 +01:00
Vitaly Baranov
470582e262 Change type of RowPolicyKind: bool -> enum. 2022-03-17 15:28:05 +01:00
Alexander Tokmakov
d04dc03fa4 Merge branch 'master' into mvcc_prototype 2022-03-17 15:24:32 +01:00
alesapin
457fa0d1ba
Merge pull request #23932 from CurtizJ/dynamic-columns 
Support of dynamic subcolumns (JSON data type)
 2022-03-17 10:45:46 +01:00
Yakov Olkhovskiy
00700988ee style fix 2022-03-17 02:31:01 -04:00
Yakov Olkhovskiy
7bb66e6702 added INTERPOLATE extension for ORDER BY WITH FILL 2022-03-17 01:51:35 -04:00
Alexander Tokmakov
4b3e13a4fe Merge branch 'master' into mvcc_prototype 2022-03-16 21:06:19 +01:00
Alexander Tokmakov
c2ac8d4a5c review fixes 2022-03-16 21:05:34 +01:00
Vitaly Baranov
39614e6e15
Merge pull request #35276 from vitlibar/fix-code-style-and-minor-corrections-after-allow-no-password 
Fix code style and other minor corrections after implementing allow_no_password.
 2022-03-16 18:55:05 +01:00
Anton Popov
2ced42ed41 add experimental settings for Object type 2022-03-16 16:51:23 +00:00
Maksim Kita
9ba53ae037
Merge pull request #34888 from kitaisreal/performance-tests-fix 
Performance tests fix
 2022-03-16 17:19:16 +01:00
Maksim Kita
dc31a4175f Fixed style check 2022-03-16 16:16:08 +00:00
Anton Popov
0ba78c3c3a Merge remote-tracking branch 'upstream/master' into HEAD 2022-03-16 15:28:09 +00:00
Kruglov Pavel
4a579768ef
Merge pull request #35311 from vdimir/pmj-dup-rows 
Attempt to fix merge join duplicate rows
 2022-03-16 12:48:29 +01:00
Maksim Kita
ed74a464bd Fixed tests 2022-03-16 11:21:18 +00:00
Nikolai Kochetov
bcd3c76296
Merge pull request #35214 from ClickHouse/add-database_replicated_allow_only_replicated_engine-setting 
Add database_replicated_allow_only_replicated_engine setting.
 2022-03-16 11:24:32 +01:00
vdimir
695708d57c
[wip] merge join duplicate rows 2022-03-15 18:51:04 +00:00
Alexander Tokmakov
b5b3d09b4a fix 2022-03-15 17:51:53 +01:00
Alexander Tokmakov
1f571b7734 Merge branch 'master' into mvcc_prototype 2022-03-15 14:45:06 +01:00
Alexander Tokmakov
9c3e4cdc6e fix 2022-03-15 14:35:48 +01:00
Anton Popov
b5a04ff938
Merge pull request #35207 from bigo-sg/add_distributed_depth_in_querylog 
Add column  distributed_depth for system.query_log and system.processes
 2022-03-15 13:38:21 +01:00
Vladimir C
af88d3f133
Merge pull request #35173 from bigo-sg/dump_ast_in_dot 2022-03-15 11:18:41 +01:00
taiyang-li
3688ff10e0 fix bug 2022-03-15 12:19:08 +08:00
Alexander Tokmakov
c8d6c13c2d fix 2022-03-15 00:08:25 +01:00
Alexander Tokmakov
9702b5177d Merge branch 'master' into mvcc_prototype 2022-03-14 21:45:38 +01:00
Alexander Tokmakov
278d779a01 log cleanup, more comments 2022-03-14 21:43:34 +01:00
Vitaly Baranov
1eb2e8693e Fix code style and other minor corrections after implementing allow_no_password. 2022-03-14 20:55:34 +01:00
Nikolai Kochetov
7526c36f30 Remove comment 2022-03-14 19:38:03 +00:00
Nikolai Kochetov
ce94b560bb Add test. 2022-03-14 19:30:52 +00:00
Maksim Kita
e30117a3d6 Fix clang-tidy warnings in Interpreters, IO folders 2022-03-14 18:17:35 +00:00
Anton Popov
063917786e minor fixes 2022-03-14 17:29:18 +00:00
Anton Popov
36ec379aeb Merge remote-tracking branch 'upstream/master' into HEAD 2022-03-14 16:28:35 +00:00
Vitaly Baranov
4af61fb9d3
Merge pull request #34738 from DevTeamBK/Issue-33953 
New setting in Server Configuration to on/off AuthType Plaintext_password and No_password
 2022-03-14 17:09:46 +01:00
alesapin
b13b415d3f
Merge pull request #35257 from azat/fix-s3-pool-shutdown 
Wait for IDiskRemote thread pool properly
 2022-03-14 13:56:37 +01:00
Nikolai Kochetov
ff09076524 A little bit better. 2022-03-14 11:31:24 +00:00
Kseniia Sumarokova
58a2d2b458
Merge pull request #35118 from zzsmdfj/issue/#31469_MaterializedMysql_mysqlDate2CkDate32 
to #31469_MaterializedMysql_mysqlDate2CkDate32
 2022-03-14 10:32:33 +01:00
Azat Khuzhin
d13fc9ec44 Wait for IDiskRemote thread pool properly 
ASan report [1]:

    WARNING: ThreadSanitizer: heap-use-after-free (pid=611)
      Read of size 4 at 0x7b48006f0250 by thread T273:
        0 MemoryTracker::allocImpl() obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:105:49 (clickhouse+0xa05d0ef)
        1 MemoryTracker::allocImpl() obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:239:22 (clickhouse+0xa05d580)
        2 MemoryTracker::alloc(long) obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:246:5 (clickhouse+0xa05dea7)
        3 DB::ThreadStatus::~ThreadStatus() obj-x86_64-linux-gnu/../src/Common/ThreadStatus.cpp:154:28 (clickhouse+0xa059b01)
        4 ThreadFromGlobalPool::ThreadFromGlobalPool<>()::'lambda'()::operator()() obj-x86_64-linux-gnu/../s>

      Previous write of size 8 at 0x7b48006f0250 by main thread:
        8 std::__1::unordered_map<std::__1::basic_string<>, DB::ProcessListForUser, std::__1::hash<>, std::__1>
        9 DB::ProcessList::~ProcessList() obj-x86_64-linux-gnu/../src/Interpreters/ProcessList.h:275:7 (clickhouse+0x17b71c9c)
        10 DB::ContextSharedPart::~ContextSharedPart() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:304:5 (clickhouse+0x17b70712)
        13 DB::SharedContextHolder::reset() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:453:44 (clickhouse+0x17b3daf9)
        14 DB::Server::main()::$_8::o>

      Thread T273 (tid=946, running) created by thread T223 at:
        0 pthread_create <null> (clickhouse+0x9f7d49d)
        15 DB::WriteBufferFromS3::makeSinglepartUpload() obj-x86_64-linux-gnu/../src/IO/WriteBufferFromS3.cpp:332:9 (clickhouse+0x17a81b6a)

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/35072/19be9c8c6433ffd1bbf8eb08975d8afbaebf3d43/stress_test__thread__actions_.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-14 12:07:47 +03:00
Maksim Kita
0dd807d19d
Merge pull request #34750 from kitaisreal/merge-tree-improve-insert-performance 
MergeTree improve insert performance
 2022-03-13 13:39:18 +01:00
Nikolai Kochetov
cdc529fa2a Add database_replicated_allow_only_replicated_engine setting. 2022-03-11 16:03:05 +00:00
Maksim Kita
5f8900cee6
Merge pull request #35079 from qieqieplus/master 
fix #33798
 2022-03-11 11:55:49 +01:00
taiyang-li
1f9e050152 add column distributed_depth for system.query_log and system.processes 2022-03-11 17:57:34 +08:00
Kseniia Sumarokova
818459b9f0
Merge pull request #33717 from kssenii/local-cache-for-remote-fs 
Local cache for remote filesystem
 2022-03-11 07:23:10 +01:00
HeenaBansal2009
3f031df225 Code refactoring 2022-03-10 22:22:51 -08:00
Maksim Kita
cbe059f4bd Updated IColumn interface to support getting stable permutation 2022-03-10 21:45:31 +00:00
Maksim Kita
765cd09d06 MergeTree improve insert performance 2022-03-10 21:45:31 +00:00
Alexander Tokmakov
7f47f20aba add kill transaction query 2022-03-10 22:29:58 +01:00
alesapin
c0d8ccc91b
Merge pull request #35178 from Varinara/master 
Added disk_name to system.part_log
 2022-03-10 22:22:37 +01:00
alexey-milovidov
58e53b06a6
Merge pull request #35134 from azat/optimize_skip_unused_shards_rewrite_in-signed 
Fix `optimize_skip_unused_shards_rewrite_in` for signed columns and negative values
 2022-03-10 23:12:05 +03:00
mergify[bot]
059682a0cd
Merge branch 'master' into asof_ftw 2022-03-10 17:29:38 +00:00
mergify[bot]
9f4ebc313b
Merge branch 'master' into master 2022-03-10 16:55:27 +00:00
Varinara
f5523f7ff0 added disk_name to system.part_log 2022-03-10 18:44:19 +03:00
Rajkumar
3d3b6d1956 clang-tidy report issues with Medium priority 2022-03-10 07:23:49 -08:00
Alexander Tokmakov
061fa6a6f2 Merge branch 'master' into mvcc_prototype 2022-03-10 13:13:04 +01:00
kssenii
787a0805a5 Merge master 2022-03-10 11:42:19 +01:00
taiyang-li
ee8fc2730e remote comment 2022-03-10 18:29:52 +08:00
taiyang-li
382417d893 fix bug 2022-03-10 18:29:00 +08:00
taiyang-li
2cd469f113 support print ast in dot format 2022-03-10 18:04:35 +08:00
HeenaBansal2009
3ce9397246 Added Suggestions from Code review 2022-03-09 20:35:01 -08:00
Alexander Tokmakov
0906b59fba fixes 2022-03-09 21:38:18 +01:00
Vladimir C
ce266b5a3e
Merge pull request #35146 from amosbird/fixpartitionprunerin 2022-03-09 13:23:45 +01:00
Nikolai Kochetov
c364908061
Merge pull request #35094 from amosbird/getridofredundantplan 
Get rid of duplicate query planing.
 2022-03-09 09:10:20 +01:00
Amos Bird
a19224bc9b
Fix partition pruner: non-monotonic function IN 2022-03-09 15:48:42 +08:00
Azat Khuzhin
ec6bae2b2a Fix optimize_skip_unused_shards_rewrite_in for signed columns 
Fixes: #35131
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-09 08:29:28 +03:00
Alexander Tokmakov
d2f838bd91 fix another race condition 2022-03-08 20:11:47 +01:00
kssenii
5260822964 Merge master 2022-03-08 18:21:28 +01:00
Azat Khuzhin
c4b6342853
Improvements for parallel_distributed_insert_select (and related) (#34728) 
* Add a warning if parallel_distributed_insert_select was ignored

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Respect max_distributed_depth for parallel_distributed_insert_select

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Print warning for non applied parallel_distributed_insert_select only for initial query

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Remove Cluster::getHashOfAddresses()

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Forbid parallel_distributed_insert_select for remote()/cluster() with different addresses

Before it uses empty cluster name (getClusterName()) which is not
correct, compare all addresses instead.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Fix max_distributed_depth check

max_distributed_depth=1 must mean not more then one distributed query,
not two, since max_distributed_depth=0 means no limit, and
distribute_depth is 0 for the first query.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Fix INSERT INTO remote()/cluster() with parallel_distributed_insert_select

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Add a test for parallel_distributed_insert_select with cluster()/remote()

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Return <remote> instead of empty cluster name in Distributed engine

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Make user with sharding_key and w/o in remote()/cluster() identical

Before with sharding_key the user was "default", while w/o it it was
empty.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-08 15:24:39 +01:00
zzsmdfj
67b9f81104 to #31469_MaterializedMysql_mysqlDate2CkDate32 2022-03-08 18:17:22 +08:00
taiyang-li
b4174b0bef merge master and fix conflicts 2022-03-08 11:39:25 +08:00
Alexander Tokmakov
8acfb8d27f Merge branch 'master' into mvcc_prototype 2022-03-07 17:40:15 +01:00
Amos Bird
fe4534d464
Get rid of duplicate query planing. 2022-03-08 00:02:58 +08:00
Anton Popov
0bc57da238 Merge remote-tracking branch 'upstream/master' into HEAD 2022-03-07 14:46:08 +00:00
tavplubix
34505be1de
Merge pull request #35081 from azat/fix-shutdown-deadlock 
Avoid possible deadlock on server shutdown
 2022-03-07 13:47:15 +01:00
Kseniia Sumarokova
cdb9a05229
Merge pull request #34605 from bigo-sg/add_metric_for_local 
Add cpu/mem metric for clickhouse-local
 2022-03-07 11:40:11 +01:00
Azat Khuzhin
081f9caa04 Avoid possible deadlock on server shutdown 
Here is an example for deadlock during shutting down DDLWorker:

Server main thread:

    6  ThreadFromGlobalPool::join () at ../src/Common/ThreadPool.h:217
    7  DB::DDLWorker::shutdown () at ../src/Interpreters/DDLWorker.cpp:123
    8  DB::DDLWorker::~DDLWorker () at ../src/Interpreters/DDLWorker.cpp:131
    9  DB::DDLWorker::~DDLWorker () at ../src/Interpreters/DDLWorker.cpp:130
    10 std::__1::default_delete<DB::DDLWorker>::operator() () at ../contrib/libcxx/include/memory:1397
    11 std::__1::unique_ptr<>::reset (this=0x7f7521d44fd0, __p=0x0) at ../contrib/libcxx/include/memory:1658
    12 DB::ContextSharedPart::shutdown () at ../src/Interpreters/Context.cpp:380 <!-- holds mutex
    13 DB::Context::shutdown () at ../src/Interpreters/Context.cpp:2677

DDLWorker thread:

    7  DB::Context::getLock () at ../src/Interpreters/Context.cpp:472 <-- trying to acquire shared.mutex
    8  DB::Context::getTCPPortSecure () at ../src/Interpreters/Context.cpp:2120
    9  DB::DDLTask::findCurrentHostID () at ../src/Interpreters/DDLTask.cpp:169
    10 DB::DDLWorker::initAndCheckTask () at ../src/Interpreters/DDLWorker.cpp:191
    11 DB::DDLWorker::scheduleTasks () at ../src/Interpreters/DDLWorker.cpp:367
    12 DB::DDLWorker::runMainThread () at ../src/Interpreters/DDLWorker.cpp:1063

v2: replace getTryLockTimed() with getTryLock() since std::recursive_mutex does not have try_lock_for()
v3: stole objects from context
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Fixes: #21318
 2022-03-06 14:31:18 +03:00
qieqieplus
5287f78d8b fix #33798 2022-03-06 12:02:17 +08:00
taiyang-li
1b9987ceb1 fix mistake in check-style 2022-03-05 14:22:56 +08:00
tavplubix
e49521163f
Merge pull request #34860 from zhangjmruc/master 
Support non-table DDLs on cross replicated cluster
 2022-03-04 19:37:51 +01:00
Kruglov Pavel
d45b6ae03c
Extract schema only once on table creation and add it to metadata (#34684) 
* Add columns description to metadata in case of schema inference

* Make better

* Remove unnecessary code

* Fix tests

* More tests

* Add tag no-fasttest

* Fix test

* Fix test

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2022-03-04 21:23:19 +03:00
Maksim Kita
c6e0922449
Merge pull request #35018 from kitaisreal/executable-udf-deterministic-in-scope-of-query-fix 
ExecutableUDF function deterministic in scope of query fix
 2022-03-04 10:46:19 +01:00
Anton Popov
df3b07fe7c Merge remote-tracking branch 'upstream/master' into HEAD 2022-03-03 22:25:28 +00:00
Anton Popov
aea7bfb59a
Merge pull request #34992 from azat/fix-asynchronous_inserts-race 
Fix race between INSERT async_insert=1 and system.asynchronous_inserts
 2022-03-03 20:55:19 +03:00
mergify[bot]
e4fba1dc99
Merge branch 'master' into executable-udf-deterministic-in-scope-of-query-fix 2022-03-03 15:17:16 +00:00
Maksim Kita
a6c510ceb1 ExecutableUDF function deterministic in scope of query fix 2022-03-03 12:40:42 +00:00
Frank Chen
b4829465d9
Improve the opentelemetry span logs for INSERT on distributed table (#34480) 2022-03-03 12:53:29 +01:00
Maksim Kita
b1a956c5f1 clang-tidy check performance-move-const-arg fix 2022-03-02 18:15:27 +00:00
Maksim Kita
1f5837359e clang-tidy check performance-noexcept-move-constructor fix 2022-03-02 18:15:27 +00:00
mreddy017
f893002b69 Fix vulnerable code related to std::move and noexcept 
This commit fixes the vulnerable code related to std::move and noexcept identified by clangtidy tool.
 2022-03-02 18:15:27 +00:00
kssenii
d5952109fb Merge master 2022-03-02 18:15:25 +01:00
mergify[bot]
93b13c0232
Merge branch 'master' into asof_ftw 2022-03-02 13:20:50 +00:00
Azat Khuzhin
57f636a1e8 Fix race between INSERT async_insert=1 and system.asynchronous_inserts 
CI report [1]:

    [c190f600f8c6] 2022.03.02 01:07:34.553012 [ 23552 ] {76b6113b-1479-46c9-90ab-e78a3c9f3dbb}  executeQuery: Code: 60. DB::Exception: Both table name and UUID are empty. (UNKNOWN_TABLE) (version 22.3.1.1) (from [::1]:42040) (comment: '02015_async_inserts_stress_long.sh') (in query: SELECT * FROM system.asynchronous_inserts FORMAT Null), Stack trace (when copying this message, always include the lines below):

    0. ClickHouse/contrib/libcxx/include/exception:133: Poco::Exception::Exception(std::__1::basic_string, std::__1::allocator > const&, int) @ 0xf50e04c in /fasttest-workspace/build/programs/clickhouse
    1. ClickHouse/src/Common/Exception.cpp:58: DB::Exception::Exception(std::__1::basic_string, std::__1::allocator > const&, int, bool) @ 0x663ebfa in /fasttest-workspace/build/programs/clickhouse
    2. DB::StorageID::assertNotEmpty() const @ 0xbc08591 in /fasttest-workspace/build/programs/clickhouse
    3. ClickHouse/contrib/libcxx/include/string:1444: DB::StorageID::getDatabaseName() const @ 0xe50d2b6 in /fasttest-workspace/build/programs/clickhouse
    4. ClickHouse/contrib/libcxx/include/string:1957: DB::StorageSystemAsynchronousInserts::fillData(std::__1::vector::mutable_ptr, std::__1::allocator::mutable_ptr > >&, std::__1::shared_ptr, DB::SelectQueryInfo const&) const @ 0xdac636c in /fasttest-workspace/build/programs/clickhouse

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/34973/e6fc6a22d5c018961c18247242dd3a40b8c54ff2/fast_test__actions_.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-02 15:28:06 +03:00
Azat Khuzhin
a49106e583 Fix non-MergeTree engines for system.*_log 
Fixes: https://github.com/ClickHouse/ClickHouse/pull/34949
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-02 13:24:40 +03:00
kssenii
a594f388a4 Merge master 2022-03-01 19:43:45 +01:00
Anton Popov
c1fdcf7a64 Merge remote-tracking branch 'upstream/master' into HEAD 2022-03-01 20:21:39 +03:00
Anton Popov
82d24f06eb
Merge pull request #34866 from CurtizJ/async-insert-table-function 
Fix async inserts to table functions
 2022-03-01 19:45:22 +03:00
Kruglov Pavel
97d634d1cc
Merge pull request #34732 from Avogar/insert-select-schema 
Support schema inference for INSERT INTO FUNCTION
 2022-03-01 13:38:16 +03:00
alesapin
4b61e4795c
Merge pull request #34949 from nikitamikhaylov/system_log_tables_and_settings 
Recreate system.{*}_log table on settings changes
 2022-03-01 11:15:19 +01:00
taiyang-li
f83132bad2 finish dev 2022-03-01 15:54:23 +08:00
taiyang-li
b31440c77a Merge branch 'master' into add_metric_for_local 2022-03-01 12:29:36 +08:00
HeenaBansal2009
c14c60f1d3 Merge branch 'master' into Issue-33953 2022-02-28 11:12:54 -08:00
HeenaBansal2009
aa8494a808 Fix: System Reload Config Failure 2022-02-28 10:51:49 -08:00
Nikita Mikhaylov
d6036f6da3 Better 
(cherry picked from commit 4ae445c9e227581ea9f1cbe9aa9d1ba82e1236c9)
 2022-02-28 15:27:52 +00:00
Anton Popov
8fa112b841 add comments 2022-02-28 17:09:46 +03:00
Kruglov Pavel
011813957d
Merge pull request #34938 from azat/create-as-ignore-ttl 
Ignore per-column TTL in CREATE TABLE AS if new table engine does not support it
 2022-02-28 16:58:15 +03:00
Azat Khuzhin
644f9168fa Ignore per-column TTL in CREATE TABLE AS if new table engine does not support it 
Follow-up for: #6968
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-28 10:29:26 +03:00
mergify[bot]
37bf9c100a
Merge branch 'master' into chained_or 2022-02-27 18:43:21 +00:00
Danila Kutenin
7b91e62ae0 Transform OR LIKE chain to multiMatchAny 
Closes #5895
 2022-02-27 16:55:02 +00:00
Alexandre Snarskii
93de1b1c99 Implement MemoryStatisticsOS for FreeBSD 2022-02-25 14:05:00 +03:00
Anton Popov
fcdebea925 Merge remote-tracking branch 'upstream/master' into HEAD 2022-02-25 13:41:30 +03:00
Anton Popov
97d57e22ea fix async inserts to table functions 2022-02-24 14:26:47 +03:00
Jianmei Zhang
d148cc30cc Support non-table DDLs on cross replicated cluster 2022-02-24 14:37:35 +08:00
Alexander Tokmakov
aa6b9a2abc Merge branch 'master' into mvcc_prototype 2022-02-23 23:22:03 +03:00
Dmitry Novik
67df01d025
Merge branch 'master' into grouping-sets-fix 2022-02-22 04:18:03 -08:00
kssenii
c637385dd0 Merge master 2022-02-22 13:17:51 +01:00
Maksim Kita
174257dad0
Merge pull request #34653 from kitaisreal/executable-udf-support-specify-argument-names 
ExecutableUserDefinedFunctions allow to specify argument names
 2022-02-22 00:02:14 +01:00
Dmitry Novik
1df43a7f57
Merge pull request #34385 from nvartolomei/nv/move-part-count 
Disable optimize_trivial_count when deduplication for part movement feature is enabled
 2022-02-21 08:53:09 -08:00
Dmitry Novik
be9082f1c4 Update comments for OvercommitTracker 2022-02-21 15:45:17 +00:00
Anton Popov
065305ab65
Merge pull request #34764 from ucasfl/hints-index 
Add name hints for data skipping indices
 2022-02-21 16:50:59 +03:00
kssenii
4543513a5d Add comments 2022-02-21 13:54:03 +01:00
Dmitry Novik
4428e7aa1b
Merge branch 'master' into nv/move-part-count 2022-02-21 02:14:23 -08:00
Mikhail f. Shiryaev
f690aba441
Merge pull request #34078 from godliness/fix-distributed-max-query-size 
Fix distributed subquery max_query_size limitation inconsistency
 2022-02-21 11:01:02 +01:00
Vitaly Baranov
7b97c986cb
Revert "Allow restrictive row policies without permissive" 2022-02-21 06:54:28 +03:00
Alexey Milovidov
522c1c1397 Allow to use default port in cluster configuration 2022-02-20 19:27:14 +01:00
feng lv
07280e0ab1 Add name hints for data skipping indices 
fix test
 2022-02-20 11:48:22 +00:00
alexey-milovidov
26d0e5438c
Merge pull request #33057 from azat/jemalloc-dev-branch 
[RFC] Update jemalloc to 5.3RC
 2022-02-20 14:19:41 +03:00
Chao Ma
895396e61a Fix distributed subquery max_query_size limitation inconsistency 2022-02-20 10:07:57 +08:00
Raúl Marín
4439b23e2b Attempt manual unpack 2022-02-19 21:01:47 +01:00
Vitaly Baranov
874b2c8dcb
Merge pull request #34596 from vitlibar/allow-restrictive-without-permissive 
Allow restrictive row policies without permissive
 2022-02-19 21:45:28 +07:00
Raúl Marín
07d2b6ba80 Remove struct packing 2022-02-19 11:16:28 +01:00
HeenaBansal2009
1b263f0c15 Added FT testcase 2022-02-18 12:58:46 -08:00
Raúl Marín
606f580922 Fix style 2022-02-18 18:37:46 +01:00
Vladimir C
9b7d011ee7
Merge pull request #34529 from vdimir/join-nullable-on-pipeline 
Apply join_use_nulls on types before join
 2022-02-18 18:34:44 +01:00
Raúl Marín
bb1d773ffd Reduce changes against master 2022-02-18 17:58:23 +01:00
avogar
653d769d34 Support schema inference for INSERT INTO FUNCTION 2022-02-18 16:19:42 +00:00
Raúl Marín
2f4c595e09 Bring back some clarity 2022-02-18 16:50:15 +01:00
Maksim Kita
0f9f30cc9e Added tests 2022-02-18 15:21:11 +00:00
Raúl Marín
f8cc024ba1 Less indirect access 2022-02-18 16:16:29 +01:00
vdimir
6ca154858c
upd TableJoin::applyKeyConvertToTable 2022-02-18 13:40:57 +00:00
vdimir
6eda8ef914
use isNullable function in join_common/convertTypeToNullable 2022-02-18 13:09:11 +00:00
Raúl Marín
57b9639872 Faster binary search 2022-02-18 14:05:42 +01:00
Raúl Marín
4ba5b931a5 AsofRowRefs: Remove extra level of indirection 2022-02-18 11:15:11 +01:00
Raúl Marín
34b110b778 Remove variant from AsofRowRefs 2022-02-18 11:02:14 +01:00
HeenaBansal2009
d16cae53b4 Initial Commit for Plaintext password feature 2022-02-17 21:25:18 -08:00
Alexander Tokmakov
f4a46a13fb fixes 2022-02-18 00:26:37 +03:00
Rajkumar
de154c5f9c Adding noexcept for move constructor 2022-02-17 13:18:12 -08:00
Azat Khuzhin
6a2abd3a68 jemalloc: remove jemalloc 4 support 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-17 21:25:58 +03:00
Raúl Marín
b751218fd6 Help the compiler with the variant dispatch code 2022-02-17 16:39:55 +01:00
Alexander Tokmakov
dae044f86b Merge branch 'master' into mvcc_prototype 2022-02-17 13:49:37 +03:00
Kruglov Pavel
7411178064
Merge pull request #34650 from azat/fix-projection-optimization 
Fix allow_experimental_projection_optimization with enable_global_with_statement
 2022-02-17 13:09:57 +03:00
Dmitry Novik
c32313b045
Merge branch 'master' into memory-overcommit-test 2022-02-17 00:50:15 -08:00
Vitaly Baranov
f4da321dbb SHOW CREATE ROW POLICY now always outputs "AS permissive" or "AS restrictive". 2022-02-17 14:18:15 +07:00
Vitaly Baranov
2de6e8e575 Change type of RowPolicyKind: bool -> enum. 2022-02-17 14:18:10 +07:00
Dmitry Novik
07d0e3c823 cleanup 2022-02-16 20:19:10 +00:00
Dmitry Novik
44d5ddc939 Fix deadlock in OvercommitTracker 2022-02-16 20:02:14 +00:00
Azat Khuzhin
774744a86d Fix allow_experimental_projection_optimization with enable_global_with_statement 
allow_experimental_projection_optimization requires one more
InterpreterSelectQuery, which with enable_global_with_statement will
apply ApplyWithAliasVisitor if the query is not subquery.

But this should not be done for queries from
MergeTreeData::getQueryProcessingStage()/getQueryProcessingStageWithAggregateProjections()
since this will duplicate WITH statements over and over.

This will also fix scalar.xml perf tests, that leads to the following
error now:

    scalar.query0.prewarm0: DB::Exception: Stack size too large.

And since it has very long query in the log, this leads to the following
perf test error:

    _csv.Error: field larger than field limit (131072)

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-16 19:14:47 +03:00
Maksim Kita
669b18e037 ExecutableUserDefinedFunctions allow to specify argument names 2022-02-16 15:31:30 +00:00
Anton Popov
edd686e1d9 fix unit test 2022-02-16 17:18:03 +03:00
kssenii
47f94120da Merge master 2022-02-16 14:43:28 +01:00
Nikolai Kochetov
f9d2dae88e
Merge pull request #34424 from yakov-olkhovskiy/ephemeral-column 
Ephemeral column issue #9436
 2022-02-16 12:04:57 +01:00
Raúl Marín
d06ef3e1db AsofRowRefs: Simplify variant code 2022-02-16 11:12:28 +01:00
Raúl Marín
47d325edd1 Improve SortedLookupVector::find 2022-02-16 11:12:28 +01:00
Raúl Marín
0a620b82a3 RowRefs: Use assert_cast instead of typeid_cast 
They couldn't fail and it meant over 1% of the query runtime:
1.23%  typeid_cast<DB::ColumnVector<unsigned int> const&, DB::IColumn const>
    - 0.53% DB::AsofRowRefs::insert
    - 0.56% DB::AsofRowRefs::findAsof
 2022-02-16 11:12:28 +01:00
Raúl Marín
b99d757633 Packed RowRef 
- sizeof(DB::AsofRowRefs::Entry<Int32>): Was 24, now 16
- sizeof(DB::AsofRowRefs::Entry<Int64>): Was 24, now 24 too
 2022-02-16 11:12:28 +01:00
Anton Popov
a661eaf39f better performance of getting storage snapshot 2022-02-16 02:17:22 +03:00
Nikolai Kochetov
1276c50271 Fix build. 2022-02-15 20:15:07 +00:00
alesapin
bc2d0ee7c7
Merge pull request #34215 from ClickHouse/revert-34211-revert-34153-add_func_tests_over_s3 
Add func tests run with s3 and fix several bugs
 2022-02-15 19:07:11 +03:00
Dmitry Novik
bb6dad7d0e Fix lock order 2022-02-15 15:04:13 +00:00
taiyang-li
e6b29167da fix style 2022-02-15 20:26:53 +08:00
Nikolai Kochetov
b2b29b7ee8 Do not use ReduceToPhysicalTransform. 2022-02-15 12:23:22 +00:00
vdimir
510a909d0d
Remove nullable_left/right_side flags from Hash/MergeJoin 2022-02-15 11:15:50 +00:00
vdimir
2de6eaf838
Fix LowCardinality using key for join_use_nulls in pipeline 2022-02-15 11:11:59 +00:00
Nikolai Kochetov
ab288642f6 Merge branch 'master' into ephemeral-column 2022-02-15 10:03:34 +00:00
Nikolai Kochetov
d6cbac1ed3
Merge pull request #34577 from ClickHouse/alwasy-remove-unused-actions-for-add-missing-defaults 
Always remove unused actions from addMissingDefaults
 2022-02-15 11:01:29 +01:00
Nikolai Kochetov
4d2c53c32b Fix clang tidy 2022-02-15 09:59:51 +00:00
taiyang-li
b4440131d0 add metrics for clickhouse-local 2022-02-15 16:25:07 +08:00
李扬
f52b67b939
Merge branch 'master' into rocksdb_metacache 2022-02-15 02:16:29 -06:00
Alexander Tokmakov
1e4e569151 Merge branch 'master' into mvcc_prototype 2022-02-15 02:26:47 +03:00
Alexander Tokmakov
ae5aa8c12d write part version before other files 2022-02-15 02:24:51 +03:00
alexey-milovidov
bc206ec423
Merge pull request #31182 from ClickHouse/memory-overcommit 
Memory overcommit
 2022-02-14 23:55:02 +03:00
Alexander Tokmakov
cbd3b45646 add EXPLAIN CURRENT TRANSACTION 2022-02-14 22:47:17 +03:00
vdimir
99ca89c0ca
Fix StorageJoin and Asof or join_use_nulls in pipeline 2022-02-14 14:14:27 +00:00
Nikolai Kochetov
22a938f83d Simplify a little bit. 2022-02-14 14:53:32 +01:00
Nikolai Kochetov
ebb5d1807d Always remove unused actipns from addMissingDefaults 2022-02-14 12:49:07 +00:00
alesapin
89373155fc Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-13 21:07:54 +03:00