thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 03:12:43 +00:00
Code Issues Packages Projects Releases Wiki Activity
92,468 Commits 9,326 Branches 1,783 Tags 2.1 GiB
c2ae762a5a
Commit Graph

6805 Commits

Author SHA1 Message Date
Alexander Tokmakov
432f1d570c
Merge pull request #37153 from azat/fix-optimize_skip_unused_shards_rewrite_in 
Fix implicit cast for optimize_skip_unused_shards_rewrite_in
 2022-05-13 12:58:28 +03:00
Igor Nikonov
f423e9bf40 Ignore AUTO_INCREMENT in CREATE TABLE to simplify migration from MySQL 
Issue #35893
 2022-05-13 09:22:52 +00:00
Kseniia Sumarokova
320efb2221
Merge branch 'master' into better-read-from-cache 2022-05-12 23:35:39 +02:00
Maksim Kita
2bf6d9c028
Merge pull request #37154 from azat/context-const 
Add const qualifier for few methods from Context
 2022-05-12 22:20:39 +02:00
Kseniia Sumarokova
6e0a910fcd
Merge branch 'master' into better-read-from-cache 2022-05-12 19:44:18 +02:00
Dmitry Novik
c5b40a9c91 WIP on GROUPING function 2022-05-12 16:40:26 +00:00
alesapin
e7296a2b28
Merge pull request #36825 from kssenii/cache-fix-1 
Removed forceful drop cache command, fix detached status state
 2022-05-12 17:18:21 +02:00
Vxider
dc33c3a9bb Merged with master 2022-05-12 23:04:07 +08:00
Azat Khuzhin
a0ba113e5e Mark Context::getInterserverIOHandler() as const 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-12 16:54:01 +03:00
Azat Khuzhin
a04b6ad314 Mark Context::getInterserverCredentials() as const 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-12 16:54:01 +03:00
Azat Khuzhin
586eda5242 Fix implicit cast for optimize_skip_unused_shards_rewrite_in 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Fixes: #37073
 2022-05-12 16:17:30 +03:00
Vladimir C
4cb6ca2311
Merge pull request #36970 from ClickHouse/concurrent_join 
Follow up for ConcurrentHashJoin
 2022-05-12 15:09:29 +02:00
kssenii
483e056bcc Merge master 2022-05-12 14:58:24 +02:00
Dmitry Novik
df071bb701
Merge pull request #37129 from ClickHouse/enable-overcommit-for-user 
Update documentation and defaults for memory overcommit
 2022-05-12 14:38:43 +02:00
mergify[bot]
7ffad69019
Merge branch 'master' into wv-populate 2022-05-12 11:52:04 +00:00
Kseniia Sumarokova
59bfcd917b
Merge pull request #36876 from kssenii/extend-cache-log 
Log into filesystem_cache_log when cache not even attempted
 2022-05-12 12:38:36 +02:00
Nikolai Kochetov
55188665f4
Merge pull request #37071 from ClickHouse/join_dup_columns_issue_contin 
Continue fixing #36199.
 2022-05-12 11:36:15 +02:00
mergify[bot]
f4bae3969d
Merge branch 'master' into enable-overcommit-for-user 2022-05-11 20:04:20 +00:00
Maksim Kita
b1c154ccb4 Fixed style check 2022-05-11 21:59:51 +02:00
Maksim Kita
75555c436b Fix usage of min_count_to_compile_sort_description setting 2022-05-11 21:59:51 +02:00
Maksim Kita
ea8ce3140a Fixed tests 2022-05-11 21:59:51 +02:00
Maksim Kita
4e7d10297b Fixed style 2022-05-11 21:59:51 +02:00
Maksim Kita
ce92f6aab1 Fixed tests 2022-05-11 21:59:51 +02:00
Maksim Kita
b3b46d2e5c Fixed tests 2022-05-11 21:59:51 +02:00
Maksim Kita
8ceb63ee6c Added JIT compilation of SortDescription 2022-05-11 21:59:51 +02:00
Nikolai Kochetov
ec34761d9f
Merge pull request #33631 from ClickHouse/grouping-sets-fix 
Support GROUPING SETS
 2022-05-11 21:28:46 +02:00
Dmitry Novik
17608b3d93 Update documentation and defaults for memory overcommit 2022-05-11 16:18:41 +00:00
Vladimir C
641e2be6ef
Merge pull request #37085 from vdimir/force-cross-to-inner-option 
Option to force cross_to_inner_join_rewrite
 2022-05-11 18:03:53 +02:00
Kruglov Pavel
af3db5a992
Merge pull request #36726 from PolyProgrammist/s3clustermacro 
Handle cluster macro in s3Cluster table function
 2022-05-11 18:00:57 +02:00
Nikolai Kochetov
2d99f0ce13 Simplify code a little bit. 2022-05-11 12:16:15 +00:00
vdimir
2f5959aca6
Upd cross_to_inner_join_rewrite setting, add test 2022-05-11 10:56:43 +00:00
Vxider
e4f43d0b01 update getSampleBlock for windowview 2022-05-11 08:47:08 +00:00
kssenii
18dd74be0c Remove unneede changes 2022-05-11 10:26:30 +02:00
Alexey Milovidov
c0acd5a325 Merge branch 'chained-or' of github.com:ClickHouse/ClickHouse into chained-or 2022-05-11 01:29:22 +02:00
Alexey Milovidov
3fcf63830c Merge branch 'master' into chained-or 2022-05-11 01:28:12 +02:00
Nikolai Kochetov
f7dbd48ee5 Simplify code a little bit. 2022-05-10 16:12:03 +00:00
Nikolai Kochetov
a02e1d2f4a Simplify code a little bit. 2022-05-10 16:00:00 +00:00
vdimir
00cd21cacf
Option to force cross_to_inner_join_rewrite 2022-05-10 15:12:17 +00:00
vdimir
9b24e0d260
Apply optimizeCountConstantAndSumOne before JOINs 2022-05-10 14:30:33 +00:00
vdimir
ff001382fb
Handle non unique column names in NotJoinedBlocks 2022-05-10 14:30:33 +00:00
vdimir
837d4bd13f
Revert "Continue fixing #36199." 
This reverts commit 1feba64a4e.
 2022-05-10 14:30:02 +00:00
Nikolai Kochetov
1feba64a4e
Continue fixing #36199. 2022-05-10 14:26:49 +00:00
kssenii
f4b45a5805 Better profile events 2022-05-10 16:00:53 +02:00
Vladimir C
4065b83ba9
Update ConcurrentHashJoin.cpp 2022-05-10 14:14:58 +02:00
Vladimir C
43935f71f7
Update ConcurrentHashJoin.cpp 2022-05-10 14:09:49 +02:00
Anton Popov
f075070fdf
Merge pull request #37046 from azat/fix-optimize_aggregation_in_order-Array 
Fix optimize_aggregation_in_order with *Array aggregate functions
 2022-05-10 14:01:01 +02:00
Anton Popov
aec30c4076
Merge pull request #37053 from CurtizJ/remove-streams-comments 
Remove last mentions of data streams
 2022-05-10 13:38:13 +02:00
Vasily Nemkov
0cbc9c18e2
No debug messge if user not set 2022-05-10 13:39:23 +03:00
Vxider
5d0a5d34c8 removing function populate in windowview 2022-05-10 15:06:58 +08:00
taiyang-li
fd878f7e7b change as requested 2022-05-10 12:06:50 +08:00
Alexey Milovidov
5d8fad8ab6
Update ConvertFunctionOrLikeVisitor.cpp 2022-05-10 02:23:24 +03:00
kssenii
2f0f76369a Better 2022-05-09 21:25:26 +02:00
Anton Popov
e911900054 remove last mentions of data streams 2022-05-09 19:15:24 +00:00
Azat Khuzhin
323ae98202 Fix optimize_aggregation_in_order with prefix GROUP BY and *Array aggregate functions 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Fixes: #35111
See-also: #37046
 2022-05-09 21:32:40 +03:00
Azat Khuzhin
6ada8a6337 Fix optimize_aggregation_in_order with *Array aggregate functions 
row_begin was wrong, and before this patch aggregator processing
{row_end, row_end} range, in other words, zero range.

Fixes: #9113 (cc @dimarub2000)
v2: add static_cast to fix UBSan
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-05-09 20:43:19 +03:00
mergify[bot]
55a6d22ad3
Merge branch 'master' into grouping-sets-fix 2022-05-09 14:02:10 +00:00
Nikolai Kochetov
a34411f51e Disable trivial_insert_select if query has aggregate functions. 
(cherry picked from commit 70ae043455)
 2022-05-09 15:56:39 +02:00
Vasily Nemkov
de9a06df5b
Merge branch 'master' into fix_session_log_crash 2022-05-09 17:00:24 +04:00
Vasily Nemkov
58953b5f28 Fixed issues with concurrent access to User object 
ContextAccess might be updated from another thread, nullifying UserPtr.
Can happen if user was dropped in concurrent query.

This could crash server under certain conditions.
 2022-05-09 15:56:50 +03:00
Alexander Tokmakov
b6be194d3a
Merge pull request #36976 from ClickHouse/replicated_database_fixes 
Fixes for Replicated database
 2022-05-09 14:41:49 +03:00
Nikolai Kochetov
ebfeca2c86 Try to move some code from SourceWithProgress to PipelineExecutor. 2022-05-09 10:28:05 +00:00
Alexey Milovidov
fec7447ab4
Update ConvertFunctionOrLikeVisitor.cpp 2022-05-09 06:16:10 +03:00
Alexey Milovidov
56abce6a75 Merge branch 'chained_or' of github.com:danlark1/ClickHouse into chained-or 2022-05-08 23:37:09 +02:00
Robert Schulze
61cbcbf073
Enable clang-tidy readability-misleading-indentation 
Official docs:

  Correct indentation helps to understand code. Mismatch of the
  syntactical structure and the indentation of the code may hide serious
  problems.
 2022-05-08 19:12:01 +02:00
Vxider
34f1821eb9 function now support for windowview populate 2022-05-08 23:43:08 +08:00
Vxider
e7c4eb2697 Merged with master 2022-05-08 18:15:31 +08:00
Alexey Milovidov
2b88c0898a
Update ConvertFunctionOrLikeVisitor.cpp 2022-05-07 14:11:06 +03:00
Kseniia Sumarokova
0f5f75a784
Merge pull request #36887 from kssenii/fix-current-size-count-in-cache 
Fix current size count in cache
 2022-05-07 11:45:20 +02:00
Dmitry Novik
8d59211f01 Remove redundant ActionType 2022-05-06 18:50:21 +00:00
Alexander Tokmakov
c806869040 fix sync database replica 2022-05-06 18:37:20 +02:00
vdimir
d712985575
Minor style changes for ConcurrentHashJoin 2022-05-06 15:17:46 +00:00
Vladimir C
bd5fab97d9
Merge pull request #36415 from bigo-sg/concurrent_join 2022-05-06 17:11:10 +02:00
Anton Popov
13e8db6299
Merge pull request #36762 from CurtizJ/dynamic-columns-12 
Fix insertion to columns of type `Object` from multiple files
 2022-05-06 14:14:32 +02:00
kssenii
32d004017b Merge origin 2022-05-06 13:54:51 +02:00
kssenii
fd1ab053f8 Merge master 2022-05-06 12:50:33 +02:00
Dmitry Novik
b4e1dbc3c8 Remove whitespaces 2022-05-05 18:35:41 +00:00
Dmitry Novik
9a251e0028 Cleanup code 2022-05-05 18:13:00 +00:00
Dmitry Novik
4cc26aa38b Merge remote-tracking branch 'origin/master' into grouping-sets-fix 
And fix execution of the query with only one grouping set
 2022-05-05 17:14:52 +00:00
Vxider
9eea003d69 Merged with master 2022-05-06 00:12:17 +08:00
mergify[bot]
2a8aad42dc
Merge branch 'master' into add-asynchronous-metrics-for-filesystem-cache 2022-05-05 15:43:38 +00:00
Vxider
a7dc1f48e5 populate support for windowview 2022-05-05 23:31:15 +08:00
Dmitry Novik
161f52292b Support distributed queries 2022-05-05 13:56:16 +00:00
Vitaly Baranov
dd51265218
Merge pull request #36864 from vitlibar/backup-improvements-4 
Backup improvements
 2022-05-05 15:37:51 +02:00
Anton Popov
fa1feb238e
Merge pull request #36853 from qoega/add-other-time-microsecons 
Add Other Query Time Microseconds Profile Event
 2022-05-05 13:58:13 +02:00
lgbo-ustc
595ba6bd9f udpate dispatchBlock() 2022-05-05 12:04:11 +08:00
lgbo-ustc
f9f241e566 fixed fast test. remove unused header 2022-05-05 11:16:33 +08:00
lgbo-ustc
29a8db1345 fixed fast test 2022-05-05 10:32:19 +08:00
lgbo-ustc
2ddccdc21c update codes 2022-05-05 10:02:50 +08:00
alesapin
aba78e5ed0
Merge pull request #36859 from ClickHouse/revert-36858-revert-35637-memory-overcommit-free 
Revert "Revert "Memory overcommit: continue query execution if memory is available""
 2022-05-04 23:43:00 +02:00
Kruglov Pavel
77e55c344c
Merge pull request #36667 from Avogar/mysqldump-format 
Add MySQLDump input format
 2022-05-04 19:49:48 +02:00
Robert Schulze
c48d9a4174
Merge pull request #36844 from rschu1ze/shared_ptr_helper3 
Remove inherited create() method + disallow copying of storages
 2022-05-04 19:16:30 +02:00
Robert Schulze
78cdb3ec64
Merge pull request #36862 from rschu1ze/new-clangtidies 
Activated a bunch of LLVM 12/13/14 clang-tidy warnings
 2022-05-04 15:26:57 +02:00
Kseniia Sumarokova
f11cfdca94
Merge pull request #36874 from kssenii/add-size-to-system-cache-log 
Add column to system.filesystem_cache_log
 2022-05-04 11:53:29 +02:00
Robert Schulze
45c0ec44b0
Fix build, pt. II 2022-05-04 10:15:25 +02:00
mergify[bot]
64084b5e32
Merge branch 'master' into shared_ptr_helper3 2022-05-03 20:46:16 +00:00
mergify[bot]
17aecac7ff
Merge branch 'master' into new-clangtidies 2022-05-03 19:44:01 +00:00
kssenii
8f99046100 Add come asynchronous metrics 2022-05-03 20:02:19 +02:00
kssenii
0266cdf125 Add entries to cache log when cache was not event attempted 2022-05-03 18:29:41 +02:00
Nikolai Kochetov
35095191eb Merge branch 'master' into refactor-something-in-part-volumes 2022-05-03 17:51:47 +02:00
mergify[bot]
ddd1c3e1b3
Merge branch 'master' into add-other-time-microsecons 2022-05-03 15:45:40 +00:00
Dmitry Novik
5ba7a55c18
Merge pull request #36650 from bigo-sg/hive_text_parallel_parsing 
Parallel parsing of hive text format
 2022-05-03 15:56:28 +02:00
kssenii
000f2c9f7a Better 2022-05-03 15:55:36 +02:00
Maksim Kita
36d1c8238a
Merge pull request #36766 from kitaisreal/evaluate-constant-expression-fix 
Fix evaluateConstantExpression for subqueries
 2022-05-03 12:32:14 +02:00
Vitaly Baranov
5257ce31f8 Improved using ThreadPool for making backup or restoring, changed columns in system.backups. 2022-05-03 11:03:13 +02:00
Vitaly Baranov
409edfd3fa Rework RestoreCoordination: make restore deterministic. 2022-05-03 11:01:44 +02:00
Vitaly Baranov
2c754f44fc Make calculation of shard_num & replica_num not dependant on match of the cluster's definitions on nodes. 2022-05-03 11:01:44 +02:00
Vitaly Baranov
2a645bb187 Fix sending 'create_table' and 'create_database' restore settings to cluster. 2022-05-03 11:01:44 +02:00
Vitaly Baranov
bddec55d35 Added ASTBackupQuery::setDatabase(). 2022-05-03 11:01:44 +02:00
Robert Schulze
0a4eccb73e
Activated a bunch of LLVM 12/13/14 clang-tidy warnings 
Omitted new checks which produce too many matches or which are
controversial (e.g. readability-identifier-length).

New checks:

- misc-misleading-bidirectional + misc-misleading-identifier

  Detects potential attack as described in the Trojan Source attack

- modernize-macro-to-enum

  Replaces groups of adjacent macros with an unscoped anonymous enum

- modernize-shrink-to-fit

  Replace copy and swap tricks on shrinkable containers with the
  shrink_to_fit() method call

- modernize-use-transparent-functors

  Prefer transparent functors to non-transparent ones

- modernize-use-uncaught-exceptions

  This check will warn on calls to std::uncaught_exception and replace
  them with calls to std::uncaught_exceptions (uncaught_exception was
  deprecated with C++17)

- performance-no-int-to-ptr

  Diagnoses every integer to pointer cast

- readability-duplicate-include

  Looks for duplicate includes and removes them

- readability-redundant-preprocessor

  Finds potentially redundant preprocessor directives

- bugprone-lambda-function-name

  Checks for attempts to get the name of a function from within a lambda
  expression

- bugprone-redundant-branch-condition

  Finds condition variables in nested if statements that were also
  checked in the outer if statement and were not changed

- bugprone-shared-ptr-array-mismatch

  Finds initializations of C++ shared pointers to non-array type that
  are initialized with an array

- bugprone-stringview-nullptr

  Checks for various ways that the const CharT* constructor of
  std::basic_string_view can be passed a null argument and replaces them
  with the default constructor in most cases

- bugprone-suspicious-memory-comparison

  Finds potentially incorrect calls to memcmp() based on properties of
  the arguments
 2022-05-03 09:22:11 +02:00
Dmitry Novik
2ed5a4013a
Revert "Revert "Memory overcommit: continue query execution if memory is available"" 2022-05-03 00:45:13 +02:00
alesapin
f0b7af0aa2
Revert "Memory overcommit: continue query execution if memory is available" 2022-05-03 00:36:50 +02:00
Dmitry Novik
71b6f89166
Merge pull request #35637 from ClickHouse/memory-overcommit-free 
Memory overcommit: continue query execution if memory is available
 2022-05-02 19:00:18 +02:00
mergify[bot]
93f5aa7488
Merge branch 'master' into aggregator-jit-lock-fix 2022-05-02 16:16:36 +00:00
Dmitry Novik
9be17ef50c
Merge pull request #35111 from azat/optimize_aggregation_in_order-prefix 
Implement partial GROUP BY key for optimize_aggregation_in_order
 2022-05-02 17:49:48 +02:00
Ilya Yatsishin
6c3bbb4933 Add Other Time Microseconds Profile Event 2022-05-02 17:13:57 +02:00
Maksim Kita
c059629f8d Aggregator JIT compilation lock fix 2022-05-02 16:21:10 +02:00
Maksim Kita
2377106370 Fix build 2022-05-02 14:27:00 +02:00
Kruglov Pavel
b98ac57090
Merge pull request #36803 from CurtizJ/fix-unflatten-nested 
Fix creation of tables with `flatten_nested = 0`
 2022-05-02 13:58:16 +02:00
Kruglov Pavel
d613f7eab0
Merge branch 'master' into mysqldump-format 2022-05-02 13:31:57 +02:00
Robert Schulze
330212e0f4
Remove inherited create() method + disallow copying 
The original motivation for this commit was that shared_ptr_helper used
std::shared_ptr<>() which does two heap allocations instead of
make_shared<>() which does a single allocation. Turned out that
1. the affected code (--> Storages/) is not on a hot path (rendering the
performance argument moot ...)
2. yet copying Storage objects is potentially dangerous and was
   previously allowed.

Hence, this change

- removes shared_ptr_helper and as a result all inherited create() methods,

- instead, Storage objects are now created using make_shared<>() by the
  caller (for that to work, many constructors had to be made public), and

- all Storage classes were marked as noncopyable using boost::noncopyable.

In sum, we are (likely) not making things faster but the code becomes
cleaner and harder to misuse.
 2022-05-02 08:46:52 +02:00
Kseniia Sumarokova
e3d2d72b39
Merge pull request #36802 from KinderRiven/local_cache_trace 
system table used for statistical query of remote file access
 2022-05-02 00:16:04 +02:00
Azat Khuzhin
3982638e9a Validate that function had been passed in CREATE FUNCTION 
Otherwise right now it leads to LOGICAL_ERROR, due to bad cast.
 2022-05-01 19:01:03 +03:00
Nikita Mikhaylov
93a65463d0
Added SYSTEM SYNC DATABASE query (#35944) 2022-05-01 15:40:18 +02:00
KinderRiven
9413aa423b fix 2022-05-01 19:45:16 +08:00
KinderRiven
7d0789472c fix 2022-05-01 19:45:16 +08:00
KinderRiven
7db67160fe fix 2022-05-01 19:45:16 +08:00
KinderRiven
e1acacf831 fix 2022-05-01 19:45:16 +08:00
KinderRiven
01210c8d38 fix header 2022-05-01 19:45:16 +08:00
KinderRiven
d80aa0fd17 impl system.filesystem_cache_log 2022-05-01 19:45:16 +08:00
KinderRiven
d595df1604 fix 2022-05-01 19:45:16 +08:00
KinderRiven
6a2e4559c8 impl system.cache_log 2022-05-01 19:45:16 +08:00
KinderRiven
ce9a6965d0 impl system.log_table 2022-05-01 19:45:16 +08:00
KinderRiven
a4b8f9e9cf impl system.cache_log 2022-05-01 19:45:16 +08:00
KinderRiven
9c9870f021 build cache_log table 2022-05-01 19:45:16 +08:00
KinderRiven
463b42b4c3 remove unused var 2022-05-01 19:45:16 +08:00
KinderRiven
5a361a0db5 impl cache_log in filecache 2022-05-01 19:45:16 +08:00
KinderRiven
4db54ff6bc add log record 2022-05-01 19:45:16 +08:00
KinderRiven
29f0471caf add cache log 2022-05-01 19:45:16 +08:00
KinderRiven
0059a660c7 impl create cache_log table 2022-05-01 19:45:16 +08:00
KinderRiven
02080aad1c impl create cache_log table 2022-05-01 19:45:16 +08:00
KinderRiven
44c9996a1e impl create cache_log table 2022-05-01 19:45:16 +08:00
KinderRiven
d5f287369e add include 2022-05-01 19:45:16 +08:00
KinderRiven
ee6ef29538 add include 2022-05-01 19:45:16 +08:00
KinderRiven
ef863a7942 add include 2022-05-01 19:45:16 +08:00
KinderRiven
021a3a788b add cache log table in system database 2022-05-01 19:45:16 +08:00
KinderRiven
6f930be28b impl cache trace 2022-05-01 19:45:16 +08:00
Alexey Milovidov
b034146ba4
Merge pull request #36799 from azat/cleanup 
Tiny cleanup
 2022-05-01 14:23:13 +03:00
Alexey Milovidov
321514a7f3
Merge pull request #36816 from azat/is_all_data_sent-on-exception 
Set is_all_data_sent on exceptions too
 2022-05-01 14:22:06 +03:00
Nikita Taranov
0fd9740c72
Log hash table's cache messages with TRACE level (#36830) 2022-05-01 12:54:54 +02:00
Alexey Milovidov
189938e935
Merge pull request #36431 from rschu1ze/bugprone-branch-clone 
Fixed warnings of clang-tidy check "bugprone-branch-clone"
 2022-05-01 13:49:37 +03:00
Alexey Milovidov
91d1ac913c
Merge pull request #36820 from azat/fix-asynchronous_metric_log 
Avoid recreation of system.asynchronous_metric_log (due to difference in codec)
 2022-05-01 13:42:17 +03:00
Robert Schulze
89aa9ae00f
Fixed clang-tidy check "bugprone-branch-clone" 
The check is currently *not* part of .clang-tidy. It complains about:
(1) "switch has multiple consecutive identical branches"
(2) "repeated branch in conditional chain"

About (1): Lots of findings in switches were about redundant
"[[fallthrough]]" in places where the compiler would not warn anyways. I
have cleaned these up.

About (2): In if-else_if-else chains, fixing the warning would usually
mean concatenating multiple if-conditions. As this would reduce
readability in most cases, I did not fix these places.

Because of (2), I also refrained from adding "bugprone-branch-clone" to
.clang-tidy.
 2022-04-30 19:40:28 +02:00
mergify[bot]
cc08ccb420
Merge branch 'master' into remove-useless-code-2 2022-04-30 12:48:15 +00:00
Nikita Mikhaylov
117ffa90cf
Fix ungrammatical error message (#36760) 2022-04-30 13:57:13 +02:00
Azat Khuzhin
d82f2ff162 Set is_all_data_sent on exceptions too 
This will avoid clickhouse-test complains about left queries after the
test, like in [1].

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/36258/9646487c093a75dc31e3e818417cfad83580b40f/stateful_tests__memory__actions_.html

Follow-up for: #36649
 2022-04-30 13:00:44 +03:00
Azat Khuzhin
a75559aa36 Avoid recreation of system.asynchronous_metric_log (due to difference in codec) 
From the log:

    2022.04.29 12:33:40.397528 [ 486 ] {} <Debug> SystemLog (system.asynchronous_metric_log): Existing table system.asynchronous_metric_log for system log has obsolete or different structure. Renaming it to asynchronous_metric_log_352.
    Old: CREATE TABLE system.asynchronous_metric_log (`event_date` Date CODEC(Delta(2), ZSTD(1)), `event_time` DateTime CODEC(Delta(4), ZSTD(1)), `metric` LowCardinality(String) CODEC(ZSTD(1)), `value` Float64 CODEC(Gorilla, ZSTD(3))) ENGINE = MergeTree PARTITION BY toYYYYMM(event_date) ORDER BY (metric, event_date, event_time) SETTINGS index_granularity = 8192
    New: CREATE TABLE system.asynchronous_metric_log (`event_date` Date CODEC(Delta(2), ZSTD), `event_time` DateTime CODEC(Delta(4), ZSTD), `metric` LowCardinality(String) CODEC(ZSTD), `value` Float64 CODEC(Gorilla, ZSTD(3))) ENGINE = MergeTree PARTITION BY toYYYYMM(event_date) ORDER BY (metric, event_date, event_time) SETTINGS index_granularity = 8192

As you can see the difference in default ZSTD level.

P.S. I decided not to fix the printer, since this may introduce changes
in metadata, that may lead to changes in metadata between ZooKeeper and
clickhouse server.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-30 12:19:29 +03:00
Alexey Milovidov
1ddb04b992
Merge pull request #36715 from amosbird/refactorbase 
Reorganize source files so that base won't depend on Common
 2022-04-30 09:40:58 +03:00
mergify[bot]
e9fde5d067
Merge branch 'master' into memory-overcommit-free 2022-04-29 19:31:59 +00:00
Nikolai Kochetov
5807ca47b0
Merge pull request #36425 from ClickHouse/pool-size-hot-reload 
Allow to increase number of threads and tasks for background executors
 2022-04-29 17:24:12 +02:00
Anton Popov
cef462f83d fix creation of tables with flatten_nested = 0 2022-04-29 14:53:08 +00:00
Azat Khuzhin
16b1d2bee4 Remove outdated comment from ThreadStatusExt 
Refs: https://github.com/ClickHouse/ClickHouse/pull/16121#discussion_r511500850

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-29 17:04:56 +03:00
Azat Khuzhin
b7b7d91bd1 Remove memory reservation for SquashingTransform 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-29 17:04:56 +03:00
Vladimir C
e1e7a67670
Merge pull request #36201 from vdimir/join_dup_columns_issue36199 2022-04-29 12:57:10 +02:00
lgbo-ustc
5021cb707c update 2022-04-29 17:41:26 +08:00
vdimir
81c24fc521
Fix assertion in join 2022-04-29 09:15:00 +00:00
Maksim Kita
5b9bb36f80
Merge pull request #36753 from kitaisreal/added-user-defined-path-config-setting 
Added user_defined_path config setting
 2022-04-29 11:10:51 +02:00
Alexey Milovidov
b81da324de
Merge pull request #36765 from ClickHouse/fix_flaky_test_non_leader_replica 
Fix flaky test `test_ddl_worker_non_leader`
 2022-04-29 11:18:49 +03:00
Amos Bird
9d30be2c59
Fix build again 2022-04-29 14:45:27 +08:00
Azat Khuzhin
0ce44f3021 Optimize optimize_aggregation_in_order with a prefix key 
Before it does lots of extra work, now, it will be significantly more
optimal (thousands of rows -> 1-2 million of rows).

v2: s/executeOnBlockSimple/executeOnBlockSmall/
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-29 06:58:28 +03:00
Azat Khuzhin
190ce217bb Disable GROUP BY statistics for optimize_aggregation_in_order 
This statistics significantly decrease performance of
optimize_aggregation_in_order with a prefix key.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-29 06:58:27 +03:00
Azat Khuzhin
767acd53fb Add ability to pass range of rows to Aggregator 
v2: fix compiled aggregate functions (seek result to row_start)
v3: fix compiled aggregate functions (seek args to row_start)
v4: change signatures for JIT
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-29 06:57:55 +03:00
Amos Bird
4a5e4274f0
base should not depend on Common 2022-04-29 10:26:35 +08:00
Maksim Kita
04429d85db Added user_defined_path config setting 2022-04-28 22:13:33 +02:00
Maksim Kita
55994afe0c Fix evaluateConstantExpression for subqueries 2022-04-28 22:09:29 +02:00
Alexander Tokmakov
6797e7dc76 fix flaky test 2022-04-28 21:39:45 +02:00
Anton Popov
1fc51e09ff fix insertion to column of type Object from multiple files via table function 2022-04-28 18:51:13 +00:00
Alexey Milovidov
d8fa806fca
Merge pull request #36360 from ClickHouse/optimize-asynchronous-metric-log 
Optimize `system.asynchronous_metric_log`
 2022-04-28 20:16:17 +03:00
Vadim Volodin
233f0b4b52 Handle cluster macro in s3Cluster table function 2022-04-28 18:38:17 +03:00
Kruglov Pavel
4d08587559
Merge branch 'master' into mysqldump-format 2022-04-28 15:58:18 +02:00
zhanglistar
7d798b3655
Count distinct optimization by using subquery of group by (#35993) 2022-04-28 14:55:37 +02:00
Maksim Kita
ec47abe9af
Merge pull request #36482 from Enmk/fix_session_log_error_on_local_and_tcp_interserver_session 
Fixed missing enum values for ClientInfo::Interface
 2022-04-28 13:06:19 +02:00
Vitaly Baranov
641a5f5e35
Merge pull request #36198 from vitlibar/backup-improvements-3 
Backup for replicated tables and other improvements
 2022-04-28 09:23:53 +02:00
Alexey Milovidov
60e512ac4b Fix for table structure comparison 2022-04-28 05:10:57 +02:00
mergify[bot]
b7e63154df
Merge branch 'master' into optimize-asynchronous-metric-log 2022-04-27 18:27:27 +00:00
Kseniia Sumarokova
0d09bb9897
Merge pull request #36699 from amosbird/bettermysqlhandler 
Slightly better mysql handler
 2022-04-27 17:07:35 +02:00
Amos Bird
1cba00478d
Slightly better mysql handler 2022-04-27 18:28:09 +08:00
lgbo-ustc
10b7e08319 update 2022-04-27 16:41:34 +08:00
lgbo-ustc
7505ce426a update 2022-04-27 13:11:24 +08:00
taiyang-li
0341880250 Merge remote-tracking branch 'origin/master' into hive_text_parallel_parsing 2022-04-27 11:15:16 +08:00
Alexey Milovidov
b8a9f1411a
Merge pull request #36649 from azat/system.processes-is_all_data_sent 
clickhouse-test: improve left queries after the test hardening
 2022-04-27 05:21:22 +03:00
lgbo-ustc
520b05b9f1 update test case tests/queries/0_stateless/02236_explain_pipeline_join.sql 2022-04-27 10:08:22 +08:00
Nikolai Kochetov
e44af67fee Merge branch 'master' into refactor-something-in-part-volumes 2022-04-26 21:08:00 +02:00
Vitaly Baranov
a8e924caf6 Make BACKUP & RESTORE synchronous by default. 2022-04-26 18:45:39 +02:00
avogar
33d845dade Add MySQLDump input format 2022-04-26 10:42:56 +00:00
lgbo-ustc
20fc676bff fixed bug: resize on left pipeline cause the order by result wrong 2022-04-26 18:27:42 +08:00
mergify[bot]
7571387a6e
Merge branch 'master' into memory-overcommit-free 2022-04-26 10:15:18 +00:00
lgbo-ustc
d96c29810a fixed bug: resize on left pipeline cause the order by result wrong 2022-04-26 18:12:14 +08:00
lgbo-ustc
0b0fa8453b fixed bug: resize on left pipeline cause the order by result wrong 2022-04-26 18:06:16 +08:00
Kseniia Sumarokova
81db081b88
Merge pull request #36639 from kssenii/better-drop-cache 
Improve drop cache query
 2022-04-26 11:25:46 +02:00
Azat Khuzhin
6339a48923 Add is_all_data_sent into system.processes 
v2: fix SHOW PROCESSLIST (does not have process list entry)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-26 12:15:00 +03:00
Vitaly Baranov
78bcb96098 Rename backup & restore setting 'async' -> 'sync', and make backup & restore async by default. 2022-04-26 09:51:19 +02:00
taiyang-li
99dee35b6e parallel parsing of hive text format 2022-04-26 14:33:10 +08:00
mergify[bot]
4e82b56a8d
Merge branch 'master' into optimize-asynchronous-metric-log 2022-04-26 05:40:49 +00:00
lgbo-ustc
c7841d226b udpate 2022-04-26 11:29:18 +08:00
lgbo-ustc
74ccc233d2 Merge remote-tracking branch 'ck/master' into concurrent_join 2022-04-26 09:21:02 +08:00
alesapin
92296484e7
Merge pull request #36348 from rschu1ze/erase_if3 
Replace remove-erase idiom by C++20 erase()/erase_if()
 2022-04-25 23:34:18 +02:00
Vitaly Baranov
000b184691 Fix style & compilation. 2022-04-25 23:05:35 +02:00
alesapin
04afe552f8 Trying to avoid segfaults 2022-04-25 22:51:09 +02:00
alesapin
1fffa56f63 Merge branch 'master' into memory-overcommit-free 2022-04-25 20:21:08 +02:00
mergify[bot]
e79de5a577
Merge branch 'master' into optimize-asynchronous-metric-log 2022-04-25 17:29:11 +00:00
kssenii
a76a35d3ce Better drop cache 2022-04-25 19:20:25 +02:00
Vitaly Baranov
ed0af9d4a0 Improve shutdown with async backup/restore. 2022-04-25 16:34:34 +02:00
Vitaly Baranov
042dc4abb2 Rename settings shard -> shard_num, replica -> replica_num. 2022-04-25 16:34:34 +02:00
Vitaly Baranov
129eca7307 CREATE queries generated by RESTORE command is now considered as internal. 2022-04-25 16:34:34 +02:00
Vitaly Baranov
68a020ecea Implement BACKUP/RESTORE ON CLUSTER. 2022-04-25 16:34:33 +02:00
Vitaly Baranov
030f3e488c Add shard_index and replica_index to params of executeDDLQueryOnCluster(). 2022-04-25 16:34:33 +02:00
Vitaly Baranov
3477665659 Add shard_index and replica_index to params of getRewrittenASTWithoutOnCluster(). 2022-04-25 16:34:33 +02:00
Vitaly Baranov
144d3aefeb Add system table system.backups; implement async mode for commands BACKUP & RESTORE 2022-04-25 16:34:33 +02:00
Kseniia Sumarokova
2106a7b895
Merge pull request #36489 from awakeljw/fork_chmaster 
fix ConvertDecimalType unexpected behavior
 2022-04-25 13:23:54 +02:00
lgbo-ustc
c68806fbff fixed typos 2022-04-25 16:16:50 +08:00
lgbo-ustc
981d560553 Merge remote-tracking branch 'ck/master' into concurrent_join 2022-04-25 13:00:04 +08:00
awakeljw
d58a21678a fix ConvertDecimalType unexpected behavior 2022-04-24 23:35:58 +08:00
lgbo-ustc
221bc9c970 update 2022-04-24 17:14:13 +08:00
lgbo-ustc
a38b580c6f update 2022-04-24 15:19:16 +08:00
Alexey Milovidov
90297e225c
Merge pull request #36430 from amosbird/fixcolumnmatcher 
Fix column matcher and column transformer
 2022-04-24 09:11:21 +03:00
Alexey Milovidov
fe004c3486
Merge pull request #36486 from kitaisreal/executable-user-defined-functions-fix-group-by 
ExecutableUserDefinedFunction fix usage in GROUP BY
 2022-04-24 06:11:57 +03:00
Alexey Milovidov
909646247d
Update TranslateQualifiedNamesVisitor.cpp 2022-04-24 05:04:13 +03:00
lgbo-ustc
056b4d1b61 update 2022-04-24 09:26:57 +08:00
Nikolai Kochetov
8c00692844 Part 8 2022-04-22 16:58:09 +00:00
Nikita Mikhaylov
9baf7b14dc Fix build 2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
aaa7112405 Even better 2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
36bdee0499 Deleted settings 2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
4818b63979 Better 2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
8c325c2cdd Better 2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
31ccb9c1c3 Done 2022-04-22 13:56:16 +00:00
Nikita Mikhaylov
224f4dc620
Made parallel_reading_from_replicas work with localhost replica (#36281) 2022-04-22 15:52:38 +02:00
yuuch
d126de6dda
Apply suggestions from code review 
Co-authored-by: Vladimir C <vdimir@clickhouse.com>
 2022-04-22 20:00:12 +08:00
lgbo-ustc
681d7d6799 update dispatchBlock() 2022-04-22 11:00:00 +08:00
Dmitry Novik
6ee62007d3 Handle GROUPING SETS with size < 2 2022-04-21 23:52:43 +00:00
Maksim Kita
583cbcb834
Merge branch 'master' into fix_session_log_error_on_local_and_tcp_interserver_session 2022-04-21 21:15:11 +02:00
Maksim Kita
9713441650
Merge pull request #36480 from Joeywzr/session_log 
add missing enum values in session_log
 2022-04-21 21:14:12 +02:00
Maksim Kita
485c1d4072 Fixed tests 2022-04-21 19:10:11 +02:00
Dmitry Novik
b5e2b38529 Optimize pipeline in case of single input 2022-04-21 14:11:58 +00:00
Maksim Kita
57444fc7d3
Merge pull request #36444 from rschu1ze/clang-tidy-fixes 
Clang tidy fixes
 2022-04-21 16:11:27 +02:00
Dmitry Novik
df63a9b205 Depricate TOTALS with GROUPING SETS 2022-04-21 13:41:59 +00:00
Dmitry Novik
77a82cc090
Merge pull request #35631 from amosbird/projection-fix1 
Fix broken SET reuse during projection analysis.
 2022-04-21 15:32:52 +02:00
mergify[bot]
be6914f6f9
Merge branch 'master' into session_log 2022-04-21 11:52:20 +00:00
Maksim Kita
397603e9be ExecutableUserDefinedFunction fix usage in GROUP BY 2022-04-21 13:44:56 +02:00
Dmitry Novik
ea6a25b8e7 Fix nullptr dereference 2022-04-21 11:14:22 +00:00
Vasily Nemkov
e2866b0541 Fixed missing enum values for ClientInfo::Interface 2022-04-21 12:16:12 +03:00
Memo
ce91a261e8 add missing enum values in session_log 2022-04-21 17:01:44 +08:00
lgbo-ustc
a9d9abb7a1 use column::scatter() 2022-04-21 16:59:30 +08:00
lgbo-ustc
e27d67ce8a update type name 2022-04-21 14:28:22 +08:00
lgbo-ustc
eac810ed85 use concatenateBlocks 2022-04-21 14:10:09 +08:00
lgbo-ustc
60964b207d update addJoinedBlock() 2022-04-21 13:19:33 +08:00
lgbo-ustc
bfd1a0e33a update codes 2022-04-21 12:14:36 +08:00
Dmitry Novik
61deae7105
Merge branch 'master' into grouping-sets-fix 2022-04-21 03:34:42 +02:00
Dmitry Novik
6e73cd6929 Implement parallel grouping sets processing 2022-04-21 01:18:40 +00:00
Alexey Milovidov
ec9b86fd03
Merge branch 'master' into fixcolumnmatcher 2022-04-21 02:15:25 +03:00
Azat Khuzhin
c6ffec1fec Fix "Cannot find column" error for distributed queries with LIMIT BY 
Consider the following example:

    SELECT k FROM (SELECT materialize('foo') AS k, -1 AS v) ORDER BY abs(v) AS _v ASC LIMIT 1 BY k

The problem here is that in case of query has LIMIT BY for
WithMergeableState* (which is the final state on the remote shard for
distributd queries) it returns the following columns:
- k
- v

While it should return:
- k
- abs(v)

So as query w/o LIMIT BY, so that initiator will be able to do the sorting

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-20 15:23:24 +03:00
lgbo-ustc
f461d18de5 update method name 2022-04-20 19:51:35 +08:00
lgbo-ustc
c844415578 add new join_algorithm parallel_hash 2022-04-20 19:47:16 +08:00
Robert Schulze
7550d9b281
Fix build 2022-04-20 13:34:17 +02:00
Alexander Tokmakov
1d30a97fd2 Merge branch 'master' into remove-useless-code-2 2022-04-20 11:45:56 +02:00
Robert Schulze
b24ca8de52
Fix various clang-tidy warnings 
When I tried to add cool new clang-tidy 14 warnings, I noticed that the
current clang-tidy settings already produce a ton of warnings. This
commit addresses many of these. Almost all of them were non-critical,
i.e. C vs. C++ style casts.
 2022-04-20 10:29:05 +02:00
Kseniia Sumarokova
766a84e64f
Merge pull request #35915 from kssenii/better-s3-settings 
Better settings configuration for s3
 2022-04-20 10:22:06 +02:00
Robert Schulze
b6d7367538
Merge remote-tracking branch 'origin/master' into erase_if3 
Conflicts:
- Interpreters/ActionsDAG.cpp
 2022-04-20 10:02:59 +02:00
Antonio Andelic
bbb0be6a44
Merge pull request #36402 from rschu1ze/clang-tidy-contains 
Activate clang-tidy warning "readability-container-contains"
 2022-04-20 08:15:46 +02:00
Kruglov Pavel
894b1b163e
Merge pull request #36055 from helifu/master2 
Don't materialize external tables if it's explain statement
 2022-04-19 20:26:27 +02:00
Kruglov Pavel
7fb7fc9daa
Merge pull request #36275 from Avogar/fix-insert-select 
Place new logic of schema inference in insert select from table function under setting
 2022-04-19 20:19:28 +02:00
Amos Bird
158a25d5fa
Fix column matcher and column transformer 2022-04-20 01:22:04 +08:00
Robert Schulze
72a331189e
Fix glitch 2022-04-19 14:59:47 +02:00
mergify[bot]
3c1dd44b80
Merge branch 'master' into fix-insert-select 2022-04-19 12:17:31 +00:00
lgbo-ustc
26ff749a32 remove unused header 2022-04-19 19:12:08 +08:00
lgbo-ustc
3f2fe3ff66 fixed code style 2022-04-19 18:47:29 +08:00
kssenii
1f9a597b77 Merge master 2022-04-19 11:31:13 +02:00
lgbo-ustc
9dfe2a6682 fixed header style 2022-04-19 16:53:24 +08:00
lgbo-ustc
3d7338581b Improve join 
now adding joined blocks from right table can be run parallelly, speedup the join process
 2022-04-19 16:07:30 +08:00
Robert Schulze
118e94523c
Activate clang-tidy warning "readability-container-contains" 
This check suggests replacing <Container>.count() by
<Container>.contains() which is more speaking and in case of
multimaps/multisets also faster.
 2022-04-18 23:53:11 +02:00
Alexey Milovidov
1e1ffbbfe2
Update SessionLog.cpp 2022-04-18 23:07:34 +03:00
mergify[bot]
daca157111
Merge branch 'master' into fix_storage_distributed_ttl 2022-04-18 17:31:59 +00:00
Alexey Milovidov
0af183826d
Merge pull request #36358 from ClickHouse/don-t-randomize-priority 
Do not randomize "priority" setting
 2022-04-18 07:04:36 +03:00
Alexey Milovidov
f6ab2bd523
Merge pull request #36312 from ClickHouse/remove-arcadia 
Remove remaining parts of Arcadia
 2022-04-18 07:02:54 +03:00
Alexey Milovidov
36595e4206
Merge pull request #36320 from ClickHouse/fix-clang-tidy-14 
Fix clang-tidy-14 (part 1)
 2022-04-18 07:02:10 +03:00
Alexey Milovidov
70d7ba5d61 Fix clang-tidy 2022-04-18 02:13:28 +02:00
Alexey Milovidov
6de5d8a554 Merge branch 'master' into fix-clang-tidy-14 2022-04-18 02:12:45 +02:00
Alexey Milovidov
0051e651c6
Update AsynchronousMetricLog.h 2022-04-18 03:01:57 +03:00
Alexey Milovidov
242919eddd Remove abbreviation 2022-04-18 01:02:49 +02:00
Alexey Milovidov
8ea14d2664 Custom ORDER BY for system.asynchronous_metric_log 2022-04-17 23:59:05 +02:00
Alexey Milovidov
e36e22812a Custom column list for system.asynchronous_metric_log 2022-04-17 23:53:28 +02:00
Alexey Milovidov
300ef741cc Custom column list for system.asynchronous_metric_log 2022-04-17 23:51:56 +02:00
Alexey Milovidov
4c09e2d30a Custom column list for system.asynchronous_metric_log 2022-04-17 23:49:39 +02:00
Alexey Milovidov
37c61d9f0a Round values for the log 2022-04-17 22:07:50 +02:00
Alexey Milovidov
1f5df287c0 Remove useless code 2022-04-17 22:02:55 +02:00
Alexey Milovidov
6807f8428d Remove microseconds from system.asynchronous_metric_log 2022-04-17 22:01:58 +02:00
Alexey Milovidov
250ca83058 Allow custom default ORDER BY for system logs 2022-04-17 21:57:44 +02:00
Alexey Milovidov
a64765bc6f Allow custom default ORDER BY for system logs 2022-04-17 21:56:36 +02:00
Alexey Milovidov
aa93f4a074 Do not randomize "priority" setting 2022-04-17 20:48:02 +02:00
Alexey Milovidov
89deaf0af1
Merge pull request #36056 from azat/client-trailing-space 
Remove trailing whitespace in default client_name
 2022-04-17 17:37:43 +03:00
Robert Schulze
1e1df8e101
Replace remove-erase idiom by C++20 erase()/erase_if() 
- makes the code less verbose while being as efficient
 2022-04-17 12:04:47 +02:00
Alexey Milovidov
294efeccfe Fix clang-tidy-14 (part 1) 2022-04-16 04:54:04 +02:00
Alexey Milovidov
cbeeb7ec4f Remove Arcadia 2022-04-16 00:20:47 +02:00
Alexey Milovidov
a0fe29e243 Remove Arcadia 2022-04-15 23:59:49 +02:00
yuuch
6919ca9a6e remove parse about the not null in ParserCollation 2022-04-15 21:31:25 +08:00
avogar
ae01646136 Place new logic of schema inference in insert select from table function under setting 2022-04-14 19:05:20 +00:00
Dmitry Novik
a16710c750 Merge remote-tracking branch 'origin/master' into grouping-sets-fix 2022-04-14 17:29:51 +00:00
yuuch
29e0b4c3bb support collation in create query 2022-04-15 00:12:31 +08:00
Alexander Tokmakov
4ce0b280e6 wait for node to disappear instead of removing 2022-04-14 15:31:05 +02:00
Anton Popov
305dd57262
Merge branch 'master' into fix_storage_distributed_ttl 2022-04-14 14:51:15 +02:00
mergify[bot]
95901466e7
Merge branch 'master' into master2 2022-04-14 10:05:03 +00:00
Yakov Olkhovskiy
c1a06ac63a
Merge pull request #24896 from hexiaoting/dev_materialized_view 
Add type check when create materialized view with To clause
 2022-04-13 22:33:54 -04:00
Maksim Kita
ca89af89a1
Merge pull request #36088 from israelg99/fix_executable 
Fix CH silently failing when it cannot execute a file
 2022-04-13 16:53:00 +02:00
kssenii
0912b038fa Merge master 2022-04-13 15:30:01 +02:00
tavplubix
43de4827d7
Merge pull request #36074 from ClickHouse/zookeeper_log_thread_query_id 
Add thread_id and query_id to zookeeper_log
 2022-04-13 15:34:04 +03:00
mergify[bot]
ba22cb76bb
Merge branch 'master' into master2 2022-04-13 11:51:43 +00:00
Kruglov Pavel
03fc08cec3
Merge pull request #36137 from azat/client-profile-events 
ProfileEvents fixes
 2022-04-13 12:27:39 +02:00
Yakov Olkhovskiy
3d52afed71 only check columns sets intersection, some refactoring 2022-04-12 15:10:11 -04:00
tavplubix
303ecc6f7d
Merge pull request #36136 from ClickHouse/transactions_fix_1 
Minor fix for transactions
 2022-04-12 21:38:47 +03:00
Azat Khuzhin
e38057ddb0 Fix code alignment in ProfileEventsExt 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-12 20:54:56 +03:00
mergify[bot]
d5b2355944
Merge branch 'master' into memory-overcommit-free 2022-04-12 16:00:12 +00:00
Julian Gilyadov
a4f56f3330
Throw exception when file cant be executed instead of displaying success 2022-04-12 17:52:44 +02:00
Yakov Olkhovskiy
5868539605 check only if TO table exists 2022-04-12 10:17:39 -04:00
mergify[bot]
0b3c15c07a
Merge branch 'master' into projection-fix1 2022-04-12 13:49:28 +00:00
Yakov Olkhovskiy
25101de7e2 check only selected columns against destination table 2022-04-12 09:29:32 -04:00
tavplubix
2e95e0db3b
Merge pull request #36133 from Algunenano/osx_atomic_exchange 
Add support for atomic exchange in OSX
 2022-04-12 14:10:36 +03:00
Alexander Tokmakov
6e4787560b fix another issue 2022-04-12 11:44:25 +02:00
Yakov Olkhovskiy
dc314d53b3 do not create view before check, use MatchColumnsMode::Name mode 2022-04-11 20:28:11 -04:00
mergify[bot]
46b9a1ef80
Merge branch 'master' into client-trailing-space 2022-04-11 22:07:37 +00:00
Yakov Olkhovskiy
783dbb2e63
Merge branch 'master' into dev_materialized_view 2022-04-11 17:47:35 -04:00
Yakov Olkhovskiy
155a2a0d42
Merge pull request #35349 from yakov-olkhovskiy/interpolate-feature 
Interpolate feature
 2022-04-11 11:15:50 -04:00
Yakov Olkhovskiy
2588f8011f comment fix 2022-04-11 11:13:29 -04:00
Alexander Tokmakov
38c80940fd slightly beter fix 2022-04-11 16:25:59 +02:00
Yakov Olkhovskiy
7293e01647 some comments added 2022-04-11 09:47:27 -04:00
Raúl Marín
6e346a7fc3 Support atomic replace under OSX 2022-04-11 14:32:55 +02:00
Vladimir C
1a604cd279
Merge pull request #36058 from xiedeyantu/master 2022-04-11 10:45:50 +02:00
Vitaly Baranov
8dcb1358b7
Merge pull request #36072 from vitlibar/backups-improve-arguments-handling-and-file-removing 
Backups: improve arguments handling and file removing
 2022-04-10 16:51:04 +02:00
xiedeyantu
5efd8a4ec0 If data type has DEFAULT NULL in table definition, make data type Nullable 2022-04-09 15:00:41 +08:00
Vitaly Baranov
758f8235c2 Collect backup entries before opening an archive for writing. 2022-04-09 01:44:46 +02:00
Alexander Tokmakov
50a32a74b7 add thread_id and query_id to zookeeper_log 2022-04-08 17:10:08 +02:00
Yakov Olkhovskiy
87c2b3e2cf fixed Nullable, tests added 2022-04-08 10:52:10 -04:00
Alexander Tokmakov
141fbc6980 Merge branch 'master' into mvcc_prototype 2022-04-08 13:38:11 +02:00
Alexander Tokmakov
49c35f3261 Merge branch 'master' into mvcc_prototype 2022-04-08 13:34:40 +02:00
Vitaly Baranov
45dea9933b
Merge pull request #35883 from azat/format-settings 
Do not allow SETTINGS after FORMAT for INSERT queries
 2022-04-08 12:38:14 +02:00
kssenii
87855c4ae7 Merge master 2022-04-08 12:32:48 +02:00
alesapin
8ec802bc62
Merge pull request #35475 from kssenii/remote-fs-cache-improvements 
Allow to write remote fs cache on all write operations. Add `system.remote_filesystem_cache` table. Add `drop remote filesystem cache (<path>)` query. Add `system.remote_data_paths` table.
 2022-04-08 12:06:26 +02:00
Vladimir C
265fa90689
Merge pull request #36033 from vdimir/cross-join-empty-list 2022-04-08 11:59:21 +02:00
Azat Khuzhin
44bf8ba568 Remove trailing whitespace in default client_name 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-08 08:09:47 +03:00
helifu
1196dc3dbd Don't materialize external tables if it's explain statement 
The explain statement of GLOBAL JOIN two distributed tables can speed up 100x:
  explain plan select ... from t1_dist global join t2_dist on ...
  explain pipeline select ... from t1_dist global join t2_dist on ...
 2022-04-08 09:00:55 +08:00
Yakov Olkhovskiy
64dcddc6e3 fixed ASTInterpolateElement::clone, fixed QueryNormalizer to exclude ASTInterpolateElement::children 2022-04-07 17:41:05 -04:00
Alexander Tokmakov
6a46da93ae Merge branch 'master' into mvcc_prototype 2022-04-07 23:22:19 +02:00
kssenii
5dce2f18b5 Better 2022-04-07 18:48:35 +02:00
Alexander Tokmakov
2e2e2b9190 fix a race condition 2022-04-07 18:17:43 +02:00
Azat Khuzhin
33d99c8ffb Introduce compatiblity mode for SETTINGS after FORMAT 
Add allow_settings_after_format_in_insert setting, OFF by default.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
v2: s/parser_settings_after_format_compact/allow_settings_after_format_in_insert/ (suggested by vitlibar)
v3: replace ParserSettings with a flag (requested by vitlibar)
 2022-04-07 16:36:34 +03:00
Alexander Tokmakov
7f54e7b422 Merge branch 'master' into mvcc_prototype 2022-04-07 15:14:06 +02:00
Kruglov Pavel
73adbb4c15
Merge pull request #35986 from amosbird/better-scalar1 
Fix performance regression of scalar query
 2022-04-07 14:07:59 +02:00
Alexander Tokmakov
8290ffa88d Merge branch 'master' into mvcc_prototype 2022-04-07 13:50:42 +02:00
vdimir
eb7ac4c3a6
Handle empty expression list in cross join 2022-04-07 11:32:32 +00:00
vdimir
69160fa3e3
remove dead code in needRewrite function 2022-04-07 10:46:39 +00:00
tavplubix
6e9c028bbb
Merge pull request #35956 from Algunenano/limit_dns_retries 
DNS cache: Add option to drop elements after several consecutive failures
 2022-04-07 13:20:36 +03:00
Nikolai Kochetov
3e1b3f14c0
Merge pull request #34355 from azat/processors-profiling 
Profiling on Processors level
 2022-04-07 12:13:14 +02:00
Amos Bird
1238babb6f
Make SelectQueryInfo pseudo-copyable 2022-04-07 17:46:50 +08:00
Amos Bird
69022850b9
Better 2022-04-07 17:46:50 +08:00
Amos Bird
9cf5935604
Fix broken SET reuse during projection analysis. 2022-04-07 17:46:49 +08:00
Yakov Olkhovskiy
7dbe8bc2dc major bugs fixed, tests added, docs updated 2022-04-07 01:21:24 -04:00
mergify[bot]
585cff1e95
Merge branch 'master' into limit_dns_retries 2022-04-06 18:23:17 +00:00
tavplubix
58db8cd9b3
Merge pull request #35946 from ClickHouse/faster_replicated_ddl 
Make some replicated DDL queries faster
 2022-04-06 19:25:25 +03:00
Amos Bird
df06f9f974
Fix performance regression of scalar query 2022-04-06 17:50:22 +08:00
Yakov Olkhovskiy
ac441b9dcf compiler suggestions 2022-04-06 00:05:06 -04:00
Yakov Olkhovskiy
90c4cd3de7
Merge branch 'master' into interpolate-feature 2022-04-05 14:39:07 -04:00
Yakov Olkhovskiy
e0d6033c39 all columns can participate in interpolate expression despite if they are selected or not, some optimization on expressionless INTERPOLATE 2022-04-05 14:26:49 -04:00
Alexander Tokmakov
37a06eec1a fixes 2022-04-05 17:36:53 +02:00
Alexander Tokmakov
d9e3e9b69f Merge branch 'master' into faster_replicated_ddl 2022-04-05 14:38:38 +02:00
Alexander Tokmakov
1fe50ad201 Merge branch 'master' into mvcc_prototype 2022-04-05 14:38:02 +02:00
Maksim Kita
c3c284e6e6
Merge pull request #35917 from kitaisreal/executable-user-defined-function-fix 
Executable user defined functions prevent executing during analysis
 2022-04-05 14:05:52 +02:00
Raúl Marín
8a05cf3927 DNS cache: Add option to drop elements after several consecutive failures 2022-04-05 13:00:27 +02:00
Nikolai Kochetov
4479b68980
Merge pull request #35623 from nickitat/function_calculation_after_sorting_and_limit 
Functions calculation after sorting
 2022-04-05 12:09:56 +02:00
alesapin
eaa72ef002
Merge pull request #35904 from ClickHouse/more_logs_on_unsuccessful_part_removal 
More logs on unsuccessful part removal
 2022-04-05 11:29:08 +02:00
Alexander Tokmakov
da00beaf7f Merge branch 'master' into mvcc_prototype 2022-04-05 11:14:42 +02:00
Alexander Tokmakov
4e9ec5dc2f make some replicated DDL faster 2022-04-05 00:51:48 +02:00
Alexey Milovidov
4d6c030d23
Revert "clang-tidy report issues with Medium priority" 2022-04-04 23:41:42 +03:00
Vladimir C
b80f24957e
Merge pull request #35674 from vdimir/asof-key-null-issue-35565 2022-04-04 16:56:04 +02:00
kssenii
4e50da722b Better s3 settings 2022-04-04 16:14:56 +02:00
Maksim Kita
2a8e479277 ExecutableUserDefinedFunction prevent function execution during query analysis 2022-04-04 15:56:01 +02:00
Nickita Taranov
698a984c07 throw if sorting column not found 2022-04-04 14:33:58 +02:00
Nickita Taranov
b095838444 stash 2022-04-04 14:33:57 +02:00
Nickita Taranov
4c51329ad6 stash 2022-04-04 14:33:57 +02:00
Alexander Tokmakov
a2167f12b8 Merge branch 'master' into mvcc_prototype 2022-04-04 14:24:23 +02:00
Nikita Taranov
bd89fcafdb
Make SortDescription::column_name always non-empty (#35805) 2022-04-04 14:17:15 +02:00
alesapin
885447342c More logs on unsuccessful part removal 2022-04-04 13:17:33 +02:00
Azat Khuzhin
e5a837e574 Add id/parent_ids into system.processors_profile_log 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-04 13:19:43 +03:00
Azat Khuzhin
99528e296c Rename need_data_elapsed_us/port_full_elapsed_us to input_wait_us/output_wait_us 
$ gg -e need_data_ -e port_full_  | cut -d: -f1 | sort -u | xargs sed -i -e s/port_full_/output_wait_/g -e s/need_data_/input_wait_/g -e s/getPortFull/getOutputWait/g -e s/getNeedData/getInputWait/g

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-04 13:19:43 +03:00
Azat Khuzhin
765b4bc45a Implement log_processors_profiles (write to processors_profile_log, OFF by default) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-04 13:19:43 +03:00
Azat Khuzhin
14538f6456 Add system.processors_profile_log 
This is the system table that will contain Processors level profiling.

v2: one entry per Processor, not 3 (PortFull/NeedData/work())
v3: us over ms
v4: Enable processors_profile_log table by default

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-04 13:19:41 +03:00
kssenii
36c583d0de Better version of cache on insert 2022-04-04 10:58:59 +02:00
Yakov Olkhovskiy
ff4d295265 style fix 2022-04-03 22:19:35 -04:00
Yakov Olkhovskiy
95ad1bf6e1 use aliases if exist for original_select_set 2022-04-03 22:10:36 -04:00
Alexey Milovidov
5a47958744
Merge pull request #35736 from CurtizJ/quota-written-bytes 
Add quota for written bytes
 2022-04-03 05:26:49 +03:00
Kruglov Pavel
687e9e7aeb
Merge pull request #35760 from Avogar/insert-from-input 
Improve schema inference for insert select from table functions
 2022-04-02 13:50:48 +02:00
Yakov Olkhovskiy
ec0ad8804a style fix 2022-04-01 21:45:58 -04:00
Yakov Olkhovskiy
0116233d36 allow INTERPOLATE to reference optimized out columns 2022-04-01 16:18:19 -04:00
Anton Popov
a10bf550c4
Update AsynchronousInsertQueue.cpp 2022-04-01 20:57:41 +02:00
Anton Popov
687942ce70 more strict quota for written bytes 2022-04-01 15:02:49 +00:00
Yakov Olkhovskiy
38993f215f
Merge pull request #35706 from yakov-olkhovskiy/ephemeral-35641 
Allow EPHEMERAL without explicit default expression
 2022-04-01 10:49:29 -04:00
kssenii
6c8e073a61 Merge master 2022-04-01 16:48:29 +02:00
Alexander Tokmakov
6591d1ceb7 Merge branch 'master' into mvcc_prototype 2022-04-01 15:38:46 +02:00
Yakov Olkhovskiy
a15996315e bugfix - columns order tracking 2022-03-31 11:51:13 -04:00
Yakov Olkhovskiy
b5682c1f02 minor refactoring 2022-03-31 08:33:50 -04:00
Kruglov Pavel
a217e9e9d0
Remove unused header 2022-03-31 14:27:43 +02:00
Alexander Tokmakov
461ecd56a4 fix race when database is Ordinary 2022-03-31 13:47:38 +02:00
Alexander Tokmakov
5a50ad9de3 Merge branch 'master' into mvcc_prototype 2022-03-31 11:35:04 +02:00
Antonio Andelic
e336302650 Merge branch 'master' into is-secure-client-info 2022-03-31 06:58:16 +00:00
Antonio Andelic
049559d510 Remove extra newline 2022-03-31 06:57:59 +00:00
Antonio Andelic
cdedcee0ce
Merge pull request #35746 from ClickHouse/use-common-io-thread-pool 
Use common IOThreadPool for S3 and URL
 2022-03-31 08:41:54 +02:00
Yakov Olkhovskiy
3b40b56e56 refactored to remove parsers rependency on datatypes 2022-03-30 23:47:16 -04:00
Nikita Taranov
30f2a942c5
Predict size of hash table for GROUP BY (#33439) 
* use AggregationMethod ctor with reserve

* add new settings

* add HashTablesStatistics

* support queries with limit

* support distributed and with external aggregation

* add new profile events

* add some tests

* add perf test

* export cache stats through AsynchronousMetrics

* rm redundant trace

* fix style

* fix 02122_parallel_formatting test

* review fixes

* fix 02122_parallel_formatting test

* apply also to two-level HTs

* try simpler strategy

* increase max_size_to_preallocate_for_aggregation for experiment

* fixes

* Revert "increase max_size_to_preallocate_for_aggregation for experiment"

This reverts commit 6cf6f75704.

* fix test

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2022-03-30 22:47:51 +02:00
Vasily Nemkov
dc88d8d571 Explicitly passing a user object to create a LogIn event 2022-03-30 23:37:24 +03:00
Yakov Olkhovskiy
6a1e116c46 refactoring 2022-03-30 16:34:19 -04:00
Alexander Tokmakov
996d57b918 try fix build with gcc 2022-03-30 17:42:47 +02:00
kssenii
0fc92fe2aa Get rid of all "remote" mentions 2022-03-30 17:15:29 +02:00
Vasily Nemkov
31e07fe078 Fixed crash when reporting successful login. 2022-03-30 17:14:55 +03:00
avogar
849596c7a2 Improve schema inference for insert select queries 2022-03-30 12:44:53 +00:00
Antonio Andelic
de2c6684a3 Remove setup and cleanup 2022-03-30 09:00:06 +00:00
Antonio Andelic
1173eab15e Use same IOThreadPool for URL and S3 2022-03-30 08:15:20 +00:00
Antonio Andelic
d14ad227b1 Polish TCP is_secure flag 2022-03-30 06:39:40 +00:00
Antonio Andelic
b982959c4c Fix writting is_secure 2022-03-30 06:39:40 +00:00
Antonio Andelic
893b8a9513 Initial implementation of is_secure 2022-03-30 06:39:40 +00:00
Anton Popov
caacc7d385 add quota for written bytes 2022-03-29 18:21:29 +00:00
Alexander Tokmakov
287d858fda Merge branch 'master' into mvcc_prototype 2022-03-29 16:24:12 +02:00
Antonio Andelic
9990abb76a Use compile-time check for Exception messages, fix wrong messages 2022-03-29 13:16:11 +00:00
Anton Popov
9610139477
Merge pull request #35629 from CurtizJ/dynamic-columns-5 
Support schema inference for type `Object` in format `JSONEachRow`
 2022-03-29 14:17:09 +02:00
Anton Popov
d42632dc13
Merge pull request #32928 from bigo-sg/rocksdb_metacache 
Speed up parts loading process of MergeTree by metadata cache
 2022-03-29 14:05:22 +02:00
Maksim Kita
a1a4552740
Merge pull request #35184 from DevTeamBK/clang-tidy-issues 
clang-tidy report issues with Medium priority
 2022-03-29 13:19:54 +02:00
taiyang-li
834ac3a1b9 Merge remote-tracking branch 'ck/master' into rocksdb_metacache 2022-03-29 18:32:06 +08:00
Anton Popov
5d8988ad4e
Merge pull request #35645 from CurtizJ/fix-quotas-async-inserts 
Fix usage of quota with async inserts
 2022-03-29 11:54:24 +02:00
Alexey Milovidov
5e262fba85
Merge pull request #35204 from azat/build-gcc 
Add build with GCC
 2022-03-29 04:55:15 +03:00
kssenii
44157e5c80 Merge master 2022-03-29 02:19:32 +02:00
Yakov Olkhovskiy
615efa1381 aliases processing fixed 2022-03-28 19:15:53 -04:00
Azat Khuzhin
bf4df5c6bb Fix SIGSEGV for build under gcc-11 (due to auto deduction) 
During building with gcc-11 you will got SIGSEGV for building
InterpretersMySQLDDLQuery (and some others), and it is due to endless
recursion:

    (gdb) bt 5
    0  0x00000000010978f2 in structural_comptypes (t1=0x7fde028c7dc8, t2=0x7fde028d1e70, strict=0) at ../../src/gcc/cp/typeck.c:1274
    1  0x00000000011c3f9d in comp_template_parms (parms1=<optimized out>, parms2=<optimized out>) at ../../src/gcc/cp/pt.c:3369
    2  0x0000000001097bd9 in structural_comptypes (t1=0x7fde028c7dc8, t2=0x7fde028d1e70, strict=<optimized out>) at ../../src/gcc/cp/typeck.c:1361
    3  0x00000000011c3f9d in comp_template_parms (parms1=<optimized out>, parms2=<optimized out>) at ../../src/gcc/cp/pt.c:3369
    4  0x0000000001097bd9 in structural_comptypes (t1=0x7fde028c7dc8, t2=0x7fde028d1e70, strict=<optimized out>) at ../../src/gcc/cp/typeck.c:1361
    (gdb) bt -X
    1397454 0x0000000001097bd9 in structural_comptypes (t1=0x7fde028d1540, t2=0x7fde028d27e0, strict=<optimized out>) at ../../src/gcc/cp/typeck.c:1361
    1397455 0x0000000000f2d8b5 in cp_tree_equal (t1=<optimized out>, t2=<optimized out>) at ../../src/gcc/cp/tree.c:4144
    1397456 0x00000000010909cb in template_args_equal (ot=0x7fde028cf578, nt=0x7fde028cfc58, partial_order=<optimized out>) at ../../src/gcc/cp/pt.c:9256
    1397457 0x0000000001090422 in template_args_equal (partial_order=false, nt=0x7fde028cfc58, ot=0x7fde028cf578) at ../../src/gcc/cp/pt.c:9295
    1397458 comp_template_args (oldargs=0x7fde028cf550, newargs=0x7fde028cfc30, oldarg_ptr=0x0, newarg_ptr=0x0, partial_order=false) at ../../src/gcc/cp/pt.c:9285
    1397459 0x00000000010a08f4 in spec_hasher::equal (e1=0x7fde028c95d0, e2=0x7ffd1194e8c0) at ../../src/gcc/cp/pt.c:1726
    1397460 0x0000000001085965 in hash_table<spec_hasher, false, xcallocator>::find_with_hash (this=0x7fde36b7f450, comparable=@0x7ffd1194e8b8: 0x7ffd1194e8c0, hash=<optimized out>) at ../../src/gcc/hash-table.h:936
    1397461 0x0000000001079698 in lookup_template_class_1 (d1=<optimized out>, arglist=0x7fde028cfc30, in_decl=0x0, context=<optimized out>, entering_scope=<optimized out>, complain=3) at ../../src/gcc/cp/pt.c:9896
    1397462 0x000000000109f8ef in lookup_template_class (complain=3, entering_scope=1, context=0x7fde27558e40, in_decl=0x0, arglist=0x7fde028cfc08, d1=0x7fde269bcd20) at ../../src/gcc/cp/pt.c:10251
    1397463 tsubst_aggr_type (t=0x7fde269bcd20, args=<optimized out>, complain=3, in_decl=0x0, entering_scope=1) at ../../src/gcc/cp/pt.c:13646
    1397464 0x000000000108f797 in tsubst (t=0x7fde269bcdc8, args=0x7fde028cf7a8, complain=3, in_decl=<optimized out>) at ../../src/gcc/cp/pt.c:16108
    1397465 0x0000000000e61bf2 in rewrite_template_parm (level=1, complain=3, tsubst_args=0x7fde028cf7a8, index=5, olddecl=0x7fde269b5600) at ../../src/gcc/cp/pt.c:28556
    1397466 rewrite_tparm_list(tree_node*, unsigned int, unsigned int, tree_node*, unsigned int, int) [clone .constprop.0] (oldelt=0x7fde269bd190, index=5, targs=0x7fde028cf7a8, targs_index=4, complain=3, level=1) at ../../src/gcc/cp/pt.c:28640
    1397467 0x00000000009f3748 in build_deduction_guide (type=type@entry=0x7fde26e13dc8, ctor=0x7fde269ac300, outer_args=outer_args@entry=0x0, complain=complain@entry=3) at ../../src/gcc/cp/pt.c:28769
    1397468 0x00000000009f444f in ctor_deduction_guides_for (complain=3, tmpl=<optimized out>) at ../../src/gcc/cp/cp-tree.h:842
    1397469 deduction_guides_for (tmpl=<optimized out>, any_dguides_p=<optimized out>, complain=3) at ../../src/gcc/cp/pt.c:29282
    1397470 0x00000000008507a8 in do_class_deduction (complain=3, flags=1, init=<optimized out>, tmpl=0x7fde26e0f980, ptype=0x7fde028c7b28) at ../../src/gcc/cp/pt.c:29402
    1397471 do_auto_deduction (type=0x7fde028c7b28, init=<optimized out>, auto_node=<optimized out>, complain=3, context=<optimized out>, outer_targs=<optimized out>, flags=1) at ../../src/gcc/cp/pt.c:29572
    1397472 0x00000000007c9569 in finish_compound_literal (type=<optimized out>, compound_literal=0x7fde028c95b8, complain=3, fcl_context=fcl_functional) at ../../src/gcc/cp/semantics.c:3060
    1397473 0x0000000001123a79 in cp_parser_functional_cast (parser=0x7fde27558da8, type=0x7fde028c7b28) at ../../src/gcc/cp/parser.c:30670
    1397474 0x0000000000fd7873 in cp_parser_postfix_expression (parser=0x7fde27558da8, address_p=<optimized out>, cast_p=<optimized out>, member_access_only_p=<optimized out>, decltype_p=false, pidk_return=0x0) at ../../src/gcc/cp/parser.c:7437
    1397475 0x0000000000fd4ddf in cp_parser_binary_expression (parser=0x7fde27558da8, cast_p=<optimized out>, no_toplevel_fold_p=false, decltype_p=<optimized out>, prec=PREC_NOT_OPERATOR, pidk=<optimized out>) at ../../src/gcc/cp/parser.c:9842
    1397476 0x0000000000fd4595 in cp_parser_assignment_expression (parser=0x7fde27558da8, pidk=<optimized out>, cast_p=<optimized out>, decltype_p=<optimized out>) at ../../src/gcc/cp/parser.c:10146
    1397477 0x0000000000fd3b90 in cp_parser_constant_expression (parser=0x7fde27558da8, allow_non_constant_p=2, non_constant_p=0x7ffd1194f1d7, strict_p=<optimized out>) at ../../src/gcc/cp/parser.c:10449
    1397478 0x0000000000fcfdd5 in cp_parser_initializer_clause (non_constant_p=<optimized out>, parser=0x7fde27558da8) at ../../src/gcc/cp/parser.c:24253
    1397479 cp_parser_initializer (parser=0x7fde27558da8, is_direct_init=<optimized out>, non_constant_p=<optimized out>, subexpression_p=<optimized out>) at ../../src/gcc/cp/parser.c:24193
    1397480 0x000000000062e5d8 in cp_parser_decomposition_declaration (init_loc=0x7ffd1194f1d8, maybe_range_for_decl=0x7ffd1194f498, decl_specifiers=0x7ffd1194f1f0, parser=0x7fde27558da8) at ../../src/gcc/cp/parser.c:14734
    1397481 cp_parser_simple_declaration (parser=0x7fde27558da8, function_definition_allowed_p=<optimized out>, maybe_range_for_decl=0x7ffd1194f498) at ../../src/gcc/cp/parser.c:14393
    1397482 0x000000000109b870 in cp_parser_init_statement (parser=0x7fde27558da8, decl=0x7ffd1194f498) at ../../src/gcc/cp/parser.c:13420
    1397483 0x00000000010996f0 in cp_parser_for (unroll=0, ivdep=false, parser=0x7fde27558da8) at ../../src/gcc/cp/parser.c:12708
    1397484 cp_parser_iteration_statement (parser=0x7fde27558da8, if_p=0x0, ivdep=<optimized out>, unroll=<optimized out>) at ../../src/gcc/cp/parser.c:13343
    1397485 0x0000000000fe5c46 in cp_parser_statement (parser=0x7fde27558da8, in_statement_expr=0x0, in_compound=<optimized out>, if_p=0x0, chain=0x0, loc_after_labels=0x0) at ../../src/gcc/cp/parser.c:11718
    1397486 0x0000000000fe15ac in cp_parser_statement_seq_opt (in_statement_expr=<optimized out>, parser=<optimized out>) at ../../src/gcc/cp/parser.c:12201
    1397487 cp_parser_compound_statement (parser=0x7fde27558da8, in_statement_expr=0x0, bcs_flags=<optimized out>, function_body=<optimized out>) at ../../src/gcc/cp/parser.c:12150

Interesting frame is 1397471, from which we can extract location:

    (gdb) p line_table[0].info_ordinary.maps[line_table[0].info_ordinary.cache]
    $54 = {
      <line_map> = {
        start_location = 1396581280
      },
      members of line_map_ordinary:
      reason = LC_RENAME,
      sysp = 0 '\000',
      m_column_and_range_bits = 8,
      m_range_bits = 0,
      to_file = 0x3eb4bb0 "/ch/src/Interpreters/MySQL/InterpretersMySQLDDLQuery.cpp",
      to_line = 46,
      included_from = 0
    }

By replicating SOURCE_LINE() macro from gcc-11 (libcpp/include/line-map.h):

    /* Converts a map and a location_t to source line.  */
    inline linenum_type
    SOURCE_LINE (const line_map_ordinary *ord_map, location_t loc)
    {
      return ((loc - ord_map->start_location)
          >> ord_map->m_column_and_range_bits) + ord_map->to_line;
    }

We got line 154:

    (gdb) p ((input_location-1396581280) >> 8) + 46
    $61 = 154

And this is:

    auto [column_name_and_type, declare_column_ast] = std::tuple{columns_name_and_type.begin(), columns_definition->children.begin()};

After rewriting it, everything works correctly.

Also here is a reproducer for gcc-11 (and no failures for gcc-12, but
gcc-12 has other issues, like [1] and one more for hash table):

    # cat /tmp/test.cpp
    #include <tuple>

    auto multi()
    {
            return std::tuple{1, 1};
    }
    double foo()
    {
            auto [a, b] = multi();
            return a - b;
    }

    # g++-11 -std=gnu++20 -c -o /dev/null -isystem /ch/contrib/libcxx/include -nostdinc++ /tmp/test.cpp
    g++-11: internal compiler error: Segmentation fault signal terminated program cc1plus
    Please submit a full bug report,
    with preprocessed source if appropriate.
    See <file:///usr/share/doc/gcc-11/README.Bugs> for instructions.
    # g++-12 -std=gnu++20 -c -o /dev/null -isystem /ch/contrib/libcxx/include -nostdinc++ /tmp/test.cpp

  [1]: https://reviews.llvm.org/D122598

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-28 22:21:45 +03:00
Vladimir C
ecdd89e811
Merge pull request #35666 from vdimir/fix-fuse 2022-03-28 20:40:43 +02:00
Alexander Tokmakov
208b242188 Merge branch 'master' into mvcc_prototype 2022-03-28 19:58:06 +02:00
vdimir
a216bc26c1
Correct check asof join key nullability 2022-03-28 13:29:34 +00:00
taiyang-li
38702be40c Merge remote-tracking branch 'ck/master' into rocksdb_metacache 2022-03-28 21:03:22 +08:00
mergify[bot]
3924609ab2
Merge branch 'master' into metric-async-insert-bytes 2022-03-28 11:37:49 +00:00
vdimir
1122db89db
Use float devision for avg after optimize_fuse_sum_count_avg 2022-03-28 11:26:42 +00:00
Kseniia Sumarokova
482a5e362b
Merge pull request #35620 from kssenii/fix-positional-arguments-with-aliases 
Fix positional arguments
 2022-03-28 09:18:49 +02:00
Anton Popov
4ddfff3084 fix usage of quota with async inserts 2022-03-28 00:46:35 +00:00
Alexey Milovidov
bb35184da1 Add metric about size of async INSERTs 2022-03-28 02:04:19 +02:00
Yakov Olkhovskiy
5a4694f340 major refactoring, simplified, optimized, bugs fixed 2022-03-27 14:32:09 -04:00
Azat Khuzhin
13f2b2c0a3 Use proper type for RadixSortTraits in RowRefs.cpp 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-27 11:32:51 +03:00
Azat Khuzhin
d518614524 Fix multi-line comment in threadPoolCallbackRunner.cpp 2022-03-26 23:42:46 +03:00
Dmitry Novik
0d75e773ab Continue query execution if memory is freed 2022-03-26 18:25:58 +00:00
mergify[bot]
93b3b46caf
Merge branch 'master' into fix-lowcardinality-hash-join 2022-03-25 22:56:59 +00:00
Anton Popov
67195bfdd5 support schema inference for type Object in format JSONEachRow 2022-03-25 21:51:53 +00:00
alesapin
14e84ff6ef
Merge pull request #34802 from ClickHouse/overcommit-doc 
Add documentation for memory overcommit
 2022-03-25 20:50:44 +01:00
kssenii
d8960cde62 Fix positional order/etc by literal 2022-03-25 20:03:12 +01:00
kssenii
5103aefd34 Fix 2022-03-25 14:33:54 +01:00
Antonio Andelic
d7fb51269a Merge branch 'master' into fix-lowcardinality-hash-join 2022-03-25 11:12:31 +00:00
Antonio Andelic
9e11e611c7 Match cardinality when adding columns 2022-03-25 11:12:02 +00:00
mergify[bot]
6267ba4445
Merge branch 'master' into fix_stupid_bug 2022-03-25 01:30:48 +00:00
Nikita Mikhaylov
64f79f0c69
Added an ability to specify cluster secret in replicated database (#35333) 2022-03-25 00:14:26 +01:00
kssenii
6769b84638 Merge master 2022-03-24 20:47:43 +01:00
Yakov Olkhovskiy
adefcfd299
Merge branch 'master' into interpolate-feature 2022-03-24 15:33:09 -04:00
Yakov Olkhovskiy
83f406b722 optimization, INTERPOLATE without expr. list, any column is allowed except WITH FILL 2022-03-24 15:29:29 -04:00
Alexander Tokmakov
7c0bdbfa93 fix stupid bug 2022-03-24 20:24:05 +01:00
Sergei Trifonov
a90e83665d
Merge pull request #35343 from ClickHouse/parallel-multipart-upload-for-s3storage 
allow parallelization of multipart S3 storage upload; fixes #34350
 2022-03-24 15:58:29 +01:00
Antonio Andelic
34b2307259 Convert columns to full when multiple disjunct used 2022-03-24 13:56:00 +00:00
tavplubix
ee611a9af4
Merge pull request #35503 from ClickHouse/fix_assertion_ddl_worker 
Fix too strict assertion in DDLWorker
 2022-03-24 13:57:07 +03:00
Nikolai Kochetov
283e20a9a5
Merge pull request #35395 from amosbird/distributedmultiplejoin 
Validate some thoughts over making sets
 2022-03-24 10:30:26 +01:00
Maksim Kita
1df1721648
Merge pull request #35546 from kitaisreal/refactor-external-models-loader 
ExternalModelsLoader refactoring
 2022-03-24 01:24:45 +01:00
Maksim Kita
3df6a86387 ExternalModelsLoader refactoring 2022-03-23 20:05:17 +01:00
mergify[bot]
abbed251c1
Merge branch 'master' into asof-join-improve-performance 2022-03-23 18:53:17 +00:00
kssenii
d2a3cfe5dc Cache on all write operations 2022-03-23 19:14:33 +01:00
Alexander Tokmakov
9aed0507b7 Merge branch 'master' into mvcc_prototype 2022-03-23 18:07:22 +01:00
mergify[bot]
13bc93c171
Merge branch 'master' into fix_assertion_ddl_worker 2022-03-23 17:06:33 +00:00
Sergei Trifonov
bf9c6be6c6 move threadPoolCallbackRunner from Common into Interpreters folder to fix split build issue 2022-03-23 17:57:19 +01:00
kssenii
c2c7f365ce Merge master 2022-03-23 16:58:35 +01:00
Amos Bird
ab7923a26c
Remove comments 2022-03-23 23:21:02 +08:00
Maksim Kita
da235f9cda
Merge pull request #34536 from kitaisreal/system-log-tables-comment-fix 
System log tables allow to specify COMMENT in ENGINE declaration
 2022-03-23 15:06:05 +01:00
Maksim Kita
561ce88ff3 Fixed tests 2022-03-23 14:25:21 +01:00
Maksim Kita
3c5c267c17 Improve performance of ASOF join 2022-03-23 12:19:38 +01:00
Alexander Tokmakov
e8b7f963b7 fix too strict assertion 2022-03-23 11:55:28 +01:00
Maksim Kita
21735db126 Fixed style check 2022-03-23 11:52:37 +01:00
Maksim Kita
1f940f9e3b
Merge pull request #35512 from CurtizJ/fix-fuzzer 
Fix crash with enabled `optimize_functions_to_subcolumns`
 2022-03-23 11:31:32 +01:00
Maksim Kita
a03bfa0fae Fixed style check 2022-03-23 10:46:27 +01:00
taiyang-li
8dbf1c60e7 merge master and fix conflict 2022-03-23 11:36:50 +08:00
Anton Popov
4ff9627f60 fix crash with enabled optimize_functions_to_subcolumns 2022-03-23 01:27:52 +00:00
Maksim Kita
1a096a87da Added support for SystemLog to specify COMMENT in ENGINE section 2022-03-22 22:44:46 +01:00
Alexander Tokmakov
3c762f566d Merge branch 'master' into mvcc_prototype 2022-03-21 20:16:29 +01:00
Vitaly Baranov
72a2f31984
Merge pull request #33985 from vitlibar/improve-backup-and-restore 
Improve backup and restore (experimental)
 2022-03-21 17:59:51 +01:00
kssenii
afd0c64a1a Better 2022-03-21 15:04:16 +01:00
Alexey Milovidov
8a04ed72af
Merge pull request #34772 from ClickHouse/cluster-dont-require-port 
Allow to use default port in cluster configuration
 2022-03-21 16:45:24 +03:00
kssenii
eabbce69a7 Add system drop remote filesystem cache query, allow to write cache on insert 2022-03-21 12:32:44 +01:00
Vitaly Baranov
7a46e7a0d1
Merge branch 'master' into improve-backup-and-restore 2022-03-21 11:49:26 +01:00
mergify[bot]
e5a5ab2a40
Merge branch 'master' into distributedmultiplejoin 2022-03-21 10:00:51 +00:00
kssenii
59b7394caf WriteSettings 2022-03-21 09:52:48 +01:00
Alexey Milovidov
0ce4696c49
Revert "[WIP] New row policies" 2022-03-21 08:41:33 +03:00
Amos Bird
243de091bb
Validate some thoughts over making sets 2022-03-21 10:58:44 +08:00
Alexander Tokmakov
9e05b12d2c Merge branch 'master' into mvcc_prototype 2022-03-20 22:42:26 +01:00
李扬
d28981e0e3
fix null pointer of TreeRewriterResult::storage_snapshot (#35439) 2022-03-20 21:07:58 +01:00
Vitaly Baranov
c884cf901f Use BackupEntryFromAppendOnlyFile for Log engine, 
and use a BackupEntriesBatch for Memort engine to improve performance.
A lot of minor corrections.
 2022-03-20 20:02:15 +01:00