thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-16 03:12:43 +00:00
Code Issues Packages Projects Releases Wiki Activity
88,310 Commits 9,326 Branches 1,783 Tags 2.1 GiB
19afeda4b1
Commit Graph

6249 Commits

Author SHA1 Message Date
taiyang-li
834ac3a1b9 Merge remote-tracking branch 'ck/master' into rocksdb_metacache 2022-03-29 18:32:06 +08:00
Anton Popov
5d8988ad4e
Merge pull request #35645 from CurtizJ/fix-quotas-async-inserts 
Fix usage of quota with async inserts
 2022-03-29 11:54:24 +02:00
Alexey Milovidov
5e262fba85
Merge pull request #35204 from azat/build-gcc 
Add build with GCC
 2022-03-29 04:55:15 +03:00
kssenii
44157e5c80 Merge master 2022-03-29 02:19:32 +02:00
Yakov Olkhovskiy
615efa1381 aliases processing fixed 2022-03-28 19:15:53 -04:00
Azat Khuzhin
bf4df5c6bb Fix SIGSEGV for build under gcc-11 (due to auto deduction) 
During building with gcc-11 you will got SIGSEGV for building
InterpretersMySQLDDLQuery (and some others), and it is due to endless
recursion:

    (gdb) bt 5
    0  0x00000000010978f2 in structural_comptypes (t1=0x7fde028c7dc8, t2=0x7fde028d1e70, strict=0) at ../../src/gcc/cp/typeck.c:1274
    1  0x00000000011c3f9d in comp_template_parms (parms1=<optimized out>, parms2=<optimized out>) at ../../src/gcc/cp/pt.c:3369
    2  0x0000000001097bd9 in structural_comptypes (t1=0x7fde028c7dc8, t2=0x7fde028d1e70, strict=<optimized out>) at ../../src/gcc/cp/typeck.c:1361
    3  0x00000000011c3f9d in comp_template_parms (parms1=<optimized out>, parms2=<optimized out>) at ../../src/gcc/cp/pt.c:3369
    4  0x0000000001097bd9 in structural_comptypes (t1=0x7fde028c7dc8, t2=0x7fde028d1e70, strict=<optimized out>) at ../../src/gcc/cp/typeck.c:1361
    (gdb) bt -X
    1397454 0x0000000001097bd9 in structural_comptypes (t1=0x7fde028d1540, t2=0x7fde028d27e0, strict=<optimized out>) at ../../src/gcc/cp/typeck.c:1361
    1397455 0x0000000000f2d8b5 in cp_tree_equal (t1=<optimized out>, t2=<optimized out>) at ../../src/gcc/cp/tree.c:4144
    1397456 0x00000000010909cb in template_args_equal (ot=0x7fde028cf578, nt=0x7fde028cfc58, partial_order=<optimized out>) at ../../src/gcc/cp/pt.c:9256
    1397457 0x0000000001090422 in template_args_equal (partial_order=false, nt=0x7fde028cfc58, ot=0x7fde028cf578) at ../../src/gcc/cp/pt.c:9295
    1397458 comp_template_args (oldargs=0x7fde028cf550, newargs=0x7fde028cfc30, oldarg_ptr=0x0, newarg_ptr=0x0, partial_order=false) at ../../src/gcc/cp/pt.c:9285
    1397459 0x00000000010a08f4 in spec_hasher::equal (e1=0x7fde028c95d0, e2=0x7ffd1194e8c0) at ../../src/gcc/cp/pt.c:1726
    1397460 0x0000000001085965 in hash_table<spec_hasher, false, xcallocator>::find_with_hash (this=0x7fde36b7f450, comparable=@0x7ffd1194e8b8: 0x7ffd1194e8c0, hash=<optimized out>) at ../../src/gcc/hash-table.h:936
    1397461 0x0000000001079698 in lookup_template_class_1 (d1=<optimized out>, arglist=0x7fde028cfc30, in_decl=0x0, context=<optimized out>, entering_scope=<optimized out>, complain=3) at ../../src/gcc/cp/pt.c:9896
    1397462 0x000000000109f8ef in lookup_template_class (complain=3, entering_scope=1, context=0x7fde27558e40, in_decl=0x0, arglist=0x7fde028cfc08, d1=0x7fde269bcd20) at ../../src/gcc/cp/pt.c:10251
    1397463 tsubst_aggr_type (t=0x7fde269bcd20, args=<optimized out>, complain=3, in_decl=0x0, entering_scope=1) at ../../src/gcc/cp/pt.c:13646
    1397464 0x000000000108f797 in tsubst (t=0x7fde269bcdc8, args=0x7fde028cf7a8, complain=3, in_decl=<optimized out>) at ../../src/gcc/cp/pt.c:16108
    1397465 0x0000000000e61bf2 in rewrite_template_parm (level=1, complain=3, tsubst_args=0x7fde028cf7a8, index=5, olddecl=0x7fde269b5600) at ../../src/gcc/cp/pt.c:28556
    1397466 rewrite_tparm_list(tree_node*, unsigned int, unsigned int, tree_node*, unsigned int, int) [clone .constprop.0] (oldelt=0x7fde269bd190, index=5, targs=0x7fde028cf7a8, targs_index=4, complain=3, level=1) at ../../src/gcc/cp/pt.c:28640
    1397467 0x00000000009f3748 in build_deduction_guide (type=type@entry=0x7fde26e13dc8, ctor=0x7fde269ac300, outer_args=outer_args@entry=0x0, complain=complain@entry=3) at ../../src/gcc/cp/pt.c:28769
    1397468 0x00000000009f444f in ctor_deduction_guides_for (complain=3, tmpl=<optimized out>) at ../../src/gcc/cp/cp-tree.h:842
    1397469 deduction_guides_for (tmpl=<optimized out>, any_dguides_p=<optimized out>, complain=3) at ../../src/gcc/cp/pt.c:29282
    1397470 0x00000000008507a8 in do_class_deduction (complain=3, flags=1, init=<optimized out>, tmpl=0x7fde26e0f980, ptype=0x7fde028c7b28) at ../../src/gcc/cp/pt.c:29402
    1397471 do_auto_deduction (type=0x7fde028c7b28, init=<optimized out>, auto_node=<optimized out>, complain=3, context=<optimized out>, outer_targs=<optimized out>, flags=1) at ../../src/gcc/cp/pt.c:29572
    1397472 0x00000000007c9569 in finish_compound_literal (type=<optimized out>, compound_literal=0x7fde028c95b8, complain=3, fcl_context=fcl_functional) at ../../src/gcc/cp/semantics.c:3060
    1397473 0x0000000001123a79 in cp_parser_functional_cast (parser=0x7fde27558da8, type=0x7fde028c7b28) at ../../src/gcc/cp/parser.c:30670
    1397474 0x0000000000fd7873 in cp_parser_postfix_expression (parser=0x7fde27558da8, address_p=<optimized out>, cast_p=<optimized out>, member_access_only_p=<optimized out>, decltype_p=false, pidk_return=0x0) at ../../src/gcc/cp/parser.c:7437
    1397475 0x0000000000fd4ddf in cp_parser_binary_expression (parser=0x7fde27558da8, cast_p=<optimized out>, no_toplevel_fold_p=false, decltype_p=<optimized out>, prec=PREC_NOT_OPERATOR, pidk=<optimized out>) at ../../src/gcc/cp/parser.c:9842
    1397476 0x0000000000fd4595 in cp_parser_assignment_expression (parser=0x7fde27558da8, pidk=<optimized out>, cast_p=<optimized out>, decltype_p=<optimized out>) at ../../src/gcc/cp/parser.c:10146
    1397477 0x0000000000fd3b90 in cp_parser_constant_expression (parser=0x7fde27558da8, allow_non_constant_p=2, non_constant_p=0x7ffd1194f1d7, strict_p=<optimized out>) at ../../src/gcc/cp/parser.c:10449
    1397478 0x0000000000fcfdd5 in cp_parser_initializer_clause (non_constant_p=<optimized out>, parser=0x7fde27558da8) at ../../src/gcc/cp/parser.c:24253
    1397479 cp_parser_initializer (parser=0x7fde27558da8, is_direct_init=<optimized out>, non_constant_p=<optimized out>, subexpression_p=<optimized out>) at ../../src/gcc/cp/parser.c:24193
    1397480 0x000000000062e5d8 in cp_parser_decomposition_declaration (init_loc=0x7ffd1194f1d8, maybe_range_for_decl=0x7ffd1194f498, decl_specifiers=0x7ffd1194f1f0, parser=0x7fde27558da8) at ../../src/gcc/cp/parser.c:14734
    1397481 cp_parser_simple_declaration (parser=0x7fde27558da8, function_definition_allowed_p=<optimized out>, maybe_range_for_decl=0x7ffd1194f498) at ../../src/gcc/cp/parser.c:14393
    1397482 0x000000000109b870 in cp_parser_init_statement (parser=0x7fde27558da8, decl=0x7ffd1194f498) at ../../src/gcc/cp/parser.c:13420
    1397483 0x00000000010996f0 in cp_parser_for (unroll=0, ivdep=false, parser=0x7fde27558da8) at ../../src/gcc/cp/parser.c:12708
    1397484 cp_parser_iteration_statement (parser=0x7fde27558da8, if_p=0x0, ivdep=<optimized out>, unroll=<optimized out>) at ../../src/gcc/cp/parser.c:13343
    1397485 0x0000000000fe5c46 in cp_parser_statement (parser=0x7fde27558da8, in_statement_expr=0x0, in_compound=<optimized out>, if_p=0x0, chain=0x0, loc_after_labels=0x0) at ../../src/gcc/cp/parser.c:11718
    1397486 0x0000000000fe15ac in cp_parser_statement_seq_opt (in_statement_expr=<optimized out>, parser=<optimized out>) at ../../src/gcc/cp/parser.c:12201
    1397487 cp_parser_compound_statement (parser=0x7fde27558da8, in_statement_expr=0x0, bcs_flags=<optimized out>, function_body=<optimized out>) at ../../src/gcc/cp/parser.c:12150

Interesting frame is 1397471, from which we can extract location:

    (gdb) p line_table[0].info_ordinary.maps[line_table[0].info_ordinary.cache]
    $54 = {
      <line_map> = {
        start_location = 1396581280
      },
      members of line_map_ordinary:
      reason = LC_RENAME,
      sysp = 0 '\000',
      m_column_and_range_bits = 8,
      m_range_bits = 0,
      to_file = 0x3eb4bb0 "/ch/src/Interpreters/MySQL/InterpretersMySQLDDLQuery.cpp",
      to_line = 46,
      included_from = 0
    }

By replicating SOURCE_LINE() macro from gcc-11 (libcpp/include/line-map.h):

    /* Converts a map and a location_t to source line.  */
    inline linenum_type
    SOURCE_LINE (const line_map_ordinary *ord_map, location_t loc)
    {
      return ((loc - ord_map->start_location)
          >> ord_map->m_column_and_range_bits) + ord_map->to_line;
    }

We got line 154:

    (gdb) p ((input_location-1396581280) >> 8) + 46
    $61 = 154

And this is:

    auto [column_name_and_type, declare_column_ast] = std::tuple{columns_name_and_type.begin(), columns_definition->children.begin()};

After rewriting it, everything works correctly.

Also here is a reproducer for gcc-11 (and no failures for gcc-12, but
gcc-12 has other issues, like [1] and one more for hash table):

    # cat /tmp/test.cpp
    #include <tuple>

    auto multi()
    {
            return std::tuple{1, 1};
    }
    double foo()
    {
            auto [a, b] = multi();
            return a - b;
    }

    # g++-11 -std=gnu++20 -c -o /dev/null -isystem /ch/contrib/libcxx/include -nostdinc++ /tmp/test.cpp
    g++-11: internal compiler error: Segmentation fault signal terminated program cc1plus
    Please submit a full bug report,
    with preprocessed source if appropriate.
    See <file:///usr/share/doc/gcc-11/README.Bugs> for instructions.
    # g++-12 -std=gnu++20 -c -o /dev/null -isystem /ch/contrib/libcxx/include -nostdinc++ /tmp/test.cpp

  [1]: https://reviews.llvm.org/D122598

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-28 22:21:45 +03:00
Vladimir C
ecdd89e811
Merge pull request #35666 from vdimir/fix-fuse 2022-03-28 20:40:43 +02:00
Alexander Tokmakov
208b242188 Merge branch 'master' into mvcc_prototype 2022-03-28 19:58:06 +02:00
vdimir
a216bc26c1
Correct check asof join key nullability 2022-03-28 13:29:34 +00:00
taiyang-li
38702be40c Merge remote-tracking branch 'ck/master' into rocksdb_metacache 2022-03-28 21:03:22 +08:00
mergify[bot]
3924609ab2
Merge branch 'master' into metric-async-insert-bytes 2022-03-28 11:37:49 +00:00
vdimir
1122db89db
Use float devision for avg after optimize_fuse_sum_count_avg 2022-03-28 11:26:42 +00:00
Kseniia Sumarokova
482a5e362b
Merge pull request #35620 from kssenii/fix-positional-arguments-with-aliases 
Fix positional arguments
 2022-03-28 09:18:49 +02:00
Anton Popov
4ddfff3084 fix usage of quota with async inserts 2022-03-28 00:46:35 +00:00
Alexey Milovidov
bb35184da1 Add metric about size of async INSERTs 2022-03-28 02:04:19 +02:00
Yakov Olkhovskiy
5a4694f340 major refactoring, simplified, optimized, bugs fixed 2022-03-27 14:32:09 -04:00
Azat Khuzhin
13f2b2c0a3 Use proper type for RadixSortTraits in RowRefs.cpp 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-27 11:32:51 +03:00
Azat Khuzhin
d518614524 Fix multi-line comment in threadPoolCallbackRunner.cpp 2022-03-26 23:42:46 +03:00
Dmitry Novik
0d75e773ab Continue query execution if memory is freed 2022-03-26 18:25:58 +00:00
mergify[bot]
93b3b46caf
Merge branch 'master' into fix-lowcardinality-hash-join 2022-03-25 22:56:59 +00:00
Anton Popov
67195bfdd5 support schema inference for type Object in format JSONEachRow 2022-03-25 21:51:53 +00:00
alesapin
14e84ff6ef
Merge pull request #34802 from ClickHouse/overcommit-doc 
Add documentation for memory overcommit
 2022-03-25 20:50:44 +01:00
kssenii
d8960cde62 Fix positional order/etc by literal 2022-03-25 20:03:12 +01:00
kssenii
5103aefd34 Fix 2022-03-25 14:33:54 +01:00
Antonio Andelic
d7fb51269a Merge branch 'master' into fix-lowcardinality-hash-join 2022-03-25 11:12:31 +00:00
Antonio Andelic
9e11e611c7 Match cardinality when adding columns 2022-03-25 11:12:02 +00:00
mergify[bot]
6267ba4445
Merge branch 'master' into fix_stupid_bug 2022-03-25 01:30:48 +00:00
Nikita Mikhaylov
64f79f0c69
Added an ability to specify cluster secret in replicated database (#35333) 2022-03-25 00:14:26 +01:00
kssenii
6769b84638 Merge master 2022-03-24 20:47:43 +01:00
Yakov Olkhovskiy
adefcfd299
Merge branch 'master' into interpolate-feature 2022-03-24 15:33:09 -04:00
Yakov Olkhovskiy
83f406b722 optimization, INTERPOLATE without expr. list, any column is allowed except WITH FILL 2022-03-24 15:29:29 -04:00
Alexander Tokmakov
7c0bdbfa93 fix stupid bug 2022-03-24 20:24:05 +01:00
Sergei Trifonov
a90e83665d
Merge pull request #35343 from ClickHouse/parallel-multipart-upload-for-s3storage 
allow parallelization of multipart S3 storage upload; fixes #34350
 2022-03-24 15:58:29 +01:00
Antonio Andelic
34b2307259 Convert columns to full when multiple disjunct used 2022-03-24 13:56:00 +00:00
tavplubix
ee611a9af4
Merge pull request #35503 from ClickHouse/fix_assertion_ddl_worker 
Fix too strict assertion in DDLWorker
 2022-03-24 13:57:07 +03:00
Nikolai Kochetov
283e20a9a5
Merge pull request #35395 from amosbird/distributedmultiplejoin 
Validate some thoughts over making sets
 2022-03-24 10:30:26 +01:00
Maksim Kita
1df1721648
Merge pull request #35546 from kitaisreal/refactor-external-models-loader 
ExternalModelsLoader refactoring
 2022-03-24 01:24:45 +01:00
Maksim Kita
3df6a86387 ExternalModelsLoader refactoring 2022-03-23 20:05:17 +01:00
mergify[bot]
abbed251c1
Merge branch 'master' into asof-join-improve-performance 2022-03-23 18:53:17 +00:00
kssenii
d2a3cfe5dc Cache on all write operations 2022-03-23 19:14:33 +01:00
Alexander Tokmakov
9aed0507b7 Merge branch 'master' into mvcc_prototype 2022-03-23 18:07:22 +01:00
mergify[bot]
13bc93c171
Merge branch 'master' into fix_assertion_ddl_worker 2022-03-23 17:06:33 +00:00
Sergei Trifonov
bf9c6be6c6 move threadPoolCallbackRunner from Common into Interpreters folder to fix split build issue 2022-03-23 17:57:19 +01:00
kssenii
c2c7f365ce Merge master 2022-03-23 16:58:35 +01:00
Amos Bird
ab7923a26c
Remove comments 2022-03-23 23:21:02 +08:00
Maksim Kita
da235f9cda
Merge pull request #34536 from kitaisreal/system-log-tables-comment-fix 
System log tables allow to specify COMMENT in ENGINE declaration
 2022-03-23 15:06:05 +01:00
Maksim Kita
561ce88ff3 Fixed tests 2022-03-23 14:25:21 +01:00
Maksim Kita
3c5c267c17 Improve performance of ASOF join 2022-03-23 12:19:38 +01:00
Alexander Tokmakov
e8b7f963b7 fix too strict assertion 2022-03-23 11:55:28 +01:00
Maksim Kita
21735db126 Fixed style check 2022-03-23 11:52:37 +01:00
Maksim Kita
1f940f9e3b
Merge pull request #35512 from CurtizJ/fix-fuzzer 
Fix crash with enabled `optimize_functions_to_subcolumns`
 2022-03-23 11:31:32 +01:00
Maksim Kita
a03bfa0fae Fixed style check 2022-03-23 10:46:27 +01:00
taiyang-li
8dbf1c60e7 merge master and fix conflict 2022-03-23 11:36:50 +08:00
Anton Popov
4ff9627f60 fix crash with enabled optimize_functions_to_subcolumns 2022-03-23 01:27:52 +00:00
Maksim Kita
1a096a87da Added support for SystemLog to specify COMMENT in ENGINE section 2022-03-22 22:44:46 +01:00
Alexander Tokmakov
3c762f566d Merge branch 'master' into mvcc_prototype 2022-03-21 20:16:29 +01:00
Vitaly Baranov
72a2f31984
Merge pull request #33985 from vitlibar/improve-backup-and-restore 
Improve backup and restore (experimental)
 2022-03-21 17:59:51 +01:00
kssenii
afd0c64a1a Better 2022-03-21 15:04:16 +01:00
Alexey Milovidov
8a04ed72af
Merge pull request #34772 from ClickHouse/cluster-dont-require-port 
Allow to use default port in cluster configuration
 2022-03-21 16:45:24 +03:00
kssenii
eabbce69a7 Add system drop remote filesystem cache query, allow to write cache on insert 2022-03-21 12:32:44 +01:00
Vitaly Baranov
7a46e7a0d1
Merge branch 'master' into improve-backup-and-restore 2022-03-21 11:49:26 +01:00
mergify[bot]
e5a5ab2a40
Merge branch 'master' into distributedmultiplejoin 2022-03-21 10:00:51 +00:00
kssenii
59b7394caf WriteSettings 2022-03-21 09:52:48 +01:00
Alexey Milovidov
0ce4696c49
Revert "[WIP] New row policies" 2022-03-21 08:41:33 +03:00
Amos Bird
243de091bb
Validate some thoughts over making sets 2022-03-21 10:58:44 +08:00
Alexander Tokmakov
9e05b12d2c Merge branch 'master' into mvcc_prototype 2022-03-20 22:42:26 +01:00
李扬
d28981e0e3
fix null pointer of TreeRewriterResult::storage_snapshot (#35439) 2022-03-20 21:07:58 +01:00
Vitaly Baranov
c884cf901f Use BackupEntryFromAppendOnlyFile for Log engine, 
and use a BackupEntriesBatch for Memort engine to improve performance.
A lot of minor corrections.
 2022-03-20 20:02:15 +01:00
Vitaly Baranov
5f195d6ead Implement writing backups as zip archive. 2022-03-20 20:02:15 +01:00
Vitaly Baranov
7f89b98308 Rework BackupSettings and RestoreSettings a little, pass StorageRestoreSettings to storages. 2022-03-20 20:02:15 +01:00
Vitaly Baranov
258a472001 Shorter names: rename IRestoreFromBackupTask -> IRestoreTask. 2022-03-20 20:02:15 +01:00
Vitaly Baranov
7a63feb3f7 Make restore tasks explicit. 2022-03-20 20:01:31 +01:00
Vitaly Baranov
9fb50422ce
Merge pull request #35345 from vitlibar/new-row-policies 
[WIP] New row policies
 2022-03-20 20:00:25 +01:00
Vitaly Baranov
afe6a1003a Add new row policy kind: simple 2022-03-20 17:34:59 +01:00
Yakov Olkhovskiy
c4daf514d6
Update InterpreterSelectQuery.cpp 
bugfix: check column existence for INTERPOLATE expression target
 2022-03-19 14:12:29 -04:00
Yakov Olkhovskiy
eb7474e73a
Merge branch 'master' into interpolate-feature 2022-03-19 03:11:14 -04:00
Yakov Olkhovskiy
a8e1671a76 type match check for INTERPOLATE expressions added, bugfix, printout fixed 2022-03-18 16:44:27 -04:00
Alexander Tokmakov
0719d01d39 more introspection 2022-03-18 14:33:59 +01:00
Alexander Tokmakov
c28008a2df more comments, minor fixes 2022-03-18 12:01:26 +01:00
Raúl Marín
fe42895745 Address clang-tidy warnings 2022-03-18 11:15:12 +01:00
Raúl Marín
aeacd42e08 ASOF: Add big integer types support 2022-03-17 19:08:33 +01:00
Alexander Tokmakov
07d952b728 use snapshots for semistructured data, durability fixes 2022-03-17 18:26:18 +01:00
Raúl Marín
e22fe541b2 Improvements based on PR comments 2022-03-17 17:02:13 +01:00
Raúl Marín
e2cec4e65b Merge remote-tracking branch 'blessed/master' into asof_ftw 2022-03-17 16:33:29 +01:00
Vitaly Baranov
98e48d7b54 SHOW CREATE ROW POLICY now always shows AS permissive/restrictive if there is TO section. 2022-03-17 15:28:53 +01:00
Vitaly Baranov
3e950d79b3 Add new setting rbac_version. 2022-03-17 15:28:51 +01:00
Vitaly Baranov
470582e262 Change type of RowPolicyKind: bool -> enum. 2022-03-17 15:28:05 +01:00
Alexander Tokmakov
d04dc03fa4 Merge branch 'master' into mvcc_prototype 2022-03-17 15:24:32 +01:00
alesapin
457fa0d1ba
Merge pull request #23932 from CurtizJ/dynamic-columns 
Support of dynamic subcolumns (JSON data type)
 2022-03-17 10:45:46 +01:00
Yakov Olkhovskiy
00700988ee style fix 2022-03-17 02:31:01 -04:00
Yakov Olkhovskiy
7bb66e6702 added INTERPOLATE extension for ORDER BY WITH FILL 2022-03-17 01:51:35 -04:00
Alexander Tokmakov
4b3e13a4fe Merge branch 'master' into mvcc_prototype 2022-03-16 21:06:19 +01:00
Alexander Tokmakov
c2ac8d4a5c review fixes 2022-03-16 21:05:34 +01:00
Vitaly Baranov
39614e6e15
Merge pull request #35276 from vitlibar/fix-code-style-and-minor-corrections-after-allow-no-password 
Fix code style and other minor corrections after implementing allow_no_password.
 2022-03-16 18:55:05 +01:00
Anton Popov
2ced42ed41 add experimental settings for Object type 2022-03-16 16:51:23 +00:00
Maksim Kita
9ba53ae037
Merge pull request #34888 from kitaisreal/performance-tests-fix 
Performance tests fix
 2022-03-16 17:19:16 +01:00
Maksim Kita
dc31a4175f Fixed style check 2022-03-16 16:16:08 +00:00
Anton Popov
0ba78c3c3a Merge remote-tracking branch 'upstream/master' into HEAD 2022-03-16 15:28:09 +00:00
Kruglov Pavel
4a579768ef
Merge pull request #35311 from vdimir/pmj-dup-rows 
Attempt to fix merge join duplicate rows
 2022-03-16 12:48:29 +01:00
Maksim Kita
ed74a464bd Fixed tests 2022-03-16 11:21:18 +00:00
Nikolai Kochetov
bcd3c76296
Merge pull request #35214 from ClickHouse/add-database_replicated_allow_only_replicated_engine-setting 
Add database_replicated_allow_only_replicated_engine setting.
 2022-03-16 11:24:32 +01:00
vdimir
695708d57c
[wip] merge join duplicate rows 2022-03-15 18:51:04 +00:00
Alexander Tokmakov
b5b3d09b4a fix 2022-03-15 17:51:53 +01:00
Alexander Tokmakov
1f571b7734 Merge branch 'master' into mvcc_prototype 2022-03-15 14:45:06 +01:00
Alexander Tokmakov
9c3e4cdc6e fix 2022-03-15 14:35:48 +01:00
Anton Popov
b5a04ff938
Merge pull request #35207 from bigo-sg/add_distributed_depth_in_querylog 
Add column  distributed_depth for system.query_log and system.processes
 2022-03-15 13:38:21 +01:00
Vladimir C
af88d3f133
Merge pull request #35173 from bigo-sg/dump_ast_in_dot 2022-03-15 11:18:41 +01:00
taiyang-li
3688ff10e0 fix bug 2022-03-15 12:19:08 +08:00
Alexander Tokmakov
c8d6c13c2d fix 2022-03-15 00:08:25 +01:00
Alexander Tokmakov
9702b5177d Merge branch 'master' into mvcc_prototype 2022-03-14 21:45:38 +01:00
Alexander Tokmakov
278d779a01 log cleanup, more comments 2022-03-14 21:43:34 +01:00
Vitaly Baranov
1eb2e8693e Fix code style and other minor corrections after implementing allow_no_password. 2022-03-14 20:55:34 +01:00
Nikolai Kochetov
7526c36f30 Remove comment 2022-03-14 19:38:03 +00:00
Nikolai Kochetov
ce94b560bb Add test. 2022-03-14 19:30:52 +00:00
Maksim Kita
e30117a3d6 Fix clang-tidy warnings in Interpreters, IO folders 2022-03-14 18:17:35 +00:00
Anton Popov
063917786e minor fixes 2022-03-14 17:29:18 +00:00
Anton Popov
36ec379aeb Merge remote-tracking branch 'upstream/master' into HEAD 2022-03-14 16:28:35 +00:00
Vitaly Baranov
4af61fb9d3
Merge pull request #34738 from DevTeamBK/Issue-33953 
New setting in Server Configuration to on/off AuthType Plaintext_password and No_password
 2022-03-14 17:09:46 +01:00
alesapin
b13b415d3f
Merge pull request #35257 from azat/fix-s3-pool-shutdown 
Wait for IDiskRemote thread pool properly
 2022-03-14 13:56:37 +01:00
Nikolai Kochetov
ff09076524 A little bit better. 2022-03-14 11:31:24 +00:00
Kseniia Sumarokova
58a2d2b458
Merge pull request #35118 from zzsmdfj/issue/#31469_MaterializedMysql_mysqlDate2CkDate32 
to #31469_MaterializedMysql_mysqlDate2CkDate32
 2022-03-14 10:32:33 +01:00
Azat Khuzhin
d13fc9ec44 Wait for IDiskRemote thread pool properly 
ASan report [1]:

    WARNING: ThreadSanitizer: heap-use-after-free (pid=611)
      Read of size 4 at 0x7b48006f0250 by thread T273:
        0 MemoryTracker::allocImpl() obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:105:49 (clickhouse+0xa05d0ef)
        1 MemoryTracker::allocImpl() obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:239:22 (clickhouse+0xa05d580)
        2 MemoryTracker::alloc(long) obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:246:5 (clickhouse+0xa05dea7)
        3 DB::ThreadStatus::~ThreadStatus() obj-x86_64-linux-gnu/../src/Common/ThreadStatus.cpp:154:28 (clickhouse+0xa059b01)
        4 ThreadFromGlobalPool::ThreadFromGlobalPool<>()::'lambda'()::operator()() obj-x86_64-linux-gnu/../s>

      Previous write of size 8 at 0x7b48006f0250 by main thread:
        8 std::__1::unordered_map<std::__1::basic_string<>, DB::ProcessListForUser, std::__1::hash<>, std::__1>
        9 DB::ProcessList::~ProcessList() obj-x86_64-linux-gnu/../src/Interpreters/ProcessList.h:275:7 (clickhouse+0x17b71c9c)
        10 DB::ContextSharedPart::~ContextSharedPart() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:304:5 (clickhouse+0x17b70712)
        13 DB::SharedContextHolder::reset() obj-x86_64-linux-gnu/../src/Interpreters/Context.cpp:453:44 (clickhouse+0x17b3daf9)
        14 DB::Server::main()::$_8::o>

      Thread T273 (tid=946, running) created by thread T223 at:
        0 pthread_create <null> (clickhouse+0x9f7d49d)
        15 DB::WriteBufferFromS3::makeSinglepartUpload() obj-x86_64-linux-gnu/../src/IO/WriteBufferFromS3.cpp:332:9 (clickhouse+0x17a81b6a)

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/35072/19be9c8c6433ffd1bbf8eb08975d8afbaebf3d43/stress_test__thread__actions_.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-14 12:07:47 +03:00
Maksim Kita
0dd807d19d
Merge pull request #34750 from kitaisreal/merge-tree-improve-insert-performance 
MergeTree improve insert performance
 2022-03-13 13:39:18 +01:00
Nikolai Kochetov
cdc529fa2a Add database_replicated_allow_only_replicated_engine setting. 2022-03-11 16:03:05 +00:00
Maksim Kita
5f8900cee6
Merge pull request #35079 from qieqieplus/master 
fix #33798
 2022-03-11 11:55:49 +01:00
taiyang-li
1f9e050152 add column distributed_depth for system.query_log and system.processes 2022-03-11 17:57:34 +08:00
Kseniia Sumarokova
818459b9f0
Merge pull request #33717 from kssenii/local-cache-for-remote-fs 
Local cache for remote filesystem
 2022-03-11 07:23:10 +01:00
HeenaBansal2009
3f031df225 Code refactoring 2022-03-10 22:22:51 -08:00
Maksim Kita
cbe059f4bd Updated IColumn interface to support getting stable permutation 2022-03-10 21:45:31 +00:00
Maksim Kita
765cd09d06 MergeTree improve insert performance 2022-03-10 21:45:31 +00:00
Alexander Tokmakov
7f47f20aba add kill transaction query 2022-03-10 22:29:58 +01:00
alesapin
c0d8ccc91b
Merge pull request #35178 from Varinara/master 
Added disk_name to system.part_log
 2022-03-10 22:22:37 +01:00
alexey-milovidov
58e53b06a6
Merge pull request #35134 from azat/optimize_skip_unused_shards_rewrite_in-signed 
Fix `optimize_skip_unused_shards_rewrite_in` for signed columns and negative values
 2022-03-10 23:12:05 +03:00
mergify[bot]
059682a0cd
Merge branch 'master' into asof_ftw 2022-03-10 17:29:38 +00:00
mergify[bot]
9f4ebc313b
Merge branch 'master' into master 2022-03-10 16:55:27 +00:00
Varinara
f5523f7ff0 added disk_name to system.part_log 2022-03-10 18:44:19 +03:00
Rajkumar
3d3b6d1956 clang-tidy report issues with Medium priority 2022-03-10 07:23:49 -08:00
Alexander Tokmakov
061fa6a6f2 Merge branch 'master' into mvcc_prototype 2022-03-10 13:13:04 +01:00
kssenii
787a0805a5 Merge master 2022-03-10 11:42:19 +01:00
taiyang-li
ee8fc2730e remote comment 2022-03-10 18:29:52 +08:00
taiyang-li
382417d893 fix bug 2022-03-10 18:29:00 +08:00
taiyang-li
2cd469f113 support print ast in dot format 2022-03-10 18:04:35 +08:00
HeenaBansal2009
3ce9397246 Added Suggestions from Code review 2022-03-09 20:35:01 -08:00
Alexander Tokmakov
0906b59fba fixes 2022-03-09 21:38:18 +01:00
Vladimir C
ce266b5a3e
Merge pull request #35146 from amosbird/fixpartitionprunerin 2022-03-09 13:23:45 +01:00
Nikolai Kochetov
c364908061
Merge pull request #35094 from amosbird/getridofredundantplan 
Get rid of duplicate query planing.
 2022-03-09 09:10:20 +01:00
Amos Bird
a19224bc9b
Fix partition pruner: non-monotonic function IN 2022-03-09 15:48:42 +08:00
Azat Khuzhin
ec6bae2b2a Fix optimize_skip_unused_shards_rewrite_in for signed columns 
Fixes: #35131
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-09 08:29:28 +03:00
Alexander Tokmakov
d2f838bd91 fix another race condition 2022-03-08 20:11:47 +01:00
kssenii
5260822964 Merge master 2022-03-08 18:21:28 +01:00
Azat Khuzhin
c4b6342853
Improvements for parallel_distributed_insert_select (and related) (#34728) 
* Add a warning if parallel_distributed_insert_select was ignored

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Respect max_distributed_depth for parallel_distributed_insert_select

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Print warning for non applied parallel_distributed_insert_select only for initial query

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Remove Cluster::getHashOfAddresses()

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Forbid parallel_distributed_insert_select for remote()/cluster() with different addresses

Before it uses empty cluster name (getClusterName()) which is not
correct, compare all addresses instead.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Fix max_distributed_depth check

max_distributed_depth=1 must mean not more then one distributed query,
not two, since max_distributed_depth=0 means no limit, and
distribute_depth is 0 for the first query.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Fix INSERT INTO remote()/cluster() with parallel_distributed_insert_select

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Add a test for parallel_distributed_insert_select with cluster()/remote()

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Return <remote> instead of empty cluster name in Distributed engine

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

* Make user with sharding_key and w/o in remote()/cluster() identical

Before with sharding_key the user was "default", while w/o it it was
empty.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-08 15:24:39 +01:00
zzsmdfj
67b9f81104 to #31469_MaterializedMysql_mysqlDate2CkDate32 2022-03-08 18:17:22 +08:00
taiyang-li
b4174b0bef merge master and fix conflicts 2022-03-08 11:39:25 +08:00
Alexander Tokmakov
8acfb8d27f Merge branch 'master' into mvcc_prototype 2022-03-07 17:40:15 +01:00
Amos Bird
fe4534d464
Get rid of duplicate query planing. 2022-03-08 00:02:58 +08:00
Anton Popov
0bc57da238 Merge remote-tracking branch 'upstream/master' into HEAD 2022-03-07 14:46:08 +00:00
tavplubix
34505be1de
Merge pull request #35081 from azat/fix-shutdown-deadlock 
Avoid possible deadlock on server shutdown
 2022-03-07 13:47:15 +01:00
Kseniia Sumarokova
cdb9a05229
Merge pull request #34605 from bigo-sg/add_metric_for_local 
Add cpu/mem metric for clickhouse-local
 2022-03-07 11:40:11 +01:00
Azat Khuzhin
081f9caa04 Avoid possible deadlock on server shutdown 
Here is an example for deadlock during shutting down DDLWorker:

Server main thread:

    6  ThreadFromGlobalPool::join () at ../src/Common/ThreadPool.h:217
    7  DB::DDLWorker::shutdown () at ../src/Interpreters/DDLWorker.cpp:123
    8  DB::DDLWorker::~DDLWorker () at ../src/Interpreters/DDLWorker.cpp:131
    9  DB::DDLWorker::~DDLWorker () at ../src/Interpreters/DDLWorker.cpp:130
    10 std::__1::default_delete<DB::DDLWorker>::operator() () at ../contrib/libcxx/include/memory:1397
    11 std::__1::unique_ptr<>::reset (this=0x7f7521d44fd0, __p=0x0) at ../contrib/libcxx/include/memory:1658
    12 DB::ContextSharedPart::shutdown () at ../src/Interpreters/Context.cpp:380 <!-- holds mutex
    13 DB::Context::shutdown () at ../src/Interpreters/Context.cpp:2677

DDLWorker thread:

    7  DB::Context::getLock () at ../src/Interpreters/Context.cpp:472 <-- trying to acquire shared.mutex
    8  DB::Context::getTCPPortSecure () at ../src/Interpreters/Context.cpp:2120
    9  DB::DDLTask::findCurrentHostID () at ../src/Interpreters/DDLTask.cpp:169
    10 DB::DDLWorker::initAndCheckTask () at ../src/Interpreters/DDLWorker.cpp:191
    11 DB::DDLWorker::scheduleTasks () at ../src/Interpreters/DDLWorker.cpp:367
    12 DB::DDLWorker::runMainThread () at ../src/Interpreters/DDLWorker.cpp:1063

v2: replace getTryLockTimed() with getTryLock() since std::recursive_mutex does not have try_lock_for()
v3: stole objects from context
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Fixes: #21318
 2022-03-06 14:31:18 +03:00
qieqieplus
5287f78d8b fix #33798 2022-03-06 12:02:17 +08:00
taiyang-li
1b9987ceb1 fix mistake in check-style 2022-03-05 14:22:56 +08:00
tavplubix
e49521163f
Merge pull request #34860 from zhangjmruc/master 
Support non-table DDLs on cross replicated cluster
 2022-03-04 19:37:51 +01:00
Kruglov Pavel
d45b6ae03c
Extract schema only once on table creation and add it to metadata (#34684) 
* Add columns description to metadata in case of schema inference

* Make better

* Remove unnecessary code

* Fix tests

* More tests

* Add tag no-fasttest

* Fix test

* Fix test

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2022-03-04 21:23:19 +03:00
Maksim Kita
c6e0922449
Merge pull request #35018 from kitaisreal/executable-udf-deterministic-in-scope-of-query-fix 
ExecutableUDF function deterministic in scope of query fix
 2022-03-04 10:46:19 +01:00
Anton Popov
df3b07fe7c Merge remote-tracking branch 'upstream/master' into HEAD 2022-03-03 22:25:28 +00:00
Anton Popov
aea7bfb59a
Merge pull request #34992 from azat/fix-asynchronous_inserts-race 
Fix race between INSERT async_insert=1 and system.asynchronous_inserts
 2022-03-03 20:55:19 +03:00
mergify[bot]
e4fba1dc99
Merge branch 'master' into executable-udf-deterministic-in-scope-of-query-fix 2022-03-03 15:17:16 +00:00
Maksim Kita
a6c510ceb1 ExecutableUDF function deterministic in scope of query fix 2022-03-03 12:40:42 +00:00
Frank Chen
b4829465d9
Improve the opentelemetry span logs for INSERT on distributed table (#34480) 2022-03-03 12:53:29 +01:00
Maksim Kita
b1a956c5f1 clang-tidy check performance-move-const-arg fix 2022-03-02 18:15:27 +00:00
Maksim Kita
1f5837359e clang-tidy check performance-noexcept-move-constructor fix 2022-03-02 18:15:27 +00:00
mreddy017
f893002b69 Fix vulnerable code related to std::move and noexcept 
This commit fixes the vulnerable code related to std::move and noexcept identified by clangtidy tool.
 2022-03-02 18:15:27 +00:00
kssenii
d5952109fb Merge master 2022-03-02 18:15:25 +01:00
mergify[bot]
93b13c0232
Merge branch 'master' into asof_ftw 2022-03-02 13:20:50 +00:00
Azat Khuzhin
57f636a1e8 Fix race between INSERT async_insert=1 and system.asynchronous_inserts 
CI report [1]:

    [c190f600f8c6] 2022.03.02 01:07:34.553012 [ 23552 ] {76b6113b-1479-46c9-90ab-e78a3c9f3dbb}  executeQuery: Code: 60. DB::Exception: Both table name and UUID are empty. (UNKNOWN_TABLE) (version 22.3.1.1) (from [::1]:42040) (comment: '02015_async_inserts_stress_long.sh') (in query: SELECT * FROM system.asynchronous_inserts FORMAT Null), Stack trace (when copying this message, always include the lines below):

    0. ClickHouse/contrib/libcxx/include/exception:133: Poco::Exception::Exception(std::__1::basic_string, std::__1::allocator > const&, int) @ 0xf50e04c in /fasttest-workspace/build/programs/clickhouse
    1. ClickHouse/src/Common/Exception.cpp:58: DB::Exception::Exception(std::__1::basic_string, std::__1::allocator > const&, int, bool) @ 0x663ebfa in /fasttest-workspace/build/programs/clickhouse
    2. DB::StorageID::assertNotEmpty() const @ 0xbc08591 in /fasttest-workspace/build/programs/clickhouse
    3. ClickHouse/contrib/libcxx/include/string:1444: DB::StorageID::getDatabaseName() const @ 0xe50d2b6 in /fasttest-workspace/build/programs/clickhouse
    4. ClickHouse/contrib/libcxx/include/string:1957: DB::StorageSystemAsynchronousInserts::fillData(std::__1::vector::mutable_ptr, std::__1::allocator::mutable_ptr > >&, std::__1::shared_ptr, DB::SelectQueryInfo const&) const @ 0xdac636c in /fasttest-workspace/build/programs/clickhouse

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/34973/e6fc6a22d5c018961c18247242dd3a40b8c54ff2/fast_test__actions_.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-02 15:28:06 +03:00
Azat Khuzhin
a49106e583 Fix non-MergeTree engines for system.*_log 
Fixes: https://github.com/ClickHouse/ClickHouse/pull/34949
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-03-02 13:24:40 +03:00
kssenii
a594f388a4 Merge master 2022-03-01 19:43:45 +01:00
Anton Popov
c1fdcf7a64 Merge remote-tracking branch 'upstream/master' into HEAD 2022-03-01 20:21:39 +03:00
Anton Popov
82d24f06eb
Merge pull request #34866 from CurtizJ/async-insert-table-function 
Fix async inserts to table functions
 2022-03-01 19:45:22 +03:00
Kruglov Pavel
97d634d1cc
Merge pull request #34732 from Avogar/insert-select-schema 
Support schema inference for INSERT INTO FUNCTION
 2022-03-01 13:38:16 +03:00
alesapin
4b61e4795c
Merge pull request #34949 from nikitamikhaylov/system_log_tables_and_settings 
Recreate system.{*}_log table on settings changes
 2022-03-01 11:15:19 +01:00
taiyang-li
f83132bad2 finish dev 2022-03-01 15:54:23 +08:00
taiyang-li
b31440c77a Merge branch 'master' into add_metric_for_local 2022-03-01 12:29:36 +08:00
HeenaBansal2009
c14c60f1d3 Merge branch 'master' into Issue-33953 2022-02-28 11:12:54 -08:00
HeenaBansal2009
aa8494a808 Fix: System Reload Config Failure 2022-02-28 10:51:49 -08:00
Nikita Mikhaylov
d6036f6da3 Better 
(cherry picked from commit 4ae445c9e227581ea9f1cbe9aa9d1ba82e1236c9)
 2022-02-28 15:27:52 +00:00
Anton Popov
8fa112b841 add comments 2022-02-28 17:09:46 +03:00
Kruglov Pavel
011813957d
Merge pull request #34938 from azat/create-as-ignore-ttl 
Ignore per-column TTL in CREATE TABLE AS if new table engine does not support it
 2022-02-28 16:58:15 +03:00
Azat Khuzhin
644f9168fa Ignore per-column TTL in CREATE TABLE AS if new table engine does not support it 
Follow-up for: #6968
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-28 10:29:26 +03:00
Alexandre Snarskii
93de1b1c99 Implement MemoryStatisticsOS for FreeBSD 2022-02-25 14:05:00 +03:00
Anton Popov
fcdebea925 Merge remote-tracking branch 'upstream/master' into HEAD 2022-02-25 13:41:30 +03:00
Anton Popov
97d57e22ea fix async inserts to table functions 2022-02-24 14:26:47 +03:00
Jianmei Zhang
d148cc30cc Support non-table DDLs on cross replicated cluster 2022-02-24 14:37:35 +08:00
Alexander Tokmakov
aa6b9a2abc Merge branch 'master' into mvcc_prototype 2022-02-23 23:22:03 +03:00
kssenii
c637385dd0 Merge master 2022-02-22 13:17:51 +01:00
Maksim Kita
174257dad0
Merge pull request #34653 from kitaisreal/executable-udf-support-specify-argument-names 
ExecutableUserDefinedFunctions allow to specify argument names
 2022-02-22 00:02:14 +01:00
Dmitry Novik
1df43a7f57
Merge pull request #34385 from nvartolomei/nv/move-part-count 
Disable optimize_trivial_count when deduplication for part movement feature is enabled
 2022-02-21 08:53:09 -08:00
Dmitry Novik
be9082f1c4 Update comments for OvercommitTracker 2022-02-21 15:45:17 +00:00
Anton Popov
065305ab65
Merge pull request #34764 from ucasfl/hints-index 
Add name hints for data skipping indices
 2022-02-21 16:50:59 +03:00
kssenii
4543513a5d Add comments 2022-02-21 13:54:03 +01:00
Dmitry Novik
4428e7aa1b
Merge branch 'master' into nv/move-part-count 2022-02-21 02:14:23 -08:00
Mikhail f. Shiryaev
f690aba441
Merge pull request #34078 from godliness/fix-distributed-max-query-size 
Fix distributed subquery max_query_size limitation inconsistency
 2022-02-21 11:01:02 +01:00
Vitaly Baranov
7b97c986cb
Revert "Allow restrictive row policies without permissive" 2022-02-21 06:54:28 +03:00
Alexey Milovidov
522c1c1397 Allow to use default port in cluster configuration 2022-02-20 19:27:14 +01:00
feng lv
07280e0ab1 Add name hints for data skipping indices 
fix test
 2022-02-20 11:48:22 +00:00
alexey-milovidov
26d0e5438c
Merge pull request #33057 from azat/jemalloc-dev-branch 
[RFC] Update jemalloc to 5.3RC
 2022-02-20 14:19:41 +03:00
Chao Ma
895396e61a Fix distributed subquery max_query_size limitation inconsistency 2022-02-20 10:07:57 +08:00
Raúl Marín
4439b23e2b Attempt manual unpack 2022-02-19 21:01:47 +01:00
Vitaly Baranov
874b2c8dcb
Merge pull request #34596 from vitlibar/allow-restrictive-without-permissive 
Allow restrictive row policies without permissive
 2022-02-19 21:45:28 +07:00
Raúl Marín
07d2b6ba80 Remove struct packing 2022-02-19 11:16:28 +01:00
HeenaBansal2009
1b263f0c15 Added FT testcase 2022-02-18 12:58:46 -08:00
Raúl Marín
606f580922 Fix style 2022-02-18 18:37:46 +01:00
Vladimir C
9b7d011ee7
Merge pull request #34529 from vdimir/join-nullable-on-pipeline 
Apply join_use_nulls on types before join
 2022-02-18 18:34:44 +01:00
Raúl Marín
bb1d773ffd Reduce changes against master 2022-02-18 17:58:23 +01:00
avogar
653d769d34 Support schema inference for INSERT INTO FUNCTION 2022-02-18 16:19:42 +00:00
Raúl Marín
2f4c595e09 Bring back some clarity 2022-02-18 16:50:15 +01:00
Maksim Kita
0f9f30cc9e Added tests 2022-02-18 15:21:11 +00:00
Raúl Marín
f8cc024ba1 Less indirect access 2022-02-18 16:16:29 +01:00
vdimir
6ca154858c
upd TableJoin::applyKeyConvertToTable 2022-02-18 13:40:57 +00:00
vdimir
6eda8ef914
use isNullable function in join_common/convertTypeToNullable 2022-02-18 13:09:11 +00:00
Raúl Marín
57b9639872 Faster binary search 2022-02-18 14:05:42 +01:00
Raúl Marín
4ba5b931a5 AsofRowRefs: Remove extra level of indirection 2022-02-18 11:15:11 +01:00
Raúl Marín
34b110b778 Remove variant from AsofRowRefs 2022-02-18 11:02:14 +01:00
HeenaBansal2009
d16cae53b4 Initial Commit for Plaintext password feature 2022-02-17 21:25:18 -08:00
Alexander Tokmakov
f4a46a13fb fixes 2022-02-18 00:26:37 +03:00
Rajkumar
de154c5f9c Adding noexcept for move constructor 2022-02-17 13:18:12 -08:00
Azat Khuzhin
6a2abd3a68 jemalloc: remove jemalloc 4 support 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-17 21:25:58 +03:00
Raúl Marín
b751218fd6 Help the compiler with the variant dispatch code 2022-02-17 16:39:55 +01:00
Alexander Tokmakov
dae044f86b Merge branch 'master' into mvcc_prototype 2022-02-17 13:49:37 +03:00
Kruglov Pavel
7411178064
Merge pull request #34650 from azat/fix-projection-optimization 
Fix allow_experimental_projection_optimization with enable_global_with_statement
 2022-02-17 13:09:57 +03:00
Dmitry Novik
c32313b045
Merge branch 'master' into memory-overcommit-test 2022-02-17 00:50:15 -08:00
Vitaly Baranov
f4da321dbb SHOW CREATE ROW POLICY now always outputs "AS permissive" or "AS restrictive". 2022-02-17 14:18:15 +07:00
Vitaly Baranov
2de6e8e575 Change type of RowPolicyKind: bool -> enum. 2022-02-17 14:18:10 +07:00
Dmitry Novik
07d0e3c823 cleanup 2022-02-16 20:19:10 +00:00
Dmitry Novik
44d5ddc939 Fix deadlock in OvercommitTracker 2022-02-16 20:02:14 +00:00
Azat Khuzhin
774744a86d Fix allow_experimental_projection_optimization with enable_global_with_statement 
allow_experimental_projection_optimization requires one more
InterpreterSelectQuery, which with enable_global_with_statement will
apply ApplyWithAliasVisitor if the query is not subquery.

But this should not be done for queries from
MergeTreeData::getQueryProcessingStage()/getQueryProcessingStageWithAggregateProjections()
since this will duplicate WITH statements over and over.

This will also fix scalar.xml perf tests, that leads to the following
error now:

    scalar.query0.prewarm0: DB::Exception: Stack size too large.

And since it has very long query in the log, this leads to the following
perf test error:

    _csv.Error: field larger than field limit (131072)

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-16 19:14:47 +03:00
Maksim Kita
669b18e037 ExecutableUserDefinedFunctions allow to specify argument names 2022-02-16 15:31:30 +00:00
Anton Popov
edd686e1d9 fix unit test 2022-02-16 17:18:03 +03:00
kssenii
47f94120da Merge master 2022-02-16 14:43:28 +01:00
Nikolai Kochetov
f9d2dae88e
Merge pull request #34424 from yakov-olkhovskiy/ephemeral-column 
Ephemeral column issue #9436
 2022-02-16 12:04:57 +01:00
Raúl Marín
d06ef3e1db AsofRowRefs: Simplify variant code 2022-02-16 11:12:28 +01:00
Raúl Marín
47d325edd1 Improve SortedLookupVector::find 2022-02-16 11:12:28 +01:00
Raúl Marín
0a620b82a3 RowRefs: Use assert_cast instead of typeid_cast 
They couldn't fail and it meant over 1% of the query runtime:
1.23%  typeid_cast<DB::ColumnVector<unsigned int> const&, DB::IColumn const>
    - 0.53% DB::AsofRowRefs::insert
    - 0.56% DB::AsofRowRefs::findAsof
 2022-02-16 11:12:28 +01:00
Raúl Marín
b99d757633 Packed RowRef 
- sizeof(DB::AsofRowRefs::Entry<Int32>): Was 24, now 16
- sizeof(DB::AsofRowRefs::Entry<Int64>): Was 24, now 24 too
 2022-02-16 11:12:28 +01:00
Anton Popov
a661eaf39f better performance of getting storage snapshot 2022-02-16 02:17:22 +03:00
Nikolai Kochetov
1276c50271 Fix build. 2022-02-15 20:15:07 +00:00
alesapin
bc2d0ee7c7
Merge pull request #34215 from ClickHouse/revert-34211-revert-34153-add_func_tests_over_s3 
Add func tests run with s3 and fix several bugs
 2022-02-15 19:07:11 +03:00
Dmitry Novik
bb6dad7d0e Fix lock order 2022-02-15 15:04:13 +00:00
taiyang-li
e6b29167da fix style 2022-02-15 20:26:53 +08:00
Nikolai Kochetov
b2b29b7ee8 Do not use ReduceToPhysicalTransform. 2022-02-15 12:23:22 +00:00
vdimir
510a909d0d
Remove nullable_left/right_side flags from Hash/MergeJoin 2022-02-15 11:15:50 +00:00
vdimir
2de6eaf838
Fix LowCardinality using key for join_use_nulls in pipeline 2022-02-15 11:11:59 +00:00
Nikolai Kochetov
ab288642f6 Merge branch 'master' into ephemeral-column 2022-02-15 10:03:34 +00:00
Nikolai Kochetov
d6cbac1ed3
Merge pull request #34577 from ClickHouse/alwasy-remove-unused-actions-for-add-missing-defaults 
Always remove unused actions from addMissingDefaults
 2022-02-15 11:01:29 +01:00
Nikolai Kochetov
4d2c53c32b Fix clang tidy 2022-02-15 09:59:51 +00:00
taiyang-li
b4440131d0 add metrics for clickhouse-local 2022-02-15 16:25:07 +08:00
李扬
f52b67b939
Merge branch 'master' into rocksdb_metacache 2022-02-15 02:16:29 -06:00
Alexander Tokmakov
1e4e569151 Merge branch 'master' into mvcc_prototype 2022-02-15 02:26:47 +03:00
Alexander Tokmakov
ae5aa8c12d write part version before other files 2022-02-15 02:24:51 +03:00
alexey-milovidov
bc206ec423
Merge pull request #31182 from ClickHouse/memory-overcommit 
Memory overcommit
 2022-02-14 23:55:02 +03:00
Alexander Tokmakov
cbd3b45646 add EXPLAIN CURRENT TRANSACTION 2022-02-14 22:47:17 +03:00
vdimir
99ca89c0ca
Fix StorageJoin and Asof or join_use_nulls in pipeline 2022-02-14 14:14:27 +00:00
Nikolai Kochetov
22a938f83d Simplify a little bit. 2022-02-14 14:53:32 +01:00
Nikolai Kochetov
ebb5d1807d Always remove unused actipns from addMissingDefaults 2022-02-14 12:49:07 +00:00
alesapin
89373155fc Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-13 21:07:54 +03:00
Yakov Olkhovskiy
579fe6c97a major rework, transform added to the insert pipe 2022-02-13 17:42:59 +00:00
Maksim Kita
340614e5ec
Merge pull request #34489 from vitlibar/make-create-row-policy-privilege-table-level 
Privileges CREATE/ALTER/DROP ROW POLICY are table-level now
 2022-02-13 15:15:49 +01:00
李扬
daa27d0bda
Merge branch 'master' into rocksdb_metacache 2022-02-12 07:50:12 -06:00
vdimir
ee09ec4dd1
Apply join_use_nulls on types before join 2022-02-11 15:56:45 +00:00
tavplubix
6d1fae834a
Merge pull request #34187 from qoega/default-table-engine 
Default table engine
 2022-02-11 18:14:29 +03:00
Alexander Tokmakov
07e66e690d Merge branch 'master' into mvcc_prototype 2022-02-11 15:53:32 +03:00
tavplubix
4a9d0c84df
Update InterpreterCreateQuery.cpp 2022-02-11 13:59:13 +03:00
Vitaly Baranov
e148b43a29 Privileges CREATE/ALTER/DROP ROW POLICY now can be granted on a table or on database.* as well as globally *.* 2022-02-11 12:35:52 +07:00
Alexander Tokmakov
78afe31397 fix 2022-02-11 02:15:43 +03:00
Alexander Tokmakov
9e893dfda7 fixes 2022-02-10 23:35:03 +03:00
Kruglov Pavel
a4f5610764
Merge pull request #34476 from CurtizJ/avoid-settings-copy 
Avoid unnecessary copying of `Settings`
 2022-02-10 14:13:46 +03:00
mergify[bot]
cb3e5f8538
Merge branch 'master' into memory-overcommit 2022-02-10 11:01:43 +00:00
Anton Popov
298838f891 avoid unnecessary copying of Settings 2022-02-10 12:13:51 +03:00
Yatsishin Ilya
33b5c2251f Merge branch 'default-table-engine' of github.com:qoega/ClickHouse into default-table-engine 2022-02-10 09:09:20 +00:00
Yatsishin Ilya
9228b10fa6 refactoring 2022-02-10 08:53:50 +00:00
Anton Popov
18940b8637 Merge remote-tracking branch 'upstream/master' into HEAD 2022-02-09 23:38:38 +03:00
mergify[bot]
d55eadc3c9
Merge branch 'master' into default-table-engine 2022-02-09 17:08:31 +00:00
Nikolai Kochetov
38fb50f736
Merge pull request #33958 from Algunenano/mv_cacheable_scalars 
Scalar cache improvements
 2022-02-09 16:46:53 +03:00
Maksim Kita
9f29c977de
Merge pull request #34234 from azat/agg-leak 
Fix memory leak in AggregatingInOrderTransform
 2022-02-09 12:42:04 +01:00
alesapin
10c3e6e546 Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-09 14:14:58 +03:00
mergify[bot]
fd5caa3196
Merge branch 'master' into short-circuit-bug 2022-02-09 11:04:19 +00:00
Kseniia Sumarokova
86956ca08b
Merge pull request #34392 from kssenii/http-buffer-skip-not-found-url-for-globs 
Allow to skip not found urls for globs
 2022-02-09 09:32:09 +01:00
Azat Khuzhin
4fa2ae76bc Fix memory leak in AggregatingInOrderTransform 
Reproducer:

    # NOTE: we need clickhouse from 33957 since right now LSan is broken due to getauxval().
    $ url=https://s3.amazonaws.com/clickhouse-builds/33957/e04b862673644d313712607a0078f5d1c48b5377/package_asan/clickhouse
    $ wget $url -o clickhouse-asan
    $ chmod +x clickhouse-asan
    $ ./clickhouse-asan server &

    $ ./clickhouse-asan client
    :) create table data (key Int, value String) engine=MergeTree() order by key
    :) insert into data select number%5, toString(number) from numbers(10e6)

    # usually it is enough one query, benchmark is just for stability of the results
    # note, that if the exception was not happen from AggregatingInOrderTransform then add --continue_on_errors and wait
    $ ./clickhouse-asan benchmark --query 'select key, uniqCombined64(value), groupArray(value) from data group by key' --optimize_aggregation_in_order=1 --memory_tracker_fault_probability=0.01, max_untracked_memory='2Mi'

LSan report:

    ==24595==ERROR: LeakSanitizer: detected memory leaks

    Direct leak of 3932160 byte(s) in 6 object(s) allocated from:
        0 0xcadba93 in realloc ()
        1 0xcc108d9 in Allocator<false, false>::realloc() obj-x86_64-linux-gnu/../src/Common/Allocator.h:134:30
        2 0xde19eae in void DB::PODArrayBase<>::realloc<DB::Arena*&>(unsigned long, DB::Arena*&) obj-x86_64-linux-gnu/../src/Common/PODArray.h:161:25
        3 0xde5f039 in void DB::PODArrayBase<>::reserveForNextSize<DB::Arena*&>(DB::Arena*&) obj-x86_64-linux-gnu/../src/Common/PODArray.h
        4 0xde5f039 in void DB::PODArray<>::push_back<>(DB::GroupArrayNodeString*&, DB::Arena*&) obj-x86_64-linux-gnu/../src/Common/PODArray.h:432:19
        5 0xde5f039 in DB::GroupArrayGeneralImpl<>::add() const obj-x86_64-linux-gnu/../src/AggregateFunctions/AggregateFunctionGroupArray.h:465:31
        6 0xde5f039 in DB::IAggregateFunctionHelper<>::addBatchSinglePlaceFromInterval() const obj-x86_64-linux-gnu/../src/AggregateFunctions/IAggregateFunction.h:481:53
        7 0x299df134 in DB::Aggregator::executeOnIntervalWithoutKeyImpl() obj-x86_64-linux-gnu/../src/Interpreters/Aggregator.cpp:869:31
        8 0x2ca75f7d in DB::AggregatingInOrderTransform::consume() obj-x86_64-linux-gnu/../src/Processors/Transforms/AggregatingInOrderTransform.cpp:124:13

    ...

    SUMMARY: AddressSanitizer: 4523184 byte(s) leaked in 12 allocation(s).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-09 09:23:56 +03:00
taiyang-li
d04ccc0489 Merge branch 'master' into rocksdb_metacache 2022-02-09 11:54:10 +08:00
Yakov Olkhovskiy
b9777be271 insertion with ephemerals requires columns list, insertion without columns list omits ephemeral 2022-02-09 01:11:29 +00:00
Anton Popov
587d7399ba support dynamic subcolumns for Memory engine 2022-02-09 03:18:53 +03:00
alesapin
02a93cb852 Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-08 19:42:27 +03:00
avogar
08c009ab50 Fix possible error 'Cannot convert column Function to mask' 2022-02-08 18:23:43 +03:00
Nikolai Kochetov
0f7c0c72bd
Merge pull request #34305 from amosbird/projection-fix27 
Fix various issues when projection is enabled by default
 2022-02-08 17:19:56 +03:00
Kruglov Pavel
720310fcf1
Merge branch 'master' into http-buffer-skip-not-found-url-for-globs 2022-02-08 16:56:22 +03:00
Yatsishin Ilya
3e2e422ddd refactoring 2022-02-08 12:21:16 +00:00
Yatsishin Ilya
bcf4303286 Merge remote-tracking branch 'origin' into default-table-engine 2022-02-08 10:35:29 +00:00
taiyang-li
b6132d490f merge master and solve conflict 2022-02-08 15:24:59 +08:00
Yakov Olkhovskiy
97daa84f6f revert unnecesarry change 2022-02-08 00:14:15 +00:00
Yakov Olkhovskiy
4d5fb56c29 added EPHEMERAL default for column (preliminary) 2022-02-07 23:21:10 +00:00
kssenii
eba3011ada Fix 2022-02-07 20:40:47 +01:00
Nicolae Vartolomei
1cdb50cf13 Disable optimize_trivial_count when deduplication for part movement feature is enabled 
Fixes #34089
 2022-02-07 18:26:49 +00:00
kssenii
2e58733750 Merge master 2022-02-07 15:23:10 +01:00
avogar
975ba9146f Fix tests naming, remove code duplication 2022-02-07 14:12:19 +03:00
mergify[bot]
dd947f964c
Merge branch 'master' into mv_cacheable_scalars 2022-02-07 10:07:26 +00:00
alesapin
ba28c2c013 Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-07 12:44:56 +03:00
Yatsishin Ilya
e0803064e6 Merge remote-tracking branch 'origin' into default-table-engine 2022-02-07 08:45:32 +00:00
Alexander Tokmakov
45be75b4db Merge branch 'master' into mvcc_prototype 2022-02-06 23:36:08 +03:00
Amos Bird
a6f0b01e6a
Fix order by after aggregation 2022-02-07 00:42:11 +08:00
Amos Bird
52aabf98fe
Revise and add more comments 2022-02-06 16:53:54 +08:00
Amos Bird
1ab773cc90
Fix aggregation_in_order with normal projection 2022-02-06 16:46:12 +08:00
Amos Bird
7674bc986e
Disable projection when there is JOIN or SAMPLE 2022-02-06 16:46:09 +08:00
Maksim Kita
bc6328e573 Fixed tests 2022-02-04 20:56:49 +00:00
Alexander Tokmakov
3956941aaf fixes 2022-02-04 21:18:20 +03:00
alesapin
c028269e6f Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-04 18:02:25 +03:00
Maksim Kita
629c8ae915
Merge pull request #34284 from CurtizJ/async-insert-in-client 
Minor fixes for #34267
 2022-02-04 13:13:47 +01:00
alesapin
2ed45b2a98 Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-04 11:23:46 +03:00
Alexander Tokmakov
6a646b413e fix 2022-02-04 00:40:17 +03:00
Alexander Tokmakov
3e700e854d cancel merges before acquiring lock for truncate 2022-02-03 23:44:53 +03:00
Alexander Tokmakov
fe30e0f162 fixes 2022-02-03 21:57:09 +03:00
Alexander Tokmakov
ca5f951558 Merge branch 'master' into mvcc_prototype 2022-02-03 18:56:44 +03:00
Anton Popov
9b844c6b42
Merge pull request #32748 from CurtizJ/read-in-order-fixed-prefix 
Support `optimize_read_in_order` if prefix of sorting key is already sorted
 2022-02-03 18:17:08 +03:00
alesapin
2a9bc7cba8 Merge branch 'master' into revert-34211-revert-34153-add_func_tests_over_s3 2022-02-03 16:20:06 +03:00
alesapin
8e33427e19 Fix check for existing table 2022-02-03 15:17:55 +03:00
Anton Popov
0755cfa584 minor fixes 2022-02-03 15:04:13 +03:00
alexey-milovidov
a6d1fcadcb
Merge pull request #34267 from CurtizJ/async-insert-in-client 
Support async inserts in clickhouse-client for queries with inlined data
 2022-02-03 05:14:47 +03:00
Maksim Kita
02c6cfba6a
Merge pull request #34225 from vitlibar/fix-checking-grants-for-show-grants 
Fix checking grants for SHOW GRANTS
 2022-02-02 22:51:51 +01:00
Yatsishin Ilya
cce0452d19 add more tests, fix style, fix MATERIALIZED VIEW POPULATE without ENGINE 2022-02-02 18:07:03 +00:00
Anton Popov
5bb1b3ce77 support async inserts in clickhouse-client for queries with inlined data 2022-02-02 20:59:37 +03:00
Vitaly Baranov
30557aebfb Add helper class to cache the result of checking access. 2022-02-02 22:22:41 +07:00
Yatsishin Ilya
8c822755cb fix AS 2022-02-02 12:33:29 +00:00
Nikolay Degterinsky
29c15f8c65
Merge pull request #33723 from FrankChen021/trace_id_order 
Parse and store opentelemetry trace-id in big-endian order
 2022-02-02 14:59:46 +03:00
tavplubix
7304fd654a
Merge pull request #34224 from CurtizJ/fix-query-metric 
Fix metric `Query`
 2022-02-02 13:39:07 +03:00
Yatsishin Ilya
759574d6f0 improve 2022-02-01 16:59:41 +00:00
Anton Popov
a4aa1f37ce remove unused code 2022-02-01 17:44:23 +03:00
Yatsishin Ilya
2a698b8da3 Merge remote-tracking branch 'origin/master' into default-table-engine 2022-02-01 13:23:52 +00:00
Vitaly Baranov
1f81e43f1e Fix checking grants for SHOW GRANTS. 2022-02-01 19:55:24 +07:00
Anton Popov
836a348a9c Merge remote-tracking branch 'upstream/master' into HEAD 2022-02-01 15:23:07 +03:00
Anton Popov
c28255850a fix metric Query 2022-02-01 15:06:49 +03:00
Azat Khuzhin
bedf208cbd Use fmt::runtime() for LOG_* for non constexpr 
Here is oneliner:

    $ gg 'LOG_\(DEBUG\|TRACE\|INFO\|TEST\|WARNING\|ERROR\|FATAL\)([^,]*, [a-zA-Z]' -- :*.cpp :*.h | cut -d: -f1 | sort -u | xargs -r sed -E -i 's#(LOG_[A-Z]*)\(([^,]*), ([A-Za-z][^,)]*)#\1(\2, fmt::runtime(\3)#'

Note, that I tried to do this with coccinelle (tool for semantic
patchin), but it cannot parse C++:

    $ cat fmt.cocci
    @@
    expression log;
    expression var;
    @@

    -LOG_DEBUG(log, var)
    +LOG_DEBUG(log, fmt::runtime(var))

I've also tried to use some macros/templates magic to do this implicitly
in logger_useful.h, but I failed to do so, and apparently it is not
possible for now.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>

v2: manual fixes
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-02-01 14:30:03 +03:00
Alexander Tokmakov
2e4ae37d98 Merge branch 'master' into mvcc_prototype 2022-02-01 13:20:03 +03:00
Ilya Yatsishin
9c2ef98d23
Apply suggestions from code review 
Co-authored-by: tavplubix <tavplubix@gmail.com>
 2022-02-01 10:03:50 +03:00
Alexander Tokmakov
5fad3fdffc throw exception on non-transactional queries 2022-02-01 01:27:55 +03:00
Amos Bird
ec7d367814
DiskLocal checker 
Add DiskLocal checker so that ReplicatedMergeTree can recover data when some of its disks are broken.
 2022-02-01 05:55:27 +08:00
Yatsishin Ilya
676fc5b20f Initial commit 2022-01-31 17:10:59 +00:00
Maksim Kita
5ef83deaa6 Update sort to pdqsort 2022-01-30 19:49:48 +00:00
Anton Popov
78b9f15abb Merge remote-tracking branch 'upstream/master' into HEAD 2022-01-30 03:24:37 +03:00
Maksim Kita
9c91a50050 Sort block refactoring 2022-01-29 15:50:55 +00:00
Maksim Kita
f3453024ff
Merge pull request #34060 from amosbird/optimizetupleorderby 
Make ORDER BY tuple almost as fast as ORDER BY columns
 2022-01-29 15:58:09 +01:00
Amos Bird
04cff632db
Revise 2022-01-29 20:55:52 +08:00
李扬
6d50d36405
Merge branch 'master' into rocksdb_metacache 2022-01-28 22:00:31 -06:00
alexey-milovidov
6535b75322
Merge pull request #34001 from azat/memory-tracker-fix 
Fix memory accounting for queries that uses < max_untracker_memory
 2022-01-29 00:59:53 +03:00
Amos Bird
2b3ce910c6
Fix collation check 2022-01-29 04:49:58 +08:00
Federico Rodriguez
c0a5bd187a ExecuteScalarSubqueriesVisitor missing static const 
Added string_view
 2022-01-28 15:00:04 -05:00
Alexander Tokmakov
fb9b2d5326 Merge branch 'master' into mvcc_prototype 2022-01-28 21:18:36 +03:00
Alexander Tokmakov
e0304c2a58 review fixes, write tid into mutation entry 2022-01-28 20:47:37 +03:00
Amos Bird
09a3ab79b7
Fix all const 2022-01-29 01:37:39 +08:00
Anton Popov
fa883f2ac3 fix tests 2022-01-28 17:23:51 +03:00
Raúl Marín
6c79e147d0 Add a few extra comments around uses_view_source and why it's propagated 2022-01-28 14:47:49 +01:00
Azat Khuzhin
162f96f8e1 Get back detachQueryIfNotDetached() into ExternalLoader 
ExternalLoader from ThreadPool (async loading) is done from the server
context, not from the query context, and the context will already go
away, so we should detachQueryIfNotDetached() to avoid trigger assertion
in ThreadStatus.

CI: https://s3.amazonaws.com/clickhouse-test-reports/34001/8cace291d17fa9553a98b2a1e8bf15b30fe5a1bd/stateless_tests__debug__actions__[3/3].html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-28 16:25:33 +03:00
Azat Khuzhin
b0c862c297 Fix memory accounting for queries that uses < max_untracker_memory 
MemoryTracker starts accounting memory directly only after per-thread
allocation exceeded max_untracker_memory (or memory_profiler_step).

But even memory under this limit should be accounted too, and there is
code to do this in ThreadStatus dtor, however due to
PullingAsyncPipelineExecutor detached the query from thread group that
memory was not accounted.

So remove CurrentThread::detachQueryIfNotDetached() from threads that
uses ThreadFromGlobalPool since it has ThreadStatus, and the query will
be detached using CurrentThread::defaultThreadDeleter.

Note, that before this patch memory accounting works for HTTP queries
due to it had been accounted from ParallelFormattingOutputFormat, but
not for TCP.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-28 16:25:33 +03:00
Raúl Marín
64799867a1 Add a comment about IInterpreterUnionOrSelectQuery::usesViewSource() 2022-01-28 12:39:56 +01:00
Nikolai Kochetov
94999e86c2
Merge pull request #33392 from azat/_shard_num 
Replace old _shard_num implementation with shardNum() function
 2022-01-28 11:02:19 +03:00
taiyang-li
3de8bde7ce Merge remote-tracking branch 'origin/master' into rocksdb_metacache 2022-01-28 09:58:52 +08:00
alexey-milovidov
63d8d75001
Merge pull request #33957 from azat/fix-getauxval 
Fix getauxval() in glibc-compatibility and fix some leaks (after LSan started to work)
 2022-01-27 23:05:23 +03:00
tavplubix
8c9266b24c
Merge pull request #33978 from amosbird/separatesystemlog 
Separate base parts out of SystemLog
 2022-01-27 20:07:55 +03:00
Amos Bird
faee95b897
Make ORDER BY tuple almost as fast as ORDER BY columns 
We have special optimizations for multiple column ORDER BY: https://github.com/ClickHouse/ClickHouse/pull/10831 . It's beneficial to also apply to tuple columns.

Before:

select * from numbers(300000000) order by (1 - number , number + 1 , number) limit 10;
2.613 sec.

After:

select * from numbers(300000000) order by (1 - number , number + 1 , number) limit 10;
0.755 sec

No tuple:

select * from numbers(300000000) order by 1 - number , number + 1 , number limit 10;
0.755 sec
 2022-01-27 21:42:08 +08:00
Azat Khuzhin
af9910b518 Fix leak of demangle result (for opentelementry) 
LSan found [1]:

    Direct leak of 5170176 byte(s) in 5049 object(s) allocated from:
        0 0xc598edd in malloc (/usr/bin/clickhouse+0xc598edd)
        1 0x39679739 in (anonymous namespace)::itanium_demangle::initializeOutputStream(char*, unsigned long*, (anonymous namespace)::itanium_demangle::OutputStream&, unsigned long) obj-x86_64-linux-gnu/../contrib/libcxxabi/src/demangle/Utility.h:178:31
        2 0x39679739 in __cxa_demangle obj-x86_64-linux-gnu/../contrib/libcxxabi/src/cxa_demangle.cpp:351:13
        3 0x28f6f3ed in DB::executeQueryImpl(char const*, char const*, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:662:44

  [1]:
https://s3.amazonaws.com/clickhouse-test-reports/33957/08f4f45fd9da923ae3e3fdd8a527c297d35247eb/stress_test__address__actions_.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-27 13:46:37 +03:00
kssenii
c91b86e220 Better 2022-01-26 21:56:08 +03:00
Raúl Marín
045c92e2b9 Fix readability issues 2022-01-26 17:36:45 +01:00
Raúl Marín
78d2caee7e Can only pre-save scalars if they are cacheable 2022-01-26 17:36:45 +01:00
Raúl Marín
e77fc9e9be Comment 2022-01-26 17:36:45 +01:00
Raúl Marín
444a8e2519 Minimize changes, improve scalar subquery for MVs 2022-01-26 17:36:45 +01:00
Raúl Marín
4b5ab80e3b Better scalar cache handling 
- Fixes global CTE scalar cache.
- Adds MVs back (views dependent on the source are cached locally and others globally
 2022-01-26 17:36:45 +01:00
tavplubix
9e91a9dfd1
Merge pull request #33429 from zzsmdfj/MaterializeMySQL_support_set_and_other_dataType 
MaterializeMySQL support SET YEAR TIME GEOMETRY dataType
 2022-01-26 11:29:46 +03:00
Maksim Kita
bf768a94e8
Merge pull request #33971 from Avogar/fix-ipv6 
Fix parsing IPv6 from query parameter and fix IPv6 to string conversion
 2022-01-25 22:45:11 +01:00
Vitaly Baranov
cc38fe3fb6 Fix checking access for the SYSTEM command. 2022-01-25 18:54:59 +07:00
Amos Bird
385caa3e0a
Separate base parts out of SystemLog 2022-01-25 18:48:43 +08:00
avogar
ab49472155 Add comment 2022-01-25 10:10:04 +03:00
avogar
ad01d68f36 Fix parsing IPv6 from query parameter and fix IPv6 to string conversion 2022-01-25 10:09:17 +03:00
zhongyuankai
a6254516e0 Fix Alter ttl modification unsupported table engine 2022-01-24 21:48:52 +08:00
Ilya Yatsishin
35c03b317e
Merge pull request #33639 from Algunenano/async_metrics_handle_failures 
AsynchronousMetrics: Ignore inaccessible sensors
 2022-01-24 14:04:07 +03:00
tavplubix
da9a38655b
Merge pull request #33847 from ClickHouse/fix_33806 
Fix logical error on RESTORE REPLICA
 2022-01-24 13:28:37 +03:00
taiyang-li
73def8b483 merge master and solve conflict 2022-01-24 11:01:43 +08:00
Maksim Kita
7fffe5846f
Merge pull request #33924 from kitaisreal/sql-user-defined-functions-invalid-lambda-crash-fix 
SQLUserDefinedFunctions invalid lambda additional cases
 2022-01-23 22:06:49 +01:00
Maksim Kita
243a280936 Updated tests 2022-01-23 17:02:43 +00:00
alexey-milovidov
d7a63dfda6
Merge pull request #33677 from ucasfl/fix-sumif 
Fix sumIf rewrite
 2022-01-22 17:12:13 +03:00
Maksim Kita
1bc3f086b1
Merge pull request #33889 from kitaisreal/sql-user-defined-functions-invalid-lambda-crash-fix 
SQLUserDefinedFunctions invalid lambda additional fixes
 2022-01-22 12:09:42 +01:00
alexey-milovidov
eb6849f7c7
Merge pull request #33842 from azat/cmake-contrib-fixes 
More cmake external modules cleanups
 2022-01-22 10:34:54 +03:00
mergify[bot]
7734f9d629
Merge branch 'master' into fix-sumif 2022-01-22 06:08:47 +00:00
Maksim Kita
3a252cd3a0 Updated tests 2022-01-21 20:49:42 +00:00
Anton Popov
e8ce091e68 Merge remote-tracking branch 'upstream/master' into HEAD 2022-01-21 20:11:18 +03:00
Alexander Tokmakov
e3b1397001 Merge branch 'master' into mvcc_prototype 2022-01-21 18:59:01 +03:00
Maksim Kita
330987a64b
Merge pull request #33868 from kitaisreal/sql-user-defined-functions-invalid-lambda-crash-fix 
SQLUserDefinedFunctions invalid lambda crash fix
 2022-01-21 16:23:12 +01:00
Maksim Kita
3b24c39294 SQLUserDefinedFunctions invalid lambda crash fix 2022-01-21 14:33:57 +00:00
Nikolai Kochetov
012453e212
Merge pull request #33588 from alexeypavlenko/fix_nested_default 
Fix default value generation for columns with dot
 2022-01-21 17:19:42 +03:00
Kruglov Pavel
daba57a933
Merge pull request #33737 from Avogar/bool-type 
Some improvements and fixes for Bool data type
 2022-01-21 15:24:58 +03:00
Alexander Tokmakov
8921725d71 fix logical error on RESTORE REPLICA 2022-01-21 14:32:19 +03:00
Azat Khuzhin
dc7affd813 contrib/libmetrohash: add ALIAS library 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-01-21 10:11:23 +03:00
zzsmdfj
411f43ec4d to MaterializeMySQL_support_set_and_other_dataType 2022-01-21 12:24:12 +08:00