Commit Graph

85447 Commits

Author SHA1 Message Date
alesapin
d0f01516db Resurrect automatic labelling 2022-03-29 15:48:57 +02:00
Anton Popov
9610139477
Merge pull request #35629 from CurtizJ/dynamic-columns-5
Support schema inference for type `Object` in format `JSONEachRow`
2022-03-29 14:17:09 +02:00
Kruglov Pavel
fe60560c03
Merge pull request #35619 from Avogar/check-remote-url-allow-hosts
Check remote_url_allow_hosts before schema inference in URL engine
2022-03-29 14:08:02 +02:00
tavplubix
ec496faae8
Merge pull request #35688 from ClickHouse/tavplubix-patch-4
Fix flaky 01037_polygon_dicts_correctness_fast
2022-03-29 15:07:10 +03:00
Anton Popov
d42632dc13
Merge pull request #32928 from bigo-sg/rocksdb_metacache
Speed up parts loading process of MergeTree by metadata cache
2022-03-29 14:05:22 +02:00
Ilya Yatsishin
45babe48bc
Merge pull request #35543 from qoega/azure-integration-test-better
Slightly better integration tests: test_backup_with_other_granularity test_azure_blob_storage_zero_copy_replication
2022-03-29 13:58:31 +02:00
Alexander Gololobov
bf376ee2f5
Merge pull request #35628 from ClickHouse/make_date
Implementation of makeDate and makeDate32
2022-03-29 13:35:26 +02:00
Maksim Kita
a1a4552740
Merge pull request #35184 from DevTeamBK/clang-tidy-issues
clang-tidy report issues with Medium priority
2022-03-29 13:19:54 +02:00
Maksim Kita
c566c52c16
Merge pull request #35683 from kitaisreal/llvm-cmake-fix
Fix enable LLVM for JIT compilation in CMake
2022-03-29 13:10:10 +02:00
taiyang-li
834ac3a1b9 Merge remote-tracking branch 'ck/master' into rocksdb_metacache 2022-03-29 18:32:06 +08:00
Anton Popov
5d8988ad4e
Merge pull request #35645 from CurtizJ/fix-quotas-async-inserts
Fix usage of quota with async inserts
2022-03-29 11:54:24 +02:00
Nikolai Kochetov
86690f18ee
Merge pull request #35667 from amosbird/projection-fix2
Another fix of projection part check
2022-03-29 10:22:01 +02:00
taiyang-li
ddd5bebe55 fix integration test 2022-03-29 11:53:51 +08:00
Amos Bird
3ab8fc1e7b
Fix projection part check
Fix projection part check when projection part type is different from
parent part type.
2022-03-29 10:39:07 +08:00
Alexey Milovidov
5e262fba85
Merge pull request #35204 from azat/build-gcc
Add build with GCC
2022-03-29 04:55:15 +03:00
Nikita Mikhaylov
85282f1b4d
Made schema inference work with s3Cluster table function (#35544) 2022-03-29 00:46:35 +02:00
mergify[bot]
c609426d09
Merge branch 'master' into llvm-cmake-fix 2022-03-28 21:16:25 +00:00
Maksim Kita
4c336c45fb Fix enable LLVM for JIT compilation in CMake 2022-03-28 23:15:31 +02:00
Alexander Gololobov
b49993f993 Fixes according to the code review 2022-03-28 22:47:39 +02:00
Kseniia Sumarokova
9a8e9198e5
Merge pull request #35593 from kssenii/named-collections-fix
Fix headers with named collections, add compression_method
2022-03-28 22:10:32 +02:00
alesapin
10f29ff7f9
Merge pull request #35679 from Avogar/fix-tests
Fix flaky tests 02149_read_in_order_fixed_prefix and 02177_issue_31009
2022-03-28 21:42:17 +02:00
alesapin
f9c1a44cae Fix black 2022-03-28 21:32:53 +02:00
alesapin
a38f63af2d
Merge pull request #35676 from ClickHouse/remove_outdated_links
Remove outdated links from CI
2022-03-28 21:24:45 +02:00
alesapin
7ab8d7178c
Merge pull request #35308 from ClickHouse/clickhouse-keeper
New package clickhouse-keeper
2022-03-28 21:23:17 +02:00
Azat Khuzhin
bf4df5c6bb Fix SIGSEGV for build under gcc-11 (due to auto deduction)
During building with gcc-11 you will got SIGSEGV for building
InterpretersMySQLDDLQuery (and some others), and it is due to endless
recursion:

    (gdb) bt 5
    0  0x00000000010978f2 in structural_comptypes (t1=0x7fde028c7dc8, t2=0x7fde028d1e70, strict=0) at ../../src/gcc/cp/typeck.c:1274
    1  0x00000000011c3f9d in comp_template_parms (parms1=<optimized out>, parms2=<optimized out>) at ../../src/gcc/cp/pt.c:3369
    2  0x0000000001097bd9 in structural_comptypes (t1=0x7fde028c7dc8, t2=0x7fde028d1e70, strict=<optimized out>) at ../../src/gcc/cp/typeck.c:1361
    3  0x00000000011c3f9d in comp_template_parms (parms1=<optimized out>, parms2=<optimized out>) at ../../src/gcc/cp/pt.c:3369
    4  0x0000000001097bd9 in structural_comptypes (t1=0x7fde028c7dc8, t2=0x7fde028d1e70, strict=<optimized out>) at ../../src/gcc/cp/typeck.c:1361
    (gdb) bt -X
    1397454 0x0000000001097bd9 in structural_comptypes (t1=0x7fde028d1540, t2=0x7fde028d27e0, strict=<optimized out>) at ../../src/gcc/cp/typeck.c:1361
    1397455 0x0000000000f2d8b5 in cp_tree_equal (t1=<optimized out>, t2=<optimized out>) at ../../src/gcc/cp/tree.c:4144
    1397456 0x00000000010909cb in template_args_equal (ot=0x7fde028cf578, nt=0x7fde028cfc58, partial_order=<optimized out>) at ../../src/gcc/cp/pt.c:9256
    1397457 0x0000000001090422 in template_args_equal (partial_order=false, nt=0x7fde028cfc58, ot=0x7fde028cf578) at ../../src/gcc/cp/pt.c:9295
    1397458 comp_template_args (oldargs=0x7fde028cf550, newargs=0x7fde028cfc30, oldarg_ptr=0x0, newarg_ptr=0x0, partial_order=false) at ../../src/gcc/cp/pt.c:9285
    1397459 0x00000000010a08f4 in spec_hasher::equal (e1=0x7fde028c95d0, e2=0x7ffd1194e8c0) at ../../src/gcc/cp/pt.c:1726
    1397460 0x0000000001085965 in hash_table<spec_hasher, false, xcallocator>::find_with_hash (this=0x7fde36b7f450, comparable=@0x7ffd1194e8b8: 0x7ffd1194e8c0, hash=<optimized out>) at ../../src/gcc/hash-table.h:936
    1397461 0x0000000001079698 in lookup_template_class_1 (d1=<optimized out>, arglist=0x7fde028cfc30, in_decl=0x0, context=<optimized out>, entering_scope=<optimized out>, complain=3) at ../../src/gcc/cp/pt.c:9896
    1397462 0x000000000109f8ef in lookup_template_class (complain=3, entering_scope=1, context=0x7fde27558e40, in_decl=0x0, arglist=0x7fde028cfc08, d1=0x7fde269bcd20) at ../../src/gcc/cp/pt.c:10251
    1397463 tsubst_aggr_type (t=0x7fde269bcd20, args=<optimized out>, complain=3, in_decl=0x0, entering_scope=1) at ../../src/gcc/cp/pt.c:13646
    1397464 0x000000000108f797 in tsubst (t=0x7fde269bcdc8, args=0x7fde028cf7a8, complain=3, in_decl=<optimized out>) at ../../src/gcc/cp/pt.c:16108
    1397465 0x0000000000e61bf2 in rewrite_template_parm (level=1, complain=3, tsubst_args=0x7fde028cf7a8, index=5, olddecl=0x7fde269b5600) at ../../src/gcc/cp/pt.c:28556
    1397466 rewrite_tparm_list(tree_node*, unsigned int, unsigned int, tree_node*, unsigned int, int) [clone .constprop.0] (oldelt=0x7fde269bd190, index=5, targs=0x7fde028cf7a8, targs_index=4, complain=3, level=1) at ../../src/gcc/cp/pt.c:28640
    1397467 0x00000000009f3748 in build_deduction_guide (type=type@entry=0x7fde26e13dc8, ctor=0x7fde269ac300, outer_args=outer_args@entry=0x0, complain=complain@entry=3) at ../../src/gcc/cp/pt.c:28769
    1397468 0x00000000009f444f in ctor_deduction_guides_for (complain=3, tmpl=<optimized out>) at ../../src/gcc/cp/cp-tree.h:842
    1397469 deduction_guides_for (tmpl=<optimized out>, any_dguides_p=<optimized out>, complain=3) at ../../src/gcc/cp/pt.c:29282
    1397470 0x00000000008507a8 in do_class_deduction (complain=3, flags=1, init=<optimized out>, tmpl=0x7fde26e0f980, ptype=0x7fde028c7b28) at ../../src/gcc/cp/pt.c:29402
    1397471 do_auto_deduction (type=0x7fde028c7b28, init=<optimized out>, auto_node=<optimized out>, complain=3, context=<optimized out>, outer_targs=<optimized out>, flags=1) at ../../src/gcc/cp/pt.c:29572
    1397472 0x00000000007c9569 in finish_compound_literal (type=<optimized out>, compound_literal=0x7fde028c95b8, complain=3, fcl_context=fcl_functional) at ../../src/gcc/cp/semantics.c:3060
    1397473 0x0000000001123a79 in cp_parser_functional_cast (parser=0x7fde27558da8, type=0x7fde028c7b28) at ../../src/gcc/cp/parser.c:30670
    1397474 0x0000000000fd7873 in cp_parser_postfix_expression (parser=0x7fde27558da8, address_p=<optimized out>, cast_p=<optimized out>, member_access_only_p=<optimized out>, decltype_p=false, pidk_return=0x0) at ../../src/gcc/cp/parser.c:7437
    1397475 0x0000000000fd4ddf in cp_parser_binary_expression (parser=0x7fde27558da8, cast_p=<optimized out>, no_toplevel_fold_p=false, decltype_p=<optimized out>, prec=PREC_NOT_OPERATOR, pidk=<optimized out>) at ../../src/gcc/cp/parser.c:9842
    1397476 0x0000000000fd4595 in cp_parser_assignment_expression (parser=0x7fde27558da8, pidk=<optimized out>, cast_p=<optimized out>, decltype_p=<optimized out>) at ../../src/gcc/cp/parser.c:10146
    1397477 0x0000000000fd3b90 in cp_parser_constant_expression (parser=0x7fde27558da8, allow_non_constant_p=2, non_constant_p=0x7ffd1194f1d7, strict_p=<optimized out>) at ../../src/gcc/cp/parser.c:10449
    1397478 0x0000000000fcfdd5 in cp_parser_initializer_clause (non_constant_p=<optimized out>, parser=0x7fde27558da8) at ../../src/gcc/cp/parser.c:24253
    1397479 cp_parser_initializer (parser=0x7fde27558da8, is_direct_init=<optimized out>, non_constant_p=<optimized out>, subexpression_p=<optimized out>) at ../../src/gcc/cp/parser.c:24193
    1397480 0x000000000062e5d8 in cp_parser_decomposition_declaration (init_loc=0x7ffd1194f1d8, maybe_range_for_decl=0x7ffd1194f498, decl_specifiers=0x7ffd1194f1f0, parser=0x7fde27558da8) at ../../src/gcc/cp/parser.c:14734
    1397481 cp_parser_simple_declaration (parser=0x7fde27558da8, function_definition_allowed_p=<optimized out>, maybe_range_for_decl=0x7ffd1194f498) at ../../src/gcc/cp/parser.c:14393
    1397482 0x000000000109b870 in cp_parser_init_statement (parser=0x7fde27558da8, decl=0x7ffd1194f498) at ../../src/gcc/cp/parser.c:13420
    1397483 0x00000000010996f0 in cp_parser_for (unroll=0, ivdep=false, parser=0x7fde27558da8) at ../../src/gcc/cp/parser.c:12708
    1397484 cp_parser_iteration_statement (parser=0x7fde27558da8, if_p=0x0, ivdep=<optimized out>, unroll=<optimized out>) at ../../src/gcc/cp/parser.c:13343
    1397485 0x0000000000fe5c46 in cp_parser_statement (parser=0x7fde27558da8, in_statement_expr=0x0, in_compound=<optimized out>, if_p=0x0, chain=0x0, loc_after_labels=0x0) at ../../src/gcc/cp/parser.c:11718
    1397486 0x0000000000fe15ac in cp_parser_statement_seq_opt (in_statement_expr=<optimized out>, parser=<optimized out>) at ../../src/gcc/cp/parser.c:12201
    1397487 cp_parser_compound_statement (parser=0x7fde27558da8, in_statement_expr=0x0, bcs_flags=<optimized out>, function_body=<optimized out>) at ../../src/gcc/cp/parser.c:12150

Interesting frame is 1397471, from which we can extract location:

    (gdb) p line_table[0].info_ordinary.maps[line_table[0].info_ordinary.cache]
    $54 = {
      <line_map> = {
        start_location = 1396581280
      },
      members of line_map_ordinary:
      reason = LC_RENAME,
      sysp = 0 '\000',
      m_column_and_range_bits = 8,
      m_range_bits = 0,
      to_file = 0x3eb4bb0 "/ch/src/Interpreters/MySQL/InterpretersMySQLDDLQuery.cpp",
      to_line = 46,
      included_from = 0
    }

By replicating SOURCE_LINE() macro from gcc-11 (libcpp/include/line-map.h):

    /* Converts a map and a location_t to source line.  */
    inline linenum_type
    SOURCE_LINE (const line_map_ordinary *ord_map, location_t loc)
    {
      return ((loc - ord_map->start_location)
          >> ord_map->m_column_and_range_bits) + ord_map->to_line;
    }

We got line 154:

    (gdb) p ((input_location-1396581280) >> 8) + 46
    $61 = 154

And this is:

    auto [column_name_and_type, declare_column_ast] = std::tuple{columns_name_and_type.begin(), columns_definition->children.begin()};

After rewriting it, everything works correctly.

Also here is a reproducer for gcc-11 (and no failures for gcc-12, but
gcc-12 has other issues, like [1] and one more for hash table):

    # cat /tmp/test.cpp
    #include <tuple>

    auto multi()
    {
            return std::tuple{1, 1};
    }
    double foo()
    {
            auto [a, b] = multi();
            return a - b;
    }

    # g++-11 -std=gnu++20 -c -o /dev/null -isystem /ch/contrib/libcxx/include -nostdinc++ /tmp/test.cpp
    g++-11: internal compiler error: Segmentation fault signal terminated program cc1plus
    Please submit a full bug report,
    with preprocessed source if appropriate.
    See <file:///usr/share/doc/gcc-11/README.Bugs> for instructions.
    # g++-12 -std=gnu++20 -c -o /dev/null -isystem /ch/contrib/libcxx/include -nostdinc++ /tmp/test.cpp

  [1]: https://reviews.llvm.org/D122598

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-03-28 22:21:45 +03:00
alesapin
ab1000c815
Merge pull request #35685 from ClickHouse/rerun-failed-jobs
Rerun failed jobs only for failed workflowa
2022-03-28 21:21:45 +02:00
alesapin
438e44e68d
Merge pull request #35659 from ClickHouse/fix_flaky_test
Fix multiple flaky tests
2022-03-28 21:18:37 +02:00
Kseniia Sumarokova
859af97fc7
Merge pull request #35614 from kssenii/fix-materialized-postgresql
Fix possible segfault in materialised postgresql
2022-03-28 21:11:44 +02:00
Vladimir C
ecdd89e811
Merge pull request #35666 from vdimir/fix-fuse 2022-03-28 20:40:43 +02:00
tavplubix
5e641e2dac
Merge pull request #35675 from ClickHouse/tavplubix-patch-2
Fix randomization of max_read_buffer_size
2022-03-28 20:45:27 +03:00
tavplubix
9a650456eb
Update 01037_polygon_dicts_correctness_fast.sh 2022-03-28 20:42:51 +03:00
Anton Popov
d677635cd8
Merge pull request #35592 from CurtizJ/dynamic-columns-4
Add parallel parsing and schema inference for format `JSONAsObject`
2022-03-28 19:29:55 +02:00
Mikhail f. Shiryaev
ea7939103a
Add debug information about event, rerun only failed jobs 2022-03-28 19:15:11 +02:00
Alexander Gololobov
93021e387e Implementation of makeDate and makeDate32 2022-03-28 18:57:24 +02:00
tavplubix
093bdc2c13
Merge pull request #35644 from ClickHouse/metric-async-insert-bytes
Add metric about size of async INSERTs
2022-03-28 19:33:11 +03:00
alesapin
9c3e9a2c9b
Update tests/queries/0_stateless/01108_restart_replicas_rename_deadlock_zookeeper.sh
Co-authored-by: Azat Khuzhin <a3at.mail@gmail.com>
2022-03-28 17:11:20 +02:00
Kruglov Pavel
933eaeae94
Fix black test 2022-03-28 16:26:20 +02:00
avogar
1f028dd48e Fix flaky tests 02149_read_in_order_fixed_prefix and 02177_issue_31009 2022-03-28 14:13:56 +00:00
tavplubix
e0dbe1785c
Merge pull request #35657 from ClickHouse/fix_flaky_test_1532
Fix flaky test `01532_primary_key_without_order_by_zookeeper`
2022-03-28 17:08:12 +03:00
tavplubix
239be83757
Merge pull request #35669 from ClickHouse/fast_test_disable_rand_settings
Disable random settings in Fast Test
2022-03-28 17:06:12 +03:00
tavplubix
1eef2d7c26
Update 01532_primary_key_without_order_by_zookeeper.sql 2022-03-28 16:56:25 +03:00
alesapin
b838a7dcb0 Remove outdated links from CI 2022-03-28 15:53:22 +02:00
alesapin
e75a0a594d Better fix for procfs_metrics 2022-03-28 15:39:02 +02:00
Kruglov Pavel
9f2cba498c
Fix timeouts in 01514_parallel_formatting test 2022-03-28 15:34:32 +02:00
tavplubix
f10930c9c8
Update clickhouse-test 2022-03-28 16:33:01 +03:00
alesapin
51ef88e5dd Fix 00385 2022-03-28 15:29:13 +02:00
tavplubix
e2573eae01
Merge pull request #35663 from ClickHouse/tavplubix-patch-2
Fix flaky test `01091_num_threads`
2022-03-28 16:27:39 +03:00
taiyang-li
38702be40c Merge remote-tracking branch 'ck/master' into rocksdb_metacache 2022-03-28 21:03:22 +08:00
alesapin
2db8d354c6
Merge pull request #35665 from ClickHouse/black-verbosity
Improve black check: show diff in the output
2022-03-28 14:57:30 +02:00
Alexander Tokmakov
e795b83148 disable random settings in fast test 2022-03-28 14:56:23 +02:00