Commit Graph

63158 Commits

Author SHA1 Message Date
Benjamin Naecker
57fd59b73b Adds support for building on Solaris-derived systems
This commit adds a number of changes to platform-detection and
compile-time directives required to support building ClickHouse on
Solaris-derived systems, most notably illumos.
2021-04-28 16:32:41 -07:00
Azat Khuzhin
8b356ba7e0 Add MSan annotation for HashTable
MSan reports [1], strictly speaking it is UB but let's suppress for now:

    [ RUN      ] HierarchyDictionariesUtils.getDescendants
    ==12==WARNING: MemorySanitizer: use-of-uninitialized-value
        0 0x6c22057 in HashTable<unsigned long, HashMapCell<unsigned long, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 15ul, 16ul>, DefaultHash<unsigned long, void>, HashTableNoState>, DefaultHash<unsigned long, void>, HashTableGrower<8ul>, Allocator<true, true> >::iterator_base<HashTable<unsigned long, HashMapCell<unsigned long, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 15ul, 16ul>, DefaultHash<unsigned long, void>, HashTableNoState>, DefaultHash<unsigned long, void>, HashTableGrower<8ul>, Allocator<true, true> >::iterator, false>::operator++() obj-x86_64-linux-gnu/../src/Common/HashTable/HashTable.h:613:17
        1 0x6c22057 in HashTable<unsigned long, HashMapCell<unsigned long, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 15ul, 16ul>, DefaultHash<unsigned long, void>, HashTableNoState>, DefaultHash<unsigned long, void>, HashTableGrower<8ul>, Allocator<true, true> >::destroyElements() obj-x86_64-linux-gnu/../src/Common/HashTable/HashTable.h:587:71
        2 0x6c22057 in HashTable<unsigned long, HashMapCell<unsigned long, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 15ul, 16ul>, DefaultHash<unsigned long, void>, HashTableNoState>, DefaultHash<unsigned long, void>, HashTableGrower<8ul>, Allocator<true, true> >::~HashTable() obj-x86_64-linux-gnu/../src/Common/HashTable/HashTable.h:691:9
        3 0x6c0df94 in HierarchyDictionariesUtils_getDescendants_Test::TestBody() obj-x86_64-linux-gnu/../src/Dictionaries/tests/gtest_hierarchy_dictionaries_utils.cpp:186:5
        4 0x731acc9 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) obj-x86_64-linux-gnu/../contrib/googletest/googletest/src/gtest.cc:2589:10
        5 0x731acc9 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) obj-x86_64-linux-gnu/../contrib/googletest/googletest/src/gtest.cc:2625:14
        6 0x7274255 in testing::Test::Run() obj-x86_64-linux-gnu/../contrib/googletest/googletest/src/gtest.cc:2664:5
        7 0x727b4e3 in testing::TestInfo::Run() obj-x86_64-linux-gnu/../contrib/googletest/googletest/src/gtest.cc:2842:11
        8 0x727e747 in testing::TestSuite::Run() obj-x86_64-linux-gnu/../contrib/googletest/googletest/src/gtest.cc:2996:28
        9 0x72c87cc in testing::internal::UnitTestImpl::RunAllTests() obj-x86_64-linux-gnu/../contrib/googletest/googletest/src/gtest.cc:5708:44
        10 0x731f035 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) obj-x86_64-linux-gnu/../contrib/googletest/googletest/src/gtest.cc:2589:10
        11 0x731f035 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) obj-x86_64-linux-gnu/../contrib/googletest/googletest/src/gtest.cc:2625:14
        12 0x72c574a in testing::UnitTest::Run() obj-x86_64-linux-gnu/../contrib/googletest/googletest/src/gtest.cc:5291:10
        13 0x6a820d9 in RUN_ALL_TESTS() obj-x86_64-linux-gnu/../contrib/googletest/googletest/include/gtest/gtest.h:2471:46
        14 0x6a820d9 in main obj-x86_64-linux-gnu/../src/Coordination/tests/gtest_for_build.cpp:1308:12
        15 0x7f579f7f00b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
        16 0x652f50d in _start (/unit_tests_dbms+0x652f50d)

      Memory was marked as uninitialized
        0 0x653bc0e in __sanitizer_dtor_callback (/unit_tests_dbms+0x653bc0e)
        1 0x6c21d16 in PairNoInit<unsigned long, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 15ul, 16ul> >::~PairNoInit() obj-x86_64-linux-gnu/../src/Common/HashTable/HashMap.h:20:8
        2 0x6c21d16 in HashMapCell<unsigned long, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 15ul, 16ul>, DefaultHash<unsigned long, void>, HashTableNoState>::~HashMapCell() obj-x86_64-linux-gnu/../src/Common/HashTable/HashMap.h:49:8
        3 0x6c21d16 in HashTable<unsigned long, HashMapCell<unsigned long, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 15ul, 16ul>, DefaultHash<unsigned long, void>, HashTableNoState>, DefaultHash<unsigned long, void>, HashTableGrower<8ul>, Allocator<true, true> >::destroyElements() obj-x86_64-linux-gnu/../src/Common/HashTable/HashTable.h:588:26
        4 0x6c21d16 in HashTable<unsigned long, HashMapCell<unsigned long, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 15ul, 16ul>, DefaultHash<unsigned long, void>, HashTableNoState>, DefaultHash<unsigned long, void>, HashTableGrower<8ul>, Allocator<true, true> >::~HashTable() obj-x86_64-linux-gnu/../src/Common/HashTable/HashTable.h:691:9
        5 0x6c0df94 in HierarchyDictionariesUtils_getDescendants_Test::TestBody() obj-x86_64-linux-gnu/../src/Dictionaries/tests/gtest_hierarchy_dictionaries_utils.cpp:186:5
        6 0x731acc9 in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) obj-x86_64-linux-gnu/../contrib/googletest/googletest/src/gtest.cc:2589:10
        7 0x731acc9 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) obj-x86_64-linux-gnu/../contrib/googletest/googletest/src/gtest.cc:2625:14
        8 0x7274255 in testing::Test::Run() obj-x86_64-linux-gnu/../contrib/googletest/googletest/src/gtest.cc:2664:5
        9 0x727b4e3 in testing::TestInfo::Run() obj-x86_64-linux-gnu/../contrib/googletest/googletest/src/gtest.cc:2842:11
        10 0x727e747 in testing::TestSuite::Run() obj-x86_64-linux-gnu/../contrib/googletest/googletest/src/gtest.cc:2996:28
        11 0x72c87cc in testing::internal::UnitTestImpl::RunAllTests() obj-x86_64-linux-gnu/../contrib/googletest/googletest/src/gtest.cc:5708:44
        12 0x731f035 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) obj-x86_64-linux-gnu/../contrib/googletest/googletest/src/gtest.cc:2589:10
        13 0x731f035 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) obj-x86_64-linux-gnu/../contrib/googletest/googletest/src/gtest.cc:2625:14
        14 0x72c574a in testing::UnitTest::Run() obj-x86_64-linux-gnu/../contrib/googletest/googletest/src/gtest.cc:5291:10
        15 0x6a820d9 in RUN_ALL_TESTS() obj-x86_64-linux-gnu/../contrib/googletest/googletest/include/gtest/gtest.h:2471:46
        16 0x6a820d9 in main obj-x86_64-linux-gnu/../src/Coordination/tests/gtest_for_build.cpp:1308:12
        17 0x7f579f7f00b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)

    SUMMARY: MemorySanitizer: use-of-uninitialized-value obj-x86_64-linux-gnu/../src/Common/HashTable/HashTable.h:613:17 in HashTable<unsigned long, HashMapCell<unsigned long, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 15ul, 16ul>, DefaultHash<unsigned long, void>, HashTableNoState>, DefaultHash<unsigned long, void>, HashTableGrower<8ul>, Allocator<true, true> >::iterator_base<HashTable<unsigned long, HashMapCell<unsigned long, DB::PODArray<unsigned long, 4096ul, Allocator<false, false>, 15ul, 16ul>, DefaultHash<unsigned long, void>, HashTableNoState>, DefaultHash<unsigned long, void>, HashTableGrower<8ul>, Allocator<true, true> >::iterator, false>::operator++()
    Exiting

  [1]: https://clickhouse-test-reports.s3.yandex.net/23664/06bb4b2b33513f9cc1cd8faed75397db1f1ed914/unit_tests_msan/test_run.txt.out.log
2021-04-28 23:05:24 +03:00
Azat Khuzhin
d83ea5844f Enable use-after-destruction detection in MSan
Refs: https://clang.llvm.org/docs/MemorySanitizer.html#use-after-destruction-detection
2021-04-28 23:02:52 +03:00
Maksim Kita
9bf96a278d
Merge pull request #23711 from ClickHouse/remove-old-compilers
Remove obsolete compilers
2021-04-28 22:51:21 +03:00
Maksim Kita
fa44db6e8a
Merge pull request #23740 from ClickHouse/ignore-cmake-in-clickhouse
Ignore cmake-in-clickhouse
2021-04-28 22:49:25 +03:00
Maksim Kita
1f1a443798
Merge pull request #23152 from amosbird/distributedwriteha
Skip unavaiable replicas when writing to distributed tables
2021-04-28 22:46:50 +03:00
Maksim Kita
4090c66e1c
Merge pull request #23722 from vdimir/odbc_test_disable_msan
Disable test_odbc_interaction for MSan build
2021-04-28 22:45:40 +03:00
Azat Khuzhin
73ab415c4c Preserve errors for INSERT into Distributed
Before this patch (and after #22208) the INSERT may fail with "Cannot
schedule a task" because the pool in DistributedBlockOutputStream
already throws exception and simply fail in writeSuffix().
2021-04-28 22:33:29 +03:00
Azat Khuzhin
590a6b9855 Add ThreadPool::finished() 2021-04-28 22:33:29 +03:00
Azat Khuzhin
91fc2fc87f tests/config/config.d/clusters.xml: fix indent 2021-04-28 21:26:12 +03:00
Nikolai Kochetov
abd8577ded Fix some tests 2021-04-28 21:21:26 +03:00
Nikolai Kochetov
69ddca1d73 Refactor join step, part 3 2021-04-28 20:32:12 +03:00
vdimir
fb88547a04
Call nestedToNullable without copying column in convertColumnToNullable for join 2021-04-28 20:18:03 +03:00
vdimir
6e4b37a45e
Merge branch 'master' into fix-join-lowcard-nullable 2021-04-28 20:06:49 +03:00
Nikolai Kochetov
9392b493d9 Skip 01823_explain_json for antlr 2021-04-28 19:33:49 +03:00
Nikolai Kochetov
77233bbdbb Ignore cmake-in-clickhouse 2021-04-28 18:10:12 +03:00
Ilya Yatsishin
8e67cab2a3
Merge pull request #23138 from qoega/add-dockerproxy-to-runner
Add dockerhub-proxy to runner
2021-04-28 17:10:28 +03:00
Nikolai Kochetov
c508868570
Merge pull request #23082 from ClickHouse/explain-output
JSON output for EXPLAIN
2021-04-28 15:39:06 +03:00
Maksim Kita
c7ff95fbd9 ExternalLoaderRepository fix arcadia 2021-04-28 14:40:35 +03:00
alexey-milovidov
07bf1f203a
Merge pull request #23709 from ClickHouse/minor-changes
Minor changes in code #14254
2021-04-28 14:33:56 +03:00
alexey-milovidov
f541e4017c
Merge pull request #23710 from ClickHouse/fix-bad-error-message
Fix illiterate error message #23700
2021-04-28 14:32:33 +03:00
Alexander Kuzmenkov
526f519c1f
Merge pull request #23446 from kssenii/globs-for-url
Support globs and failover for URL table function
2021-04-28 13:51:49 +03:00
Maksim Kita
b544037f03
Merge pull request #23436 from kitaisreal/storage-dictionary-updated
Refactored StorageDictionary
2021-04-28 13:11:48 +03:00
Alexander Kuzmenkov
0c1e897e80
Merge pull request #23707 from ClickHouse/aku/perf-stable-query-index
stable query indexes in perf test
2021-04-28 13:06:53 +03:00
Nikolai Kochetov
181273730d
Merge pull request #23683 from ClickHouse/enable_async_socket_for_remote
Enable async_socket_for_remote
2021-04-28 12:29:50 +03:00
alexey-milovidov
9b9ce0c3e5
Merge pull request #23712 from ClickHouse/followup-examples
Follow-up for #23644
2021-04-28 12:21:41 +03:00
vdimir
b8acfa8312
Fix typo in tests/integration/README.md 2021-04-28 11:40:24 +03:00
vdimir
67efca21dd
Disable test_odbc_interaction for MSan build 2021-04-28 11:39:55 +03:00
alexey-milovidov
d3f4fb404a
Merge pull request #23650 from arenadata/ADQM-284
Kerberized HDFS test is fluky - retries and extra output if failure
2021-04-28 07:38:47 +03:00
Alexey Milovidov
d3b4f22e4e Follow-up for #23644 2021-04-28 07:32:23 +03:00
alexey-milovidov
fdc5a44202
Merge pull request #23644 from nikitamikhaylov/delete-unit-tests
Move non gtest unit tests to /examples folder
2021-04-28 07:28:18 +03:00
Alexey Milovidov
eda56e52d8 Remove obsolete compilers 2021-04-28 07:16:48 +03:00
Amos Bird
fd3aef0feb
Update tests/integration/test_insert_into_distributed/test.py
Co-authored-by: Azat Khuzhin <a3at.mail@gmail.com>
2021-04-28 12:16:27 +08:00
alexey-milovidov
5dd1c72502
Update 40_bug-report.md 2021-04-28 06:53:00 +03:00
Alexey Milovidov
2cf9da72ff Fix illiterate error message #23700 2021-04-28 06:49:46 +03:00
Alexey Milovidov
f0927cbb43 Minor changes in code #14254 2021-04-28 06:33:28 +03:00
Alexey Milovidov
c1d5713110 Minor changes in code #14254 2021-04-28 06:31:15 +03:00
Alexey Milovidov
b66f9af7ce Update documentation for min/max 2021-04-28 04:41:38 +03:00
alexey-milovidov
76c78e3cec
Update max.md 2021-04-28 04:37:44 +03:00
alexey-milovidov
d1fdc311be
Update min.md 2021-04-28 04:37:06 +03:00
Alexander Kuzmenkov
cc0dd732d9 stable query indexes in perf test 2021-04-28 00:58:37 +03:00
Ilya Yatsishin
2f7a7c04f8
Merge pull request #23672 from qoega/cyrus-sasl-msan 2021-04-27 23:18:00 +03:00
Maksim Kita
3747a0c8e5 Fixed TRUNCATE TABLE for Dictionary storage 2021-04-27 22:13:21 +03:00
Maksim Kita
2e5f97c6ff Fixed code review issues 2021-04-27 22:09:03 +03:00
Nikolai Kochetov
3a25e46a8f Refactor join step, part 2 2021-04-27 20:30:33 +03:00
Nikolai Kochetov
5837fdabf2
Merge pull request #23681 from ClickHouse/devirt-agg-merge
A little bit faster merge of aggregating states.
2021-04-27 20:25:52 +03:00
alexey-milovidov
16f81a0303
Merge pull request #23662 from ClickHouse/skip-library-bridge-under-msan
Skip integration test for library bridge under MSan
2021-04-27 19:25:18 +03:00
Nikolai Kochetov
a51a380645 Refactor join step, part 1 2021-04-27 17:00:16 +03:00
Nikolai Kochetov
96e84a44d1
Merge pull request #23594 from lehasm/alexey-sm-DOCSUP-6702-document-timezoneOffset
DOCSUP-6702: document the timezoneOffset function
2021-04-27 15:42:17 +03:00
Pavel Kovalenko
838ede6c62 Rename delegate -> impl. 2021-04-27 15:11:38 +03:00