Commit Graph

1935 Commits

Author SHA1 Message Date
Yuriy Chernyshov
1f1c61ce94 Fix undefined behavior due to unaligned read 2021-06-22 13:00:05 +03:00
Yuriy Chernyshov
23486ff86c Make block length size_t 2021-06-22 12:59:44 +03:00
Yatsishin Ilya
6a79fef8bd submodule 2021-06-22 12:35:56 +03:00
Yatsishin Ilya
8a70a9ba7c update cmake 2021-06-22 10:39:27 +03:00
Yatsishin Ilya
fdd3481c2f Merge remote-tracking branch 'origin' into update-h3 2021-06-22 09:53:16 +03:00
alesapin
2f24ad6165 Merge branch 'fix_some_tests' into update_buffer_size_in_nuraft 2021-06-21 13:20:27 +03:00
Kruglov Pavel
a54cbef5b7
Merge pull request #24341 from Avogar/arrow
Support complex types in Arrow/Parquet/ORC
2021-06-21 11:05:57 +03:00
Nikolay Degterinsky
00154d3163 Minor fixes 2021-06-20 12:31:07 +00:00
Nikolay Degterinsky
844f5fc62b Updated submodules to include new READMEs 2021-06-19 23:15:16 +00:00
Azat Khuzhin
efb6433a39 Fix container-overflow in replxx during incremental search (Ctrl-R)
ASAN report:

    =================================================================
    ==7686==ERROR: AddressSanitizer: container-overflow on address 0x6200000bf080 at pc 0x00002a787e79 bp 0x7fffffffa2f0 sp 0x7fffffffa2e8
    READ of size 4 at 0x6200000bf080 thread T0
        0 0x2a787e78 in replxx::calculate_displayed_length(char32_t const*, int) obj-x86_64-linux-gnu/../contrib/replxx/src/util.cxx:66:15
        1 0x2a75786c in replxx::Replxx::ReplxxImpl::dynamicRefresh(replxx::Prompt&, char32_t*, int, int) obj-x86_64-linux-gnu/../contrib/replxx/src/replxx_impl.cxx:2201:3
        2 0x2a7453f0 in replxx::Replxx::ReplxxImpl::incremental_history_search(char32_t) obj-x86_64-linux-gnu/../contrib/replxx/src/replxx_impl.cxx:2008:3
        3 0x2a73eecc in replxx::Replxx::ReplxxImpl::action(unsigned long long, replxx::Replxx::ACTION_RESULT (replxx::Replxx::ReplxxImpl::* const&)(char32_t), char32_t) obj-x86_64-linux-gnu/../contrib/replxx/src/replxx_impl.cxx:1246:29
        4 0x2a73eecc in replxx::Replxx::ReplxxImpl::invoke(replxx::Replxx::ACTION, char32_t) obj-x86_64-linux-gnu/../contrib/replxx/src/replxx_impl.cxx:318:70
        5 0x2a74ed29 in std::__1::__function::__policy_func<replxx::Replxx::ACTION_RESULT (char32_t)>::operator()(char32_t&&) const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2221:16
        6 0x2a74ed29 in std::__1::function<replxx::Replxx::ACTION_RESULT (char32_t)>::operator()(char32_t) const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2560:12
        7 0x2a74ed29 in replxx::Replxx::ReplxxImpl::get_input_line() obj-x86_64-linux-gnu/../contrib/replxx/src/replxx_impl.cxx🔢11
        8 0x2a74dd3c in replxx::Replxx::ReplxxImpl::input(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) obj-x86_64-linux-gnu/../contrib/replxx/src/replxx_impl.cxx:580:8
        9 0x2a2a4075 in ReplxxLineReader::readOneLine(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) obj-x86_64-linux-gnu/../base/common/ReplxxLineReader.cpp:112:29
        10 0x2a29b499 in LineReader::readLine(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) obj-x86_64-linux-gnu/../base/common/LineReader.cpp:81:26
        11 0xb580f02 in DB::Client::mainImpl() obj-x86_64-linux-gnu/../programs/client/Client.cpp:665:33
        12 0xb575825 in DB::Client::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) obj-x86_64-linux-gnu/../programs/client/Client.cpp:300:20
        13 0x2a3aff25 in Poco::Util::Application::run() obj-x86_64-linux-gnu/../contrib/poco/Util/src/Application.cpp:334:8
        14 0xb54c810 in mainEntryClickHouseClient(int, char**) obj-x86_64-linux-gnu/../programs/client/Client.cpp:2702:23
        15 0xb326d8a in main obj-x86_64-linux-gnu/../programs/main.cpp:360:12
        16 0x7ffff7dcbb24 in __libc_start_main (/usr/lib/libc.so.6+0x27b24)
        17 0xb2794ad in _start (/src/ch/tmp/upstream/clickhouse-asan+0xb2794ad)

    0x6200000bf080 is located 0 bytes inside of 3672-byte region [0x6200000bf080,0x6200000bfed8)
    allocated by thread T0 here:
        0 0xb3231dd in operator new(unsigned long) (/src/ch/tmp/upstream/clickhouse-asan+0xb3231dd)
        1 0x2a75fb15 in void* std::__1::__libcpp_operator_new<unsigned long>(unsigned long) obj-x86_64-linux-gnu/../contrib/libcxx/include/new:235:10
        2 0x2a75fb15 in std::__1::__libcpp_allocate(unsigned long, unsigned long) obj-x86_64-linux-gnu/../contrib/libcxx/include/new:261:10
        3 0x2a75fb15 in std::__1::allocator<char32_t>::allocate(unsigned long) obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:840:38
        4 0x2a75fb15 in std::__1::allocator_traits<std::__1::allocator<char32_t> >::allocate(std::__1::allocator<char32_t>&, unsigned long) obj-x86_64-linux-gnu/../contrib/libcxx/include/__memory/allocator_traits.h:468:21
        5 0x2a75fb15 in std::__1::vector<char32_t, std::__1::allocator<char32_t> >::__vallocate(unsigned long) obj-x86_64-linux-gnu/../contrib/libcxx/include/vector:993:37
        6 0x2a75fb15 in std::__1::enable_if<(__is_cpp17_forward_iterator<char32_t*>::value) && (is_constructible<char32_t, std::__1::iterator_traits<char32_t*>::reference>::value), void>::type std::__1::vector<char32_t, std::__1::allocator<char32_t> >::assign<char32_t*>(char32_t*, char32_t*) obj-x86_64-linux-gnu/../contrib/libcxx/include/vector:1460:9
        7 0x2a745242 in std::__1::vector<char32_t, std::__1::allocator<char32_t> >::operator=(std::__1::vector<char32_t, std::__1::allocator<char32_t> > const&) obj-x86_64-linux-gnu/../contrib/libcxx/include/vector:1405:9
        8 0x2a745242 in replxx::UnicodeString::assign(replxx::UnicodeString const&) obj-x86_64-linux-gnu/../contrib/replxx/src/unicodestring.hxx:83:9
        9 0x2a745242 in replxx::Replxx::ReplxxImpl::incremental_history_search(char32_t) obj-x86_64-linux-gnu/../contrib/replxx/src/replxx_impl.cxx:1993:24
        10 0x2a73eecc in replxx::Replxx::ReplxxImpl::action(unsigned long long, replxx::Replxx::ACTION_RESULT (replxx::Replxx::ReplxxImpl::* const&)(char32_t), char32_t) obj-x86_64-linux-gnu/../contrib/replxx/src/replxx_impl.cxx:1246:29
        11 0x2a73eecc in replxx::Replxx::ReplxxImpl::invoke(replxx::Replxx::ACTION, char32_t) obj-x86_64-linux-gnu/../contrib/replxx/src/replxx_impl.cxx:318:70
        12 0x2a74ed29 in std::__1::__function::__policy_func<replxx::Replxx::ACTION_RESULT (char32_t)>::operator()(char32_t&&) const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2221:16
        13 0x2a74ed29 in std::__1::function<replxx::Replxx::ACTION_RESULT (char32_t)>::operator()(char32_t) const obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2560:12
        14 0x2a74ed29 in replxx::Replxx::ReplxxImpl::get_input_line() obj-x86_64-linux-gnu/../contrib/replxx/src/replxx_impl.cxx🔢11
        15 0x2a74dd3c in replxx::Replxx::ReplxxImpl::input(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) obj-x86_64-linux-gnu/../contrib/replxx/src/replxx_impl.cxx:580:8
        16 0x2a2a4075 in ReplxxLineReader::readOneLine(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) obj-x86_64-linux-gnu/../base/common/ReplxxLineReader.cpp:112:29
        17 0x2a29b499 in LineReader::readLine(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) obj-x86_64-linux-gnu/../base/common/LineReader.cpp:81:26
        18 0xb580f02 in DB::Client::mainImpl() obj-x86_64-linux-gnu/../programs/client/Client.cpp:665:33
        19 0xb575825 in DB::Client::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) obj-x86_64-linux-gnu/../programs/client/Client.cpp:300:20
        20 0x2a3aff25 in Poco::Util::Application::run() obj-x86_64-linux-gnu/../contrib/poco/Util/src/Application.cpp:334:8
        21 0xb54c810 in mainEntryClickHouseClient(int, char**) obj-x86_64-linux-gnu/../programs/client/Client.cpp:2702:23
        22 0xb326d8a in main obj-x86_64-linux-gnu/../programs/main.cpp:360:12
        23 0x7ffff7dcbb24 in __libc_start_main (/usr/lib/libc.so.6+0x27b24)

    HINT: if you don't care about these errors you may set ASAN_OPTIONS=detect_container_overflow=0.
    If you suspect a false positive see also: https://github.com/google/sanitizers/wiki/AddressSanitizerContainerOverflow.
    SUMMARY: AddressSanitizer: container-overflow obj-x86_64-linux-gnu/../contrib/replxx/src/util.cxx:66:15 in replxx::calculate_displayed_length(char32_t const*, int)
    Shadow bytes around the buggy address:
      0x0c408000fdc0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
      0x0c408000fdd0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0c408000fde0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0c408000fdf0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
      0x0c408000fe00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
    =>0x0c408000fe10:[fc]fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
      0x0c408000fe20: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
      0x0c408000fe30: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
      0x0c408000fe40: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
      0x0c408000fe50: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
      0x0c408000fe60: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
    Shadow byte legend (one shadow byte represents 8 application bytes):
      Addressable:           00
      Partially addressable: 01 02 03 04 05 06 07
      Heap left redzone:       fa
      Freed heap region:       fd
      Stack left redzone:      f1
      Stack mid redzone:       f2
      Stack right redzone:     f3
      Stack after return:      f5
      Stack use after scope:   f8
      Global redzone:          f9
      Global init order:       f6
      Poisoned by user:        f7
      Container overflow:      fc
      Array cookie:            ac
      Intra object redzone:    bb
      ASan internal:           fe
      Left alloca redzone:     ca
      Right alloca redzone:    cb
      Shadow gap:              cc
    ==7686==ABORTING

Refs: https://github.com/ClickHouse-Extras/replxx/pull/16

v2: fix test, do not use /dev/null since it client will lock it
2021-06-19 18:41:33 +03:00
Yatsishin Ilya
a1e3a3f2c1 update h3 2021-06-18 17:21:53 +03:00
mergify[bot]
c723dd7d40
Merge branch 'master' into arrow 2021-06-18 12:18:57 +00:00
Yatsishin Ilya
b47ce8de89 update 2021-06-18 13:34:15 +03:00
alesapin
9e77ca7182
Merge branch 'master' into nlp 2021-06-17 21:47:41 +03:00
alesapin
dc893f9644 Update NuRaft buffer to 64 bit size 2021-06-17 19:37:08 +03:00
Yuriy Chernyshov
2bb1c75fb5 Rename & reimport murmurhash sources from smhasher repo 2021-06-17 10:58:42 +03:00
Kseniia Sumarokova
60d220bd14
Merge pull request #24413 from kssenii/postgres-fixes-schema
postgres non-default schema for insert query
2021-06-16 22:18:23 +03:00
Pavel Kruglov
83919363ea Fix broken contrib 2021-06-16 16:55:34 +03:00
Pavel Kruglov
81998a5c6f Fix 2021-06-16 16:12:23 +03:00
Pavel Kruglov
5c6bda6294 Remove useless srcs 2021-06-16 15:57:36 +03:00
kssenii
8a7af3bc23 Update libpqxx 2021-06-16 09:38:33 +03:00
kssenii
fb1c3d0360 Update libpqxx 2021-06-15 18:47:50 +03:00
Pavel Kruglov
83c843a20b Fix tests and build 2021-06-15 16:15:27 +03:00
Pavel Kruglov
02e68655b4 Update orc and arrow 2021-06-15 16:15:27 +03:00
kssenii
7c81b37748 Update libpqxx 2021-06-15 15:25:34 +03:00
bharatnc
b17c335074 h3 - update submodule 2021-06-14 11:38:38 -07:00
alexey-milovidov
77721734ed
Merge pull request #21267 from filimonov/kafka-failover
Kafka failover issue fix
2021-06-14 01:42:42 +03:00
alexey-milovidov
a2c6b98b94
Merge pull request #25146 from oxidecomputer/master
Adds a better way to include binary resources
2021-06-13 16:13:34 +03:00
kssenii
8cbd9ec733 Fixes 2021-06-10 16:23:04 +00:00
Alexey Milovidov
2dd871077e Update roaring just in case 2021-06-10 02:36:36 +03:00
Benjamin Naecker
c3506bf16d Adds a better way to include binary resources
- Uses a small assembly file to include binary resources, rather than
  objcopy
- Updates `base/common/getResource.cpp` for this new method of inclusion
- Removes linux-only guards in CMake files, as this solution is
  cross-platform.

The resulting binary resources are available in the ClickHouse server
binary on Linux, macOS, and illumos platforms. FreeBSD has not been
tested, but will likely work as well.
2021-06-09 14:03:30 -07:00
Mikhail Filimonov
3627ab7cbe
Fix kafka failover issue (#21118).
Distringuish lack of assignment vs lack of partitions. Bit better / clearer logging.
2021-06-09 16:02:56 +02:00
kssenii
0521851afa Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into postgres-fixes-schema 2021-06-08 17:20:23 +00:00
Nikolay Degterinsky
71fa39cfc3 Updated lemmagen-c 2021-06-07 22:19:45 +00:00
Nikolay Degterinsky
1f69c33a98 Updated boost 2021-06-07 21:39:49 +00:00
Maksim Kita
23bf4cf09f
Merge pull request #25034 from sdk2/patch-1
fix jemalloc build on OS_DARWIN
2021-06-07 21:08:28 +03:00
Ivan
71f1a101c1
More pytest fixes (#23538) 2021-06-07 20:29:19 +03:00
sdk2
717f7ed01b
Update contrib/jemalloc-cmake/CMakeLists.txt 2021-06-07 16:38:24 +08:00
Alexey Milovidov
8d85145725 Remove experimental compression codecs 2021-06-06 02:26:52 +03:00
Alexey Milovidov
17962459f5 Merge branch 'master' into issue-16775 2021-06-06 02:18:28 +03:00
Nikolay Degterinsky
fb217ca916 added LemmaGen lemmatizer 2021-06-05 00:52:35 +00:00
Nikolay Degterinsky
41f8c9546a merged registerFunctions 2021-06-05 00:49:13 +00:00
Nikolay Degterinsky
480da5838e merged and resolved the conflict in Context.cpp 2021-06-03 20:23:14 +00:00
Nikolay Degterinsky
fc8e7c5d98 moved wordnet-blast to fork 2021-06-03 19:53:09 +00:00
Nikolay Degterinsky
ed12fb5604 added WordNet synonyms extensions 2021-06-03 19:28:12 +00:00
Nikolay Degterinsky
876f51ab95 minor fix 2021-06-03 19:20:35 +00:00
Nikolay Degterinsky
b47cebfcd5 minor fix in CMakeLists.txt 2021-06-03 15:07:41 +00:00
Nikolay Degterinsky
8d07b0f9ee moved stemmer to submodule 2021-06-03 15:03:56 +00:00
mergify[bot]
438b45e6c0
Merge branch 'master' into add-sqlite-support 2021-06-03 12:26:12 +00:00
Arslan G
6f7dc47b2b Merge branch 'master' into add-sqlite-support 2021-06-02 00:24:29 +03:00
mergify[bot]
43295665b9
Merge branch 'master' into update_nuraft 2021-06-01 20:34:53 +00:00
Maksim Kita
b4570bc06b Reverted libunwind from upstream 2021-06-01 21:09:41 +03:00
Nikita Mikhaylov
5dda97bef1
Merge pull request #24691 from nikitamikhaylov/debug_host_ip_change
Fix flaky integration tests
2021-06-01 15:55:51 +03:00
alesapin
0bd323b3cd Fix build 2021-06-01 10:58:53 +03:00
Maksim Kita
fd6a6fb1c9 Updated libunwind 2021-06-01 01:46:46 +03:00
alesapin
4189cca043 Update NuRaft 2021-05-30 22:28:35 +03:00
Nikita Mikhaylov
7b04181dd9 suppress msan cassandra 2021-05-28 16:22:50 +03:00
Nikolay Degterinsky
63bb8abf89 Merge branch 'master' into nlp 2021-05-28 10:26:42 +00:00
Yatsishin Ilya
99c3a917f4 Fix avro 2021-05-27 16:11:46 +03:00
Alexey Milovidov
20f59ffbb9 Merge branch 'master' into issue-16775 2021-05-26 03:46:43 +03:00
Maksim Kita
9934caeedf Revent libunwind 2021-05-25 13:40:45 +03:00
Maksim Kita
45a4bdc5d9
Merge pull request #24460 from kitaisreal/manually-disable-llvm-parts-submodule-update
Manually disable LLVM parts submodule update
2021-05-24 22:51:01 +03:00
Maksim Kita
7416cf4643 Manually disable LLVM parts submodule update 2021-05-24 20:52:12 +03:00
alexey-milovidov
246d6f35a8
Merge pull request #24419 from kitaisreal/libunwind-update-version
Libunwind update version
2021-05-24 17:32:20 +03:00
Alexey Milovidov
e206959390 Use entropy coders from zstd 2021-05-24 06:18:57 +03:00
Alexey Milovidov
6f70feed2f Merge branch 'master' into issue-16775 2021-05-24 05:56:57 +03:00
Alexey Milovidov
8d2fa6cf50 Allow to build without experimental compression libraries and remove from fasttest 2021-05-24 01:18:12 +03:00
Maksim Kita
ef58cb399a Updated submodule version 2021-05-23 17:36:10 +03:00
Alexey Milovidov
1431679f51 Remove Lizard framing format because we do not need it 2021-05-23 03:23:11 +03:00
Alexey Milovidov
de69a9f3af Fix warning in LZSSE 2021-05-23 03:04:25 +03:00
kssenii
c02088483d Update CmakeLists.txt 2021-05-22 17:26:12 +00:00
Maksim Kita
ac4db9fdcc Libunwing update version 2021-05-22 16:04:00 +00:00
alexey-milovidov
1ed30f7246
Merge branch 'master' into issue-16775 2021-05-22 17:14:23 +03:00
Maksim Kita
42e91df363 Updated submodule to llvm12 2021-05-22 15:22:38 +03:00
kssenii
3e6d3d4ecb Postgres schema for insert 2021-05-22 08:50:23 +00:00
BoloniniD
33755f1f3d Implemented YAMLParser class 2021-05-21 21:56:22 +03:00
alexey-milovidov
57348632dc
Merge pull request #24196 from Algunenano/gcc11_compat
Gcc11 compatibility
2021-05-19 22:20:22 +03:00
Arslan G
b17417f39c Fix submodules 2021-05-19 14:52:02 +03:00
Nikolai Kochetov
906bd73e79 Try mute grpc for msan. 2021-05-17 17:19:35 +03:00
Nikolai Kochetov
f18a79c857 Try mute grpc for msan. 2021-05-17 17:13:32 +03:00
Raúl Marín
97ff3a6d74 re2_st: Avoid replacing re2::re2_internal 2021-05-17 13:49:48 +02:00
Arslan G
dde9e30043 Merge branch 'master' into add-sqlite-support 2021-05-17 14:38:24 +03:00
Raúl Marín
2adf974395 Update re2 to 2021-04-01 for GCC 11 compatibility
Fixes the build with gcc version 11.1.0
2021-05-17 13:30:55 +02:00
Arslan G
ea51840f85 Init SQLite integration 2021-05-17 14:02:35 +03:00
Maksim Kita
af1eaea3e5
Merge pull request #24147 from amosbird/simdjsonfix
Bump simdjson to fix crash when alloc fails
2021-05-16 11:40:02 +03:00
alexey-milovidov
3695eb9c90
Merge pull request #24135 from Algunenano/zstd_1.5.0
Update zstd to v1.5.0
2021-05-15 17:04:30 +03:00
Amos Bird
fe6a956499
bump simdjson to fix crash when alloc fails 2021-05-15 14:15:51 +08:00
Benjamin Naecker
3252228852 Updates submodules bringing in several upstream changes to build on
illumos
2021-05-14 19:38:36 -07:00
Raúl Marín
5d2778dd2d Update zstd to v1.5.0 2021-05-15 00:57:24 +02:00
alesapin
f2a8b5b2c6 Fix concurrent snapshot read/write 2021-05-12 23:28:01 +03:00
Kruglov Pavel
fb038c1985
Merge pull request #23894 from pingyu/datasketches-uniq-again
Add uniqThetaSketch again
2021-05-11 11:24:43 +03:00
Nikolay Degterinsky
c1150ed2a5 added line in CMakelists.txt 2021-05-10 10:41:11 +00:00
Nikolay Degterinsky
0b43910164 added snowball stemmer 2021-05-10 09:34:29 +00:00
fibersel
94b159c510 add newlines 2021-05-06 16:34:27 +03:00
fibersel
d60c70a8da add submodules 2021-05-06 16:17:55 +03:00
fibersel
ef7b1750ef remove contribs 2021-05-06 15:58:21 +03:00
fibersel
a2f70306df add contribs 2021-05-06 15:55:03 +03:00
fibersel
7c9acb9f53 add dedicated cmake files for compression libraries 2021-05-06 15:26:03 +03:00
fibersel
9cfddff290 add new codecs from contrib to cmake 2021-05-06 15:08:58 +03:00
Nikolay Degterinsky
541986b8b5 added cz & ar languages to stem 2021-05-06 11:50:00 +00:00