Commit Graph

61382 Commits

Author SHA1 Message Date
Kruglov Pavel
5b52657428
Merge pull request #22725 from azat/buffer-lock-metrics
Add metric to track how much time is spend during waiting for Buffer layer lock
2021-04-07 14:35:57 +03:00
alesapin
feff1175f4 Merge branch 'master' into johnskopis/dynamic-interserver-creds-v20 2021-04-07 13:55:38 +03:00
Pavel Kruglov
19b147c7ac Allow more replica change events in hedged tests due to high server load during testing 2021-04-07 13:30:47 +03:00
alesapin
a1164a7e4c More consistent 2021-04-07 13:21:53 +03:00
alesapin
36c0e601a9 Better non-dirty fix 2021-04-07 13:18:07 +03:00
robot-clickhouse
a020b357f0 Update version_date.tsv after release 21.3.5.42 2021-04-07 12:47:25 +03:00
Nikolai Kochetov
d62e9c0b95
Merge pull request #22619 from olgarev/revolg-DOCSUP-8219-Date_time64_extended_range
DOCSUP-8219: DateTime64 extended range
2021-04-07 12:29:42 +03:00
Amos Bird
1364d939a1
Avoid using harmful function rand() 2021-04-07 17:04:35 +08:00
Ilya Yatsishin
ed2d82603e
Merge pull request #22588 from foolchi/fix_dist_map_select
fix dist map type select
2021-04-07 11:53:23 +03:00
Vladimir
864ab4a7ab
Merge pull request #21370 from vdimir/fix-merge-engine-join-group-by 2021-04-07 11:52:27 +03:00
alesapin
451ba892fa
Merge pull request #22708 from CurtizJ/fix-flaky-tests-2
Fix test 01702_system_query_log
2021-04-07 11:51:57 +03:00
Vladimir
db0550013c
Merge pull request #22325 from azat/dist-async-insert-header-check
Check only column name and type for async distributed blocks
2021-04-07 11:51:12 +03:00
alesapin
05eeec16c1 Fix potential segfault on Keeper startup 2021-04-07 11:49:10 +03:00
Maksim Kita
7baafcb567
Merge pull request #22630 from kitaisreal/direct-dictionary-dict-get-multiple-columns-optimization
DirectDictionary dictGet multiple columns optimization
2021-04-07 10:29:42 +03:00
alesapin
9fd251eaaf
Merge pull request #22514 from ClickHouse/merge_tree_deduplication
Non replicated merge tree deduplication
2021-04-07 10:19:43 +03:00
alexey-milovidov
cb6097e8ae
Merge pull request #21559 from ka1bi4/romanzhukov-DOCSUP-7045-https-replication
DOCSUP-7045: Edit and translate to Russian (Upgrading CH and https interserver replication)
2021-04-07 07:24:44 +03:00
alexey-milovidov
80cb871e20
Merge pull request #22684 from ClickHouse/exception-safety-in-zk-client-2
Always call event.set for zk client callbacks.
2021-04-07 07:20:41 +03:00
alexey-milovidov
a7e216969a
Merge pull request #22592 from azat/uncaught-exceptions
Fix some uncaught exceptions (in SCOPE_EXIT) under memory pressure
2021-04-07 07:17:26 +03:00
Olga Revyakina
b87af2c4a8 Fixed typos 2021-04-07 05:31:13 +03:00
alesapin
c77a006325
Merge pull request #22707 from ClickHouse/don_t_fsync_on_log_rotate
Remove redundant fsync on coordination logs rotation
2021-04-06 23:51:28 +03:00
Nikita Mikhaylov
0d96ed99b7 update_perf_test 2021-04-06 22:23:16 +03:00
Nikita Mikhaylov
f3f550e811 better perf test 2021-04-06 22:23:16 +03:00
Nikita Mikhaylov
7a318d25ae add perf test 2021-04-06 22:23:16 +03:00
alexey-milovidov
5d672d4529 Update FormatFactory.cpp 2021-04-06 22:23:16 +03:00
Nikita Mikhailov
71a2c85d9a better 2021-04-06 22:23:16 +03:00
Nikita Mikhailov
af85ce2fd1 better 2021-04-06 22:23:16 +03:00
Nikita Mikhailov
37f48d13b4 add test 2021-04-06 22:23:16 +03:00
Nikita Mikhailov
282775b10c save 2021-04-06 22:23:16 +03:00
keenwolf
1a4961499b the Native format is not supported after test 2021-04-06 22:23:16 +03:00
keenwolf
94c8e7af1a remove const for value-copy-return 2021-04-06 22:23:16 +03:00
keenwolf
3d3923b518 update comments 2021-04-06 22:23:16 +03:00
keenwolf
b95f54edce Small fix 2021-04-06 22:23:16 +03:00
keenwolf
139bb678a9 Branch with Format isColumnOriented() or not 2021-04-06 22:23:15 +03:00
keenwolf
4eaf718fd2 fix the getColumsForNames() to bring the whole column info from metadata 2021-04-06 22:23:15 +03:00
keenwolf
9d6e2b4bad update comments 2021-04-06 22:23:15 +03:00
keenwolf
de365a5852 fix 2021-04-06 22:23:15 +03:00
keenwolf
6f9bad682f Only read needed columns for formats as parquet etc 2021-04-06 22:23:15 +03:00
Maksim Kita
8c3ccaa247
Merge pull request #22686 from tetafro/master
Fix grammar in docs
2021-04-06 21:57:09 +03:00
Maksim Kita
e38206ee02
Merge pull request #22721 from kitaisreal/memory-storage-sync-comments-and-code
MemoryStorage sync comments and code
2021-04-06 21:56:41 +03:00
Azat Khuzhin
c4a7e81287 Add metric to track how much time is spend during waiting for Buffer layer lock
It uses very fast CLOCK_MONOTONIC_COARSE, so this should not be a
problem.
Also note that there is no sense in using microseconds/nanoseconds since
accuracy of CLOCK_MONOTONIC_COARSE usually milliseconds.
2021-04-06 21:13:24 +03:00
Azat Khuzhin
f157278b72 Safer SCOPE_EXIT
It executes the code in the dtor, that should never throw.
2021-04-06 21:10:37 +03:00
Azat Khuzhin
9394e6e5ed Add SCOPE_EXIT_SAFE/SCOPE_EXIT_MEMORY_SAFE helpers 2021-04-06 21:10:37 +03:00
Azat Khuzhin
b25a4e066c Lock MEMORY_LIMIT_EXCEEDED in ThreadStatus::detachQuery()
Found with fuzzer [1]:

    <Fatal> BaseDaemon: (version 21.5.1.6440, build id: 3B097C902DDAA35688D90750552ED499DC5D10A0) (from thread 8012) Terminate called for uncaught exception:
    Code: 241, e.displayText() = DB::Exception: Memory limit (for user) exceeded: would use 153.51 MiB (attempt to allocate chunk of 4194368 bytes), maximum: 150.00 MiB, Stack trace (when copying this message, always include the lines below):

    0. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/exception:133: Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x26fdcdd9 in /usr/bin/clickhouse
    1. ./obj-x86_64-linux-gnu/../src/Common/Exception.cpp:57: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0xad0df02 in /usr/bin/clickhouse
    2. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/string:1444: DB::Exception::Exception<char const*, char const*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, char const*&&, char const*&&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&, long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&) @ 0xacc7bef in /usr/bin/clickhouse
    3. ./obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:219: MemoryTracker::alloc(long) @ 0xacc65eb in /usr/bin/clickhouse
    4. ./obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:0: MemoryTracker::alloc(long) @ 0xacc5dad in /usr/bin/clickhouse
    5. ./obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:0: MemoryTracker::alloc(long) @ 0xacc5dad in /usr/bin/clickhouse
    6. ./obj-x86_64-linux-gnu/../src/Common/AllocatorWithMemoryTracking.h:35: AllocatorWithMemoryTracking<DB::Field>::allocate(unsigned long) @ 0xad0a2fe in /usr/bin/clickhouse
    7. void std::__1::vector<DB::Field, AllocatorWithMemoryTracking<DB::Field> >::__push_back_slow_path<DB::Field>(DB::Field&&) @ 0x11712a51 in /usr/bin/clickhouse
    8. ./obj-x86_64-linux-gnu/../src/Interpreters/ThreadStatusExt.cpp:356: DB::ThreadStatus::detachQuery(bool, bool) @ 0x1f5d5237 in /usr/bin/clickhouse
    9. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:0: void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::PipelineExecutor::executeImpl(unsigned long)::$_4>(DB::PipelineExecutor::executeImpl(unsigned long)::$_4&&)::'lambda'(), void ()> >(std::__1::__function::__policy_storage const*) @ 0x20c488e6 in /usr/bin/clickhouse
    10. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:0: ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0xad9f6cc in /usr/bin/clickhouse
    11. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:1655: void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()> >(void*) @ 0xada8264 in /usr/bin/clickhouse
    12. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so
    13. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
     (version 21.5.1.6440)

  [1]: https://clickhouse-test-reports.s3.yandex.net/22583/69296876005c0fa171c755f8b224e4d58192c402/stress_test_(address).html#fail1
2021-04-06 21:10:36 +03:00
Nikolai Kochetov
e7e17adc64
Update CHANGELOG.md 2021-04-06 19:46:47 +03:00
Alexander Kuzmenkov
f968f6bab6 return prlimit for gcc, it was not its fault 2021-04-06 19:46:24 +03:00
alesapin
95881bf2dd Fix duplicated headers 2021-04-06 19:31:24 +03:00
Alexander Kuzmenkov
d9ad06bd6d fix ubsan 2021-04-06 19:24:56 +03:00
alesapin
80d441ec95 Reset timeouts to default 2021-04-06 19:11:29 +03:00
alesapin
b4fc13de8e fix style 2021-04-06 17:02:56 +03:00
alesapin
4633afe962 Some fixes 2021-04-06 16:56:14 +03:00