Commit Graph

48340 Commits

Author SHA1 Message Date
Alexey Milovidov
4cd7de175e Adjustments 2020-09-22 06:17:09 +03:00
Alexey Milovidov
2f09e36778 Allow scale 1000 in benchmark 2020-09-22 06:13:28 +03:00
Alexey Milovidov
923724c0f4 Improvements 2020-09-22 06:06:37 +03:00
alexey-milovidov
05b5901595
Merge pull request #15115 from ClickHouse/hardware-benchmark-as-single-script
Added a script to perform hardware benchmark in a single command
2020-09-22 05:44:20 +03:00
alexey-milovidov
e5ea51f5df
Merge pull request #15113 from ClickHouse/add-results-from-seodo
Add benchmark results from seo.do
2020-09-22 05:43:26 +03:00
Alexey Milovidov
ddfd512241 Adjustments 2020-09-22 05:22:53 +03:00
Alexey Milovidov
7db544e241 Added a script to perform hardware benchmark in a single command 2020-09-22 05:19:11 +03:00
Alexey Milovidov
f9402c95fa Add benchmark results from seo.do 2020-09-22 04:17:40 +03:00
alexey-milovidov
df122b90b4
Update performance-test.md 2020-09-22 02:07:12 +03:00
alexey-milovidov
3c1a679612
Update performance-test.md 2020-09-22 02:06:19 +03:00
alexey-milovidov
e180a037fc
Merge pull request #15092 from ClickHouse/distributed-local-shard-enum-conversion-test
Added a test for Enum conversion in Distributed table if one shard is local
2020-09-22 01:40:49 +03:00
alexey-milovidov
40ff1cf912
Update adopters.md 2020-09-22 00:45:53 +03:00
Nikita Mikhaylov
123f389f1a
Merge pull request #14689 from ClickHouse/aku/broken-perf
Add context to errors in ExpressionAction
2020-09-21 23:33:42 +03:00
Alexander Kuzmenkov
4696e22799
Update perf.py 2020-09-21 22:08:41 +03:00
Alexey Milovidov
6e596b9c7d Added a test for Enum conversion in Distributed table if one shard is local 2020-09-21 21:15:41 +03:00
alexey-milovidov
e29c4c3cc4
Merge pull request #15024 from ClickHouse/fix_renameat2_einval_on_macos
Fix EINVAL in renameat2 on MacOS
2020-09-21 21:08:17 +03:00
alexey-milovidov
5e56f14d5f
Merge pull request #15027 from ClickHouse/perf-test-separate-profiler
Separate profile runs in performance test
2020-09-21 21:07:14 +03:00
alexey-milovidov
c240f464d2
Merge pull request #15061 from azat/UAF-from-ThreadPool-jobs
Fix possible UAF from jobs in the ThreadPool on shutdown
2020-09-21 20:49:46 +03:00
alexey-milovidov
10d3a1d730
Update ThreadPool.cpp 2020-09-21 20:49:15 +03:00
alesapin
263d570f7e
Merge pull request #15076 from ClickHouse/fix_modifying_ast_create_in_database_memory
Fix data race when using DatabaseMemory::getCreateTableQueryImpl(...)
2020-09-21 20:49:06 +03:00
alexey-milovidov
083bc291d2
Merge pull request #15067 from ClickHouse/dependabot/pip/docs/tools/mkdocs-macros-plugin-0.4.13
Bump mkdocs-macros-plugin from 0.4.9 to 0.4.13 in /docs/tools
2020-09-21 20:46:40 +03:00
alexey-milovidov
2f35aa673e
Merge pull request #15060 from ClickHouse/control-heavy-builds
Control heavy translation units
2020-09-21 20:44:23 +03:00
alexey-milovidov
7744eccd0a
Merge pull request #15064 from den-crane/tests/CTE
more tests for CTE
2020-09-21 20:41:53 +03:00
Alexander Kuzmenkov
1eb5137bbf
Merge pull request #15062 from ClickHouse/add-test-4476
Add a test for #4476
2020-09-21 20:41:25 +03:00
Nikolai Kochetov
c77a6967a3 Remove commented code. 2020-09-21 19:22:23 +03:00
Nikolai Kochetov
8a3549ebd3 Merge branch 'master' into control-heavy-builds 2020-09-21 17:09:33 +03:00
Nikolai Kochetov
334289fd57
Merge pull request #15082 from ClickHouse/try-improve-array-has-built-time
Update SourceVisitor for GatherUtils.
2020-09-21 17:08:14 +03:00
Nikolai Kochetov
0f3eab1e48
Merge pull request #15084 from amosbird/lef
Don't hide interface symbols
2020-09-21 17:07:52 +03:00
alesapin
084b40affe
Merge pull request #15069 from ClickHouse/fix_test_images
Use llvm-symbolizer-11 in all test images.
2020-09-21 17:02:27 +03:00
Amos Bird
741912cafd
Don't hide interface symbols 2020-09-21 21:03:17 +08:00
tavplubix
fd2cc343e6
Update renameat2.cpp 2020-09-21 15:37:34 +03:00
Alexander Kuzmenkov
679c5278e4 reword the diagnostic about short queries in perf test 2020-09-21 15:14:47 +03:00
alesapin
76189de495
Merge pull request #15070 from Artemeey/patch-1
Описание move_factor
2020-09-21 14:21:14 +03:00
tavplubix
7f629c8185
Update DatabaseMemory.cpp 2020-09-21 14:12:45 +03:00
Nikolai Kochetov
efa493fbcb Update SourceVisitor for GatherUtils. 2020-09-21 13:29:31 +03:00
tavplubix
450c1a5709
Update renameat2.cpp 2020-09-21 13:19:02 +03:00
Alexander Tokmakov
ef85ce90c3 fix build 2020-09-21 13:17:33 +03:00
alesapin
1bd76c820c Back to 19.10 2020-09-21 12:05:10 +03:00
Artemeey
a7aa714a28
Update mergetree.md
описание move_factor
2020-09-21 11:14:37 +03:00
Artemeey
aa39059890
Update storage_policies.md
Описание move_factor
2020-09-21 11:10:27 +03:00
alesapin
fd469f1266 Remove redundant file 2020-09-21 10:41:23 +03:00
alesapin
27613f8346 Fix symbolizers path in test images 2020-09-21 10:38:11 +03:00
dependabot-preview[bot]
0119eb9c0d
Bump mkdocs-macros-plugin from 0.4.9 to 0.4.13 in /docs/tools
Bumps [mkdocs-macros-plugin](https://github.com/fralau/mkdocs_macros_plugin) from 0.4.9 to 0.4.13.
- [Release notes](https://github.com/fralau/mkdocs_macros_plugin/releases)
- [Commits](https://github.com/fralau/mkdocs_macros_plugin/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
2020-09-21 06:17:03 +00:00
Denis Zhuravlev
aa25df9d45 order by added for quieries with limit 2020-09-20 19:43:19 -03:00
alexey-milovidov
3e0509af69
Merge pull request #13573 from amosbird/rms
specific ReplicatedMergeTree settings
2020-09-21 01:36:33 +03:00
Denis Zhuravlev
fe0f8117af more tests for CTE 2020-09-20 19:22:57 -03:00
Alexey Milovidov
6e1bd79d51 Add a test for #4476 2020-09-21 00:25:47 +03:00
Alexey Milovidov
035d7cb47f Enable control for heavy translation units 2020-09-20 23:46:32 +03:00
Azat Khuzhin
30f96dc525 Fix possible UAF from jobs in the ThreadPool on shutdown
ThreadPoolImpl<>::worker signaling that job is done while still storing
std::function<> object, and this can lead to problems on shutdown, since
in in this cast std::function<> can refers to some global/static object
that had been already destroyed (typical example is Logger).

I believe that this is exactly what TSAN reports about (decoded
manually, since llvm-symbolizer does not work in the test env):

<details>

```

- 2020-09-20 17:44:43   Write of size 8 at 0x7b1000008f78 by main thread (mutexes: write M1432):
    operator delete(void*, unsigned long)
    ??:0:0

    Poco::Logger::~Logger()
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Logger.cpp:39:1
    non-virtual thunk to Poco::Logger::~Logger()
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Logger.cpp:0:0

    Poco::RefCountedObject::release() const
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/RefCountedObject.h:82:24
    Poco::Logger::shutdown()
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Logger.cpp:346:16

    Poco::AutoLoggerShutdown::~AutoLoggerShutdown()
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Logger.cpp:459:4

    cxa_at_exit_wrapper(void*)
    ??:0:0

```

```
- 2020-09-20 17:44:43   Previous atomic read of size 4 at 0x7b1000008f78 by thread T116:

    __tsan_atomic32_load
    ??:0:0

    int std::__1::__cxx_atomic_load<int>(std::__1::__cxx_atomic_base_impl<int> const*, std::__1::memory_order)
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/atomic:970:12
    std::__1::__atomic_base<int, false>::load(std::__1::memory_order) const
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/atomic:1487:17
    std::__1::__atomic_base<int, false>::operator int() const
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/atomic:1491:53
    Poco::Logger::is(int) const
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/Logger.h:1264:9
    MemoryTracker::logPeakMemoryUsage() const
    /build/obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:59:5

    MemoryTracker::~MemoryTracker()
    /build/obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:46:13

    DB::ThreadGroupStatus::~ThreadGroupStatus()
    /build/obj-x86_64-linux-gnu/../src/Common/ThreadStatus.h:51:7
    std::__1::__shared_ptr_emplace<DB::ThreadGroupStatus, std::__1::allocator<DB::ThreadGroupStatus> >::__on_zero_shared()
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3602:23

    std::__1::__shared_count::__release_shared()
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3440:9
    std::__1::__shared_weak_count::__release_shared()
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3482:27
    std::__1::shared_ptr<DB::ThreadGroupStatus>::~shared_ptr()
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:4207:19
    DB::PipelineExecutor::executeImpl(unsigned long)::$_4::~$_4()
    /build/obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:720:34
    ThreadFromGlobalPool::ThreadFromGlobalPool<DB::PipelineExecutor::executeImpl(unsigned long)::$_4>(DB::PipelineExecutor::executeImpl(unsigned long)::$_4&&)::'lambda'()::~()
    /build/obj-x86_64-linux-gnu/../src/Common/ThreadPool.h:161:54
    std::__1::__compressed_pair_elem<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::PipelineExecutor::executeImpl(unsigned long)::$_4>(DB::PipelineExecutor::executeImpl(unsigned long)::$_4&&)::'lambda'(), 0, false>::~__compressed_pair_elem()
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2188:8
    std::__1::__function::__alloc_func<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::PipelineExecutor::executeImpl(unsigned long)::$_4>(DB::PipelineExecutor::executeImpl(unsigned long)::$_4&&)::'lambda'(), std::__1::allocator<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::Pip>
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:1559:38
    std::__1::__function::__func<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::PipelineExecutor::executeImpl(unsigned long)::$_4>(DB::PipelineExecutor::executeImpl(unsigned long)::$_4&&)::'lambda'(), std::__1::allocator<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::PipelineE>
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:1706:10

    std::__1::__function::__value_func<void ()>::~__value_func()
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:1828:19
    std::__1::function<void ()>::~function()
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2460:43
    ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>)
    /build/obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:268:5

    void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()::operator()() const
    /build/obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:116:73
    decltype(std::__1::forward<void>(fp)(std::__1::forward<void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(fp0)...)) std::__1::__invoke<void ThreadPoolImpl<std::__1::thread>::scheduleI>
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3519:1
    void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'(>
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/thread:273:5
    void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned lon>
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/thread:284:5

    __tsan_thread_start_func
    ??:0:0
```

```
- 2020-09-20 17:44:43   Mutex M1432 (0x0000181213a8) created at:
    pthread_mutex_init
    ??:0:0

    Poco::MutexImpl::MutexImpl()
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Mutex_POSIX.cpp:64:6

    Poco::Mutex::Mutex()
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Mutex.cpp:34:8

    __cxx_global_var_init
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Logger.cpp:28:15
    _GLOBAL__sub_I_Logger.cpp
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Logger.cpp:0:0

    __libc_csu_init
    ??:0:0
```

</details>
2020-09-20 23:46:28 +03:00
Alexey Milovidov
5301de7656 Control heavy translation units 2020-09-20 23:43:28 +03:00