Commit Graph

5524 Commits

Author SHA1 Message Date
Nikolai Kochetov
62ef40394a
Merge pull request #14856 from ClickHouse/query-plan-for-creating-sets
Query plan for creating sets
2020-09-22 16:19:20 +03:00
Alexander Kuzmenkov
478c7309d4
Merge pull request #14843 from ClickHouse/aku/global-pool
Exception on double init of global thread pool
2020-09-22 13:44:39 +03:00
Artem Zuikov
2280482530
Keep LowCardinality in MergeJoin left key result type (#15088) 2020-09-22 13:16:22 +03:00
Nikita Mikhaylov
123f389f1a
Merge pull request #14689 from ClickHouse/aku/broken-perf
Add context to errors in ExpressionAction
2020-09-21 23:33:42 +03:00
alexey-milovidov
e29c4c3cc4
Merge pull request #15024 from ClickHouse/fix_renameat2_einval_on_macos
Fix EINVAL in renameat2 on MacOS
2020-09-21 21:08:17 +03:00
alexey-milovidov
c240f464d2
Merge pull request #15061 from azat/UAF-from-ThreadPool-jobs
Fix possible UAF from jobs in the ThreadPool on shutdown
2020-09-21 20:49:46 +03:00
alexey-milovidov
10d3a1d730
Update ThreadPool.cpp 2020-09-21 20:49:15 +03:00
alesapin
263d570f7e
Merge pull request #15076 from ClickHouse/fix_modifying_ast_create_in_database_memory
Fix data race when using DatabaseMemory::getCreateTableQueryImpl(...)
2020-09-21 20:49:06 +03:00
Nikolai Kochetov
c77a6967a3 Remove commented code. 2020-09-21 19:22:23 +03:00
Nikolai Kochetov
334289fd57
Merge pull request #15082 from ClickHouse/try-improve-array-has-built-time
Update SourceVisitor for GatherUtils.
2020-09-21 17:08:14 +03:00
Nikolai Kochetov
0f3eab1e48
Merge pull request #15084 from amosbird/lef
Don't hide interface symbols
2020-09-21 17:07:52 +03:00
Amos Bird
741912cafd
Don't hide interface symbols 2020-09-21 21:03:17 +08:00
tavplubix
fd2cc343e6
Update renameat2.cpp 2020-09-21 15:37:34 +03:00
tavplubix
7f629c8185
Update DatabaseMemory.cpp 2020-09-21 14:12:45 +03:00
Nikolai Kochetov
efa493fbcb Update SourceVisitor for GatherUtils. 2020-09-21 13:29:31 +03:00
tavplubix
450c1a5709
Update renameat2.cpp 2020-09-21 13:19:02 +03:00
Alexander Tokmakov
ef85ce90c3 fix build 2020-09-21 13:17:33 +03:00
Nikolai Kochetov
54c1f04b38 Review fixes. 2020-09-21 11:36:12 +03:00
alexey-milovidov
3e0509af69
Merge pull request #13573 from amosbird/rms
specific ReplicatedMergeTree settings
2020-09-21 01:36:33 +03:00
Azat Khuzhin
30f96dc525 Fix possible UAF from jobs in the ThreadPool on shutdown
ThreadPoolImpl<>::worker signaling that job is done while still storing
std::function<> object, and this can lead to problems on shutdown, since
in in this cast std::function<> can refers to some global/static object
that had been already destroyed (typical example is Logger).

I believe that this is exactly what TSAN reports about (decoded
manually, since llvm-symbolizer does not work in the test env):

<details>

```

- 2020-09-20 17:44:43   Write of size 8 at 0x7b1000008f78 by main thread (mutexes: write M1432):
    operator delete(void*, unsigned long)
    ??:0:0

    Poco::Logger::~Logger()
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Logger.cpp:39:1
    non-virtual thunk to Poco::Logger::~Logger()
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Logger.cpp:0:0

    Poco::RefCountedObject::release() const
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/RefCountedObject.h:82:24
    Poco::Logger::shutdown()
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Logger.cpp:346:16

    Poco::AutoLoggerShutdown::~AutoLoggerShutdown()
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Logger.cpp:459:4

    cxa_at_exit_wrapper(void*)
    ??:0:0

```

```
- 2020-09-20 17:44:43   Previous atomic read of size 4 at 0x7b1000008f78 by thread T116:

    __tsan_atomic32_load
    ??:0:0

    int std::__1::__cxx_atomic_load<int>(std::__1::__cxx_atomic_base_impl<int> const*, std::__1::memory_order)
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/atomic:970:12
    std::__1::__atomic_base<int, false>::load(std::__1::memory_order) const
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/atomic:1487:17
    std::__1::__atomic_base<int, false>::operator int() const
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/atomic:1491:53
    Poco::Logger::is(int) const
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/Logger.h:1264:9
    MemoryTracker::logPeakMemoryUsage() const
    /build/obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:59:5

    MemoryTracker::~MemoryTracker()
    /build/obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:46:13

    DB::ThreadGroupStatus::~ThreadGroupStatus()
    /build/obj-x86_64-linux-gnu/../src/Common/ThreadStatus.h:51:7
    std::__1::__shared_ptr_emplace<DB::ThreadGroupStatus, std::__1::allocator<DB::ThreadGroupStatus> >::__on_zero_shared()
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3602:23

    std::__1::__shared_count::__release_shared()
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3440:9
    std::__1::__shared_weak_count::__release_shared()
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:3482:27
    std::__1::shared_ptr<DB::ThreadGroupStatus>::~shared_ptr()
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:4207:19
    DB::PipelineExecutor::executeImpl(unsigned long)::$_4::~$_4()
    /build/obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:720:34
    ThreadFromGlobalPool::ThreadFromGlobalPool<DB::PipelineExecutor::executeImpl(unsigned long)::$_4>(DB::PipelineExecutor::executeImpl(unsigned long)::$_4&&)::'lambda'()::~()
    /build/obj-x86_64-linux-gnu/../src/Common/ThreadPool.h:161:54
    std::__1::__compressed_pair_elem<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::PipelineExecutor::executeImpl(unsigned long)::$_4>(DB::PipelineExecutor::executeImpl(unsigned long)::$_4&&)::'lambda'(), 0, false>::~__compressed_pair_elem()
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:2188:8
    std::__1::__function::__alloc_func<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::PipelineExecutor::executeImpl(unsigned long)::$_4>(DB::PipelineExecutor::executeImpl(unsigned long)::$_4&&)::'lambda'(), std::__1::allocator<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::Pip>
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:1559:38
    std::__1::__function::__func<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::PipelineExecutor::executeImpl(unsigned long)::$_4>(DB::PipelineExecutor::executeImpl(unsigned long)::$_4&&)::'lambda'(), std::__1::allocator<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::PipelineE>
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:1706:10

    std::__1::__function::__value_func<void ()>::~__value_func()
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:1828:19
    std::__1::function<void ()>::~function()
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:2460:43
    ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>)
    /build/obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:268:5

    void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()::operator()() const
    /build/obj-x86_64-linux-gnu/../src/Common/ThreadPool.cpp:116:73
    decltype(std::__1::forward<void>(fp)(std::__1::forward<void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()>(fp0)...)) std::__1::__invoke<void ThreadPoolImpl<std::__1::thread>::scheduleI>
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/type_traits:3519:1
    void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'(>
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/thread:273:5
    void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned lon>
    /build/obj-x86_64-linux-gnu/../contrib/libcxx/include/thread:284:5

    __tsan_thread_start_func
    ??:0:0
```

```
- 2020-09-20 17:44:43   Mutex M1432 (0x0000181213a8) created at:
    pthread_mutex_init
    ??:0:0

    Poco::MutexImpl::MutexImpl()
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Mutex_POSIX.cpp:64:6

    Poco::Mutex::Mutex()
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Mutex.cpp:34:8

    __cxx_global_var_init
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Logger.cpp:28:15
    _GLOBAL__sub_I_Logger.cpp
    /build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/src/Logger.cpp:0:0

    __libc_csu_init
    ??:0:0
```

</details>
2020-09-20 23:46:28 +03:00
alexey-milovidov
8630d1baf4
Merge pull request #15033 from amosbird/fixstringkey
Fix incorrect key condition of fixed strings.
2020-09-20 19:51:24 +03:00
alexey-milovidov
85483f8532
Merge pull request #14853 from ClickHouse/akz/optimized_index_binary_search
Optimized marks selection algorithm for continuous marks ranges
2020-09-20 19:48:45 +03:00
alexey-milovidov
f7cae4a57d
Merge pull request #14975 from ClickHouse/fix_merge_tree_settings_check
Fix check for merge tree settings sanity on server startup
2020-09-20 19:43:40 +03:00
alexey-milovidov
55eb533fda
Merge pull request #15034 from amosbird/fff
Improve performance of grouping by single LowCardinality(FixedString)
2020-09-20 19:42:38 +03:00
alexey-milovidov
7b35a6a59a
Merge pull request #15028 from ClickHouse/fix-bar
Fix buffer overflow in "bar" function
2020-09-20 19:04:15 +03:00
alexey-milovidov
3f5d7843f6
Merge pull request #15029 from ClickHouse/cast-single-argument
Proper exception message for wrong number of arguments of CAST
2020-09-20 19:03:47 +03:00
alexey-milovidov
068e9f372e
Merge pull request #15030 from ClickHouse/base64-unpoison
Maybe fix MSan report in base64
2020-09-20 19:03:28 +03:00
Vitaly Baranov
46ab2f76cd
Merge pull request #14919 from vitlibar/add-login-function-to-access-storage
Add login() function to IAccessStorage
2020-09-20 16:58:22 +03:00
alesapin
5ffee8808a
Alter remove column properties and TTLs (#14742) 2020-09-20 16:27:33 +03:00
Amos Bird
8ca4d6634e
better fixed string group by support 2020-09-20 15:20:09 +08:00
Amos Bird
fcee786320
Fix incorrect key condition of fixed strings. 2020-09-20 11:53:29 +08:00
alexey-milovidov
1474b80ecc
Merge pull request #15008 from ClickHouse/clang-tidy-11-without-enabling
Fix terribly wrong code
2020-09-20 02:00:54 +03:00
Alexey Milovidov
0029c5512d Maybe fix MSan report in base64 2020-09-19 22:43:26 +03:00
Alexey Milovidov
5a47e64685 Proper exception message for wrong number of arguments of CAST 2020-09-19 22:34:24 +03:00
Alexey Milovidov
dae3b40a55 Fix buffer overflow in "bar" function 2020-09-19 22:15:16 +03:00
alexey-milovidov
c185db46f9
Merge pull request #15023 from ClickHouse/remove-opencl
Remove OpenCL
2020-09-19 21:31:40 +03:00
alexey-milovidov
7ae4d01744
Update convertMySQLDataType.cpp 2020-09-19 19:31:08 +03:00
Alexey Milovidov
9fa49a6843 Fix clang-tidy 2020-09-19 19:29:15 +03:00
Alexey Milovidov
c7a98132ac Slightly better code 2020-09-19 18:13:51 +03:00
Alexey Milovidov
b87ecc2ec1 Even more 2020-09-19 18:09:12 +03:00
Alexey Milovidov
c304032707 Even more 2020-09-19 18:02:11 +03:00
Alexey Milovidov
772eccf03d Move code to .cpp 2020-09-19 17:55:08 +03:00
Alexey Milovidov
059646743b Remove OpenCL 2020-09-19 17:20:00 +03:00
Alexander Tokmakov
46aa307d0a fix EINVAL in renameat2 on MacOS 2020-09-19 17:19:06 +03:00
alexey-milovidov
2cab30ba8d
Merge pull request #14521 from hexiaoting/dev_like
Add new feature: SHOW DATABASES like 'xxx'
2020-09-19 16:34:37 +03:00
alexey-milovidov
acbdc51460
Merge pull request #14771 from amosbird/withsub
First attempt of supporting CTE
2020-09-19 16:25:34 +03:00
alexey-milovidov
eb9ee723c3
Merge pull request #11913 from ClickHouse/compact-parts-by-default
Enable compact parts by default for small parts
2020-09-19 15:31:15 +03:00
Alexey Milovidov
2264622f69 Fix clang-tidy 2020-09-19 15:24:43 +03:00
tavplubix
6e0bdaf46d
Merge pull request #14535 from zhang2014/fix/datetime
ISSUES-4006 support decimal data type for MaterializedMySQL
2020-09-19 14:05:32 +03:00
alexey-milovidov
65517da62b
Merge pull request #14495 from nikitamikhaylov/update-permutation-bugfix-3
updatePermumation with Nullable
2020-09-19 13:53:55 +03:00