thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 10:52:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
98,856 Commits 9,328 Branches 1,783 Tags 2.1 GiB
148995894f
Commit Graph

87 Commits

Author SHA1 Message Date
Anton Popov
148995894f add system table asynchronous_inserts_log 2022-10-03 21:35:55 +00:00
Zhiguo Zhou
223c1230b6 Optimize the lock contentions for ThreadGroupStatus::mutex 
The release of ThreadGroupStatus::finished_threads_counters_memory
via the getProfileEventsCountersAndMemoryForThreads method brings
lots of lock contentions for ThreadGroupStatus::mutex and lowers
the overall performance. This commit optimizes this performance
issue by replacing the method call with an equivalent but more
lightweight code block.
 2022-09-22 16:23:16 +08:00
Frank Chen
f17d56b528 Merge branch 'master' into tracing_context_propagation 2022-08-30 14:24:36 +08:00
Azat Khuzhin
269453a646 Avoid leaking of ThreadGroupStatus::finished_threads_counters_memory 
Cleanup them in ThreadStatus::detachQuery(), anyway they cannot be
received after by the client.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-29 11:08:50 +02:00
Frank Chen
99c37ce6c6
Merge branch 'master' into tracing_context_propagation 2022-08-25 10:07:16 +08:00
Alexey Milovidov
74e1f4dc61 Fix clang-tidy 2022-08-20 17:09:20 +02:00
Frank Chen
40c6e4c0d6 Merge remote-tracking branch 'origin/master' into tracing_context_propagation 2022-08-02 10:02:09 +08:00
Azat Khuzhin
b20407fab9 Fix NOEXCEPT_SCOPE (before it calls std::terminate and looses the exception) 
Current implementation of NOEXCEPT_SCOPE will not work, you cannot
rethrow exception outside the catch block, this will simply terminate
(via std::terminate) the program.
In other words NOEXCEPT_SCOPE macro will simply call std::terminate on
exception and will lost original exception.

But if NOEXCEPT_SCOPE will accept the code that should be runned w/o
exceptions, then it can catch exception and log it, rewrite it in this
way.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-07-14 17:16:18 +03:00
Frank Chen
2e8c530bed Move thread trace context out of ThreadStatus 2022-07-07 17:41:10 +08:00
Yatsishin Ilya
b4e62e06e7 Use native Map 2022-07-04 14:07:29 +00:00
Alexander Tokmakov
39c0219c11 fixes 2022-06-16 19:41:32 +02:00
Robert Schulze
1a0b5f33b3
More consistent use of platform macros 
cmake/target.cmake defines macros for the supported platforms, this
commit changes predefined system macros to our own macros.

__linux__ --> OS_LINUX
__APPLE__ --> OS_DARWIN
__FreeBSD__ --> OS_FREEBSD
 2022-06-10 10:22:31 +02:00
Azat Khuzhin
16b1d2bee4 Remove outdated comment from ThreadStatusExt 
Refs: https://github.com/ClickHouse/ClickHouse/pull/16121#discussion_r511500850

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-04-29 17:04:56 +03:00
Robert Schulze
b24ca8de52
Fix various clang-tidy warnings 
When I tried to add cool new clang-tidy 14 warnings, I noticed that the
current clang-tidy settings already produce a ton of warnings. This
commit addresses many of these. Almost all of them were non-critical,
i.e. C vs. C++ style casts.
 2022-04-20 10:29:05 +02:00
kssenii
36c583d0de Better version of cache on insert 2022-04-04 10:58:59 +02:00
Azat Khuzhin
cb70544dfe Move LockMemoryExceptionInThread and MemoryTrackerBlockerInThread 2022-01-10 22:39:10 +03:00
Azat Khuzhin
c1dea66907 Move TraceCollector into Interpreters 
Since now it relies on SystemLog that is in Interpreters, and it cannot
be moved into Common, since it has lots of dependencies.
 2022-01-10 22:35:42 +03:00
Azat Khuzhin
aee034a597 Use explicit template instantiation for SystemLog 
- Move some code into module part to avoid dependency from IStorage in SystemLog
- Remove extra headers from SystemLog.h
- Rewrite some code that was relying on headers that was included by SystemLog.h

v2: rebase
v3: squash move into module part with explicit template instantiation
    (to make each commit self compilable after rebase)
 2022-01-10 22:01:41 +03:00
Raúl Marín
3a7e524f9e Style 2021-12-09 18:24:28 +01:00
Raúl Marín
3fc4167c54 Rework how progress is reported in views 2021-12-09 17:08:29 +01:00
Azat Khuzhin
f0628073bb Use comments that can be parsed by clang-tidy in ThreadStatus::initQueryProfiler() 2021-11-26 01:13:55 +03:00
Azat Khuzhin
2840405c8a Fix typo in ThreadStatus::query_profiler_enabled 2021-11-26 01:13:55 +03:00
Azat Khuzhin
baf14444e6 Cleanup ProfileEvents and CurrentMetrics 2021-11-10 21:15:27 +03:00
Dmitry Novik
a33610aa32 Make read of Counters snapshot non-atomic 2021-10-12 13:46:32 +03:00
Dmitry Novik
356723427d WIP on ProfileEvents forwarding 2021-10-11 17:39:23 +03:00
Dmitry Novik
803b8623c1 Fix TCPHandler::sendProfileEvents 2021-10-11 17:29:50 +03:00
Dmitry Novik
e9b1e05461 Send profile events from all threads of current group 2021-10-11 17:29:50 +03:00
Nikolai Kochetov
340b53ef85 Remove some more streams. 2021-10-08 17:03:54 +03:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
Nikolai Kochetov
02e5e662fd Refactor a little bit. 2021-09-21 13:35:41 +03:00
Alexey Milovidov
53ca1ebca6 Be in style 2021-08-25 03:58:49 +03:00
Raúl Marín
820a6e0987 Refactor MV stage metric setup and teardown 2021-08-10 10:49:12 +02:00
Raúl Marín
db0cdd696f Dependent views: Report progress 2021-07-27 16:29:34 +02:00
Raúl Marín
0d9bf0a08e Avoid havin multiple profilers in the same thread 2021-07-27 16:29:34 +02:00
Raúl Marín
14e8029e01 Initial basic tests and change name to not include uuid 2021-07-27 16:29:33 +02:00
Raúl Marín
eb994d8f04 Fix a couple of bugs 
- Don't use threads if there is only one dependent view and parallel_view
- Reset the current_thread when destroying views ThreadStatus
 2021-07-27 16:29:33 +02:00
Raúl Marín
7c1fcc94b0 View log: Make sure to only profile events during view execution 2021-07-27 16:29:33 +02:00
Raúl Marín
e0ee7839f7 Only use threads when necessary and log written status 2021-07-27 16:29:31 +02:00
Raúl Marín
6b9ec2a62e WIP 2021-07-27 16:28:28 +02:00
Raúl Marín
efe1938437 WIP: Slow improvements 2021-07-27 16:26:29 +02:00
Raúl Marín
ea5c02a605 WIP 2021-07-27 16:26:27 +02:00
Nikolai Kochetov
afc1fe7f3d Make ContextPtr const by default. 2021-05-31 17:49:02 +03:00
Alexey Milovidov
54fb40c041 Fix OpenTelemetry 2021-05-05 01:42:14 +03:00
Azat Khuzhin
9b9e0a9bbc Block all memory tracking limits in destructors 2021-04-14 23:38:42 +03:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr (#22297) 
* Replace all Context references with std::weak_ptr

* Fix shared context captured by value

* Fix build

* Fix Context with named sessions

* Fix copy context

* Fix gcc build

* Merge with master and fix build

* Fix gcc-9 build
 2021-04-11 02:33:54 +03:00
Azat Khuzhin
b25a4e066c Lock MEMORY_LIMIT_EXCEEDED in ThreadStatus::detachQuery() 
Found with fuzzer [1]:

    <Fatal> BaseDaemon: (version 21.5.1.6440, build id: 3B097C902DDAA35688D90750552ED499DC5D10A0) (from thread 8012) Terminate called for uncaught exception:
    Code: 241, e.displayText() = DB::Exception: Memory limit (for user) exceeded: would use 153.51 MiB (attempt to allocate chunk of 4194368 bytes), maximum: 150.00 MiB, Stack trace (when copying this message, always include the lines below):

    0. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/exception:133: Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x26fdcdd9 in /usr/bin/clickhouse
    1. ./obj-x86_64-linux-gnu/../src/Common/Exception.cpp:57: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0xad0df02 in /usr/bin/clickhouse
    2. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/string:1444: DB::Exception::Exception<char const*, char const*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, char const*&&, char const*&&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&, long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&) @ 0xacc7bef in /usr/bin/clickhouse
    3. ./obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:219: MemoryTracker::alloc(long) @ 0xacc65eb in /usr/bin/clickhouse
    4. ./obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:0: MemoryTracker::alloc(long) @ 0xacc5dad in /usr/bin/clickhouse
    5. ./obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:0: MemoryTracker::alloc(long) @ 0xacc5dad in /usr/bin/clickhouse
    6. ./obj-x86_64-linux-gnu/../src/Common/AllocatorWithMemoryTracking.h:35: AllocatorWithMemoryTracking<DB::Field>::allocate(unsigned long) @ 0xad0a2fe in /usr/bin/clickhouse
    7. void std::__1::vector<DB::Field, AllocatorWithMemoryTracking<DB::Field> >::__push_back_slow_path<DB::Field>(DB::Field&&) @ 0x11712a51 in /usr/bin/clickhouse
    8. ./obj-x86_64-linux-gnu/../src/Interpreters/ThreadStatusExt.cpp:356: DB::ThreadStatus::detachQuery(bool, bool) @ 0x1f5d5237 in /usr/bin/clickhouse
    9. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:0: void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::PipelineExecutor::executeImpl(unsigned long)::$_4>(DB::PipelineExecutor::executeImpl(unsigned long)::$_4&&)::'lambda'(), void ()> >(std::__1::__function::__policy_storage const*) @ 0x20c488e6 in /usr/bin/clickhouse
    10. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:0: ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0xad9f6cc in /usr/bin/clickhouse
    11. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:1655: void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()> >(void*) @ 0xada8264 in /usr/bin/clickhouse
    12. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so
    13. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
     (version 21.5.1.6440)

  [1]: https://clickhouse-test-reports.s3.yandex.net/22583/69296876005c0fa171c755f8b224e4d58192c402/stress_test_(address).html#fail1
 2021-04-06 21:10:36 +03:00
Alexander Tokmakov
ffaa8e34a6 minor code improvements around ThreadStatus 2021-01-28 16:57:36 +03:00
Amos Bird
272311f6bd
Extensive information in system.query_log 2020-12-18 14:13:58 +08:00
Alexander Kuzmenkov
d9dd9341ce Fix the OpenTelemetry test 2020-11-30 12:11:17 +03:00
Alexander Kuzmenkov
5cd23e269d cleanup 2020-11-19 19:55:56 +03:00