thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-12 01:12:12 +00:00
Code Issues Packages Projects Releases Wiki Activity
78,527 Commits 9,297 Branches 1,782 Tags 2.1 GiB
007e323515
Commit Graph

67 Commits

Author SHA1 Message Date
Azat Khuzhin
f0628073bb Use comments that can be parsed by clang-tidy in ThreadStatus::initQueryProfiler() 2021-11-26 01:13:55 +03:00
Azat Khuzhin
2840405c8a Fix typo in ThreadStatus::query_profiler_enabled 2021-11-26 01:13:55 +03:00
Azat Khuzhin
baf14444e6 Cleanup ProfileEvents and CurrentMetrics 2021-11-10 21:15:27 +03:00
Dmitry Novik
a33610aa32 Make read of Counters snapshot non-atomic 2021-10-12 13:46:32 +03:00
Dmitry Novik
356723427d WIP on ProfileEvents forwarding 2021-10-11 17:39:23 +03:00
Dmitry Novik
803b8623c1 Fix TCPHandler::sendProfileEvents 2021-10-11 17:29:50 +03:00
Dmitry Novik
e9b1e05461 Send profile events from all threads of current group 2021-10-11 17:29:50 +03:00
Nikolai Kochetov
340b53ef85 Remove some more streams. 2021-10-08 17:03:54 +03:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
Nikolai Kochetov
02e5e662fd Refactor a little bit. 2021-09-21 13:35:41 +03:00
Alexey Milovidov
53ca1ebca6 Be in style 2021-08-25 03:58:49 +03:00
Raúl Marín
820a6e0987 Refactor MV stage metric setup and teardown 2021-08-10 10:49:12 +02:00
Raúl Marín
db0cdd696f Dependent views: Report progress 2021-07-27 16:29:34 +02:00
Raúl Marín
0d9bf0a08e Avoid havin multiple profilers in the same thread 2021-07-27 16:29:34 +02:00
Raúl Marín
14e8029e01 Initial basic tests and change name to not include uuid 2021-07-27 16:29:33 +02:00
Raúl Marín
eb994d8f04 Fix a couple of bugs 
- Don't use threads if there is only one dependent view and parallel_view
- Reset the current_thread when destroying views ThreadStatus
 2021-07-27 16:29:33 +02:00
Raúl Marín
7c1fcc94b0 View log: Make sure to only profile events during view execution 2021-07-27 16:29:33 +02:00
Raúl Marín
e0ee7839f7 Only use threads when necessary and log written status 2021-07-27 16:29:31 +02:00
Raúl Marín
6b9ec2a62e WIP 2021-07-27 16:28:28 +02:00
Raúl Marín
efe1938437 WIP: Slow improvements 2021-07-27 16:26:29 +02:00
Raúl Marín
ea5c02a605 WIP 2021-07-27 16:26:27 +02:00
Nikolai Kochetov
afc1fe7f3d Make ContextPtr const by default. 2021-05-31 17:49:02 +03:00
Alexey Milovidov
54fb40c041 Fix OpenTelemetry 2021-05-05 01:42:14 +03:00
Azat Khuzhin
9b9e0a9bbc Block all memory tracking limits in destructors 2021-04-14 23:38:42 +03:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr (#22297) 
* Replace all Context references with std::weak_ptr

* Fix shared context captured by value

* Fix build

* Fix Context with named sessions

* Fix copy context

* Fix gcc build

* Merge with master and fix build

* Fix gcc-9 build
 2021-04-11 02:33:54 +03:00
Azat Khuzhin
b25a4e066c Lock MEMORY_LIMIT_EXCEEDED in ThreadStatus::detachQuery() 
Found with fuzzer [1]:

    <Fatal> BaseDaemon: (version 21.5.1.6440, build id: 3B097C902DDAA35688D90750552ED499DC5D10A0) (from thread 8012) Terminate called for uncaught exception:
    Code: 241, e.displayText() = DB::Exception: Memory limit (for user) exceeded: would use 153.51 MiB (attempt to allocate chunk of 4194368 bytes), maximum: 150.00 MiB, Stack trace (when copying this message, always include the lines below):

    0. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/exception:133: Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x26fdcdd9 in /usr/bin/clickhouse
    1. ./obj-x86_64-linux-gnu/../src/Common/Exception.cpp:57: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0xad0df02 in /usr/bin/clickhouse
    2. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/string:1444: DB::Exception::Exception<char const*, char const*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, char const*&&, char const*&&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&, long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&) @ 0xacc7bef in /usr/bin/clickhouse
    3. ./obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:219: MemoryTracker::alloc(long) @ 0xacc65eb in /usr/bin/clickhouse
    4. ./obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:0: MemoryTracker::alloc(long) @ 0xacc5dad in /usr/bin/clickhouse
    5. ./obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:0: MemoryTracker::alloc(long) @ 0xacc5dad in /usr/bin/clickhouse
    6. ./obj-x86_64-linux-gnu/../src/Common/AllocatorWithMemoryTracking.h:35: AllocatorWithMemoryTracking<DB::Field>::allocate(unsigned long) @ 0xad0a2fe in /usr/bin/clickhouse
    7. void std::__1::vector<DB::Field, AllocatorWithMemoryTracking<DB::Field> >::__push_back_slow_path<DB::Field>(DB::Field&&) @ 0x11712a51 in /usr/bin/clickhouse
    8. ./obj-x86_64-linux-gnu/../src/Interpreters/ThreadStatusExt.cpp:356: DB::ThreadStatus::detachQuery(bool, bool) @ 0x1f5d5237 in /usr/bin/clickhouse
    9. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:0: void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::PipelineExecutor::executeImpl(unsigned long)::$_4>(DB::PipelineExecutor::executeImpl(unsigned long)::$_4&&)::'lambda'(), void ()> >(std::__1::__function::__policy_storage const*) @ 0x20c488e6 in /usr/bin/clickhouse
    10. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:0: ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0xad9f6cc in /usr/bin/clickhouse
    11. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:1655: void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()> >(void*) @ 0xada8264 in /usr/bin/clickhouse
    12. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so
    13. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
     (version 21.5.1.6440)

  [1]: https://clickhouse-test-reports.s3.yandex.net/22583/69296876005c0fa171c755f8b224e4d58192c402/stress_test_(address).html#fail1
 2021-04-06 21:10:36 +03:00
Alexander Tokmakov
ffaa8e34a6 minor code improvements around ThreadStatus 2021-01-28 16:57:36 +03:00
Amos Bird
272311f6bd
Extensive information in system.query_log 2020-12-18 14:13:58 +08:00
Alexander Kuzmenkov
d9dd9341ce Fix the OpenTelemetry test 2020-11-30 12:11:17 +03:00
Alexander Kuzmenkov
5cd23e269d cleanup 2020-11-19 19:55:56 +03:00
Alexander Kuzmenkov
6cb378e072 cleanup 2020-11-19 18:52:11 +03:00
Alexander Kuzmenkov
1570320e20 fixes for context hierarchy 2020-11-18 20:43:18 +03:00
Alexander Kuzmenkov
0530c40cd8 fixes 2020-11-10 08:50:32 +03:00
Alexander Kuzmenkov
26229ed231 tmp spans for threads 
(doesn't compile because of json metadata changes)
 2020-11-09 18:07:38 +03:00
Azat Khuzhin
3be8a56f5c Add log_queries_min_query_duration_ms 
Only queries slower then the value of this setting will go to system.query_log,
i.e. something like slow_query_log in mysql.

v2: log_queries_min_time renamed to log_queries_min_query_duration_ms
v3: add current_database into system.query_thread_log
v4: rewrite test using current_database
v5: fix query_duration_ms in system.query_thread_log
 2020-11-02 21:34:54 +03:00
Azat Khuzhin
3b188921dd Fix query_thread_log.query_duration_ms unit 
There is a similar problem as in [1], the problem is that
getCurrentTimeNanoseconds() returns time using CLOCK_MONOTONIC.

  [1]: c0e15ba348 ("Fix RealTimeMicroseconds ProfileEvents")
 2020-10-31 11:15:27 +03:00
Azat Khuzhin
5bab7a5bc7 Add current_database into query_thread_log 2020-10-30 21:16:10 +03:00
Mikhail Filimonov
41971e073a
Fix typos reported by codespell 2020-10-27 12:04:03 +01:00
alexey-milovidov
34b9d15b66
Update ThreadStatusExt.cpp 2020-10-24 21:34:54 +03:00
Azat Khuzhin
0cccf3049a Fix parent memory tracker during query detaching 2020-10-23 21:07:53 +03:00
Azat Khuzhin
082f30a3a1 Introduce ThreadStatus::applyQuerySettings() to reduce copy-paste 2020-10-17 15:01:53 +03:00
Azat Khuzhin
c694bbe822 Read memory/thread related settings in ThreadStatus::attachQueryContext() 
Previous it was read only ThreadStatus::initializeQuery() but there is
no context there yet.
Plus executeQueryImpl() calls only attachQueryContext() and nothing
more.

This is an issue only for queries via HTTP, since via TCP there is
PullingAsyncPipelineExecutor which calls attachTo() by itself.
 2020-10-17 15:01:53 +03:00
Azat Khuzhin
c0e15ba348 Fix RealTimeMicroseconds ProfileEvents 
RUsageCounters::real_time (RealTimeMicroseconds) was constructed from
CLOCK_REALTIME (or similar) in the ThreadStatus::initPerformanceCounters()
(ThreadStatusExt.cpp), while the intention is to store CLOCK_MONOTONIC
(time from boot, i.e. /proc/uptime) values there (in ThreadProfileEvents.h)
 2020-10-07 21:32:36 +03:00
bharatnc
4f8266c7a1 Make trace_log test to pass 2020-09-25 10:39:37 -07:00
bharatnc
b27579e3f7 Use std::chrono instead of clock_gettime 2020-09-25 10:39:37 -07:00
bharatnc
324a06145c EventTime - construct time in s and us from same timespec 2020-09-25 10:39:37 -07:00
bharatnc
f252523442 Query,QueryThread Logs - add event_time_microseconds field 2020-09-25 10:39:37 -07:00
bharatnc
dc765b77e7 Use std::chrono instead of clock_gettime 2020-09-15 09:37:32 -07:00
bharatnc
e4e3105014 Construct query_start_time(_microseconds) from same timespec 2020-09-14 20:04:16 -07:00
bharatnc
44e168bea7 query_start_time_microseconds field in system.query_log 
Add a new field called `query_start_time_microseconds`  that will add
`microseconds`. This new field will be of type `DateTime64`.
 2020-09-04 09:42:18 -07:00