Nikolai Kochetov
afc1fe7f3d
Make ContextPtr const by default.
2021-05-31 17:49:02 +03:00
Alexey Milovidov
54fb40c041
Fix OpenTelemetry
2021-05-05 01:42:14 +03:00
Azat Khuzhin
9b9e0a9bbc
Block all memory tracking limits in destructors
2021-04-14 23:38:42 +03:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr ( #22297 )
...
* Replace all Context references with std::weak_ptr
* Fix shared context captured by value
* Fix build
* Fix Context with named sessions
* Fix copy context
* Fix gcc build
* Merge with master and fix build
* Fix gcc-9 build
2021-04-11 02:33:54 +03:00
Azat Khuzhin
b25a4e066c
Lock MEMORY_LIMIT_EXCEEDED in ThreadStatus::detachQuery()
...
Found with fuzzer [1]:
<Fatal> BaseDaemon: (version 21.5.1.6440, build id: 3B097C902DDAA35688D90750552ED499DC5D10A0) (from thread 8012) Terminate called for uncaught exception:
Code: 241, e.displayText() = DB::Exception: Memory limit (for user) exceeded: would use 153.51 MiB (attempt to allocate chunk of 4194368 bytes), maximum: 150.00 MiB, Stack trace (when copying this message, always include the lines below):
0. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/exception:133: Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x26fdcdd9 in /usr/bin/clickhouse
1. ./obj-x86_64-linux-gnu/../src/Common/Exception.cpp:57: DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, bool) @ 0xad0df02 in /usr/bin/clickhouse
2. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/string:1444: DB::Exception::Exception<char const*, char const*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >(int, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, char const*&&, char const*&&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&, long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&&) @ 0xacc7bef in /usr/bin/clickhouse
3. ./obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:219: MemoryTracker::alloc(long) @ 0xacc65eb in /usr/bin/clickhouse
4. ./obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:0: MemoryTracker::alloc(long) @ 0xacc5dad in /usr/bin/clickhouse
5. ./obj-x86_64-linux-gnu/../src/Common/MemoryTracker.cpp:0: MemoryTracker::alloc(long) @ 0xacc5dad in /usr/bin/clickhouse
6. ./obj-x86_64-linux-gnu/../src/Common/AllocatorWithMemoryTracking.h:35: AllocatorWithMemoryTracking<DB::Field>::allocate(unsigned long) @ 0xad0a2fe in /usr/bin/clickhouse
7. void std::__1::vector<DB::Field, AllocatorWithMemoryTracking<DB::Field> >::__push_back_slow_path<DB::Field>(DB::Field&&) @ 0x11712a51 in /usr/bin/clickhouse
8. ./obj-x86_64-linux-gnu/../src/Interpreters/ThreadStatusExt.cpp:356: DB::ThreadStatus::detachQuery(bool, bool) @ 0x1f5d5237 in /usr/bin/clickhouse
9. ./obj-x86_64-linux-gnu/../src/Processors/Executors/PipelineExecutor.cpp:0: void std::__1::__function::__policy_invoker<void ()>::__call_impl<std::__1::__function::__default_alloc_func<ThreadFromGlobalPool::ThreadFromGlobalPool<DB::PipelineExecutor::executeImpl(unsigned long)::$_4>(DB::PipelineExecutor::executeImpl(unsigned long)::$_4&&)::'lambda'(), void ()> >(std::__1::__function::__policy_storage const*) @ 0x20c488e6 in /usr/bin/clickhouse
10. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/functional:0: ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0xad9f6cc in /usr/bin/clickhouse
11. ./obj-x86_64-linux-gnu/../contrib/libcxx/include/memory:1655: void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void ThreadPoolImpl<std::__1::thread>::scheduleImpl<void>(std::__1::function<void ()>, int, std::__1::optional<unsigned long>)::'lambda1'()> >(void*) @ 0xada8264 in /usr/bin/clickhouse
12. start_thread @ 0x9609 in /usr/lib/x86_64-linux-gnu/libpthread-2.31.so
13. __clone @ 0x122293 in /usr/lib/x86_64-linux-gnu/libc-2.31.so
(version 21.5.1.6440)
[1]: https://clickhouse-test-reports.s3.yandex.net/22583/69296876005c0fa171c755f8b224e4d58192c402/stress_test_(address).html#fail1
2021-04-06 21:10:36 +03:00
Alexander Tokmakov
ffaa8e34a6
minor code improvements around ThreadStatus
2021-01-28 16:57:36 +03:00
Amos Bird
272311f6bd
Extensive information in system.query_log
2020-12-18 14:13:58 +08:00
Alexander Kuzmenkov
d9dd9341ce
Fix the OpenTelemetry test
2020-11-30 12:11:17 +03:00
Alexander Kuzmenkov
5cd23e269d
cleanup
2020-11-19 19:55:56 +03:00
Alexander Kuzmenkov
6cb378e072
cleanup
2020-11-19 18:52:11 +03:00
Alexander Kuzmenkov
1570320e20
fixes for context hierarchy
2020-11-18 20:43:18 +03:00
Alexander Kuzmenkov
0530c40cd8
fixes
2020-11-10 08:50:32 +03:00
Alexander Kuzmenkov
26229ed231
tmp spans for threads
...
(doesn't compile because of json metadata changes)
2020-11-09 18:07:38 +03:00
Azat Khuzhin
3be8a56f5c
Add log_queries_min_query_duration_ms
...
Only queries slower then the value of this setting will go to system.query_log,
i.e. something like slow_query_log in mysql.
v2: log_queries_min_time renamed to log_queries_min_query_duration_ms
v3: add current_database into system.query_thread_log
v4: rewrite test using current_database
v5: fix query_duration_ms in system.query_thread_log
2020-11-02 21:34:54 +03:00
Azat Khuzhin
3b188921dd
Fix query_thread_log.query_duration_ms unit
...
There is a similar problem as in [1], the problem is that
getCurrentTimeNanoseconds() returns time using CLOCK_MONOTONIC.
[1]: c0e15ba348
("Fix RealTimeMicroseconds ProfileEvents")
2020-10-31 11:15:27 +03:00
Azat Khuzhin
5bab7a5bc7
Add current_database into query_thread_log
2020-10-30 21:16:10 +03:00
Mikhail Filimonov
41971e073a
Fix typos reported by codespell
2020-10-27 12:04:03 +01:00
alexey-milovidov
34b9d15b66
Update ThreadStatusExt.cpp
2020-10-24 21:34:54 +03:00
Azat Khuzhin
0cccf3049a
Fix parent memory tracker during query detaching
2020-10-23 21:07:53 +03:00
Azat Khuzhin
082f30a3a1
Introduce ThreadStatus::applyQuerySettings() to reduce copy-paste
2020-10-17 15:01:53 +03:00
Azat Khuzhin
c694bbe822
Read memory/thread related settings in ThreadStatus::attachQueryContext()
...
Previous it was read only ThreadStatus::initializeQuery() but there is
no context there yet.
Plus executeQueryImpl() calls only attachQueryContext() and nothing
more.
This is an issue only for queries via HTTP, since via TCP there is
PullingAsyncPipelineExecutor which calls attachTo() by itself.
2020-10-17 15:01:53 +03:00
Azat Khuzhin
c0e15ba348
Fix RealTimeMicroseconds ProfileEvents
...
RUsageCounters::real_time (RealTimeMicroseconds) was constructed from
CLOCK_REALTIME (or similar) in the ThreadStatus::initPerformanceCounters()
(ThreadStatusExt.cpp), while the intention is to store CLOCK_MONOTONIC
(time from boot, i.e. /proc/uptime) values there (in ThreadProfileEvents.h)
2020-10-07 21:32:36 +03:00
bharatnc
4f8266c7a1
Make trace_log test to pass
2020-09-25 10:39:37 -07:00
bharatnc
b27579e3f7
Use std::chrono instead of clock_gettime
2020-09-25 10:39:37 -07:00
bharatnc
324a06145c
EventTime - construct time in s and us from same timespec
2020-09-25 10:39:37 -07:00
bharatnc
f252523442
Query,QueryThread Logs - add event_time_microseconds field
2020-09-25 10:39:37 -07:00
bharatnc
dc765b77e7
Use std::chrono instead of clock_gettime
2020-09-15 09:37:32 -07:00
bharatnc
e4e3105014
Construct query_start_time(_microseconds) from same timespec
2020-09-14 20:04:16 -07:00
bharatnc
44e168bea7
query_start_time_microseconds field in system.query_log
...
Add a new field called `query_start_time_microseconds` that will add
`microseconds`. This new field will be of type `DateTime64`.
2020-09-04 09:42:18 -07:00
Alexey Milovidov
31cbdd1a56
system.crash_log: development
2020-07-09 07:15:45 +03:00
Alexander Kuzmenkov
af8d62bbcb
Add TLB misses perf counters.
2020-06-29 15:48:18 +03:00
Alexey Milovidov
df19db1509
Added a test for history in clickhouse-client
2020-06-07 20:29:34 +03:00
Alexander Kuzmenkov
6e3bbf83e3
thread-local perf events
2020-05-29 16:04:32 +03:00
Alexander Kuzmenkov
db84f0e892
Merge remote-tracking branch 'origin/master' into HEAD
2020-05-27 16:30:13 +03:00
Alexey Milovidov
7e1813825b
Return old names of macros
2020-05-24 01:24:01 +03:00
Alexey Milovidov
ee4ffbc332
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "\3{}", \4);/'
2020-05-23 19:47:56 +03:00
Alexey Milovidov
8d2e80a5e2
find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+"\)' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+, "[^"]+")\)/\1_FORMATTED(\2)/'
2020-05-23 19:42:39 +03:00
Andrey Skobtsov
4816abed28
Settings for measuring only particular metrics via perf events
2020-05-20 21:25:49 +03:00
Andrey Skobtsov
cae41826a8
Merge branch 'master' into perf_events_metrics
...
# Conflicts:
# src/Common/ThreadProfileEvents.cpp
# src/Common/ThreadProfileEvents.h
# src/Common/ThreadStatus.cpp
# src/Common/ThreadStatus.h
2020-05-17 00:21:22 +03:00
alexey-milovidov
d9400473f9
Update ThreadStatusExt.cpp
2020-05-03 01:20:14 +03:00
Alexey Milovidov
439777ef23
Allow to set memory_profiler_sample_probability per query
2020-05-01 22:17:56 +03:00
Alexey Milovidov
db3241ccc7
Add sampling memory profiler
2020-04-30 16:30:10 +03:00
Alexey Milovidov
293ae88e7f
Add sampling memory profiler
2020-04-30 16:25:17 +03:00
Alexey Milovidov
0a7edce036
Checkpoint
2020-04-22 09:01:33 +03:00
Andrey Skobtsov
afaa7eca51
Merge branch 'master' into perf_events_metrics
...
# Conflicts:
# dbms/src/Common/ProfileEvents.cpp
2020-04-13 23:59:48 +03:00
Ivan Lezhankin
06446b4f08
dbms/ → src/
2020-04-03 18:14:31 +03:00