Only queries slower then the value of this setting will go to system.query_log,
i.e. something like slow_query_log in mysql.
v2: log_queries_min_time renamed to log_queries_min_query_duration_ms
v3: add current_database into system.query_thread_log
v4: rewrite test using current_database
v5: fix query_duration_ms in system.query_thread_log
This should significantly reduce the MemoryTracking drift, test shows
that there is 0 drift after query storm (100 queries, via http/tcp/tcp
in one session).
TL;DR;
To track memory, clickhouse creates memory tracker object for each
thread **explicitly**, but until it is not created the memory
allocations are not under account.
There should not be lot of allocations w/o memory tracker, since most of
the time it is created early enough, but even this maybe enough to
trigger some problems.
Plus sometimes it is not possible to create it, for example some 3d
party library does not allow to do this explicitly:
- for example before #15740 allocations from librdkafka threads,
- or even worse, poco threads, they don't have any routines to do this.
This won't be a problem for `MemoryTracking` metric if the deallocation
will be done from the same thread w/o memory tracker (or vise versa),
but this is not always true.
NOTE, that this will slow down per-thread allocations w/o memory
tracker, since before this patch there were no memory tracking for them
while now they will be accounted in total_memory_tracker, and for
total_memory_tracker max_untracked_memory is always reached.
But this should not be significant.
* New metrics provider (Procfs) + Refactored TasksStatsCounters
* Trivial statless test that ProcFS is provided
* Trivial perf test for ProcfsMetricsProvider
Co-authored-by: alexey-milovidov <milovidov@yandex-team.ru>
