Commit Graph

71 Commits

Author SHA1 Message Date
Alexey Milovidov
5b16681311 Merge branch 'master' into try-about-on-current-thread-join 2024-06-04 18:11:34 +02:00
Azat Khuzhin
4ad88d04b4 Move initGlobalProfiler() into ThreadStatusExt.cpp and fix examples
To avoid undefined references in examples:

    May 11 01:58:40 ld.lld-17: error: undefined symbol: DB::Context::hasTraceCollector() const
    May 11 01:58:40 >>> referenced by ThreadStatus.cpp:132 (/build/src/Common/ThreadStatus.cpp:132)
    May 11 01:58:40 >>>               ThreadStatus.cpp.o:(DB::ThreadStatus::initGlobalProfiler(unsigned long, unsigned long)) in archive src/libclickhouse_common_iod.a
    May 11 01:58:40 clang++-17: error: linker command failed with exit code 1 (use -v to see invocation)

Move it firstly into ThreadStatusExt and then do not try to use it from
the ThreadPool.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-05-13 11:05:36 +02:00
Alexander Tokmakov
d8e97b51bf Merge branch 'master' into cannot_allocate_thread 2024-04-10 21:21:42 +02:00
Alexander Tokmakov
c53b20a770 fix 2024-04-03 20:57:12 +02:00
Alexander Tokmakov
98ac8031e0 add fault injection 2024-04-03 19:49:23 +02:00
alesapin
f5c5146153 [RFC] Try to add global Real and CPU trace collector 2024-04-02 18:04:53 +02:00
Azat Khuzhin
cc296a1b8a Move some code of ThreadPool into module part
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-27 17:06:17 +01:00
Azat Khuzhin
55d5a3affa Correctly wait background threads
There are some places that make it possible to trigger use-after-free
from threads because some global variable had been destroyed before, for
example some logger.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-09 17:54:16 +01:00
Alexey Milovidov
d56cbda185 Add metrics for the number of queued jobs, which is useful for the IO thread pool 2023-11-18 19:07:59 +01:00
Alexey Milovidov
9ce1663e6a
Merge branch 'master' into try-about-on-current-thread-join 2023-09-30 07:40:01 +03:00
Yakov Olkhovskiy
6e769237a7 added setting 'enable_job_stack_trace', default is off 2023-06-28 01:39:06 +00:00
Yakov Olkhovskiy
f11e1c25ee added to fatal errors handler, added comments 2023-06-24 17:24:54 +00:00
Yakov Olkhovskiy
be17a81792 recursively collect call stacks on job scheduling and add all stack traces to exceptions printout 2023-06-24 00:12:31 +00:00
Sergei Trifonov
0d1f2e297b
Unify priorities: lower value means higher priority (#50205) 2023-05-26 15:55:30 +02:00
Alexey Milovidov
5a44dc26e7 Fixes for clang-17 2023-05-13 02:57:31 +02:00
Michael Kolupaev
e133633359 Parallel decoding with one row group per thread 2023-04-17 04:58:32 +00:00
Azat Khuzhin
011480924a Use forward declaration of ThreadPool
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-04-07 11:25:35 +02:00
Azat Khuzhin
f38a7aeabe ThreadPool metrics introspection
There are lots of thread pools and simple local-vs-global is not enough
already, it is good to know which one in particular uses threads.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-03-29 10:46:59 +02:00
Vitaly Baranov
096d0132af More corrections. 2023-03-21 21:52:59 +01:00
Vitaly Baranov
85d38493f6 Improve ThreadPool
ThreadPool::setMaxThreads() can now start new threads (if there are scheduled jobs in the queue for them);
ThreadPool::setMaxFreeThreads() can now finish free threads.
2023-03-21 21:52:59 +01:00
Anton Popov
31e8b692f4 fix typo 2023-01-31 01:13:23 +00:00
Anton Popov
839cd614fb fix memory leak in azure sdk 2023-01-31 01:01:10 +00:00
Nikolai Kochetov
aee7bb0d91
Merge pull request #42577 from ClickHouse/clean-thread-id
Clear thread::id when ThreadFromGlobalPool exits.
2022-10-24 12:03:28 +02:00
Nikolai Kochetov
06d46ca51b Clear thread::id when ThreadFromGlobalPool exits. 2022-10-22 13:28:22 +00:00
Azat Khuzhin
4e76629aaf Fixes for -Wshorten-64-to-32
- lots of static_cast
- add safe_cast
- types adjustments
  - config
  - IStorage::read/watch
  - ...
- some TODO's (to convert types in future)

P.S. That was quite a journey...

v2: fixes after rebase
v3: fix conflicts after #42308 merged
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-21 13:25:19 +02:00
Nikolai Kochetov
2ddc55a92a Try abort on current thread join. 2022-10-21 10:05:25 +00:00
Nikolai Kochetov
ad3b324382
Merge branch 'master' into try-about-on-current-thread-join 2022-10-20 21:51:41 +02:00
Nikolai Kochetov
d09a5e8fd7 Revert "Attempt to fix abort from parallel parsing (#42496)"
This reverts commit 4d703b792c.
2022-10-20 21:46:26 +02:00
Nikolai Kochetov
ecc23867db Try abort on current thread join. 2022-10-20 19:43:42 +00:00
Nikita Mikhaylov
4d703b792c
Attempt to fix abort from parallel parsing (#42496) 2022-10-20 17:13:18 +02:00
Frank Chen
20191932df Fix style
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2022-09-13 00:41:05 +08:00
Frank Chen
7e1f2901da Fix
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2022-09-13 00:06:17 +08:00
Frank Chen
ebaa24ecae Fix flaky tests
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2022-09-12 22:15:30 +08:00
Frank Chen
d9a516288c Resolve comments 2022-09-01 11:56:10 +08:00
Frank Chen
922a0dee74 Fix style
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2022-08-30 15:14:12 +08:00
Frank Chen
83cbdef3c6 Fix context re-initialization for ThreadPool 2022-08-30 12:26:23 +08:00
Frank Chen
cd19366b44 Move classes into DB::OpenTelemetry namespace 2022-08-24 16:41:40 +08:00
Frank Chen
57dde8c250 Clean up header inclusion
Signed-off-by: Frank Chen <frank.chen021@outlook.com>
2022-08-02 15:43:19 +08:00
Frank Chen
40c6e4c0d6 Merge remote-tracking branch 'origin/master' into tracing_context_propagation 2022-08-02 10:02:09 +08:00
Azat Khuzhin
68789895a4 ThreadPool: incapsulate Poco::Event and std:🧵:id into State struct
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-13 15:51:49 +03:00
Azat Khuzhin
7ffa15b94b ThreadPool: fix thread_id assignment
As found by @KochetovNicolai before this patch, lambda in
ThreadFromGlobalPool() ctor assigns value only to a copy of the
thread_id value, and so check in joinable() had been working
incorrectly, fix this by changing the value not the shared_ptr itself.

Also it is not safe to assign thread_id w/o atomics, since this can be
racy, so wrap id with std::atomic<>

Fixes: #28431
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-13 15:42:46 +03:00
Azat Khuzhin
25eb82f120 ThreadPool: do not use joinable() internally
joinable() should be used only outside, since internally it is enough to
know `state` to know that something is wrong.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-13 15:42:46 +03:00
Azat Khuzhin
3473b80077 ThreadPool: add some clarification comments
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-13 15:42:46 +03:00
Frank Chen
32931857f0 Propagate tracing context across threads 2022-07-07 17:41:43 +08:00
Nikita Mikhaylov
36bdee0499 Deleted settings 2022-04-22 13:56:16 +00:00
Maksim Kita
e7772ed434 Fix clang-tidy warnings in Common folder 2022-03-14 18:17:35 +00:00
Tomáš Hromada
445b0ba7cc
Expose all GlobalThreadPool config params (#31285) 2021-11-12 16:24:47 +03:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
alexey-milovidov
28911a959f
Merge pull request #28431 from azat/thread-pool-joinable
[RFC] Do not allow ThreadFromGlobalPool::join() from the spawned/occupated thread
2021-09-11 05:44:56 +03:00
Nikita Mikhaylov
ea0fbf81af Renaming 2021-09-06 12:01:16 +00:00