thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 02:41:59 +00:00
Code Issues Packages Projects Releases Wiki Activity
108,434 Commits 9,328 Branches 1,783 Tags 2.1 GiB
da326dda03
Commit Graph

56 Commits

Author SHA1 Message Date
Azat Khuzhin
2a8f116c18 Forward declaration of ConcurrentBoundedQueue in ThreadStatus 
ThreadStatus is the header that recomplies almost all ClickHouse
modules.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-01-21 16:02:09 +01:00
Raúl Marín
227af9a074 Optimizations around ThreadStatus 2022-10-13 21:49:21 +02:00
Frank Chen
99c37ce6c6
Merge branch 'master' into tracing_context_propagation 2022-08-25 10:07:16 +08:00
Alexey Milovidov
74e1f4dc61 Fix clang-tidy 2022-08-20 17:09:20 +02:00
Frank Chen
a3b6ad2a65
Merge branch 'master' into tracing_context_propagation 2022-08-18 20:59:07 +08:00
Raúl Marín
11a274e990 Clean up constinit usage and add a comment 2022-08-11 13:27:53 +02:00
Raúl Marín
d7a545e30d Try to optimize CurrentMemoryTracker alloc and free 2022-08-10 22:05:09 +02:00
Frank Chen
40c6e4c0d6 Merge remote-tracking branch 'origin/master' into tracing_context_propagation 2022-08-02 10:02:09 +08:00
Alexander Tokmakov
fdd2db4ace
Move check for denied allocations (#38858) 
* move check for denied allocations

* remove include

* make hardening softer
 2022-07-07 14:24:36 +02:00
Frank Chen
2e8c530bed Move thread trace context out of ThreadStatus 2022-07-07 17:41:10 +08:00
Alexey Milovidov
25cc406201
Update src/Common/ThreadStatus.cpp 
Co-authored-by: Dmitry Novik <n0vik@clickhouse.com>
 2022-06-26 01:29:06 +03:00
Alexey Milovidov
e33f236d50 Slight improvement 2022-06-25 03:59:01 +02:00
Alexey Milovidov
b89f01438e Lower mutex scope 2022-06-25 03:05:55 +02:00
Anton Popov
e911900054 remove last mentions of data streams 2022-05-09 19:15:24 +00:00
Azat Khuzhin
aee034a597 Use explicit template instantiation for SystemLog 
- Move some code into module part to avoid dependency from IStorage in SystemLog
- Remove extra headers from SystemLog.h
- Rewrite some code that was relying on headers that was included by SystemLog.h

v2: rebase
v3: squash move into module part with explicit template instantiation
    (to make each commit self compilable after rebase)
 2022-01-10 22:01:41 +03:00
Azat Khuzhin
6aebc3e94c Do not loose ProfileEvents in case of thread destroyed before 
v2: drop std::move and add copy ctor for ProfileEvents::Counter::Snapshot
v2: remove std::move
 2021-12-19 23:43:05 +03:00
Azat Khuzhin
909ce68b5a Do not suppress exception in ThreadStatus::~ThreadStatus() 2021-12-19 23:43:05 +03:00
Alexey Milovidov
8b4a6a2416 Remove cruft 2021-10-28 02:10:39 +03:00
Nikita Mikhaylov
10487f75b2
Merge pull request #28364 from ClickHouse/profile-events-wip 
Forward profile events to client
 2021-10-12 13:16:01 +03:00
Dmitry Novik
9071a7151e Fix communication & race conditions 2021-10-11 17:39:23 +03:00
Dmitry Novik
356723427d WIP on ProfileEvents forwarding 2021-10-11 17:39:23 +03:00
Azat Khuzhin
9515430517 Fix compilation with glibc 2.34 (MINSIGSTKSZ defined as sysconf(_SC_SIGSTKSZ)) 
In glibc 2.34 MINSIGSTKSZ had been defined to sysconf(_SC_SIGSTKSZ) [1].

  [1]: https://sourceware.org/git/?p=glibc.git;a=commit;h=6c57d320484988e87e446e2e60ce42816bf51d53
 2021-10-07 00:18:56 +03:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
Sergei Semin
f74350c148 fix special build 2021-09-12 02:49:31 +03:00
Raúl Marín
820a6e0987 Refactor MV stage metric setup and teardown 2021-08-10 10:49:12 +02:00
Raúl Marín
fc23e212c2 ThreadStatus: Don't change current thread if it's not this 2021-07-27 16:29:33 +02:00
Azat Khuzhin
3c76986311 Fix alternative stack under osx (MINSIGSTKSZ is 32K) 
Fixes: #25632
 2021-06-24 00:33:33 +03:00
alexey-milovidov
9a4125dd7c
Update ThreadStatus.cpp 2021-06-21 12:06:41 +03:00
Azat Khuzhin
12eadc2b5e Add a guard page for alternative signal handling stack 2021-06-20 20:08:50 +03:00
Azat Khuzhin
4dcbf3a5c5 Replace __pthread_get_minstack() with a const 2021-06-20 14:03:04 +03:00
Azat Khuzhin
9a3a1397e6 Fix alternative stack for SIGSEGV handling 
Previous stack size of 4096 wasn't enough, and this lead to that on
SIGSEGV signal handler may overwrite some TLS data (like in the
following example [1]) and in real binary "current_thread" was
overwritten, and this leads to lack of query-id on SIGSEGV, like here
[2].

  [1]: https://gist.github.com/azat/2ee360fd4f2828d363b0926431afacc6
  [2]: https://clickhouse-test-reports.s3.yandex.net/24411/79563953201ce6249a8fd49e22be3902ca4ee43a/fuzzer_ubsan/report.html#fail1

Fix this by using __pthread_get_minstack() that takes TLS block into
account. Yes it is private, but let's try (like rust tried).

This patch will also decrease TLS, since now the per-thread stack will
be allocated in the heap, not on the stack of each thread.

Refs: https://sourceware.org/legacy-ml/libc-alpha/2012-06/msg00335.html
Refs: https://maskray.me/blog/2021-02-14-all-about-thread-local-storage
Refs: https://sourceware.org/bugzilla/show_bug.cgi?id=11787
 2021-06-19 22:47:01 +03:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr (#22297) 
* Replace all Context references with std::weak_ptr

* Fix shared context captured by value

* Fix build

* Fix Context with named sessions

* Fix copy context

* Fix gcc build

* Merge with master and fix build

* Fix gcc-9 build
 2021-04-11 02:33:54 +03:00
Alexander Tokmakov
004517009e fix 2021-01-29 18:13:27 +03:00
Alexander Tokmakov
ffaa8e34a6 minor code improvements around ThreadStatus 2021-01-28 16:57:36 +03:00
Azat Khuzhin
46c3266c6a Respect MINSIGSTKSZ for alternative stack to fix under aarch64 
And acording to sigaltstack(2) alignment is not required:

    When a signal handler is invoked on the alternate stack, the kernel
    automatically aligns the address given in ss.ss_sp to a suitable address
    boundary for the underlying hardware architecture.

Fixes: #18785
 2021-01-07 18:28:23 +03:00
Alexey Milovidov
be884a89f8 Minor fixes for min/sim hash 2020-12-29 13:16:43 +03:00
Alexey Milovidov
7f280bef37 Merge branch 'master' into sigaltstack 2020-12-17 05:09:10 +03:00
Alexander Kuzmenkov
b16c5a1748 Merge remote-tracking branch 'origin/master' into HEAD 2020-11-17 22:54:54 +03:00
Alexander Tokmakov
b94cc5c4e5 remove more stringstreams 2020-11-10 21:22:26 +03:00
Alexander Kuzmenkov
26229ed231 tmp spans for threads 
(doesn't compile because of json metadata changes)
 2020-11-09 18:07:38 +03:00
Alexey Milovidov
fd84d16387 Fix "server failed to start" error 2020-11-07 03:14:53 +03:00
alexey-milovidov
a591d1f32a
Update ThreadStatus.cpp 2020-10-26 22:46:25 +03:00
Alexey Milovidov
0ba2fa8105 Fix "Arcadia" 2020-10-26 11:56:30 +03:00
Alexey Milovidov
88e846b36d Fix build on Mac OS 2020-10-26 07:42:22 +03:00
Alexey Milovidov
068235c2d7 Disable alt stack for sanitizers 2020-10-26 05:18:36 +03:00
Alexey Milovidov
4e2c0ba226 Provide diagnostics on stack overflow 2020-10-25 03:10:05 +03:00
Azat Khuzhin
72d7b6117e Use total_memory_tracker when there is no other MemoryTracker object. 
This should significantly reduce the MemoryTracking drift, test shows
that there is 0 drift after query storm (100 queries, via http/tcp/tcp
in one session).

TL;DR;

To track memory, clickhouse creates memory tracker object for each
thread **explicitly**, but until it is not created the memory
allocations are not under account.
There should not be lot of allocations w/o memory tracker, since most of
the time it is created early enough, but even this maybe enough to
trigger some problems.

Plus sometimes it is not possible to create it, for example some 3d
party library does not allow to do this explicitly:
- for example before #15740 allocations from librdkafka threads,
- or even worse, poco threads, they don't have any routines to do this.
This won't be a problem for `MemoryTracking` metric if the deallocation
will be done from the same thread w/o memory tracker (or vise versa),
but this is not always true.

NOTE, that this will slow down per-thread allocations w/o memory
tracker, since before this patch there were no memory tracking for them
while now they will be accounted in total_memory_tracker, and for
total_memory_tracker max_untracked_memory is always reached.
But this should not be significant.
 2020-10-23 21:07:52 +03:00
Alexey Milovidov
31cbdd1a56 system.crash_log: development 2020-07-09 07:15:45 +03:00
Alexey Milovidov
7ba25bd799 Make it work 2020-06-20 14:17:15 +03:00
Alexander Kuzmenkov
6e3bbf83e3 thread-local perf events 2020-05-29 16:04:32 +03:00