mirror of
https://github.com/ClickHouse/ClickHouse.git
synced 2024-10-06 00:20:51 +00:00
ClickHouse® is a real-time analytics DBMS
35231662b3
While reading from AggregatingMergeTree with
SimpleAggregateFunction(String) in primary key and
optimize_aggregation_in_order perf top shows:
Samples: 1M of event 'cycles', 4000 Hz, Event count (approx.): 287759760270 lost: 0/0 drop: 0/0
Children Self Shared Object Symbol
+ 12.64% 11.39% clickhouse [.] memcpy
+ 9.08% 0.23% [unknown] [.] 0000000000000000
+ 8.45% 8.40% clickhouse [.] ProfileEvents::increment # <-- this, and in debug it has not 0.08x overhead, but 5.8x overhead
+ 7.68% 7.67% clickhouse [.] LZ4_compress_fast_extState
+ 5.29% 5.22% clickhouse [.] DB::IAggregateFunctionHelper<DB::AggregateFunctionNullUnary<true, true> >::addFree
The reason is obvious, ProfileEvents is atomic counters (and also they
are nested):
<details>
```
Samples: 7M of event 'cycles', 4000 Hz, Event count (approx.): 450726149337
ProfileEvents::increment /usr/bin/clickhouse [Percent: local period]
Percent│
│
│
│ Disassembly of section .text:
│
│ 00000000078d8900 <ProfileEvents::increment(unsigned long, unsigned long)@@Base>:
│ ProfileEvents::increment(unsigned long, unsigned long):
0.17 │ push %rbp
0.00 │ mov %rsi,%rbp
0.04 │ push %rbx
0.20 │ mov %rdi,%rbx
0.17 │ sub $0x8,%rsp
0.26 │ → callq DB::CurrentThread::getProfileEvents
│ ProfileEvents::Counters::increment(unsigned long, unsigned long):
0.00 │ lea 0x0(,%rbx,8),%rdi
0.05 │ nop
│ unsigned long std::__1::__cxx_atomic_fetch_add<unsigned long, unsigned long>(std::__1::__cxx_atomic_base_impl<unsigned long>*, unsigned long, std::__1::memory_order):
1.02 │ mov (%rax),%rdx
97.04 │ lock add %rbp,(%rdx,%rdi,1)
│ ProfileEvents::Counters::increment(unsigned long, unsigned long):
0.21 │ mov 0x10(%rax),%rax
0.04 │ test %rax,%rax
0.00 │ → jne 78d8920 <ProfileEvents::increment(unsigned long, unsigned long)@@Base+0x20>
│ ProfileEvents::increment(unsigned long, unsigned long):
0.38 │ add $0x8,%rsp
0.00 │ pop %rbx
0.04 │ pop %rbp
0.38 │ ← retq
```
</details>
These ProfileEvents was ArenaAllocChunks (it shows ~1.5M events per
second), and the reason is that the table has
SimpleAggregateFunction(String) in PK, which requires Arena.
But most of the time there Arena wasn't even used, so avoid this cost by
re-creating Arena only if it was "used" (i.e. has new chunks).
Another possibility is to avoid populating Arena::head in ctor, but this
will make the Arena code more complex, so for now this was preferred.
Also as a long-term solution it worth looking at implementing them via
RCU (to move the extra overhead out from the write code path into read
side).
|
||
|---|---|---|
| .github | ||
| base | ||
| benchmark | ||
| cmake | ||
| contrib | ||
| debian | ||
| docker | ||
| docs | ||
| programs | ||
| src | ||
| tests | ||
| utils | ||
| website | ||
| .arcignore | ||
| .clang-format | ||
| .clang-tidy | ||
| .editorconfig | ||
| .gitattributes | ||
| .gitignore | ||
| .gitmodules | ||
| .potato.yml | ||
| .vimrc | ||
| AUTHORS | ||
| CHANGELOG.md | ||
| CMakeLists.txt | ||
| CODE_OF_CONDUCT.md | ||
| CONTRIBUTING.md | ||
| docker-compose.yml | ||
| format_sources | ||
| LICENSE | ||
| PreLoad.cmake | ||
| README.md | ||
| release | ||
| SECURITY.md | ||
| uncrustify.cfg | ||
| ya.make | ||
ClickHouse is an open-source column-oriented database management system that allows generating analytical data reports in real time.
Useful Links
- Official website has quick high-level overview of ClickHouse on main page.
- Tutorial shows how to set up and query small ClickHouse cluster.
- Documentation provides more in-depth information.
- YouTube channel has a lot of content about ClickHouse in video format.
- Slack and Telegram allow to chat with ClickHouse users in real-time.
- Blog contains various ClickHouse-related articles, as well as announcements and reports about events.
- Code Browser with syntax highlight and navigation.
- Yandex.Messenger channel shares announcements and useful links in Russian.
- Contacts can help to get your questions answered if there are any.
- You can also fill this form to meet Yandex ClickHouse team in person.
Upcoming Events
- The Second ClickHouse Meetup East (online) on October 31, 2020.
- ClickHouse for Enterprise Meetup (online in Russian) on November 10, 2020.