Commit Graph

114 Commits

Author SHA1 Message Date
Azat Khuzhin
3e627e2861 Add profile events for fsync
The following new provile events had been added:

- FileSync - Number of times the F_FULLFSYNC/fsync/fdatasync function was called for files.
- DirectorySync - Number of times the F_FULLFSYNC/fsync/fdatasync function was called for directories.
- FileSyncElapsedMicroseconds - Total time spent waiting for F_FULLFSYNC/fsync/fdatasync syscall for files.
- DirectorySyncElapsedMicroseconds - Total time spent waiting for F_FULLFSYNC/fsync/fdatasync syscall for directories.

v2: rewrite test to sh with retries
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-31 23:19:30 +03:00
Antonio Andelic
37f799550b
Merge pull request #38072 from lingpeng0314/master
Add Keeper related monitoring data
2022-07-10 09:39:57 +02:00
Alexander Tokmakov
cfe0e434e5
Revert "Add support for io_uring read method" 2022-06-24 12:19:41 +03:00
Alexey Milovidov
812ab9bd6b
Merge pull request #36103 from sauliusvl/uring
Add support for io_uring read method
2022-06-24 00:34:29 +03:00
Saulius Valatka
160c95a6b8 unpoisson memory read by io_uring, handle interrupted syscalls, do not throw from monitor thread 2022-06-21 13:20:21 +03:00
Saulius Valatka
7b5e13a9eb implement pending request queue to prevent CQ overflows, more error checking, reduce size of io_uring 2022-06-21 13:17:56 +03:00
Saulius Valatka
0dc7854af1 add initial io_uring support 2022-06-21 09:59:46 +03:00
lingpeng0314
c18dd55400 Resolve comments
- refine metric description
2022-06-21 10:30:19 +08:00
lingpeng0314
f06b19bdbf Add keeper monitoring data 2022-06-20 11:32:18 +08:00
Antonio Andelic
367df12626 Add sync support to client 2022-06-14 09:50:30 +00:00
mergify[bot]
ba49c6bb46
Merge branch 'master' into memory-overcommit-improvement 2022-05-31 13:17:06 +00:00
Anton Popov
16e839ac71 add profile events for introspection of part types 2022-05-25 14:54:49 +00:00
Dmitry Novik
e9187ec4b7 Overcommit: update defaults, exception message and add ProfileEvent 2022-05-23 14:35:09 +00:00
kssenii
7700d51ae4 Update profile events descriptions 2022-05-12 15:02:07 +02:00
Kseniia Sumarokova
5ae28f6fd8
Update ProfileEvents.cpp 2022-05-12 12:28:48 +02:00
kssenii
c5e84950d6 Update test, add descriptions to profile events 2022-05-11 14:57:51 +02:00
kssenii
f4b45a5805 Better profile events 2022-05-10 16:00:53 +02:00
kssenii
2f0f76369a Better 2022-05-09 21:25:26 +02:00
Anton Popov
fa1feb238e
Merge pull request #36853 from qoega/add-other-time-microsecons
Add Other Query Time Microseconds Profile Event
2022-05-05 13:58:13 +02:00
Ilya Yatsishin
6c3bbb4933 Add Other Time Microseconds Profile Event 2022-05-02 17:13:57 +02:00
Alexey Milovidov
d08da0f63c Everything related to S3 metrics was a trash 2022-04-24 03:46:35 +02:00
Alexey Milovidov
6de4f2bbff Everything related to S3 metrics was a trash 2022-04-24 03:42:18 +02:00
Kseniia Sumarokova
a383d9ee6a
Merge pull request #35916 from filimonov/kafka_metrics
Add some metrics to engine Kafka
2022-04-06 11:58:17 +02:00
Mikhail Filimonov
53c7376e37
Add some metrics to engine Kafka 2022-04-04 20:25:53 +02:00
Maksim Kita
2a8e479277 ExecutableUserDefinedFunction prevent function execution during query analysis 2022-04-04 15:56:01 +02:00
Nikita Taranov
30f2a942c5
Predict size of hash table for GROUP BY (#33439)
* use AggregationMethod ctor with reserve

* add new settings

* add HashTablesStatistics

* support queries with limit

* support distributed and with external aggregation

* add new profile events

* add some tests

* add perf test

* export cache stats through AsynchronousMetrics

* rm redundant trace

* fix style

* fix 02122_parallel_formatting test

* review fixes

* fix 02122_parallel_formatting test

* apply also to two-level HTs

* try simpler strategy

* increase max_size_to_preallocate_for_aggregation for experiment

* fixes

* Revert "increase max_size_to_preallocate_for_aggregation for experiment"

This reverts commit 6cf6f75704.

* fix test

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-03-30 22:47:51 +02:00
taiyang-li
834ac3a1b9 Merge remote-tracking branch 'ck/master' into rocksdb_metacache 2022-03-29 18:32:06 +08:00
Alexey Milovidov
6fbf5bd9ad More correct 2022-03-28 02:11:27 +02:00
Alexey Milovidov
bb35184da1 Add metric about size of async INSERTs 2022-03-28 02:04:19 +02:00
taiyang-li
8dbf1c60e7 merge master and fix conflict 2022-03-23 11:36:50 +08:00
kssenii
47f94120da Merge master 2022-02-16 14:43:28 +01:00
李扬
daa27d0bda
Merge branch 'master' into rocksdb_metacache 2022-02-12 07:50:12 -06:00
taiyang-li
b6132d490f merge master and solve conflict 2022-02-08 15:24:59 +08:00
kssenii
2e58733750 Merge master 2022-02-07 15:23:10 +01:00
mergify[bot]
dd947f964c
Merge branch 'master' into mv_cacheable_scalars 2022-02-07 10:07:26 +00:00
Anton Popov
5bb1b3ce77 support async inserts in clickhouse-client for queries with inlined data 2022-02-02 20:59:37 +03:00
Raúl Marín
4b5ab80e3b Better scalar cache handling
- Fixes global CTE scalar cache.
- Adds MVs back (views dependent on the source are cached locally and others globally
2022-01-26 17:36:45 +01:00
taiyang-li
73def8b483 merge master and solve conflict 2022-01-24 11:01:43 +08:00
Kseniia Sumarokova
2c43e4eb57
Merge branch 'master' into local-cache-for-remote-fs 2022-01-22 22:53:55 +03:00
kssenii
a566099759 Local cache for remote filesystem 2022-01-19 00:42:11 +03:00
taiyang-li
1e102bc1b2 merge master 2022-01-01 09:01:06 +08:00
taiyang-li
aa97634d9b merge master 2021-12-28 18:38:50 +08:00
taiyang-li
2a1fe52b9f rename symbols && fix uts && add setting use_metadata_cache 2021-12-28 18:06:13 +08:00
李扬
91334ecf50
Merge branch 'master' into rocksdb_metacache 2021-12-27 23:20:41 -06:00
Vitaly Baranov
f80a3e3247
Merge pull request #30549 from aiven/kmichel-server-reload
Start/stop servers when `listen_host`/`*_port` changes
2021-12-27 14:51:00 +03:00
taiyang-li
9036b18c2f merge master 2021-12-27 15:12:48 +08:00
taiyang-li
e7401d2a5e wrap rocksdb metacache related code with USE_ROCKSDB 2021-12-27 11:50:59 +08:00
taiyang-li
94a7a09598 Merge remote-tracking branch 'origin/master' into rocksdb_metacache 2021-12-27 10:14:24 +08:00
Kevin Michel
ffc1fca296
Start/stop servers when listen_host/*_port changes
This allows starting and stopping separately each protocol server
without restarting ClickHouse.

This also allows adding or removing `listen_host` entries, which
start and stops servers for all enabled ports.

When stopping a server, the listening socket is immediately closed
(and available for another server).

Protocols with persistent connections try to wait for any currently
running query to finish before closing the connection, but idle
connection are closed quickly (depending on how often the protocol
is polled).

An extra ProfileEvent is added, `MainConfigLoads`, it is
incremented every time the configuration is reloaded. This helps
when trying to assess whether the new configuration was applied.
2021-12-24 08:26:02 +01:00
liyang
37ba8004ff Speep up mergetree starting up process 2021-12-18 16:39:59 +08:00