Commit Graph

94 Commits

Author SHA1 Message Date
kssenii
32d004017b Merge origin 2022-05-06 13:54:51 +02:00
kssenii
8f99046100 Add come asynchronous metrics 2022-05-03 20:02:19 +02:00
Nikita Taranov
30f2a942c5
Predict size of hash table for GROUP BY (#33439)
* use AggregationMethod ctor with reserve

* add new settings

* add HashTablesStatistics

* support queries with limit

* support distributed and with external aggregation

* add new profile events

* add some tests

* add perf test

* export cache stats through AsynchronousMetrics

* rm redundant trace

* fix style

* fix 02122_parallel_formatting test

* review fixes

* fix 02122_parallel_formatting test

* apply also to two-level HTs

* try simpler strategy

* increase max_size_to_preallocate_for_aggregation for experiment

* fixes

* Revert "increase max_size_to_preallocate_for_aggregation for experiment"

This reverts commit 6cf6f75704.

* fix test

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-03-30 22:47:51 +02:00
taiyang-li
b4174b0bef merge master and fix conflicts 2022-03-08 11:39:25 +08:00
Alexandre Snarskii
93de1b1c99 Implement MemoryStatisticsOS for FreeBSD 2022-02-25 14:05:00 +03:00
Azat Khuzhin
6a2abd3a68 jemalloc: remove jemalloc 4 support
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-02-17 21:25:58 +03:00
taiyang-li
3de8bde7ce Merge remote-tracking branch 'origin/master' into rocksdb_metacache 2022-01-28 09:58:52 +08:00
Raúl Marín
ac1665cdcb AsynchronousMetrics: Ignore inaccessible sensors 2022-01-14 15:52:08 +01:00
taiyang-li
90a92dd14a fix stateless test 2022-01-11 11:09:52 +08:00
taiyang-li
3d409a4526 add metrics MergeTreeMetadataCacheSize 2022-01-10 12:20:39 +08:00
Kevin Michel
ffc1fca296
Start/stop servers when listen_host/*_port changes
This allows starting and stopping separately each protocol server
without restarting ClickHouse.

This also allows adding or removing `listen_host` entries, which
start and stops servers for all enabled ports.

When stopping a server, the listening socket is immediately closed
(and available for another server).

Protocols with persistent connections try to wait for any currently
running query to finish before closing the connection, but idle
connection are closed quickly (depending on how often the protocol
is polled).

An extra ProfileEvent is added, `MainConfigLoads`, it is
incremented every time the configuration is reloaded. This helps
when trying to assess whether the new configuration was applied.
2021-12-24 08:26:02 +01:00
Alexandre Snarskii
c36c64a9a0 more corrections for disk size calculations 2021-11-09 12:28:59 +03:00
Alexey Milovidov
8b4a6a2416 Remove cruft 2021-10-28 02:10:39 +03:00
Maksim Kita
7beaeb8b13
Merge branch 'master' into indexcache 2021-10-15 12:59:09 +03:00
Alexey Milovidov
7e084304fb Skip "no data" exception when reading thermal sensors 2021-09-11 10:03:06 +03:00
Amos Bird
a2256b1307
Dedicated Mark/Uncompressed cache for skip indices 2021-08-23 15:27:31 +08:00
Azat Khuzhin
67ebcef978 Reopen sensors (/sys/class/thermal) 2021-08-11 10:16:44 +03:00
Azat Khuzhin
3f91f61c3c Reopen EDAC files (/sys/devices/system/edac) 2021-08-11 10:16:44 +03:00
Azat Khuzhin
eed5052a86 Reopen hwmon sensors on error (/sys/class/hwmon)
Sensors maybe recreated (i.e. on module load/unload, or suspend/resume
cycle), so descriptors should be reopened.
2021-08-11 10:16:44 +03:00
Azat Khuzhin
ec4e8ca594 Firstly write current exception and then reopen block devices 2021-08-11 10:16:44 +03:00
Maksim Kita
2054ecb1ce
Merge pull request #27031 from excitoon-favorites/asyncmetrics
Improved logging of `hwmon` sensor errors in `AsynchronousMetrics`
2021-08-10 14:00:36 +03:00
alexey-milovidov
fad30f6949
Merge pull request #27361 from myrrc/bugfix/error-parsing-proc-meminfo
Fixing reading of /proc/meminfo when kB suffix is not present
2021-08-07 18:38:15 +03:00
Mike Kot
e2cee2576d Fixing reading of /proc/meminfo 2021-08-06 22:33:21 +03:00
Azat Khuzhin
55b3d21601 Fix polling of /sys/block in case of block devices reopened on error
Found on CI [1].

  [1]: https://clickhouse-test-reports.s3.yandex.net/27248/4f3b80ff33c846465983aa2bc9ae9490e1118b15/fuzzer_ubsan/report.html#fail1
2021-08-06 00:34:51 +03:00
Azat Khuzhin
8a7c9e1124 Fix polling of /sys/block
- rescan /sys/block on errors (disk had been replaced and you will got
  ENOENT)
- rescan /sys/block periodically (each 5m), for newly added devices
  (i.e. new disk to RAID array can be added)
2021-08-04 22:41:53 +03:00
Vladimir Chebotarev
b32c024571
Update AsynchronousMetrics.cpp 2021-07-30 20:03:57 +03:00
Vladimir Chebotarev
3c3fbf9a53 Improved logging of some errors in AsynchronousMetrics. 2021-07-30 17:36:28 +03:00
Alexey Milovidov
924319dc32 Less logging in AsynchronousMetrics 2021-07-16 10:32:02 +03:00
Alexey Milovidov
822cc0fec3 Lowered amount of syscalls in AsynchronousMetrics 2021-07-12 04:12:34 +03:00
Alexey Milovidov
1ce40d6629 Fix bad code (default function argument) 2021-07-11 04:01:23 +03:00
Alexey Milovidov
ee1b3696a2 Fix error in AsynchronousMetrics 2021-07-08 06:15:30 +03:00
Raúl Marín
d87607b160 AsynchronousMetrics: Don't assume temperature is always positive 2021-07-07 11:09:38 +02:00
Alexey Milovidov
94a024c7c8 Fix warning 2021-07-06 01:12:49 +03:00
Alexey Milovidov
ac1baaf6d4 Comments 2021-07-05 22:37:12 +03:00
Alexey Milovidov
1fde0e13cc A check just in case 2021-07-05 22:32:33 +03:00
Alexey Milovidov
22ba93789b Fix warning 2021-07-05 22:29:36 +03:00
Alexey Milovidov
783ec5116a Skip errors 2021-07-05 05:55:11 +03:00
Alexey Milovidov
c3b975ecb1 Add network statistics 2021-07-05 05:47:33 +03:00
Alexey Milovidov
70bdc166e0 Add block device statistics 2021-07-05 05:24:36 +03:00
Alexey Milovidov
dda4d0e34e Add EDAC 2021-07-05 04:18:12 +03:00
Alexey Milovidov
d7eca6ec35 Better temperatures 2021-07-05 03:56:14 +03:00
Alexey Milovidov
347ceac618 Better temperatures 2021-07-05 03:51:22 +03:00
Alexey Milovidov
5d5d2efd48 More temperatures 2021-07-05 03:40:28 +03:00
Alexey Milovidov
d437ec2e1d Merge branch 'master' into system-metrics 2021-07-05 03:03:07 +03:00
Alexey Milovidov
295afd29a1 Fix style 2021-07-05 03:02:23 +03:00
Alexey Milovidov
c059d0a0ee More metrics 2021-07-05 01:41:09 +03:00
Alexey Milovidov
0f8ea9b8f6 More metrics 2021-07-05 01:33:32 +03:00
Alexey Milovidov
b5840210c1 Adjustments 2021-07-05 00:54:46 +03:00
Alexey Milovidov
3a10d3802b Development 2021-07-05 00:33:00 +03:00
Alexey Milovidov
08aca329bd Development 2021-07-05 00:22:58 +03:00