Commit Graph

5 Commits

Author SHA1 Message Date
alexey-milovidov
1d5a4cfa32
Revert "[RFC] Fix memory tracking with min_bytes_to_use_mmap_io" 2021-04-19 01:30:53 +03:00
Azat Khuzhin
6064f3ef1d Fix memory tracking with min_bytes_to_use_mmap_io
With min_bytes_to_use_mmap_io=1 memory regions for mmap() will not be
accounted. But this creates issues, since if you will run simple
"SELECT * FROM big_enough_table" it will mmap lot's of files and the
process RSS will grow at some point, and before the kernel will
understand that it needs to unmap physical pages the clickhouse-server
will sync total_memory_tracker with RSS of the process and the
allocations will fail by per-server memory limits.

Also note, that this does not address all the issues with mmap()
regions.

v2: move track/untrack into the MMapReadBufferFromFileWithCache to avoid
data-race [1]

  [1]: https://clickhouse-test-reports.s3.yandex.net/23211/6acf6cb9b22cee7db8d1aa523a6208f63e792f7a/functional_stateful_tests_(thread).html#fail1
2021-04-17 10:37:38 +03:00
Alexey Milovidov
2a8ac01cdb Rename as suggested by Kita 2021-03-28 22:24:28 +03:00
Alexey Milovidov
50f712e198 Integrate mmap cache to the infrastructure 2021-03-28 04:10:30 +03:00
Alexey Milovidov
bcdf3dce36 Add cache for mmapped files 2021-03-27 02:22:51 +03:00