Commit Graph

27 Commits

Author SHA1 Message Date
Robert Schulze
c2ee774350
Add query cache metrics to system.asynchronous_metrics
Cf. https://github.com/ClickHouse/ClickHouse/pull/52384#issuecomment-1653241216
2023-07-27 15:19:01 +00:00
Robert Schulze
bdaf82922a
Use formatAST() 2023-07-19 12:56:15 +00:00
Robert Schulze
f22452c78c
Beautify pretty-printing of the query string in SYSTEM.QUERY_CACHE.
Output of SYSTEM.QUERY_CACHE

- before this PR:

    SELECT * FROM system.query_cache

    Query id: 4989008b-b84c-4e57-bfe5-7fb551814812

    Row 1:
    ──────
    query:   	   SELECT 1 SETTINGS
    [...]

- after this PR:

    SELECT * FROM system.query_cache

    Query id: 4989008b-b84c-4e57-bfe5-7fb551814812

    Row 1:
    ──────
    query:   	   SELECT 1 SETTINGS use_query_cache = 1
    [...]
2023-07-19 09:10:26 +00:00
Robert Schulze
fae545ae5c
Merge pull request #51132 from rschu1ze/inefficient-query-cache
Fix ineffective query cache for SELECTs with subqueries
2023-06-26 16:54:54 +02:00
Robert Schulze
a9f1393862
Make the fix more minimal 2023-06-22 11:46:27 +00:00
Robert Schulze
a1c131c0da
Query Cache: Try to fix bad cast from ColumnConst to ColumnVector<char8_t>
Resolves #49445

The query cache buffers query result chunks and eventually squashes
them before insertion into the cache. Here, squashing failed because not
all chunks were of the same type. Looks like chunks of the same
underlying type (e.g. UInt8) in a query result be of mixed const, sparse
or low-cardinality type. Fix this by always materializing the data
regardless of the compression. Strange thing is that the failing query
in the stress test (*) isn't able to reproduce the bug, and I haven't
managed to trigger the issue otherwise, so no test case is added.

(*) SELECT 1 UNION ALL SELECT 1 INTERSECT SELECT 1
    E.g. here: https://s3.amazonaws.com/clickhouse-test-reports/0/18817517ed6f8849e3d979e10fbb273e0edf0eaa/stress_test__debug_/fatal_messages.txt
2023-06-22 11:46:22 +00:00
Robert Schulze
69e7c1cc82
Introduce convenience ctor for Key construction during read 2023-06-17 18:22:15 +00:00
Robert Schulze
0adc98865d
Add variable for access to found key components 2023-06-17 12:43:14 +00:00
robot-clickhouse-ci-1
d1ad3ea24e
Merge pull request #49593 from ClickHouse/rs/qc-empty-chunk
Query Cache: Safeguard against empty chunks
2023-05-07 02:20:54 +02:00
Alexey Milovidov
62d8677773
Update QueryCache.cpp 2023-05-06 18:25:43 +03:00
Robert Schulze
883e0abea1
Query Cache: Safeguard against empty chunks
Reading from the query cache is implemented using processor
"SourceFromChunks" which inherits from ISource. The latter has logic
which finishes processing (= calls .finish() on the output port +
returns Status::Finished) when the derived class returns an empty chunk.
If this empty chunk is not the last chunk, i.e. if it is followed by
non-empty chunks, the query result will be incorrect. This situation
should theoretically never occur in practice but who knows ... To be on
the safe side, writing into the query cache now rejects empty chunks and
thereby avoids this scenario.
2023-05-06 14:22:23 +00:00
Robert Schulze
0598bfd79e
Mark query cache as non-experimental 2023-05-06 13:58:40 +00:00
Robert Schulze
945a5130cf
Simplify simplify simplify 2023-05-05 16:28:51 +00:00
Robert Schulze
b565257a39
Support totals and extremes in query cache 2023-04-24 22:23:27 +00:00
Robert Schulze
38d5f12a32
Query Cache: Implement per-user quotas 2023-04-23 16:13:51 +00:00
Robert Schulze
7fe0562de5
Make the cfg parameter naming more consistent 2023-04-23 14:00:22 +00:00
Robert Schulze
a09db7f4f5
Fix squashing in query cache 2023-04-13 20:09:29 +00:00
Anton Popov
4757d76fbf fix query cache with sparse columns 2023-04-06 16:01:23 +00:00
Robert Schulze
b324d0ed5e
Merge branch 'master' into qc-compression 2023-03-29 15:09:22 +02:00
Robert Schulze
0712627c05
Query cache: Fix config naming ambiguity
Add 'max_' prefix to 'size' setting to make it more clear that the cache
consumes no space if not used.
2023-03-24 12:52:09 +00:00
Robert Schulze
d8493780b2
Query cache: Enable compression and squashing of result blocks
ClickHouse reads table data in blocks of 'max_block_size' rows. Due to
filtering, aggregation, etc., result blocks are typically much smaller
than 'max_block_size' but there are also cases where they are much
bigger. Setting 'query_cache_squash_partial_results' (enabled by
default) now controls if result blocks are squashed (if they are tiny)
or split (if they are large) into blocks of 'max_block_size' size before
insertion into the query result cache. This reduces performance of
writes into the query cache but improves compressability of cache
entries and provides more natural block granularity when query results
are later served from the query cache.

Entries in the query cache are now also compressed by default. This
reduces the overall memory consumption at the cost of slower writes into
/ reads from the query cache. To disable compression, use setting
'query_cache_compress_entries'.
2023-03-20 20:11:41 +00:00
Robert Schulze
73afae2d3f
Introduce TTLCachePolicy 2023-03-14 09:45:20 +00:00
Robert Schulze
f72286c8f3
Merge branch 'master' into qc-max_cache_size 2023-02-03 10:54:35 +01:00
Robert Schulze
216636f984
Make query cache global settings re-configurable 2023-02-01 10:35:19 +00:00
Robert Schulze
8a9493aeb9
Update QueryCache.cpp 2023-02-01 11:28:00 +01:00
Robert Schulze
f6259fbd65
Query cache: Improve logging 2023-02-01 10:24:17 +00:00
Robert Schulze
325c6bdf3d
Renaming: "Query Result Cache" --> "Query Cache"
Reasons:

- The cache will at some point store intermediate results as opposed to
  only query results. We should change the terminology now without
  having to worry about backward compat.

- Equivalent caches in MySQL (1) and Starrocks (2) are called "query
  cache".

- The new name is ca. 13.8% more catchy.

(1) https://dev.mysql.com/doc/refman/5.6/en/query-cache.html
(2) https://docs.starrocks.io/en-us/2.5/using_starrocks/query_cache
2023-01-31 09:54:34 +00:00