Default settings are not very efficient, since they do not even reuse
connections.
And when each query requires connection you can have only ~80 QPS, while
by simply enabling connection reuse (connection_auto_close=false) you
can have ~500 QPS (and by increasing connection_pool_size you can have
better QPS throughput).
So this patch allows to pass through some connection related settings
for the StorageMySQL engine, like:
- connection_pool_size=16
- connection_max_tries=3
- connection_auto_close=true
v2: remove connection_pool_default_size
v3: remove num_tries_on_connection_loss
This will reduce amount of data that may be lost at shutdown.
v2: replace dynamic_cast with IStorage::isBuffer (@kitaisreal)
v3: replace IStorage::isBuffer with IStorage::flush (@alexey-milovidov)
v4: flush() for StorageProxy/StorageTableFunction
TODO (suggested by Nikolai)
1. Build query plan fro current query (inside storage::read) up to WithMergableState
2. Check, that plan is simple enough: Aggregating - Expression - Filter - ReadFromStorage (or simplier)
3. Check, that filter is the same as filter in projection, and also expression calculates the same aggregation keys as in projection
4. Return WithMergableState if projection applies
3 will be easier to do with ActionsDAG, cause it sees all functions, and dependencies are direct (but it is possible with ExpressionActions also)
Also need to figure out how prewhere works for projections, and
row_filter_policies.
wip
* Excludes views from syncing in MaterializeMySQL
* Adds integration test for ignoring MySQL views on initial replication
* Fixes bug in initial integration test for excluding views from replicating to ClickHouse on initial replication
* Fixes bug in initial integration test for excluding views from replicating to ClickHouse on initial replication
* Replace assert with check_query
Co-authored-by: Christian Frøystad <christian@securepractice.no>
Co-authored-by: Ivan <5627721+abyss7@users.noreply.github.com>
