thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-17 21:24:28 +00:00
Code Issues Packages Projects Releases Wiki Activity
72,343 Commits 9,139 Branches 1,774 Tags 2.1 GiB
0e267c50b4
Commit Graph

682 Commits

Author SHA1 Message Date
kssenii
0a3589524e File movement 2021-05-29 00:57:53 +03:00
kssenii
2a631aaf08 Final fixes 2021-05-29 00:34:44 +03:00
Arslan G
fbc5734302 Clean up code and fix tests 2021-05-28 20:00:26 +03:00
kssenii
a78c9e63e7 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-05-23 12:07:26 +00:00
kssenii
866b29fb5a Return list fds with Poco, more canonical 2021-05-23 10:56:13 +03:00
kssenii
1c43b333ea Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-22 21:26:48 +03:00
kssenii
3e6d3d4ecb Postgres schema for insert 2021-05-22 08:50:23 +00:00
alexey-milovidov
a25f1910cc
Merge pull request #24320 from ClickHouse/remove-auto-array 
Remove AutoArray
 2021-05-22 02:19:59 +03:00
kssenii
096d117f68 Review fixes 2021-05-22 00:12:46 +03:00
Alexey Milovidov
ea30bb7abf Improve implementation 2021-05-21 04:17:18 +03:00
Azat Khuzhin
4f41ebcae3 Add settings (for connections) for MySQL storage engine 
Default settings are not very efficient, since they do not even reuse
connections.
And when each query requires connection you can have only ~80 QPS, while
by simply enabling connection reuse (connection_auto_close=false) you
can have ~500 QPS (and by increasing connection_pool_size you can have
better QPS throughput).

So this patch allows to pass through some connection related settings
for the StorageMySQL engine, like:
- connection_pool_size=16
- connection_max_tries=3
- connection_auto_close=true

v2: remove connection_pool_default_size
v3: remove num_tries_on_connection_loss
 2021-05-20 22:44:52 +03:00
kssenii
3dee003f9b Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-20 19:20:09 +03:00
kssenii
5f083b779d A little better, fix checks 2021-05-18 14:25:19 +03:00
kssenii
9b8df78fdd Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-17 17:42:05 +03:00
Arslan G
dde9e30043 Merge branch 'master' into add-sqlite-support 2021-05-17 14:38:24 +03:00
Arslan G
ea51840f85 Init SQLite integration 2021-05-17 14:02:35 +03:00
kssenii
1ee77eae89 Better way to distinguish between user query and replication thread query 2021-05-16 21:35:37 +00:00
kssenii
8a4711b716 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-05-16 20:55:18 +00:00
Azat Khuzhin
23981ffd0e Introduce IStorage::flushAndShutdown() 2021-05-16 13:23:53 +03:00
Azat Khuzhin
f20799f683 Destroy Buffer() tables before others (within one database) 
This will reduce amount of data that may be lost at shutdown.

v2: replace dynamic_cast with IStorage::isBuffer (@kitaisreal)
v3: replace IStorage::isBuffer with IStorage::flush (@alexey-milovidov)
v4: flush() for StorageProxy/StorageTableFunction
 2021-05-16 13:23:53 +03:00
feng lv
c6f8ab9826 fix 2021-05-13 02:05:53 +00:00
kssenii
7c81103ea3 Add replication identifier, allow proper access to _version and _sign columns from nested 2021-05-12 22:48:42 +00:00
kssenii
407db17e2e Remove setting, use native transaction 2021-05-12 22:33:15 +00:00
kssenii
0955e5f0be Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-05-12 15:11:15 +00:00
kssenii
0527f0ea33 Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-12 16:54:18 +03:00
Nikolai Kochetov
a1ba67e029
Merge pull request #20202 from amosbird/projection 
Projections
 2021-05-12 09:38:07 +03:00
kssenii
81797393f1 fs::last_write_time, fs::space 2021-05-12 09:19:11 +03:00
Maksim Kita
72d46beca0
Merge pull request #23979 from azat/dict-preallocate 
Reimplement preallocate for hashed/sparse_hashed dictionaries
 2021-05-11 20:15:46 +03:00
feng lv
9035e3a643 fix test and conflict 2021-05-11 16:22:24 +00:00
kssenii
393b026f2e Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-11 16:49:23 +03:00
Amos Bird
264cff6415
Projections 
TODO (suggested by Nikolai)

1. Build query plan fro current query (inside storage::read) up to WithMergableState
2. Check, that plan is simple enough: Aggregating - Expression - Filter - ReadFromStorage (or simplier)
3. Check, that filter is the same as filter in projection, and also expression calculates the same aggregation keys as in projection
4. Return WithMergableState if projection applies

3 will be easier to do with ActionsDAG, cause it sees all functions, and dependencies are direct (but it is possible with ExpressionActions also)

Also need to figure out how prewhere works for projections, and
row_filter_policies.

wip
 2021-05-11 18:12:23 +08:00
kssenii
626e87bae5 Some fixes 2021-05-10 13:51:05 +00:00
kssenii
ae1191d0c0 Get rid of metadata file, rely only on lsn 2021-05-10 09:10:02 +00:00
kssenii
4ac023e511 Small fixes 2021-05-10 08:45:52 +00:00
kssenii
993823bcc4 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-05-09 21:26:41 +00:00
kssenii
ab1a05a1f4 Poco::Path to fs::path, less concatination 2021-05-09 14:59:49 +03:00
Azat Khuzhin
80f4059deb Drop unused getDictionaryConfigurationFromAST.h 2021-05-09 14:25:45 +03:00
kssenii
e26cb4bee0 Fix style check, fix yandex check 2021-05-09 13:45:50 +03:00
Alexey Milovidov
3025f9e141 More fixes for PVS-Studio 2021-05-08 19:12:49 +03:00
Alexey Milovidov
8d62c42eb9 A bunch of changes for PVS-Studio 2021-05-08 19:11:50 +03:00
kssenii
5c24f0943c Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-05-08 14:54:44 +00:00
kssenii
266567e04c Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-08 17:49:05 +03:00
alexey-milovidov
680860aae7
Merge pull request #23912 from azat/CU 
Cleanup IDatabase.h from extra headers
 2021-05-08 16:42:56 +03:00
kssenii
02288359c5 Less manual concatenation of paths 2021-05-08 13:59:55 +03:00
kssenii
35f999bf04 Poco::createFile to fs::createFile 2021-05-08 02:41:47 +03:00
kssenii
c3eab60b8b Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-08 02:40:17 +03:00
kssenii
140bf7e2ea Less Poco 2021-05-07 15:29:26 +03:00
kssenii
912878ad7f Review fixes 2021-05-07 12:05:34 +00:00
Azat Khuzhin
3672af8bbe Make DatabaseConnectionMySQL self compilable (missing Core/NamesAndTypes.h) 2021-05-06 21:03:03 +03:00
kssenii
b96f047db7 Fix clang tidy 2021-05-06 07:02:13 +00:00
Azat Khuzhin
6348497d1b Cleanup IDatabase.h from extra headers 
From IDatabase.h depends ~300 CU.

- drop unused
- use forward declaration whenever is possible
 2021-05-06 09:07:59 +03:00
kssenii
8900e93bd8 New impl 2021-05-06 00:52:59 +00:00
kssenii
ed5c2321cc Some fixes 2021-05-04 10:57:05 +00:00
kssenii
eff26f9d54 Add comments, small improvements 2021-05-03 21:42:06 +00:00
kssenii
3e3396bd9c Refactor code, add comments 2021-05-03 09:52:32 +00:00
feng lv
39f68bf5ff fix conflict 2021-05-02 16:33:45 +00:00
kssenii
14d355a6ac Fix background update 2021-05-02 11:53:03 +00:00
kssenii
32b2917141 Update s3/* 2021-05-01 18:10:04 +03:00
kssenii
ee06936596 Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-01 17:24:31 +03:00
kssenii
14be3a0783 Better reload from snapshot 2021-05-01 11:49:45 +00:00
kssenii
7643d84095 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-04-30 22:02:51 +00:00
kssenii
e717b4c5c7 Update databases/* 2021-04-30 22:38:07 +03:00
kssenii
6e0cfdaeb5 Update Databases/* part 2 2021-04-30 20:24:36 +03:00
kssenii
1b75f12fcb Update Databases/* 2021-04-30 19:48:42 +03:00
Maksim Kita
318c4bb80d Add examples folder filter to ya.make.in 2021-04-30 11:25:52 +03:00
Maksim Kita
2e5f97c6ff Fixed code review issues 2021-04-27 22:09:03 +03:00
Maksim Kita
8e508a37b7 Fixed dictionaries_lazy_load during StorageDictionary CREATE query 2021-04-27 12:15:35 +03:00
Maksim Kita
f37599112a Added EXCHANGE DICTIONARIES query support 2021-04-26 23:36:15 +03:00
Maksim Kita
c93666e0de Fixed code review issues 2021-04-26 13:52:41 +03:00
Maksim Kita
43680d8ced Fixed tests 2021-04-26 13:52:41 +03:00
Maksim Kita
561b045991 Fixed tests 2021-04-26 13:52:41 +03:00
Maksim Kita
64f49a981b Fixed tests 2021-04-26 13:52:41 +03:00
Maksim Kita
2e3e90b25b Fixed tests 2021-04-26 13:52:41 +03:00
Maksim Kita
e5079c87db Added test 2021-04-26 13:52:41 +03:00
Maksim Kita
3cb4d1bdcb Fixed style check 2021-04-26 13:52:41 +03:00
Maksim Kita
a9917a6540 Refactored StorageDictionary 2021-04-26 13:52:41 +03:00
tavplubix
fc2e2848bf
Merge pull request #23413 from TCeason/modify_MaterializeMySQL_var_check 
modify MaterializeMySQL check MySQL var logical
 2021-04-26 09:52:59 +03:00
feng lv
0f5c05854c fix conflict 2021-04-23 12:22:51 +00:00
feng lv
4ffe199d39 Implement table comments 2021-04-23 12:18:23 +00:00
TCeason
51fe9fc771 modify MaterializeMySQL check MySQL var logical 
Supprot for MySQL 5.7.9: https://github.com/ClickHouse/ClickHouse/issues/4006#issuecomment-817622685
 2021-04-21 10:16:04 +08:00
tavplubix
70c89eeff6
Merge pull request #23200 from ClickHouse/fix_prewhere_in_materialize_mysql 
Fix prewhere in MaterializeMySQL
 2021-04-20 20:05:07 +03:00
tavplubix
738d8a757b
Merge pull request #23189 from ClickHouse/minor_fixes_in_attach_query 
Minor fixes in ATTACH query
 2021-04-20 20:04:26 +03:00
alexey-milovidov
cbbff1bc4b
Merge pull request #23319 from ClickHouse/mysql-insert-without-database 
Allow to INSERT into remote MySQL storage with empty database #19281
 2021-04-20 19:30:44 +03:00
tavplubix
bef0ab182e
Merge branch 'master' into fix_prewhere_in_materialize_mysql 2021-04-20 15:38:48 +03:00
Alexander Tokmakov
4108a235f5 make separate functions 2021-04-20 14:22:02 +03:00
Kseniia Sumarokova
b43656bf89
Merge pull request #22697 from TCeason/feature/add_MaterializeMySQL_sync_bytes_judgment 
Add MySQL read history data bytes judgment
 2021-04-20 10:26:42 +03:00
Alexey Milovidov
ac68dd8e97 Fix bad code; allow to INSERT into MySQL without database name 2021-04-20 00:29:37 +03:00
TCeason
472c131420 Add MySQL read history data bytes judgment 2021-04-19 10:51:50 +08:00
kssenii
88b50621d4 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-04-18 14:40:37 +00:00
Alexander Tokmakov
0354312294 minor fixes in attach query 2021-04-17 22:43:02 +03:00
Alexander Tokmakov
3b4bd18809 fix prewhere in materialize mysql 2021-04-16 18:51:27 +03:00
Alexander Tokmakov
ebf0a3119e fix data_type_default_nullable on attach 2021-04-16 13:29:33 +03:00
Christian
969b63d1ac
Excludes views from syncing in MaterializeMySQL (#22760) 
* Excludes views from syncing in MaterializeMySQL

* Adds integration test for ignoring MySQL views on initial replication

* Fixes bug in initial integration test for excluding views from replicating to ClickHouse on initial replication

* Fixes bug in initial integration test for excluding views from replicating to ClickHouse on initial replication

* Replace assert with check_query

Co-authored-by: Christian Frøystad <christian@securepractice.no>
Co-authored-by: Ivan <5627721+abyss7@users.noreply.github.com>
 2021-04-14 14:19:07 +03:00
kssenii
01075677cf Better 2021-04-11 20:22:33 +00:00
kssenii
027f67affb Fix checks 2021-04-11 09:54:10 +00:00
kssenii
beae1c5fa4 Use ContextPtr 2021-04-11 08:08:32 +00:00
kssenii
c1adece94f Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-04-11 06:00:18 +00:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr (#22297) 
* Replace all Context references with std::weak_ptr

* Fix shared context captured by value

* Fix build

* Fix Context with named sessions

* Fix copy context

* Fix gcc build

* Merge with master and fix build

* Fix gcc-9 build
 2021-04-11 02:33:54 +03:00
kssenii
bc228f4010 Better way to drop nested table for single storage 2021-04-10 19:26:53 +00:00
kssenii
1c501e7d97 Support replica identity index 2021-04-10 16:10:28 +00:00
kssenii
12f98e8b11 Much better 2021-04-09 10:22:06 +00:00
kssenii
f0be5c6938 Fix style, build, ya check 2021-04-09 08:07:56 +00:00
kssenii
3e41c82df1 Add mutex to tryGetTable only for materialized table (not nested) 2021-04-08 23:38:27 +00:00
kssenii
bd207daa74 Use only atomic database 2021-04-08 22:38:46 +00:00
kssenii
4482a35a3a Check table in a suggested way 2021-04-08 20:39:16 +00:00
kssenii
8586565ce9 Move connection-related filed to Core/PostgreSQL 2021-04-08 12:43:02 +00:00
kssenii
9fa21a6272 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-04-08 12:18:54 +00:00
Azat Khuzhin
f157278b72 Safer SCOPE_EXIT 
It executes the code in the dtor, that should never throw.
 2021-04-06 21:10:37 +03:00
Kseniia Sumarokova
ad85467128
Merge pull request #22668 from kssenii/postgres-add-bool 
Add bool type in postgres engine
 2021-04-06 13:29:33 +03:00
kssenii
cc2fdc6c62 Add bool type 2021-04-05 14:49:52 +00:00
kssenii
13d1f17d3e Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into replicas-shards-for-mysql-and-postgres 2021-04-03 08:00:09 +00:00
kssenii
2de6b99f84 Better names 2021-04-02 16:12:14 +00:00
kssenii
99f0b08ac3 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into replicas-shards-for-mysql-and-postgres 2021-04-01 13:48:33 +00:00
kssenii
522d170588 Add default port 2021-04-01 11:30:20 +00:00
kssenii
747967a9bb Better 2021-04-01 10:27:24 +00:00
kssenii
ba5ddd5c32 Do not parse replicas in PoolWithFailover 2021-03-31 14:03:52 +00:00
Maksim Kita
55984e849c
Merge branch 'master' into hierarchy-dictionaries-updated 2021-03-31 15:55:07 +03:00
kssenii
ce05087b1b Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into replicas-shards-for-mysql-and-postgres 2021-03-30 17:27:10 +00:00
Alexander Tokmakov
a5b07f2f03 Merge branch 'master' into test_multiple_nodes 2021-03-29 22:43:25 +03:00
tavplubix
3c0f5a57ec
Merge pull request #21535 from ClickHouse/distributed_ddl_improvements 
Distributed DDL improvements
 2021-03-29 22:40:11 +03:00
tavplubix
e3cf2d99b7
Update DatabaseReplicated.cpp 2021-03-29 16:44:41 +03:00
kssenii
22b515fbc9 Add namespace, simplify names 2021-03-27 20:14:02 +00:00
kssenii
f40c582e7a Replicas/shards for postgres storage 2021-03-27 20:11:48 +00:00
kssenii
ae868208c2 Use pool with failover in mysql storage 2021-03-27 14:39:45 +00:00
Maksim Kita
9f2f0d1095 Refactored hierarchy dictionaries interface 2021-03-26 18:42:32 +03:00
tavplubix
5dfd0045a6
Merge branch 'master' into distributed_ddl_improvements 2021-03-22 18:13:19 +03:00
kssenii
ed158e2b77 Proper setting 2021-03-20 14:42:03 +00:00
kssenii
4c37ff3fbc Less lambdas 2021-03-20 13:17:30 +00:00
kssenii
91b79f95bc Better templates 2021-03-20 12:24:35 +00:00
kssenii
caffccd73e Fixes 2021-03-20 11:53:41 +00:00
kssenii
6bb81630e5 Small fixes 2021-03-20 10:32:11 +00:00
kssenii
619f0f51fd Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-03-20 09:47:05 +00:00
kssenii
3b32c8fe9a Block on empty connection pool 2021-03-19 21:00:45 +00:00
kssenii
9057aad798 Better version 2021-03-19 08:11:36 +00:00
kssenii
f1ef87d966 Fix 2021-03-18 20:04:54 +00:00
Alexander Tokmakov
ce31d4608b fix tests 2021-03-18 22:13:21 +03:00
Alexander Tokmakov
343b8c906d Merge branch 'master' into test_multiple_nodes 2021-03-18 16:01:04 +03:00
Alexander Tokmakov
fdae70df6c fix tests 2021-03-18 15:50:03 +03:00
kssenii
3903d59d30 Better 2021-03-17 14:34:04 +00:00
kssenii
ae64a24844 Add connection pool 2021-03-17 13:55:47 +00:00
Nikita Mikhaylov
f51b41b7a4
Merge pull request #21710 from kssenii/replica-priorities 
Support replicas priority for postgres dictionary source
 2021-03-17 16:02:00 +03:00
kssenii
87c740730b Rename to MaterializePostgreSQL 2021-03-17 10:52:12 +00:00
Alexander Tokmakov
ff79be8af9 fix 2021-03-16 23:01:20 +03:00
kssenii
f8a99804c1 Add double quotes 2021-03-16 11:15:43 +00:00
kssenii
a03e849a8f Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-03-16 10:06:10 +00:00
tavplubix
6a265f468e
Merge branch 'master' into distributed_ddl_improvements 2021-03-15 11:24:14 +03:00
tavplubix
6ff8f468f6
Update DatabaseReplicated.cpp 2021-03-15 11:22:42 +03:00
kssenii
8eea6a87e9 Support replicas priority for postgres dictionary source 2021-03-14 08:51:24 +00:00
tavplubix
26ebd54eaf
Merge pull request #20961 from havardk/reconnect 
MaterializeMySQL: Attempt to reconnect to MySQL if the connection is lost
 2021-03-10 20:07:11 +03:00
Alexander Tokmakov
ba3d51383c fix 2021-03-10 00:41:04 +03:00