Commit Graph

877 Commits

Author SHA1 Message Date
Nikolay Degterinsky
bdc0d65c6b Fixes after merge 2021-09-22 16:57:17 +03:00
Nikolay Degterinsky
1b1d90deb8 Merge branch 'master' into query_parameters 2021-09-22 13:53:56 +03:00
kssenii
b51840a205 Merge branch 'master' of github.com:ClickHouse/ClickHouse into named-collections 2021-09-22 11:45:22 +03:00
tavplubix
e54bd40102
Merge pull request #29216 from ClickHouse/fix_29122
Fix terminate on uncaught exception
2021-09-21 20:12:51 +03:00
Alexander Tokmakov
7e346ec6ec fix terminate on uncaught exception 2021-09-21 11:04:22 +03:00
bharatnc
51e45050c1 Update outdated ya.make files
This runs the `utils/generate-ya-make/generate-ya-make.sh` script to
update the ya.make files that are not updated.

I wonder why they were not updated and how it was missed.
2021-09-20 18:28:07 -07:00
Nikolai Kochetov
a790d391c0 Merge branch 'master' into rewrite-pushing-to-views 2021-09-20 19:43:15 +03:00
tavplubix
922cf7ee20
Merge pull request #28373 from ClickHouse/tables_topsort
Resolve table dependencies on metadata loading
2021-09-20 14:46:47 +03:00
Nikolai Kochetov
341553febd Fix build. 2021-09-16 20:40:42 +03:00
kssenii
1650c92407 Better 2021-09-16 11:38:26 +03:00
kssenii
221c09589c Review fixes 2021-09-15 21:11:49 +03:00
Nikolay Degterinsky
6adfdd2f79 Fix style 2021-09-15 19:36:55 +03:00
kssenii
489a92c067 Review fixes 2021-09-15 18:36:49 +03:00
kssenii
84c8757049 Merge branch 'master' of github.com:ClickHouse/ClickHouse into named-collections 2021-09-15 12:28:31 +03:00
kssenii
710c8d12dd Merge branch 'master' of github.com:ClickHouse/ClickHouse into materialized-postgresql 2021-09-15 12:28:09 +03:00
Nikolai Kochetov
b997214620 Rename QueryPipeline to QueryPipelineBuilder. 2021-09-14 20:48:18 +03:00
Nikolai Kochetov
0e267c50b4 Merge branch 'master' into rewrite-pushing-to-views 2021-09-14 16:13:54 +03:00
Alexander Tokmakov
df56e99b87 fixes after review 2021-09-13 22:11:16 +03:00
Nikolay Degterinsky
9199b100ee Fix build 2021-09-13 17:44:02 +03:00
Alexander Tokmakov
8ac19caca9 Merge branch 'master' into tables_topsort 2021-09-13 16:58:39 +03:00
Nikolay Degterinsky
84aea776c2 Merge branch 'master' into query_parameters 2021-09-13 12:54:48 +03:00
kssenii
ff602c1fdd Fix build check 2021-09-11 19:29:23 +03:00
Azat Khuzhin
ff17de531c Join MaterializedMySQLSyncThread only if it is joinable
It is possible to trigger
MaterializedMySQLSyncThread::stopSynchronization() from the same thread
in case of some exception at startup, when some interpreter holds the
storage refcnt, and later MaterializedMySQLSyncThread will try to join
itself from the main thread.

Here is a stack trace for example:

<details>

    4  0x000000000f7ae45c in Poco::Event::wait (this=0x7f1b90c38170) at ../contrib/poco/Foundation/include/Poco/Event.h:97
    5  ThreadFromGlobalPool::join (this=0x7f1b90c23120) at ../src/Common/ThreadPool.h:210
    6  DB::MaterializeMySQLSyncThread::stopSynchronization (this=0x7f1b8024ca68) at ../src/Databases/MySQL/MaterializeMySQLSyncThread.cpp:229
    7  DB::MaterializeMySQLSyncThread::~MaterializeMySQLSyncThread (this=0x7f1b8024ca68) at ../src/Databases/MySQL/MaterializeMySQLSyncThread.cpp:85
    8  0x000000000f816dc8 in DB::DatabaseMaterializeMySQL<DB::DatabaseAtomic>::~DatabaseMaterializeMySQL (this=0x7f1b8024c918) at ../src/Databases/MySQL/DatabaseMaterializeMySQL.h:21
    11 std::__1::shared_ptr<DB::IDatabase>::~shared_ptr (this=<optimized out>) at ../contrib/libcxx/include/memory:3212
    12 0x000000000f8726a6 in DB::InterpreterCreateQuery::createTable (this=<optimized out>, create=...) at ../src/Interpreters/InterpreterCreateQuery.cpp:952
    13 0x000000000f87735c in DB::InterpreterCreateQuery::execute (this=0x7f1aeef59860) at ../src/Interpreters/InterpreterCreateQuery.cpp:1225
    14 0x000000000fe22253 in DB::executeQueryImpl (begin=<optimized out>, end=<optimized out>, context=..., internal=true, stage=DB::QueryProcessingStage::Complete, has_query_tail=<optimized out>, istr=<optimized out>) at ../src/Interpreters/executeQuery.cpp:574
    15 0x000000000fe208e3 in DB::executeQuery (query=..., context=..., internal=<optimized out>, stage=DB::QueryProcessingStage::FetchColumns, may_have_embedded_data=<optimized out>) at ../src/Interpreters/executeQuery.cpp:933
    16 0x000000000faafcde in DB::MySQLInterpreter::InterpreterMySQLDDLQuery<DB::MySQLInterpreter::InterpreterCreateImpl>::execute (this=<optimized out>) at ../src/Interpreters/MySQL/InterpretersMySQLDDLQuery.h:75
    17 0x000000000faade78 in DB::InterpreterExternalDDLQuery::execute (this=<optimized out>) at ../src/Interpreters/InterpreterExternalDDLQuery.cpp:64
    18 0x000000000fe22253 in DB::executeQueryImpl (begin=<optimized out>, end=<optimized out>, context=..., internal=true, stage=DB::QueryProcessingStage::Complete, has_query_tail=<optimized out>, istr=<optimized out>) at ../src/Interpreters/executeQuery.cpp:574
    19 0x000000000fe208e3 in DB::executeQuery (query=..., context=..., internal=<optimized out>, stage=DB::QueryProcessingStage::FetchColumns, may_have_embedded_data=<optimized out>) at ../src/Interpreters/executeQuery.cpp:933
    20 0x000000000f7ba406 in DB::tryToExecuteQuery (query_to_execute=..., query_context=..., database=..., comment=...) at ../src/Databases/MySQL/MaterializeMySQLSyncThread.cpp:69
    21 0x000000000f7d4b88 in DB::dumpDataForTables() (connection=..., query_prefix=..., database_name=..., mysql_database_name=..., context=..., is_cancelled=..., need_dumping_tables=...) at ../src/Databases/MySQL/MaterializeMySQLSyncThread.cpp:326
    22 DB::MaterializeMySQLSyncThread::prepareSynchronized(DB::MaterializeMetadata&)::$_1::operator()() const (this=<optimized out>) at ../src/Databases/MySQL/MaterializeMySQLSyncThread.cpp:391
    29 DB::commitMetadata() (function=..., persistent_tmp_path=..., persistent_path=...) at ../src/Databases/MySQL/MaterializeMetadata.cpp:197
    30 0x000000000f80a000 in DB::MaterializeMetadata::transaction(DB::MySQLReplication::Position const&, std::__1::function<void ()> const&) (this=0x7f1b6375c3d8, position=..., fun=...) at ../src/Databases/MySQL/MaterializeMetadata.cpp:230
    31 0x000000000f7b2231 in DB::MaterializeMySQLSyncThread::prepareSynchronized (this=0x7f1b8024ca68, metadata=...) at ../src/Databases/MySQL/MaterializeMySQLSyncThread.cpp:388
    32 0x000000000f7b169c in DB::MaterializeMySQLSyncThread::synchronization (this=0x7f1b8024ca68) at ../src/Databases/MySQL/MaterializeMySQLSyncThread.cpp:180
    33 0x000000000f7d4074 in DB::MaterializeMySQLSyncThread::startSynchronization()::$_0::operator()() const (this=<optimized out>) at ../src/Databases/MySQL/MaterializeMySQLSyncThread.cpp:236

</details>
2021-09-11 09:27:12 +03:00
Nikolai Kochetov
f569a3e3f7 Merge branch 'master' into rewrite-pushing-to-views 2021-09-09 20:30:23 +03:00
kssenii
f26da04cdf Fix checks 2021-09-09 09:53:58 +03:00
kssenii
bed2688dad Review fixes 2021-09-09 01:25:08 +03:00
kssenii
797f858bc6 Merge branch 'master' of github.com:ClickHouse/ClickHouse into materialized-postgresql 2021-09-08 22:52:48 +03:00
kssenii
ab6937ee45 Merge branch 'master' of github.com:ClickHouse/ClickHouse into named-collections 2021-09-08 20:43:37 +03:00
mergify[bot]
9f0c50335f
Merge branch 'master' into tables_topsort 2021-09-08 10:55:11 +00:00
Maksim Kita
7a4a0b0ede
Merge pull request #28474 from azat/mysql-connection_no_block
Introduce connection_wait_timeout for MySQL engine.
2021-09-07 12:28:02 +03:00
Nikolay Degterinsky
22f59e1714 Add support for table and database query parameters 2021-09-07 01:19:02 +03:00
kssenii
76e49334fa Minor fixes 2021-09-05 01:59:44 +03:00
kssenii
a1e4d2e230 Fix style check and pvs check 2021-09-05 01:03:15 +03:00
kssenii
ac2d9a73a8 User managed slots 2021-09-04 14:49:40 +03:00
Nikolai Kochetov
66a76ab70f Rewrite PushingToViewsBlockOutputStream part 6 2021-09-03 20:29:36 +03:00
kssenii
a5f56bb588 Some other 2021-09-03 16:25:40 +03:00
kssenii
f17d1bc7a5 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into materialized-postgresql 2021-09-03 09:03:48 +00:00
kssenii
6efbee320b Done MySQL 2021-09-03 11:30:28 +03:00
Azat Khuzhin
ed70ed6f71 Introduce connection_no_wait setting for MySQL engine.
This will allow to avoid superfluous sleep during query execution, since
this not only not desired behavoiur, but also may hang the server, since
if you will execute enough queries that will use MySQL database but will
not allow enough connections (or your MySQL server is too slow) then you
may run out of threads in the global thread pool.

Also note that right now it is possible to get deadlock when the mysql
pool is full, consider the following scenario:

- you have m1 and m2 mysql tables
- you have q1 and q2 queries, bot queries join m1 and m2
- q1 allocated connection for m1 but cannot allocate connection for m2
- q2 allocated connection for m2 but cannot allocate connection for m1
- but to resolve the lock one should give up on the locking while it is not possible right now...

And then you got no free threads and this:

    # grep -h ^202 /proc/$(pgrep clickhouse-serv)/task/*/syscall  | cut -d' ' -f2 | sort | uniq -c | sort -nr | head
       1554 0x7ffb60b92fe8 # mutex in mysqlxx::PoolWithFailover::get
       1375 0x7ffb9f1c4748 # mutex in ::PoolEntryHelper::~PoolEntryHelper from DB::MultiplexedConnections::invalidateReplica
       1160 0x7ffb612918b8 # mutex in mysqlxx::PoolWithFailover::get
         42 0x7ffb9f057984 # mutex in ThreadPoolImpl<std::__1::thread>::worker

    *NOTE: 202 is a `futex` with WAIT*

(Went with `syscall` because debugging 10k+ threads is not easy, and
eventually it may TRAP)
2021-09-02 22:23:37 +03:00
Alexander Tokmakov
1e2844f999 support function IN 2021-09-02 16:48:41 +03:00
Alexander Tokmakov
666a3aee99 add another test 2021-09-02 16:34:46 +03:00
kssenii
7a45775f4f Complete postgres 2021-09-02 13:46:19 +03:00
Alexander Tokmakov
024a24aaf7 better code, moar logging 2021-09-02 12:19:37 +03:00
Alexander Tokmakov
c8d8f0a38c fix 2021-08-31 15:07:00 +03:00
Alexander Tokmakov
b387f05d9c resolve table dependencies on metadata loading 2021-08-30 22:29:22 +03:00
tavplubix
b47f34aa17
Merge pull request #28313 from amosbird/fastload
Fast data parts loading by delaying table startup process
2021-08-30 13:53:55 +03:00
Amos Bird
de9e591619
Fast load by delaying table startup 2021-08-30 10:15:32 +08:00
kssenii
9d0444774a Fix tests 2021-08-29 11:50:03 +03:00
kssenii
378e1f74aa Fix tests 2021-08-28 19:51:04 +03:00
kssenii
cc90d787f8 Slightly better 2021-08-28 17:45:15 +03:00
kssenii
fd621381c7 Allow modify some other settings 2021-08-28 17:36:27 +03:00
kssenii
a2b0996ac3 Dynamically remove tables 2021-08-28 17:35:59 +03:00
kssenii
174340074c Dynamically add tables complete 2021-08-28 17:35:44 +03:00
kssenii
4cd62227cf Almost done 2021-08-28 17:34:23 +03:00
tavplubix
c36fe210b8
Revert "Add test for #13398" 2021-08-27 21:07:10 +03:00
kssenii
0d5000f917 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into mysql 2021-08-24 07:30:42 +00:00
kssenii
962f415740 Add test and settings 2021-08-23 22:18:04 +00:00
Vitaly Baranov
63e8bc1f20 Added new commands BACKUP and RESTORE. 2021-08-23 23:41:45 +03:00
Vitaly Baranov
2ac97c7fc3 Make IDatabase::getTablesIterator() const. 2021-08-23 23:41:45 +03:00
kssenii
200d75646a Fix 2021-08-13 20:58:03 +00:00
Maksim Kita
124a87684f Removed some data streams 2021-08-11 23:39:01 +03:00
Mikhail Filimonov
af6249c8ec
Less include <Common/Stopwatch.h> 2021-08-11 16:20:59 +02:00
Maksim Kita
b8b24684b4
Merge pull request #27273 from ClickHouse/remove-streams-from-dicts
Remove streams from dicts
2021-08-09 15:58:37 +03:00
Maksim Kita
1db4eddafc
Merge pull request #27225 from kitaisreal/database-replicated-worker-logs-to-keep-race-fix
DatabaseReplicatedWorker logs_to_keep race fix
2021-08-06 14:16:36 +03:00
tavplubix
14ef66e49b
Merge pull request #25895 from ClickHouse/improve_create_or_replace
Improve CREATE OR REPLACE query
2021-08-06 11:58:46 +03:00
Nikolai Kochetov
39ff5ab478 Merge branch 'master' into remove-streams-from-dicts 2021-08-06 11:46:27 +03:00
Nikolai Kochetov
13f95f3fdf Streams -> Processors for dicts, part 3. 2021-08-06 11:41:45 +03:00
Kseniia Sumarokova
5edd9e0513
Merge pull request #27198 from kssenii/postgres-db-schema
Support schema for postgres database engine
2021-08-05 18:30:47 +03:00
Maksim Kita
1ae59b6cce DatabaseReplicatedWorker logs_to_keep race fix 2021-08-05 17:05:03 +03:00
kssenii
e8dd257468 Update DatabaseFactory.cpp 2021-08-05 09:04:00 +00:00
kssenii
1e38d48364 Fixes 2021-08-05 07:25:47 +00:00
kssenii
f06703a7c9 Support schema for postgres database engine 2021-08-04 20:52:45 +00:00
Alexander Tokmakov
ff2572c243 fix 2021-08-03 16:28:03 +03:00
Alexander Tokmakov
42a8bb6872 fix assertions in Replicated database 2021-08-02 16:19:11 +03:00
Alexander Tokmakov
fc9ab2cda7 Merge branch 'master' into rename_materialize_mysql 2021-07-27 22:38:40 +03:00
Alexander Kuzmenkov
ece7d000dd
Merge pull request #26736 from Algunenano/ch_local_remote_localhost
CH local: Treat localhost:port as a remote database
2021-07-27 15:30:54 +03:00
Alexander Tokmakov
a7e644f774 support old name 2021-07-26 21:24:32 +03:00
Alexander Tokmakov
63ab38ee09 rename MaterializeMySQL to MaterializedMySQL 2021-07-26 21:17:28 +03:00
kssenii
f3129b80f9 Fix sqlite attach 2021-07-26 07:20:41 +00:00
Raúl Marín
383c982715 CH local: Treat localhost:port as a remote database 2021-07-23 13:16:35 +02:00
mergify[bot]
39f71f4a08
Merge branch 'master' into fix_no_column_materialized_mysql 2021-07-19 12:27:38 +00:00
kssenii
dd519f9434 Fix 2021-07-15 19:02:31 +03:00
kssenii
f20ea87bfb Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into add-sqlite-database-path-check 2021-07-14 20:49:05 +00:00
Kseniia Sumarokova
e42a26a585
Merge pull request #26234 from jasine/master
convert timestamp and  timestamptz data types to DateTime64 in postgres engine
2021-07-14 23:42:51 +03:00
kssenii
4fb0762bbb Merge branch 'add-sqlite-database-path-check' of github.com:kssenii/ClickHouse into add-sqlite-database-path-check 2021-07-14 20:08:11 +00:00
kssenii
e875f1e289 Fix yandex check 2021-07-14 20:07:55 +00:00
Kseniia Sumarokova
ef0ebaf55c
Update SQLiteUtils.cpp 2021-07-14 20:20:41 +03:00
kssenii
d62cbbb8a5 Unification 2021-07-14 13:16:30 +00:00
kssenii
b393603b57 Fix build 2021-07-13 22:25:26 +00:00
Nikita Mikhaylov
601d04f55d ignore 2021-07-13 22:16:27 +00:00
kssenii
da09ce3f20 Remove unused 2021-07-13 19:51:02 +00:00
kssenii
a90df59b92 Better 2021-07-13 19:46:06 +00:00
mergify[bot]
d5898ecba6
Merge branch 'master' into fix_no_column_materialized_mysql 2021-07-13 14:03:05 +00:00
kssenii
9688efde96 Merge branch 'add-sqlite-database-path-check' of github.com:kssenii/ClickHouse into add-sqlite-database-path-check 2021-07-13 12:40:43 +00:00
kssenii
f86817d57c Check inclusion inside user_files 2021-07-13 12:38:37 +00:00
Kseniia Sumarokova
c615084ba7
Update DatabaseSQLite.cpp 2021-07-13 14:40:07 +03:00
kssenii
48ffd3b81d Add check for sqlite database path 2021-07-13 09:40:46 +00:00
jasine
4ec8d23fce
Merge branch 'ClickHouse:master' into master 2021-07-13 15:12:40 +08:00
kssenii
bcc77fca62 Merge branch 'add-sqlite-support' of github.com:g-arslan/ClickHouse into g-arslan-add-sqlite-support 2021-07-12 22:39:34 +00:00
kssenii
976f4dffda Add table function sqlite, fix type conversions 2021-07-12 21:09:23 +00:00
Kseniia Sumarokova
08ab42483c
Update DatabaseFactory.cpp 2021-07-12 20:40:30 +03:00
jasine
192ee2df98
Update fetchPostgreSQLTableStructure.cpp 2021-07-12 12:39:17 +08:00
kssenii
10611ef92a Normal exceptions 2021-07-10 11:00:59 +00:00
kssenii
18c2abaaf9 Fix database engine attach, more tests 2021-07-10 10:31:21 +00:00
kssenii
d250d79e33 Generate ya make 2021-07-10 08:07:55 +00:00
kssenii
78094a82ab Better 2021-07-10 08:05:23 +00:00
kssenii
1bf9a9f29f Add .cmake 2021-07-09 10:27:53 +00:00
kssenii
ff56f1787c Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into g-arslan-add-sqlite-support 2021-07-07 15:19:12 +00:00
Alexander Tokmakov
0f377fc4d8 Merge branch 'master' into fix_no_column_materialized_mysql 2021-07-05 22:07:56 +03:00
kssenii
32b7d7b750 More correct startup on create query 2021-07-04 19:53:05 +00:00
kssenii
b39888f199 Do not allow inconsistency in publication 2021-07-04 08:54:41 +00:00
kssenii
0ce4c565d1 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-07-03 11:48:15 +00:00
kssenii
f01c8edbff Fix access to tables after restart 2021-07-01 07:33:58 +00:00
Kostiantyn Storozhuk
a9510f25f9 Style fix 2021-06-29 15:22:08 +08:00
Kostiantyn Storozhuk
296593716d Reverted constructor change 2021-06-29 14:37:53 +08:00
Kostiantyn Storozhuk
c2c78929cb Implemented MySQL column comments support 2021-06-28 22:44:38 +08:00
Kseniia Sumarokova
ff8c44179c
Update DatabaseAtomic.cpp 2021-06-28 17:37:22 +03:00
Kseniia Sumarokova
a96c7a3a70
Update DatabaseAtomic.cpp 2021-06-28 17:32:03 +03:00
Kseniia Sumarokova
b6b7924413
Update DatabaseAtomic.cpp 2021-06-28 10:20:33 +03:00
kssenii
aa7133c239 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-06-27 20:08:00 +00:00
kssenii
c3a2fc0584 Review fixes 2021-06-27 19:17:17 +00:00
alexey-milovidov
fb89505656
Merge pull request #25538 from kssenii/pg-dix
Fix postrges array conversion
2021-06-27 19:35:24 +03:00
alexey-milovidov
6e25d11e1f
Merge pull request #25582 from azat/atomic-unlink-symlink
Remove only symlinks during force_restore_data of Atomic engine
2021-06-27 18:19:59 +03:00
kssenii
3d90f0710c Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-06-26 22:04:47 +00:00
Maksim Kita
3c9ae7b5ba Added more tests 2021-06-26 14:28:13 +03:00
Azat Khuzhin
50d9d3efd5 Remove only symlinks during force_restore_data of Atomic engine 2021-06-23 10:17:31 +03:00
Kseniia Sumarokova
77a3a1416d
Update fetchPostgreSQLTableStructure.cpp 2021-06-21 17:26:14 +03:00
kssenii
0f9fc33a4e Fix postgres arrays 2021-06-21 11:27:10 +00:00
Maksim Kita
2c4c2680f7
Merge pull request #25430 from kitaisreal/database-mysql-rename
DatabaseMySQL rename
2021-06-18 01:30:52 +03:00
Maksim Kita
cf77e9f2c6 DatabaseMySQL rename 2021-06-17 22:26:34 +03:00
Maksim Kita
67e9b85951 Merge ext into common 2021-06-16 23:28:41 +03:00
kssenii
0521851afa Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into postgres-fixes-schema 2021-06-08 17:20:23 +00:00
kssenii
e23bc14582 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-06-03 19:44:55 +00:00
mergify[bot]
438b45e6c0
Merge branch 'master' into add-sqlite-support 2021-06-03 12:26:12 +00:00
Arslan G
4fe9a24472 Fix review a little, try add simple test 2021-06-02 03:53:30 +03:00
Arslan G
dc0a1001d8 Fix style 2021-06-02 01:21:01 +03:00
Arslan G
6f7dc47b2b Merge branch 'master' into add-sqlite-support 2021-06-02 00:24:29 +03:00
Arslan G
bf4171e0ea Some review fixes 2021-06-01 23:52:12 +03:00
kssenii
12ea45e956 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-06-01 17:43:34 +00:00
tavplubix
e2941a8006
Update DatabaseReplicated.cpp 2021-06-01 15:15:13 +03:00
Alexander Tokmakov
94d2aed336 fix tests 2021-06-01 13:24:06 +03:00
Alexander Tokmakov
fe482109da Merge branch 'master' into fix_database_replicated_enqueue_race 2021-06-01 11:49:29 +03:00
tavplubix
e9ff0b6d70
Merge pull request #23657 from kssenii/poco-file-to-std-fs
Poco::File to std::filesystem
2021-05-31 23:17:02 +03:00
Nikolai Kochetov
afc1fe7f3d Make ContextPtr const by default. 2021-05-31 17:49:02 +03:00
Alexander Tokmakov
0035997e02 fix race on enqueue query 2021-05-31 16:31:03 +03:00
kssenii
0a3589524e File movement 2021-05-29 00:57:53 +03:00
kssenii
2a631aaf08 Final fixes 2021-05-29 00:34:44 +03:00
Arslan G
fbc5734302 Clean up code and fix tests 2021-05-28 20:00:26 +03:00
kssenii
a78c9e63e7 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-05-23 12:07:26 +00:00
kssenii
866b29fb5a Return list fds with Poco, more canonical 2021-05-23 10:56:13 +03:00
kssenii
1c43b333ea Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-22 21:26:48 +03:00
kssenii
3e6d3d4ecb Postgres schema for insert 2021-05-22 08:50:23 +00:00
alexey-milovidov
a25f1910cc
Merge pull request #24320 from ClickHouse/remove-auto-array
Remove AutoArray
2021-05-22 02:19:59 +03:00
kssenii
096d117f68 Review fixes 2021-05-22 00:12:46 +03:00
Alexey Milovidov
ea30bb7abf Improve implementation 2021-05-21 04:17:18 +03:00
Azat Khuzhin
4f41ebcae3 Add settings (for connections) for MySQL storage engine
Default settings are not very efficient, since they do not even reuse
connections.
And when each query requires connection you can have only ~80 QPS, while
by simply enabling connection reuse (connection_auto_close=false) you
can have ~500 QPS (and by increasing connection_pool_size you can have
better QPS throughput).

So this patch allows to pass through some connection related settings
for the StorageMySQL engine, like:
- connection_pool_size=16
- connection_max_tries=3
- connection_auto_close=true

v2: remove connection_pool_default_size
v3: remove num_tries_on_connection_loss
2021-05-20 22:44:52 +03:00
kssenii
3dee003f9b Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-20 19:20:09 +03:00
kssenii
5f083b779d A little better, fix checks 2021-05-18 14:25:19 +03:00
kssenii
9b8df78fdd Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-17 17:42:05 +03:00
Arslan G
dde9e30043 Merge branch 'master' into add-sqlite-support 2021-05-17 14:38:24 +03:00
Arslan G
ea51840f85 Init SQLite integration 2021-05-17 14:02:35 +03:00
kssenii
1ee77eae89 Better way to distinguish between user query and replication thread query 2021-05-16 21:35:37 +00:00
kssenii
8a4711b716 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-05-16 20:55:18 +00:00
Azat Khuzhin
23981ffd0e Introduce IStorage::flushAndShutdown() 2021-05-16 13:23:53 +03:00
Azat Khuzhin
f20799f683 Destroy Buffer() tables before others (within one database)
This will reduce amount of data that may be lost at shutdown.

v2: replace dynamic_cast with IStorage::isBuffer (@kitaisreal)
v3: replace IStorage::isBuffer with IStorage::flush (@alexey-milovidov)
v4: flush() for StorageProxy/StorageTableFunction
2021-05-16 13:23:53 +03:00
feng lv
c6f8ab9826 fix 2021-05-13 02:05:53 +00:00
kssenii
7c81103ea3 Add replication identifier, allow proper access to _version and _sign columns from nested 2021-05-12 22:48:42 +00:00
kssenii
407db17e2e Remove setting, use native transaction 2021-05-12 22:33:15 +00:00
kssenii
0955e5f0be Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-05-12 15:11:15 +00:00
kssenii
0527f0ea33 Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-12 16:54:18 +03:00
Nikolai Kochetov
a1ba67e029
Merge pull request #20202 from amosbird/projection
Projections
2021-05-12 09:38:07 +03:00
kssenii
81797393f1 fs::last_write_time, fs::space 2021-05-12 09:19:11 +03:00
Maksim Kita
72d46beca0
Merge pull request #23979 from azat/dict-preallocate
Reimplement preallocate for hashed/sparse_hashed dictionaries
2021-05-11 20:15:46 +03:00
feng lv
9035e3a643 fix test and conflict 2021-05-11 16:22:24 +00:00
kssenii
393b026f2e Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-11 16:49:23 +03:00
Amos Bird
264cff6415
Projections
TODO (suggested by Nikolai)

1. Build query plan fro current query (inside storage::read) up to WithMergableState
2. Check, that plan is simple enough: Aggregating - Expression - Filter - ReadFromStorage (or simplier)
3. Check, that filter is the same as filter in projection, and also expression calculates the same aggregation keys as in projection
4. Return WithMergableState if projection applies

3 will be easier to do with ActionsDAG, cause it sees all functions, and dependencies are direct (but it is possible with ExpressionActions also)

Also need to figure out how prewhere works for projections, and
row_filter_policies.

wip
2021-05-11 18:12:23 +08:00
kssenii
626e87bae5 Some fixes 2021-05-10 13:51:05 +00:00
kssenii
ae1191d0c0 Get rid of metadata file, rely only on lsn 2021-05-10 09:10:02 +00:00
kssenii
4ac023e511 Small fixes 2021-05-10 08:45:52 +00:00
kssenii
993823bcc4 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-05-09 21:26:41 +00:00
kssenii
ab1a05a1f4 Poco::Path to fs::path, less concatination 2021-05-09 14:59:49 +03:00
Azat Khuzhin
80f4059deb Drop unused getDictionaryConfigurationFromAST.h 2021-05-09 14:25:45 +03:00
kssenii
e26cb4bee0 Fix style check, fix yandex check 2021-05-09 13:45:50 +03:00
Alexey Milovidov
3025f9e141 More fixes for PVS-Studio 2021-05-08 19:12:49 +03:00
Alexey Milovidov
8d62c42eb9 A bunch of changes for PVS-Studio 2021-05-08 19:11:50 +03:00
kssenii
5c24f0943c Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into pg-ch-replica 2021-05-08 14:54:44 +00:00
kssenii
266567e04c Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-08 17:49:05 +03:00
alexey-milovidov
680860aae7
Merge pull request #23912 from azat/CU
Cleanup IDatabase.h from extra headers
2021-05-08 16:42:56 +03:00
kssenii
02288359c5 Less manual concatenation of paths 2021-05-08 13:59:55 +03:00
kssenii
35f999bf04 Poco::createFile to fs::createFile 2021-05-08 02:41:47 +03:00
kssenii
c3eab60b8b Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-08 02:40:17 +03:00
kssenii
140bf7e2ea Less Poco 2021-05-07 15:29:26 +03:00
kssenii
912878ad7f Review fixes 2021-05-07 12:05:34 +00:00
Azat Khuzhin
3672af8bbe Make DatabaseConnectionMySQL self compilable (missing Core/NamesAndTypes.h) 2021-05-06 21:03:03 +03:00
kssenii
b96f047db7 Fix clang tidy 2021-05-06 07:02:13 +00:00
Azat Khuzhin
6348497d1b Cleanup IDatabase.h from extra headers
From IDatabase.h depends ~300 CU.

- drop unused
- use forward declaration whenever is possible
2021-05-06 09:07:59 +03:00
kssenii
8900e93bd8 New impl 2021-05-06 00:52:59 +00:00
kssenii
ed5c2321cc Some fixes 2021-05-04 10:57:05 +00:00
kssenii
eff26f9d54 Add comments, small improvements 2021-05-03 21:42:06 +00:00
kssenii
3e3396bd9c Refactor code, add comments 2021-05-03 09:52:32 +00:00