Commit Graph

256 Commits

Author SHA1 Message Date
Alexander Tokmakov
d682c2ccdb fix 2022-10-05 22:13:16 +02:00
Robert Schulze
fd86829824
Consolidate config_core.h into config.h
Less duplication, less confusion ...
2022-09-28 13:31:57 +00:00
Alexey Milovidov
ebe7d2d25e Fix stack overflow on recursive Buffer tables and system.tables 2022-08-25 21:26:32 +02:00
kssenii
0dda03c94b Fix checks 2022-08-10 00:06:58 +02:00
Alexander Tokmakov
6a5171c829
Update DatabaseCatalog.cpp 2022-08-04 17:10:13 +03:00
Alexander Tokmakov
a269cf29e4 remove unused dirs from store on all disks 2022-08-03 19:49:16 +02:00
Alexander Tokmakov
5ef7f5f494
Merge pull request #39229 from azat/fix-noexcept-scope
[RFC] Fix NOEXCEPT_SCOPE (before it calls std::terminate and looses the exception)
2022-07-15 18:14:56 +03:00
avogar
9291d33080 Pass const std::string_view & by value, not by reference 2022-07-14 16:11:57 +00:00
Azat Khuzhin
b20407fab9 Fix NOEXCEPT_SCOPE (before it calls std::terminate and looses the exception)
Current implementation of NOEXCEPT_SCOPE will not work, you cannot
rethrow exception outside the catch block, this will simply terminate
(via std::terminate) the program.
In other words NOEXCEPT_SCOPE macro will simply call std::terminate on
exception and will lost original exception.

But if NOEXCEPT_SCOPE will accept the code that should be runned w/o
exceptions, then it can catch exception and log it, rewrite it in this
way.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-07-14 17:16:18 +03:00
Vitaly Baranov
e367d96964 Fix style. 2022-06-30 15:10:33 +02:00
Vitaly Baranov
aa97bf5125 Improve handling predefined databases and tables. 2022-06-30 08:37:17 +02:00
Alexander Tokmakov
4f54abf67f better names for macros 2022-06-28 16:27:18 +02:00
Alexander Tokmakov
f4883f1f7e add some TSA aanotations 2022-06-27 22:48:27 +02:00
Alexander Tokmakov
59a14d2a50 add docs 2022-06-24 17:34:41 +02:00
Alexander Tokmakov
379fecf7f9 try suppress failures with MaterializedPostgreSQL 2022-06-24 17:15:15 +02:00
Alexander Tokmakov
ed8341025b make code less bad 2022-06-22 18:31:42 +02:00
Alexander Tokmakov
dc1f596326 fix 2022-06-22 00:50:16 +02:00
Alexander Tokmakov
13e3d40db4 fix 2022-06-21 15:11:03 +02:00
Alexander Tokmakov
ba0fcec993 add background cleanup of store/ subdirs 2022-06-21 12:35:47 +02:00
Vitaly Baranov
0102626532 Disable the 'BACKUP ALL' command (it's not quite clear what to do with predefined databases). 2022-06-15 20:32:35 +02:00
Vitaly Baranov
1198e86295 Fix storing temporary tables and skipping system tables while making a backup. 2022-06-15 20:32:34 +02:00
Vitaly Baranov
592f568f83 Move backup/restore code to storages and databases - part 2. 2022-06-15 20:32:31 +02:00
mergify[bot]
f90dddccba
Merge branch 'master' into fix-temp-table-drop 2022-05-31 09:10:00 +00:00
Alexey Milovidov
c50791dd3b Fix clang-tidy-14, part 1 2022-05-27 22:52:14 +02:00
Vxider
54d6f98122 flush and shutdown temporary table before drop 2022-05-27 04:50:36 +00:00
Robert Schulze
45c0ec44b0
Fix build, pt. II 2022-05-04 10:15:25 +02:00
Robert Schulze
330212e0f4
Remove inherited create() method + disallow copying
The original motivation for this commit was that shared_ptr_helper used
std::shared_ptr<>() which does two heap allocations instead of
make_shared<>() which does a single allocation. Turned out that
1. the affected code (--> Storages/) is not on a hot path (rendering the
performance argument moot ...)
2. yet copying Storage objects is potentially dangerous and was
   previously allowed.

Hence, this change

- removes shared_ptr_helper and as a result all inherited create() methods,

- instead, Storage objects are now created using make_shared<>() by the
  caller (for that to work, many constructors had to be made public), and

- all Storage classes were marked as noncopyable using boost::noncopyable.

In sum, we are (likely) not making things faster but the code becomes
cleaner and harder to misuse.
2022-05-02 08:46:52 +02:00
Amos Bird
4a5e4274f0
base should not depend on Common 2022-04-29 10:26:35 +08:00
Robert Schulze
118e94523c
Activate clang-tidy warning "readability-container-contains"
This check suggests replacing <Container>.count() by
<Container>.contains() which is more speaking and in case of
multimaps/multisets also faster.
2022-04-18 23:53:11 +02:00
Raúl Marín
6e346a7fc3 Support atomic replace under OSX 2022-04-11 14:32:55 +02:00
Antonio Andelic
9990abb76a Use compile-time check for Exception messages, fix wrong messages 2022-03-29 13:16:11 +00:00
Maksim Kita
1f5837359e clang-tidy check performance-noexcept-move-constructor fix 2022-03-02 18:15:27 +00:00
Stig Bakken
ff46e8bb51 Drop support for DatabaseOrdinary in MaterializedMySQL
1. Dropped support for DatabaseOrdinary for MaterializeMySQL. It
   is marked as experimental, and dropping support makes the code
   more maintaible, and speeds up integration tests by 50%.

2. Get rid of thread name logic for StorageMaterializeMySQL wrapping,
   use setInternalQuery instead (similar to MaterializedPostgreSQL).
2021-11-18 11:46:51 +01:00
Nikolay Degterinsky
2f6a565b06 Merge branch 'master' into query_parameters 2021-11-16 10:37:58 +00:00
Nikolay Degterinsky
7b5954b736 Merge branch 'master' into query_parameters 2021-11-11 11:43:00 +00:00
Alexander Tokmakov
c4108ab672 support alter 2021-11-02 17:30:22 +03:00
Alexander Tokmakov
598dbe04db fix tests, support rename 2021-11-02 15:58:45 +03:00
Alexander Tokmakov
2b566c5bdf basic check for dependencies on drop table 2021-11-01 22:20:18 +03:00
Alexey Milovidov
8b4a6a2416 Remove cruft 2021-10-28 02:10:39 +03:00
Nikolay Degterinsky
63faf0212d Merge branch 'master' into query_parameters 2021-10-13 23:55:09 +03:00
tavplubix
7742b96497
Remove metadata leftovers on drop database (#30054)
* remove metadata leftovers on drop database

* Update InterpreterDropQuery.cpp

* Update DatabaseCatalog.cpp
2021-10-13 13:34:18 +03:00
Nikolay Degterinsky
e03bf0c526 Merge branch 'master' into query_parameters 2021-10-04 17:23:17 +03:00
Alexey Milovidov
fe6b7c77c7 Rename "common" to "base" 2021-10-02 10:13:14 +03:00
Nikolay Degterinsky
1b1d90deb8 Merge branch 'master' into query_parameters 2021-09-22 13:53:56 +03:00
tavplubix
922cf7ee20
Merge pull request #28373 from ClickHouse/tables_topsort
Resolve table dependencies on metadata loading
2021-09-20 14:46:47 +03:00
Nikolay Degterinsky
84aea776c2 Merge branch 'master' into query_parameters 2021-09-13 12:54:48 +03:00
kssenii
797f858bc6 Merge branch 'master' of github.com:ClickHouse/ClickHouse into materialized-postgresql 2021-09-08 22:52:48 +03:00
mergify[bot]
9f0c50335f
Merge branch 'master' into tables_topsort 2021-09-08 10:55:11 +00:00
Nikolay Degterinsky
22f59e1714 Add support for table and database query parameters 2021-09-07 01:19:02 +03:00
kssenii
7bea8200d8 Revert "Merge pull request #28082 from zhongyuankai/add-system-of-table_views"
This reverts commit e5bcfba89e, reversing
changes made to d77f243651.
2021-09-06 19:20:33 +00:00
Azat Khuzhin
bf6ba796f8 Fix UUID overlap in DROP TABLE for internal DDL from MaterializeMySQL
This will fix race with DatabaseCatalog::loadMarkedAsDroppedTables(),
since MaterializeMySQL, and MaterializedMySQLSyncThread in background,
will be started earlier then
DatabaseCatalog::loadMarkedAsDroppedTables() and will move those tables
to metadata_dropped, and after loadMarkedAsDroppedTables() will start
and try to load partially dropped tables and will hit UUID overlap:

    12:02:51.536783 [ 3026034 ] {} <Information> Application: starting up
    12:02:53.019282 [ 3026034 ] {} <Information> DatabaseMaterializeMySQL<Atomic> (mysql): Total 9 tables and 0 dictionaries.
    12:02:53.041699 [ 3026200 ] {} <Debug> mysql.data (7143b65f-6982-4600-b143-b65f6982e600): Loading data parts
    12:02:53.041740 [ 3026200 ] {} <Debug> mysql.data (7143b65f-6982-4600-b143-b65f6982e600): There are no data parts
    12:02:53.620382 [ 3026034 ] {} <Information> DatabaseMaterializeMySQL<Atomic> (mysql): Starting up tables.
    12:03:00.669730 [ 3026183 ] {} <Debug> executeQuery: (internal) /*Materialize MySQL step 1: execute MySQL DDL for dump data*/ DROP TABLE mysql.data
    12:03:00.741894 [ 3026269 ] {} <Information> DatabaseCatalog: Trying load partially dropped table mysql.data (7143b65f-6982-4600-b143-b65f6982e600) from /var/lib/clickhouse/metadata_dropped/mysql.data.7143b65f-6982-4600-b143-b65f6982e600.sql
    12:03:00.742582 [ 3026269 ] {} <Debug> mysql.data (7143b65f-6982-4600-b143-b65f6982e600): Loading data parts
    12:03:00.742650 [ 3026269 ] {} <Debug> mysql.data (7143b65f-6982-4600-b143-b65f6982e600): There are no data parts
    12:03:00.773137 [ 3026034 ] {} <Error> Application: Caught exception while loading metadata: Code: 57, e.displayText() = DB::Exception: Mapping for table with UUID=7143b65f-6982-4600-b143-b65f6982e600 already exists. It happened due to UUID collision, most likely because some not random UUIDs were manually specified in CREATE queries., Stack trace (when copying this message, always include the lines below):
    12:03:01.224557 [ 3026034 ] {} <Error> Application: DB::Exception: Mapping for table with UUID=7143b65f-6982-4600-b143-b65f6982e600 already exists. It happened due to UUID collision, most likely because some not random UUIDs were manually specified in CREATE queries.

Cc: @zhang2014
2021-09-04 10:41:28 +03:00
kssenii
f17d1bc7a5 Merge branch 'master' of https://github.com/ClickHouse/ClickHouse into materialized-postgresql 2021-09-03 09:03:48 +00:00
Alexander Tokmakov
b387f05d9c resolve table dependencies on metadata loading 2021-08-30 22:29:22 +03:00
tavplubix
b47f34aa17
Merge pull request #28313 from amosbird/fastload
Fast data parts loading by delaying table startup process
2021-08-30 13:53:55 +03:00
Amos Bird
de9e591619
Fast load by delaying table startup 2021-08-30 10:15:32 +08:00
kssenii
174340074c Dynamically add tables complete 2021-08-28 17:35:44 +03:00
kssenii
4cd62227cf Almost done 2021-08-28 17:34:23 +03:00
zhongyuankai
042d17531c system.views add columns type 2021-08-26 20:05:01 +08:00
zhongyuankai
3e18e22cfb update api 2021-08-25 20:20:05 +08:00
zhongyuankai
64efa917d8 review system.views 2021-08-25 18:11:40 +08:00
OnePiece
40673aa1f5
Update DatabaseCatalog.cpp 2021-08-24 20:05:29 +08:00
Alexander Tokmakov
63ab38ee09 rename MaterializeMySQL to MaterializedMySQL 2021-07-26 21:17:28 +03:00
kssenii
f01c8edbff Fix access to tables after restart 2021-07-01 07:33:58 +00:00
Amos Bird
9dfa4c06bd
Try reusing built sets during multi-pass analysis 2021-06-03 22:06:05 +08:00
tavplubix
e9ff0b6d70
Merge pull request #23657 from kssenii/poco-file-to-std-fs
Poco::File to std::filesystem
2021-05-31 23:17:02 +03:00
Nikolai Kochetov
afc1fe7f3d Make ContextPtr const by default. 2021-05-31 17:49:02 +03:00
kssenii
0a3589524e File movement 2021-05-29 00:57:53 +03:00
kssenii
2a631aaf08 Final fixes 2021-05-29 00:34:44 +03:00
kssenii
096d117f68 Review fixes 2021-05-22 00:12:46 +03:00
kssenii
9b8df78fdd Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-17 17:42:05 +03:00
feng lv
8c82e3f1b9 fix style
fix
2021-05-14 15:50:05 +00:00
feng lv
c6f8ab9826 fix 2021-05-13 02:05:53 +00:00
kssenii
0527f0ea33 Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-12 16:54:18 +03:00
kssenii
81797393f1 fs::last_write_time, fs::space 2021-05-12 09:19:11 +03:00
feng lv
9035e3a643 fix test and conflict 2021-05-11 16:22:24 +00:00
kssenii
393b026f2e Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-11 16:49:23 +03:00
Amos Bird
264cff6415
Projections
TODO (suggested by Nikolai)

1. Build query plan fro current query (inside storage::read) up to WithMergableState
2. Check, that plan is simple enough: Aggregating - Expression - Filter - ReadFromStorage (or simplier)
3. Check, that filter is the same as filter in projection, and also expression calculates the same aggregation keys as in projection
4. Return WithMergableState if projection applies

3 will be easier to do with ActionsDAG, cause it sees all functions, and dependencies are direct (but it is possible with ExpressionActions also)

Also need to figure out how prewhere works for projections, and
row_filter_policies.

wip
2021-05-11 18:12:23 +08:00
Alexey Milovidov
bdc372cb9a Fix bad code (found by PVS-Studio) 2021-05-08 19:11:40 +03:00
kssenii
266567e04c Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-05-08 17:49:05 +03:00
Azat Khuzhin
c757b532c6 Make DatabaseCatalog.cpp self compilable (missing AbstractConfiguration) 2021-05-06 09:07:58 +03:00
feng lv
39f68bf5ff fix conflict 2021-05-02 16:33:45 +00:00
kssenii
deb4903af8 Merge branch 'master' of github.com:ClickHouse/ClickHouse into poco-file-to-std-fs 2021-04-28 20:57:13 +03:00
kssenii
5e429b1266 Change in Interpreters/* 2021-04-28 20:52:00 +03:00
Maksim Kita
c93666e0de Fixed code review issues 2021-04-26 13:52:41 +03:00
Maksim Kita
561b045991 Fixed tests 2021-04-26 13:52:41 +03:00
feng lv
4ffe199d39 Implement table comments 2021-04-23 12:18:23 +00:00
Ivan
495c6e03aa
Replace all Context references with std::weak_ptr (#22297)
* Replace all Context references with std::weak_ptr

* Fix shared context captured by value

* Fix build

* Fix Context with named sessions

* Fix copy context

* Fix gcc build

* Merge with master and fix build

* Fix gcc-9 build
2021-04-11 02:33:54 +03:00
alexey-milovidov
b02a82250b
Merge pull request #22858 from vitlibar/fix-hanging-in-temporary-live-view-cleaner
Fix hanging in TemporaryLiveViewCleaner
2021-04-09 09:25:38 +03:00
Vitaly Baranov
5bcb7a0fed Fix hanging in TemporaryLiveViewCleaner 2021-04-08 16:14:49 +03:00
Maksim Kita
32831e37ba Added table function dictionary 2021-03-19 15:47:27 +03:00
Alexander Tokmakov
18f6b5bbad add timeouts 2021-02-04 22:41:44 +03:00
Alexander Tokmakov
066fb4c82b fix 2021-02-03 23:02:37 +03:00
Alexander Tokmakov
6456ccf0da better test 2021-02-02 22:39:04 +03:00
Alexander Tokmakov
5f0f8ae3cd fix duplicate UUIDs of LiveView on server startup 2021-01-13 21:13:46 +03:00
Mikhail Filimonov
24abcc970b
Tests and fixes 2020-12-14 15:31:13 +01:00
Ivan
0ba712a2d3
Merge branch 'master' into allow_atomic_database_inside_materialize_mysql 2020-12-08 16:45:51 +03:00
Alexey Milovidov
4ecfe8fef4 Remove unused headers 2020-12-08 02:10:22 +03:00
Alexander Tokmakov
e82c63e05c fixes 2020-11-24 15:28:54 +03:00
Alexander Tokmakov
dce0bf8efc Merge branch 'master' into allow_atomic_database_inside_materialize_mysql 2020-11-24 15:28:31 +03:00
Amos Bird
1d9d586e20
Make global_context consistent. 2020-11-20 18:23:14 +08:00
nikitamikhaylov
c0940f2ebd done 2020-11-13 02:27:18 +03:00
Alexander Tokmakov
33fa54cce6 better scheduling of drop table task 2020-11-01 20:37:54 +03:00
Alexander Tokmakov
b5ccb5ed5b review suggestions 2020-10-27 23:52:49 +03:00
tavplubix
934f64a2fd
Update DatabaseCatalog.cpp 2020-10-26 00:51:30 +03:00
Alexander Tokmakov
f8f2d62d70 minor improvements in DatabaseCatalog 2020-10-25 20:27:21 +03:00
tavplubix
790950ea56
Merge pull request #15934 from ClickHouse/ddl_queries_fixes
Fixes for DDL queries
2020-10-23 23:52:13 +03:00
Alexander Tokmakov
2f9611f1ae fix logical error with explicit UUIDs 2020-10-15 19:01:23 +03:00
Alexander Tokmakov
cb8d132cca fix deadlock with DDLGuard 2020-10-13 18:00:36 +03:00
Alexander Tokmakov
32776da69a Merge branch 'master' into allow_atomic_database_inside_materialize_mysql 2020-10-02 20:09:37 +03:00
alesapin
4ab39d2bee
Fix style 2020-09-29 18:19:30 +03:00
Alexander Tokmakov
cf86e19f38 fix build 2020-09-29 17:15:05 +03:00
Alexander Tokmakov
5d19ed04f6 allow synchronous drop/detach table for Atomic 2020-09-29 16:47:08 +03:00
Alexander Tokmakov
0ecf8e595f cleanup code, add test 2020-09-15 16:41:53 +03:00
Alexander Tokmakov
3671c91c37 Merge branch 'master' into allow_atomic_database_inside_materialize_mysql 2020-09-15 12:43:01 +03:00
Alexander Tokmakov
a812f7eb75 allow using Atomic database inside MaterializeMySQL 2020-09-14 22:25:02 +03:00
Alexander Tokmakov
b68782d285 enable more tests with Atomic database 2020-09-09 16:32:50 +03:00
Nikolai Kochetov
b7e53208d1 Fix tests. 2020-09-04 11:36:47 +03:00
Alexander Tokmakov
e686616756 add exclusive DDLGuard for database 2020-08-18 18:15:27 +03:00
Alexander Tokmakov
5b92f43f12 Merge branch 'master' into database_atomic_improvements 2020-08-03 14:35:40 +03:00
Vitaly Baranov
668653600c Use SettingMaxThreads only in Settings, call getNumberOfPhysicalCPUCores() instead of SettingMaxThreads::getAuto(). 2020-07-31 19:11:27 +03:00
Alexander Tokmakov
99f0adad0a fixes 2020-07-17 21:17:04 +03:00
Alexander Tokmakov
f6949b2f47 use dictionary ids 2020-07-15 22:25:31 +03:00
Alexander Tokmakov
9c4efa1f36 fix, add test 2020-07-09 01:50:15 +03:00
Alexander Tokmakov
7a5d532c7b implement rename database for atomic 2020-07-07 15:11:58 +03:00
Alexander Tokmakov
d1be5ec641 fix symlinks, add short syntax of attach database 2020-07-06 16:41:43 +03:00
Alexander Kuzmenkov
c678708c0c Put clickhouse-local data to /tmp by default
This fixes https://github.com/ClickHouse/ClickHouse/issues/9848
Also fixes https://github.com/ClickHouse/ClickHouse/issues/11926
2020-06-24 22:03:28 +03:00
Alexander Kuzmenkov
dacbe8e445
Update src/Interpreters/DatabaseCatalog.cpp 2020-06-04 14:06:07 +03:00
Alexander Kuzmenkov
13762854c6
Update src/Interpreters/DatabaseCatalog.cpp 2020-06-04 14:05:34 +03:00
Alexander Kuzmenkov
91b1d381bb fixup 2020-06-03 16:34:36 +03:00
Alexander Kuzmenkov
07e4bb7050 Remove assorted synonyms of LOGICAL_ERROR.
We don't need any special handling for them on the client, and, on the
contrary, have to handle them as logical errors in tests.
2020-06-03 16:18:42 +03:00
tavplubix
a4e40fb5f2
Merge pull request #11269 from ClickHouse/add_context_to_idatabase_methods
Add context to IDatabase methods
2020-05-29 21:15:35 +03:00
Alexey Milovidov
23d7947549 Fix constraints for DEFAULT fields; fix constraints for temporary tables; improve performance of INSERT for small blocks 2020-05-29 05:08:48 +03:00
Alexander Tokmakov
c071e0a473 add context to DatabaseCatalog methods 2020-05-29 02:01:18 +03:00
Alexander Tokmakov
3cff070f89 add context to IDatabase methods 2020-05-28 23:10:45 +03:00
Alexey Milovidov
7e1813825b Return old names of macros 2020-05-24 01:24:01 +03:00
Alexey Milovidov
7e2fb9ad65 Apply all transformations again 2020-05-23 22:38:30 +03:00
Alexey Milovidov
29762240de Remove duplicate whitespaces (preparation) 2020-05-23 22:31:54 +03:00
Alexey Milovidov
1f13515a65 Make all LOG in single line (preparation) 2020-05-23 21:31:37 +03:00
Alexey Milovidov
f69cbdcbfc find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" \+ [^+]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" \+ ([^+]+)\);/\1_FORMATTED(\2, "\3{}", \4);/' 2020-05-23 20:09:37 +03:00
Alexey Milovidov
8042e5febe find {base,src,programs} -name '*.h' -or -name '*.cpp' | xargs grep -l -P 'LOG_\w+\([^,]+, "[^"]+" << [^<]+ << "[^"]+" << [^<]+\);' | xargs sed -i -r -e 's/(LOG_\w+)\(([^,]+), "([^"]+)" << ([^<]+) << "([^"]+)" << ([^<]+)\);/\1_FORMATTED(\2, "\3{}\5{}", \4, \6);/' 2020-05-23 19:58:15 +03:00
tavplubix
04a24c59dc
Add tests with DatabaseAtomic (#10714)
* add tests with DatabaseAtomic

* fix
2020-05-07 14:29:58 +03:00
tavplubix
7d6ab32eb6
Update DatabaseCatalog.cpp 2020-04-24 01:08:26 +03:00
Alexander Tokmakov
8be4d51111 fix build 2020-04-23 22:50:05 +03:00
Alexander Tokmakov
f413061e0d remove Context from some IDatabase methods 2020-04-23 19:51:48 +03:00
Alexander Tokmakov
e02b5790af fix 2020-04-22 23:49:13 +03:00
Alexander Tokmakov
1833ac7f16 fixes 2020-04-22 23:43:10 +03:00
Alexander Tokmakov
d1024e76bd fix 2020-04-13 18:46:36 +03:00
Alexander Tokmakov
20e6593f96 disable by default 2020-04-12 21:52:59 +03:00
Alexander Tokmakov
f0edb65e33 add test 2020-04-11 02:02:15 +03:00
Alexander Tokmakov
140cd88c20 fixes 2020-04-09 02:59:39 +03:00