thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-04 21:42:39 +00:00
Code Issues Packages Projects Releases Wiki Activity
132,312 Commits 9,185 Branches 1,782 Tags 2.3 GiB
1593d33883
Commit Graph

1448 Commits

Author SHA1 Message Date
kssenii
76be22e8e8 Fix build without libpqxx 2023-12-11 20:16:13 +01:00
serxa
c1537a8757 Merge branch 'master' into dependency-failure 2023-12-11 16:53:47 +00:00
kssenii
48ce040621 Fix 2023-12-11 17:30:23 +01:00
Raúl Marín
a55a0c068e Merge remote-tracking branch 'blessed/master' into backup_1 2023-12-11 15:27:21 +01:00
kssenii
a06179da00 Merge remote-tracking branch 'origin/master' into fix-materialized-pg-issue-with-generated-columns 2023-12-11 11:15:00 +01:00
Azat Khuzhin
2707e38c69 Fix dead-lock in DatabaseMaterializedPostgreSQL after interface changes 
Deadlock for ATTACH:

    /src/ch/clickhouse/src/Common/ThreadFuzzer.cpp:0::pthread_mutex_lock
    /src/ch/clickhouse/contrib/llvm-project/libcxx/src/mutex.cpp:39::std::__1::mutex::lock()
    /src/ch/clickhouse/src/Databases/DatabaseOrdinary.cpp:240::DB::DatabaseOrdinary::waitTableStarted(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) const
    /src/ch/clickhouse/src/Databases/DatabasesCommon.cpp:203::DB::DatabaseWithOwnTablesBase::tryGetTable(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context const>) const
    /src/ch/clickhouse/src/Databases/PostgreSQL/DatabaseMaterializedPostgreSQL.cpp:335::DB::DatabaseMaterializedPostgreSQL::attachTableUnlocked(std::__1::shared_ptr<DB::Context const>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::IStorage> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&)
    /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__mutex_base💯:DB::IDatabase::attachTable(std::__1::shared_ptr<DB::Context const>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::IStorage> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&)
    /src/ch/clickhouse/contrib/llvm-project/libcxx/include/string:1499::DB::DatabaseMaterializedPostgreSQL::createTable(std::__1::shared_ptr<DB::Context const>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::IStorage> const&, std::__1::shared_ptr<DB::IAST> const&)
    /src/ch/clickhouse/contrib/llvm-project/libcxx/include/string:1499::DB::InterpreterCreateQuery::doCreateTable(DB::ASTCreateQuery&, DB::InterpreterCreateQuery::TableProperties const&, std::__1::unique_ptr<DB::DDLGuard, std::__1::default_delete<DB::DDLGuard>>&)
    /src/ch/clickhouse/src/Interpreters/InterpreterCreateQuery.cpp:0::DB::InterpreterCreateQuery::createTable(DB::ASTCreateQuery&)
    /src/ch/clickhouse/src/Interpreters/InterpreterCreateQuery.cpp:1751::DB::InterpreterCreateQuery::execute()
    /src/ch/clickhouse/src/Interpreters/executeQuery.cpp:0::DB::executeQueryImpl(char const*, char const*, std::__1::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*)
    /src/ch/clickhouse/src/Interpreters/executeQuery.cpp:1287::DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum)
    /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612::DB::TCPHandler::runImpl()
    /src/ch/clickhouse/src/Server/TCPHandler.cpp:2294::DB::TCPHandler::run()
    /src/ch/clickhouse/base/poco/Net/src/TCPServerConnection.cpp:57::Poco::Net::TCPServerConnection::start()
    /src/ch/clickhouse/contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48::Poco::Net::TCPServerDispatcher::run()
    /src/ch/clickhouse/base/poco/Foundation/src/ThreadPool.cpp:202::Poco::PooledThread::run()
    /src/ch/clickhouse/base/poco/Foundation/include/Poco/SharedPtr.h:139::Poco::ThreadImpl::runnableEntry(void*)

CI: https://s3.amazonaws.com/clickhouse-test-reports/55271/ece8c86ebc02908e36786ca32d68f9ce0ddcb35d/integration_tests__tsan__[1_6].html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-10 16:06:18 +01:00
Azat Khuzhin
ece8c86ebc Fix attaching table in Atomic engine after attachTableUnlocked() interface 2023-12-10 08:28:10 +01:00
Azat Khuzhin
c14721bb03 Move lazy table initialization interface into IDatabase 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-09 23:08:29 +01:00
Azat Khuzhin
f75484a0b1 Make IDatabase::attachTableUnlocked() virtual and use it for overrides (mostly) 2023-12-09 23:08:29 +01:00
Azat Khuzhin
cc9afe6f6e Replace --no-system-tables with loading virtual tables of system database lazily 
First of all the problem is that that simple 'SELECT 1' cannot be runned
without system.one, which makes --no-system-tables almost useless:

    $ ./clickhouse-debug local --no-system-tables -q "select 1"
    Code: 81. DB::Exception: Database system does not exist. (UNKNOWN_DATABASE)

Secondly there are just too much flags, and this one
(--no-system-tables) is too damn specific.

This patch should improve startup time of the clickhouse-local almost
3x in debug builds.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-12-09 23:08:29 +01:00
serxa
58138861ba fix style 2023-12-08 22:34:47 +00:00
serxa
1e7407f82c Customazable dependency failure handling for AsyncLoader 2023-12-08 20:14:55 +00:00
kssenii
ffd0d16f79 Merge remote-tracking branch 'origin/master' into fix-materialized-pg-issue-with-generated-columns 2023-12-08 12:42:21 +01:00
Mike Kot
4ef9065567 fix conversion error while getting psql table structure for empty array 2023-12-07 13:38:32 +00:00
凌涛
d0675488ac rebase 2023-12-07 11:14:55 +08:00
Ryan Jacobs
6a698d9480
clang-tidy 2023-12-06 18:56:54 -08:00
Ryan Jacobs
b57340bde2
postgresql integration: Throw errors instead of assuming array_ndim == 1 2023-12-06 11:54:45 -08:00
Kseniia Sumarokova
9d8e3f8bd9
Update fetchPostgreSQLTableStructure.cpp 2023-12-06 16:47:58 +01:00
kssenii
806061642a Support GENERATED and DEFAULT columns 2023-12-06 15:01:43 +01:00
凌涛
9e05e79d66
Merge branch 'master' into feature/more_warnings 2023-12-05 11:50:05 +08:00
Alexey Milovidov
02439eee58
Merge pull request #49351 from ClickHouse/async-loader-integration 
Asynchronous loading of tables
 2023-12-04 17:16:12 +01:00
Ryan Jacobs
17aacda9e5
fix: format code for stylecheck 2023-12-02 06:46:16 -08:00
Bharat Nallan Chakravarthy
efbf39ea06 improve TableNameHints 2023-12-02 00:45:21 -08:00
Ryan Jacobs
a90458eb65
Fix several issues regarding PostgreSQL array_ndims usage. 
1. Properly quote column identifiers.
2. Handle empty tables that have array columns.
3. Throw a more user friendly error when column value = NULL
   when calling array_dims(column)
4. Handle column value being the empty array {}
 2023-12-01 13:15:24 -08:00
凌涛
0b3cfcc8d4
Update DatabaseLazy.cpp 2023-12-01 11:23:31 +08:00
凌涛
2ae934ec05
Update DatabasesCommon.cpp 2023-12-01 10:51:32 +08:00
Vitaly Baranov
1bc1563e0e Stop using INCONSISTENT_METADATA_FOR_BACKUP that much. If possible prefer to continue scanning instead of stopping and starting the scanning for backup again. 2023-11-30 21:37:42 +01:00
凌涛
d9f4b4d2c0 support parts, tables, databases 2023-11-30 10:56:31 +08:00
Alexander Tokmakov
9f849fbe06 fix 2023-11-29 17:12:04 +01:00
Bharat Nallan Chakravarthy
4ca3d0d5c4 better hints when table doesn't exist 2023-11-28 21:34:02 -08:00
Alexander Tokmakov
67f0a0e3e4 don't throw if noop when dropping db replica in batch 2023-11-28 23:57:55 +01:00
serxa
8d8418ada8 get rid of defaults to make clang-tidy happy 2023-11-28 18:21:47 +00:00
凌涛
98a03ac36c support table 2023-11-28 18:28:29 +08:00
serxa
9d6a9d3573 do not rethrow load errors during db shutdown 2023-11-26 19:31:39 +00:00
Alexey Milovidov
9fa112af9a
Merge pull request #53721 from takakawa/possible_wrong_type_conversion_bugfix 
[bugfix] possible postgresql logical replication error: wrong type conversion
 2023-11-26 02:48:24 +01:00
serxa
6937c0de19 wait for database to load before shutdown 2023-11-25 15:42:37 +00:00
serxa
0fc25fbda8 Merge branch 'master' into async-loader-integration 2023-11-23 09:32:43 +00:00
Raúl Marín
281060329f Merge remote-tracking branch 'blessed/master' into backup_1 2023-11-22 10:35:18 +01:00
Raúl Marín
a1bb31525a
Merge pull request #57029 from ClickHouse/fix_DatabaseFilesystem_path_validation 
Fix file path validation for DatabaseFileSystem
 2023-11-22 09:19:19 +01:00
santrancisco
f4d936fc55
Fix style 2023-11-21 17:39:12 +11:00
santrancisco
3eef0601e3
Fix file path validation for DatabaseFileSystem 2023-11-21 16:29:40 +11:00
Raúl Marín
44874859bb Fix style 2023-11-20 12:21:12 +01:00
Raúl Marín
aadb786b2b Merge remote-tracking branch 'blessed/master' into backup_1 2023-11-20 10:53:31 +01:00
santrancisco
f05f572b3c
Fixed trailing whitespace 2023-11-20 13:37:27 +11:00
santrancisco
ab5f3d12b6
Fix sqlite file path validation to make sure it does not skip validation on relative path 2023-11-20 12:51:30 +11:00
serxa
9023f61349 Merge branch 'master' into async-loader-integration 2023-11-19 13:05:58 +00:00
Alexey Milovidov
09f263e281
Update fetchPostgreSQLTableStructure.cpp 2023-11-19 13:20:01 +03:00
Alexey Milovidov
d56cbda185 Add metrics for the number of queued jobs, which is useful for the IO thread pool 2023-11-18 19:07:59 +01:00
Nikolay Degterinsky
3828131dc6
Merge pull request #56828 from evillique/fix-joinget-dependency 
Fix failure to start due to table dependency in joinGet
 2023-11-18 15:18:22 +01:00
Raúl Marín
210a0ee045 Merge remote-tracking branch 'blessed/master' into backup_1 2023-11-17 18:50:24 +01:00
Alexander Tokmakov
ac31007475
Update DatabaseReplicated.cpp (#56796) 2023-11-16 21:00:39 +01:00
Raúl Marín
19931fee67 Prepare the introduction of more keeper faults 2023-11-16 15:46:39 +01:00
Nikolay Degterinsky
1d7eecaeec Fix failure to start due to table dependency in joinGet 2023-11-16 06:08:12 +00:00
Alexey Milovidov
9bf62dd622 Remove orphan header files 2023-11-14 07:43:11 +01:00
Nikolay Degterinsky
02ce2a0e83
Merge pull request #56489 from evillique/fix-drop-ttl-dependency 
Fix startup failure due to TTL dependency
 2023-11-13 17:40:19 +01:00
Kseniia Sumarokova
d3b6310fe1
Merge pull request #56609 from jsc0218/LessAggressiveRescheduleWithUnsuccessfulConn 
Less aggressive reschedule with unavailable conn of PG
 2023-11-13 09:01:23 +01:00
jsc0218
9c954046af style fix again 2023-11-12 23:23:56 +00:00
jsc0218
c63a6f5cf5 change comment style 2023-11-12 19:31:38 +00:00
Robert Schulze
3f49e1c5ed
Merge remote-tracking branch 'rschu1ze/master' into pcg 2023-11-12 11:43:19 +00:00
jsc0218
b5a79b1bda less aggressive reschedule when unavailable connection exists, to avoid repeated interrupting other normal routines 2023-11-11 02:20:58 +00:00
Nikolay Degterinsky
05945ff80d
Merge pull request #56486 from evillique/fix-replicated-database-cluster-port 
Fix default port for Replicated database cluster
 2023-11-10 13:37:45 +01:00
Kseniia Sumarokova
ecd98006ce
Merge pull request #56306 from ClickHouse/fix-backup-restore-flatten-nested 
Fix restore from backup with `flatten_nested` and `data_type_default_nullable`
 2023-11-10 11:43:17 +01:00
kssenii
41a880e57c Review fix 2023-11-09 16:03:51 +01:00
Nikolay Degterinsky
895ab2e57a
Fix style 2023-11-09 06:06:26 +01:00
Nikolay Degterinsky
44eb73980f Fix startup failure due to TTL dependency 2023-11-09 04:31:49 +00:00
Nikolay Degterinsky
881dde00b2 Fix default port for Replicated database cluster 2023-11-08 23:40:54 +00:00
Robert Schulze
e46dbcb2f1
Use pcg/randomSeed() random generator/seed instead of std::mt19937/std::random_device 2023-11-07 18:54:50 +00:00
kssenii
9178fd4ad1 Fix case with replicated database 2023-11-07 16:02:51 +01:00
serxa
19aa1375fc Merge branch 'master' into async-loader-integration 2023-11-07 10:25:38 +00:00
serxa
d50747847d rework AsyncLoaderPoolId.h to match settings and metrics 2023-11-06 13:08:27 +00:00
serxa
7e220182eb better 2023-11-03 16:41:32 +00:00
kssenii
ea5796fcb6 Fix 2023-11-03 16:46:55 +01:00
serxa
d0b93d90c7 wait for db to be started iff it was async-started 2023-11-03 15:31:32 +00:00
serxa
9f954ece81 fix 2023-11-02 14:55:53 +00:00
Alexander Tokmakov
977380b171
Update DatabaseReplicated.cpp 2023-11-01 19:53:08 +01:00
serxa
59c0f4c652 remove wrong waits 2023-10-27 10:45:39 +00:00
serxa
be11170a12 implement waits inside tables iterator 2023-10-27 10:08:00 +00:00
Sergei Trifonov
a5d7c2b6a2
Merge branch 'master' into async-loader-integration 2023-10-27 10:39:12 +02:00
Alexander Tokmakov
c588a7f386 Merge branch 'master' into make_code_worse 2023-10-26 17:39:10 +02:00
Alexander Tokmakov
fa963d60d0
Merge pull request #55641 from evillique/resubmit-cluster-groups 
Resubmit #54421
 2023-10-26 15:53:53 +02:00
Nikolay Degterinsky
d5ee863594 Fix 2023-10-25 19:06:50 +00:00
Nikolay Degterinsky
97c99132e0 Simplify the implementation 2023-10-25 19:02:48 +00:00
Alexander Tokmakov
0f4260bf26 fix some bad code by making it worse 2023-10-25 19:51:51 +02:00
takakawa
fe981ed8db
Merge branch 'ClickHouse:master' into possible_wrong_type_conversion_bugfix 2023-10-25 17:57:58 +08:00
serxa
08e63f2a24 fix build 2023-10-24 21:49:15 +00:00
serxa
02b6497167 get rid of atomics 2023-10-24 12:08:27 +00:00
serxa
749cdb9954 Merge branch 'master' into async-loader-integration 2023-10-24 10:09:49 +00:00
serxa
d3f54a29e3 add startup waits in all DDL queries and global DDL worker 2023-10-24 10:09:10 +00:00
Sergei Trifonov
966f1b6378
Merge pull request #55877 from ClickHouse/serxa-patch-3 
Fix digest check in replicated ddl worker
 2023-10-23 14:25:20 +02:00
Anton Popov
5819bcd07a
Support asynchronous inserts for native protocol (#54730) 
* support async insert for native protocol

* use separate queue for async inserts via native protocol

* fix test

* better logging for async inserts and more tests

* disable mixed internal and external data in async inserts

* fix tests

* fix quota in async inserts

* disable async insert for secondary query of distributed
 2023-10-20 18:39:48 +02:00
Sergei Trifonov
fd93bc74bd
Fix digest check in replicated ddl worker 2023-10-20 18:03:16 +02:00
kssenii
4bc32c6387 Fix uncaught exception in background tasj 2023-10-16 17:31:30 +02:00
serxa
0e17da22eb Merge branch 'master' into async-loader-integration 2023-10-16 11:08:29 +00:00
kssenii
3cd175d11d Fix test 2023-10-16 13:01:36 +02:00
Nikolay Degterinsky
b737866d02 Fixes 2023-10-14 19:29:21 +00:00
Nikolay Degterinsky
c1fba94d67 Resubmit #54421 
This reverts commit 3e60451ac7.
 2023-10-14 19:04:01 +00:00
Kseniia Sumarokova
310e922b40
Merge branch 'master' into materialized-pg-allow-unique-identifier 2023-10-13 19:14:50 +02:00
Alexander Tokmakov
3e60451ac7 Revert "Merge pull request #54421 from evillique/cluster-groups" 
This reverts commit 211c80e1a0, reversing
changes made to b53a98217c.
 2023-10-13 18:55:51 +02:00
Alexander Tokmakov
83daf766c9 Revert "Merge pull request #55587 from azat/fix-replica-group" 
This reverts commit ebfad5c8b8, reversing
changes made to e64abfc557.
 2023-10-13 18:55:22 +02:00
Azat Khuzhin
db1b9408f3 Fix replica groups for Replicated database engine 
Should fix 02447_drop_database_replica test

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-10-13 12:32:17 +02:00