* Add a codec Encrypted() for encrypting columns on disk
While this is implemented as a compression codec, it does not actually compress data. It instead encrypts data on disk. The key is obtained by executing a user-specified command at the server startup, or if it's not specified the codec refuses to process any data. For now the only supported cipher is 'AES-128-GCM-SIV'.
* Try to enforce table identification in CollectJoinOnKeysMatcher
* Support filtering conditions in JOIN ON for HashJoin
* Correct handle non equi join
* Update test 00878_join_unexpected_results
* Join on filters calculated as one row before join
* Do not lookup key in hash join if condition for row is not hold
* better
* Support filtering conditions in JOIN ON for MergeJoin
* Support Nullable mask in JOIN ON section
* Fix style in Interpreters/TableJoin.cpp
* Change return type of getColumnAsMask in join_common to ColumnPtr
* Handle Nullable(Nothing) type in JOIN ON section, add test cases
* Fix type cast JoinCommon::getColumnAsMask
* Check type if conditions in JOIN ON section, support functions
* Update tests with JOIN ON
* Style changes, add comments for conditions in JOIN ON section
* Add test cases for join on condtions
* JOIN ON key1 = key2 AND (cond1 OR cond2)
* Remove CollectJoinOnKeysVisitor has_join_keys
* Add test cases for join on nullable/lc conditions
* Fix style
* Change error code 48 to 403 in join on tests
* Fix whitespace
Should fix the following exception under osx:
Stack size too large. Stack address: 0x700001a58000, frame address: 0x700001a53f40, stack size: 540864, maximum stack size: 524288 (version 21.7.1.1)
Broken batches may be because of abnormal server shutdown (and lack of
fsync), and ignoring the whole batch is not great in this case, so apply
the same split logic here too.
v2: rename exception
v3: catch missing exception
v4: fix marking the file as broken multiple times (fixes
test_insert_distributed_async_send with setting enabled)
Add distributed_directory_monitor_split_batch_on_failure setting (OFF by
default), that will split the batch and send files one by one in case of
retriable errors.
v2: more error codes
* initial commit: add setting and stub
* typo
* added test stub
* fix
* wip merging new integration test and code proto
* adding steps interpreters
* adding firstly proposed solution (moving parts etc)
* added checking zookeeper path existence
* fixing the include
* fixing and sorting includes
* fixing outdated struct
* fix the name
* added ast ptr as level of indirection
* fix ref
* updating the changes
* working on test stub
* fix iterator -> reference
* revert rocksdb submodule update
* fixed show privileges test
* updated the test stub
* replaced rand() with thread_local_rng(), updated the tests
updated the test
fixed test config path
test fix
removed error messages
fixed the test
updated the test
fixed string literal
fixed literal
typo: =
* fixed the empty replica error message
* updated the test and the code with logs
* updated the possible test cases, updated
* added the code/test milestone comments
* updated the test (added more testcases)
* replaced native assert with CH one
* individual replicas recursive delete fix
* updated the AS db.name AST
* two small logging fixes
* manually generated AST fixes
* Updated the test, added the possible algo change
* Some thoughts about optimizing the solution:
ALTER MOVE PARTITION .. TO TABLE -> move to detached/ + ALTER ... ATTACH
* fix
* Removed the replica sync in test as it's invalid
* Some test tweaks
* tmp
* Rewrote the algo by using the executeQuery instead of
hand-crafting the ASTPtr.
Two questions still active.
* tr: logging active parts
* Extracted the parts moving algo into a separate helper function
* Fixed the test data and the queries slightly
* Replaced query to system.parts to direct invocation,
started building the test that breaks on various parts.
* Added the case for tables when at least one replica is alive
* Updated the test to test replicas restoration by detaching/attaching
* Altered the test to check restoration without replica restart
* Added the tables swap in the start if the server failed last time
* Hotfix when only /replicas/replica... path was deleted
* Restore ZK paths while creating a replicated MergeTree table
* Updated the docs, fixed the algo for individual replicas restoration case
* Initial parts table storage fix, tests sync fix
* Reverted individual replica restoration to general algo
* Slightly optimised getDataParts
* Trying another solution with parts detaching
* Rewrote algo without any steps, added ON CLUSTER support
* Attaching parts from other replica on restoration
* Getting part checksums from ZK
* Removed ON CLUSTER, finished working solution
* Multiple small changes after review
* Fixing parallel test
* Supporting rewritten form on cluster
* Test fix
* Moar logging
* Using source replica as checksum provider
* improve test, remove some code from parser
* Trying solution with move to detached + forget
* Moving all parts (not only Committed) to detached
* Edited docs for RESTORE REPLICA
* Re-merging
* minor fixes
Co-authored-by: Alexander Tokmakov <avtokmakov@yandex-team.ru>
But note, that tests with EXPECT_EXIT() does not work with --gtest_filter:
a) only this test is included into unit_tests_dbms
$ src/unit_tests_dbms --gtest_filter=VersionNumber*
Running main() from ../contrib/googletest/googletest/src/gtest_main.cc
Note: Google Test filter = VersionNumber*
[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from VersionNumber
[ RUN ] VersionNumber.VersionNumber
[ OK ] VersionNumber.VersionNumber (0 ms)
[ RUN ] VersionNumber.fromString
Segmentation fault (core dumped)
(gdb) bt
0 0x00007fffda323d22 in raise () from /usr/lib/libc.so.6
1 0x00007fffda30d862 in abort () from /usr/lib/libc.so.6
2 0x00007fffdbe686f5 in DB::handle_error_code (msg=..., code=49, remote=false, trace=...) at Exception.cpp:49
3 0x00007fffdbe68846 in DB::Exception::Exception (this=0x7fffd5c5a240, msg=..., code=49, remote_=false) at Exception.cpp:60
4 0x00007fffe26b2cb3 in DB::Exception::Exception<unsigned long> (this=0x7fffd5c5a240, code=49, fmt=..., args=@0x7fffffffc7e8: 4) at Exception.h:40
5 0x00007fffdbf4d201 in DB::VersionNumber::VersionNumber (this=0x7fffffffcc20, vec=...) at VersionNumber.cpp:17
6 0x00007fffdbf4d650 in DB::VersionNumber::fromString (version=..., strict=true) at VersionNumber.cpp:57
7 0x00005555555db53d in VersionNumber_fromString_Test::TestBody (this=0x7fffd5c12330) at gtest_version_number.cpp:24
b) regular build
$ src/unit_tests_dbms --gtest_filter=VersionNumber*
Running main() from ../contrib/googletest/googletest/src/gtest_main.cc
Note: Google Test filter = VersionNumber*
[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from VersionNumber
[ RUN ] VersionNumber.VersionNumber
[ OK ] VersionNumber.VersionNumber (0 ms)
[ RUN ] VersionNumber.fromString
[ OK ] VersionNumber.fromString (495 ms)
[----------] 2 tests from VersionNumber (495 ms total)
[----------] Global test environment tear-down
Segmentation fault (core dumped)
(gdb) bt
0 testing::TestInfo::should_run (this=0xe0) at gtest.h:760
1 0x00007ffff7f6edd5 in testing::TestSuite::ShouldRunTest (test_info=0xe0) at gtest.h:1009
2 0x00007ffff7f6ebb2 in testing::internal::CountIf<std::__1::vector<testing::TestInfo*, std::__1::allocator<testing::TestInfo*> >, bool (*)(testing::TestInfo const*)> (c=..., predicate=0x7ffff7f6edc0 <testing::TestSuite::ShouldRunTest(testing::TestInfo const*)>)
at gtest-internal-inl.h:294
3 0x00007ffff7f41ae5 in testing::TestSuite::test_to_run_count (this=0x7fffd5028500) at gtest.cc:2919
4 0x00007ffff7f41719 in testing::internal::SumOverTestSuiteList (case_list=..., method=(int (testing::TestSuite::*)(const testing::TestSuite * const)) 0x7ffff7f41ac0 <testing::TestSuite::test_to_run_count() const>) at gtest.cc:377
5 0x00007ffff7f41ab9 in testing::internal::UnitTestImpl::test_to_run_count (this=0x7fffd5d5c000) at gtest.cc:1012
6 0x00007ffff7f4b59d in testing::UnitTest::test_to_run_count (this=0x7ffff7fbcb80 <testing::UnitTest::GetInstance()::instance>) at gtest.cc:5059
7 0x00007ffff7f4c4ed in testing::internal::PrettyUnitTestResultPrinter::OnTestIterationEnd (this=0x7fffd5ca4a68, unit_test=...) at gtest.cc:3589
8 0x00007ffff7f4daf9 in testing::internal::TestEventRepeater::OnTestIterationEnd (this=0x7fffd5c83770, unit_test=..., iteration=0) at gtest.cc:3852
9 0x00007ffff7f57570 in testing::internal::UnitTestImpl::RunAllTests (this=0x7fffd5d5c000) at gtest.cc:5737
10 0x00007ffff7fa3a84 in testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (object=0x7fffd5d5c000,
method=(bool (testing::internal::UnitTestImpl::*)(testing::internal::UnitTestImpl * const)) 0x7ffff7f56f20 <testing::internal::UnitTestImpl::RunAllTests()>, location=0x7ffff7f11d9b "auxiliary test code (environments or event listeners)") at gtest.cc:2589
11 0x00007ffff7f71788 in testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool> (object=0x7fffd5d5c000,
method=(bool (testing::internal::UnitTestImpl::*)(testing::internal::UnitTestImpl * const)) 0x7ffff7f56f20 <testing::internal::UnitTestImpl::RunAllTests()>, location=0x7ffff7f11d9b "auxiliary test code (environments or event listeners)") at gtest.cc:2625
12 0x00007ffff7f56ea3 in testing::UnitTest::Run (this=0x7ffff7fbcb80 <testing::UnitTest::GetInstance()::instance>) at gtest.cc:5291
The #22109 adds the check but at compilation time, which is pointless,
move the check into runtime.
Remember nested epoll is required for
async_socket_for_remote/use_hedged_requests, otherwise remote queries
may stuck.
I'm using such hack to hide the password:
$ clickhouse-client --config <(echo '<config><password></password></config>')
But this had been broken in #21858
v2: add a test
v3: fail for unsupported extensions
