thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-23 16:12:01 +00:00
Code Issues Packages Projects Releases Wiki Activity
119,459 Commits 9,168 Branches 1,781 Tags 2.1 GiB
203ec1d230
Commit Graph

119459 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
lgbo
344bc05156
wip: reserve hash table size (#50875) 2023-07-10 10:44:01 +02:00
Raúl Marín
dbdfc07790
Merge branch 'master' into async_to_query_log 2023-07-10 10:32:56 +02:00
Victor Krasnov
e311d31d87 Merge branch 'master' into ADQM-950-B 2023-07-10 15:50:09 +08:00
Alexey Gerasimchuck
2c8564e45e
Merge branch 'master' into ADQM-970 2023-07-10 15:53:08 +10:00
robot-ch-test-poll3
cb26301a5a
Merge pull request #51996 from simpl1g/patch-1 
[DOCS] Add REMOVE SAMPLE BY to docs
 2023-07-10 04:59:34 +02:00
Alexey Milovidov
4509939572
Merge branch 'master' into add-test-34626 2023-07-10 01:26:04 +03:00
Alexey Milovidov
7b3b6531ee
Merge pull request #51305 from ClickHouse/add-test-44816 
Add a test for #44816
 2023-07-10 01:24:53 +03:00
Alexey Milovidov
dd1c528d2f
Merge pull request #51985 from azat/cleanup-remote-servers 
[RFC] Cleanup remote_servers in dist config.xml
 2023-07-10 01:24:06 +03:00
Alexey Milovidov
1daa8828ce
Merge pull request #51990 from ClickHouse/add-check-for-stateful-tests 
Add a check to validate that the stateful tests are stateful
 2023-07-10 00:59:21 +03:00
Alexey Milovidov
dd80bdb859
Merge pull request #51993 from azat/symbol-index-cleanup 
Cleanup SymbolIndex after reload got removed
 2023-07-10 00:58:16 +03:00
vdimir
790b438b7c
add test_delete_race_leftovers 2023-07-09 16:19:00 +00:00
Azat Khuzhin
2db092f9d8 Cleanup remote_servers in dist config.xml 
At first, there was no such amount of clusters in dist config, they
added when someone need to write some new cluster for tests.

So let's move them to the clusters.xml that is deployed only for tests,
and leave only default cluster.

And cleanup also some configs that had been copied from dist config in
the repo (about test_config_* integration tests, this should be OK,
since there are more_clusters.xml as well, that covers additional
cases).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-09 16:55:27 +02:00
Han Fei
03201bb3ec
Merge pull request #51795 from ClickHouse/revert-51390-revert-50951 
Revert "Revert "Merge pull request #50951 from ZhiguoZh/20230607-toyear-fix""
 2023-07-09 16:48:35 +02:00
Han Fei
4f48d92db4
Merge branch 'master' into revert-51390-revert-50951 2023-07-09 15:16:50 +02:00
robot-clickhouse
96d3060349
Merge pull request #51646 from amosbird/clickhouse_as_library_2 
Make common macros extendable
 2023-07-09 14:42:58 +02:00
Konstantin Ilchenko
3b954a2952
[DOCS] Add REMOVE SAMPLE BY to docs 2023-07-09 14:38:16 +02:00
Azat Khuzhin
3c18a181c9 Fix using of pools from the main thread 
Otherwise it is not possible to use clickhouse-disks with S3:

    $ clickhouse-disks -C /src/ch/clickhouse/tests/queries/0_stateless/02802_clickhouse_disks_s3_copy.xml --log-level test --disk s3_plain_native_copy copy default/test default/test.copy
    Failed to make request to: http://localhost:11111/test?list-type=2&max-keys=1&prefix=clickhouse-disks%2Fdefault%2Ftest.copy: Code: 49. DB::Exception: current_thread is not initialized. (LOGICAL_ERROR), Stack trace (when copying this message, always include the lines below):

    0. ./.cmake-llvm16/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000001ad7c872 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    1. ./.cmake-llvm16/./src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000011e2c4b7 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    2. ./.cmake-llvm16/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const (&) [34], void>(int, char const (&) [34]) @ 0x000000000d341e58 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    3. ./.cmake-llvm16/./src/Common/MemoryTrackerSwitcher.h:19: DB::(anonymous namespace)::SingleEndpointHTTPSessionPool::allocObject() @ 0x0000000012010e5a in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    4. ./.cmake-llvm16/./src/Common/PoolBase.h:174: PoolBase<Poco::Net::HTTPClientSession>::get(long) @ 0x0000000012011a6f in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    5. ./.cmake-llvm16/./contrib/llvm-project/libcxx/include/variant:797: DB::makePooledHTTPSession(Poco::URI const&, Poco::URI const&, DB::ConnectionTimeouts const&, unsigned long, bool, bool) @ 0x000000001200ec69 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    6. ./.cmake-llvm16/./src/IO/HTTPCommon.cpp:0: DB::makePooledHTTPSession(Poco::URI const&, DB::ConnectionTimeouts const&, unsigned long, bool, bool) @ 0x000000001200d909 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    7. ./.cmake-llvm16/./contrib/llvm-project/libcxx/include/variant:797: void DB::S3::PocoHTTPClient::makeRequestInternalImpl<true>(Aws::Http::HttpRequest&, DB::S3::ClientConfigurationPerRequest const&, std::shared_ptr<DB::S3::PocoHTTPResponse>&, Aws::Utils::RateLimits::RateLimiterInterface*, Aws::Utils::RateLimits::RateLimiterInterface*) const @ 0x00000000163f5157 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    8. ./.cmake-llvm16/./contrib/llvm-project/libcxx/include/string:1499: DB::S3::PocoHTTPClient::makeRequestInternal(Aws::Http::HttpRequest&, std::shared_ptr<DB::S3::PocoHTTPResponse>&, Aws::Utils::RateLimits::RateLimiterInterface*, Aws::Utils::RateLimits::RateLimiterInterface*) const @ 0x00000000163f465d in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    9. ./.cmake-llvm16/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:622: DB::S3::PocoHTTPClient::MakeRequest(std::shared_ptr<Aws::Http::HttpRequest> const&, Aws::Utils::RateLimits::RateLimiterInterface*, Aws::Utils::RateLimits::RateLimiterInterface*) const @ 0x00000000163f4454 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    10. ./.cmake-llvm16/./contrib/aws/src/aws-cpp-sdk-core/source/client/AWSClient.cpp:506: Aws::Client::AWSClient::AttemptOneRequest(std::shared_ptr<Aws::Http::HttpRequest> const&, Aws::AmazonWebServiceRequest const&, char const*, char const*, char const*) const @ 0x000000001ae2a922 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    11. ./.cmake-llvm16/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: Aws::Client::AWSClient::AttemptExhaustively(Aws::Http::URI const&, Aws::AmazonWebServiceRequest const&, Aws::Http::HttpMethod, char const*, char const*, char const*) const @ 0x000000001ae28299 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    12. ./.cmake-llvm16/./contrib/aws/src/aws-cpp-sdk-core/include/aws/core/utils/Outcome.h:160: Aws::Client::AWSXMLClient::MakeRequest(Aws::Http::URI const&, Aws::AmazonWebServiceRequest const&, Aws::Http::HttpMethod, char const*, char const*, char const*) const @ 0x000000001ae3c9ed in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    13. ./.cmake-llvm16/./contrib/aws/src/aws-cpp-sdk-core/source/client/AWSXmlClient.cpp:66: Aws::Client::AWSXMLClient::MakeRequest(Aws::AmazonWebServiceRequest const&, Aws::Endpoint::AWSEndpoint const&, Aws::Http::HttpMethod, char const*, char const*, char const*) const @ 0x000000001ae3c995 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    14. ./.cmake-llvm16/./contrib/aws/generated/src/aws-cpp-sdk-s3/source/S3Client.cpp:0: Aws::S3::S3Client::ListObjectsV2(Aws::S3::Model::ListObjectsV2Request const&) const @ 0x000000001aee6666 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    15. ./.cmake-llvm16/./contrib/aws/src/aws-cpp-sdk-core/include/aws/core/utils/Outcome.h:160: DB::S3::Client::ListObjectsV2(DB::S3::ExtendedRequest<Aws::S3::Model::ListObjectsV2Request> const&) const @ 0x00000000163cee42 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    16. ./.cmake-llvm16/./contrib/aws/src/aws-cpp-sdk-core/include/aws/core/utils/Outcome.h:120: DB::S3ObjectStorage::listObjects(String const&, std::vector<DB::RelativePathWithMetadata, std::allocator<DB::RelativePathWithMetadata>>&, int) const @ 0x0000000016b582e2 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    17. ./.cmake-llvm16/./contrib/llvm-project/libcxx/include/vector:543: DB::IObjectStorage::existsOrHasAnyChild(String const&) const @ 0x000000001644ebe9 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    18. ./.cmake-llvm16/./src/Disks/ObjectStorages/MetadataStorageFromPlainObjectStorage.cpp:0: DB::MetadataStorageFromPlainObjectStorage::exists(String const&) const @ 0x0000000016b54a64 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    19. ./.cmake-llvm16/./src/Disks/IDisk.cpp:145: DB::IDisk::copyDirectoryContent(String const&, std::shared_ptr<DB::IDisk> const&, String const&) @ 0x0000000016b38fa0 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    20. ./.cmake-llvm16/./contrib/llvm-project/libcxx/include/string:1499: DB::CommandCopy::execute(std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::Context>&, Poco::Util::LayeredConfiguration&) @ 0x0000000012050403 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    21. ./.cmake-llvm16/./contrib/llvm-project/libcxx/include/vector:434: DB::DisksApp::main(std::vector<String, std::allocator<String>> const&) @ 0x000000001204bf02 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    22. ./.cmake-llvm16/./base/poco/Util/src/Application.cpp:0: Poco::Util::Application::run() @ 0x000000001ac7a666 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    23. ./.cmake-llvm16/./programs/disks/DisksApp.cpp:0: mainEntryClickHouseDisks(int, char**) @ 0x000000001204c550 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    24. ./.cmake-llvm16/./programs/main.cpp:0: main @ 0x000000000cfbadc4 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    25. ? @ 0x00007ffff7dc9850 in ?
    26. __libc_start_main @ 0x00007ffff7dc990a in ?
    27. _start @ 0x000000000cfba1ee in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
     (version 23.7.1.1)
    AWSXmlClient: HTTP response code: -1
    Resolved remote host IP address:
    Request ID:
    Exception name:
    Error message: Code: 49. DB::Exception: current_thread is not initialized. (LOGICAL_ERROR) (version 23.7.1.1)
    0 response headers:
    If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.
    Request failed, now waiting 1600 ms before attempting again.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-09 14:21:27 +02:00
robot-ch-test-poll1
38c1e136b8
Merge pull request #51883 from qoega/sqlloogic-improvements 
Sqlloogic improvements
 2023-07-09 10:57:24 +02:00
Azat Khuzhin
d520413454 Cleanup SymbolIndex after reload got removed 
Remove MultiVersion for SymbolIndex structure since after #51873 it is
useless.

Follow-up for: #51873
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-09 09:25:11 +02:00
Azat Khuzhin
ac972661f9 Add exclusion for "API mode: {}" from S3 in 00002_log_and_exception_messages_formatting 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-09 08:21:18 +02:00
Azat Khuzhin
b958363630 tests: temporary fix the 02802_clickhouse_disks_s3_copy 
In #51135 the behavior of the `clickhouse-disks copy` had been changed,
let's temporary update the test (and continue discussion about this
change in that PR).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-09 08:20:03 +02:00
Azat Khuzhin
1844ac37d7 Remove create_read_buffer argument for copyS3File() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-09 08:20:03 +02:00
Azat Khuzhin
559d328178 Rename BACKUP setting native_copy to allow_s3_native_copy 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-09 08:20:03 +02:00
Azat Khuzhin
1590ffa3b1 Remove copyS3FileNative() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-09 08:20:03 +02:00
Azat Khuzhin
29dc9abfca Fix test_backup_restore_s3 after logging for native copying changed 
Check profile events instead of some odd logs.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-09 08:20:03 +02:00
Azat Khuzhin
84c720b33e Beatify test_backup_restore_s3 (using per-query profile events) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-09 08:20:03 +02:00
Azat Khuzhin
5d63b8be0d Add a test for allow_native_copy using clickhouse-disks (first ever) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-09 08:20:03 +02:00
Azat Khuzhin
5835e72fd6 More generic approach to disable native copy 
Previous patch implements this only for BACKUP/RESTORE, but it can be
useful for regular disks as well, so add allow_native_copy for disks.

Note, that there is s3_allow_native_copy query setting, since it looks
redundant, since it make sense only for S3 disks, and not on a per query
basis.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-09 08:20:02 +02:00
Azat Khuzhin
00cf66ab7a Add ability to disable native copy for BACKUP/RESTORE 
Native copy uses CopyObject for S3/GCS, but in GCS the CopyObject is
buggy - it does not always works.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-09 08:19:10 +02:00
Alexey Milovidov
59eadca95c
Merge branch 'master' into less-logs-2 2023-07-09 08:49:44 +03:00
Alexey Milovidov
612a785fd0
Merge pull request #51674 from ClickHouse/system_jemalloc 
System table with jemalloc stats
 2023-07-09 08:48:05 +03:00
Alexey Milovidov
d2fca36983
Merge branch 'master' into clickhouse_as_library_2 2023-07-09 08:47:53 +03:00
Alexey Milovidov
1a5ae9899d
Merge pull request #51710 from velavokr/velavokr-warn-cpu-utilization 
A cmake warning on job limits underutilizing CPU
 2023-07-09 08:47:01 +03:00
Alexey Milovidov
e74d42fe0d
Merge pull request #51802 from ClickHouse/fix-tsan-thread-local 
Fix tsan signal-unsafe call
 2023-07-09 08:46:00 +03:00
Alexey Milovidov
84600287a6
Merge pull request #50066 from bigo-sg/improve_geohash 
Allow geoHashEncode when its arguments are mixed with const and non-const columns
 2023-07-09 08:44:35 +03:00
robot-ch-test-poll4
8b329e83bd
Merge pull request #51959 from ClickHouse/fixes_for_transactions2 
Fix some issues with transactions
 2023-07-09 06:55:27 +02:00
Alexey Milovidov
28ad54299a
Merge branch 'master' into add-check-for-stateful-tests 2023-07-09 07:54:11 +03:00
Alexey Milovidov
dc7dbf87d8
Merge pull request #51989 from ClickHouse/move-test 
Move a test to the right place
 2023-07-09 07:53:42 +03:00
robot-clickhouse-ci-1
4666838a5d
Merge pull request #49438 from azat/build/unwind 
Force libunwind usage (removes gcc_eh support)
 2023-07-09 05:38:34 +02:00
Alexey Milovidov
52632af9b3
Update connection.py 2023-07-09 06:18:10 +03:00
Alexey Milovidov
01f4f2ab41
Merge branch 'master' into sqlloogic-improvements 2023-07-09 06:16:15 +03:00
Alexey Milovidov
3d4800995f
Merge pull request #49732 from nickitat/impr_prefetch 
Improve reading with prefetch
 2023-07-09 06:10:58 +03:00
Alexey Milovidov
b5b814860a
Merge branch 'master' into add-test-44816 2023-07-09 05:50:24 +03:00
Alexey Milovidov
e2c8cea381
Merge pull request #51801 from ClickHouse/fix_broken_parts_handling 
Fix broken parts handling in `ReplicatedMergeTree`
 2023-07-09 05:32:51 +03:00
Alexey Milovidov
780a54afb4
Merge pull request #51545 from azat/dist/fix-selector-SIGSEGV 
Fix SIGSEGV for clusters with zero weight across all shards (fixes INSERT INTO FUNCTION clusterAllReplicas())
 2023-07-09 05:31:39 +03:00
Alexey Milovidov
7311469c32 Fix test_extreme_deduplication 2023-07-09 04:17:15 +02:00
robot-clickhouse
a61bc7cfa5 Automatic style fix 2023-07-09 02:03:47 +00:00
Alexey Milovidov
de0837fe21 Check that functional tests cleanup their tables 2023-07-09 03:55:25 +02:00
Alexey Milovidov
29f625e7bb Add a check to validate that the stateful tests are stateful 2023-07-09 03:43:18 +02:00
Alexey Milovidov
8d9e1d41c5 Move a test to the right place 2023-07-09 03:39:05 +02:00