thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-26 17:41:59 +00:00
Code Issues Packages Projects Releases Wiki Activity
119,317 Commits 9,185 Branches 1,781 Tags 2.1 GiB
16165d9498
Commit Graph

119317 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Konstantin Ilchenko
3b954a2952
[DOCS] Add REMOVE SAMPLE BY to docs 2023-07-09 14:38:16 +02:00
Azat Khuzhin
3c18a181c9 Fix using of pools from the main thread 
Otherwise it is not possible to use clickhouse-disks with S3:

    $ clickhouse-disks -C /src/ch/clickhouse/tests/queries/0_stateless/02802_clickhouse_disks_s3_copy.xml --log-level test --disk s3_plain_native_copy copy default/test default/test.copy
    Failed to make request to: http://localhost:11111/test?list-type=2&max-keys=1&prefix=clickhouse-disks%2Fdefault%2Ftest.copy: Code: 49. DB::Exception: current_thread is not initialized. (LOGICAL_ERROR), Stack trace (when copying this message, always include the lines below):

    0. ./.cmake-llvm16/./contrib/llvm-project/libcxx/include/exception:134: Poco::Exception::Exception(String const&, int) @ 0x000000001ad7c872 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    1. ./.cmake-llvm16/./src/Common/Exception.cpp:94: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000011e2c4b7 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    2. ./.cmake-llvm16/./contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception<char const (&) [34], void>(int, char const (&) [34]) @ 0x000000000d341e58 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    3. ./.cmake-llvm16/./src/Common/MemoryTrackerSwitcher.h:19: DB::(anonymous namespace)::SingleEndpointHTTPSessionPool::allocObject() @ 0x0000000012010e5a in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    4. ./.cmake-llvm16/./src/Common/PoolBase.h:174: PoolBase<Poco::Net::HTTPClientSession>::get(long) @ 0x0000000012011a6f in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    5. ./.cmake-llvm16/./contrib/llvm-project/libcxx/include/variant:797: DB::makePooledHTTPSession(Poco::URI const&, Poco::URI const&, DB::ConnectionTimeouts const&, unsigned long, bool, bool) @ 0x000000001200ec69 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    6. ./.cmake-llvm16/./src/IO/HTTPCommon.cpp:0: DB::makePooledHTTPSession(Poco::URI const&, DB::ConnectionTimeouts const&, unsigned long, bool, bool) @ 0x000000001200d909 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    7. ./.cmake-llvm16/./contrib/llvm-project/libcxx/include/variant:797: void DB::S3::PocoHTTPClient::makeRequestInternalImpl<true>(Aws::Http::HttpRequest&, DB::S3::ClientConfigurationPerRequest const&, std::shared_ptr<DB::S3::PocoHTTPResponse>&, Aws::Utils::RateLimits::RateLimiterInterface*, Aws::Utils::RateLimits::RateLimiterInterface*) const @ 0x00000000163f5157 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    8. ./.cmake-llvm16/./contrib/llvm-project/libcxx/include/string:1499: DB::S3::PocoHTTPClient::makeRequestInternal(Aws::Http::HttpRequest&, std::shared_ptr<DB::S3::PocoHTTPResponse>&, Aws::Utils::RateLimits::RateLimiterInterface*, Aws::Utils::RateLimits::RateLimiterInterface*) const @ 0x00000000163f465d in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    9. ./.cmake-llvm16/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:622: DB::S3::PocoHTTPClient::MakeRequest(std::shared_ptr<Aws::Http::HttpRequest> const&, Aws::Utils::RateLimits::RateLimiterInterface*, Aws::Utils::RateLimits::RateLimiterInterface*) const @ 0x00000000163f4454 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    10. ./.cmake-llvm16/./contrib/aws/src/aws-cpp-sdk-core/source/client/AWSClient.cpp:506: Aws::Client::AWSClient::AttemptOneRequest(std::shared_ptr<Aws::Http::HttpRequest> const&, Aws::AmazonWebServiceRequest const&, char const*, char const*, char const*) const @ 0x000000001ae2a922 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    11. ./.cmake-llvm16/./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: Aws::Client::AWSClient::AttemptExhaustively(Aws::Http::URI const&, Aws::AmazonWebServiceRequest const&, Aws::Http::HttpMethod, char const*, char const*, char const*) const @ 0x000000001ae28299 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    12. ./.cmake-llvm16/./contrib/aws/src/aws-cpp-sdk-core/include/aws/core/utils/Outcome.h:160: Aws::Client::AWSXMLClient::MakeRequest(Aws::Http::URI const&, Aws::AmazonWebServiceRequest const&, Aws::Http::HttpMethod, char const*, char const*, char const*) const @ 0x000000001ae3c9ed in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    13. ./.cmake-llvm16/./contrib/aws/src/aws-cpp-sdk-core/source/client/AWSXmlClient.cpp:66: Aws::Client::AWSXMLClient::MakeRequest(Aws::AmazonWebServiceRequest const&, Aws::Endpoint::AWSEndpoint const&, Aws::Http::HttpMethod, char const*, char const*, char const*) const @ 0x000000001ae3c995 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    14. ./.cmake-llvm16/./contrib/aws/generated/src/aws-cpp-sdk-s3/source/S3Client.cpp:0: Aws::S3::S3Client::ListObjectsV2(Aws::S3::Model::ListObjectsV2Request const&) const @ 0x000000001aee6666 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    15. ./.cmake-llvm16/./contrib/aws/src/aws-cpp-sdk-core/include/aws/core/utils/Outcome.h:160: DB::S3::Client::ListObjectsV2(DB::S3::ExtendedRequest<Aws::S3::Model::ListObjectsV2Request> const&) const @ 0x00000000163cee42 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    16. ./.cmake-llvm16/./contrib/aws/src/aws-cpp-sdk-core/include/aws/core/utils/Outcome.h:120: DB::S3ObjectStorage::listObjects(String const&, std::vector<DB::RelativePathWithMetadata, std::allocator<DB::RelativePathWithMetadata>>&, int) const @ 0x0000000016b582e2 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    17. ./.cmake-llvm16/./contrib/llvm-project/libcxx/include/vector:543: DB::IObjectStorage::existsOrHasAnyChild(String const&) const @ 0x000000001644ebe9 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    18. ./.cmake-llvm16/./src/Disks/ObjectStorages/MetadataStorageFromPlainObjectStorage.cpp:0: DB::MetadataStorageFromPlainObjectStorage::exists(String const&) const @ 0x0000000016b54a64 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    19. ./.cmake-llvm16/./src/Disks/IDisk.cpp:145: DB::IDisk::copyDirectoryContent(String const&, std::shared_ptr<DB::IDisk> const&, String const&) @ 0x0000000016b38fa0 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    20. ./.cmake-llvm16/./contrib/llvm-project/libcxx/include/string:1499: DB::CommandCopy::execute(std::vector<String, std::allocator<String>> const&, std::shared_ptr<DB::Context>&, Poco::Util::LayeredConfiguration&) @ 0x0000000012050403 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    21. ./.cmake-llvm16/./contrib/llvm-project/libcxx/include/vector:434: DB::DisksApp::main(std::vector<String, std::allocator<String>> const&) @ 0x000000001204bf02 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    22. ./.cmake-llvm16/./base/poco/Util/src/Application.cpp:0: Poco::Util::Application::run() @ 0x000000001ac7a666 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    23. ./.cmake-llvm16/./programs/disks/DisksApp.cpp:0: mainEntryClickHouseDisks(int, char**) @ 0x000000001204c550 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    24. ./.cmake-llvm16/./programs/main.cpp:0: main @ 0x000000000cfbadc4 in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
    25. ? @ 0x00007ffff7dc9850 in ?
    26. __libc_start_main @ 0x00007ffff7dc990a in ?
    27. _start @ 0x000000000cfba1ee in /src/ch/clickhouse/.cmake-llvm16/programs/clickhouse
     (version 23.7.1.1)
    AWSXmlClient: HTTP response code: -1
    Resolved remote host IP address:
    Request ID:
    Exception name:
    Error message: Code: 49. DB::Exception: current_thread is not initialized. (LOGICAL_ERROR) (version 23.7.1.1)
    0 response headers:
    If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.
    Request failed, now waiting 1600 ms before attempting again.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-09 14:21:27 +02:00
robot-ch-test-poll1
38c1e136b8
Merge pull request #51883 from qoega/sqlloogic-improvements 
Sqlloogic improvements
 2023-07-09 10:57:24 +02:00
Azat Khuzhin
d520413454 Cleanup SymbolIndex after reload got removed 
Remove MultiVersion for SymbolIndex structure since after #51873 it is
useless.

Follow-up for: #51873
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-09 09:25:11 +02:00
Azat Khuzhin
ac972661f9 Add exclusion for "API mode: {}" from S3 in 00002_log_and_exception_messages_formatting 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-09 08:21:18 +02:00
Azat Khuzhin
b958363630 tests: temporary fix the 02802_clickhouse_disks_s3_copy 
In #51135 the behavior of the `clickhouse-disks copy` had been changed,
let's temporary update the test (and continue discussion about this
change in that PR).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-09 08:20:03 +02:00
Azat Khuzhin
1844ac37d7 Remove create_read_buffer argument for copyS3File() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-09 08:20:03 +02:00
Azat Khuzhin
559d328178 Rename BACKUP setting native_copy to allow_s3_native_copy 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-09 08:20:03 +02:00
Azat Khuzhin
1590ffa3b1 Remove copyS3FileNative() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-09 08:20:03 +02:00
Azat Khuzhin
29dc9abfca Fix test_backup_restore_s3 after logging for native copying changed 
Check profile events instead of some odd logs.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-09 08:20:03 +02:00
Azat Khuzhin
84c720b33e Beatify test_backup_restore_s3 (using per-query profile events) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-09 08:20:03 +02:00
Azat Khuzhin
5d63b8be0d Add a test for allow_native_copy using clickhouse-disks (first ever) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-09 08:20:03 +02:00
Azat Khuzhin
5835e72fd6 More generic approach to disable native copy 
Previous patch implements this only for BACKUP/RESTORE, but it can be
useful for regular disks as well, so add allow_native_copy for disks.

Note, that there is s3_allow_native_copy query setting, since it looks
redundant, since it make sense only for S3 disks, and not on a per query
basis.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-09 08:20:02 +02:00
Azat Khuzhin
00cf66ab7a Add ability to disable native copy for BACKUP/RESTORE 
Native copy uses CopyObject for S3/GCS, but in GCS the CopyObject is
buggy - it does not always works.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-09 08:19:10 +02:00
Alexey Milovidov
59eadca95c
Merge branch 'master' into less-logs-2 2023-07-09 08:49:44 +03:00
Alexey Milovidov
612a785fd0
Merge pull request #51674 from ClickHouse/system_jemalloc 
System table with jemalloc stats
 2023-07-09 08:48:05 +03:00
Alexey Milovidov
d2fca36983
Merge branch 'master' into clickhouse_as_library_2 2023-07-09 08:47:53 +03:00
Alexey Milovidov
1a5ae9899d
Merge pull request #51710 from velavokr/velavokr-warn-cpu-utilization 
A cmake warning on job limits underutilizing CPU
 2023-07-09 08:47:01 +03:00
Alexey Milovidov
e74d42fe0d
Merge pull request #51802 from ClickHouse/fix-tsan-thread-local 
Fix tsan signal-unsafe call
 2023-07-09 08:46:00 +03:00
Alexey Milovidov
84600287a6
Merge pull request #50066 from bigo-sg/improve_geohash 
Allow geoHashEncode when its arguments are mixed with const and non-const columns
 2023-07-09 08:44:35 +03:00
robot-ch-test-poll4
8b329e83bd
Merge pull request #51959 from ClickHouse/fixes_for_transactions2 
Fix some issues with transactions
 2023-07-09 06:55:27 +02:00
Alexey Milovidov
28ad54299a
Merge branch 'master' into add-check-for-stateful-tests 2023-07-09 07:54:11 +03:00
Alexey Milovidov
dc7dbf87d8
Merge pull request #51989 from ClickHouse/move-test 
Move a test to the right place
 2023-07-09 07:53:42 +03:00
robot-clickhouse-ci-1
4666838a5d
Merge pull request #49438 from azat/build/unwind 
Force libunwind usage (removes gcc_eh support)
 2023-07-09 05:38:34 +02:00
Alexey Milovidov
52632af9b3
Update connection.py 2023-07-09 06:18:10 +03:00
Alexey Milovidov
01f4f2ab41
Merge branch 'master' into sqlloogic-improvements 2023-07-09 06:16:15 +03:00
Alexey Milovidov
3d4800995f
Merge pull request #49732 from nickitat/impr_prefetch 
Improve reading with prefetch
 2023-07-09 06:10:58 +03:00
Alexey Milovidov
b5b814860a
Merge branch 'master' into add-test-44816 2023-07-09 05:50:24 +03:00
Alexey Milovidov
e2c8cea381
Merge pull request #51801 from ClickHouse/fix_broken_parts_handling 
Fix broken parts handling in `ReplicatedMergeTree`
 2023-07-09 05:32:51 +03:00
Alexey Milovidov
780a54afb4
Merge pull request #51545 from azat/dist/fix-selector-SIGSEGV 
Fix SIGSEGV for clusters with zero weight across all shards (fixes INSERT INTO FUNCTION clusterAllReplicas())
 2023-07-09 05:31:39 +03:00
Alexey Milovidov
7311469c32 Fix test_extreme_deduplication 2023-07-09 04:17:15 +02:00
robot-clickhouse
a61bc7cfa5 Automatic style fix 2023-07-09 02:03:47 +00:00
Alexey Milovidov
de0837fe21 Check that functional tests cleanup their tables 2023-07-09 03:55:25 +02:00
Alexey Milovidov
29f625e7bb Add a check to validate that the stateful tests are stateful 2023-07-09 03:43:18 +02:00
Alexey Milovidov
8d9e1d41c5 Move a test to the right place 2023-07-09 03:39:05 +02:00
Alexey Milovidov
ab93967fb4
Merge pull request #51171 from ClickHouse/retry 
Decrease default timeouts for S3 and HTTP requests
 2023-07-09 03:49:56 +03:00
Alexey Milovidov
1167e9a452
Merge pull request #51987 from ClickHouse/auto/v22.8.20.11-lts 
Update version_date.tsv and changelogs after v22.8.20.11-lts
 2023-07-09 03:22:35 +03:00
Alexey Milovidov
8180ee9591
Merge pull request #51986 from ClickHouse/auto/v23.6.2.18-stable 
Update version_date.tsv and changelogs after v23.6.2.18-stable
 2023-07-09 03:22:16 +03:00
robot-clickhouse
c968fe808f Update version_date.tsv and changelogs after v22.8.20.11-lts 2023-07-08 23:17:41 +00:00
robot-clickhouse
45d36b736a Update version_date.tsv and changelogs after v23.6.2.18-stable 2023-07-08 23:14:02 +00:00
Alexey Milovidov
508a9c1800
Merge pull request #51969 from ClickHouse/docker-network-prune 
Fix Docker
 2023-07-08 23:45:58 +03:00
robot-clickhouse
1343e5cc45
Merge pull request #51853 from kitaisreal/cache-dictionary-request-only-unique-keys-from-source 
CacheDictionary request only unique keys from source
 2023-07-08 20:58:16 +02:00
Azat Khuzhin
a10aa9ad50 Force libunwind usage (removes gcc_eh support) 
libunwind is reentrant and signal safe, and works faster then then
gcc_eh (plus it has some custom patches for problems that have been
found during it's usage in ClickHouse).

gcc_eh may be missing in the system (if gcc was not installed), and
even if it exists clickhouse uses -nodefaultlibs, so some care should be
made to make it work.

Also this library is tiny and there shouln't be any problem to require
it always (there is already tendency to require some contrib libraries,
i.e. poco).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-07-08 20:55:50 +02:00
Alexey Milovidov
66b6823202
Merge branch 'master' into add-test-34626 2023-07-08 21:53:24 +03:00
Alexey Milovidov
5d385f5f14
Merge pull request #51881 from qoega/per-column-pk-syntax 
Add support for PRIMARY KEY syntax in column definition
 2023-07-08 21:52:29 +03:00
Alexey Milovidov
2a8c7d0ea2
Update src/Parsers/ParserCreateQuery.cpp 
Co-authored-by: Nikolay Degterinsky <43110995+evillique@users.noreply.github.com>
 2023-07-08 21:52:19 +03:00
Alexey Milovidov
0440dde315
Merge pull request #51873 from ClickHouse/remove-useless-system-query 
Remove useless code
 2023-07-08 21:11:41 +03:00
Alexey Milovidov
eff85a7273
Merge pull request #51966 from ClickHouse/fix-LSan 
Fix unrelated messages from LSan in clickhouse-client
 2023-07-08 21:06:01 +03:00
Alexey Milovidov
b44c582c40
Merge pull request #51951 from ClickHouse/more_logs_on_shutdown 
Better logs on shutdown
 2023-07-08 19:51:38 +03:00
Alexey Milovidov
a4b9b65180
Merge pull request #51967 from ClickHouse/allow-oom-in-fuzzer-with-sanitizer 
Allow OOM in AST Fuzzer with Sanitizers
 2023-07-08 19:50:16 +03:00