thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-11-30 19:42:00 +00:00
Code Issues Packages Projects Releases Wiki Activity
104,963 Commits 9,190 Branches 1,782 Tags 2.2 GiB
6c4f53bdca
Commit Graph

5523 Commits

Author SHA1 Message Date
Antonio Andelic
9ac829d4c4
Merge pull request #41766 from JackyWoo/manual_snapshot_for_keeper 
Manually snapshot creation for keeper
 2022-11-07 09:18:58 +01:00
Antonio Andelic
e08f94d0f8
Merge pull request #42659 from ClickHouse/optimize-entire-partition 
Support for optimizing old parts for entire partition only
 2022-11-07 09:06:11 +01:00
Igor Nikonov
503649a638
Merge branch 'master' into igor/insert_zk_retries_retry 2022-11-06 22:55:13 +01:00
Kseniia Sumarokova
cd5c6acbd2
Merge pull request #42960 from kssenii/fix-pg-db-datetime-table-attach 
Fix postgres db engine attaching table with datetime
 2022-11-06 22:39:15 +01:00
robot-clickhouse
f44e4fe7c6 Automatic style fix 2022-11-05 22:36:32 +00:00
Igor Nikonov
e94b9cd003 Fixes 
+ enable retries in integration test
+ typo in fault injection traces
 2022-11-05 22:28:32 +00:00
JackyWoo
9404e089fc
Merge branch 'master' into manual_snapshot_for_keeper 2022-11-05 17:37:36 +08:00
Igor Nikonov
859022660e Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-11-04 23:31:51 +00:00
Aleksandr Musorin
5cb69d8a22 changed type name for S3_Plain storage 
renamed a disk for S3PlainObjectStorage in system.disks table from s3 to s3_plain
 2022-11-04 17:35:51 +01:00
Vitaly Baranov
52b1f4aed9
Merge pull request #42484 from vitlibar/mask-sensitive-info-in-logs 
Mask some information in logs
 2022-11-04 14:09:38 +01:00
Kruglov Pavel
a7a5a70189
Merge pull request #42814 from kgoralski/failed_metrics_for_async_inserts 
Add `FailedAsyncInsertQuery` event metric for async inserts
 2022-11-04 13:41:39 +01:00
alesapin
82729a88c8
Merge pull request #42833 from vitlibar/backup-s3-increase-upload-part-size-exponentially 
Increase the size of upload part exponentially for backup to S3.
 2022-11-04 13:41:10 +01:00
kssenii
c9b4bc66b5 Fiux 2022-11-04 12:19:30 +01:00
Krzysztof Góralski
f526ddba68 Update tests/integration/test_failed_async_inserts/test.py 
Co-authored-by: Kruglov Pavel <48961922+Avogar@users.noreply.github.com>
 2022-11-03 22:26:24 +01:00
Krzysztof Góralski
14e922aca1 style check fixes test.py 2022-11-03 22:26:24 +01:00
Krzysztof Góralski
69f56d8d68 style check fixes 2022-11-03 22:26:24 +01:00
Krzysztof Góralski
be53a3596c Review: Failed queries metrics for Async Inserts, added integration test instead of stateless one 2022-11-03 22:26:24 +01:00
Daniil Rubin
99d57325bb Merge branch 'master' into hudi 2022-11-03 19:29:23 +00:00
Kruglov Pavel
b124875257
Merge branch 'master' into improve-streaming-engines 2022-11-03 13:22:06 +01:00
zzsmdfj
eeb9c57c59 to fix issue 42856 and MaterializedMySQL improvement 2022-11-03 20:21:02 +08:00
Antonio Andelic
66ffdda973
Merge branch 'master' into optimize-entire-partition 2022-11-03 09:52:13 +01:00
Antonio Andelic
25b19e6ba0
Merge branch 'master' into manual_snapshot_for_keeper 2022-11-03 08:55:01 +01:00
Igor Nikonov
9a315b7899 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-11-02 15:32:54 +00:00
Vitaly Baranov
e0133688bc
Merge branch 'master' into mask-sensitive-info-in-logs 2022-11-02 16:26:13 +01:00
avogar
9f39a6a049 Fix possible heap-use-after-free 2022-11-02 14:17:48 +00:00
Mikhail f. Shiryaev
550c776ffe
Wait for all files are in sync before archiving them 2022-11-02 13:16:27 +01:00
Vitaly Baranov
b9f2f17331 Add test and logging. 2022-11-01 12:23:20 +01:00
Vitaly Baranov
73f5664109 Fix tests. 2022-11-01 11:10:58 +01:00
Igor Nikonov
9cc64a0fcc Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-10-31 18:21:21 +00:00
Vitaly Baranov
d1c69a0753 Avoid changing query if it doesn't contain password. 2022-10-31 17:04:44 +01:00
Azat Khuzhin
8345c5f881 tests: cover ATTACH of BACKUP from s3_plain disk 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:26 +01:00
Vitaly Baranov
901ae126e1 Improve test test_on_cluster. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
aa5cfd6980 Improve tests. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
854e1e918f Use assert_cast() instead of as() in some places. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
43efbad642 Wipe passwords from distributed queries too. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
dcf8724adf Move prepareQueryForLogging() to a separate header. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
5d2a222fe4 Mask sensitive information in logs. 2022-10-31 10:50:33 +01:00
avogar
8e13d1f1ec Improve and refactor Kafka/StorageMQ/NATS and data formats 2022-10-28 16:41:10 +00:00
Azat Khuzhin
7a5432feaa Fix possible SIGSEGV for web disks when file does not exists 
It can be triggered in multiple ways, either when file does not exists
and you are trying to create MergeTree table from web (that has special
code for UUID handling) or by simply OPTIMIZE TABLE FINAL for MergeTree
table that is located on web disk, in both cases you will get the
following:

<details>

<summary>stacktrace</summary>

    2022.10.28 14:08:40.631226 [ 6043 ] {6165bf5f-e76b-4bca-941c-7c7ff5e3b46b} <Trace> ContextAccess (default): Access granted: OPTIMIZE ON default.data_from_web
    2022.10.28 14:08:40.632017 [ 6043 ] {6165bf5f-e76b-4bca-941c-7c7ff5e3b46b} <Debug> default.data_from_web (a3e65e1f-5fd4-47ed-9dbd-307f2586b52d) (MergerMutator): Selected 1 parts from all_1_1_0 to all_1_1_0
    2022.10.28 14:08:40.632496 [ 6043 ] {6165bf5f-e76b-4bca-941c-7c7ff5e3b46b} <Trace> default.data_from_web (a3e65e1f-5fd4-47ed-9dbd-307f2586b52d): Trying to reserve 1.00 MiB using storage policy from min volume index 0
    2022.10.28 14:08:40.632752 [ 6043 ] {6165bf5f-e76b-4bca-941c-7c7ff5e3b46b} <Trace> DiskObjectStorage(DiskWebServer): Reserved 1.00 MiB on remote disk `web_disk`, having unreserved 16.00 EiB.
    2022.10.28 14:08:40.634155 [ 6043 ] {a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1} <Debug> MergeTask::PrepareStage: Merging 1 parts: from all_1_1_0 to all_1_1_0 into Compact
    2022.10.28 14:08:40.634498 [ 6043 ] {a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1} <Trace> WebObjectStorage: Loading metadata for directory: http://127.0.0.1:8080/store/a3e/a3e65e1f-5fd4-47ed-9dbd-307f2586b52d/tmp_merge_all_1_1_1
    2022.10.28 14:08:40.635025 [ 6043 ] {a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1} <Trace> DiskWeb: Adding directory: store/a3e/a3e65e1f-5fd4-47ed-9dbd-307f2586b52d/tmp_merge_all_1_1_1/
    2022.10.28 14:08:40.635355 [ 6043 ] {a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1} <Trace> ReadWriteBufferFromHTTP: Sending request to http://127.0.0.1:8080/store/a3e/a3e65e1f-5fd4-47ed-9dbd-307f2586b52d/tmp_merge_all_1_1_1/.index
    2022.10.28 14:08:40.639618 [ 6043 ] {a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1} <Trace> DiskWeb: Cannot load disk metadata. Error: Code: 86. DB::Exception: Received error from remote server /store/a3e/a3e65e1f-5fd4-47ed-9dbd-307f2586b52d/tmp_merge_all_1_1_1/.index. HTTP status code: 404 Not Found, body: <!doctype html><html><head><title>404 Not Found</title><style>
    body { background-color: #fcfcfc; color: #333333; margin: 0; padding:0; }
    h1 { font-size: 1.5em; font-weight: normal; background-color: #9999cc; min-height:2em; line-height:2em; border-bottom: 1px inset black; margin: 0; }
    h1, p { padding-left: 10px; }
    code.url { background-color: #eeeeee; font-family:monospace; padding:0 2px;}
    </style>
    </head><body><h1>Not Found</h1><p>The requested resource <code class="url">/store/a3e/a3e65e1f-5fd4-47ed-9dbd-307f2586b52d/tmp_merge_all_1_1_1/.index</code> was not found on this server.</p></body></html>: while loading disk metadata. (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER) (version 22.11.1.1)
    2022.10.28 14:08:40.640527 [ 5488 ] {} <Trace> BaseDaemon: Received signal 11
    2022.10.28 14:08:40.641529 [ 9027 ] {} <Fatal> BaseDaemon: ########################################
    2022.10.28 14:08:40.642759 [ 9027 ] {} <Fatal> BaseDaemon: (version 22.11.1.1, build id: 12145DA78CE5E9EBB10A034177FAE5967EF81A4A) (from thread 6043) (query_id: a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1) (query: optimize table data_from_web final) Received signal Segmentation fault (11)
    2022.10.28 14:08:40.643260 [ 9027 ] {} <Fatal> BaseDaemon: Address: NULL pointer. Access: read. Unknown si_code.
    2022.10.28 14:08:40.643769 [ 9027 ] {} <Fatal> BaseDaemon: Stack trace: 0x7ffff416c0f2 0x7ffff7cd1ca8 0x7ffff679ae5e 0x7fffd52e7906 0x7fffd50c65aa 0x7fffca7a0d42 0x7fffcaee79ec 0x7fffcaf242f8 0x7fffcaf242b5 0x7fffcaf2427d 0x7fffcaf24255 0x7fffcaf2421d 0x7ffff65c3686 0x7ffff65c2295 0x7fffcaeee2a9 0x7fffcaef2c43 0x7fffcaee3c0e 0x7fffcc4a7851 0x7fffcc4a768f 0x7fffcc4abb2d 0x7fffcfdce828 0x7fffd03e3eaa 0x7fffd03dfe3b 0x7fffc8ec42d4 0x7fffc8ed51d2 0x7ffff4bdd839 0x7ffff4bde0a8 0x7ffff48ab261 0x7ffff48a769a 0x7ffff48a6335 0x7ffff409f8fd 0x7ffff4121a60
    2022.10.28 14:08:40.644411 [ 9027 ] {} <Fatal> BaseDaemon: 4. ? @ 0x7ffff416c0f2 in ?
    2022.10.28 14:08:40.676390 [ 9027 ] {} <Fatal> BaseDaemon: 5. /src/ch/clickhouse/src/Common/StringUtils/StringUtils.cpp:9: detail::startsWith(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, char const*, unsigned long) @ 0x1ca8 in /src/ch/clickhouse/.cmake/src/Common/StringUtils/libstring_utilsd.so
    2022.10.28 14:08:40.730727 [ 9027 ] {} <Fatal> BaseDaemon: 6. /src/ch/clickhouse/src/Common/StringUtils/StringUtils.h:19: startsWith(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) @ 0x59ae5e in /src/ch/clickhouse/.cmake/src/libclickhouse_common_iod.so
    2022.10.28 14:08:40.923955 [ 9027 ] {} <Fatal> BaseDaemon: 7. /src/ch/clickhouse/src/Disks/ObjectStorages/Web/MetadataStorageFromStaticFilesWebServer.cpp:58: DB::MetadataStorageFromStaticFilesWebServer::exists(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) const @ 0x6e7906 in /src/ch/clickhouse/.cmake/src/libdbmsd.so
    2022.10.28 14:08:41.291996 [ 9027 ] {} <Fatal> BaseDaemon: 8. /src/ch/clickhouse/src/Disks/ObjectStorages/DiskObjectStorage.cpp:181: DB::DiskObjectStorage::exists(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) const @ 0x4c65aa in /src/ch/clickhouse/.cmake/src/libdbmsd.so
    2022.10.28 14:08:41.704697 [ 9027 ] {} <Fatal> BaseDaemon: 9. /src/ch/clickhouse/src/Storages/MergeTree/DataPartStorageOnDisk.cpp:74: DB::DataPartStorageOnDisk::exists() const @ 0xda0d42 in /src/ch/clickhouse/.cmake/src/libclickhouse_storages_mergetreed.so
    2022.10.28 14:08:43.032459 [ 9027 ] {} <Fatal> BaseDaemon: 10. /src/ch/clickhouse/src/Storages/MergeTree/MergeTask.cpp:147: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x14e79ec in /src/ch/clickhouse/.cmake/src/libclickhouse_storages_mergetreed.so
    ...
    Segmentation fault (core dumped)

</details>

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-28 15:15:49 +02:00
Antonio Andelic
5fae8b0cc9 Merge branch 'master' into optimize-entire-partition 2022-10-28 11:44:10 +00:00
Igor Nikonov
3682913172 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-10-27 22:01:53 +00:00
Alexander Tokmakov
86b81a2b6f
Merge pull request #42234 from arthurpassos/fix_ares_fd_process_race_condition 
Fix c-ares crash
 2022-10-27 17:17:22 +03:00
Igor Nikonov
649e8d302c Sync with retiring branch igor/insert_zk_retries 2022-10-26 17:41:44 +00:00
Igor Nikonov
8435cba352 Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry 2022-10-26 17:35:47 +00:00
Antonio Andelic
35a9672704 Merge branch 'master' into optimize-entire-partition 2022-10-26 11:09:25 +00:00
Antonio Andelic
97aaebfa18 Address PR comments 2022-10-26 10:06:56 +00:00
Antonio Andelic
611c2e2bd7 Support for optimizing old parts for entire partition only 2022-10-25 13:40:18 +00:00
Nikolai Kochetov
5c4444237e
Merge pull request #42617 from ClickHouse/revert-revert-41268-disable-s3-parallel-write-for-part-moves-to-disk-s3 
Revert revert 41268 disable s3 parallel write for part moves to disk s3
 2022-10-25 14:29:27 +02:00
Nikita Taranov
49f6692a2e
Adapt internal data structures to 512-bit era (#42564) 
* impl

* update tests

* fix tests
 2022-10-25 13:56:28 +02:00
Raúl Marín
0016bc2b73
Improve the time to recover keeper connections (#42541) 
Co-authored-by: Nikita Mikhaylov <nikitamikhaylov@clickhouse.com>
Co-authored-by: robot-clickhouse <robot-clickhouse@clickhouse.com>
Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>
 2022-10-25 13:26:29 +02:00
JackyWoo
9a36a509fe fix test 2022-10-25 17:15:49 +08:00
Alexey Milovidov
9ee7131f67
Merge pull request #42423 from ClickHouse/optimize-partitions-in-background 
Optimize partitions in background
 2022-10-24 19:41:50 +02:00
Nikolai Kochetov
beb40e4579 Split tests. 2022-10-24 14:22:06 +00:00
Vitaly Baranov
cd158b021f
Merge pull request #42562 from vitlibar/subqueries-in-row-policy-filters 
Fix using subqueries in row policy filters
 2022-10-24 13:38:15 +02:00
Igor Nikonov
54013009d3 Code migration from #41734 to align diff and git blame. 
Initial implementation was different and it changed the entire
ReplicatedMergeTreeSink::commitPart() which change history provided by git blame.
Then RetriesControl.retryLoop() was introduced later which significantly reduces
the diff since it's like while() used before.
So, check outing the current version will keep more original history in
git blame, which is useful here
 2022-10-24 08:20:11 +00:00
JackyWoo
42e391a019 fix test 2022-10-22 22:47:03 +08:00
JackyWoo
39c88c74e8 check whether last manually created snapshot is done 2022-10-22 22:31:17 +08:00
JackyWoo
bf291790b7
Merge branch 'master' into manual_snapshot_for_keeper 2022-10-22 16:57:50 +08:00
Azat Khuzhin
dbb4f8216f Merge remote-tracking branch 'upstream/master' into build/shorten-64-to-32 
Conflicts:
- src/Functions/GregorianDate.h
 2022-10-22 08:20:05 +02:00
Nikolay Degterinsky
22f87e2bf9
Merge pull request #42549 from evillique/disks-better-logs 
Better logs in clickhouse-disks
 2022-10-22 04:31:44 +02:00
Azat Khuzhin
4eeb7337ed Merge remote-tracking branch 'upstream/master' into build/shorten-64-to-32 
Conflicts:
- src/Common/intExp.h
- src/Functions/DateTimeTransforms.h
- src/Functions/FunctionsConversion.h
 2022-10-21 22:51:56 +02:00
Alexander Tokmakov
7d11054134
Merge branch 'master' into fix_ares_fd_process_race_condition 2022-10-21 18:27:22 +03:00
Vitaly Baranov
2033ea1216 Support subqueries in row policy filters. 2022-10-21 15:42:29 +02:00
Alexander Tokmakov
5ffd026261
Merge pull request #42529 from azat/tests/integration/fix-sanitizer-reports 
Fix sanitizer reports in integration tests
 2022-10-21 16:01:43 +03:00
Alexander Tokmakov
1343f62e8f
Update test.py 2022-10-21 15:59:27 +03:00
Azat Khuzhin
0365bc56aa Add backward compatiblity check for some functions 
* State of aggregate functions

This test goes though all aggregate functions that:
- has only one argument
- support string as an argument

And do a simple check by creating the aggregate state with one string.

Yes this is not covers everything (does not cover functions with
different number of arguments, types, different states in case of
multiple values - uniqCombined, but as for uniqCombined it will be
checked via uniqHLL12), but at least something.

And about type, String had been selected, since it more likely that
there will be used some hash function internally.

* Function(String)

Check every function which has only one argument, and it's type is
String.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-21 13:25:43 +02:00
Antonio Andelic
3091d609fa
Merge branch 'master' into optimize-partitions-in-background 2022-10-21 11:01:41 +02:00
Nikolay Degterinsky
d7d0794d2f Better logs in clickhouse-disks 2022-10-20 23:17:29 +00:00
Nikolai Kochetov
57ffcb7ed0 Revert "Merge pull request #41681 from ClickHouse/revert-41268-disable-s3-parallel-write-for-part-moves-to-disk-s3" 
This reverts commit 7956c2becf, reversing
changes made to 57be648984.
 2022-10-20 19:51:27 +02:00
Arthur Passos
50f9acb221
Merge branch 'master' into fix_ares_fd_process_race_condition 2022-10-20 13:35:58 -03:00
Arthur Passos
2fe59e7e8f lower thread and iterations number 2022-10-20 13:30:20 -03:00
Alexander Tokmakov
905a4ded42
Merge pull request #42535 from ClickHouse/tavplubix-patch-7 
Disable `test_storage_nats`, because it's permanently broken
 2022-10-20 16:58:46 +03:00
Alexander Tokmakov
5b03bbce2e
Merge pull request #41981 from ClickHouse/fixes_for_replicated_merge_tree2 
Some fixes for ReplicatedMergeTree
 2022-10-20 16:57:51 +03:00
Alexander Tokmakov
89ff3d4731
Update test.py 2022-10-20 16:54:52 +03:00
Azat Khuzhin
c399b021cf Fix sanitizer reports in integration tests 
Before they was ignored because first there was a check for a sign of
sanitizer (==================), but it was done by
clickhouse-server.log, while sanitizer write to stderr.log.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-20 15:45:40 +02:00
Antonio Andelic
9d7e44409a
Merge pull request #41342 from ClickHouse/keeper-upload-snapshot-to-s3 
Upload Keeper snapshots to S3
 2022-10-20 09:14:59 +02:00
Antonio Andelic
77eb353839
Merge pull request #42243 from ClickHouse/use-multiread-more 
Use MultiRead where possible
 2022-10-20 09:04:54 +02:00
Alexander Tokmakov
f958c155f3
Merge branch 'master' into fixes_for_replicated_merge_tree2 2022-10-19 17:25:24 +03:00
Antonio Andelic
05e6b94b54 Polishing 2022-10-19 12:59:28 +00:00
Antonio Andelic
e5408aac99 Simplify logic 2022-10-19 12:34:20 +00:00
Arthur Passos
7f0a2d32c8 fix test errors 2022-10-18 20:15:58 -03:00
Vitaly Baranov
1365105bc4 Implement backup to S3 2022-10-19 00:04:41 +02:00
Alexander Tokmakov
ea77ff9022 Merge branch 'master' into fixes_for_replicated_merge_tree2 2022-10-18 17:25:25 +02:00
Antonio Andelic
e828dce331 Disable auto optimize by default 2022-10-18 12:11:23 +00:00
Antonio Andelic
bd2522fdd0 Merge branch 'master' into optimize-partitions-in-background 2022-10-18 09:13:59 +00:00
Antonio Andelic
ee2f5977d5
Merge branch 'master' into keeper-upload-snapshot-to-s3 2022-10-18 10:02:50 +02:00
chen
163b3c9f52
fix test 
fix test
 2022-10-18 14:53:21 +08:00
xiedeyantu
10acf43db4 fix test 2022-10-18 09:20:59 +08:00
Vitaly Baranov
eb637a6f81
Merge pull request #42232 from azat/backups/s3 
Support BACKUP to S3 with as-is path/data structure
 2022-10-18 02:37:07 +02:00
Yakov Olkhovskiy
3ce1fa09d6
Merge pull request #41198 from ClickHouse/composable-protocol 
Composable protocol
 2022-10-17 11:23:12 -04:00
Antonio Andelic
7c7395575f Merge branch 'master' into keeper-upload-snapshot-to-s3 2022-10-17 11:02:36 +00:00
Antonio Andelic
878dee40b1 Use ZooKeeper in a single test 2022-10-17 07:36:49 +00:00
xiedeyantu
2a356715da fix test 2022-10-16 16:46:00 +08:00
Arthur Passos
f414edebae fix black style 2022-10-14 15:54:48 -03:00
Arthur Passos
299d848473 style fix? 2022-10-14 15:23:10 -03:00
Arthur Passos
78202f9660
Merge branch 'master' into fix_ares_fd_process_race_condition 2022-10-14 12:56:39 -03:00
Arthur Passos
6d22bb78e0 custom py script in integ tests 2022-10-14 12:39:08 -03:00
Anton Popov
95fdb0a877 remove outdated parts asynchronously 2022-10-14 14:52:57 +00:00
Azat Khuzhin
8830f0608d Support BACKUP to S3 with as-is path/data structure 
Right now backup to S3 does not make a lot of sense, since:
- it has random names, and to decoding them
- requires metadata from local disk (/var/lib/disks/DISK/BACKUP_NAME)
- or send_metadata (but it is also tricky even with it)

So this patch adds simpler interface for S3, it is only suitable for
BACKUP/RESTORE, so don't try to use it for MergeTree engine.

It is done by adding separate disk - `s3_plain` for this, that:
- does not support any extended features, like renames/hardlinks/attrs/...
  (so basically everything that MergeTree requires)
- only write/read/unlink/list files

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-14 12:59:25 +02:00
nvartolomei
ea47ab1400
assert unused value in test_replicated_merge_tree_compatibility 2022-10-12 18:12:25 +01:00
Yakov Olkhovskiy
6c8e52eb67
Merge branch 'master' into composable-protocol 2022-10-08 20:33:32 -04:00
Anton Popov
9f1104c2ad do not load inactive parts 2022-10-08 01:39:46 +00:00
robot-clickhouse
3bed015a1f Automatic style fix 2022-10-08 00:09:24 +00:00
Yakov Olkhovskiy
6bb62d4d03 add test for PROXYv1 2022-10-08 00:01:58 +00:00
Vitaly Baranov
69ebf12dab
Merge pull request #42146 from azat/backups/metadata-overflow-fix 
Fix reusing of files > 4GB from base backup
 2022-10-08 00:22:28 +02:00
Alexander Tokmakov
b8d9066004 Revert "Resurrect parallel distributed insert select with s3Cluster (#41535)" 
This reverts commit 860e34e760.
 2022-10-07 15:53:30 +02:00
Azat Khuzhin
94566abda9 Fix reusing of files > 4GB from base backup 
Previosly u64 numbers was truncated to u32 numbers during writing to the
mdatadata xml file, and further incremental backup cannot reuse them,
since the file in base backup is smaller.

P.S. There can be other places, I thought about enabling
-Wshorten-64-to-32, but there are lots of warnings right now.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-07 14:56:09 +02:00
Yakov Olkhovskiy
2229087896
Merge branch 'master' into composable-protocol 2022-10-07 02:23:26 -04:00
Yakov Olkhovskiy
997fa5e295 review suggestions 2022-10-07 01:16:22 +00:00
Nikita Mikhaylov
860e34e760
Resurrect parallel distributed insert select with s3Cluster (#41535) 2022-10-06 13:47:32 +02:00
Nikita Mikhaylov
766107df0a
Fixed the restriction on maximum size of replicated fetches (#42090) 2022-10-06 13:18:46 +02:00
Kseniia Sumarokova
316824c02a
Merge pull request #42085 from elenatorro/use_send_metadata_setting_on_storage_config 
Use send_metadata instead of send_object_metadata
 2022-10-06 12:33:25 +02:00
Kseniia Sumarokova
3cff776109
Merge pull request #41254 from filimonov/system_disks_unreserved_space 
Add unreserved_space column to system.disks
 2022-10-06 12:28:23 +02:00
Elena
3914b8afda Remove 'send_metadata' setting 2022-10-05 14:54:08 +02:00
Elena
5bbd2d1729 Use send_metadata instead of send_object_metadata 2022-10-05 13:43:59 +02:00
Kseniia Sumarokova
6775c2c0a8
Update test.py 2022-10-04 16:52:53 +02:00
Kseniia Sumarokova
0b0c7e359f
Update test.py 2022-10-04 16:04:57 +02:00
Kseniia Sumarokova
39d5dfb3f6
Fix integration test 2022-10-04 16:03:11 +02:00
Kseniia Sumarokova
1ffa3291d6
Update storage_conf.xml 2022-10-04 11:18:02 +02:00
kssenii
2ab884359e More efficient WriteBufferFromAzureBlobStorage 2022-10-03 22:15:04 +02:00
kssenii
c6d7cd5820 Fix write into azure blob storage 2022-10-03 18:54:07 +02:00
Kseniia Sumarokova
0bfbb5c38a
Merge branch 'master' into system_disks_unreserved_space 2022-10-03 14:25:47 +02:00
Alexander Tokmakov
a2c29076d6 avoid detached broken part duplicates 2022-10-03 12:27:17 +02:00
Alexander Tokmakov
7b5b7e4963
Merge pull request #41945 from ClickHouse/enable_more_s3_tests 
Enable some disabled S3 tests
 2022-10-03 13:25:50 +03:00
Alexander Tokmakov
035be1feae fix 2022-10-02 14:58:33 +02:00
robot-clickhouse
4cd2dc8d0d Automatic style fix 2022-10-02 12:57:04 +00:00
Alexander Tokmakov
4ce8f47400 fix 2022-10-02 14:50:24 +02:00
Vitaly Baranov
baf48ec400
Merge pull request #41562 from vitlibar/refactor-wiping-info-from-logs 
Refactor wiping sensitive information from logs.
 2022-10-02 12:58:58 +02:00
Alexander Tokmakov
e40917a9c5 fix 2022-10-02 00:44:46 +02:00
Kseniia Sumarokova
fe6de361c6
Merge pull request #41933 from ClickHouse/kssenii-patch-5 
Fix broken test_disks_app_func
 2022-10-02 00:17:28 +02:00
Vitaly Baranov
a5b2a49ab1 Refactor wiping sensitive information from logs. 2022-10-01 17:17:45 +02:00
Alexey Milovidov
645dd997d3
Update test.py 2022-10-01 18:16:25 +03:00
robot-clickhouse
26ad8b2f13 Automatic style fix 2022-10-01 11:55:58 +00:00
Kseniia Sumarokova
3afec69021
Update test.py 2022-10-01 13:49:50 +02:00
kssenii
fb3bcd2fe9 Fix use_environment_credentials 2022-09-30 12:25:39 +02:00
Alexander Tokmakov
bd305960e7
Merge pull request #41898 from ClickHouse/fix-tests-sync-replica 
Another sync replicas in test_recovery_replica
 2022-09-29 19:36:20 +03:00
alesapin
0097f15ee7
Merge pull request #41832 from ClickHouse/make_copy_instead_of_hardlink 
Use copy for some operations instead of hardlinks
 2022-09-29 17:12:08 +02:00
Alexander Tokmakov
4e422b8046
Merge pull request #41741 from ClickHouse/fix_intersecting_parts 
Fix intersecting parts
 2022-09-29 18:00:22 +03:00
robot-clickhouse
acdc3504aa Automatic style fix 2022-09-29 11:21:41 +00:00
Kseniia Sumarokova
6270f513fc
Update test.py 2022-09-29 13:03:03 +02:00
Yakov Olkhovskiy
028af59702
Update tests/integration/test_recovery_replica/test.py 
Co-authored-by: Alexander Tokmakov <tavplubix@clickhouse.com>
 2022-09-28 12:33:38 -04:00
alesapin
905d4585ac
Merge branch 'master' into make_copy_instead_of_hardlink 2022-09-28 16:47:45 +02:00
Yakov Olkhovskiy
10f6ba90e0
add sync_replicas 2022-09-28 10:00:59 -04:00
alesapin
4b4c63749a Correct test version 2022-09-28 14:38:11 +02:00
Daniil Rubin
95b5d2d29d
Merge branch 'master' into hudi 2022-09-28 14:24:55 +03:00
alesapin
2eef384807 Add comments and test 2022-09-28 13:09:48 +02:00
Alexey Milovidov
e5ffd74a56
Merge branch 'master' into trying_to_fix_compressed_marks 2022-09-28 02:30:10 +03:00
Alexey Milovidov
12dc33f4fd
Merge branch 'master' into nv/fix-mutate-fetch 2022-09-28 01:28:19 +03:00
Alexander Tokmakov
0915bfdb17 fix 2022-09-27 17:42:36 +02:00
Alexander Tokmakov
b4a10c330c fix 2022-09-27 17:34:24 +02:00
Antonio Andelic
036d1c8cbc
Merge pull request #40991 from ClickHouse/start-embedded-keeper-async 
Always start embedded Keeper in async mode
 2022-09-27 14:44:19 +02:00
robot-clickhouse
08af2a32bf Automatic style fix 2022-09-27 09:53:26 +00:00
alesapin
4d2a5d752d Add failing test 2022-09-27 11:36:20 +02:00
Antonio Andelic
655bde8e3f Merge branch 'master' into keeper-upload-snapshot-to-s3 2022-09-27 07:28:26 +00:00
JackyWoo
d1c85f68e8 manually snapshot creation for keeper 2022-09-26 18:29:15 +08:00
robot-clickhouse
7a6386c732 Automatic style fix 2022-09-25 16:13:17 +00:00
Yakov Olkhovskiy
1407f6bcda add test 2022-09-25 15:46:12 +00:00
kssenii
3725397040 Try fix azure tests 2022-09-23 15:24:10 +02:00
Alexander Tokmakov
3eebceaa38
Update test.py 2022-09-23 14:01:00 +03:00
Kseniia Sumarokova
ea43cb5648
Merge pull request #41261 from kssenii/s3-header-auth 
Support s3 authorisation headers from ast arguments
 2022-09-23 12:48:08 +02:00
Kseniia Sumarokova
a0c5da3459
Merge pull request #41652 from kssenii/fix-disk-web 
Fixes for disk web
 2022-09-23 12:43:50 +02:00
kssenii
0e8343e6f3 Revert "Better config for cache" 
This reverts commit 0ca6aadbdb.
 2022-09-22 17:57:44 +02:00
kssenii
109d2ad022 Better 2022-09-22 16:22:05 +02:00
Antonio Andelic
44a3d6babe Address PR comments 2022-09-22 13:03:27 +00:00
Kseniia Sumarokova
79304422b3
Update test.py 2022-09-22 13:11:35 +02:00
kssenii
cb442a3ad7 Fixes for disk web 2022-09-21 21:26:55 +02:00
kssenii
92c2219e82 Fix test 2022-09-21 20:51:09 +02:00
Antonio Andelic
6798b500e9 Wait on startup for Keeper 2022-09-21 15:12:16 +00:00
Antonio Andelic
aa700836b8 Merge branch 'master' into start-embedded-keeper-async 2022-09-21 13:46:54 +00:00
Kseniia Sumarokova
1e7308832c
Merge branch 'master' into system_disks_unreserved_space 2022-09-21 13:04:39 +02:00
kssenii
f917b268b3 Merge remote-tracking branch 'upstream/master' into s3-header-auth 2022-09-21 12:56:25 +02:00
nvartolomei
8d55637af8
Update test.py 2022-09-21 10:10:21 +01:00
Antonio Andelic
fc6c1faa57 Merge branch 'master' into keeper-upload-snapshot-to-s3 2022-09-21 06:57:58 +00:00
nvartolomei
842b0bb44b
Explain why hostName function is used in test mutation 2022-09-20 22:41:14 +01:00
nvartolomei
3d6545cea1
Reformat test.py to match style 2022-09-20 22:04:20 +01:00
Nicolae Vartolomei
45f1f572f0 Get rid of scary setting for testing fetching byte identical parts 2022-09-20 19:27:49 +00:00
Nicolae Vartolomei
90666a43a8 CLICKHOUSE-2153: Move new_data_part shared pointer to allow Outdated part to be deleted immediately on mutation failure 2022-09-20 18:23:19 +00:00
kssenii
b8079e4577 Merge remote-tracking branch 'upstream/master' into rename-some-cache-commands 2022-09-20 13:02:18 +02:00
alesapin
8d88b68a85
Merge pull request #41415 from ClickHouse/revert_prefixes_s3 
Revert prefixes s3 + add retries on write requests unexpected errors
 2022-09-20 12:55:04 +02:00
Mikhail Filimonov
fb3ebbb2d2
Apply style fix for tests/integration/test_disk_types/test.py 2022-09-20 12:12:01 +02:00
Vitaly Baranov
f58f8104b5
Merge pull request #41474 from vitlibar/mask-some-info-in-logs 
Mask some information in logs.
 2022-09-20 12:11:02 +02:00
Alexey Milovidov
03bbf0676f
Merge branch 'master' into revert_prefixes_s3 2022-09-20 06:14:07 +03:00
Sergei Trifonov
cf2db48c29
Merge pull request #40631 from ClickHouse/readonly-settings-allow 
Allow to modify constrained settings in readonly mode
 2022-09-20 02:18:14 +02:00
Sergei Trifonov
358c5a0139
Merge pull request #41532 from ClickHouse/s3-api-calls-events-for-disk-s3 
Detailed S3 profile events `DiskS3*` per S3 API call split for S3 ObjectStorage
 2022-09-20 02:10:50 +02:00
Vitaly Baranov
b1199878ad
Merge pull request #39977 from vitlibar/improve-recovery-of-replicated-access-storage 
Improve recovery of ReplicatedAccessStorage after errors.
 2022-09-20 00:36:30 +02:00
Sergei Trifonov
7902922ae4
Merge branch 'master' into readonly-settings-allow 2022-09-19 21:00:12 +02:00
serxa
7a1d4a404a add DiskS3* profile events per S3 API calls 2022-09-19 18:10:47 +00:00
serxa
f8aa738511 more conventional profile events names 2022-09-19 17:23:22 +00:00
Vitaly Baranov
052ecaa211
Merge branch 'master' into improve-recovery-of-replicated-access-storage 2022-09-19 15:26:21 +02:00
Vitaly Baranov
f75dd93965 Mask some information in logs. 2022-09-19 15:00:06 +02:00
Sergei Trifonov
ff2e1ff8d8
Merge branch 'master' into support_s3_api_event_metrics 2022-09-19 14:35:45 +02:00
Sergei Trifonov
26da32f53f
Update tests/integration/test_profile_events_s3/test.py 2022-09-19 14:35:23 +02:00
Alexander Tokmakov
c6efb1773e
Merge branch 'master' into fix_drop_partition_on_cluster 2022-09-19 15:18:05 +03:00
kssenii
e3cd3686af Rename 2022-09-19 14:02:51 +02:00
Sergei Trifonov
f7a1ed3141
Merge branch 'master' into readonly-settings-allow 2022-09-19 12:52:36 +02:00
Azat Khuzhin
00d4ac542d tests: fix test_hive_query flakiness 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-19 07:54:49 +02:00
alesapin
1d0dbfb112
Merge pull request #41444 from ClickHouse/one_wrong_config_assert 
One more validation for broken config in keeper
 2022-09-18 15:56:48 +02:00
Alexey Milovidov
6b752bff3a
Merge branch 'master' into revert_prefixes_s3 2022-09-18 07:10:57 +03:00
alesapin
b6d61318d0 One more validation for broken config in keeper 2022-09-18 00:04:16 +02:00
robot-clickhouse
838bc678cc Automatic style fix 2022-09-17 18:38:12 +00:00
Alexey Milovidov
c480531423 Remove trash error message from integration tests 2022-09-17 20:27:19 +02:00
Alexey Milovidov
ecfd8cb5a6 Remove trash error message from integration tests 2022-09-17 20:24:56 +02:00
Vitaly Baranov
406c76ee4e Fix test. 2022-09-16 19:02:06 +02:00
Vitaly Baranov
1206e77883 Add test checking that ReplicatedAccessStorage can continue after reloading ZooKeeper. 2022-09-16 19:02:06 +02:00
alesapin
456da30c9f Revert "Merge pull request #41249 from ClickHouse/revert-40968-s3-sharding-2" 
This reverts commit ac60300997, reversing
changes made to a9c272283a.
 2022-09-16 14:06:26 +02:00
Robert Schulze
b32b02d844
Merge pull request #40897 from ClickHouse/catboost-bridge-resurrected 
Move CatBoost evaluation into clickhouse-library-bridge
 2022-09-16 13:12:09 +02:00
alesapin
4e780cf99f
Merge pull request #41378 from ClickHouse/rename_test 
Check test flakiness
 2022-09-16 11:51:04 +02:00
Antonio Andelic
296fe01fb8 Merge branch 'master' into keeper-upload-snapshot-to-s3 2022-09-16 08:00:15 +00:00
Alexey Milovidov
da01982652
Merge pull request #41046 from azat/build/llvm-15 
Switch to llvm/clang 15
 2022-09-16 07:31:06 +03:00
Alexander Tokmakov
03c193ccca fix ON CLUSTER with Replicated database cluster 2022-09-15 21:15:57 +02:00
alesapin
1dd50e3ce8 Rename test 2022-09-15 17:07:42 +02:00
Antonio Andelic
c7775b7f16 Add tests for S3 snapshot upload 2022-09-15 13:37:17 +00:00
serxa
a960fdea89 Merge branch 'readonly-settings-allow' of github.com:ClickHouse/ClickHouse into readonly-settings-allow 2022-09-15 11:38:06 +00:00
serxa
4971c1f04d review fixes 2022-09-15 11:34:14 +00:00
kssenii
559c696230 Fix 2022-09-14 20:35:28 +02:00
mateng0915
372a0b7794 added events into s3 integration test 2022-09-14 19:19:47 +08:00
alesapin
51a302a70f
Merge branch 'master' into revert-40968-s3-sharding-2 2022-09-14 11:53:14 +02:00
Mikhail Filimonov
a136465eb4
Fix test integration/test_disk_types/test.py 2022-09-14 11:08:37 +02:00
Antonio Andelic
36f46891dd
Merge pull request #41215 from ClickHouse/keeper-close-connection-on-shutdown 
Close sessions on Keeper shutdown
 2022-09-13 19:09:34 +02:00
kssenii
420ac4eb43 s3 header auth in ast 2022-09-13 15:13:28 +02:00
Antonio Andelic
baf7255cff
Merge pull request #39976 from ClickHouse/keeper-storage 
KeeperMap storage engine
 2022-09-13 14:15:09 +02:00
Alexander Tokmakov
4d146b05a9
Merge pull request #38262 from PolyProgrammist/fix-ordinary-system-unfreeze 
Fix SYSTEM UNFREEZE for ordinary database
 2022-09-13 14:55:11 +03:00
alesapin
9af591a328
Revert "Sharding s3 key names (2)" 2022-09-13 12:29:02 +02:00
Antonio Andelic
45cde90219 Fix test 2022-09-13 09:51:31 +00:00
Sergei Trifonov
9cd78585c3 fix tests 2022-09-13 00:12:40 +02:00
Robert Schulze
fac1be9700
chore: restore SYSTEM RELOAD MODEL(S) and moniting view SYSTEM.MODELS 
- This commit restores statements "SYSTEM RELOAD MODEL(S)" which provide
  a mechanism to update a model explicitly. It also saves potentially
  unnecessary reloads of a model from disk after it's initial load.

  To keep the complexity low, the semantics of "SYSTEM RELOAD MODEL(S)
  was changed from eager to lazy. This means that both statements
  previously immedately reloaded the specified/all models, whereas now
  the statements only trigger an unload and the first call to
  catboostEvaluate() does the actual load.

- Monitoring view SYSTEM.MODELS is also restored but with some obsolete
  fields removed. The view was not documented in the past and for now it
  remains undocumented. The commit is thus not considered a breach of
  ClickHouse's public interface.
 2022-09-12 19:33:02 +00:00
Azat Khuzhin
b698a4ff65 Apply changes to http handlers on fly without server restart 
This has been implemented by simply restarting http servers in case of
http_handlers directive in configuration xml had been changed.

But, for this I have to change the handlers interface to accept
configuration separatelly, since the configuration that contains in the
server is the configuration with which server had been started.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
Co-authored-by: Antonio Andelic
 2022-09-12 17:34:51 +02:00
Antonio Andelic
deee0d639f Address PR comments 2022-09-12 14:43:52 +00:00
Sergei Trifonov
fe88a7991d
Merge branch 'master' into readonly-settings-allow 2022-09-12 16:20:40 +02:00
Antonio Andelic
1cd43d8e5c Merge branch 'master' into keeper-storage 2022-09-12 13:42:51 +00:00
Antonio Andelic
e985ee3354 Close sessions on Keeper shutdown 2022-09-12 12:22:48 +00:00
kssenii
02c3d8b0a0 Fix 2022-09-11 16:42:56 +02:00
Alexey Milovidov
5802c2fdd2
Merge pull request #40713 from ClickHouse/remove-useless-line 
Remove one line from XML, because we do not care
 2022-09-11 09:53:30 +03:00
Alexey Milovidov
a5388affb8
Merge pull request #41023 from ClickHouse/remove-trash-3 
Remove trash from config
 2022-09-11 09:52:45 +03:00
Alexey Milovidov
91338ea771
Merge branch 'master' into remove-useless-line 2022-09-11 02:03:39 +03:00
Azat Khuzhin
46eafb4732 tests: skip test_send_crash_reports under ASan 
Since it also does not fits into timeouts [1]:

    2022-09-08 21:16:20 [ 377 ] DEBUG : Command:['docker', 'exec', '-u', 'root', 'roottestsendcrashreports_node_1', 'bash', '-c', 'pkill -SEGV clickhouse'] (cluster.py:95, run_and_check)
    ...
    2022-09-08 21:16:22 [ 377 ] DEBUG : run container_id:roottestsendcrashreports_node_1 detach:False nothrow:False cmd: ['cat', '/result.txt'] (cluster.py:1744, exec_in_container)
    ...
    2022-09-08 21:16:36 [ 377 ] DEBUG : Stdout:INITIAL_STATE (cluster.py:103, run_and_check)

And server logs:

    2022.09.08 21:16:21.112076 [ 228 ] {} <Fatal> BaseDaemon: ########################################
    2022.09.08 21:16:21.112170 [ 228 ] {} <Fatal> BaseDaemon: (version 22.9.1.1, build id: 0F7336E0A4D64134C51C8365DADCB78A9B39AA3B) (from thread 1) (no query) Received signal Segmentation fault (11)
    2022.09.08 21:16:21.112244 [ 228 ] {} <Fatal> BaseDaemon: Address: 0xde Access: read. Unknown si_code.
    2022.09.08 21:16:21.112321 [ 228 ] {} <Fatal> BaseDaemon: Stack trace: 0x7fbe21d09376 0x40a4f71a 0xe293a4b 0xdc5c51a 0x38dee227 0xdc326a0 0x38e319d9 0xdc2b4e2 0xdc25fdb 0x7fbe21b2c083 0xdb636ae
    2022.09.08 21:16:21.112419 [ 228 ] {} <Fatal> BaseDaemon: 3. pthread_cond_wait @ 0x7fbe21d09376 in ?
    2022.09.08 21:16:21.122914 [ 228 ] {} <Fatal> BaseDaemon: 4. ./build_docker/../contrib/libcxx/src/condition_variable.cpp:0: std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&) @ 0x40a4f71a in /usr/bin/clickhouse
    2022.09.08 21:16:21.233016 [ 228 ] {} <Fatal> BaseDaemon: 5.1. inlined from ./build_docker/../contrib/libcxx/include/atomic:952: unsigned long std::__1::__cxx_atomic_load<unsigned long>(std::__1::__cxx_atomic_base_impl<unsigned long> const*, std::__1::memory_order)
    2022.09.08 21:16:21.233135 [ 228 ] {} <Fatal> BaseDaemon: 5.2. inlined from ../contrib/libcxx/include/atomic:1582: std::__1::__atomic_base<unsigned long, false>::load(std::__1::memory_order) const
    2022.09.08 21:16:21.233183 [ 228 ] {} <Fatal> BaseDaemon: 5.3. inlined from ../contrib/libcxx/include/atomic:1586: std::__1::__atomic_base<unsigned long, false>::operator unsigned long() const
    2022.09.08 21:16:21.233234 [ 228 ] {} <Fatal> BaseDaemon: 5.4. inlined from ../src/Daemon/BaseDaemon.cpp:967: operator()
    2022.09.08 21:16:21.233303 [ 228 ] {} <Fatal> BaseDaemon: 5.5. inlined from ../contrib/libcxx/include/__mutex_base:402: void std::__1::condition_variable::wait<BaseDaemon::waitForTerminationRequest()::$_0>(std::__1::unique_lock<std::__1::mutex>&, BaseDaemon::waitForTerminationRequest()::$_0)
    2022.09.08 21:16:21.233334 [ 228 ] {} <Fatal> BaseDaemon: 5. ../src/Daemon/BaseDaemon.cpp:967: BaseDaemon::waitForTerminationRequest() @ 0xe293a4b in /usr/bin/clickhouse
    2022.09.08 21:16:21.350675 [ 228 ] {} <Fatal> BaseDaemon: 6. ./build_docker/../programs/server/Server.cpp:0: DB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) @ 0xdc5c51a in /usr/bin/clickhouse
    2022.09.08 21:16:21.394092 [ 228 ] {} <Fatal> BaseDaemon: 7. ./build_docker/../contrib/poco/Util/src/Application.cpp:0: Poco::Util::Application::run() @ 0x38dee227 in /usr/bin/clickhouse
    2022.09.08 21:16:21.654195 [ 228 ] {} <Fatal> BaseDaemon: 8. ./build_docker/../programs/server/Server.cpp:466: DB::Server::run() @ 0xdc326a0 in /usr/bin/clickhouse
    2022.09.08 21:16:21.666991 [ 228 ] {} <Fatal> BaseDaemon: 9. ./build_docker/../contrib/poco/Util/src/ServerApplication.cpp:0: Poco::Util::ServerApplication::run(int, char**) @ 0x38e319d9 in /usr/bin/clickhouse
    2022.09.08 21:16:21.916078 [ 228 ] {} <Fatal> BaseDaemon: 10. ./build_docker/../programs/server/Server.cpp:0: mainEntryClickHouseServer(int, char**) @ 0xdc2b4e2 in /usr/bin/clickhouse
    2022.09.08 21:16:21.929922 [ 228 ] {} <Fatal> BaseDaemon: 11. ./build_docker/../programs/main.cpp:0: main @ 0xdc25fdb in /usr/bin/clickhouse
    2022.09.08 21:16:21.929981 [ 228 ] {} <Fatal> BaseDaemon: 12. __libc_start_main @ 0x7fbe21b2c083 in ?
    2022.09.08 21:16:30.357032 [ 228 ] {} <Fatal> BaseDaemon: 13. _start @ 0xdb636ae in /usr/bin/clickhouse
    2022.09.08 21:16:31.383233 [ 228 ] {} <Fatal> BaseDaemon: Integrity check of the executable skipped because the reference checksum could not be read. (calculated checksum: 6200AC7C1270DC293DF3302E1C64399B)
    ...
    2022.09.08 21:16:40.564453 [ 228 ] {} <Information> SentryWriter: Sending crash report

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/41046/a0b85eaca8d4003c9fbc4571b30830d30f1984e9/integration_tests__asan__[3/3].html

Though another option is to increase waiting time.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-09-10 21:38:35 +02:00
robot-clickhouse
5bc2845e49 Automatic style fix 2022-09-10 03:17:34 +00:00
Alexey Milovidov
f2227e125f Fix test 2022-09-10 05:11:09 +02:00
Alexey Milovidov
61278c81e4 Merge branch 'master' into remove-useless-line 2022-09-10 05:07:10 +02:00
robot-clickhouse
79ccce23e9 Automatic style fix 2022-09-09 20:09:54 +00:00
Igor Nikonov
dbfb448290 Fix: integration test, standalone keeper mode 
There is logic regarding which keeper binary use to start keeper cluster in an integration test
There 2 options:
(1) standalone keeper binary (expected binary name clickhouse-keeper)
(2) clickhouse binary with keeper inside

Fixed:
- option (1) didn't work since docker_compose_keeper.yaml didn't create
target clickhouse-keeper at all
- if clickhouse-keeper existed, option (1) was taken but
  clickhouse-keeper could be just a link to clickhouse binary (the link
  is created always during build if cmake option BUILD_STANDALONE_KEEPER is OFF)
 2022-09-09 14:51:34 +00:00
robot-clickhouse
aeac3916ef Automatic style fix 2022-09-09 12:27:13 +00:00
Antonio Andelic
32483aeec8 Merge branch 'master' into keeper-storage 2022-09-09 12:24:01 +00:00
Sergei Trifonov
76eb001e78 add test for changeable_in_readonly + fix SQL in docs 2022-09-09 14:01:29 +02:00
Alexander Tokmakov
48927ba0ac
Merge branch 'master' into no-hardlinks-while-making-backup-of-mergetree-in-atomic-db 2022-09-09 14:24:44 +03:00
Vitaly Baranov
10629a66e5 Fix black. 2022-09-08 20:58:51 +02:00
Vitaly Baranov
e48cd2b6f6 Add more test cases. 2022-09-08 18:14:25 +02:00
robot-clickhouse
807b09f6b0 Automatic style fix 2022-09-08 15:52:00 +00:00
Sergei Trifonov
f92b6a6176
Merge branch 'master' into readonly-settings-allow 2022-09-08 17:44:32 +02:00
Sergei Trifonov
62541ab764 fix more tests and clang tidy build 2022-09-08 17:43:09 +02:00
Antonio Andelic
b99996961d Address PR comments 2022-09-08 12:45:36 +00:00
Vitaly Baranov
122009a2bd Use table lock if database is ordinary and zero-copy-replication is enabled. 2022-09-08 13:54:59 +02:00
Robert Schulze
c16707ff00
chore: delete obsolete modelEvaluate() function + SYSTEM.MODELS view 
- The deleted function modelEvaluate() was superseded by
  catboostEvaluate().

- Also delete the external model repository, as modelEvaluate() was it's
  last user. Additionally remove the system view SYSTEM.MODELS for
  inspecting the repository.

- SYSTEM RELOAD MODELS is also obsolete. HOWEVER, it was retained and
  made a no-op instead of deleted.

  Why?
  The reason is that RBAC in distributed setups works by storing
  privileges (granted and revoked) as plain SQL statements in Keeper.
  Nodes read these statements at startup and parse them. If a privilege
  for SYSTEM RELOAD MODELS exists but parser doesn't recognize it
  nodes would fail to come up.

  Considered but rejected alternatives:
  - Ignore SYSTEM RELOAD MODELS during parsing RBAC privileges and
    return an error for regular SYSTEM RELOAD MODELS SQL. Special-case
    of no-op behavior, too brittle.
  - Remove SYSTEM RELOAD MODELS manually from Keeper via command-line
    manipulation of Keeper nodes or via SQL by dropping the privileges.
    Needs user intervention during upgrade.
 2022-09-08 09:10:11 +00:00
Robert Schulze
60f9f6855d
feat: implement catboost in library-bridge 
This commit moves the catboost model evaluation out of the server
process into the library-bridge binary. This serves two goals: On the
one hand, crashes / memory corruptions of the catboost library no longer
affect the server. On the other hand, we can forbid loading dynamic
libraries in the server (catboost was the last consumer of this
functionality), thus improving security.

SQL syntax:

  SELECT
    catboostEvaluate('/path/to/model.bin', FEAT_1, ..., FEAT_N) > 0 AS prediction,
    ACTION AS target
  FROM amazon_train
  LIMIT 10

Required configuration:

  <catboost_lib_path>/path/to/libcatboostmodel.so</catboost_lib_path>

*** Implementation Details ***

The internal protocol between the server and the library-bridge is
simple:

- HTTP GET on path "/extdict_ping":
  A ping, used during the handshake to check if the library-bridge runs.

- HTTP POST on path "extdict_request"
  (1) Send a "catboost_GetTreeCount" request from the server to the
      bridge, containing a library path (e.g /home/user/libcatboost.so) and
      a model path (e.g. /home/user/model.bin). Rirst, this unloads the
      catboost library handler associated to the model path (if it was
      loaded), then loads the catboost library handler associated to the
      model path, then executes GetTreeCount() on the library handler and
      finally sends the result back to the server. Step (1) is called once
      by the server from FunctionCatBoostEvaluate::getReturnTypeImpl(). The
      library path handler is unloaded in the beginning because it contains
      state which may no longer be valid if the user runs
      catboost("/path/to/model.bin", ...) more than once and if "model.bin"
      was updated in between.
  (2) Send "catboost_Evaluate" from the server to the bridge, containing
      the model path and the features to run the interference on. Step (2)
      is called multiple times (once per chunk) by the server from function
      FunctionCatBoostEvaluate::executeImpl(). The library handler for the
      given model path is expected to be already loaded by Step (1).

Fixes #27870
 2022-09-08 09:01:32 +00:00
Vitaly Baranov
31ed722572
Merge pull request #41044 from vitlibar/more-conventional-conversion-yaml-to-xmk 
More conventional conversion yaml to xml
 2022-09-07 13:46:32 +02:00
Vitaly Baranov
9c847ceec9 No hardlinks while making backup of MergeTree in atomic database. 2022-09-07 11:44:50 +02:00
Daniil Rubin
f75376b7f8 Apply black formatter to tests 2022-09-07 11:05:40 +03:00
Daniil Rubin
4cc397da84 Add tests for deltalake storage 2022-09-07 11:05:40 +03:00
Daniil Rubin
0e9c3f299c Fix bug in test 2022-09-07 11:05:40 +03:00
Daniil Rubin
7d8cc20c46 Add select test 2022-09-07 11:05:40 +03:00
Daniil Rubin
6b95faf2b3 Add createquery test 2022-09-07 11:05:40 +03:00
Daniil Rubin
3e87fd0ee4 Add test template 2022-09-07 11:05:40 +03:00
Vitaly Baranov
63e992d52d Edit test configs. 2022-09-06 17:09:26 +02:00
Sergei Trifonov
6217559175
Merge branch 'master' into readonly-settings-allow 2022-09-06 16:10:48 +02:00
Antonio Andelic
5eb034a920 Merge branch 'master' into start-embedded-keeper-async 2022-09-06 12:14:47 +00:00
Antonio Andelic
4a68bfef39 Fix tests with async Keeper start 2022-09-06 12:12:17 +00:00
Sergei Trifonov
f77809ddbc
Merge pull request #40900 from ClickHouse/s3-detailed-metrics 
S3 detailed metrics
 2022-09-06 13:20:57 +02:00
Anton Ivashkin
e91d22a5b7 Merge branch 'fix-ordinary-system-unfreeze' of github.com:PolyProgrammist/ClickHouse into fix-ordinary-system-unfreeze 2022-09-06 12:18:05 +03:00
Anton Ivashkin
5e7cf1f238 Fix test_merge_tree_s3/test.py::test_freeze_system_unfreeze[node] 2022-09-06 12:14:40 +03:00
Alexey Milovidov
59bd94dcf8
Merge pull request #40975 from vitlibar/fix-access-rights-for-describe 
Fix access rights for DESCRIBE TABLE
 2022-09-06 02:38:34 +03:00
Alexey Milovidov
4c30dbc905
Merge pull request #40968 from ClickHouse/s3-sharding-2 
Sharding s3 key names (2)
 2022-09-05 22:47:25 +03:00
ianton-ru
39e1fc7a0f
Merge branch 'master' into fix-ordinary-system-unfreeze 2022-09-05 17:10:59 +03:00
Sergei Trifonov
1c247308f1
Merge branch 'master' into s3-detailed-metrics 2022-09-05 15:55:41 +02:00
Sergei Trifonov
225773634c fix integration test for profile events s3 2022-09-05 15:54:28 +02:00
Kseniia Sumarokova
cff1dd6a8e
Merge branch 'master' into move-strange-test 2022-09-05 15:19:57 +02:00
Antonio Andelic
e07539d1b9 Merge branch 'master' into keeper-storage 2022-09-05 09:54:13 +00:00
Antonio Andelic
3a0581e990
Merge pull request #40543 from Lloyd-Pottiger/feat/support-read-only-for-embeddedrocksdb 
Add read-only support for EmbeddedRocksDB
 2022-09-05 09:31:07 +02:00
Alexey Milovidov
cda8052df0 Fix trash 2022-09-05 03:46:10 +02:00
Alexey Milovidov
e70d526896 Fix trash 2022-09-05 03:35:43 +02:00
Alexey Milovidov
bff96b0af6
Merge branch 'master' into fix-access-rights-for-describe 2022-09-05 04:34:57 +03:00
Alexey Milovidov
cc50aa39a2 Merge branch 's3-sharding-2' of github.com:ClickHouse/ClickHouse into s3-sharding-2 2022-09-05 02:25:01 +02:00
Alexey Milovidov
79e3723272 Merge branch 'master' into s3-sharding-2 2022-09-05 02:24:35 +02:00
robot-clickhouse
89d40b6f70 Automatic style fix 2022-09-05 00:23:36 +00:00
Alexey Milovidov
a0b5567d3e Move strange test 2022-09-05 02:16:48 +02:00
Alexey Milovidov
007680d93f Preparation 2022-09-05 02:13:51 +02:00
Vitaly Baranov
6c164905ca Add test. 2022-09-04 18:57:51 +02:00
Alexey Milovidov
c4adc9ed8f Remove trash 2022-09-04 04:28:08 +02:00
Alexey Milovidov
13a129bee7 Merge branch 'master' into remove-useless-line 2022-09-04 03:24:40 +02:00
Lloyd-Pottiger
ec68ed8a93 fix test 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-09-03 10:44:07 +08:00
Sergei Trifonov
005b2588d8
Merge branch 'master' into readonly-settings-allow 2022-09-02 16:37:43 +02:00
Lloyd-Pottiger
b76af9284e style check 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-09-02 19:29:57 +08:00
Lloyd-Pottiger
f499381f57 move functional test to integration test 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-09-02 18:45:37 +08:00
Antonio Andelic
9ca20b97c2
Merge branch 'master' into keeper-storage 2022-09-02 12:13:07 +02:00
Lloyd-Pottiger
5a6b2106b5 Merge branch 'master' of github.com:ClickHouse/ClickHouse into feat/support-read-only-for-embeddedrocksdb 2022-09-02 18:10:00 +08:00
robot-clickhouse
46f74fcbaf Automatic style fix 2022-09-02 07:49:09 +00:00
Antonio Andelic
b02bf49939 Use path prefix 2022-09-02 07:34:26 +00:00
Lloyd-Pottiger
59dccd6e49 fix test 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-09-02 11:14:49 +08:00
Sergei Trifonov
7fec55eea4 work in progress 2022-09-02 04:12:05 +02:00
Frank Chen
9d63cbe811 Merge 'origin/master' into tracing_context_propagation to resolve conflicts 2022-09-01 23:18:59 +08:00
Vitaly Baranov
6dcca686cb
Merge pull request #40827 from vitlibar/fix-incremental-backups-for-log-family 
Fix incremental backups for Log family.
 2022-09-01 13:13:32 +02:00
Sema Checherinda
7b59fdc042
Merge pull request #40779 from CheSema/detached-parts-metric 
Metric for the number of detached parts
 2022-09-01 12:24:42 +02:00
Vitaly Baranov
007ae0e6cc Fix incremental backups for Log family. 2022-08-31 12:57:28 +02:00
Antonio Andelic
9dd1a9859d Merge branch 'master' into keeper-storage 2022-08-31 10:12:11 +00:00
root
ea517de61a style check correction 2022-08-30 23:20:19 -03:00
root
f02f7ce99c addressed minor change requests 2022-08-30 23:20:19 -03:00
root
84a617675d style check correction in test.py 2022-08-30 23:20:19 -03:00
root
3ced3c9933 style check correction in test.py 2022-08-30 23:20:19 -03:00
root
1d29a494db Modified test code to cover various scenarios for custom keys 2022-08-30 23:20:19 -03:00
root
d5db88fa27 Structured-logging-custom-keys PR init 2022-08-30 23:20:19 -03:00
Sema Checherinda
9bb4f556c3 work on remarks on the review 2022-08-30 20:18:35 +02:00
Antonio Andelic
3c098bd0cb Add integration test for KeeperMap 2022-08-30 13:41:57 +00:00
Alexander Tokmakov
022f440ad0
Merge pull request #40769 from arthurpassos/caresptrresolver-hosts-patch 
Fix CaresPTRResolver not reading hosts file
 2022-08-30 14:35:10 +03:00
alesapin
2588901bc9
Merge pull request #40780 from ClickHouse/add_cache_to_zero_copy_test 
Add cache to zero copy replication test
 2022-08-30 12:42:05 +02:00
Kseniia Sumarokova
c88db2ef97
Merge pull request #40751 from kssenii/fix-mysql-timeouts 
Fix issue with mysql db / table function timeouts
 2022-08-30 11:59:01 +02:00
Frank Chen
f17d56b528 Merge branch 'master' into tracing_context_propagation 2022-08-30 14:24:36 +08:00
Sema Checherinda
e436b4f4cc Metric for the number of detached parts 2022-08-30 02:12:57 +02:00
alesapin
ac41de73df Add cache to zero copy replication test 2022-08-30 00:17:01 +02:00
Kseniia Sumarokova
8001560082
Fix test 2022-08-30 00:14:00 +02:00
Vitaly Baranov
33f72fb011
Merge pull request #40060 from ClickHouse/vitlibar-increase-timeout-for-test_concurrent_backups 
Increase timeout for test_concurrent_backups
 2022-08-29 22:25:56 +02:00
Arthur Passos
dd49b44abb Fix host_regexp hosts file tst 2022-08-29 15:58:18 -03:00
Arthur Passos
961365c7a4 Fix CaresPTRResolver not reading hosts file 2022-08-29 15:11:39 -03:00
Kseniia Sumarokova
c5c48e44ea
Merge branch 'master' into fix-mysql-timeouts 2022-08-29 19:33:29 +02:00
kssenii
0a6c4b9265 Fix 2022-08-29 16:20:53 +02:00
Vladimir C
5cbe7e0846
Merge pull request #40548 from ClickHouse/vdimir/warn-suppress-40330 
Add config option warning_supress_regexp
 2022-08-29 14:02:00 +02:00
Alexander Tokmakov
8d90d30d37
Merge pull request #40589 from ClickHouse/remove_wrong_code_from_mutations 
Remove wrong code for skipping mutations in MergeTree
 2022-08-29 14:18:59 +03:00
Alexander Tokmakov
eda0582ec0
Merge pull request #40641 from ClickHouse/fix_startup_of_dropped_replica 
Do not try to strartup dropped replica
 2022-08-29 14:15:15 +03:00
Vitaly Baranov
2bec3d3a7c Increase timeout for test_concurrent_backups 2022-08-29 13:13:43 +02:00
alesapin
7ce0afc0df
Merge pull request #40670 from Avogar/kafka 
Add setting to disable limit on kafka_num_consumers
 2022-08-29 10:53:35 +02:00
Alexey Milovidov
b72fceb441
Merge pull request #40708 from lesandie/test_s3_table_functions 
Added integration test for s3 table function
 2022-08-29 03:49:53 +03:00
Alexey Milovidov
f462160bd1 Merge remote-tracking branch 'origin/master' into remove-useless-line 2022-08-28 21:00:29 +02:00
Diego Nieto (lesandie)
3d50dbea34 Black reformatting 2022-08-28 20:05:39 +02:00
Alexey Milovidov
fc29303ff3
Merge pull request #40685 from ClickHouse/nlp-functional-test 
Rewrite NLP tests from integration to functional
 2022-08-28 17:05:42 +03:00
Alexey Milovidov
f518844f06
Merge pull request #40709 from ClickHouse/all_new_table_functions_must_be_documented 
All new table functions must be documented
 2022-08-28 17:05:20 +03:00
Alexey Milovidov
18c56b722f Fix integration test 2022-08-28 04:30:27 +02:00
Alexey Milovidov
b42d26acfe Remove one line from XML, because we do not care 2022-08-28 02:44:02 +02:00
Alexey Milovidov
5b57c91ad6 Rename table function MeiliSearch to meilisearch 2022-08-28 02:08:51 +02:00
Diego Nieto (lesandie)
9a07489151 Added integration test for s3 table function 2022-08-27 18:33:04 +02:00
Alexey Milovidov
6414a01987 Rewrite NLP tests from integration to functional 2022-08-26 22:36:34 +02:00
Kruglov Pavel
d4d0ec1c95
Fix style 2022-08-26 16:44:54 +02:00
Alexander Tokmakov
7fa0cc02f2 fix tests 2022-08-26 15:33:04 +02:00
avogar
b8061532ac Add setting to disable limit on kafka_num_consumers 2022-08-26 12:49:39 +00:00
Kseniia Sumarokova
47d48a7b5c
Merge pull request #40579 from kssenii/add-test 
Add test for drop fs cache on cluster
 2022-08-26 12:55:13 +02:00
Lloyd-Pottiger
0d33327eda
Merge branch 'master' into feat/support-read-only-for-embeddedrocksdb 2022-08-26 15:36:21 +08:00
Kseniia Sumarokova
e2014c9872
Update test.py 2022-08-25 23:01:31 +02:00
Alexander Tokmakov
eae2667a1c
Merge pull request #40148 from ClickHouse/replicate-tables-readonly 
`ReplicatedMergeTree` tables as readonly until attach is finished
 2022-08-25 19:39:19 +03:00
Kseniia Sumarokova
e46d34c722
Update test.py 2022-08-25 18:37:25 +02:00
Alexander Tokmakov
a6f55b1430 fix 2022-08-25 16:27:12 +02:00
robot-clickhouse
6e538f04c4 Automatic style fix 2022-08-25 12:30:57 +00:00
Alexander Tokmakov
0aea4c7184 fix 2022-08-25 14:22:27 +02:00
alesapin
7d1e3ecfa3
Merge pull request #40506 from ClickHouse/fix_bad_exception 
Fix lost errors in AWS CPP SDK
 2022-08-25 13:07:35 +02:00
alesapin
c4f8811701
Merge pull request #40395 from ClickHouse/data_source_description 
Add copy shortcut for backups if dest and source support it
 2022-08-25 12:36:30 +02:00
Kseniia Sumarokova
73bb5a6b9e
Update test.py 2022-08-25 12:10:46 +02:00
robot-clickhouse
669ae365c1 Automatic style fix 2022-08-25 11:00:01 +02:00
vdimir
8bc29abdfa Test warning_supress_regexp in test_log_levels_update 2022-08-25 11:00:01 +02:00
Frank Chen
99c37ce6c6
Merge branch 'master' into tracing_context_propagation 2022-08-25 10:07:16 +08:00
Kseniia Sumarokova
bfc3180c9e
Update test.py 2022-08-25 00:20:50 +02:00
Alexey Milovidov
aaafb797c0
Merge pull request #40541 from ClickHouse/paranoid_check_for_s3_uploads 
Add paranoid option to check blobs to S3 after upload
 2022-08-24 23:51:22 +02:00
Alexander Tokmakov
3cd26aafe4 remove wrong code for skipping mutations in MergeTree 2022-08-24 20:58:59 +02:00
Antonio Andelic
da0d6056f4 Merge branch 'master' into replicate-tables-readonly 2022-08-24 17:29:33 +00:00
Lloyd-Pottiger
9d7ace3286
Merge branch 'master' into feat/support-read-only-for-embeddedrocksdb 2022-08-24 23:52:34 +08:00
Lloyd-Pottiger
dc64341876 add tests 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-08-24 23:42:13 +08:00
Alexander Tokmakov
f68db41c1c
Merge pull request #40575 from Avogar/try-fix-s3-test 
Fix flaky schema inference cache tests
 2022-08-24 18:24:05 +03:00
alesapin
eeb0a3584e Merge branch 'master' into data_source_description 2022-08-24 15:26:18 +02:00
Kseniia Sumarokova
6ac35dabcf
Update test.py 2022-08-24 15:03:02 +02:00
kssenii
1033c14859 Add test 2022-08-24 15:01:21 +02:00
alesapin
f0b733313f Merge branch 'master' into fix_bad_exception 2022-08-24 14:44:40 +02:00
alesapin
cac3005f4d Merge branch 'master' into paranoid_check_for_s3_uploads 2022-08-24 14:43:11 +02:00
Alexander Tokmakov
f9f85a0e8b Revert "Parallel distributed insert select from *Cluster table functions (#39107)" 
This reverts commit d3cc234986.
 2022-08-24 15:17:15 +03:00
avogar
93d0113b14 Fix flacky schema inference cache tests 2022-08-24 11:28:56 +00:00
Alexander Tokmakov
c1d933b746
Update test.py 2022-08-24 13:39:45 +03:00
alesapin
dc95d86d6e
Merge pull request #40064 from ClickHouse/keeper-force-recovery-more-retries 
Add retries for more commands in `test_keeper_force_recovery`
 2022-08-23 15:44:28 +02:00
robot-clickhouse
f0b18a1082 Automatic style fix 2022-08-23 13:21:29 +00:00
alesapin
07286dd042 Add paranoid option to check blobs to S3 after upload 2022-08-23 15:12:09 +02:00
alesapin
7c7fa9472f One more comment 2022-08-23 14:07:50 +02:00
alesapin
2f478774e5 Add comment to proxy 2022-08-23 14:05:05 +02:00
Vadim Volodin
9c371d518f fix enable_unfreeze handler and sanitizer hints 2022-08-23 13:53:06 +03:00
alesapin
802d56df49 Merge branch 'master' into fix_bad_exception 2022-08-23 12:22:05 +02:00
Antonio Andelic
0765495be1
Merge pull request #39986 from Lloyd-Pottiger/feat/support-ttl-for-embeddedrocksdb 
Support TTL for EmbeddedRocksDB
 2022-08-23 08:27:47 +02:00
robot-clickhouse
315d38c365 Automatic style fix 2022-08-22 22:43:00 +00:00
alesapin
780f39b108 Fix lost errors in AWS CPP SDK 2022-08-23 00:34:39 +02:00
Lloyd-Pottiger
85a1a36167 remove read_only option 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-08-22 23:31:55 +08:00
alesapin
b4d6afd5b3 Black 2022-08-22 13:51:48 +02:00
alesapin
704d7fdc41 Fix copy and use disks 2022-08-21 18:18:35 +02:00
Antonio Andelic
76ecb28daf Increase timeouts 2022-08-20 11:52:10 +00:00
Frank Chen
024fef1aec
Merge branch 'master' into tracing_context_propagation 2022-08-20 14:23:36 +08:00
Azat Khuzhin
b92d8b302a tests: fix test_storage_s3::test_wrong_format_usage flakiness 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-08-19 21:51:23 +02:00
Sergei Trifonov
7c18d5b34f
Merge pull request #40348 from ClickHouse/threads-soft-limit-rename 
split concurrent_threads_soft_limit setting into two
 2022-08-19 13:39:32 +02:00
Kruglov Pavel
c4f9508b80
Merge pull request #40355 from Avogar/better-schema-cache-test 
Make test for s3 schema inference cache better
 2022-08-19 12:17:49 +02:00
Antonio Andelic
5e2c8869a9 Merge branch 'master' into replicate-tables-readonly 2022-08-19 09:41:11 +00:00
Antonio Andelic
3f7d28cc6b Address PR comments 2022-08-19 09:39:59 +00:00
Lloyd-Pottiger
51f53af482 fix tests 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-08-19 15:24:00 +08:00
avogar
67f848a428 Fix style 2022-08-18 17:15:30 +00:00
avogar
1fb99f51b0 Make test for s3 schema inference cache better 2022-08-18 16:50:59 +00:00
Frank Chen
a3b6ad2a65
Merge branch 'master' into tracing_context_propagation 2022-08-18 20:59:07 +08:00
Sergei Trifonov
42a711015f split concurrent_threads_soft_limit setting into two: concurrent_threads_soft_limit_{num,ratio_to_cores} 2022-08-18 14:51:08 +02:00
Nikolai Kochetov
5a85531ef7
Merge pull request #38286 from Avogar/schema-inference-cache 
Add schema inference cache for s3/hdfs/file/url
 2022-08-18 13:07:50 +02:00
Kseniia Sumarokova
d8fd1f9c37
Merge pull request #36171 from kssenii/make-cache-composable 
Make cache composable, allow not to evict certain files (regarding .idx, .mrk, ..), delete old cache version
 2022-08-18 12:37:34 +02:00
Vladimir C
7ac0eb1af1
Merge pull request #40035 from ClickHouse/vdimir/cluster-discovery-observer 
Add observer mode to (zoo)keeper cluster discovery feature
 2022-08-18 10:53:49 +02:00
Lloyd-Pottiger
340c8d1c18 address comments 
Signed-off-by: Lloyd-Pottiger <yan1579196623@gamil.com>
 2022-08-17 23:27:18 +08:00
Lloyd-Pottiger
19c4385881
Merge branch 'master' into feat/support-ttl-for-embeddedrocksdb 2022-08-17 22:17:10 +08:00