Antonio Andelic
e84e05d849
Modify visitor
2022-11-23 10:27:38 +00:00
Antonio Andelic
84884df595
Substitute UDFs in CREATE queries
2022-11-23 09:28:08 +00:00
Alexander Tokmakov
7417d8b7c0
Merge pull request #43038 from Algunenano/revert_SingleValueDataString
...
Fix regression on SingleValueDataString
2022-11-22 14:28:30 +03:00
Alexander Tokmakov
cab33c02af
Merge branch 'master' into revert_SingleValueDataString
2022-11-22 14:27:58 +03:00
Antonio Andelic
843401c3a3
Merge pull request #43323 from ClickHouse/fix-describe-delta-lake-hudi
...
Fix `DESCRIBE` for `deltaLake` and `hudi` table functions
2022-11-22 09:09:39 +01:00
Anton Popov
0479d8d9f5
Merge pull request #43387 from ClickHouse/fix_default_value_used_in_row_level_filter
...
Fix default value used in row level filter
2022-11-22 01:46:45 +01:00
Vitaly Baranov
6c2b53f0ae
Merge pull request #42940 from filimonov/exceptions_with_masker
...
Add SensitiveDataMasker to exceptions messages
2022-11-21 23:20:15 +01:00
Kruglov Pavel
581e57be9f
Merge pull request #43236 from xiedeyantu/fix_hdfscluster_connection_fail
...
fix skip_unavailable_shards does not work using hdfsCluster table function
2022-11-21 19:22:16 +01:00
Igor Nikonov
47ce029cbf
Merge pull request #43326 from ClickHouse/igor/test_read_only_fast_fix
...
Fix: make test_read_only_table more stable
2022-11-21 18:25:14 +01:00
Nikolay Degterinsky
65b5523bdb
Merge pull request #43350 from evillique/fix-parser-5
...
Fix function parameters parsing
2022-11-21 16:23:02 +01:00
Kruglov Pavel
406c50277b
Merge pull request #42779 from Avogar/add-format-func-doc
...
Add documentation for 'format' table function
2022-11-21 15:16:26 +01:00
Raúl Marín
35e9e169a1
Merge branch 'master' into revert_SingleValueDataString
2022-11-21 15:11:13 +01:00
Alexey Milovidov
bb5756bb11
Merge pull request #43403 from ClickHouse/table-function-add-comment
...
Add a comment
2022-11-21 15:06:20 +01:00
Kseniia Sumarokova
a59dac6337
Merge pull request #43143 from azat/disks/s3-check-fix
...
Add server UUID for disks access checks (read/read-by-offset/write/delete) to avoid possible races
2022-11-21 15:03:46 +01:00
Raúl Marín
a52bdca989
Detect the other corrupted state too
2022-11-21 13:33:16 +01:00
Alexey Milovidov
d0560fda00
Merge pull request #43393 from ClickHouse/auto/v22.8.9.24-lts
...
Update version_date.tsv and changelogs after v22.8.9.24-lts
2022-11-21 13:09:16 +01:00
Alexander Tokmakov
fffb711097
add tests
2022-11-21 12:54:21 +01:00
alesapin
f4ef20b5e4
Merge pull request #43295 from ClickHouse/cancel-lambda-api-url
...
Cancel lambda api url
2022-11-21 12:03:37 +01:00
Han Fei
874a1a9d83
Merge pull request #43408 from azat/tests/fix-01926_order_by_desc_limit
...
Fix flaky 01926_order_by_desc_limit
2022-11-21 11:58:33 +01:00
Antonio Andelic
3cb202a63b
Merge pull request #43026 from JackyWoo/keeper_manually_assign_leader
...
Keeper support manually assigning leader
2022-11-21 09:30:00 +01:00
Antonio Andelic
9a8d82f0fc
Merge branch 'master' into fix-describe-delta-lake-hudi
2022-11-21 08:23:33 +00:00
Antonio Andelic
af647baeb2
Merge pull request #43315 from ClickHouse/fix-merge-sample-by
...
Fix `SAMPLE BY` with Merge tables
2022-11-21 09:09:46 +01:00
Nikolay Degterinsky
f3c16ec8ed
Merge pull request #41874 from evillique/better-set-params
...
Don't require manual string serialization to set query parameters
2022-11-21 04:09:16 +01:00
Azat Khuzhin
1831aa45d9
Fix flaky 01926_order_by_desc_limit
...
CI captures [1] too long logs flush:
$ clickhouse-local --file query_log.tsv.gz --input-format TabSeparatedWithNamesAndTypes -q "select event_time, query from table where Settings['log_comment'] = '01926_order_by_desc_limit.sql' and type = 'QueryFinish' and (lower(query) LIKE lower('SELECT s FROM order_by_desc ORDER BY u%') or query like '%query_log%') format Vertical"
Row 1:
──────
event_time: 2022-11-20 12:46:42
query: SELECT s FROM order_by_desc ORDER BY u DESC LIMIT 10 FORMAT Null
SETTINGS max_memory_usage = '400M';
Row 2:
──────
event_time: 2022-11-20 12:46:47
query: SELECT s FROM order_by_desc ORDER BY u LIMIT 10 FORMAT Null
SETTINGS max_memory_usage = '400M';
Row 3:
──────
event_time: 2022-11-20 12:46:54
query: SELECT read_rows < 110000 FROM system.query_log
WHERE type = 'QueryFinish' AND current_database = currentDatabase()
AND event_time > now() - INTERVAL 10 SECOND
AND lower(query) LIKE lower('SELECT s FROM order_by_desc ORDER BY u%');
[1]: https://s3.amazonaws.com/clickhouse-test-reports/43143/c393af2812cb56a60a99d48c15c0443d2da98a8b/stateless_tests__tsan__[2/5].html
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-20 21:03:22 +01:00
Azat Khuzhin
c393af2812
Fix is_read_only property of local disks
...
But it wasn't a problem before, since before it was not possible to have
readonly==true.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-20 16:35:41 +01:00
Azat Khuzhin
e6e223adc2
Check if local disk is readonly before access checks
2022-11-20 16:35:32 +01:00
Azat Khuzhin
cb60576221
Change DiskLocal::setup() signature (it always return true)
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-20 16:31:37 +01:00
Azat Khuzhin
38c009214a
Change the name pattern for memory disks
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-20 16:28:35 +01:00
Azat Khuzhin
3e42ffd372
tests: fix hdfs disks (chicken and an egg problem with current cluster.py)
...
Right now cluster.py first creates HDFS and then creates clickhouse in
one API call, so you cannot interract and add missing mkdirs for the
clickhouse, fix this by using root dir where it is possible.
2022-11-20 16:28:35 +01:00
Azat Khuzhin
0678fba3d1
tests: fix test_disk_types (by adding mkdir for HDFS)
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-20 16:28:35 +01:00
Azat Khuzhin
11be9b9ad1
Create disk directory before access check for local disk
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-20 16:28:35 +01:00
Azat Khuzhin
dddcca5cc1
Fix deadlock in DiskRestartProxy on disk restart
...
stacktrace:
contrib/libcxx/src/condition_variable.cpp:47::std::__1::condition_variable::wait(std::__1::unique_lock<std::__1::mutex>&)
contrib/libcxx/src/shared_mutex.cpp:65::std::__1::shared_timed_mutex::lock_shared()
src/Disks/DiskRestartProxy.cpp:229::DB::DiskRestartProxy::writeFile(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, unsigned long, DB::WriteMode, DB::WriteSettings const&)
src/Disks/IDisk.cpp:0::DB::IDisk::checkAccessImpl(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&)
contrib/libcxx/include/string:1499::DB::IDisk::checkAccess()
src/Disks/IDisk.cpp:0::DB::IDisk::startup(std::__1::shared_ptr<DB::Context const>, bool)
src/Disks/DiskRestartProxy.cpp:375::DB::DiskRestartProxy::restart(std::__1::shared_ptr<DB::Context const>)
contrib/libcxx/include/__memory/shared_ptr.h:701::DB::InterpreterSystemQuery::restartDisk(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&)
src/Interpreters/InterpreterSystemQuery.cpp:508::DB::InterpreterSystemQuery::execute()
src/Interpreters/executeQuery.cpp:0::DB::executeQueryImpl(char const*, char const*, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum, DB::ReadBuffer*)
src/Interpreters/executeQuery.cpp:1083::DB::executeQuery(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::shared_ptr<DB::Context>, bool, DB::QueryProcessingStage::Enum)
src/Server/TCPHandler.cpp:0::DB::TCPHandler::runImpl()
src/Server/TCPHandler.cpp:1904::DB::TCPHandler::run()
contrib/poco/Net/src/TCPServerConnection.cpp:57::Poco::Net::TCPServerConnection::start()
contrib/libcxx/include/__memory/unique_ptr.h:48::Poco::Net::TCPServerDispatcher::run()
contrib/poco/Foundation/src/ThreadPool.cpp:213::Poco::PooledThread::run()
contrib/poco/Foundation/include/Poco/SharedPtr.h:156::Poco::ThreadImpl::runnableEntry(void*)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-20 16:28:35 +01:00
Azat Khuzhin
44f23c2568
Make disks checks only for clickhouse-server
...
This will fix clickhouse-disks
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-20 16:28:35 +01:00
Azat Khuzhin
2efd29f49d
Implement access (read/read-by-offset/write/delete) check for all disks
...
Previously we had such access (read/write/delete) checks only for S3 and
Azure disks (read/read-by-offset/write/delete), this patch adds check
for all disks.
Also I've added the disk name into IDisk interface, since it is required
for the error message in IDisk::checkAccess(), but I have to add
DiskEncrypted::encrypted_name due DiskEncrypted inherits from
DiskDecorator not from IDisk, and so does not have ability to set disk
name (DiskEncrypted could pass the disk name to the DiskDecorator, but
it is not used anywere, and besides this will require to duplicate the
name for each user of DiskDecorator).
Also from nwo on, skip_access_check will make sense for every disk, plus
now startup() called for each disk (before it was missed for some of
them).
And I've added skip_access_check as as a member for DiskRestartProxy,
since it calls startup() on restart().
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-20 16:28:35 +01:00
Azat Khuzhin
324b1a7658
Add server UUID for the S3 disks checks to avoid possible races
...
Otherwise, if you are lucky enough, race condition is possible, and you
can get some errors because one server already removed the file while
another was trying to read it.
But it was possible only for:
- batch deletes check for "s3" disk
- and all checks for "s3_plain" disk, since this disk does not uses
random names
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-20 16:11:45 +01:00
Kseniia Sumarokova
f0dbfbb0f4
Merge pull request #42800 from azat/disks/web-fix
...
Do not suppress exceptions in web disk (and fix retries for requests from web disk)
2022-11-20 16:07:45 +01:00
Kseniia Sumarokova
c12cfab953
Merge pull request #43343 from azat/disks/write-once
...
Allow to "drop tables" from s3_plain disk (so as from web disk)
2022-11-20 16:04:50 +01:00
Antonio Andelic
77eddd702a
Merge branch 'master' into fix-merge-sample-by
2022-11-20 14:38:53 +01:00
Alexey Milovidov
b3a06175e7
Add a comment
2022-11-20 07:47:20 +01:00
Alexander Tokmakov
854657d372
fix another issue
2022-11-19 23:14:53 +01:00
Alexander Tokmakov
a00f7d142d
fix flaky test
2022-11-19 21:02:48 +01:00
Alexander Tokmakov
09de30a51a
fix overflow, add more tests
2022-11-19 18:58:40 +01:00
Alexander Tokmakov
9464c32bb8
Merge branch 'master' into revert_SingleValueDataString
2022-11-19 18:00:04 +01:00
Nikita Taranov
0b4e643c27
Add check to CompressionCodecDelta ( #43255 )
...
* Add check to CompressionCodecDelta
* Apply suggestions from code review
* Update src/Compression/CompressionCodecDelta.cpp
2022-11-19 14:16:14 +01:00
alesapin
93a4950dae
Merge pull request #43222 from zhongyuankai/optimize_ttl
...
Priority is given to deleting completely expired Parts
2022-11-19 13:51:51 +01:00
alesapin
3d497cb9d4
Merge pull request #43373 from ClickHouse/trying_to_split_checks
...
Splitting checks in CI more
2022-11-19 13:42:15 +01:00
robot-clickhouse
4ee5d8bd43
Update version_date.tsv and changelogs after v22.8.9.24-lts
2022-11-19 11:51:14 +00:00
Azat Khuzhin
4f6703c972
Add is_read_only/is_write_once/is_remote/is_broken to system.disks
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-19 10:11:39 +01:00
Azat Khuzhin
c029549859
Allow to drop tables from s3_plain disk (so as from web disk)
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-19 10:10:27 +01:00
Azat Khuzhin
e2726e03cc
Override DiskDecorator::isReadOnly()
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-19 10:10:27 +01:00