thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-17 11:52:27 +00:00
Code Issues Packages Projects Releases Wiki Activity
100,878 Commits 9,326 Branches 1,784 Tags 2.1 GiB
679591cf0a
Commit Graph

33136 Commits

Author SHA1 Message Date
Alexander Tokmakov
599ccb9939
Update MergeTreeData.h 2022-10-31 16:18:17 +03:00
Alexander Tokmakov
b2b9479afa
Update src/Storages/MergeTree/MergeTreeData.h 
Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>
 2022-10-31 16:15:35 +03:00
SmitaRKulkarni
54035270b2
Merge branch 'master' into 42414_Enable_functions_in_readonly_mode 2022-10-31 14:15:29 +01:00
SmitaRKulkarni
7e9dd825c0
Merge branch 'master' into 42452_Bug_fix_parent_node_corrupted 2022-10-31 14:03:44 +01:00
Smita Kulkarni
c97c78e3ff Updated normaliser to clone the alias ast. 
Implementation:
* Updated QueryNormalizer to clone alias ast, when its replaced. Previously just assigning the same leads to exception in LogicalExpressinsOptimizer as it would be the same parent being inserted again.
* This bug is not seen with new analyser (allow_experimental_analyzer), so no changes for it. I added a test for the same.
Testing:
* Added a test for or function with alias and const where 02475_or_function_alias_and_const_where.sql
 2022-10-31 13:49:31 +01:00
Maksim Kita
bca22ec5f5 Fixed code review issues 2022-10-31 12:30:00 +01:00
Nikolay Degterinsky
d8d479a8c0
Merge pull request #42804 from evillique/fix-create-user-parser 
Fix a bug in ParserCreateUserQuery
 2022-10-31 12:29:06 +01:00
Sergei Trifonov
8c67257a1c
Merge pull request #42665 from ClickHouse/token-bucket-throttler 
Replace throttler algorithm by token bucket
 2022-10-31 12:28:18 +01:00
Azat Khuzhin
c93262170d Remove ReadOnlyMetadataStorage 
Throw exceptions from IMetadataStorage instead to avoid introducing
extra abstractions.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:30 +01:00
Azat Khuzhin
51bd0c2ac1 Make ReadOnlyMetadataStorage really readonly 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:26 +01:00
Azat Khuzhin
bceca73f6f Throw NOT_IMPLEMENTED form ReadOnlyMetadataStorage::getLastChanged() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:26 +01:00
Azat Khuzhin
3b7abbbff4 Rename s/listPrefix/findAllFiles, s/listPrefixInPath/getDirectoryContents/ 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:26 +01:00
Azat Khuzhin
fa7535c90d Slightly optimize MetadataStorageFromPlainObjectStorage::isDirectory() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:26 +01:00
Azat Khuzhin
82ea67eb51 Slightly better MetadataStorageFromPlainObjectStorage::isFile() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:26 +01:00
Azat Khuzhin
95fb2ad3cf Implement ATTACH of MergeTree table for s3_plain disk 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:26 +01:00
Azat Khuzhin
4e42521f44 Introduce ReadOnlyMetadataStorage 
And use it for:
- MetadataStorageFromPlainObjectStorage
- MetadataStorageFromStaticFilesWebServer

This will allow to reduce ~100-200 lines of duplicated code, and plus
make the code less error prone.

Note, for now I tried to make this without behaviour changes.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:26 +01:00
Azat Khuzhin
f8ba24f040 Implement S3ObjectStorage::listPrefixInPath() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:26 +01:00
Azat Khuzhin
18e4fdf40f Introduce IObjectStorage::listPrefixInPath() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:25 +01:00
Azat Khuzhin
ee18140c48 Remove listPrefix() implementation for disks that does not support send_metadata 
The reason for removing is it because not compatible with restoring
(with send_metadata set) anyway:
- HDFS - is not compatible with send_metadata, and besides it's
  implementaion is not correct, since it is simply `ls -l`, while the
  following is required: `find . -maxdepth 1 -type f`
- Web - is not compatible with send_metadata anyway
- Local - is not compatible with send_metadata anyway

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:25 +01:00
Azat Khuzhin
88db8ae7fa Add a comment for IObjectStorage::listPrefix() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:25 +01:00
Azat Khuzhin
94d9600fb8 Implement MetadataStorageFromPlainObjectStorage::getLastChanged() (as for web) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:25 +01:00
Azat Khuzhin
b34ffda272 Implement MetadataStorageFromPlainObjectStorage::getLastModified() (used by MergeTree) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:25 +01:00
Azat Khuzhin
f6d0c03ee5 Fix path to files in MetadataStorageFromPlainObjectStorage 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:25 +01:00
Maksim Kita
d760c78754 Fixed code review issues 2022-10-31 11:45:12 +01:00
Nikolai Kochetov
742d9af397
Merge pull request #42582 from azat/memory-tracker/fixes 
Fix frequent memory drift message and clarify things in comments
 2022-10-31 11:43:20 +01:00
Maksim Kita
115fcaffc5 Fixed tests 2022-10-31 11:30:56 +01:00
Vitaly Baranov
c07fe6921f Add comments. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
854e1e918f Use assert_cast() instead of as() in some places. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
43efbad642 Wipe passwords from distributed queries too. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
a30bfada63 Wipe passwords from backup logs too. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
dcf8724adf Move prepareQueryForLogging() to a separate header. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
f2bd560866 Fix IAST::clone() overrides for some queries. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
c2cc2ccc99 Make as_table_function a child of ASTCreateQuery (to help writing visitors). 2022-10-31 10:50:33 +01:00
Vitaly Baranov
5d2a222fe4 Mask sensitive information in logs. 2022-10-31 10:50:33 +01:00
Azat Khuzhin
5fe44f2736 Fix lowerUTF8()/upperUTF8() in case of symbol was in between 16-byte boundary 
In lowerUTF8()/upperUTF8() there is an SSE optimization that handles
16 byte at a time, but only for ASCII, for UTF8 symbols converion will
be done by symbol.

Consider the following example:

    КВ АМ И СЖ
             ^ - offset is 15, length of sequence is 2
                 so first byte of a symbol is in first 16 bytes
                 second byte of a symbol is not ther

And in this case it will be handled incorrectly because it will try to
process oly these 16 bytes w/o looking forward.

This had been broken by #41286, before this patch it does not looks at
the row boundaries but only at the string end and so this sutation
wasn't possible.

Fixes: #42756
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 08:17:38 +01:00
Duc Canh Le
d309e8793a
Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-10-31 15:13:39 +08:00
Azat Khuzhin
32febf5155 Remove dead code in LowerUpperUTF8Impl::array() 
AFAICS it was there before since it was possible to overrun the
expected_end, since utf8.convert() was called with "src_end - src" not
"expected_end - src".

Refs: 5a21f3908b054a0efc90c65a12fbe151c74d90dc:dbms/include/DB/Functions/FunctionsString.h
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 08:00:29 +01:00
Gabriel
94866979c5
Merge branch 'master' into comment_typo 2022-10-31 13:29:53 +08:00
Gabriel
620caeb07c Fix typo in comments 2022-10-31 13:20:58 +08:00
Nikolay Degterinsky
05e178b6d3
Merge pull request #42767 from azat/disks/web-sigsegv-fix 
Fix possible SIGSEGV for web disks when file does not exists
 2022-10-30 20:27:21 +01:00
Alexander Tokmakov
f53df7870c fix race between drop and failed insert 2022-10-30 17:59:47 +01:00
Nikolay Degterinsky
978aa16e29 Fix a bug in ParserCreateUserQuery 2022-10-30 16:42:57 +00:00
Alexander Tokmakov
c68ab231f9 fix accessing part in Deleting state 2022-10-30 17:30:51 +01:00
Roman Vasin
24a4f4afe7
Merge branch 'master' into ADQM-605 2022-10-30 18:03:07 +03:00
Anton Popov
8de75bf29c better code with projections 2022-10-29 14:39:59 +00:00
Nikolay Degterinsky
6d1b547586
Merge pull request #42724 from evillique/fix-parser-bug 
Fix a bug in ParserFunction
 2022-10-29 13:59:20 +02:00
clarkcaoliu
ca5dbe88db match function can use index if prefix 2022-10-29 18:25:29 +08:00
avogar
fe0aea2e3a Support parallel parsing for LineAsString input format 2022-10-28 21:56:09 +00:00
Miel Donkers
fb1623a5f8 Add SSL_CERTIFICATE auth method to all places missing 2022-10-28 21:17:59 +02:00
Sergei Trifonov
c948303fa0
Merge branch 'master' into token-bucket-throttler 2022-10-28 16:21:55 +02:00
Alexey Milovidov
f64e632707
Merge pull request #42469 from amosbird/issue_42456 
Fix misbehavior of key analysis
 2022-10-28 16:18:00 +02:00
Azat Khuzhin
7a5432feaa Fix possible SIGSEGV for web disks when file does not exists 
It can be triggered in multiple ways, either when file does not exists
and you are trying to create MergeTree table from web (that has special
code for UUID handling) or by simply OPTIMIZE TABLE FINAL for MergeTree
table that is located on web disk, in both cases you will get the
following:

<details>

<summary>stacktrace</summary>

    2022.10.28 14:08:40.631226 [ 6043 ] {6165bf5f-e76b-4bca-941c-7c7ff5e3b46b} <Trace> ContextAccess (default): Access granted: OPTIMIZE ON default.data_from_web
    2022.10.28 14:08:40.632017 [ 6043 ] {6165bf5f-e76b-4bca-941c-7c7ff5e3b46b} <Debug> default.data_from_web (a3e65e1f-5fd4-47ed-9dbd-307f2586b52d) (MergerMutator): Selected 1 parts from all_1_1_0 to all_1_1_0
    2022.10.28 14:08:40.632496 [ 6043 ] {6165bf5f-e76b-4bca-941c-7c7ff5e3b46b} <Trace> default.data_from_web (a3e65e1f-5fd4-47ed-9dbd-307f2586b52d): Trying to reserve 1.00 MiB using storage policy from min volume index 0
    2022.10.28 14:08:40.632752 [ 6043 ] {6165bf5f-e76b-4bca-941c-7c7ff5e3b46b} <Trace> DiskObjectStorage(DiskWebServer): Reserved 1.00 MiB on remote disk `web_disk`, having unreserved 16.00 EiB.
    2022.10.28 14:08:40.634155 [ 6043 ] {a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1} <Debug> MergeTask::PrepareStage: Merging 1 parts: from all_1_1_0 to all_1_1_0 into Compact
    2022.10.28 14:08:40.634498 [ 6043 ] {a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1} <Trace> WebObjectStorage: Loading metadata for directory: http://127.0.0.1:8080/store/a3e/a3e65e1f-5fd4-47ed-9dbd-307f2586b52d/tmp_merge_all_1_1_1
    2022.10.28 14:08:40.635025 [ 6043 ] {a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1} <Trace> DiskWeb: Adding directory: store/a3e/a3e65e1f-5fd4-47ed-9dbd-307f2586b52d/tmp_merge_all_1_1_1/
    2022.10.28 14:08:40.635355 [ 6043 ] {a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1} <Trace> ReadWriteBufferFromHTTP: Sending request to http://127.0.0.1:8080/store/a3e/a3e65e1f-5fd4-47ed-9dbd-307f2586b52d/tmp_merge_all_1_1_1/.index
    2022.10.28 14:08:40.639618 [ 6043 ] {a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1} <Trace> DiskWeb: Cannot load disk metadata. Error: Code: 86. DB::Exception: Received error from remote server /store/a3e/a3e65e1f-5fd4-47ed-9dbd-307f2586b52d/tmp_merge_all_1_1_1/.index. HTTP status code: 404 Not Found, body: <!doctype html><html><head><title>404 Not Found</title><style>
    body { background-color: #fcfcfc; color: #333333; margin: 0; padding:0; }
    h1 { font-size: 1.5em; font-weight: normal; background-color: #9999cc; min-height:2em; line-height:2em; border-bottom: 1px inset black; margin: 0; }
    h1, p { padding-left: 10px; }
    code.url { background-color: #eeeeee; font-family:monospace; padding:0 2px;}
    </style>
    </head><body><h1>Not Found</h1><p>The requested resource <code class="url">/store/a3e/a3e65e1f-5fd4-47ed-9dbd-307f2586b52d/tmp_merge_all_1_1_1/.index</code> was not found on this server.</p></body></html>: while loading disk metadata. (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER) (version 22.11.1.1)
    2022.10.28 14:08:40.640527 [ 5488 ] {} <Trace> BaseDaemon: Received signal 11
    2022.10.28 14:08:40.641529 [ 9027 ] {} <Fatal> BaseDaemon: ########################################
    2022.10.28 14:08:40.642759 [ 9027 ] {} <Fatal> BaseDaemon: (version 22.11.1.1, build id: 12145DA78CE5E9EBB10A034177FAE5967EF81A4A) (from thread 6043) (query_id: a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1) (query: optimize table data_from_web final) Received signal Segmentation fault (11)
    2022.10.28 14:08:40.643260 [ 9027 ] {} <Fatal> BaseDaemon: Address: NULL pointer. Access: read. Unknown si_code.
    2022.10.28 14:08:40.643769 [ 9027 ] {} <Fatal> BaseDaemon: Stack trace: 0x7ffff416c0f2 0x7ffff7cd1ca8 0x7ffff679ae5e 0x7fffd52e7906 0x7fffd50c65aa 0x7fffca7a0d42 0x7fffcaee79ec 0x7fffcaf242f8 0x7fffcaf242b5 0x7fffcaf2427d 0x7fffcaf24255 0x7fffcaf2421d 0x7ffff65c3686 0x7ffff65c2295 0x7fffcaeee2a9 0x7fffcaef2c43 0x7fffcaee3c0e 0x7fffcc4a7851 0x7fffcc4a768f 0x7fffcc4abb2d 0x7fffcfdce828 0x7fffd03e3eaa 0x7fffd03dfe3b 0x7fffc8ec42d4 0x7fffc8ed51d2 0x7ffff4bdd839 0x7ffff4bde0a8 0x7ffff48ab261 0x7ffff48a769a 0x7ffff48a6335 0x7ffff409f8fd 0x7ffff4121a60
    2022.10.28 14:08:40.644411 [ 9027 ] {} <Fatal> BaseDaemon: 4. ? @ 0x7ffff416c0f2 in ?
    2022.10.28 14:08:40.676390 [ 9027 ] {} <Fatal> BaseDaemon: 5. /src/ch/clickhouse/src/Common/StringUtils/StringUtils.cpp:9: detail::startsWith(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, char const*, unsigned long) @ 0x1ca8 in /src/ch/clickhouse/.cmake/src/Common/StringUtils/libstring_utilsd.so
    2022.10.28 14:08:40.730727 [ 9027 ] {} <Fatal> BaseDaemon: 6. /src/ch/clickhouse/src/Common/StringUtils/StringUtils.h:19: startsWith(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) @ 0x59ae5e in /src/ch/clickhouse/.cmake/src/libclickhouse_common_iod.so
    2022.10.28 14:08:40.923955 [ 9027 ] {} <Fatal> BaseDaemon: 7. /src/ch/clickhouse/src/Disks/ObjectStorages/Web/MetadataStorageFromStaticFilesWebServer.cpp:58: DB::MetadataStorageFromStaticFilesWebServer::exists(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) const @ 0x6e7906 in /src/ch/clickhouse/.cmake/src/libdbmsd.so
    2022.10.28 14:08:41.291996 [ 9027 ] {} <Fatal> BaseDaemon: 8. /src/ch/clickhouse/src/Disks/ObjectStorages/DiskObjectStorage.cpp:181: DB::DiskObjectStorage::exists(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) const @ 0x4c65aa in /src/ch/clickhouse/.cmake/src/libdbmsd.so
    2022.10.28 14:08:41.704697 [ 9027 ] {} <Fatal> BaseDaemon: 9. /src/ch/clickhouse/src/Storages/MergeTree/DataPartStorageOnDisk.cpp:74: DB::DataPartStorageOnDisk::exists() const @ 0xda0d42 in /src/ch/clickhouse/.cmake/src/libclickhouse_storages_mergetreed.so
    2022.10.28 14:08:43.032459 [ 9027 ] {} <Fatal> BaseDaemon: 10. /src/ch/clickhouse/src/Storages/MergeTree/MergeTask.cpp:147: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x14e79ec in /src/ch/clickhouse/.cmake/src/libclickhouse_storages_mergetreed.so
    ...
    Segmentation fault (core dumped)

</details>

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-28 15:15:49 +02:00
Roman Vasin
5755728b33 Add Date32 and DateTime64 into exception messages 2022-10-28 13:11:09 +00:00
Alexander Tokmakov
11f1f5ae4d
Merge pull request #42747 from jrdi/fix-implicit-cast-CaresPTRResolver 
Minor fix implicit cast CaresPTRResolver
 2022-10-28 15:50:11 +03:00
Roman Vasin
15f3f56812 Change 1 argument of function to first argument of function etc 2022-10-28 12:44:47 +00:00
Antonio Andelic
5fae8b0cc9 Merge branch 'master' into optimize-entire-partition 2022-10-28 11:44:10 +00:00
Sergei Trifonov
eda264b492
Merge branch 'master' into token-bucket-throttler 2022-10-28 11:38:13 +02:00
Maksim Kita
835daee0f3
Merge pull request #42728 from ucasfl/fix 
Fix truncate table does not hold lock correctly
 2022-10-28 11:44:24 +03:00
Maksim Kita
ba8b806957
Merge pull request #42704 from kitaisreal/order-by-limit-by-duplicate-elimination-improve-performance 
OrderByLimitByDuplicateEliminationPass improve performance
 2022-10-28 11:43:53 +03:00
Jordi Villar
534dbc159c
Merge branch 'master' into fix-implicit-cast-CaresPTRResolver 2022-10-28 08:00:22 +02:00
Duc Canh Le
848f5b5d65
Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-10-28 13:40:17 +08:00
Duc Canh Le
c080964391 correct 'notLike' in key condition 2022-10-28 13:39:01 +08:00
Suzy Wang
6da80f4dcb
Merge branch 'master' into formatIpv6_s390x_support 2022-10-28 00:32:37 -04:00
Duc Canh Le
69311bd5cd Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-10-28 11:59:50 +08:00
Igor Nikonov
a4ce0e344a Fix build 
src/Common/CaresPTRResolver.cpp:126:27
error: implicit conversion loses integer precision: 'int64_t' (aka
'long') to 'int' [-Werror,-Wshorten-64-to-32]
 2022-10-27 21:11:41 +00:00
Sergei Trifonov
8892da3a4c
Merge branch 'master' into token-bucket-throttler 2022-10-27 22:40:59 +02:00
Anton Popov
7715afa595
Merge pull request #42618 from CurtizJ/refactor-data-part 
Try to save `IDataPartStorage` interface
 2022-10-27 22:20:44 +02:00
Jordi Villar
073d81c31f Use nfds_t instead 2022-10-27 20:36:42 +02:00
serxa
c4f25228f7 smaller burst to be closer to old behaviour 2022-10-27 17:18:50 +00:00
Jordi Villar
1fa24161e7 Minor fix implicit cast CaresPTRResolver 2022-10-27 18:34:37 +02:00
Alexander Tokmakov
86b81a2b6f
Merge pull request #42234 from arthurpassos/fix_ares_fd_process_race_condition 
Fix c-ares crash
 2022-10-27 17:17:22 +03:00
Nikolai Kochetov
24e1aaf324
Merge pull request #42576 from ClickHouse/table-function-merge-invalid-where-type 
Invalid type in where for Merge table (logical error)
 2022-10-27 13:03:27 +02:00
Kruglov Pavel
e099817449
Merge branch 'master' into Avogar-patch-3 2022-10-27 12:46:18 +02:00
Vladimir C
31e8f92cd9
Merge pull request #42664 from ClickHouse/vdimir/followup-42274 2022-10-27 12:20:46 +02:00
Nikolay Degterinsky
b2ab692d87 Safer getResult() 2022-10-27 10:03:03 +00:00
Maksim Kita
275d076cb7 Fixed style check 2022-10-27 11:26:53 +02:00
Maksim Kita
5fcbc26800 OrderByLimitByDuplicateEliminationPass improve performance 2022-10-27 11:23:30 +02:00
flynn
2b08fe6963 Fix truncate table does not hold lock correctly 2022-10-27 08:56:18 +00:00
serxa
accf78f1ff fix the first add() call 2022-10-27 08:39:08 +00:00
Robert Schulze
31ef6c66cb
Merge pull request #42513 from ClickHouse/update-libcxx-to-15 
Build with libcxx(abi) 15
 2022-10-27 10:30:36 +02:00
Sergei Trifonov
ec1389cbe7
Update src/Common/Throttler.cpp 
Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>
 2022-10-27 10:30:00 +02:00
Márcio Martins
aaf71d157d Use return type template parameter to select throwing behavior 2022-10-27 10:26:55 +02:00
Márcio Martins
1ca230ff6d Allow coercing in JSONExtract* functions 2022-10-27 10:26:55 +02:00
Márcio Martins
40b572c23b Remove redundant default case in switch statement 2022-10-27 10:26:55 +02:00
Márcio Martins
c3bcb1899d Add convenience ctor to ReadBufferFromMemory 2022-10-27 10:26:55 +02:00
Márcio Martins
0fc0e81c0c Add Element::type() to JSONParser 2022-10-27 10:26:55 +02:00
Antonio Andelic
10aa7091ca
Merge pull request #42605 from ClickHouse/correctly-handle-prewhere-type 
Correctly handle non-native integers in PREWHERE
 2022-10-27 09:25:38 +02:00
SmitaRKulkarni
e860ecc80c
Merge branch 'master' into 42414_Enable_functions_in_readonly_mode 2022-10-27 08:26:44 +02:00
李扬
24ffaee18c
Merge branch 'master' into ascii_function 2022-10-26 21:43:46 -05:00
Nikolay Degterinsky
488c220046 Fix bug in ParserFunction 2022-10-27 01:56:25 +00:00
Suzy Wang
fd1281a34d
Merge branch 'master' into formatIpv6_s390x_support 2022-10-26 19:04:55 -04:00
Nikolai Kochetov
313cbb1058 Fix test. 2022-10-26 17:49:36 +00:00
Maksim Kita
f04f663818 Fixed tests 2022-10-26 18:55:14 +02:00
Smita Kulkarni
6a8fa2d4a5 Added new field allow_readonly in system.table_functions to allow using table functions in readonly mode. 
Implementation:
* Added a new field allow_readonly to table system.table_functions.
* Updated to use new field allow_readonly to allow using table functions in readonly mode.
* Added TableFunctionProperties similar to AggregateFunctionProperties.
* The functions allowed in readonly mode are in this set table_functions_allowed_in_readonly_mode.
Testing:
* Added a test for filesystem tests/queries/0_stateless/02473_functions_in_readonly_mode.sh
Documentation:
* Updated the english documentation for Table Functions.
 2022-10-26 18:45:23 +02:00
Roman Vasin
09fe9c3ed1 Use {} in exceptions 2022-10-26 16:07:56 +00:00
Nikolai Kochetov
1520bcd53c Fix error code. 2022-10-26 15:50:10 +00:00
Anton Popov
ca4df62e37
Merge branch 'master' into refactor-data-part 2022-10-26 17:49:20 +02:00
Azat Khuzhin
7b69a70e82 Fix frequent memory drift message and clarify things in comments 
Somethine like:

    2022.09.28 06:33:34.001433 [ 3133669 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 562.20 MiB, peak 562.21 MiB, will set to 562.20 MiB (RSS), difference: -70.46 MiB
    2022.09.28 06:33:35.001639 [ 3133669 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 562.20 MiB, peak 562.21 MiB, will set to 562.20 MiB (RSS), difference: -70.45 MiB

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-26 17:38:37 +02:00
Azat Khuzhin
2faefc0c09 Add amount of memory used by allocator into logs 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-26 17:38:37 +02:00
Azat Khuzhin
f5f6f1b593 Add profile events for jemalloc purge 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-26 17:38:36 +02:00
Maksim Kita
6a0e3fc31d Subqueries remove unnecessary table expression modifiers 2022-10-26 17:19:29 +02:00
Maksim Kita
2b44a00fbc UNION node remove unnecessary union modes 2022-10-26 16:47:40 +02:00
Mikhail f. Shiryaev
fdecac65b8
Merge pull request #42699 from ClickHouse/22.11-prepare 
Update version after release
 2022-10-26 16:31:48 +02:00
Maksim Kita
1c17e9d454 Fixed tests 2022-10-26 15:54:37 +02:00
Mikhail f. Shiryaev
d3461abd1f
Update version to 22.11.1.1 2022-10-26 15:27:09 +02:00
李扬
04f8f38580
Merge branch 'master' into ascii_function 2022-10-26 07:29:49 -05:00
Kruglov Pavel
b8ee5600be
Merge pull request #42579 from ucasfl/set 
Fix create set with wrong header when data type is lowcardinality
 2022-10-26 14:17:14 +02:00
Nikolai Kochetov
db422fa903
Merge pull request #42410 from ClickHouse/fix-multi-arg-monotonicity 
Fix possible LOGICAL_ERROR in binary ariphmetics monotonicity.
 2022-10-26 14:11:23 +02:00
Kruglov Pavel
219553df8d
Merge pull request #42580 from hanfei1991/fei/limit_max_rows 
fix behaviour of max_rows_to_read for trival limit queries
 2022-10-26 14:09:59 +02:00
Maksim Kita
c47ca522f4 Fixed style check 2022-10-26 13:27:14 +02:00
Antonio Andelic
35a9672704 Merge branch 'master' into optimize-entire-partition 2022-10-26 11:09:25 +00:00
vdimir
14d0f6457b
Add tests and doc for some url-related functions 2022-10-26 10:52:57 +00:00
Maksim Kita
1b6293f6db MergeTree indexes use ActionsDAG 2022-10-26 12:44:37 +02:00
李扬
c7e5eb756b
Update src/Functions/ascii.cpp 
Co-authored-by: Vladimir C <vdimir@clickhouse.com>
 2022-10-26 18:18:15 +08:00
李扬
48c37c52e6
Update src/Functions/ascii.cpp 
Co-authored-by: Vladimir C <vdimir@clickhouse.com>
 2022-10-26 18:18:04 +08:00
Antonio Andelic
97aaebfa18 Address PR comments 2022-10-26 10:06:56 +00:00
SmitaRKulkarni
5df138da82
Merge pull request #42376 from ClickHouse/36316_Support_glob_for_recursive_directory_traversal 
Added ** glob support for recursive directory traversal to filesystem…
 2022-10-26 11:49:20 +02:00
Maksim Kita
6671c454de
Merge pull request #42651 from kitaisreal/analyzer-query-tree-node-remove-method 
Analyzer IQueryTreeNode remove getName method
 2022-10-26 12:36:38 +03:00
Maksim Kita
cd2a8966aa
Merge pull request #42649 from kitaisreal/analyzer-change-setting 
Analyzer change setting into allow_experimental_analyzer
 2022-10-26 12:26:15 +03:00
Antonio Andelic
979d180931
Merge branch 'master' into correctly-handle-prewhere-type 2022-10-26 11:13:16 +02:00
taiyang-li
add5360a1b implement function ascii 2022-10-26 03:02:00 -05:00
Roman Vasin
3eca9ada3b Add Date32 to formatDateTIme 2022-10-26 08:00:12 +00:00
Alexey Milovidov
011ae8675a
Merge pull request #42633 from ClickHouse/revert-41726-fix-slow-json-extract-with-low-cardinality 
Revert "Low cardinality cases moved to the function for its corresponding type"
 2022-10-26 08:10:27 +02:00
Nikolay Degterinsky
8c6d5cdbac
Merge pull request #42514 from evillique/fix-aliases 
Fix aliases
 2022-10-26 07:45:37 +02:00
serxa
471b391ab2 fix 2022-10-26 00:00:40 +00:00
serxa
27599ab70c fix comment 2022-10-25 23:58:25 +00:00
serxa
fc223aa673 replace throttler algorithm by token bucket 2022-10-25 23:52:31 +00:00
Anton Popov
79889198b1 fix style check 2022-10-25 23:03:37 +00:00
Anton Popov
d8b09b430f add comments 2022-10-25 22:27:11 +00:00
Anton Popov
c4d4f2dbed better interface 2022-10-25 22:14:06 +00:00
Robert Schulze
728342ec7c
Un-inline stuff 2022-10-25 21:41:08 +00:00
Alexey Milovidov
2b4be633ff
Merge branch 'master' into revert-41726-fix-slow-json-extract-with-low-cardinality 2022-10-25 22:57:12 +02:00
ltrk2
2c902bbc65
Implement support for different UUID binary formats (#42108) 
* Implement support for different UUID binary formats

* Declare error codes that the implementation uses

* Make single-argument constructor explicit

* Modernize parts of the solution

* Make improvements based on review comments

* Declare an error code being used
 2022-10-25 21:57:50 +02:00
Nikolai Kochetov
bfb5e6eae4 Fix logical error for invalid prewhere type in StorageMerge. 2022-10-25 13:58:29 +00:00
Antonio Andelic
611c2e2bd7 Support for optimizing old parts for entire partition only 2022-10-25 13:40:18 +00:00
Nikolai Kochetov
c4685f6994
Merge branch 'master' into fix-42575 2022-10-25 15:31:32 +02:00
Anton Popov
e2ff7aa261
Merge pull request #42623 from CurtizJ/fix-bad-cast 
Fix bad cast in `arrayElement` with `Map` with `Nullable` values and `Nullable` index
 2022-10-25 14:59:52 +02:00
Anton Popov
c8199bc125
Merge branch 'master' into refactor-data-part 2022-10-25 14:31:05 +02:00
Nikolai Kochetov
5c4444237e
Merge pull request #42617 from ClickHouse/revert-revert-41268-disable-s3-parallel-write-for-part-moves-to-disk-s3 
Revert revert 41268 disable s3 parallel write for part moves to disk s3
 2022-10-25 14:29:27 +02:00
Vladimir C
8fabe1515c
Merge pull request #42274 from dentiscalprum/fix_domain 2022-10-25 13:56:58 +02:00
Nikita Taranov
49f6692a2e
Adapt internal data structures to 512-bit era (#42564) 
* impl

* update tests

* fix tests
 2022-10-25 13:56:28 +02:00
Raúl Marín
0016bc2b73
Improve the time to recover keeper connections (#42541) 
Co-authored-by: Nikita Mikhaylov <nikitamikhaylov@clickhouse.com>
Co-authored-by: robot-clickhouse <robot-clickhouse@clickhouse.com>
Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>
 2022-10-25 13:26:29 +02:00
Antonio Andelic
af0c54e055 Small fix 2022-10-25 11:13:41 +00:00
Maksim Kita
a28d1f01fd Analyzer IQueryTreeNode remove getName method 2022-10-25 12:54:17 +02:00
Alexander Gololobov
759d8bcd80
Merge pull request #42609 from filimonov/patch-6 
Fix incorrect trace log line on dict reload
 2022-10-25 12:51:00 +02:00
Antonio Andelic
451a50d9e9 Merge branch 'master' into correctly-handle-prewhere-type 2022-10-25 10:48:35 +00:00
Maksim Kita
e6bfff3de2 Analyzer change setting into allow_experimental_analyzer 2022-10-25 12:35:25 +02:00
JackyWoo
c7a0ebeb05 little fix 2022-10-25 17:46:24 +08:00
Maksim Kita
06fe6f3c8b
Merge pull request #31796 from kitaisreal/identifier-resolver 
Added Analyzer
 2022-10-25 12:36:08 +03:00
Nikolay Degterinsky
62a33b12d5
Small refactor 2022-10-25 11:24:42 +02:00
JackyWoo
9a36a509fe fix test 2022-10-25 17:15:49 +08:00
Nikolay Degterinsky
fba827ecea
Merge branch 'master' into fix_progress_bar 2022-10-25 10:50:16 +02:00
Nikolay Degterinsky
b63ac4826b
Merge branch 'master' into fix-aliases 2022-10-25 10:34:52 +02:00
Alexey Milovidov
90c62fe46e
Merge branch 'master' into revert-41726-fix-slow-json-extract-with-low-cardinality 2022-10-25 09:31:46 +02:00
Robert Schulze
efd47c9eff
Merge pull request #42476 from ClickHouse/add-datetime-typedefs 
Add convenience typedefs for Date/Date32/DateTime/DateTime64 columns
 2022-10-25 08:45:57 +02:00
Alexey Milovidov
4ac34117ac
Merge pull request #42635 from CurtizJ/fix-build 
Fix build
 2022-10-25 08:40:57 +02:00
Alexey Milovidov
c48127534b
Merge pull request #41753 from pkit/morton_coding 
add Morton Coding (ZCurve)
 2022-10-25 06:10:37 +02:00
Alexey Milovidov
079ffa9827
Merge pull request #42353 from azat/build/musl-static 
Make getResource() independent from the order of the sections
 2022-10-25 05:57:22 +02:00
Anton Popov
e30d425869 fix build 2022-10-25 01:55:02 +00:00
xiedeyantu
163001382b Fix no progress indication on INSERT FROM INFILE 2022-10-25 09:43:29 +08:00
Nikolay Degterinsky
8a9cfa9b17
Merge branch 'master' into fix-aliases 2022-10-25 02:28:40 +02:00
Anton Popov
cbfa887e52 fix merge of projections 2022-10-24 23:51:57 +00:00
Anton Popov
eed21ad4ca
Revert "Low cardinality cases moved to the function for its corresponding type" 2022-10-25 01:30:32 +02:00
Anton Popov
f865b28da4 fix drop of MergeTree tables 2022-10-24 22:38:53 +00:00
Guo Wangyang
4e294b9bfc
ColumnVector: re-enable AVX512_VBMI/AVX512_VBMI2 optimized filter and index (#41765) 
* Revert "Merge pull request #41752 from ClickHouse/revert-revert-revert"

This reverts commit 7bb245720a, reversing
changes made to 78002dc248.

* Revert "Merge pull request #41797 from ClickHouse/revert-41247"

This reverts commit 69b13d9f31, reversing
changes made to 1719e000ea.

* AVX512VBMI::vectorIndexImpl: Fix heap buffer overflow when limit == 0

* use SIMD_ELEMENTS instead of SIMD_BYTES to avoid confusing
 2022-10-24 23:11:46 +02:00
Nikolay Degterinsky
171ed8dd4e Apply suggestions from review 2022-10-24 20:38:04 +00:00
Alexey Milovidov
9ee7131f67
Merge pull request #42423 from ClickHouse/optimize-partitions-in-background 
Optimize partitions in background
 2022-10-24 19:41:50 +02:00
Anton Popov
b58ca49f05 fix bad cast in arrayElement with Map and Nullable index 2022-10-24 17:34:46 +00:00
Robert Schulze
655233e1c5
Add convenience typedefs for Date/Date32/DateTime/DateTime64 columns 2022-10-24 16:55:41 +00:00
SmitaRKulkarni
96c8260230
Merge branch 'master' into 36316_Support_glob_for_recursive_directory_traversal 2022-10-24 18:34:19 +02:00
Anton Popov
dcb7667808 share transaction for projections 2022-10-24 16:28:19 +00:00
Sergei Trifonov
bef3c71453
Merge pull request #41726 from AlfVII/fix-slow-json-extract-with-low-cardinality 
Low cardinality cases moved to the function for its corresponding type
 2022-10-24 18:18:40 +02:00
Nikolai Kochetov
93310fe8b1 Fix typo. 2022-10-24 15:51:21 +00:00
Nikolai Kochetov
d5bd905a41 Try to fix data race on zookeeper vs DDLWorker at server shutdown. 2022-10-24 15:09:35 +00:00
Anton Popov
128fdd785e better interface 2022-10-24 14:44:22 +00:00
Maksim Kita
d0e64eb15b Fixed tests 2022-10-24 16:14:05 +02:00
Nikolai Kochetov
14e3bb6354 Another case. 2022-10-24 13:58:26 +00:00
JackyWoo
b5d1c4e657 replace snpd with lgif 2022-10-24 20:08:58 +08:00
Nikolai Kochetov
5dabbf89ad Fixing build. 2022-10-24 13:43:24 +02:00
Vitaly Baranov
cd158b021f
Merge pull request #42562 from vitlibar/subqueries-in-row-policy-filters 
Fix using subqueries in row policy filters
 2022-10-24 13:38:15 +02:00
Han Fei
8f00d0d1cc fix tidy 2022-10-24 12:56:36 +02:00
Nikolai Kochetov
aee7bb0d91
Merge pull request #42577 from ClickHouse/clean-thread-id 
Clear thread::id when ThreadFromGlobalPool exits.
 2022-10-24 12:03:28 +02:00
filimonov
0762783fbd
Fix incorrect trace log line on dict reload 2022-10-24 11:58:37 +02:00
JackyWoo
2f30c817bf little fix 2022-10-24 17:23:47 +08:00
Robert Schulze
240637f8f1
Merge branch 'master' into update-libcxx-to-15 2022-10-24 11:11:01 +02:00
Antonio Andelic
5579d139eb Avoid crash with big int in prewhere 2022-10-24 08:37:52 +00:00
Maksim Kita
9a02d25c0b Fixed tests 2022-10-24 10:22:38 +02:00
Maksim Kita
3862dd96fe Fixed tests 2022-10-24 10:22:38 +02:00
Maksim Kita
8ac7f4423b Fixed tests 2022-10-24 10:22:38 +02:00
Maksim Kita
d083200d65 Fixed code review issues 2022-10-24 10:22:38 +02:00
Maksim Kita
561d585100 Fixed tests 2022-10-24 10:22:38 +02:00
Maksim Kita
39b2d25a43 Added tests 2022-10-24 10:22:38 +02:00
Maksim Kita
90048ce75d Fixed code review issues 2022-10-24 10:22:38 +02:00
Maksim Kita
dbc0870eb5 Fixed tests 2022-10-24 10:22:38 +02:00
Maksim Kita
82fae0125c Added FullSortingMerge JOIN support 2022-10-24 10:22:38 +02:00
Maksim Kita
1410befbe1 Added AggregateFunctionsArithmeticOperationsPass 2022-10-24 10:22:38 +02:00
Maksim Kita
144b33c75d Fixed tests 2022-10-24 10:22:38 +02:00
Maksim Kita
852dd8c2bb Updated UnionNode documentation 2022-10-24 10:22:37 +02:00
Maksim Kita
8610feb06d Added UniqInjectiveFunctionsEliminationPass 2022-10-24 10:22:37 +02:00
Maksim Kita
c82bb1128f Fixed nested IN functions 2022-10-24 10:22:37 +02:00
Maksim Kita
fe3eda4c2b Added SumIfToCountIfPass 2022-10-24 10:22:37 +02:00
Maksim Kita
43ec5c830c Added FunctionToSubcolumnsPass 2022-10-24 10:22:37 +02:00
Maksim Kita
98eba24a34 Fix LIMIT when query has WITH TOTALS 2022-10-24 10:22:37 +02:00
Maksim Kita
f4e59b217f Added RBAC support 2022-10-24 10:22:37 +02:00
Maksim Kita
b50e2ce470 Added SETTINGS support 2022-10-24 10:22:37 +02:00
Maksim Kita
5a0298ea34 Fixed tests 2022-10-24 10:22:37 +02:00
Maksim Kita
da85af7e44 Added OrderByLimitByDuplicateEliminationPass 2022-10-24 10:22:37 +02:00
Maksim Kita
7851dfc324 Fixed tests 2022-10-24 10:22:37 +02:00
Maksim Kita
eba9607696 Analyzer move passes into separate folder 2022-10-24 10:22:37 +02:00
Maksim Kita
3a22e37691 Fix normalization of UNION inside DESCRIBE query 2022-10-24 10:22:37 +02:00
Maksim Kita
77ea9a11cd Fixed code review issues 2022-10-24 10:22:21 +02:00
Maksim Kita
6ec5474466 Updated IQueryTreeNode interface with weak pointers array 2022-10-24 10:22:21 +02:00
Maksim Kita
2f282eeec9 Updated InDepthQueryTreeVisitor implementation 2022-10-24 10:22:21 +02:00
Maksim Kita
49a129008b Fixed code review issues 2022-10-24 10:22:21 +02:00
Maksim Kita
239f1855aa Updated projection names implementation 2022-10-24 10:22:21 +02:00
Maksim Kita
048e776c48 Fixed tests 2022-10-24 10:22:21 +02:00
Maksim Kita
400de59a37 Fixed style check 2022-10-24 10:22:21 +02:00
Maksim Kita
86896b76c5 Fix GROUP BY validation for subqueries 2022-10-24 10:22:21 +02:00
Maksim Kita
0892f485bb Fix lambda tests 2022-10-24 10:22:21 +02:00
Maksim Kita
f2d801be2c Added CustomizeFunctionsPass 2022-10-24 10:22:21 +02:00
Maksim Kita
b1471bdedf Added CountDistinctPass 2022-10-24 10:22:21 +02:00
Maksim Kita
2aff015d29 Added NormalizeCountVariantsPass 2022-10-24 10:22:21 +02:00
Maksim Kita
54e2b6d28a Added EXISTS support 2022-10-24 10:22:21 +02:00
Maksim Kita
f5ee7e0bee Added IfChainToMultiIfPass 2022-10-24 10:22:21 +02:00
Maksim Kita
7c96c0afbd Added OrderByTupleEliminationPass 2022-10-24 10:22:21 +02:00
Maksim Kita
11324aa4b4 Added IfConstantConditionPass 2022-10-24 10:22:21 +02:00
Maksim Kita
3d5b209861 Added support for dictGet, joinGet functions to handle identifier as first argument 2022-10-24 10:22:21 +02:00
Maksim Kita
7f26bcc275 Added unqualified matcher support for JOIN with USING 2022-10-24 10:22:21 +02:00
Maksim Kita
b7d0fd6bae Added validation for subqueries and table functions in JOINs 2022-10-24 10:22:21 +02:00
Maksim Kita
d62bd792ac Added positional arguments support 2022-10-24 10:22:21 +02:00
Maksim Kita
10438aea01 Added JOIN support for different algorithm 2022-10-24 10:22:21 +02:00
Maksim Kita
a0d9251f3a Added ASOF join support 2022-10-24 10:22:21 +02:00
Maksim Kita
3782aed7a9 Separate planner expression analysis 2022-10-24 10:22:20 +02:00
Maksim Kita
8a39c3e1d5 Separate planner JOIN TREE processing 2022-10-24 10:22:20 +02:00
Maksim Kita
c7d51ca83d Rename InterpolateColumnNode into InterpolateNode 2022-10-24 10:22:20 +02:00
Maksim Kita
2b2a70082e Rename SortColumnNode into SortNode 2022-10-24 10:22:20 +02:00
Maksim Kita
fe84e871f8 Fixed build 2022-10-24 10:22:20 +02:00
Maksim Kita
8cadb1b318 Added WINDOW functions support 2022-10-24 10:22:20 +02:00
Maksim Kita
85ac02c9db Planner improve action names 2022-10-24 10:22:20 +02:00
Maksim Kita
890d9678b9 Fixed tests 2022-10-24 10:22:20 +02:00
Maksim Kita
a06832508e Fixed tests 2022-10-24 10:22:20 +02:00
Maksim Kita
4f68305dbe Fixed tests 2022-10-24 10:22:20 +02:00
Maksim Kita
ca93ee7479 Fixed tests 2022-10-24 10:22:20 +02:00
Maksim Kita
fc860ec050 Planner collect table expression data 2022-10-24 10:22:20 +02:00
Maksim Kita
1844673aa8 Updated tests 2022-10-24 10:22:20 +02:00
Maksim Kita
fed146e198 Added SAMPLE BY support. Added SAMPLE BY, FINAL support for JOINS. 2022-10-24 10:22:20 +02:00
Maksim Kita
5128262129 Added GROUP BY with constant keys support 2022-10-24 10:22:20 +02:00
Maksim Kita
64c60718f8 Updated SORT COLUMN with fill 2022-10-24 10:22:20 +02:00
Maksim Kita
4110460bcd Added LIMIT BY support 2022-10-24 10:22:20 +02:00
Maksim Kita
81e4c67f17 Added INTERPOLATE support 2022-10-24 10:22:20 +02:00