thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-19 21:03:51 +00:00
Code Issues Packages Projects Releases Wiki Activity
100,911 Commits 9,338 Branches 1,784 Tags 2.2 GiB
3378c7f1b2
Commit Graph

33141 Commits

Author SHA1 Message Date
taiyang-li
6770723027 modify as requested 2022-11-03 12:02:31 +08:00
Yakov Olkhovskiy
170670da59 fallback to stderr 2022-11-03 03:28:57 +00:00
Yakov Olkhovskiy
b30f2ac12d allow '--progress err' to preserve old behavior, adjust tests 2022-11-03 03:16:55 +00:00
taiyang-li
aeea21e4d7 Merge branch 'master' into ascii_function 2022-11-03 11:13:20 +08:00
SmitaRKulkarni
f26d121f6d
Merge branch 'master' into 42452_Bug_fix_parent_node_corrupted 2022-11-02 22:40:10 +01:00
Smita Kulkarni
816f2319f7 Updated finished_asts to fix fast test fails with Normalise AST is too deep and updated ConvertFunctionOrLikeData to merge conditions by aliasorColumnName instead of ASTPtr - 42452 Bug fix for parent node corrupted 2022-11-02 19:35:22 +01:00
Alexander Tokmakov
e67d05604c Merge branch 'master' into fix_intersecting_parts2 2022-11-02 19:30:27 +01:00
Alexander Tokmakov
d8b3a2af84 fix race between INSERT and DROP 2022-11-02 19:26:50 +01:00
Vitaly Baranov
32194c1200 Add max limitation for the size of an uploaded part. 2022-11-02 17:53:54 +01:00
Robert Schulze
494d6ca3df
Cleanup implementation of regexpReplace(All|One) 
This is a pure refactoring, there are no semantic changes.

Cherry-picked from #42682.
 2022-11-02 15:49:53 +00:00
vdimir
503f71b285
Do not set treat_local_port_as_remote in ClusterDiscovery 2022-11-02 15:34:06 +00:00
Vitaly Baranov
e0133688bc
Merge branch 'master' into mask-sensitive-info-in-logs 2022-11-02 16:26:13 +01:00
Alexander Tokmakov
00c9e50ee3 fix race between INSERT and ALTER PARTITION 2022-11-02 16:25:26 +01:00
Dmitry Novik
a860b25249 Improve ValidationChecker readability 2022-11-02 15:24:09 +00:00
Alexander Tokmakov
acbad99b24 remove abandonable_lock part 2 2022-11-02 14:28:49 +01:00
Kruglov Pavel
38124b6533
Merge pull request #42780 from Avogar/parallel-parsing 
Support parallel parsing for LineAsString input format
 2022-11-02 13:21:53 +01:00
ltrk2
14141c9071 Add support for FixedString input to base64 functions 2022-11-02 08:14:28 -04:00
Robert Schulze
8ec4c77352
Merge pull request #42855 from GoGoWen/fix_typo 
Fix typo
 2022-11-02 12:54:52 +01:00
Antonio Andelic
c4b717b343
Update src/Storages/MergeTree/MergeTreeDataMergerMutator.cpp 
Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>
 2022-11-02 08:24:54 +01:00
zzsmdfj
61123b7f54 to fix join to subqueries rewriter null pointer 2022-11-02 11:34:15 +08:00
Duc Canh Le
ec382d7903 add comment 2022-11-02 09:50:41 +08:00
Duc Canh Le
c6598dc66d
Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-11-02 09:42:01 +08:00
Dmitry Novik
e06572cc27 Validate Query Tree in debug 2022-11-01 23:03:07 +00:00
Alexander Tokmakov
4f50a999bb remove abandonable_lock part 1 2022-11-01 21:35:25 +01:00
Azat Khuzhin
e58008b711 Fix rare possible hung on query cancellation 
Some queries can hung after cancelling (because enough rows had been
read for Distributed queries) or after KILL. Example of such query is a
Distributed query that uses DelayedSource that uses ExpandPipeline to
add new processors, and in this case it is possible that all already
existing processes was cancelled but new had been added and now
PipelineExecutor will wait for them undefinitelly since nobody will read
from them and nobody will cancel them either.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-11-01 18:17:47 +01:00
Vitaly Baranov
53ae7aba9a
Merge pull request #42628 from azat/s3-plain-fixes 
Implement ATTACH of MergeTree table for s3_plain disk
 2022-11-01 18:00:46 +01:00
Vitaly Baranov
085fb80190 Spare stack in InDepthNodeVisitor. 2022-11-01 17:20:12 +01:00
Yakov Olkhovskiy
16159cc7a9
Merge pull request #42412 from ClibMouse/formatIpv6_s390x_support 
format ipv6 in s390x, endianess support
 2022-11-01 12:02:44 -04:00
Anton Popov
88674ef964
Merge pull request #42794 from CurtizJ/better-code-in-check-part 
Slightly better code with projections
 2022-11-01 16:05:27 +01:00
zhongyuankai
c1e2b2af74 Optimize TTL merge, completely expired parts can be removed in time 2022-11-01 22:58:15 +08:00
Anton Popov
876dca48da
Merge pull request #36969 from CurtizJ/dynamic-columns-14 
Support `Object` type inside other types
 2022-11-01 15:20:02 +01:00
Duc Canh Le
07d9720ed5
Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-11-01 22:16:09 +08:00
Kruglov Pavel
c3a7211c1c
Merge pull request #41502 from marcioapm/coercing-json-extract 
Add coercing in JSONExtract
 2022-11-01 14:52:41 +01:00
Kruglov Pavel
7cbb534388
Merge pull request #42536 from ClickHouse/Avogar-patch-3 
Fix typo in setting name that led to bad usage of schema inference cache
 2022-11-01 14:03:47 +01:00
Kruglov Pavel
21d50f76ea
Merge pull request #41979 from Avogar/s3-cluster-schema-inference 
Fix schema inference in s3Cluster and improve in hdfsCluster
 2022-11-01 14:00:21 +01:00
Vitaly Baranov
b9f2f17331 Add test and logging. 2022-11-01 12:23:20 +01:00
Alexander Gololobov
f67f0183f6
Merge pull request #42812 from azat/utf8-fix-followup 
Fix lowerUTF8()/upperUTF8() in case of symbol was in between 16-byte boundary
 2022-11-01 12:01:00 +01:00
Maksim Kita
fe48a1ce7e
Merge pull request #42681 from kitaisreal/merge-tree-indexes-use-actions-dag 
MergeTree indexes use ActionsDAG
 2022-11-01 13:55:17 +03:00
Maksim Kita
27a72f096c
Merge pull request #42705 from kitaisreal/analyzer-small-improvements 
Analyzer improve subqueries representation
 2022-11-01 13:54:12 +03:00
GoGoWen2021
2b0cde9ef9 fix typo 2022-11-01 17:46:44 +08:00
Smita Kulkarni
b689f7ab70 Updated registerFunction method to take TableFunctionFactoryData as input , moved table_functions_allowed_in_readonly_mode inside TableFunctionFactory- 42414 Enable table functions in readonly mode 2022-11-01 10:13:10 +01:00
Duc Canh Le
3545db0a6c
Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-11-01 16:54:26 +08:00
SmitaRKulkarni
a1a3eb4f10
Merge branch 'master' into 42452_Bug_fix_parent_node_corrupted 2022-11-01 09:46:05 +01:00
Vitaly Baranov
9d46547daa
Merge branch 'master' into session-log-support-sslcertificate-auth 2022-11-01 08:32:09 +01:00
Alexey Milovidov
1b3ef58b51
Merge branch 'master' into formatIpv6_s390x_support 2022-10-31 23:20:53 +01:00
Yakov Olkhovskiy
bab0e06e3d
Merge pull request #42458 from Clark0/match-func-use-index 
match function can use index if it's a condition on string prefix #37333
 2022-10-31 17:33:58 -04:00
Zhiguo Zhou
d18d08bcc7
Remove short-circuit evaluation in AssociativeApplierImpl::apply (#42214) 
The short-circuit evaluation was implemented when applying the
saturable operators (and, or) on a vector of ColumnUInt8. However,
its control flow would be compiled as a series of conditional
branch instructions which are hard to predict by the hardware and
at the same time hinder the vectorization optimization by the
compiler. This commit removes the short-circuit and evaluates the
whole expression.
 2022-10-31 21:36:15 +01:00
Vitaly Baranov
914ab51992 Increase the size of upload part exponentially for backup to S3. 2022-10-31 17:54:41 +01:00
Vitaly Baranov
d1c69a0753 Avoid changing query if it doesn't contain password. 2022-10-31 17:04:44 +01:00
Maksim Kita
f099bf63b3
Merge pull request #42809 from Gabriel39/comment_typo 
Fix typo in comments
 2022-10-31 18:30:32 +03:00
Anton Popov
2ae3cfa9e0
Merge branch 'master' into dynamic-columns-14 2022-10-31 16:15:19 +01:00
Alexander Tokmakov
226931229b
Merge pull request #42805 from ClickHouse/fix_some_races_in_merge_tree 
Fix some races in MergeTree
 2022-10-31 17:53:20 +03:00
Márcio Martins
399d024c2c Fix array index argument type check for JSONExtract* functions 2022-10-31 15:08:34 +01:00
Márcio Martins
c33bd7794f
Merge branch 'master' into coercing-json-extract 2022-10-31 15:04:48 +01:00
Gabriel
22cf4b9457
Merge branch 'master' into comment_typo 2022-10-31 21:53:16 +08:00
Alexander Tokmakov
599ccb9939
Update MergeTreeData.h 2022-10-31 16:18:17 +03:00
Alexander Tokmakov
b2b9479afa
Update src/Storages/MergeTree/MergeTreeData.h 
Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>
 2022-10-31 16:15:35 +03:00
SmitaRKulkarni
54035270b2
Merge branch 'master' into 42414_Enable_functions_in_readonly_mode 2022-10-31 14:15:29 +01:00
SmitaRKulkarni
7e9dd825c0
Merge branch 'master' into 42452_Bug_fix_parent_node_corrupted 2022-10-31 14:03:44 +01:00
Smita Kulkarni
c97c78e3ff Updated normaliser to clone the alias ast. 
Implementation:
* Updated QueryNormalizer to clone alias ast, when its replaced. Previously just assigning the same leads to exception in LogicalExpressinsOptimizer as it would be the same parent being inserted again.
* This bug is not seen with new analyser (allow_experimental_analyzer), so no changes for it. I added a test for the same.
Testing:
* Added a test for or function with alias and const where 02475_or_function_alias_and_const_where.sql
 2022-10-31 13:49:31 +01:00
Maksim Kita
bca22ec5f5 Fixed code review issues 2022-10-31 12:30:00 +01:00
Nikolay Degterinsky
d8d479a8c0
Merge pull request #42804 from evillique/fix-create-user-parser 
Fix a bug in ParserCreateUserQuery
 2022-10-31 12:29:06 +01:00
Sergei Trifonov
8c67257a1c
Merge pull request #42665 from ClickHouse/token-bucket-throttler 
Replace throttler algorithm by token bucket
 2022-10-31 12:28:18 +01:00
Azat Khuzhin
c93262170d Remove ReadOnlyMetadataStorage 
Throw exceptions from IMetadataStorage instead to avoid introducing
extra abstractions.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:30 +01:00
Azat Khuzhin
51bd0c2ac1 Make ReadOnlyMetadataStorage really readonly 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:26 +01:00
Azat Khuzhin
bceca73f6f Throw NOT_IMPLEMENTED form ReadOnlyMetadataStorage::getLastChanged() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:26 +01:00
Azat Khuzhin
3b7abbbff4 Rename s/listPrefix/findAllFiles, s/listPrefixInPath/getDirectoryContents/ 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:26 +01:00
Azat Khuzhin
fa7535c90d Slightly optimize MetadataStorageFromPlainObjectStorage::isDirectory() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:26 +01:00
Azat Khuzhin
82ea67eb51 Slightly better MetadataStorageFromPlainObjectStorage::isFile() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:26 +01:00
Azat Khuzhin
95fb2ad3cf Implement ATTACH of MergeTree table for s3_plain disk 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:26 +01:00
Azat Khuzhin
4e42521f44 Introduce ReadOnlyMetadataStorage 
And use it for:
- MetadataStorageFromPlainObjectStorage
- MetadataStorageFromStaticFilesWebServer

This will allow to reduce ~100-200 lines of duplicated code, and plus
make the code less error prone.

Note, for now I tried to make this without behaviour changes.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:26 +01:00
Azat Khuzhin
f8ba24f040 Implement S3ObjectStorage::listPrefixInPath() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:26 +01:00
Azat Khuzhin
18e4fdf40f Introduce IObjectStorage::listPrefixInPath() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:25 +01:00
Azat Khuzhin
ee18140c48 Remove listPrefix() implementation for disks that does not support send_metadata 
The reason for removing is it because not compatible with restoring
(with send_metadata set) anyway:
- HDFS - is not compatible with send_metadata, and besides it's
  implementaion is not correct, since it is simply `ls -l`, while the
  following is required: `find . -maxdepth 1 -type f`
- Web - is not compatible with send_metadata anyway
- Local - is not compatible with send_metadata anyway

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:25 +01:00
Azat Khuzhin
88db8ae7fa Add a comment for IObjectStorage::listPrefix() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:25 +01:00
Azat Khuzhin
94d9600fb8 Implement MetadataStorageFromPlainObjectStorage::getLastChanged() (as for web) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:25 +01:00
Azat Khuzhin
b34ffda272 Implement MetadataStorageFromPlainObjectStorage::getLastModified() (used by MergeTree) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:25 +01:00
Azat Khuzhin
f6d0c03ee5 Fix path to files in MetadataStorageFromPlainObjectStorage 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:25 +01:00
Maksim Kita
d760c78754 Fixed code review issues 2022-10-31 11:45:12 +01:00
Nikolai Kochetov
742d9af397
Merge pull request #42582 from azat/memory-tracker/fixes 
Fix frequent memory drift message and clarify things in comments
 2022-10-31 11:43:20 +01:00
Maksim Kita
115fcaffc5 Fixed tests 2022-10-31 11:30:56 +01:00
Vitaly Baranov
c07fe6921f Add comments. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
854e1e918f Use assert_cast() instead of as() in some places. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
43efbad642 Wipe passwords from distributed queries too. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
a30bfada63 Wipe passwords from backup logs too. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
dcf8724adf Move prepareQueryForLogging() to a separate header. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
f2bd560866 Fix IAST::clone() overrides for some queries. 2022-10-31 10:50:33 +01:00
Vitaly Baranov
c2cc2ccc99 Make as_table_function a child of ASTCreateQuery (to help writing visitors). 2022-10-31 10:50:33 +01:00
Vitaly Baranov
5d2a222fe4 Mask sensitive information in logs. 2022-10-31 10:50:33 +01:00
Azat Khuzhin
5fe44f2736 Fix lowerUTF8()/upperUTF8() in case of symbol was in between 16-byte boundary 
In lowerUTF8()/upperUTF8() there is an SSE optimization that handles
16 byte at a time, but only for ASCII, for UTF8 symbols converion will
be done by symbol.

Consider the following example:

    КВ АМ И СЖ
             ^ - offset is 15, length of sequence is 2
                 so first byte of a symbol is in first 16 bytes
                 second byte of a symbol is not ther

And in this case it will be handled incorrectly because it will try to
process oly these 16 bytes w/o looking forward.

This had been broken by #41286, before this patch it does not looks at
the row boundaries but only at the string end and so this sutation
wasn't possible.

Fixes: #42756
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 08:17:38 +01:00
Duc Canh Le
d309e8793a
Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-10-31 15:13:39 +08:00
Azat Khuzhin
32febf5155 Remove dead code in LowerUpperUTF8Impl::array() 
AFAICS it was there before since it was possible to overrun the
expected_end, since utf8.convert() was called with "src_end - src" not
"expected_end - src".

Refs: 5a21f3908b054a0efc90c65a12fbe151c74d90dc:dbms/include/DB/Functions/FunctionsString.h
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 08:00:29 +01:00
Gabriel
94866979c5
Merge branch 'master' into comment_typo 2022-10-31 13:29:53 +08:00
Gabriel
620caeb07c Fix typo in comments 2022-10-31 13:20:58 +08:00
Nikolay Degterinsky
05e178b6d3
Merge pull request #42767 from azat/disks/web-sigsegv-fix 
Fix possible SIGSEGV for web disks when file does not exists
 2022-10-30 20:27:21 +01:00
Alexander Tokmakov
f53df7870c fix race between drop and failed insert 2022-10-30 17:59:47 +01:00
Nikolay Degterinsky
978aa16e29 Fix a bug in ParserCreateUserQuery 2022-10-30 16:42:57 +00:00
Alexander Tokmakov
c68ab231f9 fix accessing part in Deleting state 2022-10-30 17:30:51 +01:00
Roman Vasin
24a4f4afe7
Merge branch 'master' into ADQM-605 2022-10-30 18:03:07 +03:00
Anton Popov
8de75bf29c better code with projections 2022-10-29 14:39:59 +00:00
Nikolay Degterinsky
6d1b547586
Merge pull request #42724 from evillique/fix-parser-bug 
Fix a bug in ParserFunction
 2022-10-29 13:59:20 +02:00
clarkcaoliu
ca5dbe88db match function can use index if prefix 2022-10-29 18:25:29 +08:00
avogar
fe0aea2e3a Support parallel parsing for LineAsString input format 2022-10-28 21:56:09 +00:00
Miel Donkers
fb1623a5f8 Add SSL_CERTIFICATE auth method to all places missing 2022-10-28 21:17:59 +02:00
Sergei Trifonov
c948303fa0
Merge branch 'master' into token-bucket-throttler 2022-10-28 16:21:55 +02:00
Alexey Milovidov
f64e632707
Merge pull request #42469 from amosbird/issue_42456 
Fix misbehavior of key analysis
 2022-10-28 16:18:00 +02:00
Azat Khuzhin
7a5432feaa Fix possible SIGSEGV for web disks when file does not exists 
It can be triggered in multiple ways, either when file does not exists
and you are trying to create MergeTree table from web (that has special
code for UUID handling) or by simply OPTIMIZE TABLE FINAL for MergeTree
table that is located on web disk, in both cases you will get the
following:

<details>

<summary>stacktrace</summary>

    2022.10.28 14:08:40.631226 [ 6043 ] {6165bf5f-e76b-4bca-941c-7c7ff5e3b46b} <Trace> ContextAccess (default): Access granted: OPTIMIZE ON default.data_from_web
    2022.10.28 14:08:40.632017 [ 6043 ] {6165bf5f-e76b-4bca-941c-7c7ff5e3b46b} <Debug> default.data_from_web (a3e65e1f-5fd4-47ed-9dbd-307f2586b52d) (MergerMutator): Selected 1 parts from all_1_1_0 to all_1_1_0
    2022.10.28 14:08:40.632496 [ 6043 ] {6165bf5f-e76b-4bca-941c-7c7ff5e3b46b} <Trace> default.data_from_web (a3e65e1f-5fd4-47ed-9dbd-307f2586b52d): Trying to reserve 1.00 MiB using storage policy from min volume index 0
    2022.10.28 14:08:40.632752 [ 6043 ] {6165bf5f-e76b-4bca-941c-7c7ff5e3b46b} <Trace> DiskObjectStorage(DiskWebServer): Reserved 1.00 MiB on remote disk `web_disk`, having unreserved 16.00 EiB.
    2022.10.28 14:08:40.634155 [ 6043 ] {a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1} <Debug> MergeTask::PrepareStage: Merging 1 parts: from all_1_1_0 to all_1_1_0 into Compact
    2022.10.28 14:08:40.634498 [ 6043 ] {a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1} <Trace> WebObjectStorage: Loading metadata for directory: http://127.0.0.1:8080/store/a3e/a3e65e1f-5fd4-47ed-9dbd-307f2586b52d/tmp_merge_all_1_1_1
    2022.10.28 14:08:40.635025 [ 6043 ] {a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1} <Trace> DiskWeb: Adding directory: store/a3e/a3e65e1f-5fd4-47ed-9dbd-307f2586b52d/tmp_merge_all_1_1_1/
    2022.10.28 14:08:40.635355 [ 6043 ] {a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1} <Trace> ReadWriteBufferFromHTTP: Sending request to http://127.0.0.1:8080/store/a3e/a3e65e1f-5fd4-47ed-9dbd-307f2586b52d/tmp_merge_all_1_1_1/.index
    2022.10.28 14:08:40.639618 [ 6043 ] {a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1} <Trace> DiskWeb: Cannot load disk metadata. Error: Code: 86. DB::Exception: Received error from remote server /store/a3e/a3e65e1f-5fd4-47ed-9dbd-307f2586b52d/tmp_merge_all_1_1_1/.index. HTTP status code: 404 Not Found, body: <!doctype html><html><head><title>404 Not Found</title><style>
    body { background-color: #fcfcfc; color: #333333; margin: 0; padding:0; }
    h1 { font-size: 1.5em; font-weight: normal; background-color: #9999cc; min-height:2em; line-height:2em; border-bottom: 1px inset black; margin: 0; }
    h1, p { padding-left: 10px; }
    code.url { background-color: #eeeeee; font-family:monospace; padding:0 2px;}
    </style>
    </head><body><h1>Not Found</h1><p>The requested resource <code class="url">/store/a3e/a3e65e1f-5fd4-47ed-9dbd-307f2586b52d/tmp_merge_all_1_1_1/.index</code> was not found on this server.</p></body></html>: while loading disk metadata. (RECEIVED_ERROR_FROM_REMOTE_IO_SERVER) (version 22.11.1.1)
    2022.10.28 14:08:40.640527 [ 5488 ] {} <Trace> BaseDaemon: Received signal 11
    2022.10.28 14:08:40.641529 [ 9027 ] {} <Fatal> BaseDaemon: ########################################
    2022.10.28 14:08:40.642759 [ 9027 ] {} <Fatal> BaseDaemon: (version 22.11.1.1, build id: 12145DA78CE5E9EBB10A034177FAE5967EF81A4A) (from thread 6043) (query_id: a3e65e1f-5fd4-47ed-9dbd-307f2586b52d::all_1_1_1) (query: optimize table data_from_web final) Received signal Segmentation fault (11)
    2022.10.28 14:08:40.643260 [ 9027 ] {} <Fatal> BaseDaemon: Address: NULL pointer. Access: read. Unknown si_code.
    2022.10.28 14:08:40.643769 [ 9027 ] {} <Fatal> BaseDaemon: Stack trace: 0x7ffff416c0f2 0x7ffff7cd1ca8 0x7ffff679ae5e 0x7fffd52e7906 0x7fffd50c65aa 0x7fffca7a0d42 0x7fffcaee79ec 0x7fffcaf242f8 0x7fffcaf242b5 0x7fffcaf2427d 0x7fffcaf24255 0x7fffcaf2421d 0x7ffff65c3686 0x7ffff65c2295 0x7fffcaeee2a9 0x7fffcaef2c43 0x7fffcaee3c0e 0x7fffcc4a7851 0x7fffcc4a768f 0x7fffcc4abb2d 0x7fffcfdce828 0x7fffd03e3eaa 0x7fffd03dfe3b 0x7fffc8ec42d4 0x7fffc8ed51d2 0x7ffff4bdd839 0x7ffff4bde0a8 0x7ffff48ab261 0x7ffff48a769a 0x7ffff48a6335 0x7ffff409f8fd 0x7ffff4121a60
    2022.10.28 14:08:40.644411 [ 9027 ] {} <Fatal> BaseDaemon: 4. ? @ 0x7ffff416c0f2 in ?
    2022.10.28 14:08:40.676390 [ 9027 ] {} <Fatal> BaseDaemon: 5. /src/ch/clickhouse/src/Common/StringUtils/StringUtils.cpp:9: detail::startsWith(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, char const*, unsigned long) @ 0x1ca8 in /src/ch/clickhouse/.cmake/src/Common/StringUtils/libstring_utilsd.so
    2022.10.28 14:08:40.730727 [ 9027 ] {} <Fatal> BaseDaemon: 6. /src/ch/clickhouse/src/Common/StringUtils/StringUtils.h:19: startsWith(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) @ 0x59ae5e in /src/ch/clickhouse/.cmake/src/libclickhouse_common_iod.so
    2022.10.28 14:08:40.923955 [ 9027 ] {} <Fatal> BaseDaemon: 7. /src/ch/clickhouse/src/Disks/ObjectStorages/Web/MetadataStorageFromStaticFilesWebServer.cpp:58: DB::MetadataStorageFromStaticFilesWebServer::exists(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) const @ 0x6e7906 in /src/ch/clickhouse/.cmake/src/libdbmsd.so
    2022.10.28 14:08:41.291996 [ 9027 ] {} <Fatal> BaseDaemon: 8. /src/ch/clickhouse/src/Disks/ObjectStorages/DiskObjectStorage.cpp:181: DB::DiskObjectStorage::exists(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) const @ 0x4c65aa in /src/ch/clickhouse/.cmake/src/libdbmsd.so
    2022.10.28 14:08:41.704697 [ 9027 ] {} <Fatal> BaseDaemon: 9. /src/ch/clickhouse/src/Storages/MergeTree/DataPartStorageOnDisk.cpp:74: DB::DataPartStorageOnDisk::exists() const @ 0xda0d42 in /src/ch/clickhouse/.cmake/src/libclickhouse_storages_mergetreed.so
    2022.10.28 14:08:43.032459 [ 9027 ] {} <Fatal> BaseDaemon: 10. /src/ch/clickhouse/src/Storages/MergeTree/MergeTask.cpp:147: DB::MergeTask::ExecuteAndFinalizeHorizontalPart::prepare() @ 0x14e79ec in /src/ch/clickhouse/.cmake/src/libclickhouse_storages_mergetreed.so
    ...
    Segmentation fault (core dumped)

</details>

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-28 15:15:49 +02:00
Roman Vasin
5755728b33 Add Date32 and DateTime64 into exception messages 2022-10-28 13:11:09 +00:00
Alexander Tokmakov
11f1f5ae4d
Merge pull request #42747 from jrdi/fix-implicit-cast-CaresPTRResolver 
Minor fix implicit cast CaresPTRResolver
 2022-10-28 15:50:11 +03:00
Roman Vasin
15f3f56812 Change 1 argument of function to first argument of function etc 2022-10-28 12:44:47 +00:00
Antonio Andelic
5fae8b0cc9 Merge branch 'master' into optimize-entire-partition 2022-10-28 11:44:10 +00:00
Sergei Trifonov
eda264b492
Merge branch 'master' into token-bucket-throttler 2022-10-28 11:38:13 +02:00
Maksim Kita
835daee0f3
Merge pull request #42728 from ucasfl/fix 
Fix truncate table does not hold lock correctly
 2022-10-28 11:44:24 +03:00
Maksim Kita
ba8b806957
Merge pull request #42704 from kitaisreal/order-by-limit-by-duplicate-elimination-improve-performance 
OrderByLimitByDuplicateEliminationPass improve performance
 2022-10-28 11:43:53 +03:00
Jordi Villar
534dbc159c
Merge branch 'master' into fix-implicit-cast-CaresPTRResolver 2022-10-28 08:00:22 +02:00
Duc Canh Le
848f5b5d65
Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-10-28 13:40:17 +08:00
Duc Canh Le
c080964391 correct 'notLike' in key condition 2022-10-28 13:39:01 +08:00
Suzy Wang
6da80f4dcb
Merge branch 'master' into formatIpv6_s390x_support 2022-10-28 00:32:37 -04:00
Duc Canh Le
69311bd5cd Merge branch 'master' into ch_canh_fix_prefix_not_like 2022-10-28 11:59:50 +08:00
Igor Nikonov
a4ce0e344a Fix build 
src/Common/CaresPTRResolver.cpp:126:27
error: implicit conversion loses integer precision: 'int64_t' (aka
'long') to 'int' [-Werror,-Wshorten-64-to-32]
 2022-10-27 21:11:41 +00:00
Sergei Trifonov
8892da3a4c
Merge branch 'master' into token-bucket-throttler 2022-10-27 22:40:59 +02:00
Anton Popov
7715afa595
Merge pull request #42618 from CurtizJ/refactor-data-part 
Try to save `IDataPartStorage` interface
 2022-10-27 22:20:44 +02:00
Jordi Villar
073d81c31f Use nfds_t instead 2022-10-27 20:36:42 +02:00
serxa
c4f25228f7 smaller burst to be closer to old behaviour 2022-10-27 17:18:50 +00:00
Jordi Villar
1fa24161e7 Minor fix implicit cast CaresPTRResolver 2022-10-27 18:34:37 +02:00
Alexander Tokmakov
86b81a2b6f
Merge pull request #42234 from arthurpassos/fix_ares_fd_process_race_condition 
Fix c-ares crash
 2022-10-27 17:17:22 +03:00
Nikolai Kochetov
24e1aaf324
Merge pull request #42576 from ClickHouse/table-function-merge-invalid-where-type 
Invalid type in where for Merge table (logical error)
 2022-10-27 13:03:27 +02:00
Kruglov Pavel
e099817449
Merge branch 'master' into Avogar-patch-3 2022-10-27 12:46:18 +02:00
Vladimir C
31e8f92cd9
Merge pull request #42664 from ClickHouse/vdimir/followup-42274 2022-10-27 12:20:46 +02:00
Nikolay Degterinsky
b2ab692d87 Safer getResult() 2022-10-27 10:03:03 +00:00
Maksim Kita
275d076cb7 Fixed style check 2022-10-27 11:26:53 +02:00
Maksim Kita
5fcbc26800 OrderByLimitByDuplicateEliminationPass improve performance 2022-10-27 11:23:30 +02:00
flynn
2b08fe6963 Fix truncate table does not hold lock correctly 2022-10-27 08:56:18 +00:00
serxa
accf78f1ff fix the first add() call 2022-10-27 08:39:08 +00:00
Robert Schulze
31ef6c66cb
Merge pull request #42513 from ClickHouse/update-libcxx-to-15 
Build with libcxx(abi) 15
 2022-10-27 10:30:36 +02:00
Sergei Trifonov
ec1389cbe7
Update src/Common/Throttler.cpp 
Co-authored-by: Yakov Olkhovskiy <99031427+yakov-olkhovskiy@users.noreply.github.com>
 2022-10-27 10:30:00 +02:00
Márcio Martins
aaf71d157d Use return type template parameter to select throwing behavior 2022-10-27 10:26:55 +02:00
Márcio Martins
1ca230ff6d Allow coercing in JSONExtract* functions 2022-10-27 10:26:55 +02:00
Márcio Martins
40b572c23b Remove redundant default case in switch statement 2022-10-27 10:26:55 +02:00
Márcio Martins
c3bcb1899d Add convenience ctor to ReadBufferFromMemory 2022-10-27 10:26:55 +02:00
Márcio Martins
0fc0e81c0c Add Element::type() to JSONParser 2022-10-27 10:26:55 +02:00
Antonio Andelic
10aa7091ca
Merge pull request #42605 from ClickHouse/correctly-handle-prewhere-type 
Correctly handle non-native integers in PREWHERE
 2022-10-27 09:25:38 +02:00
SmitaRKulkarni
e860ecc80c
Merge branch 'master' into 42414_Enable_functions_in_readonly_mode 2022-10-27 08:26:44 +02:00
李扬
24ffaee18c
Merge branch 'master' into ascii_function 2022-10-26 21:43:46 -05:00
Nikolay Degterinsky
488c220046 Fix bug in ParserFunction 2022-10-27 01:56:25 +00:00
Suzy Wang
fd1281a34d
Merge branch 'master' into formatIpv6_s390x_support 2022-10-26 19:04:55 -04:00
Nikolai Kochetov
313cbb1058 Fix test. 2022-10-26 17:49:36 +00:00
Maksim Kita
f04f663818 Fixed tests 2022-10-26 18:55:14 +02:00
Smita Kulkarni
6a8fa2d4a5 Added new field allow_readonly in system.table_functions to allow using table functions in readonly mode. 
Implementation:
* Added a new field allow_readonly to table system.table_functions.
* Updated to use new field allow_readonly to allow using table functions in readonly mode.
* Added TableFunctionProperties similar to AggregateFunctionProperties.
* The functions allowed in readonly mode are in this set table_functions_allowed_in_readonly_mode.
Testing:
* Added a test for filesystem tests/queries/0_stateless/02473_functions_in_readonly_mode.sh
Documentation:
* Updated the english documentation for Table Functions.
 2022-10-26 18:45:23 +02:00
Roman Vasin
09fe9c3ed1 Use {} in exceptions 2022-10-26 16:07:56 +00:00
Nikolai Kochetov
1520bcd53c Fix error code. 2022-10-26 15:50:10 +00:00
Anton Popov
ca4df62e37
Merge branch 'master' into refactor-data-part 2022-10-26 17:49:20 +02:00
Azat Khuzhin
7b69a70e82 Fix frequent memory drift message and clarify things in comments 
Somethine like:

    2022.09.28 06:33:34.001433 [ 3133669 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 562.20 MiB, peak 562.21 MiB, will set to 562.20 MiB (RSS), difference: -70.46 MiB
    2022.09.28 06:33:35.001639 [ 3133669 ] {} <Trace> AsynchronousMetrics: MemoryTracking: was 562.20 MiB, peak 562.21 MiB, will set to 562.20 MiB (RSS), difference: -70.45 MiB

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-26 17:38:37 +02:00
Azat Khuzhin
2faefc0c09 Add amount of memory used by allocator into logs 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-26 17:38:37 +02:00
Azat Khuzhin
f5f6f1b593 Add profile events for jemalloc purge 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-26 17:38:36 +02:00
Maksim Kita
6a0e3fc31d Subqueries remove unnecessary table expression modifiers 2022-10-26 17:19:29 +02:00
Maksim Kita
2b44a00fbc UNION node remove unnecessary union modes 2022-10-26 16:47:40 +02:00
Mikhail f. Shiryaev
fdecac65b8
Merge pull request #42699 from ClickHouse/22.11-prepare 
Update version after release
 2022-10-26 16:31:48 +02:00
Maksim Kita
1c17e9d454 Fixed tests 2022-10-26 15:54:37 +02:00
Mikhail f. Shiryaev
d3461abd1f
Update version to 22.11.1.1 2022-10-26 15:27:09 +02:00
李扬
04f8f38580
Merge branch 'master' into ascii_function 2022-10-26 07:29:49 -05:00
Kruglov Pavel
b8ee5600be
Merge pull request #42579 from ucasfl/set 
Fix create set with wrong header when data type is lowcardinality
 2022-10-26 14:17:14 +02:00
Nikolai Kochetov
db422fa903
Merge pull request #42410 from ClickHouse/fix-multi-arg-monotonicity 
Fix possible LOGICAL_ERROR in binary ariphmetics monotonicity.
 2022-10-26 14:11:23 +02:00
Kruglov Pavel
219553df8d
Merge pull request #42580 from hanfei1991/fei/limit_max_rows 
fix behaviour of max_rows_to_read for trival limit queries
 2022-10-26 14:09:59 +02:00
Maksim Kita
c47ca522f4 Fixed style check 2022-10-26 13:27:14 +02:00
Antonio Andelic
35a9672704 Merge branch 'master' into optimize-entire-partition 2022-10-26 11:09:25 +00:00
vdimir
14d0f6457b
Add tests and doc for some url-related functions 2022-10-26 10:52:57 +00:00
Maksim Kita
1b6293f6db MergeTree indexes use ActionsDAG 2022-10-26 12:44:37 +02:00
李扬
c7e5eb756b
Update src/Functions/ascii.cpp 
Co-authored-by: Vladimir C <vdimir@clickhouse.com>
 2022-10-26 18:18:15 +08:00
李扬
48c37c52e6
Update src/Functions/ascii.cpp 
Co-authored-by: Vladimir C <vdimir@clickhouse.com>
 2022-10-26 18:18:04 +08:00
Antonio Andelic
97aaebfa18 Address PR comments 2022-10-26 10:06:56 +00:00
SmitaRKulkarni
5df138da82
Merge pull request #42376 from ClickHouse/36316_Support_glob_for_recursive_directory_traversal 
Added ** glob support for recursive directory traversal to filesystem…
 2022-10-26 11:49:20 +02:00
Maksim Kita
6671c454de
Merge pull request #42651 from kitaisreal/analyzer-query-tree-node-remove-method 
Analyzer IQueryTreeNode remove getName method
 2022-10-26 12:36:38 +03:00
Maksim Kita
cd2a8966aa
Merge pull request #42649 from kitaisreal/analyzer-change-setting 
Analyzer change setting into allow_experimental_analyzer
 2022-10-26 12:26:15 +03:00
Antonio Andelic
979d180931
Merge branch 'master' into correctly-handle-prewhere-type 2022-10-26 11:13:16 +02:00
taiyang-li
add5360a1b implement function ascii 2022-10-26 03:02:00 -05:00
Roman Vasin
3eca9ada3b Add Date32 to formatDateTIme 2022-10-26 08:00:12 +00:00
Alexey Milovidov
011ae8675a
Merge pull request #42633 from ClickHouse/revert-41726-fix-slow-json-extract-with-low-cardinality 
Revert "Low cardinality cases moved to the function for its corresponding type"
 2022-10-26 08:10:27 +02:00
Nikolay Degterinsky
8c6d5cdbac
Merge pull request #42514 from evillique/fix-aliases 
Fix aliases
 2022-10-26 07:45:37 +02:00
serxa
471b391ab2 fix 2022-10-26 00:00:40 +00:00
serxa
27599ab70c fix comment 2022-10-25 23:58:25 +00:00
serxa
fc223aa673 replace throttler algorithm by token bucket 2022-10-25 23:52:31 +00:00
Anton Popov
79889198b1 fix style check 2022-10-25 23:03:37 +00:00
Anton Popov
d8b09b430f add comments 2022-10-25 22:27:11 +00:00
Anton Popov
c4d4f2dbed better interface 2022-10-25 22:14:06 +00:00
Robert Schulze
728342ec7c
Un-inline stuff 2022-10-25 21:41:08 +00:00
Alexey Milovidov
2b4be633ff
Merge branch 'master' into revert-41726-fix-slow-json-extract-with-low-cardinality 2022-10-25 22:57:12 +02:00
ltrk2
2c902bbc65
Implement support for different UUID binary formats (#42108) 
* Implement support for different UUID binary formats

* Declare error codes that the implementation uses

* Make single-argument constructor explicit

* Modernize parts of the solution

* Make improvements based on review comments

* Declare an error code being used
 2022-10-25 21:57:50 +02:00
Nikolai Kochetov
bfb5e6eae4 Fix logical error for invalid prewhere type in StorageMerge. 2022-10-25 13:58:29 +00:00
Antonio Andelic
611c2e2bd7 Support for optimizing old parts for entire partition only 2022-10-25 13:40:18 +00:00
Nikolai Kochetov
c4685f6994
Merge branch 'master' into fix-42575 2022-10-25 15:31:32 +02:00
Anton Popov
e2ff7aa261
Merge pull request #42623 from CurtizJ/fix-bad-cast 
Fix bad cast in `arrayElement` with `Map` with `Nullable` values and `Nullable` index
 2022-10-25 14:59:52 +02:00
Anton Popov
c8199bc125
Merge branch 'master' into refactor-data-part 2022-10-25 14:31:05 +02:00
Nikolai Kochetov
5c4444237e
Merge pull request #42617 from ClickHouse/revert-revert-41268-disable-s3-parallel-write-for-part-moves-to-disk-s3 
Revert revert 41268 disable s3 parallel write for part moves to disk s3
 2022-10-25 14:29:27 +02:00
Vladimir C
8fabe1515c
Merge pull request #42274 from dentiscalprum/fix_domain 2022-10-25 13:56:58 +02:00
Nikita Taranov
49f6692a2e
Adapt internal data structures to 512-bit era (#42564) 
* impl

* update tests

* fix tests
 2022-10-25 13:56:28 +02:00
Raúl Marín
0016bc2b73
Improve the time to recover keeper connections (#42541) 
Co-authored-by: Nikita Mikhaylov <nikitamikhaylov@clickhouse.com>
Co-authored-by: robot-clickhouse <robot-clickhouse@clickhouse.com>
Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>
 2022-10-25 13:26:29 +02:00
Antonio Andelic
af0c54e055 Small fix 2022-10-25 11:13:41 +00:00
Maksim Kita
a28d1f01fd Analyzer IQueryTreeNode remove getName method 2022-10-25 12:54:17 +02:00
Alexander Gololobov
759d8bcd80
Merge pull request #42609 from filimonov/patch-6 
Fix incorrect trace log line on dict reload
 2022-10-25 12:51:00 +02:00