thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-15 10:52:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
100,408 Commits 9,328 Branches 1,783 Tags 2.1 GiB
f39cb41d3e
Commit Graph

32872 Commits

Author SHA1 Message Date
taiyang-li
f39cb41d3e change as requested 2022-11-03 16:44:57 +08:00
taiyang-li
6770723027 modify as requested 2022-11-03 12:02:31 +08:00
taiyang-li
aeea21e4d7 Merge branch 'master' into ascii_function 2022-11-03 11:13:20 +08:00
Kruglov Pavel
38124b6533
Merge pull request #42780 from Avogar/parallel-parsing 
Support parallel parsing for LineAsString input format
 2022-11-02 13:21:53 +01:00
Robert Schulze
8ec4c77352
Merge pull request #42855 from GoGoWen/fix_typo 
Fix typo
 2022-11-02 12:54:52 +01:00
Vitaly Baranov
53ae7aba9a
Merge pull request #42628 from azat/s3-plain-fixes 
Implement ATTACH of MergeTree table for s3_plain disk
 2022-11-01 18:00:46 +01:00
Yakov Olkhovskiy
16159cc7a9
Merge pull request #42412 from ClibMouse/formatIpv6_s390x_support 
format ipv6 in s390x, endianess support
 2022-11-01 12:02:44 -04:00
Anton Popov
88674ef964
Merge pull request #42794 from CurtizJ/better-code-in-check-part 
Slightly better code with projections
 2022-11-01 16:05:27 +01:00
Anton Popov
876dca48da
Merge pull request #36969 from CurtizJ/dynamic-columns-14 
Support `Object` type inside other types
 2022-11-01 15:20:02 +01:00
Kruglov Pavel
c3a7211c1c
Merge pull request #41502 from marcioapm/coercing-json-extract 
Add coercing in JSONExtract
 2022-11-01 14:52:41 +01:00
Kruglov Pavel
7cbb534388
Merge pull request #42536 from ClickHouse/Avogar-patch-3 
Fix typo in setting name that led to bad usage of schema inference cache
 2022-11-01 14:03:47 +01:00
Kruglov Pavel
21d50f76ea
Merge pull request #41979 from Avogar/s3-cluster-schema-inference 
Fix schema inference in s3Cluster and improve in hdfsCluster
 2022-11-01 14:00:21 +01:00
Alexander Gololobov
f67f0183f6
Merge pull request #42812 from azat/utf8-fix-followup 
Fix lowerUTF8()/upperUTF8() in case of symbol was in between 16-byte boundary
 2022-11-01 12:01:00 +01:00
Maksim Kita
fe48a1ce7e
Merge pull request #42681 from kitaisreal/merge-tree-indexes-use-actions-dag 
MergeTree indexes use ActionsDAG
 2022-11-01 13:55:17 +03:00
Maksim Kita
27a72f096c
Merge pull request #42705 from kitaisreal/analyzer-small-improvements 
Analyzer improve subqueries representation
 2022-11-01 13:54:12 +03:00
GoGoWen2021
2b0cde9ef9 fix typo 2022-11-01 17:46:44 +08:00
Alexey Milovidov
1b3ef58b51
Merge branch 'master' into formatIpv6_s390x_support 2022-10-31 23:20:53 +01:00
Yakov Olkhovskiy
bab0e06e3d
Merge pull request #42458 from Clark0/match-func-use-index 
match function can use index if it's a condition on string prefix #37333
 2022-10-31 17:33:58 -04:00
Zhiguo Zhou
d18d08bcc7
Remove short-circuit evaluation in AssociativeApplierImpl::apply (#42214) 
The short-circuit evaluation was implemented when applying the
saturable operators (and, or) on a vector of ColumnUInt8. However,
its control flow would be compiled as a series of conditional
branch instructions which are hard to predict by the hardware and
at the same time hinder the vectorization optimization by the
compiler. This commit removes the short-circuit and evaluates the
whole expression.
 2022-10-31 21:36:15 +01:00
Maksim Kita
f099bf63b3
Merge pull request #42809 from Gabriel39/comment_typo 
Fix typo in comments
 2022-10-31 18:30:32 +03:00
Anton Popov
2ae3cfa9e0
Merge branch 'master' into dynamic-columns-14 2022-10-31 16:15:19 +01:00
Alexander Tokmakov
226931229b
Merge pull request #42805 from ClickHouse/fix_some_races_in_merge_tree 
Fix some races in MergeTree
 2022-10-31 17:53:20 +03:00
Márcio Martins
399d024c2c Fix array index argument type check for JSONExtract* functions 2022-10-31 15:08:34 +01:00
Márcio Martins
c33bd7794f
Merge branch 'master' into coercing-json-extract 2022-10-31 15:04:48 +01:00
Gabriel
22cf4b9457
Merge branch 'master' into comment_typo 2022-10-31 21:53:16 +08:00
Alexander Tokmakov
599ccb9939
Update MergeTreeData.h 2022-10-31 16:18:17 +03:00
Alexander Tokmakov
b2b9479afa
Update src/Storages/MergeTree/MergeTreeData.h 
Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>
 2022-10-31 16:15:35 +03:00
Maksim Kita
bca22ec5f5 Fixed code review issues 2022-10-31 12:30:00 +01:00
Nikolay Degterinsky
d8d479a8c0
Merge pull request #42804 from evillique/fix-create-user-parser 
Fix a bug in ParserCreateUserQuery
 2022-10-31 12:29:06 +01:00
Sergei Trifonov
8c67257a1c
Merge pull request #42665 from ClickHouse/token-bucket-throttler 
Replace throttler algorithm by token bucket
 2022-10-31 12:28:18 +01:00
Azat Khuzhin
c93262170d Remove ReadOnlyMetadataStorage 
Throw exceptions from IMetadataStorage instead to avoid introducing
extra abstractions.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:30 +01:00
Azat Khuzhin
51bd0c2ac1 Make ReadOnlyMetadataStorage really readonly 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:26 +01:00
Azat Khuzhin
bceca73f6f Throw NOT_IMPLEMENTED form ReadOnlyMetadataStorage::getLastChanged() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:26 +01:00
Azat Khuzhin
3b7abbbff4 Rename s/listPrefix/findAllFiles, s/listPrefixInPath/getDirectoryContents/ 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:26 +01:00
Azat Khuzhin
fa7535c90d Slightly optimize MetadataStorageFromPlainObjectStorage::isDirectory() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:26 +01:00
Azat Khuzhin
82ea67eb51 Slightly better MetadataStorageFromPlainObjectStorage::isFile() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:26 +01:00
Azat Khuzhin
95fb2ad3cf Implement ATTACH of MergeTree table for s3_plain disk 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:26 +01:00
Azat Khuzhin
4e42521f44 Introduce ReadOnlyMetadataStorage 
And use it for:
- MetadataStorageFromPlainObjectStorage
- MetadataStorageFromStaticFilesWebServer

This will allow to reduce ~100-200 lines of duplicated code, and plus
make the code less error prone.

Note, for now I tried to make this without behaviour changes.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:26 +01:00
Azat Khuzhin
f8ba24f040 Implement S3ObjectStorage::listPrefixInPath() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:26 +01:00
Azat Khuzhin
18e4fdf40f Introduce IObjectStorage::listPrefixInPath() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:25 +01:00
Azat Khuzhin
ee18140c48 Remove listPrefix() implementation for disks that does not support send_metadata 
The reason for removing is it because not compatible with restoring
(with send_metadata set) anyway:
- HDFS - is not compatible with send_metadata, and besides it's
  implementaion is not correct, since it is simply `ls -l`, while the
  following is required: `find . -maxdepth 1 -type f`
- Web - is not compatible with send_metadata anyway
- Local - is not compatible with send_metadata anyway

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:25 +01:00
Azat Khuzhin
88db8ae7fa Add a comment for IObjectStorage::listPrefix() 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:25 +01:00
Azat Khuzhin
94d9600fb8 Implement MetadataStorageFromPlainObjectStorage::getLastChanged() (as for web) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:25 +01:00
Azat Khuzhin
b34ffda272 Implement MetadataStorageFromPlainObjectStorage::getLastModified() (used by MergeTree) 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:25 +01:00
Azat Khuzhin
f6d0c03ee5 Fix path to files in MetadataStorageFromPlainObjectStorage 
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 12:02:25 +01:00
Maksim Kita
d760c78754 Fixed code review issues 2022-10-31 11:45:12 +01:00
Nikolai Kochetov
742d9af397
Merge pull request #42582 from azat/memory-tracker/fixes 
Fix frequent memory drift message and clarify things in comments
 2022-10-31 11:43:20 +01:00
Maksim Kita
115fcaffc5 Fixed tests 2022-10-31 11:30:56 +01:00
Azat Khuzhin
5fe44f2736 Fix lowerUTF8()/upperUTF8() in case of symbol was in between 16-byte boundary 
In lowerUTF8()/upperUTF8() there is an SSE optimization that handles
16 byte at a time, but only for ASCII, for UTF8 symbols converion will
be done by symbol.

Consider the following example:

    КВ АМ И СЖ
             ^ - offset is 15, length of sequence is 2
                 so first byte of a symbol is in first 16 bytes
                 second byte of a symbol is not ther

And in this case it will be handled incorrectly because it will try to
process oly these 16 bytes w/o looking forward.

This had been broken by #41286, before this patch it does not looks at
the row boundaries but only at the string end and so this sutation
wasn't possible.

Fixes: #42756
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 08:17:38 +01:00
Azat Khuzhin
32febf5155 Remove dead code in LowerUpperUTF8Impl::array() 
AFAICS it was there before since it was possible to overrun the
expected_end, since utf8.convert() was called with "src_end - src" not
"expected_end - src".

Refs: 5a21f3908b054a0efc90c65a12fbe151c74d90dc:dbms/include/DB/Functions/FunctionsString.h
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2022-10-31 08:00:29 +01:00