Commit Graph

49851 Commits

Author SHA1 Message Date
avogar
4a86f4a7b9 Fix style changes 2023-11-22 18:24:34 +00:00
avogar
007353a2dd Add _size virtual column to s3/file/hdfs/url/azureBlobStorage engines 2023-11-22 18:12:36 +00:00
Azat Khuzhin
cf3cd099a5 Split HashedDictionary CU
Before HashedDictionary.cpp exceeds 50MiB, now:

    -rw-r--r-- 1 azat azat  37M Nov 22 17:56 SparseHashedDictionary.cpp.o
    -rw-r--r-- 1 azat azat  34M Nov 22 17:56 HashedDictionary.cpp.o
    -rw-r--r-- 1 azat azat 716K Nov 22 17:56 registerHashedDictionary.cpp.o

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-11-22 19:00:40 +01:00
Julia Kartseva
753b22f64c
Merge pull request #56171 from ClibMouse/dev-seriesPeriodDetect
Implemented series period detect method using pocketfft lib
2023-11-22 09:49:58 -08:00
Alexander Tokmakov
d277bfb225 setActualPartName for executeReplaceRange 2023-11-22 18:48:25 +01:00
Nikolai Kochetov
1919ec723d Cancel executor in ~CreatingSetsTransform 2023-11-22 17:31:29 +00:00
Nikolay Degterinsky
761b55ccf4 Make overflow not possible 2023-11-22 16:37:30 +00:00
vdimir
7fb4a90d0b
Analyzer: Fix logical error in LogicalExpressionOptimizerVisitor 2023-11-22 16:34:31 +00:00
Alexey Milovidov
662c653eec
Merge pull request #57034 from jrdi/fix-bloom-filter-fpr
Avoid returning biggest resolution when fpr > 0.283
2023-11-22 17:28:51 +01:00
Alexey Milovidov
21d3ebcf35 Disable checksums for builds with fuzzer 2023-11-22 17:19:59 +01:00
vdimir
ffbe85d3a0
Merge pull request #56668 from ClickHouse/vdimir/analyzer_s3_partition_pruning
Analyzer: filtering by virtual columns for StorageS3
2023-11-22 16:44:44 +01:00
Nikolay Degterinsky
a6b659cf04 Fix buffer overflow in Gorilla codec 2023-11-22 15:21:27 +00:00
Nikolay Degterinsky
09496715b0 Add a check for empty metadata path 2023-11-22 15:04:54 +00:00
Duc Canh Le
44a3dc1cd8 add new method to check if two types are exactly identical
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2023-11-22 15:02:34 +00:00
Dmitry Novik
7539928814
Merge pull request #56079 from ZhiguoZh/20231027-combine-filter-avx512
Optimize DB::combineFilters with AVX512_VBMI2 intrinsic
2023-11-22 15:42:33 +01:00
Robert Schulze
f184d69115
Remove equivalent_integer typedef 2023-11-22 14:26:46 +00:00
Andrej Hoos
9b549fd4fc
Optimize query uniqueness check in ProcessList 2023-11-22 15:26:44 +01:00
Kruglov Pavel
e1904cbdf1
Cancel PipelineExecutor properly in case of exception in spawnThreads 2023-11-22 14:43:52 +01:00
Kruglov Pavel
9fb1acc947
Merge pull request #56810 from Avogar/iceberg-metadata-files
Iceberg metadata files
2023-11-22 14:40:04 +01:00
Alexey Milovidov
3463d9af7d
Merge pull request #57048 from ClickHouse/vdimir/pmj_crash_57016
Fix Nullptr dereference in partial merge join with joined_subquery_re…
2023-11-22 13:44:19 +01:00
János Benjamin Antal
ab935e3dd7 Use the google proto files when importing protobuf schemas 2023-11-22 12:39:41 +00:00
Alexey Milovidov
d0653f244b
Merge pull request #57062 from ClickHouse/add-comment-to-transformPipeline
Better comment for ITransformingStep::transformPipeline
2023-11-22 13:37:11 +01:00
Alexey Milovidov
54410f16c6
Merge pull request #57063 from ClickHouse/fix-duplicate-storage-set-analyzer
Fix `Duplicate set` for StorageSet with analyzer.
2023-11-22 13:36:44 +01:00
Kruglov Pavel
96a3cefff6
Merge pull request #56118 from edef1c/large-arrow-list
Support reading arrow::LargeListArray
2023-11-22 13:28:46 +01:00
Alexey Milovidov
77fc95ab67 Minor changes 2023-11-22 12:13:47 +01:00
Alexey Milovidov
9cf7ebeced Minor changes 2023-11-22 12:13:47 +01:00
Alexey Milovidov
dd54c7e224 Fix error 2023-11-22 12:13:47 +01:00
Alexey Milovidov
84f669c113 Fix error 2023-11-22 12:13:47 +01:00
Alexey Milovidov
aa41bc49bf Faster build time 2023-11-22 12:13:47 +01:00
Alexey Milovidov
bba3e2a189 Faster build time 2023-11-22 12:13:47 +01:00
Alexey Milovidov
c72b256c63 Fix error 2023-11-22 12:13:47 +01:00
Alexey Milovidov
a3c4c84c87 Fix error 2023-11-22 12:13:47 +01:00
Alexey Milovidov
d25b556223 Avoid excessive allocation in Arena 2023-11-22 12:13:47 +01:00
Alexey Milovidov
d1015aae8e
Merge pull request #57073 from ClickHouse/implement-bit-hamming-distance-for-big-integers
Implement `bitHammingDistance` for big integers
2023-11-22 11:56:33 +01:00
Kseniia Sumarokova
2880e6437e
Merge pull request #56936 from jrdi/fs-cache-hit-profile-events
Add CachedReadBufferReadFromCache{Hits,Misses} profile events
2023-11-22 11:26:00 +01:00
Igor Nikonov
7c0198a26c Fix clang-tidy warning 2023-11-22 09:48:18 +00:00
Raúl Marín
e92044cec0
Merge pull request #57052 from Algunenano/fix_race_remotesource
Fix race condition in RemoteSource
2023-11-22 09:23:57 +01:00
Raúl Marín
a1bb31525a
Merge pull request #57029 from ClickHouse/fix_DatabaseFilesystem_path_validation
Fix file path validation for DatabaseFileSystem
2023-11-22 09:19:19 +01:00
Antonio Andelic
b18cb1e5cd
Merge pull request #57047 from ClickHouse/keeper-small-fixes
Small Keeper fixes
2023-11-22 08:47:58 +01:00
Antonio Andelic
8c88dbe5aa
Merge pull request #57033 from ClickHouse/fix-fuzzbits
Fix `fuzzBits` with `ARRAY JOIN`
2023-11-22 08:46:28 +01:00
Jordi Villar
3fcc6031a5 Fix unit test 2023-11-22 08:33:41 +01:00
Jordi Villar
80d21e2ccd Adding a unit test 2023-11-22 08:33:41 +01:00
Jordi Villar
0a14b39aed Avoid returning bigger resolution when fpr > 0.283 2023-11-22 08:33:41 +01:00
Nikolay Degterinsky
58b51a7f23 Better metadata path 2023-11-22 05:37:51 +00:00
Alexey Milovidov
5cb55d3f8c Fix style 2023-11-22 01:24:58 +01:00
Alexey Milovidov
de5556b458 Add a test 2023-11-22 00:29:43 +01:00
Alexey Milovidov
8ed04c7645 Merge branch 'master' into implement-bit-hamming-distance-for-big-integers 2023-11-22 00:10:11 +01:00
Ilya Golshtein
c2816ec589 merge_row_policy: make clang-tidy happy 2023-11-21 22:06:18 +00:00
Dmitry Novik
1e0b4f979c
Add extra test 2023-11-21 22:55:55 +01:00
Kseniia Sumarokova
e4f66b8469
Merge pull request #55158 from kssenii/fs-cache-improvement
fs cache improvement for big reads
2023-11-21 21:50:00 +01:00
Kseniia Sumarokova
04aa0b10fc
Merge pull request #56990 from ClickHouse/s3queue-fix-incorrect-ref-count-from-exception-in-ctor
S3Queue: fix metadata reference increment
2023-11-21 21:37:23 +01:00
Alexey Milovidov
fd8714ac29 Implement bitHammingDistance for big integers 2023-11-21 18:38:00 +01:00
Alexander Tokmakov
990e3ea1c8 Merge branch 'master' into rmt_check_shutdown_flags_in_retry_loops 2023-11-21 18:01:16 +01:00
vdimir
a139ae97eb
Merge pull request #52918 from ClickHouse/vdimir/s3_blob_log
Add system table with blob storage operations log
2023-11-21 17:40:42 +01:00
Igor Nikonov
63a1625b77 Merge remote-tracking branch 'origin/master' into pr-cleanup-narrow-dependency 2023-11-21 16:05:48 +00:00
Igor Nikonov
ce98dfb251 Settings pointer to reference 2023-11-21 16:04:54 +00:00
Nikolai Kochetov
c85d739486 Fix Duplicate set for StorageSet with analyzer. 2023-11-21 15:57:53 +00:00
Nikolai Kochetov
b1ec8a3670 Better comment for ITransformingStep::transformPipeline 2023-11-21 15:35:09 +00:00
SmitaRKulkarni
e0ceb5d6a5
Merge pull request #56004 from ClickHouse/Fix_parameterized_view_explain_ast
Bug fix explain ast with parameterized view
2023-11-21 16:02:28 +01:00
Bhavna Jindal
eb35e61aea added validateFunctionArgumentTypes 2023-11-21 06:52:48 -08:00
Bhavna Jindal
4767859ba5 added validateFunctionArgumentTypes 2023-11-21 06:52:48 -08:00
Bhavna Jindal
f0d9037e74 Added embeded function documentation 2023-11-21 06:52:48 -08:00
Bhavna Jindal
64292f36f5 fixed getReturnTypeImpl to accept only array argument 2023-11-21 06:52:48 -08:00
Bhavna Jindal
464cdc4a91 fixed getReturnTypeImpl to accept only array argument 2023-11-21 06:52:48 -08:00
Bhavna Jindal
0843d5e890 added macro to enable/disable pocketfft lib 2023-11-21 06:52:47 -08:00
Bhavna Jindal
90df67a6b1 fix header file issue and added edge case handling 2023-11-21 06:52:47 -08:00
Bhavna Jindal
2c1cf7ed43 fix header file issue and added edge case handling 2023-11-21 06:52:47 -08:00
Bhavna Jindal
56a7fe6c86 fix header file issue and added edge case handling 2023-11-21 06:52:47 -08:00
Bhavna Jindal
ad99cb902f changed method name, updated pocketfft repo reference 2023-11-21 06:52:47 -08:00
Bhavna Jindal
62d69946eb Added testcases, documentation and addressed comments 2023-11-21 06:52:47 -08:00
Bhavna Jindal
3fa104ec11 format file 2023-11-21 06:52:47 -08:00
Bhavna Jindal
c4c12509ee removed commented code and fixed a bug 2023-11-21 06:52:47 -08:00
Bhavna Jindal
2235adcae5 Implemented series periods detect method using pocketfft lib 2023-11-21 06:52:47 -08:00
Igor Nikonov
66f6a6575f Cleanup iteration: settings usage 2023-11-21 13:29:04 +00:00
vdimir
8f34b36293
Merge pull request #56538 from ClickHouse/vdimir/join_on_nullsafe_optimization
Rewrite equality with is null check in JOIN ON section
2023-11-21 14:17:58 +01:00
Antonio Andelic
90bb796d63 Small Keeper fixes 2023-11-21 13:11:33 +00:00
Raúl Marín
47abb9e879 Fix race condition in RemoteSource 2023-11-21 14:07:14 +01:00
凌涛
6052cfc990
Modified spelling 2023-11-21 20:34:11 +08:00
vdimir
ee4d5d758f
Fix Nullptr dereference in partial merge join with joined_subquery_requires_alias = 0 2023-11-21 12:15:12 +00:00
Igor Nikonov
83c6ac4f6c Remove storage as parameter dependency, only columns are needed 2023-11-21 12:14:12 +00:00
Antonio Andelic
2ae32d3383
Update src/Functions/getClientHTTPHeader.cpp 2023-11-21 13:12:44 +01:00
Alexander Gololobov
8cf93fe133
Merge pull request #56905 from ClickHouse/disk_encrypted_create_dirs
Allow delegate disk to handle retries for createDirectories
2023-11-21 12:51:05 +01:00
凌涛
e86aa433c5
Merge branch 'master' into feature/getHTTPHeader 2023-11-21 19:48:47 +08:00
凌涛
5e581a9fa4 optimize code 2023-11-21 19:47:39 +08:00
alesapin
227bb30d02
Merge pull request #56986 from kevinmingtarja/implicit-constraint-collapsing-merged-tree
Add implicit constraint for CollapsingMergeTree sign column
2023-11-21 12:43:51 +01:00
Robert Schulze
6851c1dbbc
Suppress bs clang-tidy warning 2023-11-21 11:15:19 +00:00
Robert Schulze
c5828cf856
Merge pull request #57000 from slvrtrn/concat-single-arg
Allow to use `concat` function with a single argument
2023-11-21 12:06:29 +01:00
kssenii
474f1ae0b6 Better 2023-11-21 11:42:23 +01:00
Raúl Marín
e34fea5495
Merge pull request #56984 from ClickHouse/fix_sqlite_file_path_validation
Fix sqlite file path validation
2023-11-21 11:40:20 +01:00
Aleksandr Musorin
ee0ae17ce2 Added optimize_trivial_approximate_count_query documentatiosn for RocksDB 2023-11-21 11:31:56 +01:00
Kseniia Sumarokova
d384762123
Merge branch 'master' into fs-cache-improvement 2023-11-21 11:24:52 +01:00
Salvatore Mesoraca
f96cd4cfb8
Update src/Storages/NamedCollectionsHelpers.cpp
Co-authored-by: Nikolay Degterinsky <43110995+evillique@users.noreply.github.com>
2023-11-21 11:14:48 +01:00
Kseniia Sumarokova
958030758a
Merge pull request #56999 from ClickHouse/s3queue-fix-empty-key-check
S3Queue minor fix
2023-11-21 11:11:08 +01:00
Antonio Andelic
f996dac75d Fix fuzzBits 2023-11-21 09:43:27 +00:00
Jordi Villar
6bd20abb90 Add CachedReadBufferReadFromCache{Hits,Misses} profile events 2023-11-21 10:34:12 +01:00
Salvatore Mesoraca
970e34f32e
NamedCollections: make exception message more informative.
Fix test 00002_log_and_exception_messages_formatting
2023-11-21 10:21:28 +01:00
vdimir
15234474d7
Implement system table blob_storage_log 2023-11-21 09:18:25 +00:00
santrancisco
f4d936fc55
Fix style 2023-11-21 17:39:12 +11:00
santrancisco
3eef0601e3
Fix file path validation for DatabaseFileSystem 2023-11-21 16:29:40 +11:00
凌涛
c5f16725ec add function getClientHTTPHeader 2023-11-21 11:12:01 +08:00
Alexey Milovidov
fdc8b0ceb1
Merge pull request #56956 from evillique/local-send-fatal-log
Send fatal logs by default in clickhouse-local
2023-11-20 23:08:13 +01:00
Alexey Milovidov
0de18bfa6f
Merge pull request #56972 from Michicosun/fix_select_query_options_pass
Something InterpreterSelectQueryAnalyzer::getSampleBlock
2023-11-20 22:46:19 +01:00
Sema Checherinda
485f1834d8
Merge pull request #56938 from CheSema/lz4-buffering
Lz4 compression: buffer block in a rare case
2023-11-20 20:33:30 +01:00
Kseniia Sumarokova
2beee5d9cb
Merge branch 'master' into s3queue-fix-empty-key-check 2023-11-20 20:17:47 +01:00
vdimir
31a6c7c1c4
Style changes around filterKeysForPartitionPruning 2023-11-20 18:08:45 +00:00
vdimir
95e9a27417
Remove ast based code from filterKeysForPartitionPruning 2023-11-20 17:59:58 +00:00
vdimir
a915eeded8
StorageS3 use filters from SourceStepWithFilter 2023-11-20 17:59:58 +00:00
vdimir
1cddfb1e6b
rewrite getBlockWithVirtuals for S3Storage 2023-11-20 17:59:57 +00:00
vdimir
cbb2e02c03
Analyzer: partition pruning for S3 2023-11-20 17:59:53 +00:00
slvrtrn
d348db8438 Merge remote-tracking branch 'origin' into concat-single-arg 2023-11-20 17:57:50 +01:00
Arthur Passos
3544ee1e5f fix build by removing some const specifiers 2023-11-20 13:52:18 -03:00
Arthur Passos
e5129990ed sign all aws headers 2023-11-20 13:38:32 -03:00
slvrtrn
6d14350886 Allow to use any integer types with fromDaysSinceYearZero 2023-11-20 17:26:31 +01:00
Nikolai Kochetov
2a59a688de
Merge pull request #56867 from kitaisreal/planner-support-transactions
Planner support transactions
2023-11-20 17:10:22 +01:00
Aleksandr Musorin
38133692bb Add total_bytes for RocksDB storage
Show total_bytes in system tables for RocksDB storage

Previously it did not work because total_rows was calculated only when optimize_trivial_approximate_count_query=1
2023-11-20 17:08:35 +01:00
Kevin Mingtarja
1d668264e1 Fix style 2023-11-20 23:54:36 +08:00
Kevin Mingtarja
d84d5692ef Enable implicit constraint for VersionedCollapsing 2023-11-20 23:52:22 +08:00
Zhiguo Zhou
be0f0b6879 Fix code style 2023-11-20 23:34:52 +08:00
kssenii
9b5e180995 Fix 2023-11-20 16:27:02 +01:00
slvrtrn
59c251bf88 Allow to use concat with a single argument 2023-11-20 16:17:31 +01:00
Alexander Tokmakov
0eaf83b02e
Merge pull request #56994 from ClickHouse/revert-56992-revert-56314-s3-aggressive-timeouts
Revert "Revert "s3 adaptive timeouts""
2023-11-20 15:14:46 +01:00
Kruglov Pavel
7f30fb7fde
Merge pull request #56941 from Avogar/fix-auto-schema
Fix generating deep nested columns in CapnProto/Protobuf schemas
2023-11-20 14:58:56 +01:00
Kseniia Sumarokova
56222ddd57
Merge pull request #56872 from ClickHouse/filecache-call-assertions-more-often
Call cache check a bit more often
2023-11-20 14:57:24 +01:00
Sema Checherinda
f999337dae
Revert "Revert "s3 adaptive timeouts"" 2023-11-20 14:53:22 +01:00
Alexander Tokmakov
5031f239c3
Revert "s3 adaptive timeouts" 2023-11-20 14:28:59 +01:00
Sema Checherinda
fafd169e7b
Update src/IO/Lz4DeflatingWriteBuffer.cpp
Co-authored-by: Antonio Andelic <antonio2368@users.noreply.github.com>
2023-11-20 14:12:52 +01:00
vdimir
06c6282eb7
fix build 2023-11-20 13:46:37 +01:00
Nikita Mikhaylov
bf141f5aff
Update src/Core/Settings.h
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
2023-11-20 13:25:51 +01:00
vdimir
28bb76d568
address review comments 2023-11-20 13:21:46 +01:00
kssenii
1ade4b797b Add a comment 2023-11-20 12:48:49 +01:00
kssenii
10fb40ece9 Fix 2023-11-20 12:44:17 +01:00
alesapin
857ce2d880
Merge branch 'master' into implicit-constraint-collapsing-merged-tree 2023-11-20 12:28:55 +01:00
Raúl Marín
44874859bb Fix style 2023-11-20 12:21:12 +01:00
Sema Checherinda
ebb66c1a9e add comments 2023-11-20 12:13:24 +01:00
Mikhail Artemenko
e305bf6549 Merge branch 'master' into fix_select_query_options_pass 2023-11-20 08:39:56 +00:00
Zhiguo Zhou
1f044b11e3 Enhance unittest MergeTree.CombineFilters 2023-11-20 16:00:14 +08:00
Kevin Mingtarja
af0c954c9e Add implicit constraint for CollapsingMergeTree sign column 2023-11-20 15:05:12 +08:00
santrancisco
f05f572b3c
Fixed trailing whitespace 2023-11-20 13:37:27 +11:00
santrancisco
ab5f3d12b6
Fix sqlite file path validation to make sure it does not skip validation on relative path 2023-11-20 12:51:30 +11:00
melvynator
cfdb663892 Remove a tab 2023-11-20 00:46:14 +01:00
Igor Nikonov
7020177e7c
Merge branch 'master' into pr-cleanup-remote-source 2023-11-19 21:33:47 +01:00
Nikolay Degterinsky
fd26fab07f Merge remote-tracking branch 'upstream/master' into local-send-fatal-log 2023-11-19 17:59:51 +00:00
Alexey Milovidov
62a87665c5 Fix build 2023-11-19 16:31:18 +01:00
Nikolay Degterinsky
9683adfb03
Merge pull request #56830 from evillique/better-add-column
Fix flattening existing Nested columns during ADD COLUMN
2023-11-19 16:21:56 +01:00
Nikolay Degterinsky
cf46214440 Always send fatal level logs 2023-11-19 15:09:52 +00:00
Nikolay Degterinsky
44289c5f19
Merge pull request #56957 from evillique/resubmit-better-except
Resubmit: Better except for SSL authentication
2023-11-19 16:00:18 +01:00
Mikhail Artemenko
b117d8ed12 create interpreter with changed select_query_options 2023-11-19 14:28:33 +00:00
Sema Checherinda
a950595c24
Merge pull request #56314 from CheSema/s3-aggressive-timeouts
s3 adaptive timeouts
2023-11-19 14:12:14 +01:00
Sema Checherinda
cacc23b8b7 safe SinkToOut d-tor 2023-11-19 12:25:42 +01:00
Sema Checherinda
24fbe620d3 fix build 2023-11-19 12:14:53 +01:00