Robert Schulze
81543c0a46
Merge remote-tracking branch 'rschu1ze/master' into fix_remove_inverted_index_files
2024-01-23 09:45:09 +00:00
Nikita Taranov
09e24ed6c5
Merge pull request #58068 from zhongyuankai/limit_materialized_view_size
...
A limit on the number of materialized views attached to a table.
2024-01-22 22:50:56 +01:00
Nikita Taranov
b161921825
impl
2024-01-22 22:33:23 +01:00
Max K
4932b1cad9
Merge branch 'master' into better-read-buffers
2024-01-22 21:52:55 +01:00
Arthur Passos
24b8bbe9fa
Allow to attach partition from table with different partition expression when destination partition expression doesn't re-partition ( #39507 )
...
* temp commit
* temp commit
* draft impl for feedback
* fix weird style changes
* fix weird style changes
* fix weird style changes
* fix weird style changes
* fix weird style changes
* aa
* aa
* Add integ tests and remove partition key restriction
* fix small incosistency in partition id
* style fix
* style fix
* style fix
* use existing DataPartStorageBuilder instead of new one
* Refactor part clone to make it more readable and maintainable
* Add MergeTreeDataPartCloner docs
* define ErrorCodes::BAD_ARGUMENTS
* Rebase
* camel case methods
* address some comments
* yet another rebase?
* Move from integ tests to stateless tests
* address more comments
* add finalize on min_max_idx files
* Add sync option to DistinctPartitionExpCloner
* just a temp test
* revert temp change
* Use echoOn to distinguish test queries
* remove comment
* fix build issue during rebase
* atempt to fix build after rebase
* finally fix build
* clear minmaxidx hyperrectangle before loading it
* Fix error on min_max files deletion where it was being assumed that partition expression contained all columns
* get it to the state it was previously
* add missing include
* getting functional?
* refactoring and renaming
* some more refactoring
* extern bad arguments
* try to fix style
* improvements and docs
* remove duplicate includes
* fix crash
* make tests more stable by ordering
* rebase once again..
* fix
* make ci happy?
* fix rebase issues
* docs
* rebase, but prolly needs to be improved
* refactor out from nasty inheritance to static methods
* fix style
* work around optional
* refactor & integrate some changes
* update column_type
* add tests by dencrane
* set utc
* fix ref file
* fix tests
* use MergeTree instead of SummingMergeTree
* mark MergeTreeDataPart::getBlock as const
* address a few comments
* compute module function name size at compile time
* simplify branching in getPartitionAstFieldsCount
* remove column_indexes argument
* merge getBlock with buildBlock
* add some const specifiers
* small adjustments
* remove no longer needed isNull check
* use std::min and max to update global min max idx
* add some assertions
* forward declare some symbols
* fix grammar
* forward decl
* try to fix build..
* remove IFunction forward decl
* Revert "use std::min and max to update global min max idx"
This reverts commit b2fe79dda7129db2610f9416087dd820246d4416
2024-01-22 21:00:04 +01:00
kssenii
1aa8e62bb7
Fix
2024-01-22 18:26:44 +01:00
Nikolai Kochetov
65c5ea31a8
Revert "Revert "Allow parallel replicas for JOIN with analyzer [part 1].""
2024-01-22 17:09:34 +01:00
Dmitry Novik
4e5249275e
Do not resolve remote table id on initiator
2024-01-22 16:45:25 +01:00
Smita Kulkarni
2e7ce5b0e2
Updated settings ptr and fetching of client from Disk & ObjectStorage
2024-01-22 16:24:43 +01:00
Anton Popov
e1e7a9cd45
Merge branch 'master' into better-read-buffers
2024-01-22 15:50:48 +01:00
Alexander Tokmakov
c2202ff347
Skip unavailable replicas in parallel distributed insert select ( #58931 )
...
* Update StorageDistributed.cpp
* Update StorageDistributed.cpp
* Update StorageDistributed.cpp
* add a test
* Update 01099_parallel_distributed_insert_select.sql
2024-01-22 15:34:46 +01:00
mochi
0606a77267
Update src/Storages/MergeTree/MutateTask.cpp
...
Co-authored-by: Dmitry Novik <mrnovikd@gmail.com>
2024-01-22 22:02:50 +08:00
Alexander Tokmakov
4315d44517
Revert "Allow parallel replicas for JOIN with analyzer [part 1]."
2024-01-22 13:30:35 +01:00
Kseniia Sumarokova
aec76d6653
Merge pull request #58877 from ClickHouse/multiple-read-file-log-storage-in-mv
...
Multiple read file log storage in mv
2024-01-22 12:02:41 +01:00
MochiXu
02b178cc9c
fix drop inverted index
2024-01-22 11:10:58 +08:00
Vitaly Baranov
01aafda27b
Merge pull request #58804 from vitlibar/allow-kill-query-for-backups
...
Allow KILL QUERY for backups
2024-01-20 23:48:18 +01:00
Igor Nikonov
4be068c73b
Merge pull request #58993 from ClickHouse/revert-58992-revert_flaky
...
Revive: Parallel replicas custom key: skip unavailable replicas
2024-01-20 19:44:56 +01:00
robot-clickhouse
273a06856a
Merge pull request #59025 from ClickHouse/cpu-scheduler
...
Split resource scheduler off `IO/` into `Common/Scheduler/`
2024-01-20 15:43:50 +01:00
Kseniia Sumarokova
5d67e26fc4
Merge pull request #58869 from ClickHouse/force-check-for-cached-custom-disk
...
Fix for custom cached disk creation
2024-01-20 14:47:36 +01:00
Kseniia Sumarokova
689e617c12
Merge pull request #58963 from ClickHouse/follow-up-disk-configuration
...
Follow up to #58357
2024-01-20 14:41:34 +01:00
serxa
824deab761
split resource scheduler off IO/ into Common/Scheduler/
2024-01-20 12:06:26 +00:00
Igor Nikonov
59025b2d3a
Merge remote-tracking branch 'origin/master' into revert-58992-revert_flaky
2024-01-19 23:38:24 +00:00
Kruglov Pavel
20379d34d5
Merge branch 'master' into variant-data-type
2024-01-19 23:17:42 +01:00
Michael Kolupaev
fd361273f0
Fix StorageURL forgetting headers on server restart
2024-01-19 11:35:12 -08:00
Alexey Milovidov
decdb62cef
Merge pull request #58800 from ZhiguoZh/20240115-combine-filter-bmi2
...
Optimize DB::combineFilters with BMI2/SSE intrinsics
2024-01-19 16:50:04 +01:00
Dmitry Novik
728276ab3f
Fix crash when indexHint() is used
2024-01-19 14:59:33 +00:00
zhongyuankai
cbb51659f6
Merge branch 'master' into limit_materialized_view_size
2024-01-19 22:00:12 +08:00
Igor Nikonov
131f71b50a
Revert "Revert flaky"
2024-01-19 13:51:30 +01:00
Raúl Marín
477489a05d
Revert "Merge pull request #57235 from ClickHouse/pr-custom-key-failover"
...
This reverts commit 8ba9b4a7ef0f39245cb8
2024-01-19 12:46:05 +00:00
kssenii
2f34ef6ff8
Follow up
2024-01-19 11:19:59 +01:00
kssenii
48e489682b
Merge remote-tracking branch 'origin/master' into broken-projections-better-handling
2024-01-19 10:49:27 +01:00
Kseniia Sumarokova
1c74e7fa08
Merge pull request #58902 from ClickHouse/fs-cache-refactorings
...
fs cache: minor refactoring
2024-01-19 10:25:45 +01:00
Robert Schulze
1bfeee1954
Some fixups
2024-01-18 20:49:50 +00:00
Robert Schulze
15700592f7
Merge remote-tracking branch 'ClickHouse/master' into column_level_compress_block
2024-01-18 19:12:57 +00:00
Kruglov Pavel
5444cde408
Merge branch 'master' into variant-data-type
2024-01-18 18:31:27 +01:00
avogar
f202d71371
Make alter with variant espansion as no-op, add tests for alters
2024-01-18 17:29:36 +00:00
Dmitry Novik
ba2dad98bb
Merge pull request #58913 from ClickHouse/fix-style
...
Style fix
2024-01-18 14:24:39 +01:00
Nikolai Kochetov
9db8b50753
Merge pull request #58838 from ClickHouse/allow-parallel-replicas-for-join-with-analyzer
...
Allow parallel replicas for JOIN with analyzer [part 1].
2024-01-18 14:01:08 +01:00
János Benjamin Antal
f34d1ae647
Use WithContext
2024-01-18 11:47:59 +00:00
Smita Kulkarni
e71977b724
Merge branch 'master' into Azure_backup
2024-01-18 08:29:01 +01:00
Duc Canh Le
22afcd46f4
add ALTER COLUMN MODIFY/RESET SETTING
...
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2024-01-18 05:17:07 +00:00
Alexander Gololobov
0a42dbafe8
Add attributes
2024-01-17 17:46:15 +01:00
Anton Popov
72b0fed4ef
Merge pull request #58729 from CurtizJ/nested-size0
...
Fix reading of offsets subcolumn (`size0`) from `Nested`
2024-01-17 17:16:27 +01:00
Alexander Gololobov
7dda18c209
Spans in mergeBlock and finalizeImpl
2024-01-17 16:07:53 +01:00
Dmitry Novik
3a1c3638e8
Style fix
2024-01-17 15:04:50 +00:00
Anton Popov
245e1cbd5c
Merge remote-tracking branch 'upstream/master' into better-read-buffers
2024-01-17 13:46:48 +00:00
kssenii
c905937ef6
Refactoring
2024-01-17 13:22:59 +01:00
Dmitry Novik
a5e0be484a
Merge pull request #58875 from ClickHouse/fix-indexhint-rpn-builder
...
Fix RPN construction for indexHint
2024-01-17 12:24:28 +01:00
Smita Kulkarni
8d88f4cf87
Update setting is_parameterized_view & settings columns for view
2024-01-17 09:42:53 +01:00
Anton Popov
299293c088
better interface in read buffers
2024-01-16 23:26:20 +00:00
János Benjamin Antal
c9b6d53948
Unify naming
2024-01-16 16:16:08 +00:00
Dmitry Novik
80291ec3f2
Add comments
2024-01-16 16:59:04 +01:00
Dmitry Novik
c325f08e79
Fix RPN construction for indexHint
2024-01-16 15:34:45 +00:00
János Benjamin Antal
1d34f4a304
Merge pull request #58809 from ClickHouse/print-entries-to-be-processed-in-trace
...
Increase log level to trace to help debug `00993_system_parts_race_condition_drop_zookeeper`
2024-01-16 16:14:22 +01:00
Kruglov Pavel
74973c1b41
Merge pull request #58537 from sunny19930321/fix/hive-threadpool-query
...
Fix hive threadpool read ORC/Parquet/... Failed
2024-01-16 14:46:22 +01:00
kssenii
8decf5039a
Fix for custom cached disk
2024-01-16 14:41:12 +01:00
Nikita Taranov
7d03ccd8db
Merge pull request #58844 from nickitat/fixxxx
...
Fix reading from an invisible part in new (not released yet) parallel replicas coordinator
2024-01-16 14:09:21 +01:00
Nikolai Kochetov
534e086a5d
Update StorageMergeTree.cpp
2024-01-16 13:50:39 +01:00
Smita Kulkarni
76ba60aa24
Merge branch 'master' into Support_parameterized_view_with_analyzer
2024-01-16 11:26:41 +01:00
zhongyuankai
cca2fa6976
fix test
2024-01-16 15:39:33 +08:00
Zhiguo Zhou
7450c35189
Optimize DB::combineFilters with BMI2/SSE intrinsics
...
The DB::combineFilters combines two UInt8 columns (filters) by
scanning the first column for non-zero elements and replacing them
with elements in the second column contiguously.
As a follow-up of #56079 , the function is optimized with the BMI2
and SSE intrinsics so that the x86 devices without AVX512 support
could also accelerate the column combinations via data parallelism.
2024-01-16 15:33:27 +08:00
Nikita Taranov
b9132f7f69
add check in read pool
2024-01-16 00:57:10 +01:00
Nikita Taranov
90d30b6d11
impl
2024-01-15 23:56:53 +01:00
alesapin
f1b1096eb4
Fix isRetry method
2024-01-15 20:10:21 +01:00
Nikolai Kochetov
733ceaa775
Allow parallel replicas for JOIN with analyzer.
2024-01-15 18:48:12 +00:00
Anton Popov
b430f3094e
fix nested size0 for storage Log
2024-01-15 17:26:44 +00:00
Kseniia Sumarokova
7ded82e51f
Merge pull request #58297 from azat/qp-alter-part
...
Support query parameters in ALTER TABLE ... PART
2024-01-15 18:18:34 +01:00
János Benjamin Antal
b04e8b11ae
Delay reading from StorageFileLog by creating a query plan step
2024-01-15 11:39:43 +00:00
János Benjamin Antal
3e1d7bf685
Merge pull request #58477 from ClickHouse/multiple-read-kafka-storage-in-mv
...
Delay reading from StorageKafka to allow multiple reads in materialized views
2024-01-15 12:31:59 +01:00
Azat Khuzhin
48a700edba
Support query parameters in ALTER TABLE ... PART
...
But, to make this happen it is not enough to simply fix the parser, the
whole ASTs for ALTER should be fixed as well, since they store `ASTPtr`
instead of `IAST *`, and so because of `ReplaceQueryParameterVisitor`
does not work for ALTER query.
So this patch converts ASTPtr to `IAST *` for ASTAlterQuery and
ASTAlterCommand (plus it also fixes when the children for
ASTAlterCommand had not been updated sometimes).
And only after this it is possible to adjust the parser (see
places with `parser_string_and_substituion` in the parser code).
v2: also fix ASTAlterCommand::clone (add missing components) and ParserAlterCommand::parseImpl
v3: fix APPLY DELETED MASK
v4: fix mutations_execute_nondeterministic_on_initiator/mutations_execute_subqueries_on_initiator
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-01-15 14:16:54 +03:00
János Benjamin Antal
f4e8e47c7a
Increase log level to trace
2024-01-15 10:44:40 +00:00
Robert Schulze
9cfdff2ddb
Merge pull request #58284 from lingtaolf/Improvement/inverted_index_support_match
...
Utilize inverted index by function `match()`
2024-01-15 11:37:23 +01:00
zhongyuankai
826b29ef4d
Max materialized views size for table
2024-01-15 10:02:57 +08:00
Alexey Milovidov
a471a54987
Merge pull request #55528 from ClickHouse/insert-quorum-host-name-check
...
Insert quorum: check host node version in addition
2024-01-15 02:23:15 +01:00
Vitaly Baranov
3d7a17af08
Allow KILL QUERY to cancel backups/restores.
2024-01-15 02:22:35 +01:00
Alexander Tokmakov
5128607995
Merge pull request #58393 from jaymebrd/feat/add-from-modifier-sync-replica-lightweight
...
add `from srcreplicas` modifier for `SYSTEM SYNC REPLICA LIGHTWEIGHT`
2024-01-14 20:53:16 +01:00
Nikita Taranov
df0c6cc029
Fix bug in new (not released yet) parallel replicas coordinator ( #58722 )
2024-01-14 16:16:20 +01:00
Duc Canh Le
ac76eecb1e
Merge branch 'master' into column_level_compress_block
...
Resolve conflicts
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2024-01-14 09:32:43 +00:00
Alexey Milovidov
65ac905032
Merge pull request #54366 from canhld94/final_no_copy
...
Implement vertical replacing FINAL
2024-01-14 04:21:07 +01:00
Alexey Milovidov
e92e3615e3
Merge branch 'master' into non-ready-set-ttl
2024-01-13 21:11:51 +01:00
Alexey Milovidov
12585ea0e4
Update TTLDescription.cpp
2024-01-13 23:10:27 +03:00
Alexey Milovidov
211c285a0b
Merge pull request #58657 from ClickHouse/analyzer-fix-index-set
...
Analyzer: Correctly handle constant set in index
2024-01-13 19:23:28 +01:00
Alexey Milovidov
d10a6e91af
Merge pull request #58770 from hdnpth/add-missing-includes
...
Add missing includes when _LIBCPP_REMOVE_TRANSITIVE_INCLUDES enabled
2024-01-13 19:22:49 +01:00
Alexey Milovidov
af8c8923cb
Merge branch 'master' into analyzer-fix-index-set
2024-01-13 04:15:10 +01:00
Alexey Milovidov
d112492c56
Remove some code
2024-01-13 03:48:04 +01:00
Artem Alperin
4139cb24ee
Add missing includes
2024-01-13 01:48:55 +03:00
Nikita Mikhaylov
8edab24189
Add comments to all columns of system tables ( #58356 )
2024-01-12 16:39:22 +01:00
Robert Schulze
b183e1f048
Merge remote-tracking branch 'rschu1ze/master' into Improvement/inverted_index_support_match
2024-01-12 14:10:11 +00:00
Robert Schulze
d11ed921be
Some fixups
2024-01-12 14:05:19 +00:00
Anton Popov
8e88605a7c
Merge remote-tracking branch 'upstream/master' into HEAD
2024-01-12 03:55:15 +00:00
Konstantin Bogdanov
633ef126c7
Merge pull request #57550 from ClickHouse/nickitat-patch-15
...
Small change in log message in MergeTreeDataMergerMutator
2024-01-12 02:02:10 +03:00
Anton Popov
eb6647d4e8
fix reading of offsets subcolumn from Nested
2024-01-11 21:55:22 +00:00
Robert Schulze
790f5890a0
Merge pull request #58611 from rschu1ze/qc-isolation
...
Improve isolation of query cache entries under re-created users or role switches
2024-01-11 17:30:24 +01:00
Nikolai Kochetov
f83cf1a6b7
Merge pull request #58638 from amosbird/fix-58620
...
Fix broken partition key analysis when doing projection optimization
2024-01-11 14:17:48 +01:00
Nikita Mikhaylov
baf4e5e894
Added comments for system tables ( #48350 )
2024-01-11 12:50:00 +01:00
Duc Canh Le
6331d8a6f2
Merge branch 'master' into final_no_copy
...
Resolve conflicts
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2024-01-11 02:55:14 +00:00
Igor Nikonov
8d4a395afa
Merge remote-tracking branch 'origin/master' into pr-custom-key-failover
2024-01-10 20:28:04 +00:00
Robert Schulze
52d75ab68f
Merge pull request #58458 from rschu1ze/replace-std_regexp-by-re2
...
Replace `std::regex` by re2
2024-01-10 17:46:54 +01:00
Kseniia Sumarokova
a6b5b7460b
Merge branch 'master' into broken-projections-better-handling
2024-01-10 16:46:18 +01:00
Dmitry Novik
ddbdd89904
Analyzer: Correctly handle constant set in index
2024-01-10 14:34:56 +00:00
Anton Popov
e87c62eead
Merge pull request #58628 from CurtizJ/remove-projections-code-2
...
Remove more projections code
2024-01-10 15:29:15 +01:00
Robert Schulze
fee98994d4
Fix tokenization in DataPartsExchange (it missed the last element)
2024-01-10 11:44:01 +00:00
Robert Schulze
499227b9cf
Merge remote-tracking branch 'rschu1ze/master' into replace-std_regexp-by-re2
2024-01-10 10:02:53 +00:00
Robert Schulze
36173bbb7b
Merge remote-tracking branch 'rschu1ze/master' into qc_isolation
2024-01-10 09:57:13 +00:00
Robert Schulze
4cacfa6b43
Merge pull request #57509 from jinjunzh/qatzstd_main
...
Add Intel-QAT-based `ZSTD_QAT` compression codec
2024-01-10 10:31:39 +01:00
Kseniia Sumarokova
1b517105db
Merge pull request #57323 from valbok/mysql-binlog-client
...
MaterializedMySQL: Introduce MySQL Binlog Client
2024-01-10 09:08:13 +01:00
Alexey Milovidov
4c1841e475
Merge branch 'master' into insert-quorum-host-name-check
2024-01-10 05:17:37 +01:00
Robert Schulze
ced9e93ac6
Merge remote-tracking branch 'rschu1ze/master' into qatzstd_main
2024-01-09 20:36:02 +00:00
Amos Bird
29afac61cc
Fix partition key analysis
2024-01-10 03:13:03 +08:00
pufit
6cf55b82f4
Merge pull request #58539 from canhld94/file_custom_compress_level
...
Allow explicitly set compression level in output format
2024-01-09 13:43:38 -05:00
Robert Schulze
c04e4eb162
Merge remote-tracking branch 'rschu1ze/master' into qc-isolation
2024-01-09 16:30:20 +00:00
Anton Popov
e0d469b0e0
remove more projections code
2024-01-09 15:48:08 +00:00
Dmitry Novik
629d4b921e
Fix style
2024-01-09 15:38:04 +00:00
Dmitry Novik
11280cbde1
Merge remote-tracking branch 'origin/master' into storage-merge-aliases-analyzer
2024-01-09 15:10:59 +00:00
Anton Popov
2fd0f7be7b
Merge pull request #55579 from CurtizJ/remove-old-projections-code
...
Remove more old code of projection analysis
2024-01-09 15:51:12 +01:00
Jayme Bird
b451c04e75
fixup wait for all replicas if no src_replicas are specified
2024-01-09 14:02:48 +00:00
Sema Checherinda
7c7e72c4b7
Merge pull request #58573 from ClickHouse/chesema-s3-client-creation
...
fix and test that S3Clients are reused
2024-01-09 13:28:20 +01:00
Robert Schulze
e9b6f413b8
Get rid of QueryCache::user_name
2024-01-09 12:24:12 +00:00
Igor Nikonov
f566423e19
Merge remote-tracking branch 'origin/master' into pr-custom-key-failover
2024-01-09 10:38:43 +00:00
Yakov Olkhovskiy
cfa41e7dc6
Merge pull request #58038 from ClickHouse/revert-58022-revert-57555-ft-dropped_tables_parts
...
Add system.dropped_tables_parts table - second attempt
2024-01-09 00:47:07 -05:00
Robert Schulze
fabc06995e
Improve isolation of query results in query cache
...
Fixes #58054
2024-01-08 20:52:30 +00:00
Igor Nikonov
7fd851b4e9
Merge remote-tracking branch 'origin/master' into pr-custom-key-failover
2024-01-08 20:43:20 +00:00
jinjunzh
cf53d9a17c
add USE_QAT into build_options
2024-01-08 11:05:36 -05:00
Anton Popov
3e933c9099
Merge remote-tracking branch 'upstream/master' into HEAD
2024-01-08 15:33:38 +00:00
Smita Kulkarni
f421e74402
Merge branch 'master' into Support_parameterized_view_with_analyzer
2024-01-08 14:10:31 +01:00
Jayme Bird
f67e283968
review fixup: simplify conditions and add meaningful comment on why we have these conditions
2024-01-08 10:02:09 +00:00
Jayme Bird
9734934d84
review fixup: don't pass in zookeeper
2024-01-08 10:02:09 +00:00
Jayme Bird
382f6404bf
fix getChildren from Keeper
2024-01-08 10:02:09 +00:00
Jayme Bird
5221b4f9bc
switch to contains to be more readable
2024-01-08 10:02:09 +00:00
Jayme Bird
18b7f1de5f
various fixups
2024-01-08 10:02:09 +00:00
Jayme Bird
b1580efffa
switch to use comma join seperated list of srcReplicas
2024-01-08 10:02:09 +00:00
Jayme Bird
128a421d84
add from <srcReplica> modified to SYSTEM SYNC REPLICA LIGHTWEIGHT
2024-01-08 10:02:09 +00:00
Raúl Marín
8f998eaa82
Merge pull request #58447 from canhld94/fix_symlink_in_user_files
...
Allow users to work with symlinks in user_files_path (again)
2024-01-08 10:40:36 +01:00
Robert Schulze
6665d23243
Merge pull request #58029 from skyoct/feat/server_settings
...
Add `changeable_without_restart` column to `system.server_settings`
2024-01-08 10:09:51 +01:00
Robert Schulze
c11a2fcb7f
Fix style
2024-01-08 08:50:34 +00:00
Duc Canh Le
149de9aa93
discard fs::canonical return
...
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2024-01-08 03:40:48 +00:00
Robert Schulze
f553b55e3a
Merge remote-tracking branch 'rschu1ze/master' into regex-std-re2
2024-01-07 22:31:35 +00:00
Robert Schulze
8e804487f3
Some fixups
2024-01-07 22:28:08 +00:00
Sema Checherinda
6f626d8294
fix auth_settings.hasUpdates function
2024-01-07 23:00:26 +01:00
Alexander Tokmakov
371d664667
Update MergeTreeData.cpp
2024-01-07 17:27:22 +01:00
Smita Kulkarni
16e682e0ae
Merge branch 'master' into Support_parameterized_view_with_analyzer
2024-01-07 16:38:01 +01:00
Robert Schulze
53965bb9f1
Merge remote-tracking branch 'rschu1ze/master' into qatzstd_main
2024-01-07 14:11:22 +00:00
Smita Kulkarni
356fc0aadb
Fix tests
2024-01-07 14:49:24 +01:00
Robert Schulze
e947ed710b
Hijack and document 'update' map to represent reload status
2024-01-07 13:09:27 +00:00
Sema Checherinda
d5f86f671d
fix and test that S3Clients reused
2024-01-07 02:19:06 +01:00
Anton Popov
8c9cf394b3
Merge remote-tracking branch 'upstream/master' into HEAD
2024-01-06 20:58:17 +00:00
Nikolai Kochetov
105789b936
Merge pull request #53705 from ClickHouse/analyzer-execution-names
...
Analyzer: always qualify execution names
2024-01-06 17:39:58 +01:00
Anton Popov
48a7402b34
fix build
2024-01-06 16:39:27 +00:00
Nikolai Kochetov
c2f00bc75a
Merge pull request #58480 from ClickHouse/try-to-remove-pk-analysis-on-ast
...
Check if I can remove KeyCondition analysis on AST.
2024-01-06 12:40:46 +01:00
Anton Popov
f252412915
Merge remote-tracking branch 'upstream/master' into HEAD
2024-01-06 03:08:52 +00:00
Alexey Milovidov
e24ec55451
Merge pull request #58482 from ClickHouse/fix_new_intersecting_parts
...
Fix a stupid case of intersecting parts
2024-01-05 21:53:21 +01:00
Alexander Tokmakov
f94f33c554
Merge pull request #57755 from azat/rmt/ALTER_METADATA-MERGE_PARTS-race
...
[RFC] Eliminate possible race between ALTER_METADATA and MERGE_PARTS
2024-01-05 17:11:57 +01:00
Sergei Trifonov
8d6b012223
Merge pull request #57877 from azat/merges-mutations-throttling
...
Add ability to throttle merges/mutations
2024-01-05 17:08:50 +01:00
Nikolai Kochetov
c6858f8666
Fixing tidy
2024-01-05 15:22:41 +00:00
Smita Kulkarni
4be844509e
Merge branch 'master' into Support_parameterized_view_with_analyzer
2024-01-05 13:58:07 +01:00
Nikolai Kochetov
8bcfa888a5
Fixing index hint
2024-01-05 11:50:09 +00:00
Nikolai Kochetov
eff6232418
Merge branch 'master' into try-to-remove-pk-analysis-on-ast
2024-01-05 10:54:46 +00:00
Nikolai Kochetov
15b66031bc
Update prepared sets.
2024-01-05 10:53:07 +00:00
Smita Kulkarni
2ee6893312
Updated to return container for getObjectsNamespace
2024-01-05 10:58:04 +01:00
Nikolai Kochetov
80c1e59d3f
Merge branch 'master' into analyzer-execution-names
2024-01-05 09:31:25 +00:00
sunny19930321
8c8f7c83ff
hive threadpool read orc failed
2024-01-05 15:58:17 +08:00
sunny19930321
f9b57bfb2c
fix: hive threadpool read orc failed
2024-01-05 14:36:28 +08:00
Robert Schulze
2166df0640
Merge pull request #57882 from lingtaolf/optimization/BF_support_rg
...
Utilize `ngrambf` and `tokenbf` indexes by function `match()`
2024-01-05 00:00:50 +01:00
Nikolai Kochetov
85c1bb80fd
Merge branch 'master' into hdfs-virtuals
2024-01-04 18:00:10 +00:00
Nikolai Kochetov
85ea5c1f82
Remove more code
2024-01-04 17:46:06 +00:00
Robert Schulze
f276a7f9d4
Merge remote-tracking branch 'ClickHouse/master' into optimization/BF_support_rg
2024-01-04 17:15:17 +00:00
Alexander Tokmakov
9f5015737b
fix a stupid case of intersecting parts
2024-01-04 17:45:42 +01:00
Nikolai Kochetov
494a32f4e4
Review fixes
2024-01-04 14:41:04 +00:00
Kruglov Pavel
7e6e835e2e
Merge pull request #57520 from Avogar/ignore-mv-with-dropped-target-table
...
Ignore MVs with dropped target table during pushing to views
2024-01-04 15:33:27 +01:00
Duc Canh Le
2e14cfb526
add settings for output compression level and window size
...
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2024-01-04 08:16:00 +00:00
Nikita Taranov
66d2db5283
New parallel replicas coordinator implementation ( #57968 )
2024-01-04 00:27:04 +01:00
Nikolai Kochetov
7a271f09ed
Check if I can remove KeyCondition analysis on AST.
2024-01-03 17:50:46 +00:00
Nikolai Kochetov
d06de83ac1
Fix KeyCondition for file/url/s3
2024-01-03 17:44:28 +00:00
János Benjamin Antal
81b5d8fddd
Delay reading from StorageKafka by creating a query plan step
2024-01-03 17:13:41 +00:00
Anton Popov
cf4604bfb6
fix build and tests
2024-01-03 16:59:13 +00:00
Anton Popov
ce1b978a2d
Merge remote-tracking branch 'upstream/master' into HEAD
2024-01-03 16:01:44 +00:00
Anton Popov
7dd128f90f
Revert "remove projection from StorageSnapshot"
...
This reverts commit a01acf5d2a
2024-01-03 16:00:50 +00:00
Anton Popov
6b6f395cc7
Merge pull request #57631 from Avogar/cache-for-reading-compact-subcolumns
...
Read column once while reading more that one subcolumn from it in Compact parts
2024-01-03 16:31:54 +01:00
Nikolai Kochetov
a3a24bdb40
Merge branch 'master' of github.com:ClickHouse/ClickHouse into hdfs-virtuals
2024-01-03 15:10:57 +00:00
Nikolai Kochetov
7834519212
Merge pull request #58255 from ClickHouse/filter-virtual-columns-storage-merge
...
Refactor StorageMerge virtual columns filtering.
2024-01-03 16:09:49 +01:00
Robert Schulze
8d4b519bb1
Replace std::regex by re2
2024-01-03 15:06:20 +00:00
Val Doroshchuk
5c221d123d
MaterializedMySQL: Introduce MySQL Binlog Client
...
One binlog connection for many databases.
Suggesting to disable this feature by default for now. It should be explicitly enabled by SETTINGS use_binlog_client=1.
But if you would permanently enable it in MaterializedMySQLSettings, it should keep old behavior and all tests should pass too.
1. Introduced `IBinlog` and its impl to read the binlog events from socket - `BinlogFromSocket`, or file - `BinlogFromFile`. Based on prev impl of `EventBase` and the same old binlog parsers. It fully keeps BC with old version. Fixed `./check-mysql-binlog` to test new impl.
2. Introduced `BinlogEventsDispatcher`, it reads the event from the source `IBinlog` and sends it to currently attached `IBinlog` instances.
3. Introduced `BinlogClient`, which is used to group a list of `BinlogEventsDispatcher` by MySQL binlog connection which is defined by `user:password@host:port`. All dispatchers with the same binlog position should be merged to one.
4. Introduced `BinlogClientFactory`, which is a singleton and it is used to track all binlogs created over the instance.
5. Introduced `use_binlog_client` setting to `MaterializedMySQL`, which forces to reuse a `BinlogClient` if it already exists in `BinlogClientCatalog` or create new one. By default, it is disabled.
6. Introduced `max_bytes_in_binlog_queue` setting to define the limit of bytes in binlog's queue of events. If bytes in the queue increases this limit, `BinlogEventsDispatcher` will stop reading new events from source `IBinlog` until the space for new events will be freed.
7. Introduced `max_milliseconds_to_wait_in_binlog_queue` setting to define max ms to wait when the max bytes exceeded.
7. Introduced `max_milliseconds_to_wait_in_binlog_queue` setting to define max ms to wait when the max bytes exceeded.
8. Introduced `max_bytes_in_binlog_dispatcher_buffer` setting to define max bytes in the binlog dispatcher's buffer before it is flushed to attached binlogs.
9. Introduced `max_flush_milliseconds_in_binlog_dispatcher` setting to define max milliseconds in the binlog dispatcher's buffer to wait before it is flushed to attached binlogs.
10. Introduced `system.mysql_binlogs` system table, which shows a list of active binlogs.
11. Introduced `UnparsedRowsEvent` and `MYSQL_UNPARSED_ROWS_EVENT`, which defines that an event is not parsed and should be explicitly parsed later.
12. Fixed bug when not possible to apply DDL since syntax error or unsupported SQL.
@larspars is the author of following:
`GTIDSets::contains()`
`ReplicationHelper`
`shouldReconnectOnException()`
2024-01-03 15:26:09 +01:00
Nikolai Kochetov
9dc1f4d99c
Update StorageMerge.cpp
2024-01-03 14:53:40 +01:00
Kruglov Pavel
c8acc7c2d1
Fix build
2024-01-03 14:44:00 +01:00
Nikolai Kochetov
aee933a437
Merge branch 'master' into hdfs-virtuals
2024-01-03 12:16:57 +00:00
Nikolai Kochetov
d5dcb6661d
Review fixes.
2024-01-03 10:12:08 +00:00
Nikolai Kochetov
78776a060c
Update src/Storages/StorageMerge.cpp
...
Co-authored-by: Dmitry Novik <n0vik@clickhouse.com>
2024-01-03 11:06:57 +01:00
Nikolai Kochetov
eeed23b1bc
Fix sanitizer assert.
2024-01-03 09:45:25 +00:00
Duc Canh Le
12fda5f309
fix 02771_multidirectory_globs_storage_file
...
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2024-01-03 09:22:29 +00:00
Duc Canh Le
3502245ecb
use fs::absolute instead of fs::canonical
...
To allow users to work with symlinks in user_files_path
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2024-01-03 04:58:26 +00:00
Alexey Milovidov
44432b1ffc
Fix build
2024-01-02 21:54:13 +01:00
Alexey Milovidov
deac0b5f3e
Merge branch 'master' into insert-quorum-host-name-check
2024-01-02 21:46:12 +01:00
avogar
488e36de46
More fixes
2024-01-02 20:20:39 +00:00
Smita Kulkarni
4122de9721
Updated tests and added settings
2024-01-02 20:19:01 +01:00
Nikolai Kochetov
9c25cb6692
Cleanup
2024-01-02 18:08:04 +00:00
Nikolai Kochetov
4f99a8bc1f
Remove more unused code.
2024-01-02 17:54:20 +00:00
Nikolai Kochetov
1b20ce5162
Cleanup
2024-01-02 17:50:06 +00:00
Nikolai Kochetov
c808b03e55
Remove unneeded code
2024-01-02 17:27:33 +00:00
Nikolai Kochetov
8936c8376a
Use predicate in getTaskIteratorExtension.
2024-01-02 17:14:16 +00:00
avogar
c5ba97f0cd
Fix tests
2024-01-02 16:38:45 +00:00
Kruglov Pavel
f2dfe8bdda
Fix build
2024-01-02 16:42:17 +01:00
Nikolai Kochetov
3e3fed1cbe
Add reading step to URL
2024-01-02 15:18:13 +00:00
Nikolai Kochetov
0f76967f97
Add reading step to Azure.
2024-01-02 13:46:08 +00:00
Alexander Tokmakov
d13abac912
Merge pull request #58333 from ClickHouse/fix_no_such_key_detached_part
...
Fix lost blobs after dropping a replica with broken detached parts
2024-01-02 14:21:55 +01:00
Nikolai Kochetov
b7cc6d4615
Fixing tests.
2024-01-02 13:08:04 +00:00
Duc Canh Le
d623702378
Merge branch 'master' into final_no_copy
...
Resolve conflicts
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2024-01-02 06:07:49 +00:00
Bharat Nallan Chakravarthy
00b5e28d6a
Merge upstream/master into ncb/system-database-engines
2023-12-30 23:10:48 -08:00
Bharat Nallan Chakravarthy
3c4ac0ba08
add system.database_engines table
2023-12-30 20:31:25 -08:00
Alexey Milovidov
7bded0a5e7
Merge pull request #58316 from ClickHouse/reintroduce_is_deleted
...
Re-introduce `is_deleted` column for ReplacingMergeTree
2023-12-31 00:57:19 +01:00
Alexey Milovidov
063463b0a8
Merge pull request #58359 from ClickHouse/all-system-tables-local
...
Attach all system tables in `clickhouse-local`
2023-12-30 17:51:27 +01:00
Alexey Milovidov
c9467d8d47
Update ReplicatedMergeTreeSink.cpp
2023-12-30 15:19:50 +03:00
Alexey Milovidov
2421173574
Merge branch 'master' into insert-quorum-host-name-check
2023-12-30 13:13:24 +01:00
Alexey Milovidov
8fc05e25fe
Merge pull request #58310 from azat/kafka-fix-stat-leak-resubmit
...
Create consumers for Kafka tables on fly with TTL (resubmit)
2023-12-30 13:03:16 +01:00
Alexey Milovidov
e1812f3b58
Merge pull request #58266 from ClickHouse/vdimir/simple_fix_tuple_elimination
...
Analyzer: fix tuple comparison when result is always null
2023-12-30 13:02:38 +01:00
Alexey Milovidov
aa6ecd2d59
Merge pull request #58343 from azat/s3/optional-gcs-compose
...
Avoid sending ComposeObject requests after upload to GCS
2023-12-30 12:40:04 +01:00
Alexey Milovidov
40ca9c202d
Merge pull request #58346 from ClickHouse/check-what-would-be-ifremove-array-joined-columns-from-key-condition
...
Check what happen if remove array joined columns from KeyCondition
2023-12-30 12:38:57 +01:00
Alexey Milovidov
f058394d92
Merge pull request #58351 from ClickHouse/fix_00002
...
Keep exception format string in retries ctl
2023-12-30 12:37:36 +01:00
Alexey Milovidov
39b239683c
Attach all system tables in clickhouse-local
2023-12-29 21:25:22 +01:00
Nikolai Kochetov
b95bdef09e
Update StorageS3 and StorageS3Cluster
2023-12-29 17:41:11 +00:00
Kruglov Pavel
f57939096c
Merge branch 'master' into ignore-mv-with-dropped-target-table
2023-12-29 17:02:23 +01:00
Nikolai Kochetov
5521e5d9b1
Refactor StorageHDFS and StorageFile virtual columns filtering
2023-12-29 15:58:01 +00:00
robot-ch-test-poll3
07ba672e37
Merge pull request #58142 from canhld94/final_less_compare
...
MergeTree FINAL to not compare rows from same non-L0 part
2023-12-29 16:47:14 +01:00
Azat Khuzhin
a12df35be4
Eliminate possible race between ALTER_METADATA and MERGE_PARTS
...
v2: move metadata version check after checking that the part is not covering part
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-29 16:46:10 +01:00
Azat Khuzhin
c7fa93d704
Add infrastructure for testing replicated MergeTree queue
...
- replicated_queue_fail_next_entry - to fail next queue entry
- replicated_queue_unfail_entries - to "unfail" all queue entries (if
any)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-29 16:43:01 +01:00
Alexander Tokmakov
1013f6b23f
Merge branch 'master' into reintroduce_is_deleted
2023-12-29 15:46:24 +01:00
Alexander Tokmakov
72a0797b88
keep exception format string in retries ctl
2023-12-29 15:21:46 +01:00
Alexey Milovidov
ea03cc82aa
Merge pull request #58320 from ClickHouse/mv3
...
Refreshable materialized views again
2023-12-29 14:44:50 +01:00
Azat Khuzhin
853fdfe775
Clean cached messages on destroy kafka consumer
...
The callchain of the kafka consumer is very tricky, so for the sake of
common sense let's just clean the messages on moving out consumer (and
in dtor, but this is just to keep that two code path in sync).
(Also reported by @filimonov)
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-29 14:30:21 +01:00
Azat Khuzhin
b3d6caf37f
Unsubscribe kafka consumer before cleaning it by TTL
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-29 14:03:53 +01:00
Nikolai Kochetov
0e8232a8c3
Check what happen if remove array joined columns from KeyCondition
2023-12-29 12:24:19 +00:00
Azat Khuzhin
f578541ded
Fix destructing kafka consumer via member orders
...
We've discussed this with @filimonov and he pointed out that everything
else (except for rdkafka_stat/rdkafka_stat_mutex) is done via members
orders, so let's do it in the same style.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-29 13:19:11 +01:00
Azat Khuzhin
8c54380d80
Avoid sending ComposeObject requests after upload to GCS
...
This should not be required anymore, but leave it as an option, since
likely this is required for old files.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-29 11:53:49 +01:00
Azat Khuzhin
f4a7789cd4
Convert various S3::Client settings into separate ClientSettings struct
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2023-12-29 11:53:49 +01:00
Duc Canh Le
91a87d6b6c
better implementation
...
Signed-off-by: Duc Canh Le <duccanh.le@ahrefs.com>
2023-12-29 07:27:10 +00:00
Igor Nikonov
208a9193f6
Merge remote-tracking branch 'origin/master' into pr-custom-key-failover
2023-12-28 21:28:36 +00:00
Alexander Tokmakov
852f397a97
fix lost blobs after dropping a replica with broken detached parts
2023-12-28 21:47:19 +01:00
Kruglov Pavel
fbd3f7cd59
Merge pull request #56132 from Avogar/flatten-only-true-nested
...
Flatten only true Nested type if flatten_nested=1, not all Array(Tuple)
2023-12-28 20:58:28 +01:00
Michael Kolupaev
c4f4516a37
Fix WriteBuffer assert if refresh is cancelled at the wrong moment
2023-12-28 18:34:28 +00:00
Nikolai Kochetov
490a8bce9e
Remove commented code.
2023-12-28 18:01:08 +00:00
Michael Kolupaev
ea138fe8c9
space
2023-12-28 17:56:06 +00:00
Michael Kolupaev
96c68e5aae
Remove pausing, enable multithreading, kick off refresh on table creation unless the query says EMPTY
2023-12-28 17:56:06 +00:00
Michael Kolupaev
4d732cdf1e
Add to system.process, improve test slightly
2023-12-28 17:56:05 +00:00
Michael Kolupaev
f0417d0ec3
Things
2023-12-28 17:56:05 +00:00
Michael Kolupaev
0fc7535eba
Fixes
2023-12-28 17:56:05 +00:00
Michael Kolupaev
609b2c216c
Fix some of the CI
2023-12-28 17:56:05 +00:00
Michael Kolupaev
8b8ef41407
Documentation
2023-12-28 17:56:05 +00:00
Michael Kolupaev
64e6deb197
Slightly more things
2023-12-28 17:56:05 +00:00
Michael Kolupaev
dda0606f67
Things
2023-12-28 17:56:05 +00:00
Michael Kolupaev
98dbd105ad
Overhaul timestamp arithmetic
2023-12-28 17:56:04 +00:00
Michael Kolupaev
a524e8c51e
Overhaul dependencies
2023-12-28 17:56:04 +00:00
