alesapin
6f059be748
Fix bug with temporary projection parts deletion
2022-11-08 15:37:03 +01:00
serxa
2de26daa56
fix build
2022-11-08 14:31:29 +00:00
SmitaRKulkarni
658036d179
Merge pull request #42708 from ClickHouse/42414_Enable_functions_in_readonly_mode
...
42414 enable functions in readonly mode
2022-11-08 15:22:31 +01:00
Alexander Tokmakov
eb19df0094
fix
2022-11-08 15:21:25 +01:00
Sergei Trifonov
8eedd1e046
Merge branch 'master' into disk-s3-throttler
2022-11-08 15:00:56 +01:00
SmitaRKulkarni
4f2e5596e3
Merge pull request #42933 from yakkomajuri/patch-2
...
fix(typo): Passible -> Possible
2022-11-08 14:54:50 +01:00
serxa
6d5d9ff421
rename ReadWriteSettings -> RequestSettings
2022-11-08 13:48:23 +00:00
Maksim Kita
0aa4fd3c2d
Fixed tests
2022-11-08 14:29:17 +01:00
Kseniia Sumarokova
d2b777c059
Merge pull request #41054 from rubin-do/hudi
...
Add Hudi and DeltaLake table engines
2022-11-08 12:03:06 +01:00
Kseniia Sumarokova
92b5ca326c
Merge pull request #42303 from xiedeyantu/system_table_add_column
...
system.detached_parts add column bytes_on_disk and path
2022-11-08 12:00:56 +01:00
Robert Schulze
2c568df714
Merge pull request #42968 from ClickHouse/bitcast
...
Less use of CH-specific bit_cast()
2022-11-08 10:54:10 +01:00
Robert Schulze
bb507356ef
Merge pull request #42209 from canhld94/ch_canh_fix_prefix_not_like
...
Add `notLike` to key condition atom map
2022-11-08 10:07:21 +01:00
Antonio Andelic
7067f9e0e1
Merge pull request #42947 from azat/mergetree/part-load-tiny
...
Attach thread pool for loading parts to the query
2022-11-08 09:17:32 +01:00
zhongyuankai
37099e57f4
Merge branch 'master' into optimze_ttl_merge
2022-11-08 12:26:46 +08:00
Azat Khuzhin
2bd4a71150
Merge branch 'master' into fetch-remove-covered
2022-11-08 00:55:00 +01:00
Kseniia Sumarokova
4fc7926c16
Merge branch 'master' into system_table_add_column
2022-11-07 23:36:57 +01:00
Kseniia Sumarokova
d695240d53
Update StorageHudi.cpp
2022-11-07 23:07:00 +01:00
alesapin
257e766e0e
Trying to fix two flaky tests
2022-11-07 20:49:41 +01:00
Alexander Tokmakov
9210e586d2
fix
2022-11-07 20:27:18 +01:00
Alexander Tokmakov
b8174a63a8
Merge branch 'master' into fix_intersecting_parts2
2022-11-07 20:26:36 +01:00
serxa
2daec0b45e
S3 request per second rate throttling + refactoring
2022-11-07 18:05:40 +00:00
xiedeyantu
d4cdc38b71
better
2022-11-07 22:49:58 +08:00
Kseniia Sumarokova
891e167509
Update StorageHudi.cpp
2022-11-07 15:17:16 +01:00
Kruglov Pavel
48163ff9c2
Merge pull request #42938 from amosbird/issue_42771
...
Fix NOT_FOUND_COLUMN_IN_BLOCK in some rare case
2022-11-07 13:10:12 +01:00
kssenii
897811371d
Minor changes
2022-11-07 12:45:26 +01:00
kssenii
819be6bb32
Merge remote-tracking branch 'upstream/master' into hudi
2022-11-07 11:30:53 +01:00
Duc Canh Le
bd2bd7149d
fix typo
2022-11-07 18:22:50 +08:00
SmitaRKulkarni
c563e124c2
Merge branch 'master' into 42414_Enable_functions_in_readonly_mode
2022-11-07 09:50:05 +01:00
Duc Canh Le
f7e65ebd72
make comments more clear
2022-11-07 16:25:36 +08:00
Robert Schulze
cdaf0becfe
Merge branch 'master' into bitcast
2022-11-07 09:24:52 +01:00
Robert Schulze
f3f50530d1
Merge branch 'master' into ch_canh_fix_prefix_not_like
2022-11-07 09:14:02 +01:00
Antonio Andelic
e08f94d0f8
Merge pull request #42659 from ClickHouse/optimize-entire-partition
...
Support for optimizing old parts for entire partition only
2022-11-07 09:06:11 +01:00
Duc Canh Le
69d6d42443
NOT LIKE only work for 'prefix%'
2022-11-06 15:26:19 +08:00
zhongyuankai
ef6065b331
fix test
2022-11-06 13:40:26 +08:00
Duc Canh Le
a8bce28c04
Merge branch 'master' into ch_canh_fix_prefix_not_like
2022-11-06 13:04:08 +08:00
Igor Nikonov
e94b9cd003
Fixes
...
+ enable retries in integration test
+ typo in fault injection traces
2022-11-05 22:28:32 +00:00
Igor Nikonov
859022660e
Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry
2022-11-04 23:31:51 +00:00
Daniil Rubin
7500cc38c0
Apply clang-format
2022-11-04 20:51:44 +00:00
Daniil Rubin
37048e986a
Add comments, fix review issues
2022-11-04 20:49:24 +00:00
Daniil Rubin
d6bbd65dde
Fix broken include
2022-11-04 16:03:12 +00:00
Robert Schulze
9c066e964d
Less use of CH-specific bit_cast()
...
Converted usage of CH-custom bit_cast to std::bit_cast if possible, i.e.
when
sizeof(From) == sizeof(To).
(The CH-custom bit_cast is able to deal with sizeof(From) != sizeof(To).)
Motivation for this came from #42847 where it is not clear how the
internal bit_cast should behave on big endian systems, so we better
avoid that situation as much as possible.
2022-11-04 15:52:48 +00:00
Vladimir C
f359411668
Merge branch 'master' into optimze_ttl_merge
2022-11-04 16:52:20 +01:00
Vitaly Baranov
52b1f4aed9
Merge pull request #42484 from vitlibar/mask-sensitive-info-in-logs
...
Mask some information in logs
2022-11-04 14:09:38 +01:00
zhongyuankai
1345f7e73d
Merge branch 'master' into optimze_ttl_merge
2022-11-04 20:54:46 +08:00
alesapin
82729a88c8
Merge pull request #42833 from vitlibar/backup-s3-increase-upload-part-size-exponentially
...
Increase the size of upload part exponentially for backup to S3.
2022-11-04 13:41:10 +01:00
Kseniia Sumarokova
ed44b20694
Merge pull request #42224 from kssenii/fit-http-buffer-retries
...
Fix http buffer retries
2022-11-04 11:50:17 +01:00
Amos Bird
d901ead1bc
Done
2022-11-04 17:26:44 +08:00
zhongyuankai
049a593272
Merge branch 'master' into optimze_ttl_merge
2022-11-04 17:22:38 +08:00
zhongyuankai
e773eb2fd1
batter
2022-11-04 17:19:34 +08:00
Azat Khuzhin
7032b2b98a
Attach thread pool for loading parts to the query
...
Detach should be done via CurrentThread::defaultThreadDeleter()
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-04 09:06:12 +01:00
Azat Khuzhin
cdd09fd72a
Improve messages for loading data parts (add part type)
...
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-11-04 09:05:06 +01:00
Daniil Rubin
99d57325bb
Merge branch 'master' into hudi
2022-11-03 19:29:23 +00:00
Daniil Rubin
a573d8aef7
Apply clang-format
2022-11-03 18:17:39 +00:00
Daniil Rubin
7b57e94d9f
Fix StorageHudi
2022-11-03 18:14:10 +00:00
Daniil Rubin
a9f8948c8d
Fix build
2022-11-03 17:28:41 +00:00
Amos Bird
7186898ffa
Experiment
2022-11-04 00:29:17 +08:00
Yakko Majuri
e68a3b0c7a
fix(typo): Passible -> Possible
2022-11-03 11:25:07 -03:00
kssenii
4fe4a07600
Add test
2022-11-03 12:14:08 +01:00
Antonio Andelic
66ffdda973
Merge branch 'master' into optimize-entire-partition
2022-11-03 09:52:13 +01:00
Alexander Tokmakov
e67d05604c
Merge branch 'master' into fix_intersecting_parts2
2022-11-02 19:30:27 +01:00
Alexander Tokmakov
d8b3a2af84
fix race between INSERT and DROP
2022-11-02 19:26:50 +01:00
Vitaly Baranov
32194c1200
Add max limitation for the size of an uploaded part.
2022-11-02 17:53:54 +01:00
Igor Nikonov
9a315b7899
Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry
2022-11-02 15:32:54 +00:00
Vitaly Baranov
e0133688bc
Merge branch 'master' into mask-sensitive-info-in-logs
2022-11-02 16:26:13 +01:00
Alexander Tokmakov
00c9e50ee3
fix race between INSERT and ALTER PARTITION
2022-11-02 16:25:26 +01:00
Alexander Tokmakov
acbad99b24
remove abandonable_lock part 2
2022-11-02 14:28:49 +01:00
Antonio Andelic
c4b717b343
Update src/Storages/MergeTree/MergeTreeDataMergerMutator.cpp
...
Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>
2022-11-02 08:24:54 +01:00
Duc Canh Le
ec382d7903
add comment
2022-11-02 09:50:41 +08:00
Duc Canh Le
c6598dc66d
Merge branch 'master' into ch_canh_fix_prefix_not_like
2022-11-02 09:42:01 +08:00
Alexander Tokmakov
4f50a999bb
remove abandonable_lock part 1
2022-11-01 21:35:25 +01:00
Nikolai Kochetov
478d3079c8
Merge branch 'master' into read-in-order-from-query-plan
2022-11-01 21:02:22 +01:00
Nikolai Kochetov
9ffebf4a0c
Fixing read-in-order for special storages.
2022-11-01 19:51:52 +00:00
Igor Nikonov
e76c3c381f
Better handling tables in read-only mode during insert
...
+ throw immediately if table has no metadata in zk
+ stop retries in case of shutdown
+ check if table is readonly at the begining of every retry
2022-11-01 18:47:51 +00:00
Anton Popov
88674ef964
Merge pull request #42794 from CurtizJ/better-code-in-check-part
...
Slightly better code with projections
2022-11-01 16:05:27 +01:00
zhongyuankai
c1e2b2af74
Optimize TTL merge, completely expired parts can be removed in time
2022-11-01 22:58:15 +08:00
Anton Popov
876dca48da
Merge pull request #36969 from CurtizJ/dynamic-columns-14
...
Support `Object` type inside other types
2022-11-01 15:20:02 +01:00
Duc Canh Le
07d9720ed5
Merge branch 'master' into ch_canh_fix_prefix_not_like
2022-11-01 22:16:09 +08:00
Kruglov Pavel
21d50f76ea
Merge pull request #41979 from Avogar/s3-cluster-schema-inference
...
Fix schema inference in s3Cluster and improve in hdfsCluster
2022-11-01 14:00:21 +01:00
Vitaly Baranov
b9f2f17331
Add test and logging.
2022-11-01 12:23:20 +01:00
Maksim Kita
fe48a1ce7e
Merge pull request #42681 from kitaisreal/merge-tree-indexes-use-actions-dag
...
MergeTree indexes use ActionsDAG
2022-11-01 13:55:17 +03:00
Duc Canh Le
3545db0a6c
Merge branch 'master' into ch_canh_fix_prefix_not_like
2022-11-01 16:54:26 +08:00
Yakov Olkhovskiy
bab0e06e3d
Merge pull request #42458 from Clark0/match-func-use-index
...
match function can use index if it's a condition on string prefix #37333
2022-10-31 17:33:58 -04:00
Igor Nikonov
9cc64a0fcc
Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry
2022-10-31 18:21:21 +00:00
Igor Nikonov
2d5050ea66
Fix review comments
2022-10-31 18:20:58 +00:00
Anton Popov
2ae3cfa9e0
Merge branch 'master' into dynamic-columns-14
2022-10-31 16:15:19 +01:00
Alexander Tokmakov
599ccb9939
Update MergeTreeData.h
2022-10-31 16:18:17 +03:00
Alexander Tokmakov
b2b9479afa
Update src/Storages/MergeTree/MergeTreeData.h
...
Co-authored-by: Sergei Trifonov <sergei@clickhouse.com>
2022-10-31 16:15:35 +03:00
SmitaRKulkarni
54035270b2
Merge branch 'master' into 42414_Enable_functions_in_readonly_mode
2022-10-31 14:15:29 +01:00
Maksim Kita
bca22ec5f5
Fixed code review issues
2022-10-31 12:30:00 +01:00
Maksim Kita
115fcaffc5
Fixed tests
2022-10-31 11:30:56 +01:00
Vitaly Baranov
5d2a222fe4
Mask sensitive information in logs.
2022-10-31 10:50:33 +01:00
Duc Canh Le
d309e8793a
Merge branch 'master' into ch_canh_fix_prefix_not_like
2022-10-31 15:13:39 +08:00
Alexander Tokmakov
f53df7870c
fix race between drop and failed insert
2022-10-30 17:59:47 +01:00
Alexander Tokmakov
c68ab231f9
fix accessing part in Deleting state
2022-10-30 17:30:51 +01:00
Anton Popov
8de75bf29c
better code with projections
2022-10-29 14:39:59 +00:00
clarkcaoliu
ca5dbe88db
match function can use index if prefix
2022-10-29 18:25:29 +08:00
Igor Nikonov
d09deaf10f
Fix style check
2022-10-28 15:17:05 +00:00
Igor Nikonov
937dd44399
Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry
2022-10-28 15:09:23 +00:00
Igor Nikonov
c53b96a11c
Add more comments
2022-10-28 15:09:01 +00:00
Alexey Milovidov
f64e632707
Merge pull request #42469 from amosbird/issue_42456
...
Fix misbehavior of key analysis
2022-10-28 16:18:00 +02:00
Antonio Andelic
5fae8b0cc9
Merge branch 'master' into optimize-entire-partition
2022-10-28 11:44:10 +00:00
Duc Canh Le
848f5b5d65
Merge branch 'master' into ch_canh_fix_prefix_not_like
2022-10-28 13:40:17 +08:00
Duc Canh Le
c080964391
correct 'notLike' in key condition
2022-10-28 13:39:01 +08:00
Duc Canh Le
69311bd5cd
Merge branch 'master' into ch_canh_fix_prefix_not_like
2022-10-28 11:59:50 +08:00
Igor Nikonov
3682913172
Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry
2022-10-27 22:01:53 +00:00
Igor Nikonov
a601c166ad
Add clean up for faults injected before zk request
...
+ for multi-op request, it create responses in multi response
+ some polishing
2022-10-27 21:35:13 +00:00
Anton Popov
7715afa595
Merge pull request #42618 from CurtizJ/refactor-data-part
...
Try to save `IDataPartStorage` interface
2022-10-27 22:20:44 +02:00
Igor Nikonov
f773436de5
Fix flaky test
2022-10-27 13:34:07 +00:00
Nikolai Kochetov
24e1aaf324
Merge pull request #42576 from ClickHouse/table-function-merge-invalid-where-type
...
Invalid type in where for Merge table (logical error)
2022-10-27 13:03:27 +02:00
Antonio Andelic
10aa7091ca
Merge pull request #42605 from ClickHouse/correctly-handle-prewhere-type
...
Correctly handle non-native integers in PREWHERE
2022-10-27 09:25:38 +02:00
SmitaRKulkarni
e860ecc80c
Merge branch 'master' into 42414_Enable_functions_in_readonly_mode
2022-10-27 08:26:44 +02:00
Nikolai Kochetov
313cbb1058
Fix test.
2022-10-26 17:49:36 +00:00
Igor Nikonov
649e8d302c
Sync with retiring branch igor/insert_zk_retries
2022-10-26 17:41:44 +00:00
Igor Nikonov
8435cba352
Merge remote-tracking branch 'origin/master' into igor/insert_zk_retries_retry
2022-10-26 17:35:47 +00:00
Maksim Kita
f04f663818
Fixed tests
2022-10-26 18:55:14 +02:00
Smita Kulkarni
6a8fa2d4a5
Added new field allow_readonly in system.table_functions to allow using table functions in readonly mode.
...
Implementation:
* Added a new field allow_readonly to table system.table_functions.
* Updated to use new field allow_readonly to allow using table functions in readonly mode.
* Added TableFunctionProperties similar to AggregateFunctionProperties.
* The functions allowed in readonly mode are in this set table_functions_allowed_in_readonly_mode.
Testing:
* Added a test for filesystem tests/queries/0_stateless/02473_functions_in_readonly_mode.sh
Documentation:
* Updated the english documentation for Table Functions.
2022-10-26 18:45:23 +02:00
Nikolai Kochetov
1520bcd53c
Fix error code.
2022-10-26 15:50:10 +00:00
Anton Popov
ca4df62e37
Merge branch 'master' into refactor-data-part
2022-10-26 17:49:20 +02:00
Mikhail f. Shiryaev
fdecac65b8
Merge pull request #42699 from ClickHouse/22.11-prepare
...
Update version after release
2022-10-26 16:31:48 +02:00
Maksim Kita
1c17e9d454
Fixed tests
2022-10-26 15:54:37 +02:00
Mikhail f. Shiryaev
d3461abd1f
Update version to 22.11.1.1
2022-10-26 15:27:09 +02:00
Nikolai Kochetov
db422fa903
Merge pull request #42410 from ClickHouse/fix-multi-arg-monotonicity
...
Fix possible LOGICAL_ERROR in binary ariphmetics monotonicity.
2022-10-26 14:11:23 +02:00
Kruglov Pavel
219553df8d
Merge pull request #42580 from hanfei1991/fei/limit_max_rows
...
fix behaviour of max_rows_to_read for trival limit queries
2022-10-26 14:09:59 +02:00
Maksim Kita
c47ca522f4
Fixed style check
2022-10-26 13:27:14 +02:00
Antonio Andelic
35a9672704
Merge branch 'master' into optimize-entire-partition
2022-10-26 11:09:25 +00:00
Maksim Kita
1b6293f6db
MergeTree indexes use ActionsDAG
2022-10-26 12:44:37 +02:00
Antonio Andelic
97aaebfa18
Address PR comments
2022-10-26 10:06:56 +00:00
SmitaRKulkarni
5df138da82
Merge pull request #42376 from ClickHouse/36316_Support_glob_for_recursive_directory_traversal
...
Added ** glob support for recursive directory traversal to filesystem…
2022-10-26 11:49:20 +02:00
Antonio Andelic
979d180931
Merge branch 'master' into correctly-handle-prewhere-type
2022-10-26 11:13:16 +02:00
Azat Khuzhin
8db31beee6
Remove covered parts for fetched part
...
Here is an example that I found on production, simplified.
Consider the following queue (nothing of this had been processed on this
replica):
- GET_PART all_0_0_0 (queue-0000000001)
- GET_PART all_1_1_0 (queue-0000000002)
...
- GET_PART all_0_1_1 (queue-0000000003)
- GET_PART all_2_2_0 (queue-0000000004)
...
- MERGE_PARTS from [all_0_1_1, all_2_2_0] to all_0_2_2 (queue-0000000005)
And now queue-0000000005 started to executing (either because
of reording, or because at that time GET_PART fails), and it
does not have any required parts, so it will fetch them, but
not all_0_0_0 and all_1_1_0, so this replica delay will set to
the time of min(queue-0000000001, queue-0000000002), while it
is not true, since it already have parts that covers those
parts.
and since MERGE_PARTS takes 30min, it increased the replica delay
eventually to 30min, for the time range of 30min, which is pretty huge.
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2022-10-26 11:10:50 +03:00
Anton Popov
79889198b1
fix style check
2022-10-25 23:03:37 +00:00
Anton Popov
d8b09b430f
add comments
2022-10-25 22:27:11 +00:00
Anton Popov
c4d4f2dbed
better interface
2022-10-25 22:14:06 +00:00
Nikolai Kochetov
bfb5e6eae4
Fix logical error for invalid prewhere type in StorageMerge.
2022-10-25 13:58:29 +00:00
Antonio Andelic
611c2e2bd7
Support for optimizing old parts for entire partition only
2022-10-25 13:40:18 +00:00
Anton Popov
c8199bc125
Merge branch 'master' into refactor-data-part
2022-10-25 14:31:05 +02:00
Raúl Marín
0016bc2b73
Improve the time to recover keeper connections ( #42541 )
...
Co-authored-by: Nikita Mikhaylov <nikitamikhaylov@clickhouse.com>
Co-authored-by: robot-clickhouse <robot-clickhouse@clickhouse.com>
Co-authored-by: Alexey Milovidov <milovidov@clickhouse.com>
2022-10-25 13:26:29 +02:00
Antonio Andelic
af0c54e055
Small fix
2022-10-25 11:13:41 +00:00
Antonio Andelic
451a50d9e9
Merge branch 'master' into correctly-handle-prewhere-type
2022-10-25 10:48:35 +00:00
Maksim Kita
06fe6f3c8b
Merge pull request #31796 from kitaisreal/identifier-resolver
...
Added Analyzer
2022-10-25 12:36:08 +03:00
Anton Popov
cbfa887e52
fix merge of projections
2022-10-24 23:51:57 +00:00
Anton Popov
f865b28da4
fix drop of MergeTree tables
2022-10-24 22:38:53 +00:00
Alexey Milovidov
9ee7131f67
Merge pull request #42423 from ClickHouse/optimize-partitions-in-background
...
Optimize partitions in background
2022-10-24 19:41:50 +02:00
SmitaRKulkarni
96c8260230
Merge branch 'master' into 36316_Support_glob_for_recursive_directory_traversal
2022-10-24 18:34:19 +02:00
Anton Popov
dcb7667808
share transaction for projections
2022-10-24 16:28:19 +00:00
Anton Popov
128fdd785e
better interface
2022-10-24 14:44:22 +00:00
Antonio Andelic
5579d139eb
Avoid crash with big int in prewhere
2022-10-24 08:37:52 +00:00
Maksim Kita
d083200d65
Fixed code review issues
2022-10-24 10:22:38 +02:00
Maksim Kita
10438aea01
Added JOIN support for different algorithm
2022-10-24 10:22:21 +02:00
Maksim Kita
890d9678b9
Fixed tests
2022-10-24 10:22:20 +02:00
