Commit Graph

11450 Commits

Author SHA1 Message Date
vdimir
0a5bc248fd
Merge branch 'master' into vdimir/join_select_inner_table 2024-11-29 15:50:25 +00:00
Mikhail Artemenko
b81ee27750
Merge pull request #71406 from azat/automatic-external-aggregation
Automatic GROUP/ORDER BY to disk based on the memory usage
2024-11-29 12:27:43 +00:00
Alexey Katsman
d7167bd103
Merge pull request #72548 from alexkats/dict
Add source query validation for dictionaries
2024-11-29 11:25:40 +00:00
alesapin
d25b15a3b1
Merge pull request #72613 from ClickHouse/nopctest
Remove flaky test_page_cache
2024-11-29 10:16:13 +00:00
Michael Kolupaev
29b722df29 Remove flaky test_page_cache 2024-11-28 21:09:12 +00:00
Pablo Marcos
d8a235b785
Merge pull request #67733 from Zawa-ll/66073-system-load-primary-key-rebase
Added statement `SYSTEM LOAD PRIMARY KEY`
2024-11-28 16:02:31 +00:00
Alex Katsman
23086e1424 Add source query validation for dictionaries 2024-11-28 15:38:07 +00:00
Azat Khuzhin
bdb39a6fef Change logic of automatic ORDER/GROUP BY to be based on available memory
- remove max_bytes_ratio_before_external_{order,group_by}_for_server
- change the way max_bytes_ratio_before_external_{order,group_by} works

Note, that it is not enough to transform ratio to bytes in
executeQuery(), since in this case it will not work for merges and
internal queries, plus, you have to reset them for Distributed engine
and update it for Merge/View/...

This patch also introduce some helpers (see MemoryTrackerUtils) and
adjust Aggregator::Params constructor to accept Settings object instead
of tons of arguments.

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-11-28 16:01:53 +01:00
Azat Khuzhin
db1f459cc4 tests: disable test_max_bytes_ratio_before_external_order_group_by_for_server under TSan
The problem is that TSan has some memory overhead which increases
process RSS to 10Gi in the middle of the query and it will fail.
Another option is to avoid syncing with RSS too frequently, but I doubt
that it is significant to run this test under TSan

v1: increase memory to 20Gi
    Fixes: https://s3.amazonaws.com/clickhouse-test-reports/71406/b33be86dee3c7616e9193c339837b9e250810557/integration_tests__tsan__[6_6].html
v2: instead just disable under TSan
    Fixes: https://s3.amazonaws.com/clickhouse-test-reports/71406/df68cf7362d825c3d83991fd74c391a49e73a8b1/integration_tests__tsan__[6_6].html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-11-28 16:01:53 +01:00
Azat Khuzhin
02aa6eac82 Cover max_bytes_ratio_before_external_{order,group_by}_for_server
Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
2024-11-28 16:01:53 +01:00
Pablo Marcos
f1f5304484 Update test_grant_and_revoke/test_with_table_engine_grant.py::test_grant_all_on_table 2024-11-28 08:59:49 +00:00
Igor Nikonov
97bd43894d Merge remote-tracking branch 'origin/master' into pr-skip-index-analysis-on-workers 2024-11-28 08:12:31 +00:00
Vitaly Baranov
e7e30d67fb
Merge pull request #72552 from vitlibar/increase-timeout-in-test_short_disconnection_doesnt_stop_restore
Increase timeout in test_short_disconnection_doesnt_stop_restore
2024-11-27 20:17:22 +00:00
robot-clickhouse
dbf217f32a Automatic style fix 2024-11-27 19:57:56 +00:00
Igor Nikonov
10d721991e Remove local_plan duplication 2024-11-27 19:46:41 +00:00
Igor Nikonov
ff2f791e57 Merge remote-tracking branch 'origin/master' into pr-skip-index-analysis-on-workers 2024-11-27 19:24:42 +00:00
Vladimir Cherkasov
9344c1e679
Merge branch 'master' into vdimir/join_select_inner_table 2024-11-27 18:05:07 +01:00
Vitaly Baranov
6657d04553 Increase timeout in test_short_disconnection_doesnt_stop_restore. 2024-11-27 17:42:28 +01:00
Nikita Taranov
d59fe5fd7f
Merge pull request #70788 from ClickHouse/parallel_hash_by_def
Use `parallel_hash` join algorithm by default
2024-11-27 16:18:02 +00:00
Vladimir Cherkasov
d77794d285
Merge branch 'master' into vdimir/join_select_inner_table 2024-11-27 11:05:26 +01:00
Robert Schulze
e9f083b26d
Merge pull request #71997 from allmazz/delete_old_mongodb_integration
Get rid of the legacy mongodb integration
2024-11-26 21:51:46 +00:00
Konstantin Bogdanov
ca2d9ccef5
Merge pull request #72429 from ClickHouse/fix-backups-except-tables-formatting
Fix `BACKUP ... EXCEPT TABLES` queries formatting and parsing
2024-11-26 18:08:19 +00:00
Vladimir Cherkasov
96be57aed2
Merge branch 'master' into vdimir/join_select_inner_table 2024-11-26 18:40:18 +01:00
Robert Schulze
6a899f97eb
Merge remote-tracking branch 'ClickHouse/master' into delete_old_mongodb_integration 2024-11-26 17:33:41 +00:00
Raúl Marín
dd092411ff
Merge pull request #72455 from Algunenano/rename_setting
Rename allowed_feature_tier to allow_feature_tier
2024-11-26 15:54:01 +00:00
vdimir
a8b0b748d0
Resubmit #68682, att2 2024-11-26 12:59:38 +00:00
Robert Schulze
45e1702f82
Merge remote-tracking branch 'ClickHouse/master' into delete_old_mongodb_integration 2024-11-26 12:44:06 +00:00
Robert Schulze
2ed07b21d3
Fix test_mask_sensitive_info 2024-11-26 12:43:08 +00:00
Raúl Marín
575e17758d Adjust tests 2024-11-26 13:34:00 +01:00
Vladimir Cherkasov
37cd162397
Merge pull request #72357 from aalexfvk/fix_move_zero_copy_part_test
Remove flaky test test_move_shared_lock_fail_keeper_unavailable and extend the stable one
2024-11-26 11:39:23 +00:00
Raúl Marín
b487f59496 Rename allowed_feature_tier to allow_feature_tier 2024-11-26 12:30:57 +01:00
Igor Nikonov
53fcfe4180 Merge remote-tracking branch 'origin/master' into pr-skip-index-analysis-on-workers 2024-11-26 10:54:28 +00:00
Igor Nikonov
2b1b4d0516 fix test_parallel_replicas_all_marks_read 2024-11-26 10:54:23 +00:00
Kseniia Sumarokova
ecd1fcb21a
Merge pull request #71301 from MikhailBurdukov/fix_zombies_in_lib_bridge
Fixed zombie processes after library bridge crash.
2024-11-26 10:03:14 +00:00
robot-clickhouse
7ed67942e8 Automatic style fix 2024-11-26 04:14:42 +00:00
Konstantin Bogdanov
b2d358c27b
Add tests 2024-11-26 05:06:14 +01:00
robot-clickhouse
8226903918 Automatic style fix 2024-11-25 21:41:10 +00:00
Igor Nikonov
38132c1451 Update test_parallel_replicas_protocol 2024-11-25 21:32:57 +00:00
Igor Nikonov
62b256422a Merge remote-tracking branch 'origin/master' into pr-skip-index-analysis-on-workers 2024-11-25 20:53:05 +00:00
Nikita Taranov
879035e841 Merge branch 'master' into parallel_hash_by_def 2024-11-25 20:18:04 +01:00
Nikita Taranov
63e9204c52 fix 2024-11-25 19:30:35 +01:00
Raúl Marín
4585a6225c
Revert "Resubmit #68682" 2024-11-25 19:20:14 +01:00
Robert Schulze
17f27275ae
Merge remote-tracking branch 'ClickHouse/master' into delete_old_mongodb_integration 2024-11-25 17:23:15 +00:00
Vladimir Cherkasov
95b21f4a44
Merge pull request #71577 from ClickHouse/vdimir/join_select_inner_table
Resubmit #68682
2024-11-25 13:24:51 +00:00
Aleksei Filatov
bf102d2a00 Remove flaky test and extend stable one 2024-11-24 14:26:38 +00:00
Alexey Milovidov
2379629fd4
Merge pull request #72304 from ClickHouse/fix_postgres_utility
Fix check in `test_postgresql_replica_database_engine_2`
2024-11-24 01:52:59 +00:00
Igor Nikonov
d146c76745 Merge remote-tracking branch 'origin/master' into pr-skip-index-analysis-on-workers 2024-11-22 23:02:33 +00:00
robot-clickhouse
7dead112d1 Automatic style fix 2024-11-22 20:11:27 +00:00
Nikita Taranov
b3344558c1 fix 2024-11-22 21:01:05 +01:00
Vladimir Cherkasov
26de447039
Merge pull request #70603 from zwy991114/check-table-structure-completely-setting
Add setting enforce_index_structure_match_on_partition_manipu..
2024-11-22 16:30:34 +00:00
robot-clickhouse
3af404f2c9 Automatic style fix 2024-11-21 15:53:25 +00:00
divanik
438aa415a6 Resolve issues 2024-11-21 15:42:47 +00:00
Igor Nikonov
207f1c29e6 Merge remote-tracking branch 'origin/master' into pr-skip-index-analysis-on-workers 2024-11-21 13:43:59 +00:00
Vladimir Cherkasov
95e72b7588
Merge branch 'master' into vdimir/join_select_inner_table 2024-11-21 12:42:38 +01:00
MikhailBurdukov
ea375aecb3 Increase timeout 2024-11-21 09:53:31 +00:00
MikhailBurdukov
4cc8d36419 Redesigned 2024-11-21 09:53:30 +00:00
MikhailBurdukov
cb4741f770 Fix zombie processes after library brigde crash. 2024-11-21 09:53:30 +00:00
Antonio Andelic
cc3c7e74ae
Merge pull request #70523 from ClickHouse/randomize-keeper-feature-flasgs-keeper
Randomize Keeper feature flags in integration tests
2024-11-21 08:20:07 +00:00
Alexey Milovidov
25bd73ea5e
Merge pull request #72023 from ClickHouse/fix-bind
Fix comments
2024-11-21 05:03:24 +00:00
Igor Nikonov
fd35a5ab9b Fix test_parallel_replicas_all_marks_read 2024-11-20 22:57:09 +00:00
Igor Nikonov
808436a75d Merge remote-tracking branch 'origin/master' into pr-skip-index-analysis-on-workers 2024-11-20 22:08:23 +00:00
Igor Nikonov
5a95219e89 Add setting parallel_replicas_skip_index_analysis_on_workers 2024-11-20 22:07:57 +00:00
Mikhail Artemenko
44b4bd38b9
Merge pull request #72045 from ClickHouse/issues/70174/cluster_versions
Enable cluster table functions for DataLake Storages
2024-11-20 21:22:37 +00:00
Nikita Mikhaylov
2b3098c641
Merge pull request #72147 from ClickHouse/fix-rabbit
Fix `test_storage_rabbitmq`
2024-11-20 16:59:40 +00:00
Nikita Mikhaylov
9120462ed4 Retry waiting 2024-11-20 13:08:24 +00:00
Raúl Marín
3ae055c74e
Merge pull request #71841 from Algunenano/experimental_tiers_switch
Implement `allowed_feature_tier` as a global switch to  disable all experimental / beta features
2024-11-20 12:39:06 +00:00
Vladimir Cherkasov
59fe7e1951
Merge pull request #71911 from ianton-ru/auxiliary_autodicovery
Auxiliary autodiscovery
2024-11-20 11:27:04 +00:00
Nikita Mikhaylov
dae913c53b
Merge pull request #72080 from ClickHouse/fix-move-partition-to-table-formatting-with-parens
Fix formatting of `MOVE PARTITION ... TO TABLE ...` alter commands
2024-11-20 00:50:19 +00:00
pufit
07be02d297
Merge pull request #70332 from zvonand/ldap-remote-roles
Passing external user roles from query originator to other nodes
2024-11-19 23:06:45 +00:00
Vladimir Cherkasov
456a41ee42
Merge pull request #71845 from aalexfvk/acquire_zero_copy_shared_lock_before_swap
Acquire zero-copy shared lock before moving a part
2024-11-19 19:38:54 +00:00
Kseniia Sumarokova
0ff7c280f9
Merge pull request #72075 from ClickHouse/kssenii-patch-13
Fix  test_disk_over_web_server/
2024-11-19 15:46:01 +00:00
divanik
296717cf76 Fixed bug 2024-11-19 14:18:30 +00:00
János Benjamin Antal
7f493c81d0 Fix test 2024-11-19 14:10:55 +00:00
János Benjamin Antal
a1fb0ad706 Make sure parens are always matching 2024-11-19 13:11:11 +00:00
Mikhail Artemenko
19aec5e572 Merge branch 'issues/70174/cluster_versions' of github.com:ClickHouse/ClickHouse into issues/70174/cluster_versions 2024-11-19 12:51:56 +00:00
Antonio Andelic
57db5cf24c Randomize correctly 2024-11-19 13:39:54 +01:00
Mikhail Artemenko
39ebe113d9 Merge branch 'master' into issues/70174/cluster_versions 2024-11-19 11:28:46 +00:00
Kseniia Sumarokova
6e1de8b8a4
Update test.py 2024-11-19 11:23:46 +01:00
Kseniia Sumarokova
9c4456df1a
Merge branch 'master' into fix-s3-queue-log-processed-rows 2024-11-19 11:21:40 +01:00
Aleksei Filatov
559d9ac517
Merge branch 'master' into acquire_zero_copy_shared_lock_before_swap 2024-11-19 13:00:31 +03:00
Andrey Zvonov
6e58cfc5b8 fix build, fix test 2024-11-19 09:53:28 +00:00
Andrey Zvonov
c4fc7a1bac fix style (2) 2024-11-19 09:53:28 +00:00
Andrey Zvonov
94e2a9cc43 fix style 2024-11-19 09:53:28 +00:00
Andrey Zvonov
9fadfb98b4 add a test that works 2024-11-19 09:53:27 +00:00
Andrey Zvonov
86eb3d6425 Revive #42537
Co-authored-by: Enmk <V.Nemkov@gmail.com>

fix memory access
2024-11-19 09:53:27 +00:00
Antonio Andelic
459fa898ed Merge branch 'master' into randomize-keeper-feature-flasgs-keeper 2024-11-19 10:00:41 +01:00
robot-clickhouse
014608fb6b Automatic style fix 2024-11-18 17:51:51 +00:00
Mikhail Artemenko
a29ded4941 add test for iceberg 2024-11-18 17:39:46 +00:00
Anton Ivashkin
9917dc66d4 Antother style fix 2024-11-18 16:18:36 +01:00
Anton Ivashkin
fb4e1feb16 Add cleanup after test 2024-11-18 14:38:46 +01:00
Sema Checherinda
67d1eb84aa Merge branch 'master' into chesema-merge-wb 2024-11-18 13:54:20 +01:00
Raúl Marín
926f7ca7a2 Style changes 2024-11-18 12:14:18 +01:00
Anton Ivashkin
2d48406a82 Fix test style 2024-11-18 11:21:32 +01:00
vdimir
b03f48807a
Merge branch 'master' into vdimir/join_select_inner_table 2024-11-18 10:06:39 +00:00
Vladimir Cherkasov
3e05ffe675
Merge pull request #71560 from xogoodnow/json-utc-formatted-log
Added UTC date format (RFC 3339) for json formatted logging
2024-11-18 09:10:52 +00:00
Alexey Milovidov
49589da56e Fix comments 2024-11-18 07:18:46 +01:00
Vitaly Baranov
e00128c717 Increase timeout in test test_long_disconnection_stops_backup. 2024-11-17 17:36:22 +01:00
Sema Checherinda
63aacef8d5 Merge branch 'master' into chesema-merge-wb 2024-11-17 00:42:03 +01:00
Kirill Nikiforov
1c5b362f17
fix style 2024-11-16 16:12:12 +04:00
Ali
59f2c44a52
Merge branch 'ClickHouse:master' into json-utc-formatted-log 2024-11-16 11:29:48 +03:30