thevar1able/ClickHouse
1
0
Fork 0
mirror of https://github.com/ClickHouse/ClickHouse.git synced 2024-12-12 01:12:12 +00:00
Code Issues Packages Projects Releases Wiki Activity
109,599 Commits 9,297 Branches 1,782 Tags 2.1 GiB
6cd4507f2f
Commit Graph

11810 Commits

Author SHA1 Message Date
alesapin
c0f28ace51 Better 2023-02-28 17:52:21 +01:00
alesapin
abd6500ee3 Better exception message when move was unseccessful 2023-02-28 17:49:16 +01:00
pufit
29296cb004
Merge pull request #46550 from ClickHouse/46231-systerm-server-settings 
Implement `system.server_settings`
 2023-02-28 11:23:15 -05:00
alesapin
bb7571427b Fix bug in zero-copy replica disk choice during fetch 2023-02-28 14:48:46 +01:00
Kruglov Pavel
a06596bdba
Fix special build 2023-02-28 14:33:41 +01:00
alesapin
712a8309c5
Merge pull request #46976 from ClickHouse/decrease-log-level 
Decrease log level in "disks"
 2023-02-28 12:28:30 +01:00
Nikolai Kochetov
738551dbf1
Merge pull request #46624 from CheSema/detached_mt 
add multitreading for StorageSystemDetachedParts
 2023-02-27 20:54:28 +01:00
Alexey Milovidov
29da7fc965 Decrease log level in "disks" 2023-02-27 17:48:38 +01:00
Anton Popov
caa1d22759
Merge branch 'master' into fix-issues-with-sparse 2023-02-27 15:25:19 +01:00
Robert Schulze
6f0709032d
Merge pull request #46921 from ClickHouse/rs/inv-idx-checksum 
A better alternative to #46344
 2023-02-27 14:44:44 +01:00
Robert Schulze
4c9b9b362c
Replace dynamic_cast by typeid_cast 2023-02-27 10:01:21 +00:00
Robert Schulze
cc0c0c6133
A better alternative to #46344 
The (experimental) inverted index writes/reads files different from the
standard files written by the other skip indexes. The original problem
was that with database engine "ordinary", DROP TABLE of a table with
inverted index finds unknown files in persistence and complains. The
same will happen with engine "atomic" but deferred. As a hotfix, the
error was silenced by explicitly adding the four files created in a
specific test to the deletion code.

This PR tries a cleaner solution where all needed files are provided via
the normal checksum structure. One drawback remains which is that the
affected files were written earlier and we don't have their checksums
available. Therefore, the inverted index is currently excluded from
CHECK TABLE.

Minimal repro:
  SET allow_experimental_inverted_index = 1;
  DROP TABLE IF EXISTS tab;
  CREATE TABLE tab(s String, INDEX af(s) TYPE inverted(2)) ENGINE = MergeTree() ORDER BY s;
  INSERT INTO tab VALUES ('Alick a01');
  CHECK TABLE tab;
  DROP TABLE IF EXISTS tab;

  run ./clickhouse-test with --db-engine Ordinary
 2023-02-26 21:06:38 +00:00
Alexey Milovidov
73aec86f5a Change exception message 2023-02-26 21:36:26 +01:00
Sema Checherinda
1666527dae do listing combined with getting size in one thread 2023-02-26 12:09:05 +01:00
Alexey Milovidov
d448a48f16
Merge pull request #46831 from azat/merge-default-prewhere 
Allow PREWHERE for Merge with different DEFAULT expression for column
 2023-02-26 10:02:34 +03:00
Robert Schulze
5e68d50b39
Merge pull request #46752 from ClickHouse/rs/periods-in-kafka-topics 
Allow configuration of Kafka topics with periods
 2023-02-26 07:49:28 +01:00
Alexey Milovidov
8f40bbd5b4
Revert "Fix several RENAME COLUMN bugs." 2023-02-26 05:47:05 +03:00
Alexey Milovidov
eb89ab471f
Merge pull request #45911 from ClickHouse/mutations_rename_hang 
Fix several `RENAME COLUMN` bugs.
 2023-02-26 01:56:40 +03:00
Alexey Milovidov
656af55831
Merge pull request #46384 from ClickHouse/fixes_for_993 
Fixes for 993
 2023-02-26 01:55:20 +03:00
Robert Schulze
fbba9e319d
Merge branch 'master' into rs/periods-in-kafka-topics 2023-02-25 21:45:20 +01:00
Robert Schulze
3618e37dd2
Reduce indentation by early out 2023-02-25 15:36:59 +00:00
Sema Checherinda
0c9643f1ed make listing in parrallel as well 2023-02-25 16:27:27 +01:00
Nikolai Kochetov
57b5f9e7d7 Fix case with no detached parts for table. 2023-02-25 13:33:20 +01:00
Sema Checherinda
e2bfa83429 cancel workers when exception 2023-02-25 13:33:20 +01:00
Sema Checherinda
08dc874a37 do not run support threads if no tasks left 2023-02-25 13:33:20 +01:00
Sema Checherinda
e5cbe4311e create limites count of support threads for a block 2023-02-25 13:33:20 +01:00
Sema Checherinda
ef0c1841af work with comments on review 2023-02-25 13:33:20 +01:00
Sema Checherinda
3b58249320 pack several files in one thread 2023-02-25 13:33:20 +01:00
Sema Checherinda
8abc1f0d5d fix build and style 2023-02-25 13:33:20 +01:00
Sema Checherinda
dce319e1c5 add multitreading for StorageSystemDetachedParts 2023-02-25 13:33:20 +01:00
Alexander Tokmakov
cad1e0b768 fix 2023-02-25 01:18:34 +01:00
Alexander Tokmakov
ca1c793cac
Merge pull request #46820 from Algunenano/avoid_KeeperMultiException_system_errors 
Don't report ZK node exists to system.errors when a block was created concurrently by a different replica
 2023-02-24 19:42:32 +03:00
Robert Schulze
6402050125
Fixup comments 2023-02-24 16:59:38 +01:00
Azat Khuzhin
86e9b131c7 Allow PREWHERE for Merge with different DEFAULT expression for column 
It can be not that special to have Merge engine w/o default expression,
while MergeTree will have it. So improve PREWHERE for such kinds, and
also allow PREWHERE for DEFAULT and MATERIALIZED (since both are
physical columns on disk and this make sense).

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-24 16:33:49 +01:00
Alexander Gololobov
b43b98f7c0
Merge pull request #46785 from ClickHouse/fix_combined_prewhere_column 
Fix combined PREWHERE column accumulated from multiple steps
 2023-02-24 14:00:17 +01:00
Kruglov Pavel
2150b836b1
Merge branch 'master' into fix-kafka-protobuf 2023-02-24 12:48:07 +01:00
Raúl Marín
7ff8d2d3d5 Don't report ZK node exists to system.errors 2023-02-24 12:36:29 +01:00
pufit
2fbe75957d Add default column for system.settings and system.server_settings 2023-02-23 18:24:57 -05:00
Anton Popov
d926713cf5 Merge remote-tracking branch 'upstream/master' into HEAD 2023-02-23 23:04:22 +00:00
Anton Popov
5e5e802348 return back optimization for ALTER MODIFY Nullable 2023-02-23 22:55:47 +00:00
Mikhail f. Shiryaev
18a20268d8
Merge pull request #46792 from ClickHouse/23.3-prepare 
Update version after release
 2023-02-23 23:51:28 +01:00
Alexander Tokmakov
fe92fd8a61 Merge branch 'master' into fix_insert_cancellation_in_native_protocol 2023-02-23 22:35:31 +01:00
Robert Schulze
6906cafa99
Change to S3 cfg syntax 2023-02-23 20:04:41 +00:00
Antonio Andelic
1efba8995d Update version to 23.3.1.2537 2023-02-23 19:35:57 +00:00
Alexander Gololobov
47f51fe187
Merge pull request #46771 from ClickHouse/fix_prewhere_logging 
More concise logging at trace level for PREWHERE steps
 2023-02-23 20:20:30 +01:00
Alexander Gololobov
8b22373488 Apply filter at last step if it has been accumulated from multiple steps 2023-02-23 16:58:28 +01:00
Alexander Gololobov
cbcb1a7852 More concise logging at trace level 2023-02-23 13:31:56 +01:00
Kseniia Sumarokova
7e064ccbd4
Merge pull request #46667 from kssenii/hide-named-collections-under-setting 
Allow to hide only values from system.named_collections while making collection visible
 2023-02-23 11:44:50 +01:00
Kseniia Sumarokova
30a8eb0c2f
Merge pull request #46670 from kssenii/hide-disk-setting 
Hide arguments of custom disk merge tree setting
 2023-02-23 11:42:53 +01:00
Alexey Milovidov
c0bc549e77
Merge pull request #46709 from azat/bg-pool-no-query 
BackgroundSchedulePool should not have any query context
 2023-02-23 10:24:46 +03:00
Alexey Milovidov
e3f24b2b68
Merge pull request #46744 from ClickHouse/remove_unused_field 
Remove unused MergeTreeReadTask::remove_prewhere_column
 2023-02-23 08:02:54 +03:00
Robert Schulze
81bf43157f
Allow configuration of Kafka topics with periods 
The Kafka table engine allows global configuration and per-Kafka-topic
configuration. The latter uses syntax <kafka_TOPIC>, e.g. for topic
"football":

  <kafka_football>
      <retry_backoff_ms>250</retry_backoff_ms>
      <fetch_min_bytes>100000</fetch_min_bytes>
  </kafka_football>

Some users had to find out the hard way that such configuration doesn't
take effect if the topic name contains a period, e.g. "sports.football".
The reason is that ClickHouse configuration framework already uses
periods as level separators to descend the configuration hierarchy.
(Besides that, per-topic configuration at the same level as global
configuration could be considered ugly.)

Note that Kafka topics may contain characters "a-zA-Z0-9._-" (*) and
a tree-like topic organization using periods is quite common in
practice.

This PR deprecates the existing per-topic configuration syntax (but
continues to support it for backward compat) and introduces a new
per-topic configuration syntax below the global Kafka configuration of
the form:

<kafka>
   <topic name="football">
       <retry_backoff_ms>250</retry_backoff_ms>
       <fetch_min_bytes>100000</fetch_min_bytes>
   </topic>
</kafka>

The period restriction doesn't apply to XML attributes, so <topic
name="sports.football"> will work. Also, everything Kafka-related is
below <kafka>.

Considered but rejected alternatives:
- Extending Poco ConfigurationView with custom separators (e.g."/"
  instead of "."). Won't work easily because ConfigurationView only
  builds a path but defers descending the configuration tree to the
  normal configuration classes.
- Reloading the configuration file in StorageKafka (instead of reading
  the loaded file) but with a custom separator. This mode is supported
  by XML configuration. Too ugly and error-prone since the true
  configuration is composed from multiple configuration files.

(*) https://stackoverflow.com/a/37067544
 2023-02-22 20:35:09 +00:00
Alexander Tokmakov
e660c0838c fix multipart requests 2023-02-22 17:54:35 +01:00
Alexander Gololobov
67dcd9694c Remove unused MergeTreeReadTask::remove_prewhere_column 2023-02-22 17:49:22 +01:00
pufit
8e7533fa57
Merge pull request #46564 from AVMusorin/update-time-distribution-queue 
Added `last_exception_time` column into distribution_queue table
 2023-02-22 10:43:35 -05:00
kssenii
c4761d6cc6 Fix checks 2023-02-22 14:27:43 +01:00
Kruglov Pavel
3ba3fdbfa3
Merge pull request #46607 from kssenii/delay-loading-of-named-collections 
Do not load named collections on server startup (on first access instead)
 2023-02-22 13:22:34 +01:00
Azat Khuzhin
2ca47a6eb6 BackgroundSchedulePool should not have any query context 
BackgroundSchedulePool is used for some peridic jobs, not from the query
context, i.e. flush of Buffer table.

And for such jobs there cannot be any query context, and more
importantly it will not work correctly since that query_context will
eventually expires.

And this is the reason of this failures [1].

  [1]: https://s3.amazonaws.com/clickhouse-test-reports/46668/015991bc5e20c787851050c2eaa13f0fef3aac00/stateless_tests_flaky_check__asan_.html

Signed-off-by: Azat Khuzhin <a.khuzhin@semrush.com>
 2023-02-22 10:50:51 +01:00
Alexander Tokmakov
592af6d652 fix incomplete interst through http 2023-02-22 02:34:03 +01:00
AVMusorin
0bf0fe488e
added last_exception_time column into distribution_queue table 2023-02-21 23:46:57 +01:00
Alexander Tokmakov
f5c232707d cancel insert queries correctly 2023-02-21 22:04:39 +01:00
Alexander Gololobov
b61bb56a5a
Merge pull request #46611 from ClickHouse/prewhere_progress 
Fix reporting progress when rows are filtered in PREWHERE
 2023-02-21 21:27:56 +01:00
kssenii
9a7c71b78e Allow to hide only values from system.named_collections 2023-02-21 18:07:57 +01:00
Alexey Milovidov
848286eca7
Merge pull request #40945 from arthurpassos/add_select_final_mt_setting 
Add user setting to force select final at query level
 2023-02-21 19:58:45 +03:00
Kseniia Sumarokova
655e7f29f7
Merge pull request #46595 from kssenii/update-rabbit-producer 
Fix rabbitmq test
 2023-02-21 16:18:55 +01:00
Anton Popov
b66a5bdcf0 fix mutations with sparse columns 2023-02-21 15:13:29 +00:00
Alexander Gololobov
5354b0d069
Merge pull request #46140 from ClickHouse/split_prewhere_dag 
Split PREWHERE DAG into multiple read steps
 2023-02-21 14:37:09 +01:00
Kseniia Sumarokova
09a979f106
Merge branch 'master' into update-rabbit-producer 2023-02-21 11:50:30 +01:00
Vladimir C
a791250a4a
Merge pull request #38614 from bharatnc/ncb/profile-events-partlog 2023-02-21 11:06:44 +01:00
Kseniia Sumarokova
24d0490ef3
Merge branch 'master' into delay-loading-of-named-collections 2023-02-21 00:24:18 +01:00
Kruglov Pavel
828a06b511
Merge branch 'master' into fix-kafka-protobuf 2023-02-20 21:45:43 +01:00
Kruglov Pavel
b0424c1021
Merge pull request #46171 from Avogar/insert-null-as-default 
Use default of column type in `insert_null_as_default` if column DEFAULT values is not specified
 2023-02-20 21:45:02 +01:00
Arthur Passos
4f2fcf763b
Merge branch 'master' into add_select_final_mt_setting 2023-02-20 16:22:14 -03:00
Kseniia Sumarokova
47604ad77e
Merge pull request #46106 from kssenii/return-materialized-postrgesql-tests-back 
Enable disabled MaterializedPostgreSQL tests back in CI
 2023-02-20 19:04:58 +01:00
Alexander Gololobov
c84a64d22f Return chunks with 0 rows from MergeTreeSource to report progress when rows are filtered in PREWHERE 2023-02-20 17:35:01 +01:00
kssenii
9d16205c8a Load named collections on first access 2023-02-20 16:33:45 +01:00
kssenii
80a0aa1327 Merge remote-tracking branch 'upstream/master' into return-materialized-postrgesql-tests-back 2023-02-20 14:37:39 +01:00
kssenii
361678ad73 rabbitmq-test-fix 2023-02-20 14:16:40 +01:00
Alexey Milovidov
d8cda3dbb8 Remove PVS-Studio 2023-02-19 23:30:05 +01:00
Arthur Passos
3d9b23e5f3
Merge branch 'master' into add_select_final_mt_setting 2023-02-19 15:01:41 -03:00
Alexey Milovidov
5359dcc014
Merge pull request #46554 from ucasfl/iceberg-fix 
fix build without avro
 2023-02-19 14:35:29 +03:00
Alexander Gololobov
520b381675 Fixes for executing actions on block with unknown row count 2023-02-19 12:14:23 +01:00
Maksim Kita
6dffe52e68
Merge pull request #46471 from kitaisreal/analyzer-planner-fixes-before-enable-by-default 
Analyzer planner fixes before enable by default
 2023-02-19 13:49:18 +03:00
flynn
7b4fbf33b3 fix 2023-02-19 03:47:14 +00:00
Alexander Gololobov
0ed0b191f8 Fix for Nullable step filter column 2023-02-18 19:12:32 +01:00
flynn
177261a5ac fix typo 2023-02-18 17:34:37 +00:00
flynn
830398f0bd fix build without avro 
fix
 2023-02-18 17:32:35 +00:00
Alexander Gololobov
88fb6c3c25 Fix for incorrect combined PREWHERE column 2023-02-18 18:29:16 +01:00
Maksim Kita
cbd961de98 Fixed code review issues 2023-02-18 17:06:00 +01:00
Nikolay Degterinsky
f93aae80b8
Merge pull request #46407 from evillique/parameters-rename-query 
Add parameterized RENAME queries
 2023-02-18 16:04:55 +01:00
Kseniia Sumarokova
b0ea047e76
Merge pull request #46533 from kssenii/fix-disk-setting 
Allow `disk` merge tree setting in query to override config merge tree setting `storage_policy`
 2023-02-18 12:36:54 +01:00
pufit
4496db66b2 Implement system.server_settings 2023-02-17 23:46:09 -05:00
Kseniia Sumarokova
bda4c94f66
Merge pull request #45384 from ucasfl/iceberg 2023-02-17 23:38:24 +01:00
Kseniia Sumarokova
a0b28edd46
Merge pull request #46521 from kssenii/fix-stress-test-logical-errors 
Fix stress test
 2023-02-17 21:01:20 +01:00
Nikolay Degterinsky
5104b2e8bf
Merge branch 'master' into parameters-rename-query 2023-02-17 17:30:37 +01:00
kssenii
afb36dd120 Allow disk setting to override storage_policy from config merge tree settings insteaf of exception 2023-02-17 17:11:29 +01:00
Maksim Kita
6c1436f78e Fixed tests 2023-02-17 16:50:46 +01:00
flynn
ebd88aab41
Merge branch 'master' into iceberg 2023-02-17 21:53:29 +08:00
Vladimir C
0f182f6164
Merge pull request #46454 from azat/merge-alias-prewhere 2023-02-17 14:10:52 +01:00
flynn
b3a9468661 fix 2023-02-17 12:42:24 +00:00