Commit Graph

62056 Commits

Author SHA1 Message Date
robot-ch-test-poll
402822e3e8
Merge pull request #68485 from ClickHouse/backport/24.8/68452
Backport #68452 to 24.8: Better inference of date times 2
2024-08-17 03:53:42 +02:00
robot-ch-test-poll2
74fd7bfc4c
Merge pull request #68481 from ClickHouse/backport/24.8/68015
Backport #68015 to 24.8: Add profile events for merges
2024-08-17 04:50:41 +04:00
robot-clickhouse
b66599667d Backport #68452 to 24.8: Better inference of date times 2 2024-08-16 22:09:16 +00:00
robot-clickhouse
25a6c51c4a Backport #68015 to 24.8: Add profile events for merges 2024-08-16 21:09:24 +00:00
robot-clickhouse
387d6a8eef Backport #66444 to 24.8: Implement new JSON data type. 2024-08-16 13:14:20 +00:00
robot-clickhouse
f9ec9040bd Backport #68382 to 24.8: Improve schema inference of date times 2024-08-15 21:06:42 +00:00
robot-clickhouse
9740ab6ae8 Backport #68386 to 24.8: Revert "[RFC] Fix settings/current_database in system.processes for async BACKUP/RESTORE" 2024-08-15 16:09:01 +00:00
robot-clickhouse
d8c8b92f6b Backport #68326 to 24.8: Fix missing sync replica mode in query SYSTEM SYNC REPLICA 2024-08-15 11:19:26 +00:00
robot-clickhouse
d60c7f666a Backport #68361 to 24.8: Rename: S3DiskNoKeyErrors -> DiskS3NoSuchKeyErrors 2024-08-15 01:41:22 +00:00
robot-ch-test-poll3
d96357ce17
Merge pull request #68349 from ClickHouse/backport/24.8/68132
Backport #68132 to 24.8: Reimplement Dynamic type
2024-08-14 18:23:02 +02:00
robot-clickhouse
1d8395b118 Backport #68132 to 24.8: Reimplement Dynamic type 2024-08-14 14:09:49 +00:00
robot-clickhouse
2dae7768e0 Backport #68288 to 24.8: Fix postgres crash 2024-08-14 12:14:55 +00:00
robot-clickhouse
40e8812a01 Update autogenerated version to 24.9.1.1 and contributors 2024-08-13 19:26:30 +00:00
Daniil Ivanik
e02b434d2f
Merge pull request #68238 from ClickHouse/divanik/remove_useless_logs_about_zero_replication
Remove unnecessary logs in MergeTreeData.cpp
2024-08-13 16:21:32 +00:00
Alexander Tokmakov
eddf948246
Merge pull request #68250 from ClickHouse/tavplubix-patch-10
Replace segfault in Replicated database with logical error
2024-08-13 16:15:09 +00:00
Nikita Taranov
dd3fa7c3b6
Merge pull request #61793 from nickitat/keep_alive_max_reqs
Implement `max_keep_alive_requests` setting for server
2024-08-13 15:06:43 +00:00
Robert Schulze
2ffcc97af2
Merge pull request #63675 from rschu1ze/vector-search
Initial implementation of vector similarity index
2024-08-13 15:06:20 +00:00
Yakov Olkhovskiy
3e8a177622
Merge pull request #61908 from ClickHouse/ci-fuzzer-enable
CI: enable libfuzzer (fixing build and docker)
2024-08-13 14:22:09 +00:00
Yarik Briukhovetskyi
39c25663ae
Merge pull request #67879 from bigo-sg/opt_orc_writer
Avoid allocating unnecessary capacity for array column while writing orc & some minor refactors
2024-08-13 12:51:11 +00:00
Robert Schulze
99282e526a
Merge pull request #68235 from sakulali/query_cache_tag
QueryCache: Add tagging
2024-08-13 10:44:10 +00:00
Yarik Briukhovetskyi
086c0f03a6
Merge pull request #65997 from yariks5s/hive_style_partitioning
Implementing Hive-style partitioning
2024-08-13 10:04:21 +00:00
pufit
ae5223854f
Merge pull request #67653 from ClickHouse/pufit/inconsistent-formating-grant-current-grants
Fix inconsistent formatting for `GRANT CURRENT GRANTS`
2024-08-13 03:21:26 +00:00
János Benjamin Antal
ac6826392d
Merge pull request #67554 from ClickHouse/fix-message-queue-sink-from-http-interface
Fix message queue sink from http interface
2024-08-12 21:29:14 +00:00
János Benjamin Antal
6eb4a71ad3
Merge pull request #68163 from azat/backups-processes
[RFC] Fix settings/current_database in system.processes for async BACKUP/RESTORE
2024-08-12 21:07:55 +00:00
Alexander Tokmakov
5a683796a0
Update DatabaseReplicated.cpp 2024-08-12 22:34:14 +02:00
Han Fei
40382451a2
Merge pull request #68186 from rschu1ze/stats-tests-refactoring
Refactor tests for (experimental) statistics
2024-08-12 18:58:19 +00:00
Robert Schulze
c22265b889
Some fixups 2024-08-12 17:45:38 +00:00
Kruglov Pavel
ba85cc8d59
Merge pull request #67043 from Avogar/improve-squashing
Improve columns squashing for String/Array/Map/Variant/Dynamic types
2024-08-12 17:14:15 +00:00
Robert Schulze
fe537045c9
Merge remote-tracking branch 'ClickHouse/master' into query_cache_tag 2024-08-12 16:16:32 +00:00
Yarik Briukhovetskyi
3a6e05eb43
try to fix includes 2024-08-12 18:03:42 +02:00
Yarik Briukhovetskyi
ea1cd66575
fix tidy 2024-08-12 17:32:43 +02:00
Robert Schulze
fb76cb90b1
Allow un-quoted skip index parameters
Previously, only this syntax to create a skip index worked:

   INDEX index_name column_name TYPE vector_similarity('hnsw', 'L2Distance')

Now, this syntax will work as well:

  INDEX index_name column_name TYPE vector_similarity(hnsw, L2Distance)
2024-08-12 15:32:25 +00:00
Robert Schulze
d2e79f0b92
Rework vector index parameters
USearch (similar to FAISS) allows to specify the distance function,
quantization, and various HNSW meta-parameters for index creation and
sarch. Some users wished for greater configurability, so let's expose
them.

Index creation now requires either
- 2 parameters (with the other 4 parameters taking on default values), or
- 6 parameters for full control

This commit also remove quantization `f64` (that would be upsampling).
2024-08-12 15:32:19 +00:00
Robert Schulze
cc5c64e1ed
Add migration helper for legacy 'annoy' and 'usearch' indexes types
Index types 'annoy' and 'usearch' were removed and replaced by
'vector_similarity' indexes in an earlier commit.

This means unfortuantely, that if customers have tables with these
indexes and upgrade, their database might not start anymore - the
system loads the metadata at startup, thinks something is wrong with
such tables, and halts immediately.

This commit adds support for loading and attaching such indexes back.
Data insert or use (search) return an error which recommends a migration
to 'vector_similarity' indexes. The implementation is generally similar
to what has recently been implemented for 'full_text' indexes [1, 2].

[1] https://github.com/ClickHouse/ClickHouse/pull/64656
[2] https://github.com/ClickHouse/ClickHouse/pull/64846
2024-08-12 15:31:27 +00:00
Robert Schulze
785b6637fa
Rename index type "usearch" to "vector_similarity"
First, index type "vector_similarity" is more speaking and user-friendly
than "usearch". Second, we should not expose the name of the library
doing the job (usearch). Of course, the docs will continue to mention
usearch (credit where credit is due).

Existing setting `allow_experimental_usearch_index` was marked obsolete.
A new settings `allow_experimental_vector_similarity_index` was added.
2024-08-12 15:30:45 +00:00
Robert Schulze
021fad920e
Cosmetics: minor stuff 2024-08-12 15:30:41 +00:00
Robert Schulze
2aa037985b
Cosmetics: simplify inheritance hierarchy 2024-08-12 15:30:38 +00:00
Robert Schulze
901906159d
Cosmetics: ApproximateNearestNeighborInformation --> Info + nest in class 2024-08-12 15:30:35 +00:00
Robert Schulze
6170aad43e
Cosmetics: ApproximateNearestNeighborIndexesCommon --> VectorSimilarityCondition 2024-08-12 15:30:30 +00:00
Robert Schulze
e20eff635e
Cosmetics: variable naming 2024-08-12 15:30:27 +00:00
Robert Schulze
1bf320a1a8
Cosmetics: metric --> distance_function (for consistent terminology) 2024-08-12 15:30:24 +00:00
Robert Schulze
3f47b42d71
Remove funny typedef 2024-08-12 15:30:21 +00:00
Robert Schulze
fb26a9e6d4
Cosmetics: whitespaces 2024-08-12 15:30:18 +00:00
Robert Schulze
0f1765a273
Cosmetics: function naming 2024-08-12 15:30:14 +00:00
Robert Schulze
a8167abca2
Cosmetics: use native types/functions 2024-08-12 15:30:10 +00:00
Robert Schulze
9ad890e399
Cosmetics: whitespaces 2024-08-12 15:30:07 +00:00
Robert Schulze
27a6931a35
Cosmetics: variable naming 2024-08-12 15:29:59 +00:00
Robert Schulze
289c27c804
Introduce version for for index files in persistence 2024-08-12 15:29:02 +00:00
Robert Schulze
74de79e52b
Addd logging of basic statistics 2024-08-12 15:28:46 +00:00
Robert Schulze
8853b3359b
Remove useless templatization
Makes the code cleaner, compile faster, and the binary smaller.
2024-08-12 15:27:06 +00:00