Commit Graph

150561 Commits

Author SHA1 Message Date
Robert Schulze
785b6637fa
Rename index type "usearch" to "vector_similarity"
First, index type "vector_similarity" is more speaking and user-friendly
than "usearch". Second, we should not expose the name of the library
doing the job (usearch). Of course, the docs will continue to mention
usearch (credit where credit is due).

Existing setting `allow_experimental_usearch_index` was marked obsolete.
A new settings `allow_experimental_vector_similarity_index` was added.
2024-08-12 15:30:45 +00:00
Robert Schulze
021fad920e
Cosmetics: minor stuff 2024-08-12 15:30:41 +00:00
Robert Schulze
2aa037985b
Cosmetics: simplify inheritance hierarchy 2024-08-12 15:30:38 +00:00
Robert Schulze
901906159d
Cosmetics: ApproximateNearestNeighborInformation --> Info + nest in class 2024-08-12 15:30:35 +00:00
Robert Schulze
6170aad43e
Cosmetics: ApproximateNearestNeighborIndexesCommon --> VectorSimilarityCondition 2024-08-12 15:30:30 +00:00
Robert Schulze
e20eff635e
Cosmetics: variable naming 2024-08-12 15:30:27 +00:00
Robert Schulze
1bf320a1a8
Cosmetics: metric --> distance_function (for consistent terminology) 2024-08-12 15:30:24 +00:00
Robert Schulze
3f47b42d71
Remove funny typedef 2024-08-12 15:30:21 +00:00
Robert Schulze
fb26a9e6d4
Cosmetics: whitespaces 2024-08-12 15:30:18 +00:00
Robert Schulze
0f1765a273
Cosmetics: function naming 2024-08-12 15:30:14 +00:00
Robert Schulze
a8167abca2
Cosmetics: use native types/functions 2024-08-12 15:30:10 +00:00
Robert Schulze
9ad890e399
Cosmetics: whitespaces 2024-08-12 15:30:07 +00:00
Robert Schulze
27a6931a35
Cosmetics: variable naming 2024-08-12 15:29:59 +00:00
Robert Schulze
289c27c804
Introduce version for for index files in persistence 2024-08-12 15:29:02 +00:00
Robert Schulze
4ad624cb7e
Cosmetics 2024-08-12 15:28:58 +00:00
Robert Schulze
74de79e52b
Addd logging of basic statistics 2024-08-12 15:28:46 +00:00
Kruglov Pavel
bba4a90a9c
Merge branch 'master' into better-dynamic 2024-08-12 17:28:09 +02:00
Robert Schulze
8853b3359b
Remove useless templatization
Makes the code cleaner, compile faster, and the binary smaller.
2024-08-12 15:27:06 +00:00
Robert Schulze
4f23f7754b
Cosmetics 2024-08-12 15:26:05 +00:00
Robert Schulze
7f611681df
Add a similar sanity check as in other skipping indexes 2024-08-12 15:26:01 +00:00
Robert Schulze
f944ef25bb
Better handling of errors during add, search, and save 2024-08-12 15:25:58 +00:00
Robert Schulze
e7c2bf49c3
Add detach/attach test 2024-08-12 15:25:55 +00:00
Robert Schulze
40bed3e20f
Remove support for WHERE-type queries
These kind of vector search similarity queries are rather obscure and
rare in practice. They require the user to specify a maximum distance
which is not intuitive to obtain. Furthermore, these queries are not
natively supported in USearch, so the vector search index had to emulate
these queries.

Therefore simplifying the code base and restricting vector search to
ORDER-BY queries only.
2024-08-12 15:25:52 +00:00
Robert Schulze
abb8e61981
Remove support code for Lp norm in vector search
It is a generalization of other norms, too expensive to calculate and
not relevant in practice. Also, Usearch doesn't support it.
2024-08-12 15:25:48 +00:00
Robert Schulze
65186f0b69
Remove tuple support
Indexes for approximate nearest neighbourhood (ANN) search (USearch) can
be build on columns of type Array(Float32) or Tuple(Float32[, Float32[, ...]]).
In practice, Arrays(Float32) is the only relevant data type.
Arrays store high-dimensional embeddings consecutively (--> cache
locality) and the additional flexibility of different data types in a
tuple is not needed for vector search.

Therefore removing support for ANN indexes over tuple columns to
simplify the code, tests and docs.
2024-08-12 15:25:39 +00:00
Robert Schulze
218421c255
Remove Annoy indexes
Annoy indexes fell out of favor in the community, at least when it comes
to vector databases. Such indexes work okay-ish low dimensions but they
suffers badly from a curse of dimensionality which makes them inapt for
a high number of dimensions.

Now that Annoy is gone, issue (*) also disappears and we can drop
'no-ubsan', 'no-cpu-aarch64', and 'no-asan' from tests.

(*) spotify/annoy#456
2024-08-12 15:24:49 +00:00
Robert Schulze
7c41939921
Fix test results (no analyzer support yet ...) 2024-08-12 15:24:22 +00:00
Robert Schulze
d7211f9d12
Fix CMake integration of usearch and annoy
Registers usearch and annoy properly via configure_config.cmake and
config.h.in like all other 3rd party libs, instead of (mis)using
target_compile_definitions.
2024-08-12 15:24:18 +00:00
Robert Schulze
a39b9cf643
Un-screw usearch's build description
No directory 'SimSIMD-map' exists, the build only worked because SimSIMD
support in usearch was (accidentally?) disabled. This commit corrects
the build description. SimSIMD support in usearch will be enabled by a
later commit.
2024-08-12 15:24:14 +00:00
divanik
eb3ffb7184 Add supportsReplication 2024-08-12 15:09:16 +00:00
Robert Schulze
85f63b056b
Merge pull request #68135 from ClickHouse/refactor-field-get
Only use Field::safeGet - Field::get prone to type punning
2024-08-12 14:25:11 +00:00
Pablo Marcos
da5b9582a9 Fix indent 2024-08-12 13:54:17 +00:00
János Benjamin Antal
6cde029ed9 Fix style 2024-08-12 13:48:44 +00:00
Pablo Marcos
f7c6eabb49 Small fix to filter by current_database in system.query_log 2024-08-12 13:44:05 +00:00
János Benjamin Antal
34643ee16c Run test only from modified files 2024-08-12 13:30:25 +00:00
Robert Schulze
037a1006fd
Merge remote-tracking branch 'ClickHouse/master' into ci-fuzzer-enable 2024-08-12 12:28:32 +00:00
Nikita Taranov
2f546fb513
Merge pull request #68098 from aiven-sal/aiven-sal/segfault
Fix UB in hopEnd, hopStart, tumbleEnd, and tumbleStart
2024-08-12 12:09:23 +00:00
János Benjamin Antal
737948470d Fix syntax 2024-08-12 11:54:34 +00:00
Sema Checherinda
5e836bc20e
Merge pull request #67472 from ClickHouse/chesema-02765
speed up system flush logs
2024-08-12 11:51:55 +00:00
avogar
b58a22aba7 Update test 2024-08-12 11:41:42 +00:00
Antonio Andelic
e9d16bc054 Use new mc restart 2024-08-12 13:39:53 +02:00
avogar
38f3131e11 Fix review comments 2024-08-12 11:39:36 +00:00
vdimir
52f37f2ec6
Merge pull request #67980 from ClickHouse/vdimir/fix_03130_convert_outer_join_to_inner_join
Fix 03130_convert_outer_join_to_inner_join
2024-08-12 11:34:10 +00:00
Pablo Marcos
06ceaee502 Fix test 01903_correct_block_size_prediction_with_default
- Don't allow random settings that affect the memory usage
- Run two queries and compare the memory usage, rather than
  having an arbitrary hardcoded value
2024-08-12 11:11:10 +00:00
János Benjamin Antal
8f124710ef Remove Log engine from Kafka integration tests
It doesn't work well when `thread_per_consumer` is used as writer can make readers starve when `shared_time_mutex` prefers writes over reads.
2024-08-12 10:27:21 +00:00
Yarik Briukhovetskyi
8bc89ac8df
Merge branch 'master' into hive_style_partitioning 2024-08-12 11:44:45 +02:00
Robert Schulze
ee433684dd
Follow-up to ClickHouse#63898, pt. III 2024-08-12 08:15:03 +00:00
Robert Schulze
0aa30b10d5
Merge pull request #68069 from rschu1ze/cmake-cleanup
Minor CMake cleanup
2024-08-12 06:43:00 +00:00
Robert Schulze
574c445be9
Refactor tests for (experimental) statistics 2024-08-12 05:56:16 +00:00
Alexey Milovidov
6016dc96aa Fix test 01172_transaction_counters 2024-08-12 07:19:54 +02:00