v1.14.0 🦫
Meilisearch v1.14 gives more granular control over which parts of filters you can disable for indexing performance optimization. This release also includes composite embedders, which can improve embedding generation during search and indexing, and a new route to retrieve multiple documents by their IDs.
🧰 All official Meilisearch integrations (including SDKs, clients, and other tools) are compatible with this Meilisearch release. Integration deployment happens between 4 to 48 hours after a new version becomes available.
Some SDKs might not include all new features. Consult the project repository for detailed information. Is a feature you need missing from your chosen SDK? Create an issue letting us know you need it, or, for open-source karma points, open a PR implementing it (we'll love you for that ❤️).
v1.14 gives you more control over which types of filter you want to disable in your searches. This allows you to further optimize indexing speeds by letting you activate only the filter features you need.
Use PATCH /indexes/INDEX_NAME/settings
to specify which filters you want to enable for each attribute in your documents:
{
"filterableAttributes": [
{
"attributePatterns": ["genre", "artist"],
"features": {
"facetSearch": true,
"filter": {
"equality": true,
"comparison": false
}
}
},
{
"attributePatterns": ["rank"],
"features": {
"facetSearch": false,
"filter": {
"equality": true,
"comparison": true
}
}
}
]
}
For more details about this feature, please refer to its public usage page
Done by @ManyTheFish in #5254.
This feature allows using different embedders at search and indexing time. This can be useful when optimizing AI-powered search performance. For example, you may prefer to use:
- A remote embedder during indexing, as remote embedders have higher bandwidth and can generate more embeddings per second
- A local embedder when answering to search queries, as local embedders have lower latency and can respond more quickly to user input
To use the feature, follow these steps:
- Enable the
Composite embedders
feature with the Meilisearch Cloud interface, or with the/experimental-features
route:
curl MEILISEARCH_URL/experimental-features \
-H 'Content-Type: application/json' \
-d '{"compositeEmbedders": true}'
- Next, create an embedder, setting its
source
to"composite"
and defining onesearchEmbedder
and oneindexingEmbedder
:
- Once the composite embedder has been created, Meilisearch will use its
indexingEmbedder
during indexing andsearchEmbedder
when responding to user queries
For more details about this feature, please refer to its public usage page.
Done by @dureuill in #5371 and #5401.
It is now possible to retrieve multiple documents by their IDs:
curl -H 'Content-Type: application/json' MEILISEARCH_URL/indexes/INDEX_UID/documents -d '{ "ids": ["cody", "finn", "brandy", "gambit"] }'
{
"results": [
{
"id": "brandy",
"info": 13765493
},
{
"id": "finn",
"info": 35863
},
{
"id": "cody",
"info": 122263
},
{
"id": "gambit",
"info": 22222
}
],
"offset": 0,
"limit": 20,
"total": 4
}
[!WARNING] Documents are not returned in the queried order. Non-existent documents are ignored.
Done by @dureuill in #5384.
- Batch together
/documents
requests using eitherPUT
orPOST
by @Kerollmops in #5293 - Display timestamped internal indexing steps on the
/batches
route by @Kerollmops in #5356 and #5364 - Introduce
exhaustiveFacetCount
parameter to/facet-search
route to retrieve an exhaustive facet count by @ManyTheFish in #5369 - Reduce RAM consumption of arroy by @irevoire in https://github.com/meilisearch/arroy/pull/105
- Experimental feature: Cache embeddings during search (Read more in the feature discussion) by @dureuill in #5418
- Extend batch progress view to include indexing of vectors by @irevoire in #5420
- Armenian characters are no longer case-sensitive by @ManyTheFish in https://github.com/meilisearch/meilisearch/pull/5454
- Avoid reindexing searchables when the order changes by @ManyTheFish in https://github.com/meilisearch/meilisearch/pull/5402
- Accept cancellation tasks even when the disk is full by @irevoire in https://github.com/meilisearch/meilisearch/pull/5492
- Geo update bug by @ManyTheFish in https://github.com/meilisearch/meilisearch/pull/5407
- Fix prefix search on attributes listed in
disabledOnAttributes
settings by @ManyTheFish in https://github.com/meilisearch/meilisearch/pull/5415 (fixes #5347 and #5452) - Fix CI to work with merge queues by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5456
- Support fetching the pooling method from the model configuration by @dureuill in #5355
- Delete unused prefixes by @Kerollmops in #5413
- Fixes mini dashboard to prevent the panel from popping up every time by @curquiza in https://github.com/meilisearch/meilisearch/pull/5436
- Enabling
rankingScoreThreshold
no longer causes_rankingScore
to be miscalculated by @barloes and @dureuill in https://github.com/meilisearch/meilisearch/pull/5313 - Validate dimensions of embedding when receiving documents with
_vectors
by @dureuill in https://github.com/meilisearch/meilisearch/pull/5478
- Dependencies updates
- Bump Ubuntu in the CI from 20.04 to 22.04 by @Kerollmops in #5338
- Bump heed to v0.22 by @irevoire and @Kerollmops in #5406
- Bump ring to v0.17.14 to compile on old aarch64 by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5423
- Bump zip from 2.2.2 to 2.3.0 by @dependabot in https://github.com/meilisearch/meilisearch/pull/5426
- CIs and tests
- Improve test performance of get_index.rs by @DerTimonius in #5210
- Ollama Integration Tests by @Kerollmops in #5308
- Ensure the settings routes are properly configured when a new field is added to the Settings struct by @MichaScant in #5149
- Skip a snapshot test on Windows by @Kerollmops in #5383
- Fix CI to work with merge queues by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5456
- Accept total batch size in human size by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5421
- Add more progress levels to measure merging and post-processing by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5422 and https://github.com/meilisearch/meilisearch/pull/5468
- Isolate word fst usage to dedicate it to typo-correction by @ManyTheFish in https://github.com/meilisearch/meilisearch/pull/5415
- Show database sizes batches by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5457 and https://github.com/meilisearch/meilisearch/pull/5464
⚠️ Please consider that the batches stats content can change anytime.
- Improve performance of computing document stats by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5465
❤️ Thanks again to our external contributors:
- Meilisearch: @MichaScant
- Heed: @oXtxNt9U
- Arroy: @ptondereau
- Charabia: @NarHakobyan, @mosuka
v1.14.0-rc.3
[!WARNING] Since this is a release candidate (RC), we do NOT recommend using it in a production environment. Is something not working as expected? We welcome bug reports and feedback about new features.
- make meilisearch accept cancellation tasks even when the disk is full by @irevoire in https://github.com/meilisearch/meilisearch/pull/5492
Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.14.0-rc.2...v1.14.0-rc.3
v1.14.0-rc.2 🦫
[!WARNING] Since this is a release candidate (RC), we do NOT recommend using it in a production environment. Is something not working as expected? We welcome bug reports and feedback about new features.
- enabling
rankingScoreThreshold
no longer causes_rankingScore
to be miscalculated by @barloes and @dureuill in https://github.com/meilisearch/meilisearch/pull/5313 - Validate dimensions of embedding when receiving documents with
_vectors
by @dureuill in https://github.com/meilisearch/meilisearch/pull/5478
- Improve performance of computing document stats by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5465
- More detailed Post Processing progress trace by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5468
- Prefer camelCase for internal database sizes db name by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5464
- @barloes made their first contribution in https://github.com/meilisearch/meilisearch/pull/5313
Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.14.0-rc.1...v1.14.0-rc.2
v1.14.0-rc.1 🦫
Add more progress levels to measure merging by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5422
Computes the durations of each indexing step and displays them in the batch route. #5356
⚠️ When working with it in the doc or the integrations, please consider that the content can change anytime. It must not be enforced.
Show database sizes batches by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5457
Computes the size of each database and displays them in the batch route. #5463
⚠️ When working with it in the doc or the integrations, please consider that the content can change anytime. It must not be enforced.
- Accept total batch size in human size by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5421
- Avoid reindexing searchable order changes by @ManyTheFish in https://github.com/meilisearch/meilisearch/pull/5402
- Geo update bug by @ManyTheFish in https://github.com/meilisearch/meilisearch/pull/5407
- Isolate word fst usage by @ManyTheFish in https://github.com/meilisearch/meilisearch/pull/5415
- Fix CI to work with merge queues by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5456
- Bump ring to v0.17.14 to compile on old aarch64 by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5423
- Bump zip from 2.2.2 to 2.3.0 by @dependabot in https://github.com/meilisearch/meilisearch/pull/5426
- Update Charabia v0.9.3 by @ManyTheFish in https://github.com/meilisearch/meilisearch/pull/5454
- Update mini-dashboard to v0.2.19 version by @curquiza in https://github.com/meilisearch/meilisearch/pull/5436
Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.14.0-rc.0...v1.14.0-rc.1
v1.14.0-rc.0 🦫
[!WARNING] Since this is a release candidate (RC), we do NOT recommend using it in a production environment. Is something not working as expected? We welcome bug reports[#new] and feedback about new features.
Meilisearch v1.14 exposes a more granular way to express your filterable attributes! 🎉 This release also improves semantic search performance, embedding indexing speed, and a new route to get multiple documents by IDs.
This feature allows for the choice of the filter features of each filterable attribute with high granularity. Activating or deactivating a feature will have an impact on the indexing time. To use the feature, use the new filterableAttributes
format in the settings route (PATCH /indexes/INDEX_UID/settings
):
{
"filterableAttributes": [
{
"attributePatterns": ["genre", "artist"],
"features": { "facetSearch": true, "filter": { "equality": true, "comparison": false } }
},
{
"attributePatterns": ["rank"],
"features": { "facetSearch": false, "filter": { "equality": true, "comparison": true } }
},
{
"attributePatterns": ["albumId"],
"features": { "facetSearch": false, "filter": { "equality": true, "comparison": false } }
},
]
}
🗒️ In this example, we activate/deactivate each feature depending on the usage of each filterable attribute, the
genre
andartist
are expected to be strings so the facet search and the equality operators (=
/!=
) are activated, but, the comparison operators are deactivated (>=
,<=
, ...) because it's not useful and will save time during the indexing process. However, the rank is expected to be a number, so the facet search is deactivated in favor of the the comparison operators. Last, the albumId is expected to be unique, so only the equality operators (=
/!=
) are activated.
For more details about this feature, please refer to its public usage page.
Done by @ManyTheFish in #5254.
This feature allows using different embedders at search and indexing time, which is useful to optimize the embedder to each situation:
- Using a remote embedder for indexing in bulk, as remote embedders provide the highest bandwidth (embeddings/s)
- Using a local embedder for answering search queries, as local embedders provide the lowest latency (time to first embedding)
To use the feature, follow these steps:
- Enable the
Composite embedders
feature from the Cloud dashboard or with the following:
curl -H 'Content-Type: application/json' MEILISEARCH_URL/experimental-features -d '{ "compositeEmbedders": true }'
- Send a settings task (
PATCH /indexes/INDEX_UID/settings
) containing an embedder withsource: "composite"
, and with the parameterssearchEmbedder
andindexingEmbedder
describing the embedder at search time (resp. indexing time). For example, using a Hugging Face inference endpoint.
{
"embedders": {
"text": {
"source": "composite",
"searchEmbedder": {
"source": "huggingFace", // locally computed embeddings using a model from the Hugging Face Hub
"model": "baai/bge-base-en-v1.5",
"revision": "a5beb1e3e68b9ab74eb54cfd186867f64f240e1a"
},
"indexingEmbedder": {
"source": "rest", // remotely computed embeddings using Hugging Face inference endpoints
"url": "https://URL.endpoints.huggingface.cloud",
"apiKey": "hf_XXXXXXX",
"documentTemplate": "Your {{doc.template}}",
"request": {
"inputs": [
"{{text}}",
"{{..}}"
]
},
"response": [
"{{embedding}}",
"{{..}}"
]
}
}
}
}
- Send documents to the index. They will be indexed remotely using the Hugging Face inference endpoint.
- Perform semantic search queries, they will be embedded using the local model fetched from the Hugging Face Hub.
For more details about this feature, please refer to its public usage page.
Done by @dureuill in #5371.
- Get multiple documents by ids by @dureuill in #5384
- Support merging update and replacement operations by @Kerollmops in #5293
- Display the internal indexing steps with timings on the
/batches
route by @Kerollmops in #5356 - exhaustive facet search by @ManyTheFish in #5369
- Reduce RAM consumption of arroy by @irevoire in https://github.com/meilisearch/arroy/pull/105
- Cache Embeddings in Search with an experimental feature by @dureuill in #5418
- Extend the batch progress view to the indexing of vectors by @irevoire in 5420
- Support fetching the pooling method from the model configuration by @dureuill in #5355
- Make sure to delete useless prefixes by @Kerollmops in #5413
- Dependencies updates
- CIs and tests
- Misc
❤️ Thanks again to our external contributors:
- Meilisearch: @MichaScant
- Heed: @oXtxNt9U
- Arroy: @ptondereau
v1.13.3 🕊️
- Bump mini-dashboard v0.2.18 by @Strift in https://github.com/meilisearch/meilisearch/pull/5385
Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.13.2...v1.13.3
v1.13.2 🕊️
- Support dumpless upgrade for all v1.13 patches by @dureuill in https://github.com/meilisearch/meilisearch/pull/5376
Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.13.1...v1.13.2
v1.13.1 🕊️
- Hotfix typo tolerance bug by @ManyTheFish in https://github.com/meilisearch/meilisearch/pull/5346
- Bump mini-dashboard to v0.2.17 by @Strift in https://github.com/meilisearch/meilisearch/pull/5367
- Documents database stats by @ManyTheFish in https://github.com/meilisearch/meilisearch/pull/5325
- Fix the dumpless upgrade log by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5360
Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.13.0...v1.13.1
v1.13.0 🕊️
Meilisearch v1.13 stabilizes AI-powered search integration! 🎉 This release also introduces two new experimental features: upgrading to new Meilisearch releases without generating a dump and making federated requests across multiple Meilisearch instances.
🧰 All official Meilisearch integrations (including SDKs, clients, and other tools) are compatible with this Meilisearch release. Integration deployment takes 4 to 48 hours after a new version becomes available.
Some SDKs might not include all new features. Consult the project repository for detailed information. Is a feature you need missing from your chosen SDK? Create an issue letting us know you need it, or, for open-source karma points, open a PR implementing it (we'll love you for that ❤️).
With v1.13, AI-powered search is available to all users by default and no longer requires manual activation.
vectorStore
is no longer an accepted value for the/experimental-features
route- Ollama URLs must end with either
/api/embed
or/api/embeddings
- Modified error codes:
invalid_embedder
has been split intoinvalid_search_embedder
andinvalid_similar_embedder
. These codes are returned when theembedder
parameter of a request to/search
,/multi-search
or/similar
specifies a non-existing embedder or is not a stringinvalid_hybrid_query
has been renamed toinvalid_search_hybrid_query
. It is returned when thehybrid
parameter contains unknown keys or is not eithernull
or an object
Done by @dureuill in https://github.com/meilisearch/meilisearch/pull/5232 & https://github.com/meilisearch/meilisearch/pull/5234
Use --experimental-dumpless-upgrade
after updating the Meilisearch binary to migrate to a new release:
./meilisearch --experimental-dumpless-upgrade
This faster and more efficient process does not require you to generate a dump and pass it to Meilisearch when upgrading to a new release.
[!WARNING] Meilisearch strongly recommends you generate a backup snapshot before migrating. This is an experimental feature, and failed upgrades may lead to database corruption.
Read more about it on the dedicated product discussion.
Done by @irevoire and @dureuill in https://github.com/meilisearch/meilisearch/pull/5264
Use remote federated search requests together with the /multi-search
route to query multiple Meilisearch instances simultaneously. This is particularly useful when handling very large databases.
First, use the /experimental-features
route to enable network
:
curl \
-X PATCH 'MEILISEARCH_URL/experimental-features/' \
-H 'Content-Type: application/json' \
--data-binary '{
"network": true
}'
Next, set up your network of Meilisearch instances with a call to PATCH /network
configuring one self
and multiple remotes
:
curl \
-X PATCH 'MEILISEARCH_URL/network/' \
-H 'Content-Type: application/json' \
--data-binary '{
"remotes": {
"ms-0": {
"url": "http://ms-1235.example.meilisearch.io",
"searchApiKey": "Ecd1SDDi4pqdJD6qYLxD3y7VZAEb4d9j6LJgt4d6xas"
},
"ms-1": {
"url": "http://ms-4242.example.meilisearch.io",
"searchApiKey": "hrVu-OMcjPGElK7692K7bwriBoGyHXTMvB5NmZkMKqQ"
}
},
"self": "ms-0"
}'
Repeat this process with every instance in your network, then add documents. Do not send the same documents to different instances. Finally, make a /multi-search
query with the new federationOptions.remote
:
curl \
-X PATCH 'MEILISEARCH_URL/multi-search/' \
-H 'Content-Type: application/json' \
--data-binary '{
"federation": {},
"queries": [
{
"q": "Batman returns dark",
"indexUid": "movies",
"federationOptions": {
"remote": "ms-0"
}
},
{
"q": "Batman returns dark",
"indexUid": "movies",
"federationOptions": {
"remote": "ms-1"
}
}
]
}'
Find more information about this feature on the public usage page
- Improve task auto-batching by exposing a way to limit the total size of a batch by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5223
- Add Prometheus metrics to measure task queue latency by @takaebato in https://github.com/meilisearch/meilisearch/pull/5178
- Speed up the request to list indexes by @irevoire in https://github.com/meilisearch/meilisearch/pull/5166
- Add support for
GITHUB_TOKEN
authentication in installation script by @Sherlouk in https://github.com/meilisearch/meilisearch/pull/5216 - Expose a new
usedDatabaseSize
field on the/stats
route by @ManyTheFish in https://github.com/meilisearch/meilisearch/pull/5314 - Expose embeddings info on the
/stats
route by @ManyTheFish in https://github.com/meilisearch/meilisearch/pull/5341
- Improve error message when an attribute is not filterable by @jameshiew in https://github.com/meilisearch/meilisearch/pull/5135
- Make sure indexing embeddings is using the right number of threads by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5322
- Dumpless upgrade will no longer corrupt the database in case of early stop by @irevoire in https://github.com/meilisearch/meilisearch/pull/5316
- Fix export/import dump that contains batches data by @irevoire in https://github.com/meilisearch/meilisearch/pull/5310
- Make sure indexing geo points works with PUT requests by @dureuill in https://github.com/meilisearch/meilisearch/pull/5332
- Add back semantic search timeout from v1.11.3 by @dureuill in https://github.com/meilisearch/meilisearch/pull/5339
- Dependency updates
- Upgrade dependencies to fix the
idna
severity issue by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5218
- Upgrade dependencies to fix the
- CIs and tests
- Split tests into separate files by @K-Kumar-01 in https://github.com/meilisearch/meilisearch/pull/5134, https://github.com/meilisearch/meilisearch/pull/5171, https://github.com/meilisearch/meilisearch/pull/5174
- Remove obsolete test code by @K-Kumar-01 in https://github.com/meilisearch/meilisearch/pull/5173
- Fix flaky batches test by @irevoire in https://github.com/meilisearch/meilisearch/pull/5175
- Remove hard coded task IDs to prevent flaky tests by @mhmoudr in https://github.com/meilisearch/meilisearch/pull/5182
- Improve test performance of get_index.rs by @DerTimonius in https://github.com/meilisearch/meilisearch/pull/5210
- Other
- Instruct users to create custom reports on the benchboard by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5029
- Fix typo in a comment by @eltociear in https://github.com/meilisearch/meilisearch/pull/5184
- Replace hardcoded string with constants by @Gnosnay in https://github.com/meilisearch/meilisearch/pull/5169
- Refactor index-scheduler by @irevoire in https://github.com/meilisearch/meilisearch/pull/5199
- Refactor indexer by @dureuill in https://github.com/meilisearch/meilisearch/pull/5168
- Auto-generate OpenAPI spec by @irevoire in https://github.com/meilisearch/meilisearch/pull/4867 & https://github.com/meilisearch/meilisearch/pull/5231
- Merge bitmaps by using
Extend::extend
by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5221 - Fix issue introduced by dumpless upgrade by @manojks1999 in https://github.com/meilisearch/meilisearch/pull/5284
- Send the OSS analytics once per day instead of once per hour by @irevoire in https://github.com/meilisearch/meilisearch/pull/5312
- Log more metrics around HTTP embedder requests and ANNs in arroy by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5288
- Fix internal error when not correctly using the
documentTemplateMaxBytes
parameter by @dureuill in https://github.com/meilisearch/meilisearch/pull/5306 - When a batch is deleted it no longer keeps some internal data in the DB @Kerollmops and @irevoire in https://github.com/meilisearch/meilisearch/pull/5272
- Fix a rare stack overflow when using remote embedders by @Kerollmops with the help of @dureuill https://github.com/meilisearch/meilisearch/pull/5294 (from v1.12.8)
- Fix Dotnet tests in sdks-tests.yml by @curquiza in https://github.com/meilisearch/meilisearch/pull/5291
- Debug log the channel congestion by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5177
- Improve unexpected panic message by @irevoire in https://github.com/meilisearch/meilisearch/pull/5300
- Introduce a compaction subcommand in meilitool by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5235
- Expose a route to get the file content associated with a task by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5326
- Fix workload sha for the benchmark by @ManyTheFish in https://github.com/meilisearch/meilisearch/pull/5342
❤️ Thanks again to our external contributors:
- Meilisearch: @takaebato, @Sherlouk, @jameshiew, @K-Kumar-01, @mhmoudr, @eltociear, @Gnosnay, @DerTimonius, @manojks1999, @ellnix, @Guikingone, @migueltarga
v1.13.0-rc.3 🕊️
[!WARNING] Since this is a release candidate (RC), we do NOT recommend using it in a production environment. Is something not working as expected? We welcome bug reports and feedback about new features.
- Embeddings stats by @ManyTheFish in https://github.com/meilisearch/meilisearch/pull/5341
- Fix workload sha by @ManyTheFish in https://github.com/meilisearch/meilisearch/pull/5342
Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.13.0-rc.2...v1.13.0-rc.3