• Do not panic when doing a dumpless upgrade on empty indexes with configured embeddings by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/6102

Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.32.1...v1.32.2

Introduce a MEILI_EXPERIMENTAL_DISABLE_FID_BASED_DATABASES_CLEANUP env var to opt out of the field ID-based database cleanup when upgrading a Meilisearch <1.32.0.

by @ManyTheFish in https://github.com/meilisearch/meilisearch/pull/6096

Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.32.0...v1.32.1

Introduces comprehensive progress tracking and logging for search operations in Meilisearch. It adds detailed timing information for each step of the search process, enabling better observability and performance analysis.

by @ManyTheFish in https://github.com/meilisearch/meilisearch/pull/6072

We accelerate document indexing by processing a large number of tasks in batches or a large number of records in parallel. We expedited the preparation of the payloads by extracting the various changes and assigning internal IDs in parallel. We achieved a 7x speedup on a four-million-document insertion using four CPUs, and the performance scales with the number of CPUs.

The indexedDocuments field in tasks using skipCreation no longer precisely reflects the number of document operations performed, specifically for POST and PUT operations. This count may be higher than the actual number of operations, but it doesn't affect the computation; only the reported count is impacted. We prioritize speed over perfect accuracy here, and the documents are still correctly indexed as before.

by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/6080

Fixed vector sort bucketing so documents with identical similarity scores are grouped together, ensuring subsequent ranking rules are applied correctly.

by @dureuill in https://github.com/meilisearch/meilisearch/pull/6081

Fixes a bug where changing searchableAttributes from ["*"] to a subset of fields left orphaned data in fid-based databases, causing corruption and warnings during search.

by @ManyTheFish in https://github.com/meilisearch/meilisearch/pull/6076

Bumps hannoy to v0.1.3-nested-rtxns, which fixes graph-related recall issues and adds a method to rebuild graph links to recover previously malformed graphs. Also fixes a minor issue in the dumpless upgrade flow where the upgrade description was not displayed correctly and related operations were not properly associated with the upgrade.

by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/6055

Updated the JavaScript SDK tests to use pnpm instead of yarn in CI workflows, switching the package manager across test configurations to ensure the SDK test suite runs correctly and consistently with the current tooling.

by @Strift in https://github.com/meilisearch/meilisearch/pull/6075

Updated the SDK tests CI workflow for the JavaScript SDKs

by @curquiza in https://github.com/meilisearch/meilisearch/pull/6050

Fix Stacked Borrows violation in IterMut.

by @dependabot[bot] in https://github.com/meilisearch/meilisearch/pull/6087

Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.31.0...v1.32.0

The recently introduced S3-streaming snapshots feature is now available as an Enterprise Edition feature. From now on, you'll need a license to use this feature when using Meilisearch as a self-hosted solution. Note that this version introduces a breaking change regarding the S3-streaming snapshot feature, which is no longer available in the Community Edition; however, on-disk snapshots remain accessible. You can read our BUSL license for contact information, if you like. Note that if you are using the Community Edition of Meilisearch between version 1.25 and this release, you can freely use the S3 Streaming feature without an Enterprise Edition license.

by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/6057

We just introduced support for IRSA authentication to do snapshots on AWS. IRSA allows the use of short-lived access and secret keys to upload snapshots through S3. This feature is available under the Enterprise Edition and can be accessed through two experimental CLI parameters.

by @paulden in https://github.com/meilisearch/meilisearch/pull/6044

Adds an optional skipCreation boolean query parameter to POST and PUT on /indexes/{index}/documents. When set to true, documents that don't exist in the index are silently ignored rather than created. Default is false, preserving existing behavior.

by @YoEight in https://github.com/meilisearch/meilisearch/pull/6053

• Fix actix payload error handling by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/6052
• Bump hannoy to v0.1.2 by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/6063

• Run windows in the PR CI by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/6058
• Make CI test titles more human friendly by @curquiza in https://github.com/meilisearch/meilisearch/pull/6062

This change fixes the parenting of the tasks when doing an index swap.

This hotfix avoids the encounter of following error during a search request:

{
  "message": "Inside `<query>`: Index `<index_uid>`: internal: missing <field_id> in the fieldids weights mapping.",
  "code": "internal",
  "type": "internal",
  "link": "https://docs.meilisearch.com/errors#internal"
}

Since v1.19.0, Meilisearch Enterprise Edition allows the automatic sharding of documents over multiple Meilisearch instances, enabling scaling to more documents than a single instance would accommodate.

Meilisearch v1.30.0 adds the ability to modify the number of participants in sharding, without having to start over sending documents to a new cluster containing the number of desired machines.

To make this possible, Meilisearch v1.30.0 introduces breaking changes. These breaking changes only affect the users of the experimental network feature who enabled automatic sharding (network.sharding = true). Users of the stable features are not affected.

// PATCH /network
{
  "self": "ms0",
  "leader": "ms0", // must be equal to `self`
  "remotes": {
    "ms0": {
      "url": "URL_OF_MS0",
      "searchApiKey": "SEARCH_API_KEY_OF_MS0",
      "writeApiKey": "WRITE_API_KEY_OF_MS0",
    },
    "ms1": {
      "url": "URL_OF_MS1",
      "searchApiKey": "SEARCH_API_KEY_OF_MS1",
      "writeApiKey": "WRITE_API_KEY_OF_MS1",
    }
  }
}

{
  "remotes": {
  // add information about the new remote
  "ms2": {
      "url": "URL_OF_MS2",
      "searchApiKey": "SEARCH_API_KEY_OF_MS2",
      "writeApiKey": "WRITE_API_KEY_OF_MS2",
    }
  // information about existing remotes does not need to be repeated
}

{
  "remotes": {
  // set removed remote to null
  "ms2": null
  // information about existing remotes does not need to be repeated
}

by @dureuill in https://github.com/meilisearch/meilisearch/pull/6000

• Fix macos-amd64 compilation by @dureuill in https://github.com/meilisearch/meilisearch/pull/6037
• Move to Rust v1.91.1 by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/6039
• Remove risk of command injection in CI by @curquiza in https://github.com/meilisearch/meilisearch/pull/6041

🌈 The Meilisearch binary is available again for meilisearch-enterprise-macos-amd64 and meilisearch-macos-amd64. It was not available for Meilisearch v1.29.

Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.29.0...v1.30.0

The git binary must now be present at build time to populate the commitSha1 field of the /version field.

This change was made for build performance reasons.

by @dureuill in https://github.com/meilisearch/meilisearch/pull/6030

The new settings indexer scales better, supports near-instant cancellations and displays the progress of the indexing operation.

Previously, the new settings indexer was enabled only if the only changes in a settings batch were to embedder settings. In Meilisearch v1.29.0, the new settings indexer will be enabled if the change is any combination of:

• searchableAttributes
• exactAttributes
• proximityPrecision
• embedders (as before)

Any other change to settings appearing in a batch will cause the batch to use the legacy settings indexer. Additionally, the new settings indexer is currently disabled by default for Cloud users.

OSS users who would like to disable the new settings indexer should pass the MEILI_EXPERIMENTAL_NO_EDITION_2024_FOR_SETTINGS environment variable to true.

by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/5983

Meilisearch v1.21.0 introduced a new vector store backend providing better performance and relevancy.

Starting with v1.29.0, any newly created index will default to the new backend. Existing indexes will be left unchanged.

It is still possible to explicitly choose the vector store backend, please refer to the relevant experimental feature discussion page.

by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/6004

You can now select models with the XLM Roberta architecture when generating embeddings locally on CPU or GPU with the huggingFace embedder.

by @qdequele in https://github.com/meilisearch/meilisearch/pull/6018

• Fix rare internal error in the experimental vector store by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/6032

Last week was a Quality of Life week, and while we still had improvements in the pipe, the bulk of our work was dedicated to maintenance tasks.

Most notably, the CI is now faster, going from over one hour to less than 30 minutes, and also more reliable as it automatically tests the dumpless upgrade.

• New workload to run declarative tests. This is, in particular, very useful to automatically test the dumpless upgrade. By @Mubelotix in https://github.com/meilisearch/meilisearch/pull/5861
• The code that applies the dumpless upgrade migrations was rewritten to make it easier to maintain. By @dureuill in https://github.com/meilisearch/meilisearch/pull/6029
• We no longer run macOS and Windows tests at PR time. These CI jobs were very slow and given their limited impact they will only be run on the nightly builds. Ny @Kerollmops in https://github.com/meilisearch/meilisearch/pull/6021
• Send notifications for Kubernetes integration when releasing by @curquiza in https://github.com/meilisearch/meilisearch/pull/6023
• Introduce xtask sub-command to generate prototype tag names with the expected nomenclature by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/6022
• Fix misc. CI issues by @dureuill in https://github.com/meilisearch/meilisearch/pull/6026

Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.28.2...v1.29.0

This release fixes a bug affecting the Prometheus metrics route in versions 1.28.0 and 1.28.1, specifically when the instance has too many tasks. The issue is visible as high memory usage and could cause the instance to be OOM-killed. If you are using the /metrics route, we recommend deleting all succeeded or failed functions in the index using the dedicated route or upgrading to at least v1.28.2.

• Limit the number of retrieved tasks to one in the metrics by @Kerollmops in https://github.com/meilisearch/meilisearch/pull/6024

Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.28.1...v1.28.2

This release features two fixes: one to ensure that we correctly upload the Linux-amd64 binaries for the Community and Enterprise editions, and one contributed by an external developer to ensure that we still return documents that don't contain the sortable attribute after those that do.

• Fix sort on /documents endpoint when field has no values by @EclipseAditya in https://github.com/meilisearch/meilisearch/pull/6016
• Fix release CI after we introduced the enterprise edition by @paulden in https://github.com/meilisearch/meilisearch/pull/6020

• @EclipseAditya made their first contribution in https://github.com/meilisearch/meilisearch/pull/6016

Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.28.0...v1.28.1

This release introduces improvements to language support and separates the community and enterprise binary editions. We now offer binaries under the BUSL-1.1 license, identified by the "enterprise" term in their names, in addition to our MIT-licensed binaries, which retain their original names. Docker images for the enterprise edition are available in the getmeili/meilisearch-enterprise repository.

• Separation of EE and CE. CE remains the default binary, and the name does not change by Louis on #6011

• Charabia v0.9.9: introduce a better word segmentation for Thai, Khmer, and German languages by @ManyTheFish in #6007
• Expose batch progress traces on the metrics route to improve the indexing debugging experience by @Kerollmops in #5956

• Remove version from the name of the test job in CI by @paulden in #6012
• Upgrade most of the dependencies by @Kerollmops in #6002
• Build x86 and ARM images on GitHub-hosted runners by @paulden in #6003
• Fix SDKs tests to use the enterprise edition and continue testing the sharding feature by @curquiza in #6013

• @paulden made their first contribution in #6003

Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.27.0...v1.28.0