Experimental features

[dureuill]

Pull Request

Related issue

• Fixes Support for enabling some experimental features at runtime #3857
• Related to Display ranking details at search #3771

What does this PR do?

Example

Using the feature to enable `scoreDetails`

❯ curl \
  -X POST 'http://localhost:7700/indexes/index-word-count-10-count/search' \
  -H 'Content-Type: application/json' \
  --data-binary '{ "q": "Batman", "limit": 1, "showRankingScoreDetails": true, "attributesToRetrieve": ["title"]}' | jsonxf

{
  "message": "Computing score details requires enabling the `score details` experimental feature. See https://github.com/meilisearch/product/discussions/674",
  "code": "feature_not_enabled",
  "type": "invalid_request",
  "link": "https://docs.meilisearch.com/errors#feature_not_enabled"
}

❯ curl \
  -X PATCH 'http://localhost:7700/experimental-features/' \
  -H 'Content-Type: application/json'  \
--data-binary '{
    "scoreDetails": true
  }'
{"scoreDetails":true,"vectorSearch":false}

❯ curl \
  -X POST 'http://localhost:7700/indexes/index-word-count-10-count/search' \
  -H 'Content-Type: application/json' \
  --data-binary '{ "q": "Batman", "limit": 1, "showRankingScoreDetails": true, "attributesToRetrieve": ["title"]}' | jsonxf
{
  "hits": [
    {
      "title": "Batman",
      "_rankingScoreDetails": {
        "words": {
          "order": 0,
          "matchingWords": 1,
          "maxMatchingWords": 1,
          "score": 1.0
        },
        "typo": {
          "order": 1,
          "typoCount": 0,
          "maxTypoCount": 1,
          "score": 1.0
        },
        "proximity": {
          "order": 2,
          "score": 1.0
        },
        "attribute": {
          "order": 3,
          "attribute_ranking_order_score": 1.0,
          "query_word_distance_score": 1.0,
          "score": 1.0
        },
        "exactness": {
          "order": 4,
          "matchType": "exactMatch",
          "score": 1.0
        }
      }
    }
  ],
  "query": "Batman",
  "processingTimeMs": 3,
  "limit": 1,
  "offset": 0,
  "estimatedTotalHits": 46
}

User standpoint

• Add new route GET/POST/PATCH/DELETE /experimental-features to switch on or off some of the experimental features in a manner persistent between instance restarts
• Use these new routes to allow setting on/off the following experimental features:
  • vector store TODO: fill in issue
  • score details (related to Display ranking details at search #3771)
• Make the way of checking feature availability and error message uniform for the Prometheus metrics experimental feature
• Save the enabled features in dump, restore from dumps
• TODO: tests:
  • Test new security permissions (do they allow access with ALL, do they prevent access when missing)
  • Test dump behavior, in particular ability to import existing v6 dumps
  • Test basic behavior when calling the rule

Implementation standpoint

• New DB "experimental-features"
• dumps are modified to save the state of that new DB as a experimental-features.json file, that is then loaded back when importing the dump. This doesn't change the dump version, as the file is optional and it missing will not cause the dump to fail

[Kerollmops]

Thank you very much 🍩
bors merge

[meili-bors]

Build failed:

• Tests on macos-12

[Kerollmops]

I hope it's just a spurious test 🤞
bors merge

[dureuill]

I hope it's just a spurious test 🤞

Looks like many workflows failed on macos this afternoon.

Maybe a shared runner has trouble coping with multiple PRs at once?

[meili-bors]

Build succeeded:

• Run Clippy
• Run Rustfmt
• Run tests in debug
• Tests on macos-12
• Tests on ubuntu-18.04
• Tests on windows-2022