How do I set up a query autocomplete API for multilingual website search with typo tolerance?

To set up a query autocomplete API for a multilingual website search with typo tolerance, you want an endpoint that can quickly return ranked suggestions as the user types, while handling language-specific text normalization, stemming/segmentation, and fuzzy matching.

1) Define the autocomplete behavior

Your API should support:

• Prefix suggestions: iph → iphone, iphone case
• Multilingual queries: English, French, German, Arabic, Japanese, etc.
• Typo tolerance: iphnoe → iphone
• Ranking by:
  • prefix match quality
  • popularity/click-through
  • recency
  • locale/language match
  • user context if available

Typical endpoint:

GET /autocomplete?q=iphnoe&lang=en&limit=10

Response:

{
  "query": "iphnoe",
  "suggestions": [
    {
      "text": "iphone",
      "score": 0.98,
      "type": "product",
      "lang": "en"
    },
    {
      "text": "iphone case",
      "score": 0.91,
      "type": "product",
      "lang": "en"
    }
  ]
}

---

2) Choose an indexing/search engine strategy

You have three common options:

Option A: Elasticsearch / OpenSearch

Good for production autocomplete, multilingual analysis, and fuzziness.

Use:

• search_as_you_type
• edge n-grams
• completion suggester
• language analyzers
• fuzziness on prefix query (carefully)

Option B: PostgreSQL + trigram indexes

Good for simpler systems and smaller scale.

Use:

• pg_trgm
• language-normalized fields
• weighted ranking

Option C: Dedicated search service

Examples: Algolia, Typesense, Meilisearch.

These often already provide:

• autocomplete
• typo tolerance
• multilingual analyzers
• ranking rules

If you want full control, OpenSearch/Elasticsearch is usually the best fit.

---

3) Prepare multilingual text normalization

For each indexed suggestion/query, store normalized forms:

• lowercase
• Unicode normalization
• accent folding: café → cafe
• language-specific stemming or tokenization
• script normalization if needed

Examples:

• German: handle ß vs ss
• French/Spanish: remove accents
• Arabic: normalize alef forms, remove diacritics
• Japanese/Chinese: use a tokenizer like Kuromoji/ICU

Suggested stored fields

For each suggestion:

{
  "text": "Télévision 4K",
  "normalized_text": "television 4k",
  "lang": "fr",
  "popularity": 1200
}

---

4) Model the autocomplete index

You can index:

• products
• categories
• articles
• previously searched queries
• entities/brands

For autocomplete, precompute suggestion strings.

Example document:

{
  "id": "q123",
  "suggestion": "iphone case",
  "suggestion_folded": "iphone case",
  "lang": "en",
  "popularity": 8421,
  "type": "query"
}

If using Elasticsearch/OpenSearch

Map fields with:

• text using analyzer
• keyword for exact matching
• completion field for suggestions

Example mapping sketch:

{
  "settings": {
    "analysis": {
      "analyzer": {
        "autocomplete_analyzer": {
          "tokenizer": "standard",
          "filter": ["lowercase", "asciifolding", "edge_ngram_filter"]
        }
      },
      "filter": {
        "edge_ngram_filter": {
          "type": "edge_ngram",
          "min_gram": 2,
          "max_gram": 20
        }
      }
    }
  },
  "mappings": {
    "properties": {
      "suggestion": {
        "type": "text",
        "analyzer": "autocomplete_analyzer",
        "search_analyzer": "standard"
      },
      "lang": { "type": "keyword" },
      "popularity": { "type": "rank_feature" }
    }
  }
}

---

5) Implement typo tolerance safely

Typos in autocomplete are tricky because too much fuzziness harms relevance and latency.

Recommended approach:

Use typo tolerance only after a minimum prefix length

• no fuzziness for very short queries like ip
• allow fuzzy matching after 3–4 characters

Use small edit distance

• AUTO or 1
• avoid 2 unless necessary

Apply fuzziness to normalized fields

For example:

• input: iphnoe
• normalized query: iphnoe
• matched against folded field

Combine prefix + fuzzy

Search strategy:

1. exact prefix match
2. prefix on normalized field
3. fuzzy match for near-miss typos
4. popularity fallback

In Elasticsearch/OpenSearch, this might mean:

• match_phrase_prefix
• prefix
• fuzzy
• boosted completion suggester

---

6) Handle language detection and locale routing

For multilingual search, detect or receive the user’s language:

• from browser Accept-Language
• from site locale
• from user profile
• from language detection on input

Then search the matching language index/field first.

Example:

• user locale fr
• query televison
• prefer French suggestions and French analyzers

If your content is truly multilingual in one index, store lang and use per-language analyzers or subfields:

• title.en
• title.fr
• title.ar

---

7) Rank suggestions properly

Ranking should combine several signals:

• prefix closeness
• typo distance
• popularity
• click-through rate
• conversion rate
• freshness/trending
• locale match
• personalization

Example scoring formula:

final_score =
  0.50 * text_match_score +
  0.25 * popularity_score +
  0.15 * locale_match_score +
  0.10 * freshness_score

Boost:

• exact prefix matches
• queries the user has clicked before
• popular suggestions in the user’s region

---

8) API design

A simple API could look like this:

GET /api/autocomplete?q=televison&lang=fr&limit=8

Optional parameters:

• lang — language/locale
• country — regional ranking
• type — products/categories/queries
• context — current category or page
• user_id — personalization
• fuzzy=true — enable typo tolerance
• highlight=true — mark matched parts

Example response:

{
  "query": "televison",
  "lang": "fr",
  "suggestions": [
    {
      "text": "télévision",
      "highlighted": "<em>télévision</em>",
      "score": 0.99
    },
    {
      "text": "télévision 4k",
      "highlighted": "<em>télévision</em> 4k",
      "score": 0.92
    }
  ]
}

---

9) Add caching and performance controls

Autocomplete must be fast, ideally under 50–100 ms.

Use:

• in-memory cache for popular prefixes
• CDN or edge caching if suggestions are static enough
• debounce in frontend, e.g. 150–300 ms
• limit results to 5–10
• reject too-short queries
• timeout fallback to exact prefix only

A good production flow:

1. user types
2. frontend debounces
3. API checks cache
4. search engine query
5. rank and return top N

---

10) Improve with query logs

Store:

• query text
• language
• suggestions shown
• clicked suggestion
• conversion outcome

Then retrain/rerank based on:

• click-through rate
• abandonment
• zero-result queries
• misspellings users frequently make

This helps you:

• add synonyms
• add popular typo variants
• promote trending entities

---

11) Recommended implementation pattern

If you want a robust setup:

Backend

• Node.js / Python / Go API service
• Elasticsearch/OpenSearch as search layer
• Redis for caching hot prefixes
• PostgreSQL or warehouse for analytics/logging

Search documents

Store:

• canonical suggestion text
• language
• popularity
• synonyms
• click metrics

Query flow

1. detect language
2. normalize input
3. query exact prefix index
4. apply fuzzy fallback if needed
5. combine ranking signals
6. return top suggestions

---

12) Practical tips

• Don’t apply fuzzy matching to every keystroke.
• Use language-specific analyzers where possible.
• Normalize accents and punctuation.
• Keep short-query behavior conservative.
• Test per language, because tokenization differs a lot.
• Track latency and relevance separately.

---

Example minimal architecture

Browser
  -> Autocomplete API
      -> Redis cache
      -> OpenSearch index
      -> Ranking layer
      -> Response JSON

---

If you want, I can also give you:

1. a full Elasticsearch/OpenSearch mapping and query example,
2. a Node.js/Python API implementation, or
3. a PostgreSQL trigram-based version.