Improving Search Relevance with Intent Identification and NLP: Inside Zomato's Architecture

Search is the primary discovery engine for hyper-local food aggregators like Zomato. Users rely on the search bar to find restaurants, specific dishes, cuisines, and delivery locations.

However, user queries are rarely clean, structured, or homogeneous. A search bar is an open text field where users combine entities arbitrarily: “best Domino’s Pizza near me”, “MCD burger”, or “garlic bread with cheese dip”.

Directly firing these multi-entity, conversational queries into a standard inverted index (such as Elasticsearch or Apache Solr) leads to poor relevance and degraded user experience. Solving this requires building an intent identification layer using Natural Language Processing (NLP).

---

Why Naive Full-Text Search Fails on Multi-Entity Queries

In a standard search architecture, data is partitioned into indices such as restaurants and dishes (or menus). Each document has weighted fields:

• Title / Name: High weight (e.g., $10$)
• Description / Ingredients: Lower weight (e.g., $1$)

+-------------------------------------------------------------+
|                        Elasticsearch                        |
|                                                             |
|  Index: Restaurants                                         |
|  - title (weight: 10)                                       |
|  - description (weight: 1)                                  |
|  - cuisine_tags (weight: 3)                                 |
|                                                             |
|  Index: Menu / Dishes                                       |
|  - dish_name (weight: 10)                                   |
|  - restaurant_id (foreign key)                              |
|  - ingredients (weight: 1)                                  |
+-------------------------------------------------------------+

When a query matches terms inside the title field, BM25 scoring awards it a disproportionately high relevance score.

The Problem with Unparsed Multi-Entity Queries

Consider the query:

$\text{"best coffee near me"}$

If fired as a raw string across the restaurants index:

1. The search engine checks for the terms best, coffee, near, and me.
2. A restaurant named “Best Coffee Cafe” or “Best Bliss” receives a massive score boost because the tokens best and coffee appear verbatim in its title field.
3. A top-rated cafe down the block named “Blue Tokai” (which serves excellent coffee but lacks the word “best” in its title) gets penalized or buried.

The search engine does not natively understand that:

• coffee $\rightarrow$ Dish / Category
• best $\rightarrow$ Quality / Rating filter
• near me $\rightarrow$ Geolocation filter (latitude/longitude radius)

The Impact of Voice Search and Verbosity

The shift toward voice search amplifies this problem. When typing, users are terse (“dominos pizza”). When speaking, users become conversational and verbose (“Jack’s ka aloo tikki burger aur fries chahiye”). Voice queries introduce filler words, code-mixing, and combined intent that completely break keyword-matching heuristics.

---

Categorizing Food Aggregator Search Queries

User queries generally fall into three common multi-entity patterns:

Query Pattern	Example	Expected Intent
Dish + Dish	”pizza with cheese dip”, “chai and samosa”	Find restaurants whose menus satisfy both dishes simultaneously.
Restaurant + Dish	”MCD burger”, “Domino’s farm fresh pizza”	Locate a specific brand/outlet and narrow the view to that exact dish.
Entity + Modifiers / Location	”veg restaurant koramangala”, “best biryani near me”	Filter by cuisine attribute (veg), geolocation polygon (koramangala), or sort by rating.

Single-intent queries (“Biryani” or “KFC”) are trivial to resolve. The engineering challenge is decomposing compound queries into structured constraints before querying the storage layer.

---

Real-World NLP Challenges in Hyper-Local Search

Building an intent and entity parser for a consumer platform presents several non-trivial challenges:

1. Lack of Labeled Ground Truth: Consumer search logs contain billions of queries, but very little human-labeled data mapping tokens to grammatical entities. Unsupervised and semi-supervised representations are necessary.
2. Multilingualism and Code-Mixing: In diverse markets like India, users constantly mix languages (e.g., Hinglish: “Dal makhni ke saath butter naan”). The system must extract the semantic entities (Dal Makhni, Butter Naan) while safely discarding language-specific filler words (ke saath).
3. Phonetic & Orthographic Variations: Non-standard transliterations are ubiquitous (“Rumali Roti” vs. “Roomali Roti”, “rajma chawal” vs. “rajma rice”).
4. Subword Merging from Speech-to-Text: Voice transcription engines often merge or split words unpredictably (e.g., “friedrice” vs. “fried rice”).

---

The ML Pipeline: Identifying Entities and Intent

To transform unstructured text into structured entity tags, the processing pipeline combines Byte-Pair Encoding (BPE), domain-trained Word2Vec embeddings, and a BiLSTM-CRF sequence tagging network.

Raw Query: "Jack's aloo tikki burger"
         │
         ▼
┌────────────────────────────────────────┐
│  1. Subword Tokenization (BPE)        │
│  - Splits text into robust sub-tokens  │
└──────────────────┬─────────────────────┘
                   │ Tokens
                   ▼
┌────────────────────────────────────────┐
│  2. Domain Embeddings (Word2Vec)      │
│  - Generates dense semantic vectors   │
│  - Trained on menus & merchant data    │
└──────────────────┬─────────────────────┘
                   │ Dense Vectors
                   ▼
┌────────────────────────────────────────┐
│  3. Sequence Tagger (BiLSTM + CRF)    │
│  - Evaluates bidirectional context    │
│  - Predicts optimal entity tag chain   │
└──────────────────┬─────────────────────┘
                   │ Tagged Sequence
                   ▼
Result: [Jack's: RESTAURANT] [aloo tikki burger: DISH]

1. Subword Tokenization with Byte-Pair Encoding (BPE)

Standard whitespace or punctuation-based tokenizers fail on compound food names, typos, and joined tokens (e.g., “cheesedip”).

Byte-Pair Encoding (BPE) provides an iterative, data-driven subword tokenization scheme:

• It builds a vocabulary from base characters and iteratively merges the most frequent adjacent character pairs found across the food and menu corpus.
• If a token has never been seen before, BPE decomposes it into familiar subwords rather than producing an <UNK> (out-of-vocabulary) token.
• It successfully normalizes spelling variants and handles joined speech-to-text outputs.

2. Domain-Specific Embeddings with Word2Vec

Generic, off-the-shelf word vectors (like standard GloVe or BERT models trained on Wikipedia) lack domain nuance for local cuisines and regional dishes.

By training a Word2Vec model directly on the platform’s corpus (millions of dish descriptions, menus, restaurant names, and user queries):

• High-dimensional semantic vectors (typically $128$ or $256$ dimensions) are generated for food vocabulary.
• Words that appear in similar textual contexts occupy adjacent positions in vector space.
• Vector algebra preserves semantic relationships:

$\vec{v}(\text{"rajma chawal"}) \approx \vec{v}(\text{"rajma rice"})$

$\vec{v}(\text{"Domino's"}) - \vec{v}(\text{"Pizza"}) + \vec{v}(\text{"Burger"}) \approx \vec{v}(\text{"McDonald's"})$

This continuous vector space helps the model generalize across synonyms and phonetically similar words.

3. Sequence Tagging with BiLSTM + CRF

Once text is tokenized and transformed into dense vectors, the query is passed into a Named Entity Recognition (NER) / Sequence Tagging architecture:

• Bi-directional LSTM (BiLSTM): Processes the token sequence both forwards (left-to-right) and backwards (right-to-left). This captures full sentence context—essential because a word like “Burger” could be part of a dish (“Veg Burger”) or a brand name (“Burger King”).
• Conditional Random Field (CRF): Placed on top of the BiLSTM layer. While the BiLSTM outputs independent classification probabilities for each token, the CRF models the statistical transitions between sequential tags (e.g., enforcing that a DISH_CONTINUATION tag cannot follow a RESTAURANT_START tag without invalid transitions).

Sequence Tagging Example

For the input query “Jack’s aloo tikki burger”:

Token:       Jack's          aloo         tikki        burger
Vector:      [0.12, ...]     [-0.43, ...] [0.88, ...]  [0.31, ...]
Tag (BIO):   B-RESTAURANT    B-DISH       I-DISH       I-DISH

• B-RESTAURANT: Beginning of Restaurant entity (Jack’s)
• B-DISH: Beginning of Dish entity (aloo)
• I-DISH: Continuation of Dish entity (tikki burger)

---

End-to-End System Architecture

By decoupling entity extraction from document retrieval, the search pipeline becomes an orchestrator that constructs precise queries rather than blasting raw text to Elasticsearch.

sequenceDiagram
    autonumber
    actor User
    participant SearchAPI as Search Service
    participant Gateway as Inference Gateway (Model)
    participant ES as Elasticsearch Cluster

    User->>SearchAPI: GET /search?q=Jack's+aloo+tikki+burger
    SearchAPI->>Gateway: POST /predict-intent {query: