Query tokenization
When you search for multiple words, the query is automatically tokenized:- Strip non-Arabic characters (keeps only Arabic letters and spaces)
- Normalize the query (unify alef variants, remove diacritics)
- Split by whitespace into individual tokens
Each word in your query becomes a separate token that must be matched independently.
Example tokenization
Input query:الله الرحمن
After processing:
AND logic implementation
All search layers use AND logic: every token must match for a verse to be included in results.Simple search AND logic
The simple search layer usesArray.every() to enforce AND logic:
- For each verse, check if every token appears in the text
- Only return verses where all tokens are found
- If even one token is missing, the verse is excluded
The
.every() method returns true only if all tokens pass the test. If any token fails, the entire verse is rejected.Linguistic search AND logic
The advanced linguistic search performs AND logic through set intersection:- Find matches for each token independently
- Start with the first token’s matching verse IDs
- Filter to keep only IDs that also match the second token
- Continue filtering for each subsequent token
- Result: Only verses that match all tokens
Visual example of set intersection
Query:الله الرحمن
Per-token match types
Each token can match through different layers:- Linguistic match: Lemma or root match found for this token
- Fuzzy match: Fuse.js found an approximate match for this token
- Each token independently tries linguistic search first
- If that fails, it falls back to fuzzy search
- The final results must satisfy all tokens via their respective match types
Multi-word scoring
Scoring accumulates across all matched tokens:Scoring example
Query:الله الرحمنVerse: “بسم الله الرحمن الرحيم” Matches found:
- Token
الله: 1 exact match → +3 points - Token
الرحمن: 1 exact match → +3 points
6
Verses matching more tokens or matching tokens multiple times receive higher scores.
Examples
Example 1: Two-word exact match
Example 2: Three-word query
Example 3: Mixed match types
Query:صلى محمد
Possible results:
- Verse A:
صلى(exact) +محمد(exact) → score: 6, matchType: ‘exact’ - Verse B:
يصلون(lemma for صلى) +محمد(exact) → score: 5, matchType: ‘exact’ - Verse C:
صلاة(root for صلى) +محمد(exact) → score: 4, matchType: ‘exact’
All results contain both tokens, but through different match layers. The
matchType reflects the best match quality found in that verse.Example 4: No results when missing a token
Query:الله قرآن
Verse text: “بسم الله الرحمن الرحيم”
Result: This verse is excluded because it contains الله but not قرآن. Both tokens must match.
Why AND logic?
AND logic provides: ✓ Precision: Results are more specific and relevant✓ User expectation: Matches natural language search behavior
✓ Reduced noise: Eliminates verses that only partially match
✓ Better UX: Users can refine searches by adding words
Common use cases
Searching for phrases
While this isn’t phrase search (word order doesn’t matter), multi-word search effectively finds verses containing all words:Narrowing results
Topic-based search
Performance considerations
The search engine optimizes multi-word queries through:- Early termination: If any token has zero matches, stop immediately
- Set operations: Efficient intersection using
Setdata structure - Deduplication: Verses appear only once even if matched by multiple layers