Suggest an edit

Improve this article

Refine the answer for “What is search used for?”. Your changes go to moderation before they’re published.

Approval required

Content

What you’re changing

Title (EN)

Short answer (EN)

Shown above the full answer for quick recall.

Answer (EN)

## Short answer

Search is used to quickly and precisely find the needed data among large volumes of information. It improves the user experience, increases conversion, speeds up work with the product, helps analyze data and make decisions, and reduces the load on support.

## Detailed answer

### Main goals of search

- Fast access to data: finding documents, products, users, logs, metrics.
- Navigation and discovery: helping the user understand the catalog and the structure of the content.
- Analytics and monitoring: quickly finding anomalies and incidents in event streams and logs.
- Automation: finding entities in background jobs, triggers, integrations.
- Personalization and recommendations: taking the user's interests into account when serving and ranking results.
- Reducing the load on support: users find answers and content on their own.

### Types of search

- Exact match: by key/ID/field.
- Substring: LIKE/ILIKE, prefix search, autocomplete.
- Full-text: by words, accounting for morphology and relevance.
- Phrase: "search by exact phrase" with word order preserved.
- Filtering/faceted: facet fields (category, price), aggregations.
- Fuzzy: accounting for typos (Levenshtein distance, BK-trees).
- Synonyms and normalization: accounting for word forms and alternative terms.
- Geo search: by coordinates and radius, sorting by distance.
- Vector/semantic: search by meaning using embeddings.

### How it works under the hood

- Indexing: tokenization, normalization, stop words, stemming/lemmatization, n-grams; building an inverted index.
- Data structures: hash tables, B-/B+ trees, tries, an inverted index, BK-trees.
- Relevance and ranking: TF-IDF, BM25, boosting important fields, freshness, popularity, personalization.
- Query processing: parsing operators (AND/OR/NOT), filters, sorting, pagination, highlighting matches, facets.
- Architecture: a separate engine (specialized systems) or FTS built into the database; asynchronous indexing from queues; eventual consistency; sharding, replication; caching.

### UX practices

- Autocomplete and suggestions, popular queries.
- Input debouncing and canceling stale requests, loading states and empty states.
- Highlighting matches, synonyms, typo correction, "did you mean".
- Filters, facets, saved queries, search history, keyboard shortcuts.

### Quality metrics

- Precision / Recall, F1 - accuracy and completeness.
- MRR, NDCG - ranking quality.
- CTR, post-click conversion, time to first result, response speed.

### Performance and scaling

- Database indexes: GIN/GIST for full text, btree for exact matches, covering indexes.
- Caching: of responses, suggestions, facets; warm-up and invalidation.
- Pagination: keyset instead of OFFSET/LIMIT on large data, query timeouts.
- Asynchronous indexing, monitoring index lag, scheduled reindexing and backups.

### Security and compliance

- Filtering by access rights (row-level security), multi-tenancy (tenant_id).
- Masking and minimizing PII, query auditing, rate limiting.

### Internationalization

- Unicode compatibility, correct collation, and case-insensitivity.
- Morphology for different languages, transliteration (for example, Ukrainian/Latin).

### Code examples

### SQL: full-text search in Postgres

```
-- Create a full-text search index on the title and body columns
CREATE INDEX idx_posts_fts ON posts USING GIN (
  to_tsvector('english', coalesce(title,'') || ' ' || coalesce(body,''))
);

-- Search with morphology support (plainto_tsquery is convenient for user input)
SELECT id, title, ts_headline('english', body, plainto_tsquery('english', $1)) AS snippet,
       ts_rank_cd(
         to_tsvector('english', coalesce(title,'') || ' ' || coalesce(body,'')),
         plainto_tsquery('english', $1)
       ) AS rank
FROM posts
WHERE to_tsvector('english', coalesce(title,'') || ' ' || coalesce(body,'')) @@ plainto_tsquery('english', $1)
ORDER BY rank DESC
LIMIT 20 OFFSET $2;

-- Example of a simple LIKE (substring search; less efficient)
SELECT id, title FROM posts WHERE title ILIKE '%' || $1 || '%' LIMIT 20;
```

### Backend: Node.js Express endpoint with pagination and filters

```
import express from 'express';
import { Pool } from 'pg';

const app = express();
const pool = new Pool({ connectionString: process.env.DATABASE_URL });

app.get('/search', async (req, res) => {
  const q = String(req.query.q || '').trim();
  const limit = Math.min(parseInt(String(req.query.limit || '20'), 10), 100);
  const cursor = req.query.cursor ? Number(req.query.cursor) : null; // keyset pagination
  const category = req.query.category ? String(req.query.category) : null;

const params = [] as any[];
  let where = [] as string[];

if (q) {
    params.push(q);
    where.push("to_tsvector('english', coalesce(title,'') || ' ' || coalesce(body,'')) @@ plainto_tsquery('english', $" + params.length + ")");
  }
  if (category) {
    params.push(category);
    where.push('category = $' + params.length);
  }
  if (cursor) {
    params.push(cursor);
    where.push('id > $' + params.length); // keyset: sort by id ASC
  }

const sql = `
    SELECT id, title, ts_rank_cd(
      to_tsvector('english', coalesce(title,'') || ' ' || coalesce(body,'')),
      COALESCE(plainto_tsquery('english', $1), to_tsquery('english', ''))
    ) AS rank
    FROM posts
    ${where.length ? 'WHERE ' + where.join(' AND ') : ''}
    ORDER BY id ASC
    LIMIT ${limit + 1}
  `;

try {
    const { rows } = await pool.query(sql, params);
    const hasMore = rows.length > limit;
    const items = hasMore ? rows.slice(0, limit) : rows;
    res.json({ items, nextCursor: hasMore ? items[items.length - 1].id : null });
  } catch (e) {
    res.status(500).json({ error: 'search_failed' });
  }
});

app.listen(3000);
```

### Frontend: React, debouncing and canceling stale requests

```
import { useEffect, useMemo, useState } from 'react';

function useDebouncedValue(value, delay = 300) {
  const [debounced, setDebounced] = useState(value);
  useEffect(() => {
    const id = setTimeout(() => setDebounced(value), delay);
    return () => clearTimeout(id);
  }, [value, delay]);
  return debounced;
}

export default function SearchBox() {
  const [q, setQ] = useState('');
  const debouncedQ = useDebouncedValue(q, 300);
  const [items, setItems] = useState([]);
  const [loading, setLoading] = useState(false);
  const [error, setError] = useState(null);

const controller = useMemo(() => new AbortController(), [debouncedQ]);

useEffect(() => {
    if (!debouncedQ) { setItems([]); return; }
    setLoading(true); setError(null);

fetch(`/search?q=${encodeURIComponent(debouncedQ)}`, { signal: controller.signal })
      .then(r => {
        if (!r.ok) throw new Error('Network error');
        return r.json();
      })
      .then(data => setItems(data.items || []))
      .catch(err => { if (err.name !== 'AbortError') setError(err.message); })
      .finally(() => setLoading(false));

return () => controller.abort();
  }, [debouncedQ]);

return (
    <div>
      <input
        placeholder="Search..."
        value={q}
        onChange={e => setQ(e.target.value)}
      />
      {loading && <div>Loading...</div>}
      {error && <div>Error: {error}</div>}
      {!loading && !error && items.length === 0 && debouncedQ && <div>Nothing found</div>}
      <ul>
        {items.map(i => <li key={i.id}>{i.title}</li>)}
      </ul>
    </div>
  );
}
```

### Algorithmic angle: binary search

```
function binarySearch(arr, x) {
  let l = 0, r = arr.length - 1;
  while (l <= r) {
    const m = l + ((r - l) >> 1);
    if (arr[m] === x) return m;
    if (arr[m] < x) l = m + 1; else r = m - 1;
  }
  return -1;
}

// Usage: the array must be sorted
console.log(binarySearch([1,3,5,7,9], 7)); // 3
```

### Search implementation checklist

1. Gather requirements: content types, fields, language, load, SLA.
2. Choose the technology: a database with FTS, or a specialized engine; evaluate cost and support.
3. Data schema and indexes: which fields are indexed, morphology, synonyms, tokenization.
4. Indexing pipeline: sources, queues, deduplication, updates, lag monitoring.
5. API: filters, sorting, pagination, highlighting, request-level security.
6. UI/UX: suggestions, autocomplete, facets, empty states, accessibility (a11y).
7. Observability: query logging, metrics, tracing, dashboards, alerts.
8. Quality and relevance: offline/online evaluations, A/B tests, collecting clicks for training.
9. Operations: backups, reindexing without downtime, sharding/replication, an update plan.

Markdown · drag & drop images · ⌘B / ⌘I shortcuts1120 words

For the reviewer

Note to the moderator (optional)

Visible only to the moderator. Helps review go faster.