"I Know It's Documented, But I Can't Find It"
A documentation site with poor search is like a library where all the books are shelved randomly with no catalog. The knowledge exists, but the cost of finding it is so high that engineers give up and ask a colleague instead — or worse, rediscover the answer themselves. Good search is not a nice-to-have; it is what determines whether your documentation investment pays off.
Standard static site generators ship with client-side search: at build time, they generate a JSON index of all content, and the search bar queries that JSON file in the browser. This works well for small doc sites (under a few hundred pages) but degrades quickly as the corpus grows. The JSON file becomes large, the search becomes slow, and relevance ranking is primitive.
The Solution: Dedicated Search Engines
Platform engineers integrate dedicated search services for large documentation sites:
-
Algolia DocSearch: The industry standard for documentation search (SaaS). It provides instant search (results appear as you type), typo tolerance, faceted filtering, and relevance ranking tuned specifically for technical documentation. The free tier covers open-source projects. Algolia crawls your site on a schedule and maintains the index automatically.
-
Meilisearch: An open-source, self-hosted alternative to Algolia written in Rust. Extremely fast (sub-50ms queries), easy to deploy as a container in your Kubernetes cluster, and GDPR-friendly because your data never leaves your infrastructure. Ideal for internal documentation that cannot be sent to a third-party SaaS.
-
OpenSearch / Elasticsearch: More complex to operate but offers the most flexibility. Used when documentation search needs to be integrated with a broader observability or knowledge platform.
How CI-Driven Indexing Works
The search index must stay in sync with the documentation. This is solved with a CI pipeline step:
┌─────────────────────────────────────────────────────┐
│ CI Pipeline (runs on every merge to main) │
│ │
│ 1. Build static site (mkdocs build) │
│ 2. Deploy HTML to hosting (S3 / GitHub Pages) │
│ 3. Run indexer: │
│ - Crawl generated HTML │
│ - Extract: page title, headings, body text, │
│ code blocks │
│ - Push structured records to Search Engine API │
│ 4. Search index is now current │
└─────────────────────────────────────────────────────┘
A minimal CI indexing step is just a Python script — using a scraping library to crawl the deployed site — invoked with the site URL, the search engine's endpoint and API key, and the target index name.
The key architectural decision is when to index: indexing after deployment (not before) ensures the search index always reflects what users actually see, not what was just built.