Can agents replace the search stack?
The article explores whether AI agents can replace traditional search stacks by using LLMs to interpret queries, call retrieval tools, and rank results. Early experiments show agents significantly improve search quality over basic retrievers, especially when allowed to make multiple diverse queries. However, agents struggle when the needed information isn't in their knowledge base, highlighting limits compared to embedding-based retrieval. The future may involve specialized agentic search models for specific domains, complementing rather than fully replacing current search APIs.
Opening excerpt (first ~120 words) tap to expand
How is search implemented where you work? Probably as a complex set of capabilities on top of retrieval. Our search APIs understand queries, call backend search systems, and finally rerank results. But if we had an agent in the loop, would we need all that? Could we replace search backends with an agent? After all, an agent understands user requests, calls retrieval tools, and evaluates relevance on its own. We see ChatGPT do this all the time, why can’t our search bar? In other words, if you give a basic BM25 backend to an agent, could it take the Search API’s job? Surprisingly good results with simple tool usage Well I’ve measured initial findings (a bit deeper than in the past). I gave agents basic search tools and got the popcorn out to watch.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at Doug Turnbull's Blog.