Show HN: Promptloop – create, run, and improve prompt evals from the terminal
Promptloop is an interactive command-line interface designed for creating and evaluating prompts efficiently. It allows users to generate test cases, run evaluations, and produce reports without leaving the terminal. The tool is built on LangChain deepagents and aims to streamline the prompt evaluation process by saving methodologies and results in a structured format.
- ▪Promptloop enables users to create, run, and improve prompt evaluations from the terminal.
- ▪It saves test cases, reports, and prompt history in a designated project folder.
- ▪The tool provides metrics such as response time and text similarity to assess prompt performance.
Opening excerpt (first ~120 words) tap to expand
Promptloop An interactive CLI agent for the full prompt-eval loop: create test cases, run evals, generate reports, and approve prompt diffs without leaving your terminal. Built on LangChain deepagents. The Prompt Eval Loop Agent harnesses are getting better, but prompts still shape what they do. promptloop turns a prompt and eval intent into a repeatable loop: It saves the methodology, test cases, reports, prompt history, and chat checkpoints under .evals/ in the target project.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at GitHub.