Deterministic First, LLM Second: An Advisory CI Pre-Screen
A new pre-screening system for pull requests has been implemented, replacing the previous Gemini system that frequently encountered errors. The new system utilizes a deterministic classifier to evaluate submissions based on specific rules, while a Groq LLM provides advisory summaries without affecting the primary decision. This design allows for seamless operation even if the LLM encounters issues, ensuring consistent CI outcomes.
- ▪The old PR review system faced numerous errors, leading to the development of a new pre-screening system.
- ▪The new system features a deterministic classifier that evaluates submissions and an LLM that provides advisory summaries.
- ▪The LLM's outputs do not block the primary CI decision, ensuring that the workflow continues even if the LLM fails.
Opening excerpt (first ~120 words) tap to expand
try { if(localStorage) { let currentUser = localStorage.getItem('current_user'); if (currentUser) { currentUser = JSON.parse(currentUser); if (currentUser.id === 3842419) { document.getElementById('article-show-container').classList.add('current-user-is-article-author'); } } } } catch (e) { console.error(e); } Jeremy Longshore Posted on May 18 • Originally published at startaitools.com Deterministic First, LLM Second: An Advisory CI Pre-Screen #cicd #githubactions #llm #deterministicsystems The old PR review system ran Gemini on every submission to the claude-code-plugins repo. It broke every time — quota errors, timeout, malformed JSON, the works. On 2026-05-15 I shipped a replacement and deleted the original on the same day. The replacement is structured around two contracts.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at DEV.to (Top).
Discussion
0 commentsMore from DEV.to (Top)
