AgentCheck – Pytest for AI Agents

pygent-test 0.1.1 pip install pygent-test Copy PIP instructions Latest version Released: Apr 28, 2026 Pytest-style behavioral regression testing for AI agents. Navigation Project description Release history Download files Verified details These details have been verified by PyPI Project links Homepage Issues Repository GitHub Statistics Repository Stars: Forks: Open issues: Open PRs: Maintainers ashutosh_023 Unverified details These details have not been verified by PyPI Meta License Expression: MIT SPDX License Expression Author: Ashutosh Rath Tags ai , agents , testing , pytest , evals , regression Requires: Python >=3.10 Classifiers Development Status 3 - Alpha Intended Audience Developers Programming Language Python :: 3 Python :: 3.10 Python :: 3.11 Python :: 3.12 Python :: 3.13 Topic Scientific/Engineering :: Artificial Intelligence Software Development :: Testing Report project as malware Project description Project details Release history Download files Project description AgentCheck AgentCheck is pytest for AI agents. Test behavior, not exact text. Install from source today: python -m pip install -e . Planned published package install: pip install pygent-test What It Does AgentCheck helps you verify agent behavior such as: which tools were used whether tools were used in the expected order whether the agent stayed within a step budget whether the agent claimed success without tool evidence whether behavior regressed against a saved baseline Current Status This repo already supports: repeated-run behavioral tests with @agent_test(...) local baseline and regression comparison CLI commands: test, bless, compare, report pytest integration a plain Python adapter an OpenAI Agents SDK adapter real live OpenAI agent tests in integration_examples/ Quick Start python -m pip install -e . python -m agentcheck.cli test examples Minimal Example from agentcheck import agent_test, expect from examples.booking_agent import SimpleBookingAgent @agent_test(runs=5, agent_factory=SimpleBookingAgent) def test_booking_agent(agent: SimpleBookingAgent): result = agent.run("Book a table for 2 tonight") check = expect(result, collect=True) check.used_tool("restaurant_search") check.used_tool("booking_tool") check.steps_less_than(5) check.did_not_claim_confirmation_without_tool("booking_tool") check.verify() return result Real Agent Testing AgentCheck has been exercised against real OpenAI Agents SDK agents. Use the included live suite: python -m agentcheck.cli test integration_examples or: python -m pytest integration_examples -q The included live tests cover: a single-tool weather assistant a multi-tool research assistant Documentation Use these docs depending on what you need: TECHNICAL_GUIDE.md Detailed developer guide covering architecture, assertions, adapters, and workflows REAL_WORLD_TESTING.md Real OpenAI Agents SDK testing setup and examples Included Demos Passing local demo: python -m agentcheck.cli test examples Intentional failure demo: python -m agentcheck.cli test regression_examples --fail-on-regression Commands python -m agentcheck.cli test <path> python -m agentcheck.cli bless <path> python -m agentcheck.cli compare python -m agentcheck.cli report Smoke Test Run a quick end-to-end validation with: python scripts/smoke_test.py To include the live OpenAI integration tests: python scripts/smoke_test.py --with-live Pytest AgentCheck tests can also run through pytest: python -m pytest examples -q python -m pytest tests -q python -m pytest…

This excerpt is published under fair use for community discussion. Read the full article at Pypi.