AMResume
Back to featured work

Public prototype · agent evaluation

Software Lifecycle Env

Deterministic benchmark environment for evaluating AI agents on testing, debugging, and maintenance workflows.

PythonFastAPIPydanticAgent EvaluationHidden Regression ChecksOpenEnvDocker

Overview

Software Lifecycle Env is a public Python benchmark environment that tests whether AI agents can inspect realistic engineering context, apply safe fixes, run validation, and avoid visible-case-only patches.

What I built

I built a benchmark environment where agents inspect an in-memory repository, patch files, run deterministic validation, and submit final answers against visible and hidden checks.

Technical focus

The technical focus is safe evaluation design: typed actions, deterministic scoring, hidden regression checks, anti-gaming controls, FastAPI wrapping, and reproducible benchmark-quality scripts.

Why it matters

This project demonstrates applied AI engineering beyond demos: designing evaluation systems that test realistic software lifecycle behavior.

System architecture

How the components connect

  1. 01

    Task fixture

  2. 02

    Typed observation

  3. 03

    Agent patch / inspect

  4. 04

    Visible + hidden checks

  5. 05

    Deterministic reward

Architecture is derived from the project's public documentation. It is a conceptual flow, not a live system diagram.

Engineering decision

Pair visible feedback with hidden regression tests and deterministic fixtures to resist superficial one-case patches.

Failure mode

An agent can pass the visible example while violating adjacent contracts or hardcoding a shortcut.

How to examine the evidence

Verified on Windows, 14 October 2026: the quality script exited successfully across all three tasks; reference solves scored 0.99 with zero hidden failures.

python scripts/check_benchmark_quality.py
Benchmark methodology and results

A bounded prototype benchmark, not a general claim about all model performance.