leeway-ui-check

Leeway UI Check is a tool for verifying the UI modification results of AI coding agents, supporting Web and Flutter scenarios. The basic workflow is: the user provides a reference screenshot and page requirements, the coding agent modifies the page, Leeway uses Playwright to capture the implementation result, and returns pixel, SSIM, layout, and text reports for use in the next round of fixes. Main capabilities - Compare reference images with page screenshots and output normal difference images, strict difference images, and regional issue localization. - Treat text, geometry, interaction, and resource errors as independent blocking conditions rather than mixing them into a single score. - Interact with coding agents through five MCP tools: submit code versions, query feedback, cancel, and deliver. - Save each round's source snapshot, report, and evidence, so the corresponding version can be reviewed before delivery. How to use The verified development environment is Windows / PowerShell, Node 22.16.0, and Python 3.13. For the first installation, clone the repository, run npm ci, create a Python virtual environment and install workers/requirements.txt, then run npm run setup:browser. After that, you can run three preset page variants through npm run build, npm run demo, and npm run cli -- report, and open the local report page (127.0.0.1:4318). When integrating with your own project, configure the reference image, viewport, source directory, and startup command from the task template, create a task and start the worker, then connect the coding agent according to the MCP guide. The overall loop is: agent modifies source code -> submit -> worker screenshots and compares -> report -> agent modifies again. Boundaries and limitations The project describes itself as an early local single-user version. The default score is used for diagnosis, and formal acceptance requires a verified profile; OCR, vision models, security isolation, and automatic convergence with real models have not yet been implemented or accepted. The Harness does not automatically guess the business behavior of buttons; interactions that need to be checked must be written into the task configuration, and the modification suggestions in the report are hypotheses to be verified. The documentation includes usage guides, MCP integration, FAQs, architecture descriptions, scoring calibration, changelogs, contribution guidelines, and security boundaries. The project uses the MIT license.