abi/screenshot-to-code
Drop in a screenshot and convert it to clean code (HTML/Tailwind/React/Vue)
How screenshot-to-code is put together
screenshot-to-code is an AI-powered tool that converts screenshots, mockups, Figma designs, and screen recordings into functional frontend code (HTML/Tailwind, React, Vue, Bootstrap, Ionic). It is a monorepo with a React/Vite/TypeScript frontend and a Python FastAPI backend, wired together via a WebSocket streaming endpoint (/generate-code) plus supporting HTTP routes. The backend orchestrates multiple LLM providers (OpenAI, Anthropic, Gemini) and Replicate for image generation, running an agentic runner with tool-calling (asset extraction, image editing, headless-Chromium screenshot preview via Playwright). Data flows: frontend captures image/video input and settings, streams over WebSocket to the backend, which builds prompts, calls provider LLMs, streams generated code variants back, and stores assets/logs on the local filesystem.
Languages
Frameworks
Datastores
Infrastructure
Major components
FastAPI backend (routes)
Exposes HTTP/WebSocket endpoints for code generation, screenshots, evals, exports, design systems, capabilities, and agent runs.
Agent runner/engine
Orchestrates the agentic code-generation loop with provider abstraction and tool-calling across OpenAI/Anthropic/Gemini.
LLM providers layer
Provider-specific adapters (anthropic, openai, gemini) implementing a common interface for streaming completions and image handling.
Agent tools
Implements callable tools: asset extraction, local assets, screenshot preview, and their parsing/runtime/summaries.
Prompts subsystem
Builds create/update/system prompts, message pipelines, design-system and policy prompt logic for each supported stack.
Image generation
Generates/edits images and removes backgrounds via Replicate for use in generated output.
Evals & fs_logging
Runs code-gen and asset-extraction benchmarks and persists prompt reports, agent runs, and cost/token usage to the filesystem.
React frontend
UI for uploading input, configuring models/stacks/settings, streaming generation, viewing/editing code and evals.
Refreshed the default model lineup based on graded test results and improved the tools for reviewing those tests.
2 weeksWeek of 2026-07-27
Week of 2026-07-27latest
Refreshed the default model lineup based on graded test results and improved the tools for reviewing those tests.
Week of 2026-07-20
A busy week adding new AI models, expanding evaluation and logging tools, tightening cost limits, and improving image and generation-control features.
Over these weeks the project focused heavily on building out its evaluation tooling and keeping its lineup of AI models current, while also improving the app's image-handling features and cost controls. The team added ways to run, log, and compare model outputs side by side, and refreshed which models are used by default based on those test results. Recent work polished the generation controls and tightened spending safeguards.
Week by week
2026-07-27Refreshed the default model lineup based on graded test results and improved the tools for reviewing those tests.latest5 changes
Chore
Updated default text models from test results
The mix of AI models used for text-to-code was refreshed based on judged evaluations, swapping in newer models at similar cost and speed.
Chore
Updated default image models from test results
The mix of AI models used for image-to-code was refreshed based on graded evaluation runs.
Feature
Support for text-based test sets
The evaluation system can now run text-brief test sets in addition to image-based ones.
Feature
Side-by-side comparison viewer
Added a viewer to compare evaluation outputs from different models next to each other.
Fix
Flag empty agent runs as failures
Agent runs that finish without producing any output are now correctly marked as failed.
2026-07-20A busy week adding new AI models, expanding evaluation and logging tools, tightening cost limits, and improving image and generation-control features.6 changes
Feature
Added new AI model options
New model variants including Gemini 3.6 Flash, Claude Opus 5, and a GPT-5.6 low-thinking option were added with pricing and configuration.
Feature
Evaluation matrix with sessions and logging
Added a matrix view of test sessions plus full agent-run logging backed by a searchable index and a dedicated viewer.
Feature
Batch image editing and background removal
Image editing and background removal can now process multiple independent items at once in parallel.
Fix
Spending safeguards on generations
Generations are now aborted if they exceed a $3 cost ceiling to prevent runaway spending.
Feature
Better generation controls and stack picker
Generation controls were unified into a single reusable card, with framework logos added to the technology-stack selector.
Refactor
More accurate, faster asset extraction
Screenshot asset extraction was made more accurate and robust and upgraded to run on Gemini 3.6 Flash.
Dependencies and code review
Dependency advisories
Security Watch
No known advisories across 0 scanned dependencies.
No known advisories in the scanned dependencies.
Code review
No concrete code-level issues confirmed.
Get this report every week for your repos.
GitZoid learns each repo, reports what changed, and flags what needs attention. One flat price for the whole team.
$19 a month, flat · First 10 outputs free · No card required