screenshot-to-code

This simple app converts a screenshot to code (HTML/Tailwind CSS, or React or Bootstrap). It uses GPT-4 Vision to generate the code and DALL-E 3 to generate similar-looking images. You can now also enter a URL to clone a live website!

https://github.com/abi/screenshot-to-code/assets/23818/6cebadae-2fe3-4986-ac6a-8fb9db030045

See the Examples section below for more demos.

🚀 Try It Out!

🆕 Try it here (bring your own OpenAI key - your key must have access to GPT-4 Vision. See FAQ section below for details). Or see Getting Started below for local install instructions.

🌟 Recent Updates

• Dec 11 - Start a new project from existing code (allows you to come back to an older project)
• Dec 7 - 🔥 🔥 🔥 View a history of your edits, and branch off them
• Nov 30 - Dark mode, output code in Ionic (thanks @dialmedu), set OpenAI base URL
• Nov 28 - 🔥 🔥 🔥 Customize your stack: React or Bootstrap or TailwindCSS
• Nov 23 - Send in a screenshot of the current replicated version (sometimes improves quality of subsequent generations)
• Nov 21 - Edit code in the code editor and preview changes live thanks to @clean99
• Nov 20 - Paste in a URL to screenshot and clone (requires ScreenshotOne free API key)
• Nov 19 - Support for dark/light code editor theme - thanks @kachbit
• Nov 16 - Added a setting to disable DALL-E image generation if you don't need that
• Nov 16 - View code directly within the app
• Nov 15 - You can now instruct the AI to update the code as you wish. It is helpful if the AI messed up some styles or missed a section.

🛠 Getting Started

The app has a React/Vite frontend and a FastAPI backend. You will need an OpenAI API key with access to the GPT-4 Vision API or Google API key for Gemini Pro Vision.

Model Configuration can also be found on the settings modal of the web ui.

Run the backend (I use Poetry for package management - pip install poetry if you don't have it):

Run with GPT-4 Vision

cd backend
echo "OPENAI_API_KEY=sk-your-key" > .env
poetry install
poetry shell
poetry run uvicorn main:app --reload --port 7001

Run with Gemini Pro Vision

You can also run this with Gemini Pro Vision by using the following command line instead:

cd backend
echo "GOOGLE_API_KEY=sk-your-key" >.env
poetry install
poetry shell
MODEL=gemini poetry run uvicorn main:app --reload --port 7001

Run the frontend:

cd frontend
yarn
yarn dev

Open http://localhost:5173 to use the app.

If you prefer to run the backend on a different port, update VITE_WS_BACKEND_URL in frontend/.env.local

For debugging purposes, if you don't want to waste GPT4-Vision credits, you can run the backend in mock mode (which streams a pre-recorded response):

MOCK=true poetry run uvicorn main:app --reload --port 7001

Configuration

• You can configure the OpenAI base URL if you need to use a proxy: Set OPENAI_BASE_URL in the backend/.env or directly in the UI in the settings dialog

Docker

If you have Docker installed on your system, in the root directory, run:

echo "OPENAI_API_KEY=sk-your-key" > .env
docker-compose up -d --build

The app will be up and running at http://localhost:5173. Note that you can't develop the application with this setup as the file changes won't trigger a rebuild.

🙋‍♂️ FAQs

• I'm running into an error when setting up the backend. How can I fix it? Try this. If that still doesn't work, open an issue.
• How do I get an OpenAI API key? See https://github.com/abi/screenshot-to-code/blob/main/Troubleshooting.md
• How can I provide feedback? For feedback, feature requests and bug reports, open an issue or ping me on Twitter.

📚 Examples

NYTimes

Original	Replica

Instagram page (with not Taylor Swift pics)

https://github.com/abi/screenshot-to-code/assets/23818/503eb86a-356e-4dfc-926a-dabdb1ac7ba1

Hacker News but it gets the colors wrong at first so we nudge it

https://github.com/abi/screenshot-to-code/assets/23818/3fec0f77-44e8-4fb3-a769-ac7410315e5d

🌍 Hosted Version

🆕 Try it here (bring your own OpenAI key - your key must have access to GPT-4 Vision. See FAQ section for details). Or see Getting Started for local install instructions.