Merge branch 'main' into hosted

README.md

@@ -1,27 +1,41 @@
 # screenshot-to-code
 
-This simple app converts a screenshot to code (HTML/Tailwind CSS, or React or Bootstrap or Vue). It uses GPT-4 Vision (or Claude 3) to generate the code and DALL-E 3 to generate similar-looking images. You can now also enter a URL to clone a live website.
+A simple tool to convert screenshots, mockups and Figma designs into clean, functional code using AI.
-
-🆕 Now, supporting Claude 3!
 
 https://github.com/abi/screenshot-to-code/assets/23818/6cebadae-2fe3-4986-ac6a-8fb9db030045
 
+Supported stacks:
+
+- HTML + Tailwind
+- React + Tailwind
+- Vue + Tailwind
+- Bootstrap
+- Ionic + Tailwind
+- SVG
+
+Supported AI models:
+
+- GPT-4 Vision
+- Claude 3 Sonnet (faster, and on par or better than GPT-4 vision for many inputs)
+- DALL-E 3 for image generation
+
 See the [Examples](#-examples) section below for more demos.
 
+We also just added experimental support for taking a video/screen recording of a website in action and turning that into a functional prototype. 
+
+![google in app quick 3](https://github.com/abi/screenshot-to-code/assets/23818/8758ffa4-9483-4b9b-bb66-abd6d1594c33)
+
+[Learn more about video here](https://github.com/abi/screenshot-to-code/wiki/Screen-Recording-to-Code).
+
 [Follow me on Twitter for updates](https://twitter.com/_abi_).
 
 ## 🚀 Try It Out!
 
-🆕 [Try it here](https://screenshottocode.com) (bring your own OpenAI key - **your key must have access to GPT-4 Vision. See [FAQ](#%EF%B8%8F-faqs) section below for details**). Or see [Getting Started](#-getting-started) below for local install instructions.
+🆕 [Try it live on the hosted version](https://screenshottocode.com) (bring your own OpenAI key - **your key must have access to GPT-4 Vision. See [FAQ](#%EF%B8%8F-faqs) section below for details**). Or see [Getting Started](#-getting-started) below for local install instructions.
-
-## 🌟 Recent Updates
-
-- Mar 8 - 🔥🎉🎁 Video-to-app: turn videos/screen recordings into functional apps
-- Mar 5 - Added support for Claude Sonnet 3 (as capable as or better than GPT-4 Vision, and faster!)
 
 ## 🛠 Getting Started
 
-The app has a React/Vite frontend and a FastAPI backend. You will need an OpenAI API key with access to the GPT-4 Vision API.
+The app has a React/Vite frontend and a FastAPI backend. You will need an OpenAI API key with access to the GPT-4 Vision API or an Anthropic key if you want to use Claude Sonnet, or for experimental video support.
 
 Run the backend (I use Poetry for package management - `pip install poetry` if you don't have it):
 
@@ -33,6 +47,8 @@ poetry shell
 poetry run uvicorn main:app --reload --port 7001
 ```
 
+If you want to use Anthropic, add the `ANTHROPIC_API_KEY` to `backend/.env` with your API key from Anthropic.
+
 Run the frontend:
 
 ```bash
@@ -51,25 +67,6 @@ For debugging purposes, if you don't want to waste GPT4-Vision credits, you can
 MOCK=true poetry run uvicorn main:app --reload --port 7001
 ```
 
-## Video to app (experimental)
-
-https://github.com/abi/screenshot-to-code/assets/23818/1468bef4-164f-4046-a6c8-4cfc40a5cdff
-
-Record yourself using any website or app or even a Figma prototype, drag & drop in a video and in a few minutes, get a functional, similar-looking app.
-
-[You need an Anthropic API key for this functionality. Follow instructions here.](https://github.com/abi/screenshot-to-code/blob/main/blog/video-to-app.md)
-
-## Configuration
-
-- You can configure the OpenAI base URL if you need to use a proxy: Set OPENAI_BASE_URL in the `backend/.env` or directly in the UI in the settings dialog
-
-## Using Claude 3
-
-We recently added support for Claude 3 Sonnet. It performs well, on par or better than GPT-4 vision for many inputs, and it tends to be faster.
-
-1. Add an env var `ANTHROPIC_API_KEY` to `backend/.env` with your API key from Anthropic
-2. When using the front-end, select "Claude 3 Sonnet" from the model dropdown
-
 ## Docker
 
 If you have Docker installed on your system, in the root directory, run:
@@ -85,6 +82,8 @@ The app will be up and running at http://localhost:5173. Note that you can't dev
 
 - **I'm running into an error when setting up the backend. How can I fix it?** [Try this](https://github.com/abi/screenshot-to-code/issues/3#issuecomment-1814777959). If that still doesn't work, open an issue.
 - **How do I get an OpenAI API key?** See https://github.com/abi/screenshot-to-code/blob/main/Troubleshooting.md
+- **How can I configure an OpenAI proxy?** - you can configure the OpenAI base URL if you need to use a proxy: Set OPENAI_BASE_URL in the `backend/.env` or directly in the UI in the settings dialog
+- **How can I update the backend host that my front-end connects to?** - Configure VITE_HTTP_BACKEND_URL and VITE_WS_BACKEND_URL in front/.env.local For example, set VITE_HTTP_BACKEND_URL=http://124.10.20.1:7001
 - **How can I provide feedback?** For feedback, feature requests and bug reports, open an issue or ping me on [Twitter](https://twitter.com/_abi_).
 
 ## 📚 Examples

backend/config.py

@@ -5,7 +5,11 @@ import os
 
 ANTHROPIC_API_KEY = os.environ.get("ANTHROPIC_API_KEY", None)
 
+# Debugging-related
+
 SHOULD_MOCK_AI_RESPONSE = bool(os.environ.get("MOCK", False))
+IS_DEBUG_ENABLED = bool(os.environ.get("IS_DEBUG_ENABLED", False))
+DEBUG_DIR = os.environ.get("DEBUG_DIR", "")
 
 # Set to True when running in production (on the hosted version)
 # Used as a feature flag to enable or disable certain features

backend/debug/DebugFileWriter.py

@@ -0,0 +1,30 @@
+import os
+import logging
+import uuid
+
+from config import DEBUG_DIR, IS_DEBUG_ENABLED
+
+
+class DebugFileWriter:
+    def __init__(self):
+        if not IS_DEBUG_ENABLED:
+            return
+
+        try:
+            self.debug_artifacts_path = os.path.expanduser(
+                f"{DEBUG_DIR}/{str(uuid.uuid4())}"
+            )
+            os.makedirs(self.debug_artifacts_path, exist_ok=True)
+            print(f"Debugging artifacts will be stored in: {self.debug_artifacts_path}")
+        except:
+            logging.error("Failed to create debug directory")
+
+    def write_to_file(self, filename: str, content: str) -> None:
+        try:
+            with open(os.path.join(self.debug_artifacts_path, filename), "w") as file:
+                file.write(content)
+        except Exception as e:
+            logging.error(f"Failed to write to file: {e}")
+
+    def extract_html_content(self, text: str) -> str:
+        return str(text.split("<html>")[-1].rsplit("</html>", 1)[0] + "</html>")

backend/llm.py

@@ -3,6 +3,8 @@ from typing import Any, Awaitable, Callable, List, cast
 from anthropic import AsyncAnthropic
 from openai import AsyncOpenAI
 from openai.types.chat import ChatCompletionMessageParam, ChatCompletionChunk
+from config import IS_DEBUG_ENABLED
+from debug.DebugFileWriter import DebugFileWriter
 
 from utils import pprint_prompt
 
@@ -143,6 +145,10 @@ async def stream_claude_response_native(
     prefix = "<thinking>"
     response = None
 
+    # For debugging
+    full_stream = ""
+    debug_file_writer = DebugFileWriter()
+
     while current_pass_num <= max_passes:
         current_pass_num += 1
 
@@ -164,10 +170,22 @@ async def stream_claude_response_native(
         ) as stream:
             async for text in stream.text_stream:
                 print(text, end="", flush=True)
+                full_stream += text
                 await callback(text)
 
-        # Return final message
         response = await stream.get_final_message()
+        response_text = response.content[0].text
+
+        # Write each pass's code to .html file and thinking to .txt file
+        if IS_DEBUG_ENABLED:
+            debug_file_writer.write_to_file(
+                f"pass_{current_pass_num - 1}.html",
+                debug_file_writer.extract_html_content(response_text),
+            )
+            debug_file_writer.write_to_file(
+                f"thinking_pass_{current_pass_num - 1}.txt",
+                response_text.split("</thinking>")[0],
+            )
 
         # Set up messages array for next pass
         messages += [
@@ -185,6 +203,9 @@ async def stream_claude_response_native(
     # Close the Anthropic client
     await client.close()
 
+    if IS_DEBUG_ENABLED:
+        debug_file_writer.write_to_file("full_stream.txt", full_stream)
+
     if not response:
         raise Exception("No HTML response found in AI response")
     else:

backend/mock_llm.py

@@ -4,14 +4,14 @@ from typing import Awaitable, Callable
 from custom_types import InputMode
 
 
-STREAM_CHUNK_SIZE = 5
+STREAM_CHUNK_SIZE = 20
 
 
 async def mock_completion(
     process_chunk: Callable[[str], Awaitable[None]], input_mode: InputMode
 ) -> str:
     code_to_return = (
-        GOOGLE_FORM_VIDEO_PROMPT_MOCK
+        TALLY_FORM_VIDEO_PROMPT_MOCK
         if input_mode == "video"
         else NO_IMAGES_NYTIMES_MOCK_CODE
     )
@@ -670,394 +670,460 @@ $(document).ready(function() {
 """
 
 GOOGLE_FORM_VIDEO_PROMPT_MOCK = """
-<thinking>
+
-To build this:
+<thinking>User flow:
-- Create a search bar that allows typing and shows placeholder text
+1. User starts on the Google search page and types in "times" in the search bar
-- Implement search suggestions that update as the user types
+2. As the user types, Google provides autocomplete suggestions related to "times"
-- Allow selecting a suggestion to perform that search
+3. User selects the "times" suggestion from the autocomplete dropdown
-- Show search results with the query and an AI-powered overview 
+4. The search results page for "times" loads, showing various results related to The New York Times newspaper
-- Have filter tabs for different search verticals 
+5. User clicks the "Generate" button under "Get an AI-powered overview for this search?"
-- Allow clicking filter tabs to add/remove them, updating the URL
+6. An AI-generated overview about The New York Times loads on the right side of the search results
-- Ensure the UI closely matches the Google style and colors
+
+Code structure:
+- HTML structure with header, search bar, autocomplete dropdown, search button
+- Search results area to display search results
+- Sidebar area to display the AI-generated overview
+- Use Tailwind CSS utility classes for styling
+- Use jQuery to handle user interactions:
+  - Typing in search bar to show/filter autocomplete suggestions
+  - Selecting autocomplete suggestion to populate search bar
+  - Clicking search button to display search results
+  - Clicking "Generate" button to display AI overview
+- Hardcode search results and AI overview content for demo purposes
 </thinking>
 
 <html>
 <head>
   <script src="https://cdn.tailwindcss.com"></script>
-  <script src="https://code.jquery.com/jquery-3.7.0.min.js"></script>
+  <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.15.3/css/all.min.css"></link>
-  <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.15.3/css/all.min.css">
+  <script src="https://code.jquery.com/jquery-3.7.1.min.js"></script>
-  <style>
-    body {
-      font-family: Arial, sans-serif;
-    }
-    .filter {
-      color: #1a73e8;
-    }
-    .filter.active {
-      background-color: #e2eeff;
-      border-bottom: 3px solid #1a73e8;
-    }
-  </style>
 </head>
 <body>
 
-<div class="flex justify-end items-center p-4 text-sm text-gray-700">
+<div class="flex flex-col h-screen">
-  <div class="mr-4">Gmail</div>
+  <div class="flex items-center gap-4 px-5 py-3">
-  <div class="mr-4">Images</div>
+    <img class="w-24" src="https://www.google.com/images/branding/googlelogo/2x/googlelogo_color_92x30dp.png" alt="Google logo">
-  <img class="h-8 w-8 rounded-full" src="https://via.placeholder.com/150" alt="User avatar">
+    <div class="flex-grow relative">
-</div>
+      <input type="text" id="search" class="w-full px-4 py-2 text-lg border border-gray-200 rounded-full focus:outline-none focus:ring-2 focus:ring-blue-500" placeholder="Search Google or type a URL">
-
+      <div id="autocomplete" class="absolute w-full bg-white border border-gray-100 rounded-md shadow-lg z-10 hidden">
-<div class="flex flex-col items-center mt-44">
+        <div class="px-4 py-2 hover:bg-gray-100 cursor-pointer">times</div>
-  <img class="w-72" src="https://www.google.com/images/branding/googlelogo/2x/googlelogo_color_272x92dp.png" alt="Google logo">
+        <div class="px-4 py-2 hover:bg-gray-100 cursor-pointer">ts-ebml</div>
-  
+        <div class="px-4 py-2 hover:bg-gray-100 cursor-pointer">tiktok ceo</div>
-  <div class="relative mt-8 w-full max-w-xl">
+        <div class="px-4 py-2 hover:bg-gray-100 cursor-pointer">two videos</div>
-    <input id="search" type="text" class="w-full px-4 py-3 text-lg rounded-full border border-gray-200 hover:shadow-lg focus:outline-none focus:shadow-lg" placeholder="Google Search">
+        <div class="px-4 py-2 hover:bg-gray-100 cursor-pointer">Taskrabbit</div>
-    <i class="absolute right-0 top-0 m-3 text-blue-500 text-xl fas fa-microphone"></i>
+        <div class="px-4 py-2 hover:bg-gray-100 cursor-pointer">translate</div>
-  </div>
+        <div class="px-4 py-2 hover:bg-gray-100 cursor-pointer">Target</div>
-
+        <div class="px-4 py-2 hover:bg-gray-100 cursor-pointer">Taylor Swift</div>
-  <div id="suggestions" class="flex flex-col w-full max-w-xl border border-gray-100 shadow-lg mt-1 rounded-lg hidden">
+        <div class="px-4 py-2 hover:bg-gray-100 cursor-pointer">Travis Kelce</div>
-  </div>
+        <div class="px-4 py-2 hover:bg-gray-100 cursor-pointer">Temu</div>
-
+      </div>
-  <div class="flex justify-center space-x-4 mt-8">
-    <button class="px-6 py-3 bg-gray-100 text-gray-800 rounded-md hover:shadow-md focus:outline-none">Google Search</button>
-    <button class="px-6 py-3 bg-gray-100 text-gray-800 rounded-md hover:shadow-md focus:outline-none">I'm Feeling Lucky</button>
-  </div>
-</div>
-
-<div id="results" class="mt-8 hidden">
-  <div class="flex justify-between items-center px-5 py-3 text-sm">
-    <div>
-      <span class="text-blue-600 mr-3 filter active" data-filter="all">All</span>
-      <span class="text-blue-600 mr-3 filter" data-filter="news">News</span>
-      <span class="text-blue-600 mr-3 filter" data-filter="images">Images</span>
-      <span class="text-blue-600 mr-3 filter" data-filter="videos">Videos</span>
-      <span class="text-blue-600 mr-3 filter" data-filter="shopping">Shopping</span>
-      <span class="text-blue-600 mr-3 filter" data-filter="maps">Maps</span>
-      <span class="text-blue-600 mr-3 filter" data-filter="books">Books</span>
-      <span class="text-blue-600 mr-3 filter" data-filter="flights">Flights</span>
-      <span class="text-blue-600 mr-3 filter" data-filter="finance">Finance</span>
     </div>
-    <div>
+    <button id="search-btn" class="bg-blue-500 hover:bg-blue-600 text-white px-4 py-2 rounded-md ml-2">
-      <span class="text-gray-600">Settings</span>
+      <i class="fas fa-search"></i>
-      <span class="text-gray-600 ml-3">Tools</span>
+    </button>
-      <span class="text-gray-600 ml-3">SafeSearch</span>
+    <img class="w-8 h-8 rounded-full ml-4" src="https://via.placeholder.com/150" alt="Profile picture of the user">
+  </div>
+  
+  <div class="flex-grow overflow-y-auto">
+    <div id="search-results" class="p-8 hidden">
+      <div class="text-sm text-gray-600 mb-4">Results for New York, NY 10022 - Choose area</div>
+      
+      <div class="bg-blue-50 p-4 mb-4">
+        <button id="overview-btn" class="bg-blue-500 hover:bg-blue-600 text-white px-4 py-2 rounded-md">Get an AI-powered overview for this search?</button>
+      </div>
+      
+      <div class="mb-4">
+        <div class="text-xl text-blue-800 mb-1">
+          <a href="https://www.nytimes.com">The New York Times</a>
+        </div>
+        <div class="text-sm text-gray-800 mb-1">https://www.nytimes.com</div>
+        <div class="text-sm text-gray-600">
+          The New York Times - Breaking News, US News, World News ...
+        </div>
+        <div class="text-sm text-gray-600">  
+          Live news, investigations, opinion, photos and video by the journalists of The New York Times from more than 150 countries around the world.
+        </div>
+      </div>
+
+      <div class="mb-4">
+        <div class="text-sm text-gray-600 mb-1">Opinion | The House Vote to Force TikTok's Sale Is a Mistake - The ...</div>
+        <div class="text-sm text-gray-600">Why Are Lawmakers Trying to Ban TikTok Instead of Doing What ...</div>
+      </div>
+
+      <div class="mb-4">  
+        <div class="text-sm text-gray-600 mb-1">The Crossword</div>
+        <div class="text-sm text-gray-600">Play the Daily New York Times Crossword puzzle edited by Will ...</div>
+      </div>
+
+      <div class="mb-4">
+        <div class="text-sm text-gray-600 mb-1">Today's Paper</div>
+        <div class="text-sm text-gray-600">Today's Paper · The Front Page · Highlights · Lawyer, Author ...</div>
+      </div>
+
+      <div class="mb-4">
+        <div class="text-sm text-gray-600 mb-1">Word game Wordle</div>
+        <div class="text-sm text-gray-600">Get 6 chances to guess a 5-letter word.Get 6 chances to guess a ...</div>
+      </div>
+
+      <div>
+        <div class="text-sm text-gray-600">Connections</div>
+        <div class="text-sm text-gray-600">A look at the links between The Times and the world it covers.</div>
+      </div>
     </div>
-  </div>
     
-  <div class="px-5 py-3">
+    <div id="overview" class="p-8 hidden">
-    <div class="text-sm text-gray-600">Results for <span id="location">New York, NY 10022</span> - Choose area</div>
+      <h2 class="text-2xl font-bold mb-4">The New York Times</h2>
-    <button id="overview" class="px-4 py-2 mt-2 text-white bg-blue-500 rounded-md">Get an AI-powered overview for this search</button>
+      <div class="mb-4">
-  </div>
+        <div class="font-bold">Newspaper</div>
-
+      </div>
-  <div id="overview-text" class="px-5 py-3"></div>
+      <div class="mb-4">  
-
+        <div class="font-bold">Owner:</div>
-  <div class="px-5 py-3">
+        <div>The New York Times Company</div>
-    <div class="text-xl text-blue-600">The New York Times</div>
+      </div>
-    <div class="text-sm text-green-700">https://www.nytimes.com</div>
+      <div class="mb-4">
-    <div class="mt-2">The New York Times - Breaking News, US News, World News ...</div>
+        <div class="font-bold">Publisher:</div>  
-    <div class="text-sm text-gray-600">Live news, investigations, opinion, photos and video by the journalists of The New York Times from more than 150 countries around the world.</div>
+        <div>The New York Times Company</div>
+      </div>
+      <div class="mb-4">
+        <div class="font-bold">Founders:</div>
+        <div>George Jones, Henry Jarvis Raymond</div>  
+      </div>
+      <div class="mb-4">
+        <div class="font-bold">Circulation:</div>
+        <div>10,360,000 news subscribers (as of February 2024)</div>
+      </div>
+      <div class="mb-4">  
+        <div class="font-bold">Editor-in-chief:</div>
+        <div>Joseph Kahn</div>
+      </div>
+      <div class="mb-4">
+        <div class="font-bold">Format:</div>
+        <div>Broadsheet</div>
+      </div>
+      <div>  
+        <div class="font-bold">Founded:</div> 
+        <div>September 18, 1851; 172 years ago</div>
+      </div>
+    </div>
   </div>
 </div>
 
 <script>
-const searchSuggestions = {
+$(document).ready(function() {
-  "t": ["times", "translate", "twitter", "target"],
+  let selectedText = "";
-  "ti": ["times", "times square", "tiktok", "tires"],
-  "tim": ["times", "times square", "time", "timer"],
-  "time": ["times", "times square", "time", "time magazine"],
-  "times": ["times", "times square", "times table", "times of india"]
-};
   
-let currentSearch = '';
+  $("#search").on("input", function() {
+    const searchText = $(this).val().trim();
     
-$('#search').on('input', function() {
+    if (searchText !== "") {
-  currentSearch = $(this).val().trim().toLowerCase();
+      $("#autocomplete").removeClass("hidden");
-
+      $("#autocomplete div").each(function() {
-  if (currentSearch.length > 0) {
+        if ($(this).text().toLowerCase().startsWith(searchText.toLowerCase())) {
-    const suggestions = searchSuggestions[currentSearch] || [];
+          $(this).removeClass("hidden");
-    const suggestionsHtml = suggestions.map(s => `<div class="px-4 py-2 hover:bg-gray-100 cursor-pointer">${s}</div>`).join('');
+        } else {
-    $('#suggestions').html(suggestionsHtml).show();
+          $(this).addClass("hidden");
-  } else {
+        }
-    $('#suggestions').empty().hide();
+      });
-  }
+    } else {
-});
+      $("#autocomplete").addClass("hidden");
-
-$('#suggestions').on('click', 'div', function() {
-  const suggestion = $(this).text();
-  $('#search').val(suggestion);
-  $('#suggestions').empty().hide();
-  performSearch(suggestion);
-});
-
-$('#search').on('keypress', function(e) {
-  if (e.which === 13) {
-    const query = $(this).val();
-    performSearch(query);
-  }
-});
-
-function performSearch(query) {
-  $('#results').show();
-  $('.filter').removeClass('active');
-  $('.filter[data-filter="all"]').addClass('active');
-  updateFilters('all');
-  $('#overview-text').empty();
-}
-
-$('.filter').on('click', function() {
-  $('.filter').removeClass('active');
-  $(this).addClass('active');
-  const filter = $(this).data('filter');
-  updateFilters(filter);
-});
-
-function updateFilters(activeFilter) {
-  const filters = ['all', 'news', 'images', 'videos', 'shopping', 'maps', 'books', 'flights', 'finance'];
-  let filtersHtml = '';
-
-  filters.forEach(filter => {
-    if (filter === activeFilter) {
-      filtersHtml += `<span class="text-blue-600 mr-3 filter active" data-filter="${filter}">${filter.charAt(0).toUpperCase() + filter.slice(1)}</span>`;
-    } else if (filters.indexOf(filter) <= filters.indexOf(activeFilter)) {
-      filtersHtml += `<span class="text-blue-600 mr-3 filter" data-filter="${filter}">${filter.charAt(0).toUpperCase() + filter.slice(1)}</span>`;
     }
   });
   
-  $('.flex.justify-between').find('div:first-child').html(filtersHtml);
+  $("#autocomplete div").click(function() {
-  window.history.pushState({}, '', `?q=${encodeURIComponent(currentSearch)}&filter=${activeFilter}`);
+    selectedText = $(this).text();
-}
+    $("#search").val(selectedText);
+    $("#autocomplete").addClass("hidden");
+  });
   
-$('#overview').on('click', function() {
+  $("#search-btn").click(function() {
-  $('#overview-text').text('Try AI-powered overviews when you search');
+    if (selectedText !== "") {
+      $("#search-results").removeClass("hidden");
+      $("#overview").addClass("hidden");
+    }
+  });
+  
+  $("#overview-btn").click(function() {
+    $("#overview").removeClass("hidden");
+  });
 });
 </script>
+
 </body>
 </html>
 
-<thinking>To further improve the app based on the original instructions:
+<thinking>
+To improve the app further and make it fully functional like the original video:
 
-1. Adjust the font sizes, colors, and spacing to more closely match the Google style.
+1. Add more visual styling to match the original, including:
-2. Add the hover effect on the search bar to expand its shadow.
+   - Google logo and favicon
-3. Position the microphone icon correctly inside the search bar.
+   - User profile icon
-4. Implement the "I'm Feeling Lucky" functionality to redirect to the first search result.
+   - Search bar styling and hover effects
-5. Add the "Search by voice" and "Search by image" options that appear on hover.
+   - Search button icon and styling
-6. Show the "Clear" button in the search bar when there's text entered.
+   - Fonts and text sizes
-7. Make the search suggestions clickable to perform the search.
+   - Spacing and layout adjustments
-8. Update the page title and URL when a search is performed.
+
-9. Show a more realistic search result with a link, description, and URL.
+2. Implement the search functionality:
-10. Add more filter options and make them functional to update the search results.
+   - Perform a search when the user presses Enter or clicks the search button
-11. Generate a more realistic AI-powered overview of the search topic.
+   - Show a loading state while searching
-12. Refine the responsive layout for different screen sizes.
+   - Display the search results dynamically based on the query
+   - Handle cases when no results are found
+
+3. Enhance the autocomplete functionality:
+   - Fetch autocomplete suggestions from a data source based on the user's input
+   - Highlight the text match in the suggestions
+   - Allow navigating the suggestions with arrow keys
+   - Update the search input when a suggestion is selected
+
+4. Improve the AI-powered overview:
+   - Fetch the overview content dynamically based on the search query
+   - Display a loading state while generating the overview
+   - Enhance the styling of the overview to match the original
+
+5. Add additional UI elements:
+   - Header navigation tabs (About, Store, Gmail, Images)
+   - Footer links (Advertising, Business, How Search works, Privacy, Terms)
+   - Pagination for search results
+   - Feedback link
+
+6. Optimize performance:
+   - Lazy-load images and content
+   - Implement caching mechanisms
+   - Minify and compress assets
+
+7. Ensure responsiveness:
+   - Make the layout responsive and mobile-friendly
+   - Adjust font sizes, padding, and margins for different screen sizes
+   - Hide/collapse certain elements on smaller screens
+
+8. Enhance accessibility:
+   - Add appropriate ARIA attributes
+   - Ensure keyboard navigation works properly
+   - Provide alternative text for images
+
+9. Implement error handling:
+   - Handle network errors gracefully
+   - Display user-friendly error messages
+   - Provide options to retry or refresh
+
+10. Add smooth animations and transitions:
+    - Fade in/out effects for loading states
+    - Smooth sliding animations for autocomplete suggestions
+    - Subtle hover and click animations for interactive elements
 </thinking>
 
 <html>
 <head>
-  <title>Google</title>
+  <title>Google Search</title>
+  <link rel="icon" href="https://www.google.com/favicon.ico" type="image/x-icon">
   <script src="https://cdn.tailwindcss.com"></script>
-  <script src="https://code.jquery.com/jquery-3.7.0.min.js"></script>
+  <link href="https://fonts.googleapis.com/css2?family=Roboto:wght@400;500&display=swap" rel="stylesheet">
   <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.15.3/css/all.min.css">
-  <link rel="preconnect" href="https://fonts.googleapis.com">
+  <script src="https://code.jquery.com/jquery-3.7.1.min.js"></script>
-  <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
-  <link href="https://fonts.googleapis.com/css2?family=Roboto:wght@400;500;700&display=swap" rel="stylesheet">
-  <style>
-    body {
-      font-family: 'Roboto', Arial, sans-serif;
-      font-size: 14px;
-      color: #202124;
-    }
-    .filter {
-      color: #1a73e8;
-    }
-    .filter.active {
-      border-bottom: 3px solid #1a73e8;
-    }
-    .search-bar:hover {
-      box-shadow: 0 1px 6px rgb(32 33 36 / 28%);
-      border-color: rgba(223,225,229,0);
-    }
-    .search-bar:focus-within {
-      box-shadow: 0 1px 6px rgb(32 33 36 / 28%);
-      border-color: rgba(223,225,229,0);
-    }
-  </style>
 </head>
-<body>
+<body class="font-roboto">
 
-<div class="flex justify-end items-center p-2 text-sm">
+<div class="flex flex-col min-h-screen">
-  <div class="mr-4 hover:underline cursor-pointer">Gmail</div>
+  <header class="flex items-center justify-end space-x-4 px-5 py-2 text-sm text-gray-700">
-  <div class="mr-4 hover:underline cursor-pointer">Images</div>
+    <a href="#" class="hover:underline">About</a>
-  <img class="h-8 w-8 rounded-full" src="https://via.placeholder.com/150" alt="User avatar">
+    <a href="#" class="hover:underline">Store</a>
-</div>
+    <a href="#" class="hover:underline">Gmail</a>
+    <a href="#" class="hover:underline">Images</a>
+    <img class="w-8 h-8 rounded-full" src="https://via.placeholder.com/150" alt="Profile picture of the user">
+  </header>
   
-<div class="flex flex-col items-center mt-40">
+  <div class="flex items-center justify-center flex-grow">
-  <img class="w-72" src="https://www.google.com/images/branding/googlelogo/2x/googlelogo_color_272x92dp.png" alt="Google logo">
+    <div class="flex flex-col items-center w-full max-w-[584px]">
-  
+      <img class="w-[272px] mb-8" src="https://www.google.com/images/branding/googlelogo/2x/googlelogo_color_272x92dp.png" alt="Google logo">
-  <div class="relative mt-6 w-full max-w-[584px]">
+      <div class="relative w-full mb-3">
-    <input id="search" type="text" class="w-full px-5 py-3 text-base rounded-full border border-gray-200 hover:shadow-lg focus:outline-none focus:shadow-lg search-bar" placeholder="Google Search" autocomplete="off">
+        <input type="text" id="search" class="w-full px-5 py-3 text-lg border border-gray-200 rounded-full hover:shadow-md focus:outline-none focus:ring-2 focus:ring-blue-500" placeholder="Search Google or type a URL" autocomplete="off">
-    <i class="absolute right-0 top-0 m-3.5 text-[#4285f4] text-xl fas fa-microphone"></i>
+        <div id="autocomplete" class="absolute w-full bg-white border border-gray-100 rounded-md shadow-lg z-10 hidden">
-    <i id="clear-search" class="absolute right-0 top-0 m-3.5 text-gray-500 text-2xl fas fa-times cursor-pointer hidden"></i>
+          <div class="px-4 py-2 hover:bg-gray-100 cursor-pointer">times</div>
-  </div>
+          <div class="px-4 py-2 hover:bg-gray-100 cursor-pointer">ts-ebml</div>
-
+          <div class="px-4 py-2 hover:bg-gray-100 cursor-pointer">tiktok ceo</div>
-  <div id="search-options" class="flex justify-center space-x-2 mt-2 text-sm text-[#4285f4] hidden">
+          <div class="px-4 py-2 hover:bg-gray-100 cursor-pointer">two videos</div>
-    <div class="flex items-center cursor-pointer">
+          <div class="px-4 py-2 hover:bg-gray-100 cursor-pointer">Taskrabbit</div>
-      <i class="fas fa-search mr-1"></i>
+          <div class="px-4 py-2 hover:bg-gray-100 cursor-pointer">translate</div>
-      <span>Search by voice</span>
+          <div class="px-4 py-2 hover:bg-gray-100 cursor-pointer">Target</div>
-    </div>
+          <div class="px-4 py-2 hover:bg-gray-100 cursor-pointer">Taylor Swift</div>
-    <div class="flex items-center cursor-pointer">
+          <div class="px-4 py-2 hover:bg-gray-100 cursor-pointer">Travis Kelce</div>
-      <i class="fas fa-camera mr-1"></i>
+          <div class="px-4 py-2 hover:bg-gray-100 cursor-pointer">Temu</div>
-      <span>Search by image</span>
+        </div>
+      </div>
+      <div class="flex space-x-2">
+        <button id="search-btn" class="bg-gray-100 hover:bg-gray-200 text-gray-800 px-4 py-2 rounded">Google Search</button>
+        <button class="bg-gray-100 hover:bg-gray-200 text-gray-800 px-4 py-2 rounded">I'm Feeling Lucky</button>
+      </div>
     </div>
   </div>
   
-  <div id="suggestions" class="flex flex-col w-full max-w-[584px] border border-gray-100 shadow-lg rounded-lg bg-white hidden">
+  <div id="search-results" class="flex-grow p-8 overflow-y-auto hidden">
+    <div class="text-sm text-gray-600 mb-4">Results for New York, NY 10022 - Choose area</div>
+    
+    <div class="bg-blue-50 p-4 mb-4">
+      <button id="overview-btn" class="bg-blue-500 hover:bg-blue-600 text-white px-4 py-2 rounded">Get an AI-powered overview for this search</button>
+    </div>
+    
+    <div id="results-list">
+      <div class="mb-4">
+        <div class="text-xl text-blue-800 mb-1">
+          <a href="https://www.nytimes.com">The New York Times</a>
+        </div>
+        <div class="text-sm text-gray-800 mb-1">https://www.nytimes.com</div>
+        <div class="text-sm text-gray-600">
+          The New York Times - Breaking News, US News, World News ...
+        </div>
+        <div class="text-sm text-gray-600">  
+          Live news, investigations, opinion, photos and video by the journalists of The New York Times from more than 150 countries around the world.
+        </div>
+      </div>
+
+      <div class="mb-4">
+        <div class="text-sm text-gray-600 mb-1">Opinion | The House Vote to Force TikTok's Sale Is a Mistake - The ...</div>
+        <div class="text-sm text-gray-600">Why Are Lawmakers Trying to Ban TikTok Instead of Doing What ...</div>
+      </div>
+
+      <div class="mb-4">  
+        <div class="text-sm text-gray-600 mb-1">The Crossword</div>
+        <div class="text-sm text-gray-600">Play the Daily New York Times Crossword puzzle edited by Will ...</div>
+      </div>
+
+      <div class="mb-4">
+        <div class="text-sm text-gray-600 mb-1">Today's Paper</div>
+        <div class="text-sm text-gray-600">Today's Paper · The Front Page · Highlights · Lawyer, Author ...</div>
+      </div>
+
+      <div class="mb-4">
+        <div class="text-sm text-gray-600 mb-1">Word game Wordle</div>
+        <div class="text-sm text-gray-600">Get 6 chances to guess a 5-letter word.Get 6 chances to guess a ...</div>
+      </div>
+
+      <div>
+        <div class="text-sm text-gray-600">Connections</div>
+        <div class="text-sm text-gray-600">A look at the links between The Times and the world it covers.</div>
+      </div>
+    </div>
+    
+    <div id="pagination" class="flex justify-center mt-8">
+      <button class="bg-blue-500 hover:bg-blue-600 text-white px-4 py-2 rounded mr-2">Previous</button>
+      <button class="bg-blue-500 hover:bg-blue-600 text-white px-4 py-2 rounded">Next</button>
+    </div>
   </div>
   
-  <div class="flex justify-center space-x-4 mt-8">
+  <div id="overview" class="flex-grow p-8 overflow-y-auto hidden">
-    <button id="search-button" class="px-6 py-2 bg-[#f8f9fa] text-[#3c4043] rounded text-sm hover:shadow-md focus:outline-none">Google Search</button>
+    <h2 class="text-2xl font-medium mb-4">The New York Times</h2>
-    <button id="lucky-button" class="px-6 py-2 bg-[#f8f9fa] text-[#3c4043] rounded text-sm hover:shadow-md focus:outline-none">I'm Feeling Lucky</button>
+    <div class="mb-4">
-  </div>
+      <div class="font-medium">Newspaper</div>
-</div>
+    </div>
-
+    <div class="mb-4">  
-<div id="results" class="mt-6 hidden">
+      <div class="font-medium">Owner:</div>
-  <div class="flex justify-between items-center px-4 py-2.5 text-sm">
+      <div>The New York Times Company</div>
-    <div>
+    </div>
-      <span class="text-[#4285f4] mr-3 filter active" data-filter="all">All</span>
+    <div class="mb-4">
-      <span class="text-[#4285f4] mr-3 filter" data-filter="news">News</span>
+      <div class="font-medium">Publisher:</div>  
-      <span class="text-[#4285f4] mr-3 filter" data-filter="images">Images</span>
+      <div>The New York Times Company</div>
-      <span class="text-[#4285f4] mr-3 filter" data-filter="videos">Videos</span>
+    </div>
-      <span class="text-[#4285f4] mr-3 filter" data-filter="shopping">Shopping</span>
+    <div class="mb-4">
-      <span class="text-[#4285f4] mr-3 filter" data-filter="maps">Maps</span>
+      <div class="font-medium">Founders:</div>
-      <span class="text-[#4285f4] mr-3 filter" data-filter="books">Books</span>
+      <div>George Jones, Henry Jarvis Raymond</div>  
-      <span class="text-[#4285f4] mr-3 filter" data-filter="flights">Flights</span>
+    </div>
-      <span class="text-[#4285f4] mr-3 filter" data-filter="finance">Finance</span>
+    <div class="mb-4">
+      <div class="font-medium">Circulation:</div>
+      <div>10,360,000 news subscribers (as of February 2024)</div>
+    </div>
+    <div class="mb-4">  
+      <div class="font-medium">Editor-in-chief:</div>
+      <div>Joseph Kahn</div>
+    </div>
+    <div class="mb-4">
+      <div class="font-medium">Format:</div>
+      <div>Broadsheet</div>
     </div>
     <div>  
-      <span class="text-[#70757a] hover:underline cursor-pointer">Settings</span>
+      <div class="font-medium">Founded:</div> 
-      <span class="text-[#70757a] ml-6 hover:underline cursor-pointer">Tools</span>
+      <div>September 18, 1851; 172 years ago</div>
-      <span class="text-[#70757a] ml-6 hover:underline cursor-pointer">SafeSearch</span>
     </div>
   </div>
   
-  <div class="px-4 py-3">
+  <footer class="bg-gray-100 text-sm text-gray-600 px-8 py-3">
-    <div class="text-[#70757a]">Results for <span id="location">New York, NY 10022</span> - <span class="hover:underline cursor-pointer">Choose area</span></div>
+    <div class="flex justify-between">
-    <button id="overview" class="px-4 py-2.5 mt-2 text-white bg-[#1a73e8] rounded text-sm hover:bg-[#1a65c7]">Get an AI-powered overview for this search</button>
+      <div class="flex space-x-6">
-  </div>
+        <a href="#" class="hover:underline">Advertising</a>
-
+        <a href="#" class="hover:underline">Business</a>
-  <div id="overview-text" class="px-4 py-3 text-sm"></div>
+        <a href="#" class="hover:underline">How Search works</a>
-
+      </div>
-  <div class="px-4 py-3">
+      <div class="flex space-x-6">
-    <div class="text-xl text-[#1a0dab] hover:underline cursor-pointer">The New York Times</div>
+        <a href="#" class="hover:underline">Privacy</a>
-    <div class="text-sm text-[#006621] hover:underline cursor-pointer">https://www.nytimes.com</div>
+        <a href="#" class="hover:underline">Terms</a>
-    <div class="mt-1">The New York Times - Breaking News, US News, World News ...</div>
+        <a href="#" class="hover:underline">Settings</a>
-    <div class="text-sm text-[#545454]">Live news, investigations, opinion, photos and video by the journalists of The New York Times from more than 150 countries around the world.</div>
+      </div>
-  </div>
+    </div>
+  </footer>
 </div>
 
 <script>
-const searchSuggestions = {
+$(document).ready(function() {
-  "t": ["times", "translate", "twitter", "target"],
+  let selectedText = "";
-  "ti": ["times", "times square", "tiktok", "tires"],
-  "tim": ["times", "times square", "time", "timer"],
-  "time": ["times", "times square", "time", "time magazine"],
-  "times": ["times", "times square", "times table", "times of india"]
-};
   
-let currentSearch = '';
+  $("#search").on("input", function() {
+    const searchText = $(this).val().trim();
     
-$('#search').on('input', function() {
+    if (searchText !== "") {
-  currentSearch = $(this).val().trim().toLowerCase();
+      $("#autocomplete").removeClass("hidden");
-
+      $("#autocomplete div").each(function() {
-  if (currentSearch.length > 0) {
+        if ($(this).text().toLowerCase().includes(searchText.toLowerCase())) {
-    const suggestions = searchSuggestions[currentSearch] || [];
+          $(this).removeClass("hidden");
-    const suggestionsHtml = suggestions.map(s => `<div class="px-4 py-2 hover:bg-gray-100 cursor-pointer">${s}</div>`).join('');
+          $(this).html($(this).text().replace(new RegExp(searchText, "gi"), "<strong>$&</strong>"));
-    $('#suggestions').html(suggestionsHtml).show();
+        } else {
-    $('#clear-search').show();
+          $(this).addClass("hidden");
-    $('#search-options').hide();
+        }
-  } else {
+      });
-    $('#suggestions').empty().hide();
+    } else {
-    $('#clear-search').hide();
+      $("#autocomplete").addClass("hidden");
-    $('#search-options').show();
-  }
-});
-
-$('#suggestions').on('click', 'div', function() {
-  const suggestion = $(this).text();
-  $('#search').val(suggestion);
-  $('#suggestions').empty().hide();
-  performSearch(suggestion);
-});
-
-$('#search').on('keypress', function(e) {
-  if (e.which === 13) {
-    const query = $(this).val();
-    performSearch(query);
-  }
-});
-
-$('#search-button').on('click', function() {
-  const query = $('#search').val();
-  performSearch(query);
-});
-
-$('#lucky-button').on('click', function() {
-  window.location.href = 'https://www.google.com/doodles';
-});
-
-$('#clear-search').on('click', function() {
-  $('#search').val('');
-  $('#suggestions').empty().hide();
-  $(this).hide();
-  $('#search-options').show();
-  $('#search').focus();
-});
-
-$('#search').on('focus', function() {
-  if ($(this).val().length > 0) {
-    $('#search-options').hide();
-  } else {
-    $('#search-options').show();
-  }
-});
-
-$('#search').on('blur', function() {
-  $('#search-options').hide();
-});
-
-function performSearch(query) {
-  $('#results').show();
-  $('.filter').removeClass('active');
-  $('.filter[data-filter="all"]').addClass('active');
-  updateFilters('all');
-  $('#overview-text').empty();
-  window.history.pushState({}, '', `?q=${encodeURIComponent(query)}`);
-  document.title = `${query} - Google Search`;
-}
-
-$('.filter').on('click', function() {
-  $('.filter').removeClass('active');
-  $(this).addClass('active');
-  const filter = $(this).data('filter');
-  updateFilters(filter);
-});
-
-function updateFilters(activeFilter) {
-  const filters = ['all', 'news', 'images', 'videos', 'shopping', 'maps', 'books', 'flights', 'finance'];
-  let filtersHtml = '';
-
-  filters.forEach(filter => {
-    if (filter === activeFilter) {
-      filtersHtml += `<span class="text-[#4285f4] mr-3 filter active" data-filter="${filter}">${filter.charAt(0).toUpperCase() + filter.slice(1)}</span>`;
-    } else if (filters.indexOf(filter) <= filters.indexOf(activeFilter)) {
-      filtersHtml += `<span class="text-[#4285f4] mr-3 filter" data-filter="${filter}">${filter.charAt(0).toUpperCase() + filter.slice(1)}</span>`;
     }
   });
   
-  $('.flex.justify-between').find('div:first-child').html(filtersHtml);
+  $("#autocomplete div").click(function() {
-  window.history.pushState({}, '', `?q=${encodeURIComponent(currentSearch)}&filter=${activeFilter}`);
+    selectedText = $(this).text();
-}
+    $("#search").val(selectedText);
+    $("#autocomplete").addClass("hidden");
+  });
   
-$('#overview').on('click', function() {
+  $("#search").keydown(function(e) {
-  $('#overview-text').text('Try AI-powered overviews when you search. These provide a quick understanding of topics based on information from across the web.');
+    if (e.which === 13) {
+      performSearch();
+    }
+  });
+  
+  $("#search-btn").click(performSearch);
+  
+  $("#overview-btn").click(function() {
+    $("#overview").removeClass("hidden");
+    $("#search-results").addClass("hidden");
+  });
+  
+  function performSearch() {
+    const query = $("#search").val().trim();
+    if (query !== "") {
+      $("#search-results").removeClass("hidden");
+      $("#overview").addClass("hidden");
+      
+      // Simulating search results for demo purposes
+      $("#results-list").empty();
+      for (let i = 0; i < 6; i++) {
+        $("#results-list").append(`
+          <div class="mb-4">
+            <div class="text-xl text-blue-800 mb-1">
+              <a href="#">Search Result ${i + 1}</a>
+            </div>
+            <div class="text-sm text-gray-800 mb-1">https://example.com/result${i + 1}</div>
+            <div class="text-sm text-gray-600">
+              Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed euismod, nulla sit amet aliquam lacinia, nisl nisl aliquam nisl, nec aliquam nisl nisl sit amet nisl.
+            </div>
+          </div>
+        `);
+      }
+    }
+  }
 });
 </script>
 

backend/routes/generate_code.py

@@ -155,7 +155,7 @@ async def stream_code(websocket: WebSocket):
             print("Using OpenAI API key from environment variable")
 
     # If we still don't have an API key, throw an error
-    if not openai_api_key:
+    if not openai_api_key and code_generation_model == "gpt_4_vision":
         print("OpenAI API key not found")
         await throw_error(
             "No OpenAI API key found. Please add your API key in the settings dialog or add it to backend/.env file."
@@ -266,7 +266,7 @@ async def stream_code(websocket: WebSocket):
 
                 if not ANTHROPIC_API_KEY:
                     await throw_error(
-                        "No Anthropic API key found. Please add the environment variable ANTHROPIC_API_KEY to backend/.env"
+                        "Video only works with Anthropic models. No Anthropic API key found. Please add the environment variable ANTHROPIC_API_KEY to backend/.env"
                     )
                     raise Exception("No Anthropic key")
 

frontend/src/App.tsx

@@ -41,6 +41,7 @@ import { CodeGenerationModel } from "./lib/models";
 import ModelSettingsSection from "./components/ModelSettingsSection";
 import { extractHtml } from "./components/preview/extractHtml";
 import useBrowserTabIndicator from "./hooks/useBrowserTabIndicator";
+import { URLS } from "./urls";
 
 const IS_OPENAI_DOWN = false;
 
@@ -396,6 +397,15 @@ function App({ navbarComponent }: Props) {
             }
           />
 
+          <a
+            className="text-xs underline text-gray-500 text-right"
+            href={URLS.tips}
+            target="_blank"
+            rel="noopener"
+          >
+            Tips for better results
+          </a>
+
           {IS_RUNNING_ON_CLOUD &&
             !(settings.openAiApiKey || settings.accessCode) &&
             subscriberTier === "free" && <OnboardingNote />}

frontend/src/components/recording/ScreenRecorder.tsx

@@ -116,7 +116,26 @@ function ScreenRecorder({
           <div className="flex items-center mr-2 text-xl gap-x-1">
             <span>Screen Recording Captured.</span>
           </div>
-          <Button onClick={kickoffGeneration}>Generate</Button>
+          {screenRecordingDataUrl && (
+            <video
+              muted
+              autoPlay
+              loop
+              className="w-[340px] border border-gray-200 rounded-md"
+              src={screenRecordingDataUrl}
+            />
+          )}
+          <div className="flex gap-x-2">
+            <Button
+              variant="secondary"
+              onClick={() =>
+                setScreenRecorderState(ScreenRecorderState.INITIAL)
+              }
+            >
+              Re-record
+            </Button>
+            <Button onClick={kickoffGeneration}>Generate</Button>
+          </div>
         </div>
       )}
     </div>

frontend/src/urls.ts

@@ -1,4 +1,5 @@
 export const URLS = {
   "intro-to-video":
-    "https://github.com/abi/screenshot-to-code/blob/main/blog/video-to-app.md",
+    "https://github.com/abi/screenshot-to-code/wiki/Screen-Recording-to-Code",
+  tips: "https://git.new/s5ywP0e",
 };