Improving the json mode prompt that gets used within react app charms (#1006)

* Improving the json mode prompt that gets used within react app charms

* updating llm cache for integration tests

* only upload jumble assets if we are running ci in the main branch

Author: jakedahn

.github/workflows/deno.yml

@@ -156,6 +156,7 @@ jobs:
 
       - name: 📤 Upload backend artifacts to R2 /jumble/${{ github.sha }}
         uses: ryand56/r2-upload-action@v1.4
+        if: github.ref == 'refs/heads/main'
         with:
           r2-account-id: ${{ secrets.R2_ACCOUNT_ID }}
           r2-access-key-id: ${{ secrets.R2_ACCESS_KEY_ID }}

charm/src/iframe/static.ts

@@ -652,19 +652,104 @@ async function fetchLLMResponse() {
 }
 \`\`\`
 
-If you need JSON to be returned from the LLM, you can enable the \`mode: 'json'\` in the \`promptPayload\`.
+
+## 3. llm Function with JSON
+
+If you need JSON to be returned from the LLM, you can enable the \`mode: 'json'\`
+in the \`promptPayload\`. However, if you do this, you'll need to make sure you
+define the schema you expect from the JSON response. Do this as plain english.
+
+For example: "Generate a traditional Vietnamese recipe in JSON format, with the
+following properties: name (string), ingredients (array of strings),
+instructions (array of strings)"
+
+NOTE: You'll need to parse the result string into an object using \`JSON.parse()\`.
+
+ANOTHER NOTE: Language model requests are globally cached based on your prompt.
+This means that identical requests will return the same result. If your llm use
+requires unique results on every request, make sure to introduce a cache-breaking
+string such as a timestamp or incrementing number/id.
 
 \`\`\`jsx
 const promptPayload = {
-  system: 'Translate all the messages to emojis, reply in JSON.',
+  system: 'Translate all the messages to emojis, reply in JSON with the following properties: an array of objects, each with original_text (string), emoji_translation (string)',
   messages: ['Hi', 'How can I help you today?', 'tell me a joke'],
   mode: 'json'
 };
 const result = await llm(promptPayload);
-console.log('JSON response from llm:', result);
+console.log('JSON response from llm:', JSON.parse(result));
+\`\`\`
+
+This \`result\` variable will be a stringified JSON object. Once you JSON.parse() it, you'll get the following object:
+
+[
+    {
+        "original_text": "Hi",
+        "emoji_translation": "👋"
+    },
+    {
+        "original_text": "How can I help you today?",
+        "emoji_translation": "🤔❓🙋‍♂️📅"
+    },
+    {
+        "original_text": "tell me a joke",
+        "emoji_translation": "🗣️👉😂"
+    }
+]
+
+Another example:
+
+\`\`\`jsx
+// Every time we run this prompt, we want a unique result, so we'll use a cache-breaking string.
+const cacheBreaker = Date.now();
+
+const promptPayload = {
+  system: "You are a professional chef specializing in Mexican cuisine. Generate a detailed, authentic Mexican recipe in JSON format with the following properties: title (string), ingredients (array of strings), instructions (array of strings), prepTime (integer in minutes), cookTime (integer in minutes)",
+  messages :["give me something spicy!" + " " + cacheBreaker],
+  mode: "json",
+};
+const result = await llm(promptPayload);
+console.log('JSON response from llm:', JSON.parse(result));
 \`\`\`
 
-## 3. readWebpage Function
+The \`result\` variable will be a stringified JSON object. Once you JSON.parse() it, you'll get the following object:
+
+{
+    "title": "Camarones a la Diabla (Devil Shrimp)",
+    "ingredients": [
+        "1.5 lbs Large Shrimp, peeled and deveined",
+        "4 tbsp Olive Oil",
+        "1 medium White Onion, finely chopped",
+        "4 cloves Garlic, minced",
+        "2-3 Habanero Peppers, finely chopped (adjust to your spice preference, remove seeds for less heat)",
+        "1 (28 oz) can Crushed Tomatoes",
+        "1/2 cup Chicken Broth",
+        "2 tbsp Tomato Paste",
+        "1 tbsp Apple Cider Vinegar",
+        "1 tbsp Dried Oregano",
+        "1 tsp Cumin",
+        "1/2 tsp Smoked Paprika",
+        "1/4 tsp Ground Cloves",
+        "Salt and Black Pepper to taste",
+        "Fresh Cilantro, chopped, for garnish",
+        "Lime wedges, for serving"
+    ],
+    "instructions": [
+        "In a large bowl, toss the shrimp with salt and pepper.",
+        "Heat the olive oil in a large skillet or Dutch oven over medium-high heat.",
+        "Add the onion and cook until softened, about 5 minutes.",
+        "Add the garlic and habanero peppers and cook for 1 minute more, until fragrant.",
+        "Stir in the crushed tomatoes, chicken broth, tomato paste, apple cider vinegar, oregano, cumin, smoked paprika, and cloves.",
+        "Bring the sauce to a simmer and cook for 15 minutes, stirring occasionally, until slightly thickened.",
+        "Add the shrimp to the sauce and cook for 3-5 minutes, or until the shrimp are pink and cooked through.",
+        "Taste and adjust seasoning with salt and pepper as needed.",
+        "Garnish with fresh cilantro and serve immediately with lime wedges. Serve with rice or tortillas."
+    ],
+    "prepTime": 20,
+    "cookTime": 30
+}
+
+## 4. readWebpage Function
 
 \`\`\`jsx
 async function fetchFromUrl() {
@@ -674,15 +759,15 @@ async function fetchFromUrl() {
 }
 \`\`\`
 
-## 4. generateImage Function
+## 5. generateImage Function
 
 \`\`\`jsx
 function ImageComponent() {
   return <img src={generateImage("A beautiful sunset over mountains")} alt="Generated landscape" />;
 }
 
 \`\`\`
-## 5. Using the Interface Functions
+## 6. Using the Interface Functions
 
 \`\`\`javascript
 // Import from modern ESM libraries:

jumble/integration/cache/llm-api-cache/62cab61d32bf8e93021879dfeece95b962e9a9ffd4ff5c5657d563b1f2de0da9.json

@@ -0,0 +1,15 @@
+{
+  "model": "anthropic:claude-3-7-sonnet-latest",
+  "system": "\nYou are creating a simple MVP based on the user's goal, using an existing data schema. Focus on the simplest implementation that works with the provided schema.\n\nGiven a user's feature request and an existing data schema, you will:\n1. Create a short title (2-5 words) that names the artifact\n2. Create a one-sentence description in the format \"A <artifact> to <goal>\"\n3. Create a concise specification (3-5 sentences max) that works with the existing schema\n4. Generate a brief implementation plan (3 steps max)\n5. Design a minimal JSON schema that represents the core data model\n\nYour response must be structured as follows:\n\n<title>\n[Short title for the artifact, 2-5 words]\n</title>\n\n<description>\n[One-sentence description in the format \"A <artifact> to <goal>\"]\n</description>\n\n<spec>\n[Concise specification that captures only the essential requirements]\n</spec>\n\n<plan>\n[Brief 3-step implementation plan using the existing schema]\n</plan>\n\n<result_schema>\n[Minimal JSON Schema in valid JSON format that represents data created by the artifact]\n</result_schema>\n\nSCHEMA GUIDELINES:\n1. Keep it minimal:\n   - Include only essential fields (5-7 properties max)\n   - Focus on the core functionality\n   - If user requested complex features, simplify for this first version\n\n2. Each property should have:\n   - A descriptive \"title\" field\n   - A brief \"description\" field\n   - A sensible default value where appropriate\n\n3. Example of a simple schema:\n```json\n{\n  \"type\": \"object\",\n  \"title\": \"Note\",\n  \"description\": \"A simple note for the user\",\n  \"properties\": {\n    \"title\": {\n      \"type\": \"string\",\n      \"title\": \"Title\",\n      \"description\": \"Title of the note\",\n      \"default\": \"New Note\"\n    },\n    \"content\": {\n      \"type\": \"string\",\n      \"title\": \"Content\",\n      \"description\": \"Content of the note\"\n    },\n    \"created\": {\n      \"type\": \"string\",\n      \"format\": \"date-time\",\n      \"title\": \"Created Date\",\n      \"description\": \"When the note was created\",\n    }\n  },\n  \"required\": [\"title\", \"content\"]\n}\n\nGUIDELINES:\n- Aim for the simplest possible solution that works with the existing schema\n- The specification should take into account the existing schema structure\n- Focus on what can be achieved quickly with the existing data model\n- Avoid suggesting modifications to the schema if possible\n\nIMPORTANT:\n- Focus on the simplest working version\n- Aim for fewer fields rather than more\n- But still capture all the important state the user is creating\n- The user can always iterate and improve the solution later\n",
+  "stream": false,
+  "messages": [
+    {
+      "role": "user",
+      "content": "\n\n<goal>count the values in current</goal>\n<plan><step>Analyze the user's request to understand what \"count the values in current\" means in this context</step>\n<step>Design a simple charm that counts values in an array or object</step>\n<step>Define the input/output data model for the charm</step>\n<step>Plan the implementation logic to handle different data types (arrays, objects, etc.)</step>\n<step>Consider edge cases like empty inputs, non-countable inputs, etc.</step></plan>\n<description>This charm counts the number of values in the provided input data. It's designed to be flexible, accepting various data types and returning a count of their elements or properties. For arrays, it counts the number of elements; for objects, it counts the number of properties; for strings, it counts the number of characters; and for other primitive values, it returns 1. The charm provides a simple utility function that helps users quickly determine the size or length of their data without having to write custom counting logic for different data types.</description>\n<data>\"Input: Any → can be array, object, string, or primitive value\\nProcess: Count elements/properties/characters based on input type\\nOutput: Number → the count of values\\n\\nArray[T] → count number of elements → Number\\nObject → count number of properties → Number\\nString → count number of characters → Number\\nPrimitive (number, boolean, etc.) → return 1 → Number\\nnull/undefined → return 0 → Number\\n\\nThe charm processes the input data by first determining its type, then applying the appropriate counting logic based on that type.\"</data>\n\n\nExisting Schema:\n```json\n{\n  \"type\": \"object\",\n  \"properties\": {\n    \"values\": {\n      \"type\": \"array\",\n      \"items\": {\n        \"type\": \"string\"\n      }\n    }\n  }\n}\n```\n\nBased on this goal and the existing schema, please provide a title, description, any additional schema,detailed specification, and implementation plan.\n"
+    },
+    {
+      "role": "assistant",
+      "content": "<title>\nValue Counter Charm\n</title>\n\n<description>\nA charm to count the number of values in the current data structure.\n</description>\n\n<spec>\nThis charm counts elements in the provided data based on its type. It will count the number of elements in the \"values\" array from the existing schema. The charm returns a simple count result with the total number of values and supports empty arrays by returning zero.\n</spec>\n\n<plan>\n1. Extract the \"values\" array from the input data\n2. Count the number of elements in the array\n3. Return a result object with the count value\n</plan>\n\n<result_schema>\n{\n  \"type\": \"object\",\n  \"title\": \"Count Result\",\n  \"description\": \"Result of counting values in the data structure\",\n  \"properties\": {\n    \"count\": {\n      \"type\": \"integer\",\n      \"title\": \"Count\",\n      \"description\": \"The number of values counted\",\n      \"default\": 0\n    },\n    \"source\": {\n      \"type\": \"string\",\n      \"title\": \"Source\",\n      \"description\": \"The source of the counted values\",\n      \"default\": \"values\"\n    },\n    \"isEmpty\": {\n      \"type\": \"boolean\",\n      \"title\": \"Is Empty\",\n      \"description\": \"Whether the source contains no values\",\n      \"default\": false\n    },\n    \"timestamp\": {\n      \"type\": \"string\",\n      \"format\": \"date-time\",\n      \"title\": \"Timestamp\",\n      \"description\": \"When the counting was performed\"\n    }\n  },\n  \"required\": [\"count\", \"source\", \"isEmpty\", \"timestamp\"]\n}\n</result_schema>"
+    }
+  ]
+}

jumble/integration/cache/llm-api-cache/867276562b188f66b047b5f0e25a7551a42a4d04206c08ab88f2698b3ed563ba.json

@@ -7,7 +7,7 @@
     },
     {
       "role": "assistant",
-      "content": "<steps>\n1. Analyze the user's request to understand what \"count the values in current\" means in this context\n2. Design a simple charm that counts values in an array or object\n3. Define the input/output data model for the charm\n4. Plan the implementation logic to handle different data types (arrays, objects, etc.)\n5. Consider edge cases like empty inputs, non-countable inputs, etc.\n</steps>\n\n<specification>\nThis charm will count the values in the input data provided to it. The purpose is to analyze the input and return a count of its elements or properties. For arrays, it will count the number of elements. For objects, it will count the number of properties. For strings, it will count the number of characters. For other primitive values (numbers, booleans, null), it will return 0 or 1 depending on whether the value exists. This charm provides a simple utility function that can be useful in data analysis, validation, or when making decisions based on the size of data structures.\n</specification>\n\n<data_model>\nInput: any → can be array, object, string, number, boolean, or null\nProcess: Count elements/properties/characters based on input type\nOutput: number → the count of values\n\nProcessing flow:\n- Array input [a, b, c] → Count: 3\n- Object input {a: 1, b: 2} → Count: 2\n- String input \"hello\" → Count: 5\n- Number input 42 → Count: 1\n- Boolean input true → Count: 1\n- Null input null → Count: 0\n- Undefined input undefined → Count: 0\n</data_model>"
+      "content": "<steps>\n1. Analyze the user's request to understand what \"count the values in current\" means in this context\n2. Design a simple charm that counts values in an array or object\n3. Define the input/output data model for the charm\n4. Plan the implementation logic to handle different data types (arrays, objects, etc.)\n5. Consider edge cases like empty inputs, non-countable inputs, etc.\n</steps>\n\n<specification>\nThis charm counts the number of values in the provided input data. It's designed to be flexible, accepting various data types and returning a count of their elements or properties. For arrays, it counts the number of elements; for objects, it counts the number of properties; for strings, it counts the number of characters; and for other primitive values, it returns 1. The charm provides a simple utility function that helps users quickly determine the size or length of their data without having to write custom counting logic for different data types.\n</specification>\n\n<data_model>\nInput: Any → can be array, object, string, or primitive value\nProcess: Count elements/properties/characters based on input type\nOutput: Number → the count of values\n\nArray[T] → count number of elements → Number\nObject → count number of properties → Number\nString → count number of characters → Number\nPrimitive (number, boolean, etc.) → return 1 → Number\nnull/undefined → return 0 → Number\n\nThe charm processes the input data by first determining its type, then applying the appropriate counting logic based on that type.\n</data_model>"
     }
   ],
   "model": "anthropic:claude-3-7-sonnet-latest",