smoketest version of ralph, basic files (#1951)

* smoketest version of ralph, basic files

* temporarily use git branch

* ask ralph to use playwright

* telling it how to deploy locally

* how to use smoketest doc

* cleanup formatting for TASKS.md

* added script to run several ralph smoketests and stop them

* clean up to point back to main, ready for landing

* added note to self to clean up sleep infinity later

Author: ellyxir

.gitignore

@@ -23,3 +23,4 @@ claude-research.key
 local-dev-toolshed.log
 local-dev-shell.log
 tools/ralph/logs/
+tools/ralph/smoketest/

tools/ralph/Dockerfile

@@ -76,6 +76,8 @@ RUN npm install -g @anthropic-ai/claude-code && \
 # --no-sandbox is required because Docker containers restrict namespace creation
 RUN claude mcp add --scope user playwright npx "@playwright/mcp@latest" -- --headless --isolated --no-sandbox
 
-# Start Common Tool servers in background and keep container alive
-# the sleep ensures that restarting the server doesnt cause the container to exit
-CMD ["/bin/sh", "-c", "/app/start-servers.sh & sleep infinity"]
+# Start servers in background, run smoketest, then exit
+# If you want to run ralph normally without the smoketest, then replace with:
+# CMD ["/bin/sh", "-c", "/app/start-servers.sh & sleep infinity"]
+# TODO(@ellyxir): remove sleep infinity, smoketest should exit when its done
+CMD ["/bin/sh", "-c", "/app/start-servers.sh & /app/labs/tools/ralph/bin/ralph-smoketest.sh; sleep infinity"]

tools/ralph/README.md

@@ -6,7 +6,73 @@ Ability to run [Ralph](https://ghuntley.com/ralph/)
 
 Claude CLI and Codex are installed
 
-## How to run Ralph
+## How to use Smoketest Ralph
+
+Smoketest Ralph runs a single task from `TASKS.md` and exits when complete. Each
+Ralph instance is assigned a specific task via the `RALPH_ID` environment
+variable.
+
+### Running smoketests
+
+**Option 1: Use the automated script (recommended)**
+
+Run multiple smoketests in parallel (tasks 1-3):
+
+```bash
+./tools/ralph/bin/run_smoketest.sh
+```
+
+This script will:
+
+- Clean up old results
+- Stop/remove any existing ralph containers
+- Start 3 smoketest containers in parallel (RALPH_ID 1, 2, and 3)
+- Results are written to `./tools/ralph/smoketest/<ID>/`
+
+Monitor progress with:
+
+```bash
+docker logs ralph_1  # or ralph_2, ralph_3
+```
+
+To stop all running smoketests:
+
+```bash
+./tools/ralph/bin/stop_smoketest.sh
+```
+
+**Option 2: Run a single smoketest manually**
+
+1. Build the Docker image (if not using pre-built):
+
+```bash
+docker build -t ellyxir/ralph tools/ralph/
+```
+
+2. Run the container with your RALPH_ID and mounted credentials:
+
+```bash
+cd ~/labs
+docker run -e RALPH_ID=3 -d -v ~/.claude.json:/home/ralph/.claude.json \
+  -v ~/.claude/.credentials.json:/home/ralph/.claude/.credentials.json \
+  -v ./tools/ralph/smoketest:/app/smoketest --name ralph ellyxir/ralph
+```
+
+Note: The container will exit automatically when the smoketest completes.
+
+### Retrieving results
+
+Results are available on the host machine in
+`./tools/ralph/smoketest/${RALPH_ID}/`:
+
+- `SCORE.txt` - Contains SUCCESS, PARTIAL, or FAILURE
+- `RESULTS.md` - Summary of work including test results
+- Pattern files created during the task
+
+No need to copy files from the container - the bind mount makes results
+immediately available on your host.
+
+## How to run Ralph (not smoketest)
 
 ### Using pre-built image from Docker Hub (recommended)
 

tools/ralph/SMOKETEST_PROMPT.md

@@ -0,0 +1,40 @@
+# Smoketest Ralph General Prompt
+
+Goal: implement the unchecked item from `./tools/ralph/TASKS.md` that matches
+your assigned RALPH_ID
+
+1. Open `./tools/ralph/TASKS.md` and find the task numbered with your RALPH_ID.
+
+2. If your assigned task is already checked `[x]`, exit with a message saying
+   the task is already complete.
+
+3. Use Claude Skills "recipe-dev" to work on the task that corresponds to your
+   RALPH_ID number.
+
+4. Format with `deno fmt` for the changed files.
+
+5. Once tests pass, deploy it locally using `./docs/common/RECIPE_DEV_DEPLOY.md`
+   for info on how to deploy. It should deploy to http://localhost:8080 and not
+   to toolshed. Servers are already running.
+
+6. Once it is deployed locally, use a subagent to test your work with MCP
+   playwright
+
+7. Check off the completed items in `TASKS.md`:
+
+8. git stage and commit with a message
+
+9. Copy the files you created for the task to /app/smoketest/${RALPH_ID}/
+
+10. Create a summary of your work in the same directory, be sure to include
+    playwright results and what you tested. file location:
+    /app/smoketest/${RALPH_ID}/RESULTS.md
+
+11. Create a /app/smoketest/${RALPH_ID}/SCORE.txt which has one of the following
+    values based on your results: SUCCESS, PARTIAL, FAILURE
+
+12. Add feedback to documentation to `./tools/ralph/LEARNINGS.md`.
+
+13. Exit
+
+Please begin.

tools/ralph/TASKS.md

@@ -1,33 +1,54 @@
 # Ralph Task List
 
-A running checklist of tasks. New items include brief implementation notes for a
-future Ralph pass.
-
-Tasks marked with [UI] mean they should add/remove/modify the UI of the pattern.
-UI tasks should wire up the functionality and call the appropriate handlers.
-
-- [] Create a counter
-  - [] Add [UI] buttons for incrementing counter
-    - **Test with Playwright**: click increment 3 times, verify counter shows 3,
-      click decrement once, verify counter shows 2
-    - **State**: The displayed count matches the pattern's `count` output field
-  - [] Create multiple counters
-    - [] Add [UI] buttons to create multiple counters
-      - **Test with Playwright**: create 3 counters, test each one
-      - **State**: Each counter maintains its own value in the pattern's
-        `counters` array
-- [] Create a shopping list
-  - [] Create [UI] for shopping list
-    - **Test with Playwright**: add "milk" and "bread", make sure you see both,
-      remove "bread", verify only "milk" remains
-    - **State**: The list shows all items from pattern's `items` array with
-      correct `completed` status
-- [] Lunch voter - list of destinations (just a string) (dedup)
-  - [] [UI] for adding list of destination (just a string) and displaying it
-    - **UI must**: Show an editable list with add/remove buttons for
-      destinations
-    - **Test with Playwright**: Deploy pattern, add at least 2 destinations via
-      UI, verify they appear in the list, remove one destination, verify it's
-      removed from both UI and charm output
-    - **State**: The displayed list matches the pattern's `destinations` output
-      field
+1. [ ] Counter
+   - Components: Count display + Increment/decrement buttons + Reset button
+   - Data: Current count value
+   - Features: Increment, decrement, reset to zero
+
+2. [ ] Shopping List with sort-by-category and budget tracking. These should be
+       3 different patterns (shopping list, category list, and budget tracker)
+       and a final pattern that combines them together and acts as a launcher.
+   - Components: Shopping list (item input + checkboxes + clear button) +
+     Category list (category input + item assignment) + Budget tracker (price
+     input + total display) + Launcher (tabs/buttons to switch between views)
+   - Data: Shopping items with name, category, price, checked status; Categories
+     with names
+   - Features: Add/remove items, assign categories, track prices, sort by
+     category, view budget totals, check off purchased items
+
+3. [ ] Calendar
+   - Components: Month view with day cells + Event list displayed in calendar +
+     Day editor (opens when clicking a day)
+   - Data: Events with date, time, description
+   - Features: View one month at a time, click day to edit its event list,
+     events shown in calendar UI
+
+4. [ ] Fitness Workout Planner
+   - Components: Exercise routine builder + Set/rep counter + Progress chart
+   - Data: Exercises with sets, reps, weight
+   - Features: Track personal records, show strength gains over time
+
+5. [ ] Lunch Voter
+   - Components: Restaurant list + Voting buttons + Vote tally display +
+     Add/remove restaurant form
+   - Data: Restaurants with vote counts
+   - Features: Add/remove restaurants, vote for favorites, see most popular
+     choice
+
+6. [ ] Study Schedule with Focus Timer
+   - Components: Study task list + Time block scheduler + Pomodoro timer + Break
+     reminders
+   - Data: Study topics, estimated duration, completion status
+   - Features: Schedule study sessions, track time spent, enforce breaks
+
+7. [ ] Travel Itinerary with Budget Tracker
+   - Components: Activity scheduler + Day-by-day timeline + Expense tracker +
+     Budget dashboard
+   - Data: Activities with time, location, cost
+   - Features: Plan entire trip, track expenses by category, budget warnings
+
+8. [ ] Contact Manager with Birthday Reminders
+   - Components: Contact list + Upcoming birthdays view + Gift idea notes +
+     Calendar integration
+   - Data: Contacts with birthdays, gift history
+   - Features: Birthday notifications, gift suggestions, relationship notes

tools/ralph/bin/ralph-smoketest.sh

@@ -0,0 +1,44 @@
+#!/usr/bin/env bash
+
+# Get the directory where this script is located
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# Get the ralph directory (parent of bin)
+RALPH_DIR="$(dirname "$SCRIPT_DIR")"
+# Get the labs directory (two levels up from bin)
+LABS_DIR="$(dirname "$(dirname "$RALPH_DIR")")"
+
+# Change to labs directory for relative paths to work
+cd "$LABS_DIR"
+
+# Check RALPH_ID is set
+if [ -z "$RALPH_ID" ]; then
+  echo "Error: RALPH_ID environment variable is not set"
+  exit 1
+fi
+
+# Ensure logs directory exists
+mkdir -p ./tools/ralph/logs
+
+# Rotate logs keeping last 5
+for i in 4 3 2 1; do
+  [ -f ./tools/ralph/logs/ralph-claude.log.$i ] && mv ./tools/ralph/logs/ralph-claude.log.$i ./tools/ralph/logs/ralph-claude.log.$((i+1))
+done
+[ -f ./tools/ralph/logs/ralph-claude.log ] && mv ./tools/ralph/logs/ralph-claude.log ./tools/ralph/logs/ralph-claude.log.1
+
+# llm command to summarize changes
+LLM="./tools/ralph/bin/llm.sh"
+
+{ printf "Your RALPH_ID is %s.\n\n" "$RALPH_ID"; cat ./tools/ralph/SMOKETEST_PROMPT.md; } | \
+claude --print --dangerously-skip-permissions \
+--verbose --output-format=stream-json 2>&1 | \
+tee -a ./tools/ralph/logs/ralph-claude.log
+
+# Auto-stash changes if any exist
+if [[ -n "$(git status --porcelain)" ]]; then
+  git add -A
+
+  # Generate commit message from staged changes
+  commit_msg=$(git diff --staged | $LLM "Summarize these changes into a short one-line description, output just that one line")
+
+  git stash push -m "$commit_msg"
+fi

tools/ralph/bin/run_smoketest.sh

@@ -0,0 +1,33 @@
+#!/usr/bin/env bash
+
+# Get the directory where this script is located
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# Get the ralph directory (parent of bin)
+RALPH_DIR="$(dirname "$SCRIPT_DIR")"
+# Get the labs directory (two levels up from bin)
+LABS="$(dirname "$(dirname "$RALPH_DIR")")"
+
+# Change to labs directory
+cd "$LABS"
+
+# Stop and remove any existing ralph containers first
+for ID in 1 2 3; do
+  docker stop ralph_$ID 2>/dev/null || true
+  docker rm ralph_$ID 2>/dev/null || true
+done
+
+# Remove existing results after containers are stopped
+rm -rf "$LABS/tools/ralph/smoketest"/[0-9]*
+
+# Run smoketests for IDs 1 through 3
+for ID in 1 2 3; do
+  echo "Starting smoketest for RALPH_ID=$ID"
+  docker run --rm -e RALPH_ID=$ID -d \
+    -v ~/.claude.json:/home/ralph/.claude.json \
+    -v ~/.claude/.credentials.json:/home/ralph/.claude/.credentials.json \
+    -v "$LABS/tools/ralph/smoketest:/app/smoketest" \
+    --name ralph_$ID \
+    ellyxir/ralph
+done
+
+echo "All smoketests started. Use 'docker logs ralph_<ID>' to monitor progress."

tools/ralph/bin/stop_smoketest.sh

@@ -0,0 +1,10 @@
+#!/usr/bin/env bash
+
+# Stop and remove all ralph smoketest containers
+for ID in 1 2 3; do
+  echo "Stopping ralph_$ID..."
+  docker stop ralph_$ID 2>/dev/null || true
+  docker rm ralph_$ID 2>/dev/null || true
+done
+
+echo "All smoketest containers stopped and removed."