smoketest version of ralph, basic files

[ellyxir]

Summary by cubic

Add a smoketest mode for Ralph that runs servers, executes a guided prompt, and then exits. Dockerfile now runs the smoketest by default, and new files guide task execution and capture results.

• New Features

  • Added ralph-smoketest.sh to run the SMOKETEST_PROMPT via Claude, rotate logs, and auto-stash changes with an LLM commit summary.
  • Added SMOKETEST_PROMPT.md with a step-by-step flow (formatting, tests, artifacts, scoring, learnings).
  • Updated TASKS.md with numbered tasks (Counter, Shopping List suite, Calendar, etc.) aligned to RALPH_ID.
  • Changed Dockerfile CMD to start servers and run ralph-smoketest.sh.

• Migration

  • Set RALPH_ID before running.
  • To run Ralph normally, change Dockerfile CMD to start-servers.sh & sleep infinity.
  • Smoketest outputs live in /app/smoketest/${RALPH_ID}/; logs in tools/ralph/logs/.

[cubic-dev-ai]

2 issues found across 6 files

Prompt for AI agents (all 2 issues)

Understand the root cause of the following 2 issues and fix them.


<file name="tools/ralph/Dockerfile">

<violation number="1" location="tools/ralph/Dockerfile:88">
`sleep infinity` keeps the container alive even after the smoketest finishes (and masks its exit status). The smoketest can’t signal failure or allow the container to exit. Remove the trailing `sleep infinity` so the shell exits with the smoketest result.</violation>
</file>

<file name="tools/ralph/TASKS.md">

<violation number="1" location="tools/ralph/TASKS.md:3">
This line introduces a Markdown bullet with no content, which renders as a blank bullet before each task. Remove the stray &#39;-&#39; or merge it with the task entry to keep the checklist readable.</violation>
</file>

_{React with 👍 or 👎 to teach cubic. Mention @cubic-dev-ai to give feedback, ask questions, or re-run the review.}

[ellyxir]

i think this is good for the first batch to enable smoke testing.
i have a bunch of things i still want to do that i can add in a followup PR:

[ ] i set up mcp playwright in Dockerfile but then we rewrite the setup with the bindmounts, need to fix this
[ ] add timing to the test, maybe in SCORE.txt or TIME.txt
[ ] change deploy step to use a skill, its a lot of text right now
[ ] have run_smoketest.sh script docker cp the .claude.json and .claude directory with docker cp, dont use -v bind mounts
[ ] enhance the smokerun run script to periodically print how many containers are still running
[ ] enhance the smokerun run script to summarise results and give a final avg score and total time
[ ] copy the playwright screenshots to the bind mounts
[ ] instead of passing ralph_id, have script slice the right task and insert the task directly into the prompt
[ ] once ben's changes land, need to update reference from "recipe-dev" to "pattern-dev" skill

Cleanup:
[ ] remove the sleep infinity in the Dockerfile
[ ] edit run and stop smoketest scripts to run 1-9