-
Notifications
You must be signed in to change notification settings - Fork 9
Ellyse/smoketest mount labs #1965
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Remove git clone from Dockerfile, use bind mount instead
- Remove MCP setup from Dockerfile (done in run script after credentials copied)
- Add 5 second sleep for mount to settle
- Update run_smoketest.sh to mount labs dir and copy credentials
- Configure Playwright MCP after copying credentials to avoid overwrite
- Update SMOKETEST_PROMPT.md with working Playwright MCP instructions
- Change recipe-dev to pattern-dev, fix paths, update port to 8000
- Add space name requirement (ralph${RALPH_ID}) to avoid parallel conflicts
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Redirect Ralph smoketest output to /app/smoketest/${RALPH_ID}/ralph.log
- Update README to document log location and bind mount behavior
- Users can now tail -f the logs instead of using docker logs
- All smoketest outputs (logs, results, patterns) now in one directory
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Changed SMOKETEST_PROMPT.md to create patterns in /app/smoketest/${RALPH_ID}/ instead of /app/labs/packages/patterns/
- Removed step 7 (copying pattern files) since they're now created in the final location
- Renumbered remaining steps (8→7, 9→8, 10→9)
- Updated README.md to document that patterns are created directly in smoketest directory for automatic cleanup
This approach simplifies the workflow and ensures patterns are automatically cleaned up when smoketests rerun. Tested successfully with ralph_1 container.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Container now exits when smoketest completes instead of staying running. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Script now monitors running containers and prints status every 10 seconds until all smoketests complete. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Show "Still running: 1 2 3" instead of "Still running: ralph_1 ralph_2 ralph_3" for cleaner output. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Show users where to find logs after smoketests start. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
- Updated SMOKETEST_PROMPT.md to instruct Ralph to name screenshots as ralph_${RALPH_ID}-<description>.png
- Updated run_smoketest.sh to automatically move screenshots to appropriate smoketest directories after completion
- Prevents screenshot conflicts between parallel Ralph instances
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Display where to find results and summarize how many SUCCESS/PARTIAL/FAILURE scores across all completed smoketests. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
- Added RALPH_IDS="1 2 3 4 5 6 7 8" to run_smoketest.sh for active tasks - Added RALPH_IDS="1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20" to stop_smoketest.sh (generous range to catch any running containers) - Replaced all hardcoded loops with $RALPH_IDS for easier configuration - Updated README with instructions on adding new tasks 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Line wrapping adjustment for screenshot naming instruction. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
1 issue found across 5 files
Prompt for AI agents (all 1 issues)
Understand the root cause of the following 1 issues and fix them.
<file name="tools/ralph/Dockerfile">
<violation number="1" location="tools/ralph/Dockerfile:78">
The new entrypoint tries to mkdir /app/smoketest/${RALPH_ID}, but that bind mount is root-owned when Docker creates it. Since the container runs as the unprivileged ralph user, mkdir fails with “Permission denied”, so the smoketest never starts. Please ensure the directory is writable (e.g., create/chown it before switching users).</violation>
</file>
React with 👍 or 👎 to teach cubic. Mention @cubic-dev-ai to give feedback, ask questions, or re-run the review.
- Run containers as host user with -u $(id -u):$(id -g) to avoid UID mismatch - Pre-create smoketest directories on host with correct ownership - Added prerequisites to README: - Docker installation link - User must be in docker group (explains sudo credential issue) - User must be logged into Claude CLI - Removed mkdir from Dockerfile since host creates directories This fixes permission issues on systems where container UID ≠ host UID. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
- Mount ~/.claude.json and ~/.claude as read-only volumes to /tmp/home - Set HOME=/tmp/home so container can write config files - Ensure smoketest parent directory exists before creating subdirs - Removed docker cp commands (no longer needed) This allows containers to run as host user without permission errors when accessing /home/ralph (which is owned by the ralph user created in Dockerfile). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
- Removed shared ~/.claude mounts that caused conflicts between containers - Copy credentials into each container's /tmp/home (writable for host user) - Each container gets its own isolated copy of Claude credentials - Fixes read-only filesystem error and prevents cross-container conflicts 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Moved installation from ralph user to root so they install to /usr/local/bin instead of /home/ralph/.npm-global/bin. This fixes MODULE_NOT_FOUND errors when container runs as host user with different UID.
- Document automatic image pull from Docker Hub - Update manual run instructions to use host user and credential copying - Replace outdated credential mount approach with current implementation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary by cubic
Switch smoketests to use a bind-mounted labs directory and a Playwright MCP testing flow, enabling parallel runs with per-task outputs. Containers now exit when done; logs, patterns, and screenshots land in each task’s directory for easy monitoring and cleanup.
New Features
Migration