chore: update seeder for testing #1123
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Seeder CI is now (correctly) failing when generating an error (this is due to needing LLM cache) |
jsantell
commented
May 5, 2025
jsantell
commented
May 5, 2025
jsantell
commented
May 5, 2025
* Add a flag for skipping verification for smoke tests (`--no-verify`) * Lazily load `openai` dependency as to not require credentials * Only use browser during verification * Add a flag for skipping report generation (`--no-report`) * Return an error exit code if any verification fails (unless `--no-verify`) * Connect a single charm gen to a parallel integration test to ensure seeder runs properly. No verification is done. * Align shared usage of utils between seeder/jumble-integration
anotherjesse
reviewed
May 5, 2025
| import { createSession, Identity } from "@commontools/identity"; | ||
| import { CharmManager } from "@commontools/charm"; | ||
| import { sleep } from "@commontools/utils/sleep"; |
There was a problem hiding this comment.
this one is funny since it is almost as long as the code required to define sleep 🤣
anotherjesse
approved these changes
May 5, 2025
| await waitForSelectorClick( | ||
| page, | ||
| "button[aria-label='register-with-passphrase']", | ||
| // Check if we need to select a method first (in case of multiple auth methods available) |
There was a problem hiding this comment.
love that these are sharing a codepath now!
| @@ -19,50 +19,6 @@ export const ensureReportDir = async (name: string) => { | |||
| } | |||
| }; | |||
|
|
|||
| // Helper function to group results by scenario | |||
| function groupResultsByScenario( | |||
There was a problem hiding this comment.
thx for deleting ... left over from the vibecoded reports
| @@ -13,5 +13,6 @@ deno check \ | |||
| jumble \ | |||
| memory \ | |||
| runner \ | |||
| seeder \ | |||
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
--no-verify)openaidependency as to not require credentials--no-report)--no-verify)