-
Notifications
You must be signed in to change notification settings - Fork 9
feat(builder): Merge OpaqueRef and Cell #2024
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
7 issues found across 48 files
Prompt for AI agents (all 7 issues)
Understand the root cause of the following 7 issues and fix them.
<file name="packages/generated-patterns/integration/patterns/counter-delayed-compute.pattern.ts">
<violation number="1" location="packages/generated-patterns/integration/patterns/counter-delayed-compute.pattern.ts:25">
Using derive here returns a computed value but never drains the pending queue or writes the new total back to the underlying cell. Because derive’s callback only receives plain values, the increments remain queued and rawValue never advances, breaking the counter’s persistence.</violation>
</file>
<file name="packages/runner/src/builder/types.ts">
<violation number="1" location="packages/runner/src/builder/types.ts:109">
Rename this interface back to IOpaquable so the module augmentation still merges into the upstream interface instead of defining a brand-new IOxpaquable type.</violation>
</file>
<file name="packages/runner/test/wish.test.ts">
<violation number="1" location="packages/runner/test/wish.test.ts:39">
Remove `.only` so the full test suite runs. Leaving this focused spec in committed code will cause the rest of the suite to be skipped during CI.</violation>
</file>
<file name="packages/runner/src/builder/opaque-ref.ts">
<violation number="1" location="packages/runner/src/builder/opaque-ref.ts:26">
`opaqueRef` now throws whenever the current frame lacks a runtime, but builder code (e.g. `recipe()`) still pushes frames without a runtime before calling `opaqueRef`, so recipe construction now crashes instead of producing a ref.</violation>
</file>
<file name="packages/api/index.ts">
<violation number="1" location="packages/api/index.ts:44">
`[CELL_BRAND]` should stay typed as `Kind`; unioning with `string` collapses the brand to `string` and destroys the compile-time differentiation between cell kinds.</violation>
</file>
<file name="packages/runner/src/builder/recipe.ts">
<violation number="1" location="packages/runner/src/builder/recipe.ts:292">
Falling back to `true` here drops the default-derived argument schema, so recipes without explicit schemas lose the structure/default metadata that downstream tooling relies on. Please restore the schema derivation instead of returning `true`.</violation>
</file>
<file name="packages/runner/src/runtime.ts">
<violation number="1" location="packages/runner/src/runtime.ts:422">
Always pass the stored default frame to popFrame so disposal removes the correct frame and surfaces any mismatch.</violation>
</file>
React with 👍 or 👎 to teach cubic. Mention @cubic-dev-ai to give feedback, ask questions, or re-run the review.
packages/generated-patterns/integration/patterns/counter-delayed-compute.pattern.ts
Show resolved
Hide resolved
5173b33 to
77aa836
Compare
…l implementations
Implements the ability to create cells before they have a full link,
allowing deferred link creation based on cause information. This also
unifies the previous RegularCell and StreamCell implementations into a
single CellImpl class.
Key architectural changes:
- Merged RegularCell and StreamCell into unified CellImpl class
- Stream detection now happens at runtime via isStream() check
- Both regular cells and streams share the same interface
- .set() automatically handles stream vs regular cell behavior
- Use NormalizedLink (with optional id/space) instead of requiring NormalizedFullLink
- Cells can start with just { path: [] } and get id/space populated later
- hasFullLink() checks for presence of both id and space fields
- ensureLink() populates missing fields from cause information
- Add .for(cause) method to associate causes with cells
- Stores cause for later link creation
- Supports optional { force } parameter for future extensions
- Returns cell for method chaining
- Deferred link creation using createRef()
- ensureLink() creates entity IDs from causes when needed
- Uses frame context or explicit .for() cause
- Helpful error messages when context is insufficient
- Update .key() to build paths incrementally
- Works with partial links (just path[])
- Inherits cause from parent cell
- Seamless transition to full links when needed
API simplification:
- Removed isStream() checks from client code (now handled internally)
- Updated createCell() signature (removed noResolve parameter)
- Consistent Cell interface regardless of stream/regular distinction
Test coverage:
- 23 new tests for optional link functionality
- All 124 existing cell tests still passing
- Full type checking passes
Rollout plan updates:
- Marked completed tasks in rollout-plan.md
- Documented implementation approach
This enables more flexible cell creation patterns where cells can be
created and manipulated before their final identity is determined,
which is essential for the recipe construction work.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Implement Strategy 1 for sharing link creation across sibling cells. When cells like .asSchema() and .withTx() create siblings, they now share a CauseContainer that holds the entity ID and cause. Key changes: - Added CauseContainer interface with id (URI) and cause fields - Each cell has own _link (space, path, schema) but shares _causeContainer - .for() sets cause in shared container, visible to all siblings - .key() creates children with new CauseContainer (different identity) - .asSchema() and .withTx() create siblings sharing CauseContainer - ensureLink() populates shared container's id from cause - Constructor signature: runtime, tx, link?, synced?, causeContainer? This allows any sibling to trigger link creation via .for() or ensureLink(), and all siblings see the created ID. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Change the .for() method to throw an error by default if a cause is
already set, with an optional boolean parameter to allow treating it
as a suggestion.
Changes:
- Renamed parameter from `options: { force?: boolean }` to `allowIfSet?: boolean`
- Default behavior: throw error if cause/link already exists
- Pass `true` to silently ignore if cause already set (treat as suggestion)
- Moved .for() signature to IAnyCell interface for consistency
- Updated tests to verify new behavior
This makes the API safer by preventing accidental overwrites while still
allowing flexibility when needed.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Audit and fix frame creation to ensure space is always available from the result cell context: 1. Modified pushFrameFromCause() to: - Extract space from unsafe_binding and set on frame.space - Accept inHandler boolean parameter (replaces event field) - This makes frame.space available as fallback in cell.ts 2. Renamed Frame.event to Frame.inHandler: - More accurate semantic: indicates handler context, not event data - Used for per-frame counter-based ID generation fallback - Simpler boolean check vs checking event truthiness 3. Updated CauseContainer to store space: - Container now holds id, space, and cause for sharing across siblings - ensureLink() checks if container has both id and space before deriving - Space from container takes precedence over frame space 4. Updated event handler in runner.ts: - Pass inHandler: true to pushFrameFromCause() - Frame now has both space (from processCell.space) and inHandler flag Benefits: - frame.space fallback in cell.ts now works correctly - clearer semantics with inHandler vs event - space always available from result cell in runner contexts - CauseContainer properly shares space across siblings
Implements the "First merge of OpaqueRef and RegularCell" from the rollout plan by adding OpaqueRef-like methods to Cell, making them API-compatible. ## Changes ### Cell implementation (cell.ts) - Add OpaqueRef methods: setPreExisting, setDefault, setSchema, connect, export, map, mapWithPattern - Update toJSON() to return null when no link exists (matches OpaqueRef behavior) - Update [toOpaqueRef]() to return this instead of creating new OpaqueRef - Add cellNodes WeakMap to track connected nodes per cell instance - Add shared mapFactory for map operations ### Type declarations - Update module augmentation to include all new methods - Update toJSON return type to LegacyJSONCellLink | null - Update export() return type to include nodes and frame ### OpaqueRef implementation (opaque-ref.ts) - Add documentation clarifying methods are shared with Cell - Update toJSON comment to note it matches Cell behavior ### Query result proxy (query-result-proxy.ts) - Update makeOpaqueRef with TODO for future Cell-based refactoring ## Key Design Decisions 1. **API Compatibility**: Cell now implements all OpaqueRef methods, making them interchangeable at the API level 2. **WeakMap for nodes**: Cell uses WeakMap instead of instance field to keep implementation clean 3. **Deprecated methods**: setPreExisting and setDefault marked deprecated with console warnings 4. **Lifecycle separation**: OpaqueRef for recipe construction (no runtime), Cell for recipe execution (has runtime) ## Testing - All tests pass (1139 steps across all runner tests) - Cell tests: 124 steps ✓ - OpaqueRef tests: 8 steps ✓ - Recipe tests: 23 steps ✓ - Integration tests: 23 steps ✓ 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
After merging dx1/disable-old-closure-support, fix type error where isOpaqueRef was used but not defined. The correct function is isOpaqueCell. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Implement createOpaqueRefFactory that creates OpaqueRefs backed by actual Cells instead of custom proxy implementations. This unifies the Cell and OpaqueRef implementations. ## Key Changes ### opaque-ref.ts - Add createOpaqueRefFactory(runtime) that returns an opaqueRef function bound to a runtime - The factory creates actual Cells underneath instead of custom proxies - Add minimal Proxy wrapper around Cell for: - Symbol.iterator support (array destructuring) - Symbol.toPrimitive that throws error directing users to use derive - isOpaqueRefMarker support - setName/setDefault/export methods that track OpaqueRef-specific metadata - Keep legacy opaqueRef for backward compatibility (used in recipe construction without runtime) ### factory.ts - Use createOpaqueRefFactory(runtime) in createBuilder - Bind the runtime-aware opaqueRef to commontools.cell ## How It Works 1. **During pattern execution** (with runtime): - createOpaqueRefFactory creates Cells directly using createCell - Proxy wrapper adds iterator and toPrimitive support - All Cell methods (connect, export, map, etc.) work natively 2. **During recipe construction** (without runtime): - Legacy opaqueRef still creates custom proxies - Used by recipe(), lift(), handler() before runtime is available ## Benefits - OpaqueRef and Cell now share the same underlying implementation - Reduces code duplication - Cell methods work directly without delegation - Better error messages via toPrimitive ## Testing - All tests pass (132 steps in cell/opaque-ref tests) - Integration tests pass (22 steps in recipes tests) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
…eRefs This is the first major step in merging OpaqueRef and Cell implementations. All builder functions now use Cell-backed OpaqueRefs during recipe execution. - Renamed `createOpaqueRefFactory` to `opaqueRefImpl` that takes runtime as first parameter - Fixed recursive proxy wrapping - child cells are now properly wrapped in proxies - Added support for `toCell` symbol to access unproxied Cell from proxy - Created `*Impl` versions that take runtime as first parameter: - `createNodeFactoryImpl`, `liftImpl`, `handlerImpl`, `deriveImpl` - `byRefImpl`, `computeImpl`, `renderImpl` - Legacy versions now throw errors directing to use `*Impl` from factory - Wrapped all built-ins in `createBuiltIns(runtime)` factory function - Includes: `str`, `ifElse`, `llm`, `llmDialog`, `generateObject`, `fetchData`, `streamData`, `compileAndRun`, `navigateTo`, `wish`, `patternTool` - Creates runtime-bound wrappers: `(...args) => *Impl(runtime, ...args)` - All builder functions now properly pass runtime through - Updated Cell's export() to include optional properties: `value`, `defaultValue`, `name`, `external` - Fixed interface conflicts between IAnyCell and IOpaquable - Removed duplicate `map` and `mapWithPattern` from IAnyCell augmentation - Aligned export types across Cell and OpaqueRef - Tests currently failing due to legacy `createNodeFactory` calls in: - `packages/runner/src/module.ts` - `packages/runner/src/builtins/index.ts` - These are runtime initialization builtins that need separate refactoring **Two-phase approach:** 1. **Recipe construction** (no runtime): Uses legacy opaqueRef for creating recipe definitions 2. **Recipe execution** (has runtime): Uses Cell-backed opaqueRef via factory This commit completes phase 2, making all pattern execution use Cells underneath. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Clean up remaining changes from the OpaqueRef/Cell refactoring: - Remove setDefault test assertions (functionality removed in favor of schemas) - Update test expectations for legacy opaqueRef export behavior - Minor formatting fixes in related files Note: Tests still failing due to runtime builtin initialization needing refactoring. This will be addressed in a follow-up commit. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
…fProxy() Moved the OpaqueRef proxy wrapping logic from `opaqueRefImpl` into a new `getAsOpaqueRefProxy()` method on CellImpl. This is cleaner architecturally and significantly simplifies the code. ## Changes - Added `getAsOpaqueRefProxy()` method to CellImpl (cell.ts:950-984) - Handles Symbol.iterator, Symbol.toPrimitive, and toCell - Recursively wraps child cells via property access - No longer needs to manually track schemas - `.key()` handles it - Simplified `opaqueRefImpl` (opaque-ref.ts:31-51) - Reduced from ~65 lines to ~20 lines - Just creates cell, sets schema/value, calls `.getAsOpaqueRefProxy()` - Removed the inline `wrapCell` function - Added method declaration to IAnyCell interface ## Benefits - Better encapsulation - proxy logic lives on Cell where it belongs - Simpler code - `.key()` already propagates schemas, no need to track separately - Single source of truth for OpaqueRef proxy behavior - All tests still pass ✓ 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
…erly isQueryResult) instead.
…sures - Change Frame type to have optional runtime, tx, and space fields - Revert all *Impl functions (liftImpl, handlerImpl, etc.) to original signatures - Update builder functions to access runtime via getTopFrame() - Remove closure-based approach from factory.ts - Update pushFrame/pushFrameFromCause to accept and propagate runtime - Update runner.ts to pass runtime when creating execution frames - Update opaqueRef to use Cell-backed implementation when runtime available - Update built-in.ts to use frame-based approach - Update module.ts rawImpl to push frame with runtime This simplifies the architecture by using the existing frame system to pass runtime context instead of creating wrapper closures for every builder function. Some tests still failing - needs investigation.
- Add defaultFrame field to Runtime class - Push default frame with runtime in constructor - Pop default frame in dispose() method - Update pushFrame to accept and propagate tx parameter - Pass frame.tx when creating cells in opaqueRefWithCell - Update test expectations for new error messages - Tests now expect 'space is required' instead of 'no frame context' - Tests now expect 'no cause provided' instead of 'no frame context' This ensures builder functions always have access to runtime via getTopFrame(), making Cell-backed OpaqueRefs work throughout the system. The default frame doesn't have space/cause, so cells still properly error when used without proper context. Tests improved from 18 failures to 8 failures.
…implementation Finalize the merge of OpaqueRef and Cell by removing the legacy proxy-based OpaqueRef implementation and making opaqueRef() always use Cell-backed OpaqueRefs. Key changes: - Remove legacy proxy-based OpaqueRef implementation from opaqueRef() - Require runtime in frame for all OpaqueRef creation - Remove unsafe_bindToRecipeAndPath() and unsafe_getExternal() from OpaqueRef interface - Remove deprecated setPreExisting() method from Cell - Fix Cell.getAsOpaqueRefProxy() to properly bind methods when accessed - Update Cell.export() to include external link when cell has an ID - Update tests to reflect new unified behavior
…dling
Introduces a CellKind type system to distinguish cell variants at construction
time rather than relying on runtime value markers or dynamic type checks. This
is a major simplification in how cells track their semantic role (regular cell,
stream, opaque reference, etc.) and continues the OpaqueRef/Cell merge effort.
- Add CellKind union type: "cell" | "opaque" | "stream" | "comparable" |
"readonly" | "writeonly"
- Change BrandedCell<T, Brand extends string> to BrandedCell<T, Kind extends CellKind>
- Store kind as private _kind field in CellImpl, set at construction time
- Pass kind parameter through all cell creation paths: createCell(),
asSchema(), withTx(), key()
- Replace magic $stream marker value with explicit kind="stream" flag
- Create dedicated stream<T>(schema?) function that calls opaqueRefWithCell
with kind="stream"
- Update StreamFunction signature: was <T>(initial?: T), now <T>(schema?: JSONSchema)
- Remove stream.set({ $stream: true }) pattern from handler creation in module.ts
- Export stream marker value from Cell.export() when kind="stream" instead of
reading from storage
Previously, cells called getTopFrame() dynamically whenever they needed frame
context (space, cause, generatedIdCounter). This was fragile and could give
wrong results if the frame changed between cell creation and use.
Now cells capture frame at construction:
- Add private _frame: Frame | undefined field to CellImpl
- Set this._frame = getTopFrame() in constructor
- Replace all getTopFrame() calls with this._frame in:
- get link() - for deriving entity IDs
- get space() - for resolving storage space
- set() - for writing with correct cause
- push() - for array operations with cause
- export() - for returning frame context
- Cells are bound to the frame they were created in, preventing closure-related bugs
- Clearer error messages when cell created outside frame context
- Consistent behavior even if frame stack changes after creation
Previously, cells tracked both value and defaultValue separately, and recipe
serialization would:
1. Collect defaultValue from all cells
2. Generate schema from defaults via createJsonSchema()
3. Track defaults separately in recipe initial state
Now defaults are just schema.default:
- opaqueRef(value, schema) treats value as schema.default, not initial value
- Merge value into schema: { ...schema, default: value as JSONValue }
- Pass merged schema to createCell with link containing schema
- Remove defaultValue field from Cell.export() return type
- Remove defaults collection and createJsonSchema() logic from recipe.ts
- Simplify argument schema construction: if string, use as description;
otherwise use as-is or true
- Remove 50+ lines of complex schema generation from defaults
- Variable renaming for clarity: cells ‚Üí allCells, nodes ‚Üí allNodes
- Remove defaultValue tracking in factoryFromRecipe
- Simplify initial value collection to only process cell.export().value
Complete rewrite of how opaqueRef creates cells:
Before:
```typescript
const cell = createCell(runtime, undefined, tx, false);
if (schema) cell.setSchema(schema);
if (value) cell.set(value);
```
After:
```typescript
if (value !== undefined) {
schema = { ...toSchemaObj(schema), default: value };
}
const cell = createCell(
runtime,
{ path: [], schema, rootSchema: schema, space },
tx,
false,
kind
);
```
Changes:
- Create cell with link containing schema, not undefined link
- Treat value parameter as default, merge into schema
- Pass kind parameter to distinguish cell types
- Include frame.space in link if available
- Remove post-creation setSchema() and set() calls
- Simplify opaqueRef() to just call opaqueRefWithCell
In node-utils.ts and elsewhere:
- Change isOpaqueRef(value) checks to isCell(value)
- This is more accurate post-merge since all OpaqueRefs are now Cells
- Remove unnecessary type assertions: (item as OpaqueRef<T>) ‚Üí just use item
cellNodes WeakMap now keys by root cell from causeContainer:
- Change from WeakMap<CellImpl<any>, ...> to WeakMap<OpaqueCell<unknown>, ...>
- In connect(): use this._causeContainer.cell as key instead of this
- Ensures all siblings (created via asSchema/withTx) share same node set
- Add cell field to CauseContainer to track root
From IOpaquable interface:
- Remove setDefault(value: Opaque<T> | T): void
- Remove setPreExisting(ref: any): void
- Keep setSchema for now (still marked deprecated)
- Rename module augmentation: IOxpaquable ‚Üí IOpaquable (was typo)
Cell.export() changes:
- Remove link field (was NormalizedFullLink, only set if hasFullLink())
- Remove defaultValue field
- Change name to use causeContainer.cause as string
- Return root cell from causeContainer instead of this
- Only include external link if cell has ID
- Simplify value field: return { $stream: true } if kind="stream", else undefined
Update cell-optional-link.test.ts to create cells within frame context:
- Move pushFrame() before new CellImpl() calls
- Cells need frame at construction time now, not just at link creation time
- Fix "should share cause with children via .key()" test expectations
Add proper runtime/storage setup to recipe.test.ts:
- Create StorageManager and Runtime before tests
- Push frame with runtime in beforeEach
- Pop frame and dispose runtime in afterEach
- Ensures recipe construction has required context
- Remove unused imports from opaque-ref.ts (recipe, createNodeFactory, etc.)
- Remove console.log("key", key) from json-utils.ts
- Remove unused linkToOpaqueRef WeakMap from query-result-proxy.ts
- Remove title.setDefault("untitled") from html-recipes.test.ts
- StreamFunction signature changed (initial value ‚Üí schema parameter)
- Cells must be created within frame context (will throw otherwise)
- setDefault() and setPreExisting() removed from API
- Cell.export() no longer returns link or defaultValue fields
This refactoring significantly reduces complexity in recipe serialization,
improves type safety, and makes cell behavior more predictable by capturing
context at creation time rather than dynamically resolving it later.
… can trigger getters like `value`
0e51e69 to
a9279ef
Compare
- undo rawImpl change, no longer necessary
…isn't correctly handled right now by validateAndTransform
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Merge OpaqueRef and Cell Implementation
Summary
This PR completes a major architectural refactoring that merges
OpaqueRefandCellinto a unified implementation. Previously, these were separate concepts with duplicated functionality. Now,OpaqueRefis implemented as a proxy wrapper aroundCell, eliminating code duplication and simplifying the mental model for working with reactive values.Key Changes
Core Architecture
StreamCellintoRegularCell(renamed toCellImpl), creating a single cell type that handles both regular and stream behaviorOpaqueRefis now a proxy wrapper aroundCellinstances, rather than a separate implementationCell Construction & Linking
linkparameter, allowing cells to be created without an immediate link.for()Method: New method to set a cause/link for cells that don't have one yet, enabling lazy link creationCell Kinds System
CellKindEnum: Introduced standardized cell kinds:"cell","opaque","stream","comparable","readonly","writeonly"BrandedCell<T, Kind>to useCellKindfor better type safetyCleanup & Deprecation
toOpaqueRef(): No longer needed since all cells are now also OpaqueRefssetDefault(),setPreExisting(), and other legacy methods that are no longer neededAPI Changes
cell()Function: Simplified signature -cell<T>(defaultValue?, schema?)stream()Function: Updated tostream<T>(schema?)for consistency.for()called with appropriate contextMigration Impact
Breaking Changes
toOpaqueRef()removed - cells are already opaque refssetDefault()deprecated - use constructor parameter insteadsetPreExisting()removed - use.for()insteadNon-Breaking Changes
Testing
cell-optional-link.test.ts(445 lines)Benefits
Summary by cubic
Merged OpaqueRef and Cell into a single, Cell-backed implementation with runtime-aware construction. This simplifies the API, enables creating cells without links, and clarifies stream behavior.
Refactors
Migration
Written for commit 9785fc6. Summary will update automatically on new commits.