spec(runner): updated plan for DX1 & adjusted specs

Author: seefeldb

docs/specs/recipe-construction/capability-wrappers.md

@@ -1,13 +1,18 @@
-# Capability Wrappers Prototype
+# Capability Types via Branded Types
+
+> **Status:** This design has been **superseded by the branded types approach**
+> described in `rollout-plan.md`. We are using TypeScript branded types with
+> symbol-based brands rather than runtime proxy wrappers.
 
 ## Objectives
 
-- Collapse `OpaqueRef` and runtime `Cell` APIs into a single object whose
+- Collapse `OpaqueRef` and runtime `Cell` APIs into a single type system whose
   surface area is gated by explicit capabilities (`Opaque`, `Readonly`,
   `Mutable`, `Writeonly`).
+- Use TypeScript's type system (branded types) rather than runtime Proxy objects
+  to enforce capability boundaries.
 - Keep recipe ergonomics intact by continuing to expose property proxies and
-  helpers like `.map()` while steering operations through capability-aware
-  wrappers.
+  helpers like `.map()`.
 - Provide a migration path that lets existing recipes run during the migration
   window by adapting the current builder implementation incrementally.
 
@@ -27,84 +32,99 @@
   nested recipes. That indirection means wrappers must cooperate with module
   factories that expect builder-time proxies yet execute against runtime cells.
 
-## Wrapper Shape
-
-- Introduce a lightweight `CapabilityCell<T, C extends Capability>` interface
-  that wraps a concrete runtime cell and exposes operations allowed by
-  capability `C`. Capabilities map to subsets of the current `Cell` API:
-  - `Opaque`: property proxies, `.map`, `.filter`, `.derive`, structural
-    equality, read-only helpers.
-  - `Readonly`: everything in `Opaque` plus `.get`, `.getAsQueryResult`, and
-    schema navigation helpers that do not mutate state.
-  - `Mutable`: superset of `Readonly` plus `.set`, `.update`, `.push`, and
-    `.redirectTo`.
-  - `Writeonly`: `.send`, `.set`, `.redirectTo`, but intentionally hides `.get`
-    to encourage event-sink authoring disciplines.
-- Use `Proxy` objects to intercept property access and method calls. Each proxy
-  lazily constructs child proxies with the same capability so nested lookups
-  remain ergonomic (e.g. `mutable.todos[0].title.set("…")`).
-- When a helper like `.map` is invoked, delegate to a capability-aware shim that
-  wraps the callee recipe factory so the inner function receives wrappers
-  instead of raw cells or `OpaqueRef`s. Internally reuse the existing
-  `createNodeFactory` path to avoid reimplementing module registration.
-
-## Construction Flow
-
-1. Builder entry points (`recipe`, `lift`, `handler`) push a frame, wrap inputs
-   in `CapabilityCell` proxies instead of `opaqueRef`, and still record the
-   graph using existing traversal utilities.
-2. For compatibility, supply adapters that allow legacy `OpaqueRef` helper
-   affordances (`setDefault`, `unsafe_bindToRecipeAndPath`) to continue working
-   until all call sites migrate. These adapters simply forward to the wrapped
-   runtime cell or translate to snapshot metadata updates.
-3. During runtime execution, `pushFrameFromCause` already seeds the frame with
-   the `cause` and `unsafe_binding`. Wrappers created while executing a lifted
-   function can therefore call `runtime.getCell` immediately because the cause
-   data is ready.
+## Type System Shape (Branded Types Approach)
+
+The actual implementation uses **branded types** rather than runtime Proxy wrappers:
+
+- Create a `CellLike<T>` type with a symbol-based brand where the value is
+  `Record<string, boolean>` representing capability flags.
+- Factor out interface parts along: reading, writing, `.send` (for stream-like),
+  and derives (currently just `.map`).
+- Define capability types by combining these factored parts with specific brand
+  configurations:
+  - `OpaqueRef<T>`: `{ opaque: true, read: false, write: false, stream: false }`
+    - Supports: property proxies, `.map`, `.filter`, `.derive`, structural equality
+  - `Cell<T>` (Mutable): `{ opaque: false, read: true, write: true, stream: true }`
+    - Supports: everything (`.get`, `.set`, `.update`, `.push`, `.redirectTo`, `.send`)
+  - `Stream<T>`: `{ opaque: false, read: false, write: false, stream: true }`
+    - Supports: `.send` only
+  - `ReadonlyCell<T>`: `{ opaque: false, read: true, write: false, stream: false }`
+    - Supports: `.get`, `.getAsQueryResult`, schema navigation
+  - `WriteonlyCell<T>`: `{ opaque: false, read: false, write: true, stream: false }`
+    - Supports: `.set`, `.update`, `.redirectTo` but hides `.get`
+- For `OpaqueRef`, keep proxy behavior where each key access returns another
+  `OpaqueRef`.
+- Simplify most wrap/unwrap types to use `CellLike`.
+
+### Comparison to Original Proxy Design
+
+The branded types approach provides compile-time safety without runtime overhead.
+The original proxy-based `CapabilityCell<T, C>` design is **not being implemented**
+because TypeScript's type system can enforce the same boundaries more efficiently.
+
+## Construction Flow (Branded Types)
+
+1. Builder entry points (`recipe`, `lift`, `handler`) push a frame and work with
+   the unified `CellLike` types instead of separate `opaqueRef` and `Cell` types.
+2. Cell creation is deferred - cells can be created without an immediate link,
+   using `.for(cause)` to establish the link later.
+3. For compatibility during migration, legacy `OpaqueRef` helper affordances
+   (`setDefault`, `unsafe_bindToRecipeAndPath`) continue working until all call
+   sites migrate.
+4. During runtime execution, `pushFrameFromCause` seeds the frame with the
+   `cause` and `unsafe_binding`. Created cells can call `runtime.getCell` when
+   their cause is ready (either automatically derived or explicitly set via
+   `.for()`).
 
 ## Type System Notes
 
-- Export capability-specific TypeScript types from `@commontools/api` such as
-  `OpaqueCell<T>`, `ReadonlyCell<T>`, `MutableCell<T>`, and `WriteonlyCell<T>`.
-  These extend a shared base that keeps the proxy type information available to
-  recipe authors.
+- Export capability-specific TypeScript types from `@commontools/api`:
+  `OpaqueRef<T>`, `Cell<T>`, `ReadonlyCell<T>`, `WriteonlyCell<T>`, and
+  `Stream<T>`.
+- All types extend a shared `CellLike<T>` base with branded capability flags.
 - Extend our JSON Schema annotations so authors can declare capabilities at any
   depth. When `asCell: true` is present, allow an `opaque`, `readonly`, or
   `writeonly` flag (or the closest JSON Schema standard equivalent if one
-  exists). Builder proxies read these flags to choose the capability for the
-  proxy returned by `key()` or nested property access.
+  exists).
 - Provide conditional helper types to map schema metadata to helper surfaces
   (e.g., array helpers only appear when `T` extends `readonly any[]`). Reuse the
   IFC-aware schema lookup utilities to keep helper availability aligned with the
   JSON schema.
-- Augment builders so `recipe` factories default to `OpaqueCell` inputs while
+- Augment builders so `recipe` factories default to `OpaqueRef` inputs while
   `lift` can declare stronger capabilities for each argument via a typed options
   bag (e.g., `{ inputs: { item: Capability.Mutable } }`).
 
-## Cause Defaults
-
-- Capability helpers (e.g., `.map`, `.filter`) should emit deterministic
-  default causes that combine the parent cell's cause with a helper-specific
-  label. Authors can still pass explicit `cause` overrides to `lift` or
-  `handler`, but most call sites inherit stable defaults automatically.
-- Expose an optional `.setCause(cause: CauseDescriptor)` chain on newly created
-  capability cells. It overrides the derived id before the cell participates in
-  the graph. If the cell has already been connected to a node or materialized
-  into a runtime cell, `.setCause` must throw to avoid inconsistencies.
-
-## Migration Strategy
-
-- Step 1: Introduce wrappers with shims that forward to existing opaque ref
-  behavior. Dual-write metadata (`export()`) so `factoryFromRecipe` can continue
-  serializing the graph until the snapshot pipeline lands.
-- Step 2: Convert builtin helpers and modules to consume wrappers. Audit
-  `packages/runner/src/builder` to replace direct `OpaqueRef` imports with
-  wrapper types while keeping `opaqueRef` available as a thin adapter.
-- Step 3: Update recipes in `recipes/` iteratively. Provide codemods that swap
-  `cell()` usage for capability-specific constructors when explicit mutability is
-  required.
-- Step 4: Remove legacy-only APIs (`setDefault`, `setPreExisting`) once the
-  snapshot work replaces path-based aliasing and defaults come from schemas.
+## Cause Defaults and `.for()` Method
+
+- Cause assignment happens in two layers:
+  1. **Automatic derivation**: Default causes are derived from frame context,
+     input cell ids, and implementation fingerprints (see `cause-derivation.md`)
+  2. **Explicit override via `.for()`**: Authors can call `.for(cause)` to
+     explicitly assign a cause before the cell is linked
+- The `.for()` method provides an explicit layer on top of automation:
+  - Optional second parameter makes it flexible (ignores if link already exists,
+    adds extension if cause already exists)
+  - Throws if cell already connected to a node or materialized into runtime cell
+- Helpers (e.g., `.map`, `.filter`) use automatic derivation by default but can
+  be overridden with explicit `cause` parameters to `lift` or `handler`.
+
+## Migration Strategy (In-Place)
+
+This is an **in-place migration** rather than a V1/V2 opt-in system:
+
+- **Step 1**: Unify Cell API types using branded types (see `rollout-plan.md`)
+  - Create `CellLike<>` and factor out capability traits
+  - Remove `ShadowRef`/`unsafe_` mechanisms
+- **Step 2**: Enable deferred cell creation with `.for()` method
+  - Change `RegularCell` constructor to make link optional
+  - Add `.for()` method for explicit cause assignment
+  - Implement automatic cause derivation as baseline
+- **Step 3**: Update recipe lifecycle to use deferred execution
+  - Run recipes like lifts with tracked cell/cause creation
+  - Remove JSON recipe representation
+- **Step 4**: Cleanup legacy APIs
+  - Remove `setDefault`, `setPreExisting` once defaults come from schemas
+  - Deprecate path-based aliasing in favor of graph snapshots (Phase 2)
 
 ## Open Questions
 

docs/specs/recipe-construction/cause-derivation.md

@@ -10,6 +10,18 @@ derive deterministic causes for the **new cells** produced inside that frame by
 combining the frame cause with deterministic fingerprints of the inputs and
 implementation.
 
+## Two-Layer Approach
+
+Cause assignment works in two complementary layers:
+
+1. **Automatic derivation** (this spec): Default causes are automatically derived
+   from frame context, input cell ids, and implementation fingerprints
+2. **Explicit override via `.for()`**: Authors can call `.for(cause)` on cells to
+   explicitly assign a cause, providing an explicit layer on top of the automation
+
+The `.for()` method acts as an escape hatch when automatic derivation isn't
+sufficient or when authors want stable, human-readable cause names.
+
 ## Frame Cause Setup
 
 1. When a lift or handler is invoked at runtime, the runtime computes a
@@ -55,14 +67,22 @@ const cause = createRef({
 
 The resulting entity id seeds the cause for the cell returned by the helper.
 
-### Explicit Overrides
-
-- Newly created capability cells expose `.setCause(cause: CauseDescriptor)` to
-  replace the derived cause with an explicit value (useful for hand-authored
-  stability keys). The override must occur **before** the cell participates in
-  the graph. If the cell has been connected to a node, read, or written, the
-  call must throw so we do not retroactively change ids that other nodes may
-  already reference.
+### Explicit Overrides via `.for()`
+
+- Newly created cells expose `.for(cause, flexible?)` to replace or refine the
+  derived cause with an explicit value:
+  - **Basic usage**: `.for(cause)` assigns the specified cause
+  - **Flexible mode**: `.for(cause, true)` provides flexibility:
+    - Ignores the `.for()` if link already exists
+    - Adds extension if cause already exists (see tracker in `rollout-plan.md`
+      lines 39-46)
+  - The override must occur **before** the cell is materialized into a runtime
+    cell or connected to a node
+  - If the cell has already been connected, the call must throw to avoid
+    inconsistencies
+- This provides an **explicit layer on top of automatic derivation** - authors
+  can use automatic derivation for most cells and only call `.for()` when they
+  need specific control.
 
 ## Propagating Causes to Nested Frames
 

docs/specs/recipe-construction/graph-snapshot.md

@@ -1,5 +1,9 @@
 # Graph Snapshot Schema
 
+> **Status:** This work is **deferred to Phase 2** (see `rollout-plan.md` and
+> `overview.md`). The snapshot format described here will be implemented as part
+> of the `process` metadata work mentioned in the rollout plan (lines 54-61).
+
 ## Motivation
 
 - `Runner.setupInternal` currently stores recipe metadata inside the process
@@ -15,6 +19,19 @@
   this graph look like" without re-running the factory, simplifying rehydration
   and teardown.
 
+## Relationship to Rollout Plan
+
+This spec describes the implementation for tasks in `rollout-plan.md` lines
+54-61:
+
+- Write metadata into result cell including `source` (from context) and
+  `process` metadata
+- The `process` field contains the graph snapshot described in this document,
+  including:
+  - The `Module` (which includes the `Program` as link using `cid:hash`)
+  - Created cells with links to module and/or schema
+  - History of previously used schemas (P2 for safe updating)
+
 ## Snapshot Envelope
 
 Store an additional `graph` payload on the result cell alongside the existing
@@ -82,40 +99,48 @@ type EventHandlerDescriptor = ReactiveModuleDescriptor & { handler: true };
     schemas so reactive nodes can express tuple-style inputs used by lifts and
     handlers.
 
-## Generation Flow
+## Generation Flow (Phase 2)
 
-1. When `Runner.startWithTx` iterates `recipe.nodes`, instrument each
-   `instantiateNode` call to record:
-   - The resolved module descriptor, including implementation reference.
-   - Normalized input links (aliases already resolved by the capability
-     wrappers so `unwrapOneLevelAndBindtoDoc` is no longer needed).
+> **Note:** This describes future implementation work after Phase 1 type
+> unification is complete.
+
+1. When `Runner.startWithTx` iterates nodes during lift/handler execution,
+   instrument each instantiation to record:
+   - The resolved module descriptor, including implementation reference
+   - Normalized input links (already resolved since cells have concrete causes
+     from Phase 1 work)
    - Either an output link (for reactive nodes) or a stream link (for event
-     handlers), whichever applies.
-   - Optional argument/result schemas attached by the builder.
+     handlers), whichever applies
+   - Optional argument/result schemas attached by the builder
 2. Compute the final snapshot object, attach it to
-   `resultCell.withTx(tx).setMetadata("graph", snapshot)`, and persist it in the
-   same transaction that finishes setup.
+   `resultCell.withTx(tx).setMetadata("process", snapshot)` (note: using
+   `process` not `graph` to match rollout plan), and persist it in the same
+   transaction that finishes setup
+
+## Rehydration Strategy (Phase 2)
 
-## Rehydration Strategy
+> **Note:** Deferred to Phase 2 implementation.
 
 - On `Runner.setupInternal`, if a prior snapshot exists, load it before
   unpacking defaults. The runtime can:
   - Reattach scheduler subscriptions by walking the nodes, reusing modules whose
-    implementation reference matches.
-  - Rehydrate handler streams directly from their stored links.
+    implementation reference matches
+  - Rehydrate handler streams directly from their stored links
 - When a handler or lift causes a graph to rebuild, diff the previous and new
   node lists. Nodes that disappear are torn down by following their stored
   output/stream links. Nodes with matching descriptors can reuse their existing
-  cells.
+  cells
+
+## Teardown and Diffing (Phase 2)
 
-## Teardown and Diffing
+> **Note:** Deferred to Phase 2 implementation.
 
 - Nodes uniquely define the dependencies; links are inferred from the cell links
   embedded inside each snapshot entry. Diffing node descriptors is sufficient to
-  drive teardown.
+  drive teardown
 - Maintain a monotonic `generation` counter on the snapshot. When a handler
   triggers rehydration, increment the counter so logs can correlate actions with
-  rebuilds.
+  rebuilds
 
 ## Outstanding Questions