Skip to content

Commit 756f1d1

Browse files
committed
update schemas for node factories, serialized state, etc.
1 parent 6ba3082 commit 756f1d1

File tree

3 files changed

+92
-81
lines changed

3 files changed

+92
-81
lines changed

docs/specs/recipe-construction/graph-snapshot.md

Lines changed: 57 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Graph Snapshot Schema
1+
# Process Snapshot Schema
22

33
> **Status:** This work is **deferred to Phase 2** (see `rollout-plan.md` and
44
> `overview.md`). The snapshot format described here will be implemented as part
@@ -8,9 +8,8 @@
88

99
- `Runner.setupInternal` currently stores recipe metadata inside the process
1010
cell (`TYPE`, `argument`, `internal`, `resultRef`). We plan to deprecate
11-
`TYPE` while still persisting the originating program reference (rename the
12-
`spell` field to `pattern`). The runtime still lacks a persisted view of the
13-
concrete nodes it instantiated.
11+
`TYPE` while still persisting the originating program reference. The runtime
12+
still lacks a persisted view of the concrete nodes it instantiated.
1413
- `instantiateNode` performs alias gymnastics through
1514
`unwrapOneLevelAndBindtoDoc` so nested recipes and closures work. A snapshot
1615
that records the resolved nodes makes this machinery obsolete: future runs can
@@ -39,52 +38,75 @@ Store an additional `process` payload on the result cell alongside the existing
3938
allow future format changes.
4039

4140
```ts
42-
interface GraphSnapshotV1 {
41+
interface ProcessSnapshotV1 {
4342
version: 1;
4443
program?: RuntimeProgram;
45-
generation?: number; // Incremented each time the runtime rebuilds the graph
46-
nodes: Array<ReactiveNodeSnapshot | EventHandlerSnapshot>;
44+
generation?: number; // Incremented each time it changes
45+
cells: Array<GeneratedCell>;
46+
predecessor?: ProcessSnapshotV1;
4747
}
4848

49-
interface ReactiveNodeSnapshot {
50-
module: ReactiveModuleDescriptor;
51-
inputs: Record<string, Binding>;
52-
output?: CellLink; // Optional; some modules only read
53-
argumentSchema?: JSONSchema;
54-
resultSchema?: JSONSchema;
49+
interface ReactiveNodeNarrowedSchema = {
50+
argumentSchema?: JSONSchema; // optional, further constraint on what
51+
resultSchema?: JSONSchema; // the module already says
5552
}
5653

57-
interface EventHandlerSnapshot {
58-
module: EventHandlerDescriptor;
59-
inputs: Record<string, Binding>;
60-
stream: CellLink; // Event stream destination
61-
argumentSchema?: JSONSchema;
54+
interface ReactiveNodeArgument = {
55+
arguments: JSONArray;
56+
} & ReactiveNodeNarrowedSchema;
57+
58+
interface HandlerNodeNarrowedSchema = {
59+
eventSchema?: JSONSchema; // see above
60+
stateSchema?: JSONSchema;
61+
resultSchema?: JSONSchema;
6262
}
6363

64-
type Binding = CellLink | JSONValue | Record<string, Binding> | Array<Binding>;
64+
interface HandlerNodeArgument = {
65+
stream: SigilLink; // Event stream
66+
state?: JSONArray; // Optional state
67+
} & HandlerNodeNarrowedSchema
6568

66-
type ReactiveModuleDescriptor =
69+
interface CurriedArguments = {
70+
curried?: { index?: number, value: JSONValue }[];
71+
}
72+
73+
interface BoxedNodeFactory = { module: NodeFactoryDescriptor }
74+
75+
interface BoxedLink = { link: NormalizedLink } // relative to this cell
76+
77+
type GeneratedCell =
78+
| BoxedLink
79+
| (BoxedLink & BoxedNodeFactory &
80+
(ReactiveNodeArgument | HandlerNodeArgument) & CurriedArgument);
81+
82+
type NodeFactoryDescriptor =
83+
| ((
84+
| {
85+
type: "javascript";
86+
implementation: string;
87+
}
88+
| {
89+
type: "ref";
90+
ref: string;
91+
}) & (
92+
| {
93+
argumentSchema: JSONSchema;
94+
resultSchema: JSONSchema;
95+
}
96+
| {
97+
eventSchema: JSONSchema;
98+
stateSchema: JSONSchema;
99+
resultSchema?: JSONSchema;
100+
}))
67101
| {
68-
type: "ref";
69-
ref: string;
70-
argumentSchema: JSONSchema;
71-
resultSchema?: JSONSchema;
102+
type: "program";
103+
program: RuntimeProgram; // Resolves to a node factory with schemas, etc.
72104
}
73-
| {
74-
type: "javascript";
75-
implementation: string | { file: string; export: string };
76-
argumentSchema: JSONSchema;
77-
resultSchema?: JSONSchema;
78-
metadata?: Record<string, JSONValue>;
79-
};
80-
81-
type EventHandlerDescriptor = ReactiveModuleDescriptor & { handler: true };
82105
```
83106
84107
- Notes:
85108
86-
- Only nodes are serialized; cells and links are implied by the cell links
87-
used in `inputs`, `output`, or `stream`.
109+
- .
88110
- `CellLink` reuses the normalized link format the runtime already
89111
understands.
90112
- `Binding` supports nested structures so aliases into deeply nested inputs
@@ -148,5 +170,3 @@ type EventHandlerDescriptor = ReactiveModuleDescriptor & { handler: true };
148170
run timestamps) or should that remain derived at runtime?
149171
- How do we store large schemas efficiently? Option: store a content hash in the
150172
snapshot and persist the full schema in a shared manifest keyed by hash.
151-
- What is the cleanest way to encode the `pattern` reference alongside the
152-
snapshot so older runtimes can ignore it while newer ones rely on it?

docs/specs/recipe-construction/node-factory-shipping.md

Lines changed: 33 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,8 @@
22

33
## Goals
44

5-
- Allow lifts, handlers, and recipes to return reusable node factories that can
6-
be shipped across spaces or persisted and later instantiated.
5+
- Allow lifts (and thus patterns) and handlers to return reusable node factories
6+
that can be shipped across spaces or persisted and later instantiated.
77
- Support partial application via a `.curry(value, argIndex = 0)` helper that
88
yields another shippable node factory reflecting the bound argument.
99
- Use a versioned sigil format (`nodeFactory@1`) that closely mirrors the graph
@@ -15,6 +15,9 @@
1515
- Any node factory created by `lift`, `handler`, or builder helpers returns a
1616
callable object that also exposes `.toJSON()` producing the `nodeFactory@1`
1717
sigil.
18+
- Passing a node factory as part of the argument to `Cell.set()` & co, or
19+
calling another factory with it, automatically converts it into it's JSON
20+
representation.
1821
- Recipes may return node factories or pass them into other lifts/recipes. When
1922
the runtime encounters a `nodeFactory@1` payload (e.g., via `Cell.get()` or as
2023
a recipe input), it materializes a callable node factory automatically.
@@ -26,57 +29,46 @@
2629

2730
Serialized form:
2831

29-
```json
32+
```ts
3033
{
3134
"/": {
32-
"nodeFactory@1": {
33-
"module": ReactiveModuleDescriptor | EventHandlerDescriptor,
34-
"program": RuntimeProgram,
35-
"curried": [
36-
{
37-
"index": 0,
38-
"value": Binding
39-
}
40-
],
41-
"argumentSchema": JSONSchema,
42-
"resultSchema": JSONSchema | null,
43-
"metadata": Record<string, JSONValue> | null
44-
}
35+
"nodeFactory@1":
36+
& BoxedNodeFactory
37+
& CurriedArguments
38+
& (ReactiveNodeNarrowedSchema | HandlerNodeNarrowedSchema)
4539
}
4640
}
4741
```
4842

4943
Notes:
5044

51-
- `module` matches the descriptors used in `graph-snapshot.md`, including
52-
implementation references and descriptor-level `argumentSchema`/`resultSchema`.
53-
- `program` stores the complete `RuntimeProgram` (as defined in
54-
`packages/runner/src/harness/types.ts`) so the runtime knows how to load the
55-
compiled artifact and entry symbol.
56-
- `curried` is ordered; each entry binds `value` to the given `index` (0-based by
57-
default). `Binding` mirrors the nested binding structure from the snapshot so
58-
bound values can include cell links.
59-
- `argumentSchema` on the sigil represents the effective schema after all
60-
currying. It must continue to describe array/prefix-array inputs.
61-
- `resultSchema` is optional and may be omitted/`null` for handlers.
62-
- `metadata` carries optional helper information (e.g., helper names) and may be
63-
omitted if unused.
45+
- `BoxedNodeFactory` the descriptors used in `graph-snapshot.md`, including
46+
implementation references and descriptor-level `argumentSchema`/`resultSchema`
47+
and link to implemention.
48+
- `CurriedArguments` contains the lists the arguments that are being supplied
49+
already, index defaulting to 0 or +1 the previous index.
50+
- For event handlers, 0 is the event stream. If it is curried, then the
51+
resulting function looks like a reactive node factory that can be called
52+
with state to bind. If it isn't curried, the result is a stream factory,
53+
just like the original, but bound to state.
54+
- `ReactiveNodeNarrowedSchema | HandlerNodeNarrowedSchema` are optional
55+
constraints due to currying over the original schemas in the module.
6456

6557
## Runtime Behavior
6658

6759
- Deserialization: When the runtime reads a value containing the `nodeFactory@1`
6860
sigil, it constructs a callable factory that:
6961
1. Applies stored currying bindings before invoking the underlying module.
70-
2. Exposes `.curry` to append additional entries to `curried` and returns a new
71-
serialized-aware wrapper.
62+
2. Exposes `.curry` to append additional entries to `curried` and returns a
63+
new serialized-aware wrapper.
7264
3. Implements `.toJSON()` to emit the sigil.
7365
- Invocation: Calling the materialized factory schedules a node instantiation
7466
identical to calling the original lift/handler. Inputs are merged with stored
7567
curry bindings using positional logic (array inputs) or schema-derived names.
76-
- Integration with snapshots: When a recipe instantiates a node from a serialized
77-
factory, the resulting node appears in the graph snapshot exactly like a
78-
regular node (same descriptor, inputs, etc.). The factory sigil itself can also
79-
be stored in recipe state for later reuse.
68+
- Integration with snapshots: When a recipe instantiates a node from a
69+
serialized factory, the resulting node appears in the graph snapshot exactly
70+
like a regular node (same descriptor, inputs, etc.). The factory sigil itself
71+
can also be stored in recipe state for later reuse.
8072

8173
## Builder Integration
8274

@@ -87,11 +79,10 @@ Notes:
8779
runtime keeps the sigil available so passing the factory back to storage or
8880
another recipe preserves currying metadata.
8981

90-
## Open Questions
82+
## Rollout
9183

92-
- Do we need an explicit way to specify argument names alongside indexes for
93-
currying when dealing with object-shaped schemas?
94-
- Should we limit the size of serialized `RuntimeProgram` payloads by hashing or
95-
referencing a shared cache entry instead of embedding the entire program?
96-
- How do we ensure backwards compatibility if the module descriptor expands? The
97-
sigil version (`nodeFactory@1`) gives us room to add future revisions.
84+
- This is blocked on multi-argument `lift`.
85+
- However, we can start with curried `recipe` (which later become a variant of
86+
`lift`) and instead of treating it as multiple inputs merge the input object
87+
instead on call. The first use-case is for actions, so we can even do this
88+
just there.

docs/specs/recipe-construction/rollout-plan.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -40,10 +40,11 @@
4040
context write those down, with schemas used.
4141
- [ ] Set `source` in context, so that created cells copy it and write it as
4242
metadata.
43-
- [ ] Add a helper for causes to complain about duplicates
4443
- [ ] Keep track of all created cells, make sure to track those as well for
4544
the recipe creation in lift/handler, so we don't need to return them in e.g.
4645
a handler.
46+
- [ ] Add a helper for cause generation that checks that no other created cell
47+
already has that cause.
4748
- [ ] In lift/handler, change how recipes are invoked to directly go off the
4849
created graph.
4950
- [ ] For each created cell (as that's always the case when introducing a
@@ -76,7 +77,6 @@
7677
- [ ] Implement `nodeFactory@1` sigil format for shipping factories
7778
- [ ] Add `.curry(value, argIndex)` method for partial application
7879
- [ ] Support rehydration of serialized factories from cells/events
79-
- [ ] Enable factories to be passed across spaces
8080
- [ ] Ensure currying metadata is preserved during serialization
8181

8282
## Open Questions

0 commit comments

Comments
 (0)