Skip to content

Conversation

@mathpirate
Copy link
Contributor

@mathpirate mathpirate commented Oct 3, 2025

Not yet ready for merge

This branch implements closure support for map on OpaqueRef and Cell

Currently includes a first draft of a closure transformer with associated fixtures; next, will include a new method map_with_pattern on OpaqueRef and Cell which will support the new signature


Summary by cubic

Adds a ClosureTransformer that rewrites OpaqueRef/Cell array map callbacks with outer-scope captures into recipe callbacks with a params object. This enables safe, explicit data flow in maps, supports nested callbacks and method chains, and adds thorough docs and fixtures.

  • New Features

    • Transforms OpaqueRef<T[]> and Cell<T[]> .map(fn) when fn captures outer variables to: map(recipe(({ elem, index?, params:{...} }) => ...), { ... }).
    • Selective capture: skips module-scoped declarations, imports, functions/handlers, and globals; captures from parent callback scope; treats property access as a unit (e.g., state.user.name).
    • Handles nested callbacks, array method chains before map (filter/slice/concat/reverse/sort/flat/flatMap), destructured params, index param, JSX, template literals, conditionals, and type assertions. Plain arrays are not transformed.
    • Automatically injects the recipe import. Adds closure-design and implementation-roadmap docs plus comprehensive fixtures.
  • Refactors

    • Added ClosureTransformer to the pipeline (runs first).
    • Dataflow analyzer: handles synthetic nodes and preserves array-map rewrite hints.
    • OpaqueRef call-expression emitter: skips rewriting the map call itself and rewrites the callee chain when needed.
    • Test setup: new closures fixture suite; updated a few expectations to reflect the new transformation.

Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

11 issues found across 46 files

Prompt for AI agents (all 11 issues)

Understand the root cause of the following 11 issues and fix them.


<file name="packages/ts-transformers/test/fixtures/closures/map-import-reference.input.tsx">

<violation number="1" location="packages/ts-transformers/test/fixtures/closures/map-import-reference.input.tsx:24">
Add a stable key to the &lt;div&gt; returned from the map so React can reconcile list items correctly.</violation>
</file>

<file name="packages/ts-transformers/test/fixtures/closures/map-index-param-used.expected.tsx">

<violation number="1" location="packages/ts-transformers/test/fixtures/closures/map-index-param-used.expected.tsx:44">
Please add a stable key to the mapped &lt;div&gt; so React can reconcile the list correctly when items change.</violation>
</file>

<file name="packages/ts-transformers/test/fixtures/closures/map-nested-callback.input.tsx">

<violation number="1" location="packages/ts-transformers/test/fixtures/closures/map-nested-callback.input.tsx:23">
Add a stable key to the element returned inside `state.items.map` so React can correctly reconcile the list.</violation>

<violation number="2" location="packages/ts-transformers/test/fixtures/closures/map-nested-callback.input.tsx:30">
Provide a unique key on the `&lt;li&gt;` rendered inside the `item.tags.map` callback to avoid reconciliation issues.</violation>
</file>

<file name="packages/ts-transformers/test/fixtures/closures/map-template-literal.input.tsx">

<violation number="1" location="packages/ts-transformers/test/fixtures/closures/map-template-literal.input.tsx:21">
Elements returned from state.items.map need a stable key so React can reconcile list updates correctly.</violation>
</file>

<file name="packages/ts-transformers/test/fixtures/closures/map-no-captures.expected.tsx">

<violation number="1" location="packages/ts-transformers/test/fixtures/closures/map-no-captures.expected.tsx:40">
Each element returned from the map call should have a stable `key` prop to avoid React reconciliation issues and warnings.</violation>
</file>

<file name="packages/ts-transformers/test/fixtures/closures/map-nested-property.input.tsx">

<violation number="1" location="packages/ts-transformers/test/fixtures/closures/map-nested-property.input.tsx:22">
Add a stable key to elements rendered inside `state.items.map` so React can reconcile list items correctly.</violation>
</file>

<file name="packages/ts-transformers/test/fixtures/closures/map-import-reference.expected.tsx">

<violation number="1" location="packages/ts-transformers/test/fixtures/closures/map-import-reference.expected.tsx:46">
Elements rendered from the map call need a stable key for correct reconciliation; please add a unique key prop (for example, `item.id`) to the `&lt;div&gt;` returned here.</violation>
</file>

<file name="packages/ts-transformers/test/fixtures/closures/map-no-captures.input.tsx">

<violation number="1" location="packages/ts-transformers/test/fixtures/closures/map-no-captures.input.tsx:19">
Each element produced by state.items.map should include a unique key prop so React can reconcile list updates; without it, updates may misapply state and trigger warnings.</violation>
</file>

<file name="packages/ts-transformers/docs/closure-implementation-roadmap.md">

<violation number="1" location="packages/ts-transformers/docs/closure-implementation-roadmap.md:147">
The &quot;Edge Cases&quot; list claims Cell arrays are transformed even though earlier sections explain they are rejected; please align this bullet with the documented behavior so the roadmap is self-consistent.</violation>
</file>

<file name="packages/ts-transformers/src/closures/transformer.ts">

<violation number="1" location="packages/ts-transformers/src/closures/transformer.ts:543">
Rewriting every callback as an arrow function changes the semantics whenever the original was a function expression (e.g., `this`/`arguments` stop working). Please preserve the original function kind when rebuilding the node.</violation>
</file>


Since this is your first cubic review, here's how it works:

  • cubic automatically reviews your code and comments on bugs and improvements
  • Teach cubic by replying to its comments. cubic learns from your replies and gets better over time
  • Ask questions if you need clarification on any suggestion

React with 👍 or 👎 to teach cubic. Mention @cubic-dev-ai to give feedback, ask questions, or re-run the review.

export default recipe<State>("ModuleScopedReference", (state) => {
return {
[UI]: (
<div>
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Oct 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a stable key to the

returned from the map so React can reconcile list items correctly.

Prompt for AI agents
Address the following comment on packages/ts-transformers/test/fixtures/closures/map-import-reference.input.tsx at line 24:

<comment>Add a stable key to the &lt;div&gt; returned from the map so React can reconcile list items correctly.</comment>

<file context>
@@ -0,0 +1,34 @@
+export default recipe&lt;State&gt;(&quot;ModuleScopedReference&quot;, (state) =&gt; {
+  return {
+    [UI]: (
+      &lt;div&gt;
+        {/* Should NOT capture module-level constant or function */}
+        {state.items.map((item) =&gt; (
</file context>
Suggested change
<div>
<div key={item.id}>
Fix with Cubic

return {
[UI]: (<div>
{/* Uses both index parameter and captures state.offset */}
{state.items.map(recipe(({ elem, index, params: { offset } }) => (<div>
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Oct 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a stable key to the mapped

so React can reconcile the list correctly when items change.

Prompt for AI agents
Address the following comment on packages/ts-transformers/test/fixtures/closures/map-index-param-used.expected.tsx at line 44:

<comment>Please add a stable key to the mapped &lt;div&gt; so React can reconcile the list correctly when items change.</comment>

<file context>
@@ -0,0 +1,49 @@
+    return {
+        [UI]: (&lt;div&gt;
+        {/* Uses both index parameter and captures state.offset */}
+        {state.items.map(recipe(({ elem, index, params: { offset } }) =&gt; (&lt;div&gt;
+            Item #{index + offset}: {elem.name}
+          &lt;/div&gt;)), { offset: state.offset })}
</file context>
Suggested change
{state.items.map(recipe(({ elem, index, params: { offset } }) => (<div>
{state.items.map(recipe(({ elem, index, params: { offset } }) => (<div key={elem.id}>
Fix with Cubic

export default recipe<State>("NestedCallback", (state) => {
return {
[UI]: (
<div>
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Oct 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a stable key to the element returned inside state.items.map so React can correctly reconcile the list.

Prompt for AI agents
Address the following comment on packages/ts-transformers/test/fixtures/closures/map-nested-callback.input.tsx at line 23:

<comment>Add a stable key to the element returned inside `state.items.map` so React can correctly reconcile the list.</comment>

<file context>
@@ -0,0 +1,38 @@
+export default recipe&lt;State&gt;(&quot;NestedCallback&quot;, (state) =&gt; {
+  return {
+    [UI]: (
+      &lt;div&gt;
+        {/* Outer map captures state.prefix, inner map closes over item from outer callback */}
+        {state.items.map((item) =&gt; (
</file context>
Suggested change
<div>
<div key={item.id}>
Fix with Cubic

{state.prefix}: {item.name}
<ul>
{item.tags.map((tag) => (
<li>{item.name} - {tag.name}</li>
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Oct 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Provide a unique key on the <li> rendered inside the item.tags.map callback to avoid reconciliation issues.

Prompt for AI agents
Address the following comment on packages/ts-transformers/test/fixtures/closures/map-nested-callback.input.tsx at line 30:

<comment>Provide a unique key on the `&lt;li&gt;` rendered inside the `item.tags.map` callback to avoid reconciliation issues.</comment>

<file context>
@@ -0,0 +1,38 @@
+            {state.prefix}: {item.name}
+            &lt;ul&gt;
+              {item.tags.map((tag) =&gt; (
+                &lt;li&gt;{item.name} - {tag.name}&lt;/li&gt;
+              ))}
+            &lt;/ul&gt;
</file context>
Suggested change
<li>{item.name} - {tag.name}</li>
<li key={tag.id}>{item.name} - {tag.name}</li>

✅ Addressed in 9ead191

<div>
{/* Template literal with captures */}
{state.items.map((item) => (
<div>{`${state.prefix} ${item.name} ${state.suffix}`}</div>
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Oct 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Elements returned from state.items.map need a stable key so React can reconcile list updates correctly.

Prompt for AI agents
Address the following comment on packages/ts-transformers/test/fixtures/closures/map-template-literal.input.tsx at line 21:

<comment>Elements returned from state.items.map need a stable key so React can reconcile list updates correctly.</comment>

<file context>
@@ -0,0 +1,26 @@
+      &lt;div&gt;
+        {/* Template literal with captures */}
+        {state.items.map((item) =&gt; (
+          &lt;div&gt;{`${state.prefix} ${item.name} ${state.suffix}`}&lt;/div&gt;
+        ))}
+      &lt;/div&gt;
</file context>
Fix with Cubic

export default recipe<State>("NestedProperty", (state) => {
return {
[UI]: (
<div>
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Oct 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a stable key to elements rendered inside state.items.map so React can reconcile list items correctly.

Prompt for AI agents
Address the following comment on packages/ts-transformers/test/fixtures/closures/map-nested-property.input.tsx at line 22:

<comment>Add a stable key to elements rendered inside `state.items.map` so React can reconcile list items correctly.</comment>

<file context>
@@ -0,0 +1,31 @@
+export default recipe&lt;State&gt;(&quot;NestedProperty&quot;, (state) =&gt; {
+  return {
+    [UI]: (
+      &lt;div&gt;
+        {state.items.map((item) =&gt; (
+          &lt;div&gt;
</file context>

✅ Addressed in 9ead191

return {
[UI]: (<div>
{/* Should NOT capture module-level constant or function */}
{state.items.map((item) => (<div>
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Oct 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Elements rendered from the map call need a stable key for correct reconciliation; please add a unique key prop (for example, item.id) to the <div> returned here.

Prompt for AI agents
Address the following comment on packages/ts-transformers/test/fixtures/closures/map-import-reference.expected.tsx at line 46:

<comment>Elements rendered from the map call need a stable key for correct reconciliation; please add a unique key prop (for example, `item.id`) to the `&lt;div&gt;` returned here.</comment>

<file context>
@@ -0,0 +1,51 @@
+    return {
+        [UI]: (&lt;div&gt;
+        {/* Should NOT capture module-level constant or function */}
+        {state.items.map((item) =&gt; (&lt;div&gt;
+            Item: {derive(item.price, _v1 =&gt; formatPrice(_v1 * (1 + TAX_RATE)))}
+          &lt;/div&gt;))}
</file context>
Suggested change
{state.items.map((item) => (<div>
{state.items.map((item) => (<div key={item.id}>
Fix with Cubic

<div>
{/* No captures - just uses the callback parameter */}
{state.items.map((item) => (
<div>Item #{item.id}: ${item.price}</div>
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Oct 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each element produced by state.items.map should include a unique key prop so React can reconcile list updates; without it, updates may misapply state and trigger warnings.

Prompt for AI agents
Address the following comment on packages/ts-transformers/test/fixtures/closures/map-no-captures.input.tsx at line 19:

<comment>Each element produced by state.items.map should include a unique key prop so React can reconcile list updates; without it, updates may misapply state and trigger warnings.</comment>

<file context>
@@ -0,0 +1,24 @@
+      &lt;div&gt;
+        {/* No captures - just uses the callback parameter */}
+        {state.items.map((item) =&gt; (
+          &lt;div&gt;Item #{item.id}: ${item.price}&lt;/div&gt;
+        ))}
+      &lt;/div&gt;
</file context>
Suggested change
<div>Item #{item.id}: ${item.price}</div>
<div key={item.id}>Item #{item.id}: ${item.price}</div>
Fix with Cubic

✅ Callback parameters (not captured)
✅ Index parameter preservation
✅ Plain arrays (not transformed)
✅ Cell arrays (ARE transformed)
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Oct 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "Edge Cases" list claims Cell arrays are transformed even though earlier sections explain they are rejected; please align this bullet with the documented behavior so the roadmap is self-consistent.

Prompt for AI agents
Address the following comment on packages/ts-transformers/docs/closure-implementation-roadmap.md at line 147:

<comment>The &quot;Edge Cases&quot; list claims Cell arrays are transformed even though earlier sections explain they are rejected; please align this bullet with the documented behavior so the roadmap is self-consistent.</comment>

<file context>
@@ -0,0 +1,439 @@
+✅ Callback parameters (not captured)
+✅ Index parameter preservation
+✅ Plain arrays (not transformed)
+✅ Cell arrays (ARE transformed)
+
+#### Test Coverage
</file context>
Suggested change
✅ Cell arrays (ARE transformed)
✅ Cell arrays (NOT transformed)

✅ Addressed in 35f40ac

}

// Create the new callback
const newCallback = factory.createArrowFunction(
Copy link
Contributor

@cubic-dev-ai cubic-dev-ai bot Oct 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rewriting every callback as an arrow function changes the semantics whenever the original was a function expression (e.g., this/arguments stop working). Please preserve the original function kind when rebuilding the node.

Prompt for AI agents
Address the following comment on packages/ts-transformers/src/closures/transformer.ts at line 543:

<comment>Rewriting every callback as an arrow function changes the semantics whenever the original was a function expression (e.g., `this`/`arguments` stop working). Please preserve the original function kind when rebuilding the node.</comment>

<file context>
@@ -0,0 +1,617 @@
+  }
+
+  // Create the new callback
+  const newCallback = factory.createArrowFunction(
+    callback.modifiers,
+    callback.typeParameters,
</file context>
Fix with Cubic

import { createNodeFactory } from "./module.ts";

let mapFactory: NodeFactory<any, any>;
let mapWithPatternFactory: NodeFactory<any, any>;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Shouldn't this be NodeFactory<any, any> | undefined?
  • Could NodeFactorys generics be unknown? (maybe not, could be too difficult)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes it should be, i removed the mapWithPatternFactory but i made this change on the mapFactory variable declared separately just above this comment

throw new Error("map_with_pattern currently only supports arrays");
}

const newArrayValue = resultWithLog.get().slice(0, initializedUpTo);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this returned function ((tx: IExtendedStorageTransaction) => { .. }) execute multiple times? If not, initializedUpTo is always 0 here (same with newArrayValue.length), though I'm not too familiar with these mechanisms

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok so it appears that the answer is yes, the returned function executes multiple times, reactively whenever inputs change. (this original comment was on the now-deleted separate map_with_pattern built-in which i have since re-unified with the preexisting map built-in but) it applies to map too so, leaving this conclusion here

import { isOpaqueRefType } from "../transformers/opaque-ref/opaque-ref.ts";
import { getHelperIdentifier } from "../transformers/opaque-ref/import-resolver.ts";

export class ClosureTransformer extends Transformer {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤘

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤘


// Collect destructured property names if the param is a destructured binding pattern
const destructuredProps = new Set<string>();
if (elemName && ts.isObjectBindingPattern(elemName)) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

future ideas: Similar to wanting a DSL for rewriting typescript, I wonder if there's some abstraction we could generally apply to the functions we rewrite to handle the transformation of args (optional args, optional destructuring), or handle rewriting functions the same way (regardless if it was an arrow function, function declaration, function expression etc.), since we could reuse them across several transformations, not worrying about all the different forms user authored TS comes in within the transformer business logic

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah... thanks... i agree, it would be better to have more conceptual unification here. i'm going to try to think about this again from the top soon and to also incorporate berni's nudge to try constructing our transformations in such a way that as much of the original code can survive untransformed as possible. some of the other transformers do a bunch of parameter name rewriting that's legacy from the original transformer whose behavior i wasn't ready to change. soon i'll take a broad pass at this

mapCall.expression.expression, // state.items
factory.createIdentifier("map_with_pattern"),
)
: factory.createIdentifier("map_with_pattern"); // Fallback (shouldn't happen)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's throw an error if we don't expect this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch, thanks, done

// Check if the callback is an arrow function or function expression
if (
callback &&
(ts.isArrowFunction(callback) || ts.isFunctionExpression(callback))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's another similar helper function defined here isFunctionDeclaration that seems to include various types of function types -- if we don't care what type of function it is, maybe we can reuse that, or just add these utils to the generic ast helpers in ast/ (i think that's the rough idea anyway)

@mathpirate mathpirate force-pushed the feature/map-closures branch from 2ddcb2a to f943229 Compare October 9, 2025 19:46

// Module-level function - should NOT be captured
function formatPrice(price: number): string {
return `$${price.toFixed(2)}`;
Copy link
Collaborator

@jsantell jsantell Oct 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if this function did something like

function formatPrice(price: number): string {
  // @ts-ignore
  globalThis.x = (globalThis.x ?? 0) + 1;
  // @ts-ignore
  return `${globalThis.x}_$${price.toFixed(2)}`;
}

import ts from "typescript";
import type { TransformationContext } from "../core/context.ts";

export interface ClosureRule {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this used?

@jsantell
Copy link
Collaborator

Probably could exclude the "jsx-key" rule from linting as it's not applicable really in our environment

@mathpirate mathpirate force-pushed the feature/map-closures branch from a679c0d to ba95dea Compare October 16, 2025 19:25
mathpirate and others added 20 commits October 20, 2025 14:09
…ill broken)

      This commit removes special-case handling for array-map callbacks and treats
      all callback parameters (builder and array-map) uniformly for fine-grained
      reactivity.

      ## What Works

      Successfully unified parameter handling:
      - Both builder (state) and map (element) parameters now excluded from
        derive dependencies
      - Property accesses (state.count, element.price) correctly included
      - Manually written mapWithPattern(recipe(...)) works perfectly with JSX
        expression transformations

      ## What's Broken

      Synthetic AST nodes (created by closures transformer) fail to detect computations:
      - All closure transformation tests failing (9 tests)
      - Root cause: synthetic AST nodes lack symbol information
      - element.price * discount in transformed map callbacks not wrapped in derive
      - Merged analysis from children has requiresRewrite: false

      ## Changes Made

      1. dataflow.ts: Removed array-map special cases at lines 362-373, 411-428
         - These forced requiresRewrite: true for array-map parameters
         - No longer needed in new uniform model

      2. helpers.ts: Unified resolvesToParameterOfKind()
         - Removed allowPropertyTraversal distinction
         - Both builder and array-map now only match root identifiers
         - Property accesses excluded from parameter detection

      3. Test fixtures: Updated expected outputs for all map callback tests
         - Now expect derive/ifElse wrappers in JSX expressions
         - Tests currently fail due to synthetic AST issue

      4. New test: manual-map-with-pattern.{input,expected}.tsx
         - Validates manually written code works correctly
         - Demonstrates desired behavior

      ## Next Steps

      Need to solve synthetic AST analysis to detect computations without symbol
      information. Options to explore:
      - Fix synthetic handling in dataflow.ts (lines 173-240)
      - Modify closures transformer to preserve more information
      - Alternative approach TBD

      🤖 Generated with [Claude Code](https://claude.com/claude-code)

      Co-Authored-By: Claude <noreply@anthropic.com>
…etection

- Unified top-level and nested map callback transformation into single visitor
- Always transform map callbacks (even with zero captures) to ensure params are opaque
- Added detection of recipe/builder parameters as opaque even when symbol resolution fails
- Fixed synthetic PropertyAccessExpression handling in dataflow analysis
- Updated test expectations to match new behavior

This sets foundation for hierarchical params refactor.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@mathpirate mathpirate force-pushed the feature/map-closures branch from d682fd0 to 96f2f72 Compare October 20, 2025 21:09
@mathpirate mathpirate merged commit f9ded59 into main Oct 21, 2025
8 checks passed
@mathpirate mathpirate deleted the feature/map-closures branch October 21, 2025 16:49
seefeldb added a commit that referenced this pull request Oct 21, 2025
@seefeldb seefeldb restored the feature/map-closures branch October 22, 2025 17:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants