feat(api): re-arranging Cell types

[seefeldb]

Unify Type System: IKeyable Interface and Cell Type Architecture

Note: This carries over some duplication between OpaqueRef and Cells as separate types. And there is duplicative use of symbols to brand types. This will be fixed in a future PR.

Overview

This PR fundamentally restructures the type system to use a Higher-Kinded Type (HKT) pattern for the IKeyable interface, replacing the previous scattered type definitions with a unified, composable architecture. This change improves type safety, eliminates circular dependencies, and provides a foundation for better variance control across the codebase.

Major Changes

1. Introduced HKT-based IKeyable<T, Wrap> Interface (packages/api/index.ts)

• Replaced multiple ad-hoc .key() method signatures with a single, reusable IKeyable<T, Wrap> interface
• Uses lightweight Higher-Kinded Types (HKT) to parameterize the return type wrapper (e.g., Cell<T>, OpaqueCell<T>, ReadonlyCell<T>)
• Properly handles covariance with out T modifier and variance guards (unknown extends K)
• Automatically unwraps nested BrandedCell types to prevent accumulation (e.g., Cell<Cell<T>> becomes Cell<T>)
• Includes comprehensive JSDoc with variance explanation and examples

Key type:

export interface IKeyable<out T, Wrap extends HKT> {
  key<K extends PropertyKey>(
    this: IsThisObject, // only allow on object types
    valueKey: K,
  ): KeyResultType<T, K, Wrap>;
}

2. Unified Cell Architecture with Capability Interfaces

Restructured the entire cell type system into composable capability interfaces:

• IAnyCell<T> - Base marker interface
• IReadable<T> - Read operations (.get())
• IWritable<T> - Write operations (.set(), .update(), .push())
• IStreamable<T> - Event sending (.send())
• IEquatable - Equality checks (.equals())
• IKeyable<T, Wrap> - Property access (.key())
• IResolvable<T, C> - Cell resolution (.resolveAsCell())
• IDerivable<T> - Derivation operations (.map(), .mapWithPattern())
• IOpaquable<T> - Legacy opaque operations (deprecated methods)

3. Brand-Based Cell Variants

Introduced a brand system using BrandedCell<T, Brand> to distinguish cell types at compile-time:

• Cell<T> - Full cell with read/write/stream capabilities (brand: "cell")
• OpaqueCell<T> - Opaque reference for keying and derivation only (brand: "opaque")
• Stream<T> - Stream-only for event sending (brand: "stream")
• ReadonlyCell<T> - Read-only variant (brand: "readonly")
• WriteonlyCell<T> - Write-only variant (brand: "writeonly")
• ComparableCell<T> - Equality checks only (brand: "comparable")

Each variant composes only the capability interfaces it needs, preventing misuse.

4. Replaced Cellify<T> with AnyCellWrapping<T>

• Renamed for clarity - this type is specifically for write operations
• Improved handling of BrandedCell unwrapping in nested structures
• Added support for ID and ID_FIELD metadata symbols
• Used consistently in .set(), .update(), .push(), and .send() methods

5. Unified CellLike and Opaque Semantics

• CellLike<T> - Simplified to just AnyCell<T> (top level must be a cell)
• Opaque<T> - Accepts T OR any cell wrapping T, recursively at any level
• Clarified the distinction: CellLike requires cells, Opaque accepts plain or wrapped values

6. Module Augmentation Refactor (packages/runner/src/cell.ts)

• Moved from augmenting concrete types to augmenting capability interfaces
• Augments IAnyCell<T>, IWritable<T>, IStreamable<T> with runtime-specific methods
• Cleaner separation between public API (@commontools/api) and runtime implementation

7. Type Utilities

• UnwrapCell<T> - Recursively unwraps BrandedCell types while preserving any and unknown
• KeyResultType<T, K, Wrap> - Implements the .key() return type logic with all variance guards

Benefits

1. Type Safety: Stronger compile-time guarantees prevent misuse of cell methods
2. Composability: Capability interfaces can be mixed and matched for new cell variants
3. Variance Control: Proper covariance handling prevents unsound type assignments
4. No Circular Dependencies: UnwrapCell and BrandedCell break previous circular type issues
5. Better Inference: .key() now correctly infers nested property types across all cell variants
6. Maintainability: Single source of truth for .key() behavior via IKeyable

Migration Notes

• Renamed: Cellify<T> ‚Üí AnyCellWrapping<T> (semantically the same for write operations)
• Renamed: OpaqueRefMethods<T> ‚Üí IOpaqueCell<T> (interface for opaque cell capabilities)
• New: All cell types now extend BrandedCell<T, Brand> for compile-time differentiation
• Deprecated: Legacy methods in IOpaquable are marked as deprecated

Testing

• All existing tests pass with updated type annotations
• Added new schema tests for UnwrapCell and nested cell handling
• Integration tests verify .key() works correctly across all cell variants

---

Summary by cubic

Unifies the Cell type system using brands and capability interfaces, and adds an HKT-based key() that returns correctly wrapped types. This improves type safety, simplifies APIs, and fixes inference across runner, CLI, and UI.

• Refactors

  • Added BrandedCell and capability interfaces (Readable, Writable, Streamable, Equatable, Derivable, Keyable, Resolvable).
  • Implemented IKeyable<T, Wrap> with HKT wrappers (Cell, ReadonlyCell, WriteonlyCell, ComparableCell); key() now unwraps brands and preserves variance.
  • Introduced AnyCell, OpaqueCell, IOpaqueCell, and IOpaquable; OpaqueRef is now an OpaqueCell-based proxy.
  • Replaced Cellify with AnyCellWrapping for set/update/push/send.
  • Renamed isOpaqueRef to isOpaqueCell; parseLink now accepts AnyCell and correctly handles stream bases.
  • Updated RegularCell/StreamCell to use AnyCellWrapping and improved schema tracking in key().
  • Tightened types in runner, CLI, UI, and tests; added explicit casts where inference is limited.
  • Added a TypeScript profiling harness under packages/api/perf to measure type-checking cost of API types.
  • Updated schema-generator and ts-transformers to detect CELL_BRAND and treat OpaqueCell correctly in generation.
  • Schema generator no longer emits ID/ID_FIELD in inferred schema output.

• Migration

  • Rename isOpaqueRef(...) to isOpaqueCell(...).
  • FetchDataFunction/StreamDataFunction now return OpaqueRef<{ pending; result; error }>.
  • key() may need explicit casts when selecting streams (e.g., const s = cell.key("handler") as unknown as Stream).
  • Use key(index) for array elements on OpaqueRef proxies instead of bracket access.
  • Update write methods to pass AnyCellWrapping (set, update, push, send).
  • OpaqueRef methods are exposed via IOpaqueCell/IOpaquable; use export()/connect() from these interfaces.

^{Written for commit 16e2226. Summary will update automatically on new commits.}

[cubic-dev-ai]

3 issues found across 27 files

Prompt for AI agents (all 3 issues)

Understand the root cause of the following 3 issues and fix them.


<file name="packages/api/index.ts">

<violation number="1" location="packages/api/index.ts:34">
`BrandedCell` dropped the underlying value type, so helpers like `UnwrapCell` now infer `unknown`, breaking key() and wrapping typings.</violation>

<violation number="2" location="packages/api/index.ts:62">
`push` now widens the non-array branch to `any[]`, so non-array cells can call `.push(...)` without a compile-time error, risking runtime failures.</violation>
</file>

<file name="packages/runner/src/link-utils.ts">

<violation number="1" location="packages/runner/src/link-utils.ts:250">
This branch has the same issue: `isAnyCell` can match streams, but streams do not have `entityId`, so `toURI` receives `undefined` and throws when a stream is supplied as the base. Pull the id from `getAsNormalizedFullLink()` (which streams implement) instead.</violation>
</file>

_{React with 👍 or 👎 to teach cubic. Mention @cubic-dev-ai to give feedback, ask questions, or re-run the review.}

[cubic-dev-ai]

This branch has the same issue: isAnyCell can match streams, but streams do not have entityId, so toURI receives undefined and throws when a stream is supplied as the base. Pull the id from getAsNormalizedFullLink() (which streams implement) instead.

Prompt for AI agents

Address the following comment on packages/runner/src/link-utils.ts at line 250:

<comment>This branch has the same issue: `isAnyCell` can match streams, but streams do not have `entityId`, so `toURI` receives `undefined` and throws when a stream is supplied as the base. Pull the id from `getAsNormalizedFullLink()` (which streams implement) instead.</comment>

<file context>
@@ -247,7 +247,7 @@ export function parseLink(
     // If no id provided, use base cell&#39;s document
     if (!id &amp;&amp; base) {
-      id = isCell(base) ? toURI(base.entityId) : base.id;
+      id = isAnyCell(base) ? toURI(base.entityId) : base.id;
     }
 
</file context>

Suggested change

	id = isAnyCell(base) ? toURI(base.entityId) : base.id;
	id = isAnyCell(base) ? base.getAsNormalizedFullLink().id : base.id;

[cubic-dev-ai]

No issues found across 17 files

[cubic-dev-ai]

No issues found across 7 files

[cubic-dev-ai]

No issues found across 1 file

[cubic-dev-ai]

Reviewed changes from recent commits (found 1 issue).

1 issue found across 1 file

Prompt for AI agents (all 1 issues)

Understand the root cause of the following 1 issues and fix them.


<file name="packages/api/index.ts">

<violation number="1" location="packages/api/index.ts:151">
The new KeyResultType no longer unwraps nested branded cells, so key(&quot;nested&quot;) on Cell&lt;{ nested: Cell&lt;number&gt; }&gt; now infers Cell&lt;Cell&lt;number&gt;&gt;, breaking the documented guarantee that nested cells don’t accumulate wrappers.</violation>
</file>

_{React with 👍 or 👎 to teach cubic. Mention @cubic-dev-ai to give feedback, ask questions, or re-run the review.}

[jsantell]

I don't yet completely grok all the new types, but the composable feature approach to cell types LGTM

[jsantell]

secondary?

[jsantell]

This is probably in your local deno cache, there are no node_modules in our workspace

[jsantell]

We don't use node in our codebase

[jsantell]

Nice specialization work here

[jsantell]

Could "inherit" from IsThisArray:

type IsThisObject = 
  | BrandedCell<JSONObject>
  | BrandedCell<Record<string, unknown | any>>
  | IsThisArray

[seefeldb]

good idea, done

[jsantell]

What is the result of unknown | any?

[seefeldb]

just unknown seems to work, so removed the any

[jsantell]

What is the meaning of the brand value (second generic)? I see "opaque" and "comparable" etc., could these be constrained to some subset? Or are all strings allowed?

[seefeldb]

It makes it so that Cell<T> & Opaque<T> isn't allowed. We could restrict it further, but in the interest of keeping types simple I didn't see the upside, this is very internal type.

[mathpirate]

parseLink was updated to accept AnyCell, but when the base is actually a stream we still try to read entityId and blow up.

I made a separate branch https://github.com/commontoolsinc/labs/tree/dx1/unify-types-stream-base-fix where I added a targeted regression test (stream-base-parse-link.test.ts) that reproduces the failure, then taught parseLink to normalize the base via getAsNormalizedFullLink() before dereferencing id/space. With that change the test passes, without it the test fails.

Unfortunately that branch has typechecking errors and I can't tell whether they're coming from my change or not... leaving this comment so you can help me figure it out Berni