Improve incremental builds (tailwindlabs#13168)

* ensure we don't crash on deleted files

* change return type of `compile` to include a `rebuild()` function

This will allow us in the future to perform incremental rebuilds after
the initial rebuild. This is purely the API change so that we can
prepare all the call sites to use this new API.

* set `@tailwind utilities` nodes

Instead of replacing the node that represents the `@tailwind utilities`
with the generated AST nodes from the rawCandidates, we will set the
nodes of the `@tailwind utilities` rule to the AST nodes instead.

This way we dont' have to remove and replace the `@tailwind utilities`
rule with `n` new nodes. This will later allow us to track the
`@tailwindcss utilities` rule itself and update its `nodes` for
incremental rebuilds.

This also requires a small change to the printer where we now need to
print the children of the `@tailwind utilities` rule. Note: we keep the
same `depth` as-if the `@tailwindcss utilities` rule was not there.
Otherwise additional indentation would be present.

* move sorting to the `ast.sort()` call

This will allow us to keep sorting AST nodes in a single spot.

* move parser functions to the `DesignSystem`

This allows us to put all the parsers in the `DesignSystem`, this allows
us to scope the parsers to the current design system (the current theme,
current utility values and variants).

The implementation of these parsers are also using a `DefaultMap`
implementation. This allows us to make use of caching and only parse a
candidate, parse a variant or compile AST nodes for a given raw
candidate once if we've already done this work in the past.

Again, this is scoped to the `DesignSystem` itself. This means that if
the corresponding theme changes, then we will create a new
`DesignSystem` entirely and the caches will be garbage collected. This
is important because a candidate like `bg-primary` can be invalid in
`DesignSystem` A, but can be valid in `DesignSystem` B and vice versa.

* ensure we sort variants alphabetically by root

For incremental rebuilds we don't know all the used variants upfront,
which means that we can't sort them upfront either (what we used to do).

This code now allows us to sort the variants deterministically when
sorting the variants themselves instead of relying on the fact that they
used to be sorted before.

The sort itself could change slightly compared to the previous
implementation (especially when you used stacked variants in your
candidates), but it will still be deterministic.

* replace `localeCompare` comparisons

Use cheaper comparisons than `localeCompare` when comparing 2 strings.
We currently don't care if it is 100% correctly sorted, but we just want
consistent sorting. This is currently faster compared to
`localeCompare`.

Another benefit is that `localeCompare` could result in
non-deterministic results if the CSS was generated on 2 separate
computers where the `locale` is different.

We could solve that by adding a dedicated locale, but it would still be
slower compared to this.

* track invalid candidates

When an incoming raw candidates doesn't produce any output, then we can
mark it as an invalid candidate. This will allow us to reduce the amount
of candidates to handle in incremental rebuilds.

* add initial incremental rebuild implementation

This includes a number of steps:

1. Track the `@tailwind utilities` rule, so that we can adjust its nodes
   later without re-parsing the full incoming CSS.
2. Add the new incoming raw candidates to the existing set of
   candidates.
3. Parse the merged set to `compileCandidates` (this can accept any
   `Iterable<string>`, which means `string[]`, `Set<string>`, ...)
4. Get the new AST nodes, update the `@tailwind utilities` rule's nodes
   and re-print the AST to CSS.

* improvement 1: ignore known invalid candidates

This will reduce the amount of candidates to handle. They would
eventually be skipped anyway, but now we don't even have to re-parse
(and hit a cache) at all.

* improvement 2: skip work, when generated AST is the same

Currently incremental rebuilds are additive, which means that we are not
keeping track if we should remove CSS again in development.

We can exploit this information, because now we can quickly check the
amoutn of generated AST nodes.

- If they are the same then nothing new is generated — this means that
  we can re-use the previous compiled CSS. We don't even have to
  re-print the AST because we already did do that work in the past.
- If there are more AST nodes, something new is generated — this means
  that we should update the `@tailwind utilities` rule and re-print the
  CSS. We can store the result for future incremental rebuilds.

* improvement 3: skip work if no new candidates are detected

- We already know a set of candidates from previous runs.
- We also already know a set of candidates that are invalid and don't
  produce anything.

This means that an incremental rebuild could give us a new set of
candidates that either already exist or are invalid.

If nothing changes, then we can re-use the compiled CSS.

This actually happens more often than you think, and the bigger your
project is the better this optimization will be.

For example:

```
// Imagine file A exists:
<div class="flex items-center justify-center"></div>
<button class="text-red-500">Delete</button>
```

```
// Now you add a second file B:
<div class="text-red-500 flex"></div>
```

You just created a brand new file with a bunch of HTML elements and
classes, yet all of the candidates in file B already exist in file A, so
nothing changes to the actual generated CSS.

Now imagine the other hundreds of files that already contain hundreds of
classes.

The beauty of this optimization is two-fold:

- On small projects, compiling is very fast even without this check.
  This means it is performant.
- On bigger projects, we will be able to re-use existing candidates.
  This means it stays performant.

* remove `getAstNodeSize`

We can move this up the tree and move it to the `rebuild` function
instead.

* remove invalid candidate tracking from `DesignSystem`

This isn't used anywhere but only in the `rebuild` of the compile step.
This allows us to remove it entirely from core logic, and move it up the
chain where it is needed.

* replace `throwOnInvalidCandidate` with `onInvalidCanidate`

This was only needed for working with `@apply`, now this logic _only_
exists in the code path where we are handling `@apply`.

* update `compile` API signature

* update callsite of `compile()` function

* fix typo

Author: RobinMalfait

oxide/crates/core/src/lib.rs

@@ -433,9 +433,9 @@ fn read_all_files_sync(changed_content: Vec<ChangedContent>) -> Vec<Vec<u8>> {
 
     changed_content
         .into_iter()
-        .map(|c| match (c.file, c.content) {
-            (Some(file), None) => std::fs::read(file).unwrap(),
-            (None, Some(content)) => content.into_bytes(),
+        .filter_map(|c| match (c.file, c.content) {
+            (Some(file), None) => std::fs::read(file).ok(),
+            (None, Some(content)) => Some(content.into_bytes()),
             _ => Default::default(),
         })
         .collect()

packages/@tailwindcss-cli/src/commands/build/index.ts

@@ -100,7 +100,8 @@ export async function handle(args: Result<ReturnType<typeof options>>) {
   )
 
   // Compile the input
-  let result = compile(input, candidates)
+  let { build } = compile(input)
+  let result = build(candidates)
 
   // Optimize the output
   if (args['--minify'] || args['--optimize']) {
@@ -193,7 +194,7 @@ export async function handle(args: Result<ReturnType<typeof options>>) {
         }
 
         // Compile the input
-        let result = compile(input, candidates)
+        result = compile(input).build(candidates)
 
         // Optimize the output
         if (args['--minify'] || args['--optimize']) {

packages/@tailwindcss-postcss/src/index.ts

@@ -53,7 +53,7 @@ function tailwindcss(opts: PluginOptions = {}): AcceptedPlugin {
 
         // No `@tailwind` means we don't have to look for candidates
         if (!hasTailwind) {
-          replaceCss(compile(root.toString(), []))
+          replaceCss(compile(root.toString()).build([]))
           return
         }
 
@@ -83,7 +83,7 @@ function tailwindcss(opts: PluginOptions = {}): AcceptedPlugin {
           })
         }
 
-        replaceCss(compile(root.toString(), candidates))
+        replaceCss(compile(root.toString()).build(candidates))
       },
     ],
   }

packages/@tailwindcss-vite/src/index.ts

@@ -63,7 +63,7 @@ export default function tailwindcss(): Plugin[] {
   }
 
   function generateCss(css: string) {
-    return compile(css, Array.from(candidates))
+    return compile(css).build(Array.from(candidates))
   }
 
   function generateOptimizedCss(css: string) {

packages/tailwindcss/src/ast.ts

@@ -105,6 +105,13 @@ export function toCss(ast: AstNode[]) {
         return css
       }
 
+      if (node.selector === '@tailwind utilities') {
+        for (let child of node.nodes) {
+          css += stringify(child, depth)
+        }
+        return css
+      }
+
       // Print at-rules without nodes with a `;` instead of an empty block.
       //
       // E.g.:

packages/tailwindcss/src/candidate.bench.ts

@@ -1,9 +1,8 @@
 import { scanDir } from '@tailwindcss/oxide'
 import { bench } from 'vitest'
-import { parseCandidate, parseVariant } from './candidate'
+import { parseCandidate } from './candidate'
 import { buildDesignSystem } from './design-system'
 import { Theme } from './theme'
-import { DefaultMap } from './utils/default-map'
 
 // FOLDER=path/to/folder vitest bench
 const root = process.env.FOLDER || process.cwd()
@@ -15,10 +14,6 @@ const designSystem = buildDesignSystem(new Theme())
 
 bench('parseCandidate', () => {
   for (let candidate of result.candidates) {
-    parseCandidate(
-      candidate,
-      designSystem.utilities,
-      new DefaultMap((variant, map) => parseVariant(variant, designSystem.variants, map)),
-    )
+    parseCandidate(candidate, designSystem)
   }
 })

packages/tailwindcss/src/candidate.test.ts

@@ -1,7 +1,7 @@
 import { expect, it } from 'vitest'
-import { parseCandidate, parseVariant } from './candidate'
+import { buildDesignSystem } from './design-system'
+import { Theme } from './theme'
 import { Utilities } from './utilities'
-import { DefaultMap } from './utils/default-map'
 import { Variants } from './variants'
 
 function run(
@@ -11,11 +11,12 @@ function run(
   utilities ??= new Utilities()
   variants ??= new Variants()
 
-  let parsedVariants = new DefaultMap((variant, map) => {
-    return parseVariant(variant, variants!, map)
-  })
+  let designSystem = buildDesignSystem(new Theme())
 
-  return parseCandidate(candidate, utilities, parsedVariants)
+  designSystem.utilities = utilities
+  designSystem.variants = variants
+
+  return designSystem.parseCandidate(candidate)
 }
 
 it('should skip unknown utilities', () => {

packages/tailwindcss/src/candidate.ts

@@ -1,3 +1,4 @@
+import type { DesignSystem } from './design-system'
 import { decodeArbitraryValue } from './utils/decode-arbitrary-value'
 import { segment } from './utils/segment'
 
@@ -206,14 +207,7 @@ export type Candidate =
       important: boolean
     }
 
-export function parseCandidate(
-  input: string,
-  utilities: {
-    has: (value: string) => boolean
-    kind: (root: string) => Omit<Candidate['kind'], 'arbitrary'>
-  },
-  parsedVariants: { get: (value: string) => Variant | null },
-): Candidate | null {
+export function parseCandidate(input: string, designSystem: DesignSystem): Candidate | null {
   // hover:focus:underline
   // ^^^^^ ^^^^^^           -> Variants
   //             ^^^^^^^^^  -> Base
@@ -228,7 +222,7 @@ export function parseCandidate(
   let parsedCandidateVariants: Variant[] = []
 
   for (let variant of rawVariants) {
-    let parsedVariant = parsedVariants.get(variant)
+    let parsedVariant = designSystem.parseVariant(variant)
     if (parsedVariant === null) return null
 
     // Variants are applied left-to-right meaning that any representing pseudo-
@@ -320,7 +314,7 @@ export function parseCandidate(
     base = base.slice(1)
   }
 
-  let [root, value] = findRoot(base, utilities)
+  let [root, value] = findRoot(base, designSystem.utilities)
 
   let modifierSegment: string | null = null
 
@@ -335,13 +329,13 @@ export function parseCandidate(
     modifierSegment = rootModifierSegment
 
     // Try to find the root and value, without the modifier present
-    ;[root, value] = findRoot(rootWithoutModifier, utilities)
+    ;[root, value] = findRoot(rootWithoutModifier, designSystem.utilities)
   }
 
   // If there's no root, the candidate isn't a valid class and can be discarded.
   if (root === null) return null
 
-  let kind = utilities.kind(root)
+  let kind = designSystem.utilities.kind(root)
 
   if (kind === 'static') {
     if (value !== null) return null
@@ -475,15 +469,7 @@ function parseModifier(modifier: string): CandidateModifier {
   }
 }
 
-export function parseVariant(
-  variant: string,
-  variants: {
-    has: (value: string) => boolean
-    kind: (root: string) => Omit<Variant['kind'], 'arbitrary'>
-    compounds: (root: string) => boolean
-  },
-  parsedVariants: { get: (value: string) => Variant | null },
-): Variant | null {
+export function parseVariant(variant: string, designSystem: DesignSystem): Variant | null {
   // Arbitrary variants
   if (variant[0] === '[' && variant[variant.length - 1] === ']') {
     /**
@@ -535,20 +521,20 @@ export function parseVariant(
     // - `group-hover/foo/bar`
     if (additionalModifier) return null
 
-    let [root, value] = findRoot(variantWithoutModifier, variants)
+    let [root, value] = findRoot(variantWithoutModifier, designSystem.variants)
 
     // Variant is invalid, therefore the candidate is invalid and we can skip
     // continue parsing it.
     if (root === null) return null
 
-    switch (variants.kind(root)) {
+    switch (designSystem.variants.kind(root)) {
       case 'static': {
         if (value !== null) return null
 
         return {
           kind: 'static',
           root,
-          compounds: variants.compounds(root),
+          compounds: designSystem.variants.compounds(root),
         }
       }
 
@@ -564,7 +550,7 @@ export function parseVariant(
               kind: 'arbitrary',
               value: decodeArbitraryValue(value.slice(1, -1)),
             },
-            compounds: variants.compounds(root),
+            compounds: designSystem.variants.compounds(root),
           }
         }
 
@@ -573,14 +559,14 @@ export function parseVariant(
           root,
           modifier: modifier === null ? null : parseModifier(modifier),
           value: { kind: 'named', value },
-          compounds: variants.compounds(root),
+          compounds: designSystem.variants.compounds(root),
         }
       }
 
       case 'compound': {
         if (value === null) return null
 
-        let subVariant = parsedVariants.get(value)
+        let subVariant = designSystem.parseVariant(value)
         if (subVariant === null) return null
         if (subVariant.compounds === false) return null
 
@@ -589,7 +575,7 @@ export function parseVariant(
           root,
           modifier: modifier === null ? null : { kind: 'named', value: modifier },
           variant: subVariant,
-          compounds: variants.compounds(root),
+          compounds: designSystem.variants.compounds(root),
         }
       }
     }