Improve incremental builds #13168
Merged
Improve incremental builds #13168
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
76e03ce to
b75828d
Compare
This was referenced Mar 9, 2024
b75828d to
26a857e
Compare
thecrypticace
approved these changes
Mar 11, 2024
32318d3 to
fa665e0
Compare
thecrypticace
approved these changes
Mar 11, 2024
This will allow us in the future to perform incremental rebuilds after the initial rebuild. This is purely the API change so that we can prepare all the call sites to use this new API.
Instead of replacing the node that represents the `@tailwind utilities` with the generated AST nodes from the rawCandidates, we will set the nodes of the `@tailwind utilities` rule to the AST nodes instead. This way we dont' have to remove and replace the `@tailwind utilities` rule with `n` new nodes. This will later allow us to track the `@tailwindcss utilities` rule itself and update its `nodes` for incremental rebuilds. This also requires a small change to the printer where we now need to print the children of the `@tailwind utilities` rule. Note: we keep the same `depth` as-if the `@tailwindcss utilities` rule was not there. Otherwise additional indentation would be present.
This will allow us to keep sorting AST nodes in a single spot.
This allows us to put all the parsers in the `DesignSystem`, this allows us to scope the parsers to the current design system (the current theme, current utility values and variants). The implementation of these parsers are also using a `DefaultMap` implementation. This allows us to make use of caching and only parse a candidate, parse a variant or compile AST nodes for a given raw candidate once if we've already done this work in the past. Again, this is scoped to the `DesignSystem` itself. This means that if the corresponding theme changes, then we will create a new `DesignSystem` entirely and the caches will be garbage collected. This is important because a candidate like `bg-primary` can be invalid in `DesignSystem` A, but can be valid in `DesignSystem` B and vice versa.
For incremental rebuilds we don't know all the used variants upfront, which means that we can't sort them upfront either (what we used to do). This code now allows us to sort the variants deterministically when sorting the variants themselves instead of relying on the fact that they used to be sorted before. The sort itself could change slightly compared to the previous implementation (especially when you used stacked variants in your candidates), but it will still be deterministic.
Use cheaper comparisons than `localeCompare` when comparing 2 strings. We currently don't care if it is 100% correctly sorted, but we just want consistent sorting. This is currently faster compared to `localeCompare`. Another benefit is that `localeCompare` could result in non-deterministic results if the CSS was generated on 2 separate computers where the `locale` is different. We could solve that by adding a dedicated locale, but it would still be slower compared to this.
When an incoming raw candidates doesn't produce any output, then we can mark it as an invalid candidate. This will allow us to reduce the amount of candidates to handle in incremental rebuilds.
This includes a number of steps: 1. Track the `@tailwind utilities` rule, so that we can adjust its nodes later without re-parsing the full incoming CSS. 2. Add the new incoming raw candidates to the existing set of candidates. 3. Parse the merged set to `compileCandidates` (this can accept any `Iterable<string>`, which means `string[]`, `Set<string>`, ...) 4. Get the new AST nodes, update the `@tailwind utilities` rule's nodes and re-print the AST to CSS.
This will reduce the amount of candidates to handle. They would eventually be skipped anyway, but now we don't even have to re-parse (and hit a cache) at all.
Currently incremental rebuilds are additive, which means that we are not keeping track if we should remove CSS again in development. We can exploit this information, because now we can quickly check the amoutn of generated AST nodes. - If they are the same then nothing new is generated — this means that we can re-use the previous compiled CSS. We don't even have to re-print the AST because we already did do that work in the past. - If there are more AST nodes, something new is generated — this means that we should update the `@tailwind utilities` rule and re-print the CSS. We can store the result for future incremental rebuilds.
- We already know a set of candidates from previous runs. - We also already know a set of candidates that are invalid and don't produce anything. This means that an incremental rebuild could give us a new set of candidates that either already exist or are invalid. If nothing changes, then we can re-use the compiled CSS. This actually happens more often than you think, and the bigger your project is the better this optimization will be. For example: ``` // Imagine file A exists: <div class="flex items-center justify-center"></div> <button class="text-red-500">Delete</button> ``` ``` // Now you add a second file B: <div class="text-red-500 flex"></div> ``` You just created a brand new file with a bunch of HTML elements and classes, yet all of the candidates in file B already exist in file A, so nothing changes to the actual generated CSS. Now imagine the other hundreds of files that already contain hundreds of classes. The beauty of this optimization is two-fold: - On small projects, compiling is very fast even without this check. This means it is performant. - On bigger projects, we will be able to re-use existing candidates. This means it stays performant.
We can move this up the tree and move it to the `rebuild` function instead.
This isn't used anywhere but only in the `rebuild` of the compile step. This allows us to remove it entirely from core logic, and move it up the chain where it is needed.
This was only needed for working with `@apply`, now this logic _only_ exists in the code path where we are handling `@apply`.
fa665e0 to
edc8523
Compare
thecrypticace
approved these changes
Mar 11, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR prepares the internals for way faster incremental rebuilds.
The biggest API change is internal, but the definition of the
compilefunction now looks like this:This will allow us to call a
rebuildfunction with new incoming candidates. In case you want to run a full build again (e.g.: when a CSS file changed), then you can call thecompilefunction again.All potential caches are scoped to the
compilefunction, which means that if you callcompileagain, you will get a fresh instance (and the old caches can be garbage collected).This PR also introduces more dedicated methods on the internal
DesignSystemfor parsing candidates, parsing variants and compiling AST nodes from candidates.All these methods are behind caches, so we don't do unnecessary work and re-use everything as much as possible. All objects will use the same instance which means that we can compare them by reference when needed.
This also includes optimizations for incremental rebuilds, such as: