Custom resolver

[dgp1130]

Hi, I'm trying to use @parcel/css to bundle CSS files, however there doesn't seem to be a way to provide a custom resolver for @import statements, the package appears to directly use the real file system. This is a challenge for my use case where I'm hoping to use Parcel for CSS bundling in a Bazel workspace. Bazel is a build tool which puts generated files in different directories from the source code, meaning that @import './foo.css' could resolve to ./foo or $project_root/dist/bin/path/to/dir/foo.css, or even a few other paths. I would like to configure @parcel/css to resolve these paths correctly, but AFAICT, this package doesn't have any direct support for that. See dgp1130/rules_prerender#46 and dgp1130/rules_prerender@7e995d6 for a little more info about the use case and my attempt to implement it.

It does seem that custom resolvers are supported in .parcelrc, but I don't get the impression @parcel/css has any understanding of that configuration file. I'm unclear how the parcel package supports custom CSS resolvers if @parcel/css doesn't, but maybe they don't compose each other in the way I think they do? If there is a more appropriate package or different level of abstraction I should be using which would support resolvers, please let me know and I'm happy to try that.

If it does make sense to support custom resolvers in @parcel/css, I'm happy to put in the effort to contribute it. I think we would just need to update BundleConfig to accept a SourceResolver from JS and then use that instead of the FileResolver. All the inputs and outputs would be strings, so serializing into and out-of JS shouldn't be too much of an issue. I think it's possible, though I haven't done too much with Rust or WASM, so any pointers or suggestions here would be appreciated.

Is this a contribution you would be interested in accepting or am I going about this problem in the wrong manner?

[devongovett]

You should use Parcel for this eg via the api https://parceljs.org/features/parcel-api/. Parcel CSS is used under the hood, and you can hook into resolution via the plugin system. Standalone, Parcel CSS has very limited bundling support.

[dgp1130]

@devongovett, thanks for the suggestion. I spent most of today trying to get things working with the Parcel API. I was kind of able to get a custom resolver working, but it turned out much more complicated than I expected. Parcel plugins seem to require being loaded from an NPM package, with some tricks available to load local plugins. However as a build system, Bazel is already capable of solving those orchestration challenges pretty easily, and Parcel's tight coupling with NPM and node_modules/ actively fights against it. The .parcelrc file and plugin packages are providing features I don't need with Bazel, which was why I was trying to aim for a lower level of abstraction in @parcel/css.

I'm wondering if there's a package under @parcel/* which provides the same hooks as the Parcel API but without all that configuration and loading? The Parcel API seems too high level (lots of automated loading and importing which doesn't work well in a Bazel context with a local plugin) while @parcel/css seems to be too low-level (can't specify a custom resolver). What I would want to do is something like:

import { Parcel, getDefaultParcelConfig, getDefaultParcelResolver } from '@parcel/something';
import { Resolver } from '@parcel/plugin';

// Create a custom resolver dynamically.
function createResolver(context: any): Resolver {
  return new Resolver({
    async resolve() { /* ... */ }
  });
}

const bundler = new Parcel({
  entries: 'input.css',
  // Pass in a config object directly, no need to automatically import anything.
  config: {
    ...getDefaultParcelConfig(),
    resolvers: [ createResolver(someUseCaseSpecificContext), getDefaultParcelResolver() ],
  },
});

const bundled = await bundler.bundle();
await fs.writeFile('output.css', bundled);

This is a bit idealized, and passing through the default configuration is a bit awkward, but I think this structure would remove a lot of the coupling between Parcel and node_modules/, making the system much more flexible for use cases outside that structure.

Does an API at this level of abstraction exist in Parcel? Or is the Parcel API the closest thing?

[devongovett]

I think you're looking for something similar to #82, though perhaps rather than providing the file contents you only need to do some kind of custom mapping of the filenames? We could add a JS API for that, but I was worried that it would impact performance since we'd need to block on the JS thread for each dependency.

When you use Parcel, the bundle API from @parcel/css is not currently used. In that case, each file is processed independently with the analyzeDependencies option, and Parcel uses the resulting information to perform bundling independently. Eventually we will probably migrate parts of that over too, but as described above, there are some challenges performance-wise.

If you're willing to write Rust, you could do this today using a custom SourceProvider wrapping around FileProvider. The Bundler API accepts an arbitrary SourceProvider and uses it to perform all filesystem operations.

[dgp1130]

Ah, I didn't realize this module was also published as a crate. I'm certainly open to writing Rust here and can explore that option.

I see your concern with the performance, making a JS call for every import is definitely a hit and I understand why you would want to avoid it. My immediate thoughts are:

1. This feature shouldn't slow down anyone who isn't using a custom resolver, so you only pay for the feature if you really need it.
2. If a user really wants a custom resolver without the performance hit, they could always implement it at the Rust level like you're suggesting.
3. For my particular use case I could actually pre-compute a map of all available imports and where they actually are on the filesystem. This is the way I would implement the resolver anyways. The Rust API could just accept this map as an input and look it up in Rust to avoid the JS callback. That's kind of specific to my use case though, so I understand if that's too limiting for what you think Parcel should support.

#82 does seem very similar. My use case is still on the filesystem, just at different paths, so a resolver alone would be good enough for me. However, I understand that providing file contents directly would probably be more flexible, and I'm fine making the fs.readFile() call if necessary. I'm not sure what the performance implications of that would be though, I'm sure it's more significant than the JS resolver function call, but maybe that's just the cost of doing what I want to do here?

[dgp1130]

I took a pass at a Rust implementation and it seems to mostly work, it's definitely much easier to integrate with Bazel than using the Parcel API since I no longer need to deal with getting the configuration file and resolver to load. The two pieces of feedback I have from this experiment are:

1. The parcel_css crate is still in alpha (1.0.0-alpha.24) as of now. Are there plans to get to a 1.0.0 release? Is this crate considered public API of Parcel?
2. SourceProvider is designed to accept only a single path, and the bundler actually joins the current path with the imported path, assuming that they are typical file paths where that makes sense. In Bazel web tooling (rules_nodejs), absolute imports are typically written as workspace_name/path/to/foo with the workspace_name/ prefix signifying an absolute path (relative paths would start with ./). This scheme doesn't work with the bundler as is today, for example:

/* my_workspace/foo/bar.css */

/* Absolute import of `test.css` from the project root. */
@import 'my_workspace/test.css';

This results in a call to the SourceProvider with the path: my_workspace/foo/my_workspace/test.css. I get that the conventions are a little weird in this regard, but it's supposed to align with a Node resolve, since my_workspace/... wouldn't be a relative import in Node, but instead look for the my_workspace NPM package, which Bazel resolves to the whole workspace.

I think the issue here is that the bundler is assuming that is ok to join the import specifier with the current file path, when that may not be true or have the intended effect. Ideally SourceProvider would have two different fields, one for the file making the import, and another for the raw import specifier.

pub trait SourceProvider: Send + Sync {
    fn read<'a>(&'a self, import_specifier: &str, imported_from_file: &Path) -> Result<&'a str>;
}

This would also allow plugins to treat specifiers as non-file paths. For example, if someone wanted to implement @import 'assets!./foo.css', they would have the freedom to parse and interpret the specifier in whatever way makes sense for their use case.

Personally I would still love to see a JS binding for path resolution, though Rust is good enough as a workaround. Sorry that this issue is covering a lot of different topics, just trying to share my experience with configuring Parcel for an uncommon use case and the challenges I encounter. Hopefully this is useful to see where it can be improved. I'm happy to take a stab at implementing some of these suggestions if you think they are the right way to go here.

[devongovett]

The parcel_css crate is still in alpha (1.0.0-alpha.24) as of now. Are there plans to get to a 1.0.0 release? Is this crate considered public API of Parcel?

The Parcel CSS Rust API has a massive surface area because it exposes structs for each individual CSS rule/property/value. The alpha status indicates that these structs are still changing as we work on improvements/features. However, the quality of the parsing/compiling/minifying itself (the parts exposed to JS) is considered stable, and it should be safe to use top-level APIs like StyleSheet and Bundler.

Ideally SourceProvider would have two different fields

I think probably we'd have a separate method on the SourceProvider trait to resolve the path since read is called for both entries and @import rules, whereas resolve would only need to be called for @import. We could have a default implementation that does source_path.with_file_name(specifier), as we do today.

Personally I would still love to see a JS binding for path resolution

Happy to also consider that, but getting the Rust implementation going first would be a good start.