Skip to content

System build automation notes

Christopher Joel edited this page Jul 18, 2024 · 1 revision

These are some notes about our Github Rust Workflow in system. Some topical background can be found in the RFCs in this repo.

Topology

  • We run lints before we run any actual tests
  • We have a step that pre-builds Wasm artifacts
  • We have a matrix for running actual tests
    • OS(macos,ubuntu) x Rust(stable, nightly)
  • We have an "analysis" step at the end of a successful run
    • Takes Cargo test results and reports any flakey tests (N/5 runs failed before one succeeded) to an associated PR

Multi-target artifact dependencies

  • A lot of what we are building is tooling that produces and/or consumes Web Assembly (aka Wasm) artifacts.
  • For producing artifacts, we use the wasm32-wasip1 Cargo target
  • For consuming artifacts, we (mostly) use whatever the native host Cargo target is
  • Therefor, some packages depend on the artifact of other packages in the same workspace but built across different toolchains
    • This is tricky for Cargo to do, because it doesn't support multiple targets in a single build and it takes a lock on the build directory, so it cannot re-invoke itself from within its own build
  • We work around the cargo->cargo problem using Docker: cargo invokes a Docker builder, which internally uses cargo and produces an artifact that is plucked out by the build script
  • The cost of this is worked-around in CI but pre-building Wasm artifacts in a separate job and putting them in a well-known location during test builds
Clone this wiki locally