Skip to content

fix: stream artifact downloads to disk to avoid OOM on large artifacts#397

Open
yeicor wants to merge 5 commits into
dawidd6:masterfrom
yeicor:copilot/fix-artifact-download-memory-issue
Open

fix: stream artifact downloads to disk to avoid OOM on large artifacts#397
yeicor wants to merge 5 commits into
dawidd6:masterfrom
yeicor:copilot/fix-artifact-download-memory-issue

Conversation

@yeicor

@yeicor yeicor commented Apr 15, 2026

Copy link
Copy Markdown

In main.js, every artifact ZIP is currently downloaded by calling Octokit's downloadArtifact, which follows the redirect and fetches the entire binary response body into a zip.data ArrayBuffer that lives in Node.js heap memory. That Buffer.from(zip.data) is then passed either to AdmZip (which also keeps everything in RAM) or written to a temp file for unzip. For large artifacts this can easily exhaust the runner's memory and kill the process with an OOM error.

This PR fixes this issue by streaming the downloads to disk.

Copilot AI review requested due to automatic review settings April 15, 2026 15:36

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Streams GitHub Actions artifact ZIP downloads directly to disk to avoid OOM when processing large artifacts.

Changes:

  • Added downloadArtifactToFile() to stream ZIP responses to a file via a custom Octokit fetch implementation.
  • Updated artifact download/extraction flow to use temp ZIP files and clean them up afterward.
  • Adjusted error handling for expired artifacts and extraction paths.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread main.js Outdated
Comment thread main.js
Comment thread main.js
@yeicor

yeicor commented Apr 15, 2026

Copy link
Copy Markdown
Author

CI passes on my fork: https://github.com/yeicor/action-download-artifact/actions

Fixes #201

@dawidd6

dawidd6 commented Apr 28, 2026

Copy link
Copy Markdown
Owner

Could you please address the copilot review comments?

@yeicor

yeicor commented Apr 29, 2026

Copy link
Copy Markdown
Author

All Copilot comments addressed with minimal changes. CI passes: https://github.com/yeicor/action-download-artifact/actions/runs/25122241313

  • Log group closure fixed using inner try/finally to always call core.endGroup().
  • Redirect explicitly set to "follow" to avoid relying on defaults.
  • Runtime already satisfies Node 18+ via node24 (added a comment about this).

Note: AdmZip still loads full ZIP into memory; use_unzip: true is required for streaming extraction (maybe this should be documented somewhere or be the new default).

@dawidd6

dawidd6 commented Apr 30, 2026

Copy link
Copy Markdown
Owner

A bit too much changes for just a change making the artifact download streamable. Would be nice to have the download + extract logic both streamable. GitHub does so in actions/toolkit using unzip-stream package. Maybe we could do that too.

@yeicor

yeicor commented Apr 30, 2026

Copy link
Copy Markdown
Author

Thanks for the feedback. This is currently good enough for me, and I don’t really have time to keep iterating on it. Happy for you (or anyone) to take this branch and build on it, though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants