fix: stream artifact downloads to disk to avoid OOM on large artifacts#397
fix: stream artifact downloads to disk to avoid OOM on large artifacts#397yeicor wants to merge 5 commits into
Conversation
Agent-Logs-Url: https://github.com/yeicor/action-download-artifact/sessions/2d0911e4-0a25-46eb-8f15-c19ce03b1645 Co-authored-by: yeicor <4929005+yeicor@users.noreply.github.com>
…e copy Agent-Logs-Url: https://github.com/yeicor/action-download-artifact/sessions/b0b20368-9d39-4096-8933-115de55a7208 Co-authored-by: yeicor <4929005+yeicor@users.noreply.github.com>
Agent-Logs-Url: https://github.com/yeicor/action-download-artifact/sessions/58991d77-6515-4700-8ff1-421d449e566d Co-authored-by: yeicor <4929005+yeicor@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Streams GitHub Actions artifact ZIP downloads directly to disk to avoid OOM when processing large artifacts.
Changes:
- Added
downloadArtifactToFile()to stream ZIP responses to a file via a custom Octokitfetchimplementation. - Updated artifact download/extraction flow to use temp ZIP files and clean them up afterward.
- Adjusted error handling for expired artifacts and extraction paths.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
CI passes on my fork: https://github.com/yeicor/action-download-artifact/actions Fixes #201 |
|
Could you please address the copilot review comments? |
|
All Copilot comments addressed with minimal changes. CI passes: https://github.com/yeicor/action-download-artifact/actions/runs/25122241313
Note: AdmZip still loads full ZIP into memory; |
|
A bit too much changes for just a change making the artifact download streamable. Would be nice to have the download + extract logic both streamable. GitHub does so in actions/toolkit using unzip-stream package. Maybe we could do that too. |
|
Thanks for the feedback. This is currently good enough for me, and I don’t really have time to keep iterating on it. Happy for you (or anyone) to take this branch and build on it, though. |
In
main.js, every artifact ZIP is currently downloaded by calling Octokit'sdownloadArtifact, which follows the redirect and fetches the entire binary response body into azip.dataArrayBufferthat lives in Node.js heap memory. ThatBuffer.from(zip.data)is then passed either toAdmZip(which also keeps everything in RAM) or written to a temp file forunzip. For large artifacts this can easily exhaust the runner's memory and kill the process with an OOM error.This PR fixes this issue by streaming the downloads to disk.