Skip to content

Commit 32bfa55

Browse files
committed
Attempt to estimate the minimum required buffer length when initializing StreamsSequenceStream instances
For most other `DecodeStream` based streams, we'll attempt to estimate the minimum `buffer` length based on the raw stream data. The purpose of this is to avoid having to unnecessarily re-size the `buffer`, thus reducing the number of *intermediate* allocations necessary when decoding the stream data. However, currently no such optimization is attempted for `StreamsSequenceStream`, and given that they can often be quite large that seems unfortunate. To improve this, at least somewhat, this patch utilizes the raw sizes of the `StreamsSequenceStream` sub-streams to estimate the minimum required `buffer` length. Most likely this patch won't have a huge effect on memory consumption, however for pathological cases it should help reduce peak memory usage slightly. One example is the PDF file in issue 2813, where currently the `StreamsSequenceStream` instances would grow their `buffer`s as `2 MiB -> 4 MiB -> 8 MiB -> 16 MiB -> 32 MiB`. With this patch, the same stream `buffers`s grow as `8 MiB -> 16 MiB -> 32 MiB`, thus avoiding a total of `12 MiB` of *intermediate* allocations (since there's two `StreamsSequenceStream` used, for rendering/text-extraction).
1 parent 3f4c2d6 commit 32bfa55

File tree

1 file changed

+13
-1
lines changed

1 file changed

+13
-1
lines changed

src/core/stream.js

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -126,6 +126,8 @@ var DecodeStream = (function DecodeStreamClosure() {
126126
var emptyBuffer = new Uint8Array(0);
127127

128128
function DecodeStream(maybeMinBufferLength) {
129+
this._rawMinBufferLength = maybeMinBufferLength || 0;
130+
129131
this.pos = 0;
130132
this.bufferLength = 0;
131133
this.eof = false;
@@ -251,7 +253,17 @@ var DecodeStream = (function DecodeStreamClosure() {
251253
var StreamsSequenceStream = (function StreamsSequenceStreamClosure() {
252254
function StreamsSequenceStream(streams) {
253255
this.streams = streams;
254-
DecodeStream.call(this, /* maybeLength = */ null);
256+
257+
let maybeLength = 0;
258+
for (let i = 0, ii = streams.length; i < ii; i++) {
259+
const stream = streams[i];
260+
if (stream instanceof DecodeStream) {
261+
maybeLength += stream._rawMinBufferLength;
262+
} else {
263+
maybeLength += stream.length;
264+
}
265+
}
266+
DecodeStream.call(this, maybeLength);
255267
}
256268

257269
StreamsSequenceStream.prototype = Object.create(DecodeStream.prototype);

0 commit comments

Comments
 (0)