Commits

Breadcrumbs

History for

onhadoop-2

User selector

All users

All time

Commit History

Commits on Feb 21, 2014

• merge from iipc/master

  nlevitt

  committed
  5e71d11

  Copy full SHA for 5e71d11

Commits on Jan 25, 2014

• get rid of RecyclingFastBufferedOutputStream, which was supposed to avoid allocating a buffer every time it was allocated, but didn't because FastBufferedOutputStream always goes and allocates its …

  Show description for dcb68a2
  nlevitt

  committed
  dcb68a2

  Copy full SHA for dcb68a2

Commits on Jan 11, 2014

• fix off-by-one mistake with integer arithmetic so RecordingInputStreamTest.testReadFullyOrUntil() test passes (how did this ever pass?)

  nlevitt

  committed
  7c6c673

  Copy full SHA for 7c6c673

• updates for heritrix FetchHTTP using httpcomponents 4.3

  nlevitt

  committed
  acec0d4

  Copy full SHA for acec0d4

Commits on Dec 11, 2013

• Removed older PublicSuffix code.

  anjackson

  committed
  a732a9e

  Copy full SHA for a732a9e

Commits on Dec 7, 2013

• fix HER-2059 with commons-httpclient workaround

  nlevitt

  committed
  0963f2e

  Copy full SHA for 0963f2e

Commits on Dec 6, 2013

• Merge branch 'master' of github.com:internetarchive/ia-web-commons

  Show description for 908c7a1
  anjackson

  committed
  908c7a1

  Copy full SHA for 908c7a1

• moving a bunch of stuff from heritrix-commons to ia-web-commons so that wayback doesn't have to depend on heritrix-commons

  nlevitt

  committed
  a54dd8e

  Copy full SHA for a54dd8e

• moving org.archive.net.PublicSuffixes to ia-web-commons

  nlevitt

  committed
  fc24be8

  Copy full SHA for fc24be8

Commits on Nov 23, 2013

• Extract outlinks/hopinfo from warc/metadata records

  vinaygoel

  committed
  5077ad8

  Copy full SHA for 5077ad8

Commits on Nov 16, 2013

• FIX: add cdxlinefactory doesn't require custom format

  Show description for 0ebbad2
  ikreymer

  committed
  0ebbad2

  Copy full SHA for 0ebbad2

Commits on Nov 4, 2013

• canonicalizer: add ExtractRule and RewriteRule

  ikreymer

  committed
  1e8c1af

  Copy full SHA for 1e8c1af

Commits on Nov 2, 2013

• apach43SLR: better reading of entire buffer

  ikreymer

  committed
  54bdd7d

  Copy full SHA for 54bdd7d

• FEATURE: add support for httpcore 4.3 for SeekableLineReader!

  ikreymer

  committed
  68dab84

  Copy full SHA for 68dab84

Commits on Nov 1, 2013

• move ThreadLocalHttpConnectionManager to original package org.archive.httpclient

  ikreymer

  committed
  da37977

  Copy full SHA for da37977

• fix package

  ikreymer

  committed
  1c58cba

  Copy full SHA for 1c58cba

• fix path for ThreadLocalHttpConnectionManager

  ikreymer

  committed
  cd43101

  Copy full SHA for cd43101

• add ThreadLocalHttpConnectionManager (from heritrix-commons)

  Show description for 154386c
  ikreymer

  committed
  154386c

  Copy full SHA for 154386c

Commits on Oct 24, 2013

• log RuntimeIOException

  ikreymer

  committed
  baf1ad8

  Copy full SHA for baf1ad8

Commits on Oct 23, 2013

• FEATURE: add getTotalCount() to cdx input sources

  ikreymer

  committed
  2d2d30e

  Copy full SHA for 2d2d30e

Commits on Oct 18, 2013

• FIX: MultiCDXInputSource: add optimization for output lazy initing of sequential cluster load

  ikreymer

  committed
  7d4fbf4

  Copy full SHA for 7d4fbf4

• MultiCDXInputSource: Make comparator public

  ikreymer

  committed
  647adec

  Copy full SHA for 647adec

Commits on Oct 17, 2013

• HttpSLR: add optional error header which can be saved

  Show description for f82aead
  ikreymer

  committed
  f82aead

  Copy full SHA for f82aead

• FIX: ApacheSLR: turn off cookies when using manual cookie

  ikreymer

  committed
  0474441

  Copy full SHA for 0474441

Commits on Oct 15, 2013

• FIX: ApacheHttp31SLR save the connected url even on error!

  Show description for 936fb49
  ikreymer

  committed
  936fb49

  Copy full SHA for 936fb49

Commits on Oct 14, 2013

• ZIPNUMLoader: attempt better error msgs by propagating full error in exception

  ikreymer

  committed
  1fc6019

  Copy full SHA for 1fc6019

• ZipNumBlock Loader: * add info on which shard failed to RuntimIOException

  Show description for c5dbc67
  ikreymer

  committed
  c5dbc67

  Copy full SHA for c5dbc67

• ADD: CloseableIteratorWrapper utility class for wrapping regular iterators

  Show description for 612cdeb
  ikreymer

  committed
  612cdeb

  Copy full SHA for 612cdeb

Commits on Oct 10, 2013

• Copied PublicSuffixes code into web-commons from heritrix-commons.

  anjackson

  committed
  7f70542

  Copy full SHA for 7f70542

• Merge branch 'master' of github.com:iipc/iipc-web-commons

  anjackson

  committed
  d03fe77

  Copy full SHA for d03fe77

• Initial build under org.netpreserve

  Show description for 299281f
  anjackson

  committed
  299281f

  Copy full SHA for 299281f

Commits on Oct 6, 2013

• ZIP & SLR improvements: store connectedUrl immediately on connection, print connectedUrl on error, skip cached url on fallback to location list

  ikreymer

  committed
  b4f639d

  Copy full SHA for b4f639d

Commits on Oct 4, 2013

• CDX: Add CloseableCompositeIterator which iterates in sequence, optimization for zipnum clusters to be loaded sequentially when only looking for last line

  ikreymer

  committed
  41d3a58

  Copy full SHA for 41d3a58

Commits on Oct 3, 2013

• ZipNum: fixes: cached locations: don't throw RunTimeException if required (handled later), add a locCacheMaxDuration to cache only if below threshold

  ikreymer

  committed
  28c2995

  Copy full SHA for 28c2995

Commits on Oct 2, 2013

• util fix: makeRangeHeader made static, ApacheHTTP31SlrFactory returns more specific type

  ikreymer

  committed
  53dd536

  Copy full SHA for 53dd536