Commits

Branch selector

 master

User selector

All users

All time

Commit History

Commits on Jun 26, 2014

• Remove the code that returns an easier version if there was a parse error since it was caused by a s3 problem.

  Aloisius

  committed
  7eddd64

  Copy full SHA for 7eddd64

• If there is an error parsing a resource, return the version one level higher (ie, http response instead of html response)

  Aloisius

  committed
  edc3820

  Copy full SHA for edc3820

Commits on Mar 20, 2014

• htmlparser bump to 2.1 to fix some serious bugs with jsp parsing

  Aloisius

  committed
  04d556c

  Copy full SHA for 04d556c

Commits on Feb 5, 2014

• Hack to limit CSS size to 100K.

  Aloisius

  committed
  347e6f1

  Copy full SHA for 347e6f1

Commits on Nov 26, 2013

• Write out WET files (warc encapsulated text) using the same parser that we have already run

  Aloisius

  committed
  dadf551

  Copy full SHA for dadf551

Commits on Nov 25, 2013

• reference pom.xml for building with CDH4

  vinaygoel

  committed
  329ff22

  Copy full SHA for 329ff22

Commits on Nov 23, 2013

• Extract outlinks/hopinfo from warc/metadata records

  vinaygoel

  committed
  5077ad8

  Copy full SHA for 5077ad8

Commits on Nov 16, 2013

• FIX: add cdxlinefactory doesn't require custom format

  Show description for 0ebbad2
  ikreymer

  committed
  0ebbad2

  Copy full SHA for 0ebbad2

Commits on Nov 4, 2013

• canonicalizer: add ExtractRule and RewriteRule

  ikreymer

  committed
  1e8c1af

  Copy full SHA for 1e8c1af

Commits on Nov 2, 2013

• apach43SLR: better reading of entire buffer

  ikreymer

  committed
  54bdd7d

  Copy full SHA for 54bdd7d

• FEATURE: add support for httpcore 4.3 for SeekableLineReader!

  ikreymer

  committed
  68dab84

  Copy full SHA for 68dab84

Commits on Nov 1, 2013

• move ThreadLocalHttpConnectionManager to original package org.archive.httpclient

  ikreymer

  committed
  da37977

  Copy full SHA for da37977

• fix package

  ikreymer

  committed
  1c58cba

  Copy full SHA for 1c58cba

• fix path for ThreadLocalHttpConnectionManager

  ikreymer

  committed
  cd43101

  Copy full SHA for cd43101

• add ThreadLocalHttpConnectionManager (from heritrix-commons)

  Show description for 154386c
  ikreymer

  committed
  154386c

  Copy full SHA for 154386c

Commits on Oct 24, 2013

• log RuntimeIOException

  ikreymer

  committed
  baf1ad8

  Copy full SHA for baf1ad8

Commits on Oct 23, 2013

• FEATURE: add getTotalCount() to cdx input sources

  ikreymer

  committed
  2d2d30e

  Copy full SHA for 2d2d30e

Commits on Oct 18, 2013

• FIX: MultiCDXInputSource: add optimization for output lazy initing of sequential cluster load

  ikreymer

  committed
  7d4fbf4

  Copy full SHA for 7d4fbf4

• MultiCDXInputSource: Make comparator public

  ikreymer

  committed
  647adec

  Copy full SHA for 647adec

Commits on Oct 17, 2013

• HttpSLR: add optional error header which can be saved

  Show description for f82aead
  ikreymer

  committed
  f82aead

  Copy full SHA for f82aead

• FIX: ApacheSLR: turn off cookies when using manual cookie

  ikreymer

  committed
  0474441

  Copy full SHA for 0474441

Commits on Oct 15, 2013

• FIX: ApacheHttp31SLR save the connected url even on error!

  Show description for 936fb49
  ikreymer

  committed
  936fb49

  Copy full SHA for 936fb49

Commits on Oct 14, 2013

• ZIPNUMLoader: attempt better error msgs by propagating full error in exception

  ikreymer

  committed
  1fc6019

  Copy full SHA for 1fc6019

• ZipNumBlock Loader: * add info on which shard failed to RuntimIOException

  Show description for c5dbc67
  ikreymer

  committed
  c5dbc67

  Copy full SHA for c5dbc67

• ADD: CloseableIteratorWrapper utility class for wrapping regular iterators

  Show description for 612cdeb
  ikreymer

  committed
  612cdeb

  Copy full SHA for 612cdeb

Commits on Oct 6, 2013

• ZIP & SLR improvements: store connectedUrl immediately on connection, print connectedUrl on error, skip cached url on fallback to location list

  ikreymer

  committed
  b4f639d

  Copy full SHA for b4f639d

Commits on Oct 4, 2013

• CDX: Add CloseableCompositeIterator which iterates in sequence, optimization for zipnum clusters to be loaded sequentially when only looking for last line

  ikreymer

  committed
  41d3a58

  Copy full SHA for 41d3a58

Commits on Oct 3, 2013

• ZipNum: fixes: cached locations: don't throw RunTimeException if required (handled later), add a locCacheMaxDuration to cache only if below threshold

  ikreymer

  committed
  28c2995

  Copy full SHA for 28c2995

Commits on Oct 2, 2013

• util fix: makeRangeHeader made static, ApacheHTTP31SlrFactory returns more specific type

  ikreymer

  committed
  53dd536

  Copy full SHA for 53dd536

Commits on Sep 26, 2013

• FIX: typo in ZipNumCluster

  ikreymer

  committed
  5643f1b

  Copy full SHA for 5643f1b

Commits on Sep 25, 2013

• zipnum fix: fix pagination for zipnum page size calculation

  ikreymer

  committed
  58b344c

  Copy full SHA for 58b344c

Commits on Sep 19, 2013

• CDX: Add setStatusCode method

  ikreymer

  committed
  70ce244

  Copy full SHA for 70ce244

Commits on Sep 5, 2013

• FIX: reset StartBoundedStringIterator to not perform prefix check, instead add endKey to ZipNumIndex reverse query

  Show description for d7e407b
  ikreymer

  committed
  d7e407b

  Copy full SHA for d7e407b

• FIX: start boundary condition for reverse lookup

  ikreymer

  committed
  5570a4e

  Copy full SHA for 5570a4e

Commits on Sep 4, 2013

• ArchiveUtils: Add a parser for ISO 14 digit timestamp

  ikreymer

  committed
  bd59f3b

  Copy full SHA for bd59f3b