Commits

Breadcrumbs

History for

ontest-openjson

User selector

All users

All time

Commit History

Commits on Jul 23, 2019

• Use openjson as replacement for the org.json dependency

  Show description for ed59bf0
  sebastian-nagel

  committed
  ed59bf0

  Copy full SHA for ed59bf0

• Replace the org.json dependency by Ted Dunning's open-json library

  Show description for fea3c86
  sebastian-nagel

  committed
  fea3c86

  Copy full SHA for fea3c86

Commits on Jul 8, 2019

• Add unit test for HTML entity decoding

  Show description for f559b79
  sebastian-nagel

  committed
  f559b79

  Copy full SHA for f559b79

Commits on Jun 27, 2019

• WAT/WET extraction: use compiled regex Pattern to get better performance

  sebastian-nagel

  committed
  3abab54

  Copy full SHA for 3abab54

Commits on Jun 19, 2019

• WAT: unescape XML/HTML character entities (#14)

  Show description for e0d23b8
  sebastian-nagel

  committed
  e0d23b8

  Copy full SHA for e0d23b8

Commits on Aug 23, 2017

• WAT extractor: get links from onClick attributes, fixes #8

  Show description for 442670b
  sebastian-nagel

  committed
  442670b

  Copy full SHA for 442670b

• WAT extracor: add rel attribute to A@/href links, fixes #10

  Show description for 7efb0c7
  sebastian-nagel

  committed
  7efb0c7

  Copy full SHA for 7efb0c7

• WET extractor: improve spacing of textual content, fixes #13

  Show description for 6031308
  sebastian-nagel

  committed
  6031308

  Copy full SHA for 6031308

Commits on Jun 5, 2017

• Limit pattern matching URLs embedded in CSS to match max. 8000 characters,

  Show description for 871fde8
  sebastian-nagel

  committed
  871fde8

  Copy full SHA for 871fde8

Commits on May 4, 2017

• Roll-back to Guava 14.0.1 to be compatible with CDH 5.11.0 which uses Guava 11.0.2

  sebastian-nagel

  committed
  65ab2ad

  Copy full SHA for 65ab2ad

• Fix HTTP-Response-Metadata for wget WARCs. Changes came from https://github.com/commoncrawl/ia-web-commons/commit/58e85a60d75707da55ed499f836e57d49347484a

  

  ldko

  authored and

  sebastian-nagel

  committed
  3fac189

  Copy full SHA for 3fac189

Commits on Apr 28, 2017

• Do not add value of preceding HTTP header field if there is no value (or

  Show description for aad4e05
  sebastian-nagel

  committed
  aad4e05

  Copy full SHA for aad4e05

• Merge with recent master of iipc/webarchive-commons, transfer all

  Show description for 58e85a6
  sebastian-nagel

  committed
  58e85a6

  Copy full SHA for 58e85a6

Commits on Mar 21, 2017

• Merge branch 'wat-improved-link-extraction' of https://github.com/sebastian-nagel/webarchive-commons into sebastian-nagel-wat-improved-link-extraction

  ldko

  committed
  f12e86f

  Copy full SHA for f12e86f

Commits on Mar 2, 2017

• Fix #25: move missing unit tests over from Heritrix3

  MohammedElsayyed

  committed
  6aa43f8

  Copy full SHA for 6aa43f8

Commits on Feb 22, 2017

• Improve HTML link extraction

  Show description for 11579c2
  sebastian-nagel

  committed
  11579c2

  Copy full SHA for 11579c2

Commits on Feb 15, 2017

• Merge pull request #69 from sebastian-nagel/cc-url-parser-ignore-empty-port

  Show description for cb91063
  ldko

  authored
  cb91063

  Copy full SHA for cb91063

• Merge pull request #68 from sebastian-nagel/HTMLResourceFactory_to_use_CharsetDetector

  Show description for ed33bae
  ldko

  authored
  ed33bae

  Copy full SHA for ed33bae

• Merge pull request #65 from dportabella/fix

  Show description for 1ee2529
  ldko

  authored
  1ee2529

  Copy full SHA for 1ee2529

• Merge pull request #62 from iipc/kris-sigur-patch-1

  Show description for c5876b3
  ldko

  authored
  c5876b3

  Copy full SHA for c5876b3

Commits on Jan 27, 2017

• CharsetDetector: remove unnecessary check for empty string (contributed by @ldko)

  sebastian-nagel

  committed
  0384028

  Copy full SHA for 0384028

Commits on Jan 18, 2017

• CSS link extraction: clip also unpaired leading and trailing quotation marks

  sebastian-nagel

  committed
  194a1fa

  Copy full SHA for 194a1fa

Commits on Jan 4, 2017

• Improve clipping of quotation marks in CSS link extraction

  Show description for b918f7f
  sebastian-nagel

  committed
  b918f7f

  Copy full SHA for b918f7f

Commits on Dec 9, 2016

• Strip empty port, do not fail

  sebastian-nagel

  committed
  9e41abc

  Copy full SHA for 9e41abc

Commits on Nov 24, 2016

• Match http-equiv meta elements with unquoted attribute values, e.g.

  Show description for 824dd82
  sebastian-nagel

  committed
  824dd82

  Copy full SHA for 824dd82

• HTML encoding detection: fix errors with empty content or empty charset values

  sebastian-nagel

  committed
  607acaa

  Copy full SHA for 607acaa

• Use CharsetDetector to guess encoding of HTML document

  sebastian-nagel

  committed
  5f223d6

  Copy full SHA for 5f223d6

Commits on Sep 21, 2016

• fix: last header was lost if LF LF (intead of CRLF CRLF)

  dportabella

  committed
  9d7abed

  Copy full SHA for 9d7abed

Commits on Aug 7, 2016

• Make regular expression to extract URLs from CSS more restrictive

  Show description for a23cfeb
  sebastian-nagel

  committed
  a23cfeb

  Copy full SHA for a23cfeb

Commits on Jul 13, 2016

• Remove invalid constant

  Show description for 0cbca57
  

  Kristinn Sigurðsson

  authored
  0cbca57

  Copy full SHA for 0cbca57

Commits on May 26, 2016

• Merge pull request #54 from vonrosen/modify-strip-session-params

  Show description for 1485fdd
  

  Kristinn Sigurðsson

  committed
  1485fdd

  Copy full SHA for 1485fdd

• Merge pull request #50 from nlevitt/her-2089

  Show description for cf5f600
  

  Kristinn Sigurðsson

  committed
  cf5f600

  Copy full SHA for cf5f600

• Merge pull request #51 from nlevitt/fix-warc-sizeOnDisk

  Show description for f5e6541
  

  Kristinn Sigurðsson

  committed
  f5e6541

  Copy full SHA for f5e6541

Commits on Mar 22, 2016

• Make canonicalizer be able to strip session id params even if they are the first params in the query string. And add session id strip test. And change IAURLCanonicalizer.java to ensure that if afte…

  Show description for 5cfff50
  

  Hunter Stern

  committed
  5cfff50

  Copy full SHA for 5cfff50

Commits on Mar 11, 2016

• Update ArchiveFileConstants.java

  jrwiebe

  committed
  28c9a1b

  Copy full SHA for 28c9a1b