Commits

Branch selector

 master

User selector

All users

All time

Commit History

Commits on Oct 31, 2023

• Merge pull request #33 from commoncrawl/reduce-log-level-to-avoid-superflous-logging

  Show description for d75823f
  jnioche

  authored
  d75823f

  Copy full SHA for d75823f

• Merge pull request #34 from commoncrawl/hadoop-recent

  Show description for 98bedc1
  jnioche

  authored
  98bedc1

  Copy full SHA for 98bedc1

Commits on Oct 8, 2023

• Upgrade to recent Hadoop (3.3.5) and further dependency upgrades

  Show description for a06199f
  sebastian-nagel

  committed
  a06199f

  Copy full SHA for a06199f

• Reduce log level of two classes called by the WAT/WET extractor

  Show description for 9b81ced
  sebastian-nagel

  committed
  9b81ced

  Copy full SHA for 9b81ced

Commits on Oct 6, 2022

• Improve anchor text extraction

  Show description for b2be0a5
  sebastian-nagel

  committed
  b2be0a5

  Copy full SHA for b2be0a5

• Improved text and anchor text extraction

  Show description for 91e7052
  sebastian-nagel

  committed
  91e7052

  Copy full SHA for 91e7052

Commits on Oct 5, 2022

• Upgrade to JDK / Java 8

  sebastian-nagel

  committed
  aa03434

  Copy full SHA for aa03434

• Use WARC header WARC-Identified-Payload-Type (if available) to identify

  Show description for b42ecb9
  sebastian-nagel

  committed
  b42ecb9

  Copy full SHA for b42ecb9

• Upgrade dependencies

  sebastian-nagel

  committed
  c4873b3

  Copy full SHA for c4873b3

Commits on Mar 16, 2021

• Bump junit from 3.8.1 to 4.13.1

  Show description for d63faaf
  

  dependabot[bot]

  authored and

  sebastian-nagel

  committed
  d63faaf

  Copy full SHA for d63faaf

• Fix InterruptibleCharSequenceTest (testInterruptibility), fixes #25

  Show description for 8accf19
  sebastian-nagel

  committed
  8accf19

  Copy full SHA for 8accf19

Commits on Jun 15, 2020

• Merge pull request #89 from sebastian-nagel/webarchive-commons-88,

  Show description for 1115246
  

  ldko

  authored and

  sebastian-nagel

  committed
  1115246

  Copy full SHA for 1115246

• Update change log to include #85, #86 and #89

  sebastian-nagel

  committed
  c0381bc

  Copy full SHA for c0381bc

• WAT extractor: do not fail on missing WARC-Filename in warcinfo record, fixes #88

  Show description for d2c7379
  sebastian-nagel

  committed
  d2c7379

  Copy full SHA for d2c7379

Commits on Jun 5, 2020

• WET extractor: add identified natural language of text content (#22)

  Show description for a1f07ad
  sebastian-nagel

  committed
  a1f07ad

  Copy full SHA for a1f07ad

• - simplify date parsing resp. do not parse if not needed

  Show description for 37b40f5
  sebastian-nagel

  committed
  37b40f5

  Copy full SHA for 37b40f5

Commits on Dec 12, 2019

• Work-around issue processing WARC request records with invalid message

  Show description for 428022b
  sebastian-nagel

  committed
  428022b

  Copy full SHA for 428022b

Commits on Oct 22, 2019

• Upgrade dependencies

  sebastian-nagel

  committed
  00109ec

  Copy full SHA for 00109ec

• Merge remote-tracking branch 'iipc/master'

  sebastian-nagel

  committed
  95c7170

  Copy full SHA for 95c7170

Commits on Aug 28, 2019

• Remove pom-cdh5.xml as changes required to run under Cloudera Hadoop

  Show description for 248fb98
  sebastian-nagel

  committed
  248fb98

  Copy full SHA for 248fb98

• WAT: only unescape complete XML/HTML character entities (fixes #19)

  Show description for bb754a3
  sebastian-nagel

  committed
  bb754a3

  Copy full SHA for bb754a3

Commits on Aug 27, 2019

• WAT: only unescape complete XML/HTML character entities (fixes #19)

  Show description for 716eae9
  sebastian-nagel

  committed
  716eae9

  Copy full SHA for 716eae9

Commits on Aug 19, 2019

• WAT: only unescape complete XML/HTML character entities (fixes #19)

  Show description for 60b6f7e
  sebastian-nagel

  committed
  60b6f7e

  Copy full SHA for 60b6f7e

Commits on Aug 13, 2019

• Merge pull request #83 from ruebot/issue-82

  Show description for 336a49e
  ldko

  authored
  336a49e

  Copy full SHA for 336a49e

Commits on Jul 30, 2019

• Merge pull request #17 from commoncrawl/dunning-org-json

  Show description for f453915
  sebastian-nagel

  authored
  f453915

  Copy full SHA for f453915

Commits on Jul 23, 2019

• Use openjson as replacement for the org.json dependency

  Show description for d188b9e
  sebastian-nagel

  committed
  d188b9e

  Copy full SHA for d188b9e

Commits on Jul 22, 2019

• Replace the org.json dependency by Ted Dunning's open-json library

  Show description for 470c353
  sebastian-nagel

  committed
  470c353

  Copy full SHA for 470c353

Commits on Jul 8, 2019

• Merge branch 'cc-14-wat-decode-html-char-entities', fixes #14

  sebastian-nagel

  committed
  c7e79be

  Copy full SHA for c7e79be

• Add unit test for HTML entity decoding

  Show description for fb514ab
  sebastian-nagel

  committed
  fb514ab

  Copy full SHA for fb514ab

• WAT/WET extraction: use compiled regex Pattern to get better performance

  sebastian-nagel

  committed
  4a1f323

  Copy full SHA for 4a1f323

• WAT: unescape XML/HTML character entities (#14)

  Show description for d5af4e5
  sebastian-nagel

  committed
  d5af4e5

  Copy full SHA for d5af4e5

Commits on May 7, 2019

• Update TravisCI config; resolves #82.

  Show description for 723b18a
  ruebot

  committed
  723b18a

  Copy full SHA for 723b18a

• [maven-release-plugin] prepare for next development iteration

  ruebot

  committed
  da029db

  Copy full SHA for da029db

• [maven-release-plugin] prepare release webarchive-commons-1.1.9

  ruebot

  committed
  2e8cdea

  Copy full SHA for 2e8cdea

Commits on Aug 23, 2017

• upgrade to CDH 5.12.0

  sebastian-nagel

  committed
  538ec1d

  Copy full SHA for 538ec1d