Commits

Branch selector

 upgrade-java-8

User selector

All users

All time

Commit History

Commits on Mar 12, 2026

• Integrate GitHub CI build

  sebastian-nagel

  committed
  cf16197

  Copy full SHA for cf16197

• Format pom.xml

  sebastian-nagel

  committed
  3e350c5

  Copy full SHA for 3e350c5

• Apply spotless code formatting rules

  sebastian-nagel

  committed
  fe0ad25

  Copy full SHA for fe0ad25

• Integrate spotless code formatter

  sebastian-nagel

  committed
  7700597

  Copy full SHA for 7700597

• Fix broken logo link

  sebastian-nagel

  committed
  6b219a0

  Copy full SHA for 6b219a0

• Require minimum Java version 8

  Show description for 008b3ec
  sebastian-nagel

  committed
  008b3ec

  Copy full SHA for 008b3ec

Commits on Dec 17, 2024

• Merge pull request #5 from commoncrawl/wat-multivalue-headers

  Show description for 5aea741
  sebastian-nagel

  authored
  5aea741

  Copy full SHA for 5aea741

Commits on Dec 13, 2024

• Adapt WATSampleOutlinks job to read multi-valued headers in WAT data

  Show description for 20eca18
  sebastian-nagel

  committed
  20eca18

  Copy full SHA for 20eca18

• Adapt WATServerType example to read multi-valued headers in WAT data

  Show description for 66bc8bf
  sebastian-nagel

  committed
  66bc8bf

  Copy full SHA for 66bc8bf

• Improve TagCounterMap

  Show description for 4b269de
  sebastian-nagel

  committed
  4b269de

  Copy full SHA for 4b269de

• Modernize examples

  Show description for 30dd7a7
  sebastian-nagel

  committed
  30dd7a7

  Copy full SHA for 30dd7a7

• Upgrade dependencies

  Show description for 4204cbe
  sebastian-nagel

  committed
  4204cbe

  Copy full SHA for 4204cbe

Commits on Jul 9, 2024

• fix(WATSampleOutLinks): use Text.getBytes() following API docs

  Show description for fabf5ca
  sebastian-nagel

  committed
  fabf5ca

  Copy full SHA for fabf5ca

Commits on May 24, 2023

• Allow to specify a bot name when search for "nofollow" HTML metadata

  Show description for 0eedb4a
  sebastian-nagel

  committed
  0eedb4a

  Copy full SHA for 0eedb4a

• Allow to specify a bot name when search for "nofollow" HTML metadata

  Show description for e7bfb1c
  sebastian-nagel

  committed
  e7bfb1c

  Copy full SHA for e7bfb1c

Commits on Jan 6, 2023

• Upgrade dependencies

  Show description for c64d3c8
  sebastian-nagel

  committed
  c64d3c8

  Copy full SHA for c64d3c8

• Use "nofollow" rel attributes and HTML/HTTP metadata when

  Show description for 33e0bbd
  sebastian-nagel

  committed
  33e0bbd

  Copy full SHA for 33e0bbd

Commits on Apr 6, 2022

• Implement new methods to access data on s3://commoncrawl/

  Show description for dbd6670
  sebastian-nagel

  committed
  dbd6670

  Copy full SHA for dbd6670

Commits on Sep 24, 2021

• WATSampleOutlinks:

  Show description for 04a8c43
  sebastian-nagel

  committed
  04a8c43

  Copy full SHA for 04a8c43

Commits on May 21, 2021

• Upgrade dependencies

  Show description for bf87278
  sebastian-nagel

  committed
  bf87278

  Copy full SHA for bf87278

• WATSampleOutlinks:

  Show description for a7992c5
  sebastian-nagel

  committed
  a7992c5

  Copy full SHA for a7992c5

Commits on May 20, 2021

• Remove obsolete jar (webarchive-commons dependency managed by Maven)

  sebastian-nagel

  committed
  ba959cd

  Copy full SHA for ba959cd

Commits on May 3, 2021

• WATSampleOutlinks:

  Show description for 17bfa39
  sebastian-nagel

  committed
  17bfa39

  Copy full SHA for 17bfa39

Commits on Jan 21, 2020

• WATSampleOutlinks:

  Show description for 8badc44
  sebastian-nagel

  committed
  8badc44

  Copy full SHA for 8badc44

Commits on Apr 9, 2019

• WATSampleOutlinks:

  Show description for 8e6a55a
  sebastian-nagel

  committed
  8e6a55a

  Copy full SHA for 8e6a55a

Commits on Mar 28, 2018

• WATSampleOutlinks:

  Show description for 7f87e6a
  sebastian-nagel

  committed
  7f87e6a

  Copy full SHA for 7f87e6a

Commits on Nov 14, 2017

• WATSampleOutlinks:

  Show description for 14aa2d8
  sebastian-nagel

  committed
  14aa2d8

  Copy full SHA for 14aa2d8

Commits on Oct 14, 2017

• Add tool to extract and sample outlinks from WAT files

  sebastian-nagel

  committed
  cb355fa

  Copy full SHA for cb355fa

Commits on Oct 9, 2017

• Update libraries (Hadoop and webarchive-commons)

  sebastian-nagel

  committed
  f1747e9

  Copy full SHA for f1747e9

Commits on Jul 28, 2017

• Update Common Crawl logo location

  sebastian-nagel

  committed
  e220036

  Copy full SHA for e220036

Commits on Feb 23, 2017

• Merge branch 'tfmorris-hadoop2'

  sebastian-nagel

  committed
  14f298d

  Copy full SHA for 14f298d

• Update Github links & Java version

  

  tfmorris

  authored and

  sebastian-nagel

  committed
  55743c8

  Copy full SHA for 55743c8

• Update to Hadoop 2

  Show description for b21b550
  

  tfmorris

  authored and

  sebastian-nagel

  committed
  b21b550

  Copy full SHA for b21b550

• Update paths to new Public Dataset bucket s3://commoncrawl/

  sebastian-nagel

  committed
  f79b7cf

  Copy full SHA for f79b7cf

• Merge branch 'tfmorris-gitignore'

  sebastian-nagel

  committed
  65c42a5

  Copy full SHA for 65c42a5