Commits

Branch selector

 cc

User selector

All users

All time

Commit History

Commits on May 22, 2026

• Add end to end tests for the SitemapInjector (#52)

  Show description for 29c642c
  lfoppiano

  authored
  29c642c

  Copy full SHA for 29c642c

Commits on May 21, 2026

• Add unit tests for plugin scoring-adaptive,

  Show description for 4861a66
  sebastian-nagel

  committed
  4861a66

  Copy full SHA for 4861a66

Commits on May 18, 2026

• Merge pull request #55 from commoncrawl/cc-integrate-upstream-improvements

  Show description for 262516f
  sebastian-nagel

  authored
  262516f

  Copy full SHA for 262516f

Commits on May 17, 2026

• NUTCH-3177 Fetcher to report idle threads not as hung threads

  Show description for 514760b
  sebastian-nagel

  committed
  514760b

  Copy full SHA for 514760b

• NUTCH-3176 URLUtil and urlnormalizer-basic: add support for IDNA2008

  Show description for 98704c5
  sebastian-nagel

  committed
  98704c5

  Copy full SHA for 98704c5

Commits on May 15, 2026

• Integrate NUTCH-3175 into build of CCF fork

  sebastian-nagel

  committed
  5cc412c

  Copy full SHA for 5cc412c

• NUTCH-3175 Implement integration testing framework for Nutch Protocol plugins using Testcontainers

  

  lewismc

  authored and

  sebastian-nagel

  committed
  f8e8583

  Copy full SHA for f8e8583

• parse: fix 'occured' -> 'occurred' typo in ParseStatus FAILED javadoc (#910)

  Show description for 2b72674
  

  SAY-5

  authored and

  sebastian-nagel

  committed
  2b72674

  Copy full SHA for 2b72674

• NUTCH-3168 Sandbox Commons JEXL usage in crawl and index pipelines (#909)

  

  lewismc

  authored and

  sebastian-nagel

  committed
  0ea90c3

  Copy full SHA for 0ea90c3

• [NUTCH-3160] Remove System.exit(..) from reusable code (#903)

  

  lfoppiano

  authored and

  sebastian-nagel

  committed
  d0099c0

  Copy full SHA for d0099c0

• NUTCH-3154 Implement integration testing framework for Nutch IndexWriter plugins using Testcontainers (#895)

  

  lewismc

  authored and

  sebastian-nagel

  committed
  3bf5541

  Copy full SHA for 3bf5541

Commits on May 8, 2026

• NUTCH-3174 protocol-okhttp: request may hang despite http.time.limit is set

  Show description for 3326bba
  sebastian-nagel

  committed
  3326bba

  Copy full SHA for 3326bba

Commits on May 4, 2026

• Revert "fix: apply URL normalisation for robots.txt redirect"

  Show description for 4560ffd
  lfoppiano

  committed
  4560ffd

  Copy full SHA for 4560ffd

• fix: apply URL normalisation for robots.txt redirect

  lfoppiano

  committed
  7f7cf6c

  Copy full SHA for 7f7cf6c

Commits on Apr 7, 2026

• Merge pull request #51 from commoncrawl/sitemap-localized-links

  Show description for 31c2ba8
  sebastian-nagel

  authored
  31c2ba8

  Copy full SHA for 31c2ba8

Commits on Apr 3, 2026

• Log sitemap processing timeout

  sebastian-nagel

  committed
  d5c8ab7

  Copy full SHA for d5c8ab7

• Extract and inject localized links from sitemaps

  sebastian-nagel

  committed
  f4f1232

  Copy full SHA for f4f1232

• SitemapInjector: update Javadoc

  sebastian-nagel

  committed
  2f1e195

  Copy full SHA for 2f1e195

Commits on Mar 28, 2026

• Fix revisit content-type (#42)

  Show description for d0c4d97
  lfoppiano

  authored
  d0c4d97

  Copy full SHA for d0c4d97

Commits on Feb 27, 2026

• Merge pull request #46 from commoncrawl/merge-upstream-master-1.22

  Show description for 2a9a6ab
  sebastian-nagel

  authored
  2a9a6ab

  Copy full SHA for 2a9a6ab

Commits on Feb 26, 2026

• WARC writer: log capture date as ISO date

  sebastian-nagel

  committed
  bf01b43

  Copy full SHA for bf01b43

• NUTCH-3132 Standardize existing Nutch metrics naming and implementation

  Show description for 044dfd2
  sebastian-nagel

  committed
  044dfd2

  Copy full SHA for 044dfd2

• NUTCH-3132 Standardize existing Nutch metrics naming and implementation

  Show description for 0eda915
  sebastian-nagel

  committed
  0eda915

  Copy full SHA for 0eda915

Commits on Feb 25, 2026

• NUTCH-3153 Update of license and notice files

  sebastian-nagel

  committed
  023010a

  Copy full SHA for 023010a

• NUTCH-3152 Job counters getGroup to use metrics constants

  sebastian-nagel

  committed
  fef49b9

  Copy full SHA for fef49b9

• NUTCH-3150 Expand Caching Hadoop Counter References (#892)

  

  lewismc

  authored and

  sebastian-nagel

  committed
  2e2374d

  Copy full SHA for 2e2374d

• NUTCH-3142 Add Error Context to Metrics (#882)

  

  lewismc

  authored and

  sebastian-nagel

  committed
  1db8e7d

  Copy full SHA for 1db8e7d

• NUTCH-3143 GitHub workflow does not run all unit tests (#890)

  

  lewismc

  authored and

  sebastian-nagel

  committed
  b8d1fc9

  Copy full SHA for b8d1fc9

• NUTCH-3143 GitHub workflow does not run all unit tests (#889)

  

  lewismc

  authored and

  sebastian-nagel

  committed
  e742fc5

  Copy full SHA for e742fc5

• NUTCH-3148 Cache Ivy dependencies in GitHub CI builds

  Show description for cc74d71
  sebastian-nagel

  committed
  cc74d71

  Copy full SHA for cc74d71

• NUTCH-3148 Cache Ivy dependencies in GitHub CI builds (#886)

  

  lewismc

  authored and

  sebastian-nagel

  committed
  fb8538b

  Copy full SHA for fb8538b

• NUTCH-1564: address code review comments.

  Show description for 366a601
  

  Isabelle Giguere

  authored and

  sebastian-nagel

  committed
  366a601

  Copy full SHA for 366a601

• NUTCH-1564: fix AdaptiveFetchSchedule for unmodified pages

  Show description for 89e6b87
  

  Isabelle Giguere

  authored and

  sebastian-nagel

  committed
  89e6b87

  Copy full SHA for 89e6b87

• NUTCH-1564: fix immediate refetch for pages not modified

  Show description for 226ac7e
  

  Isabelle Giguere

  authored and

  sebastian-nagel

  committed
  226ac7e

  Copy full SHA for 226ac7e

• NUTCH-3144 URLUtil unit tests fail after upgrade to crawler-commons 1.6

  Show description for e3d0af3
  sebastian-nagel

  committed
  e3d0af3

  Copy full SHA for e3d0af3