Commits

Breadcrumbs

History for

onmaster

User selector

All users

All time

Commit History

Commits on Dec 4, 2024

• Deprecate some classes specific to HttpClient 3

  Show description for 8988fbb
  ato

  committed
  8988fbb

  Copy full SHA for 8988fbb

Commits on Dec 3, 2024

• Use Files.createLink instead of shelling out to ln

  ato

  committed
  4bb03ba

  Copy full SHA for 4bb03ba

• Remove dependency on dsiutils

  ato

  committed
  3ae5720

  Copy full SHA for 3ae5720

Commits on Nov 27, 2024

• Merge pull request #98 from sebastian-nagel/metadata-multivalued

  Show description for dcbb052
  ato

  authored
  dcbb052

  Copy full SHA for dcbb052

• URLParser and WaybackURLKeyMaker fail on URLs with IPv6 address hostname

  sebastian-nagel

  committed
  a4748d9

  Copy full SHA for a4748d9

Commits on Nov 9, 2024

• Make MetaData multi-valued to preserve values of repeating WARC and HTTP headers

  Show description for 835f4e1
  sebastian-nagel

  committed
  835f4e1

  Copy full SHA for 835f4e1

Commits on Oct 15, 2024

• Fix javadoc build errors affecting JDK 8

  ato

  committed
  ce4b7b8

  Copy full SHA for ce4b7b8

• Fix javadoc build errors

  ato

  committed
  74bc86c

  Copy full SHA for 74bc86c

Commits on Sep 27, 2024

• Make MetaData multi-valued to preserve values of repeating WARC and HTTP headers

  sebastian-nagel

  committed
  f4f6655

  Copy full SHA for f4f6655

Commits on Jun 11, 2020

• WAT extractor: do not fail on missing WARC-Filename in warcinfo record, fixes #88

  Show description for a2cc42c
  sebastian-nagel

  committed
  a2cc42c

  Copy full SHA for a2cc42c

Commits on Oct 25, 2019

• Merge pull request #85 from sebastian-nagel/cc-8-wat-extract-onclick-links

  Show description for 3d0cdd7
  ato

  authored
  3d0cdd7

  Copy full SHA for 3d0cdd7

Commits on Oct 22, 2019

• ExtractingParseObserver: extract rel, hreflang and type attributes

  Show description for 26b1e7a
  sebastian-nagel

  committed
  26b1e7a

  Copy full SHA for 26b1e7a

• ExtractingParseObserver: get links from onClick attributes

  Show description for 79aed91
  sebastian-nagel

  committed
  79aed91

  Copy full SHA for 79aed91

Commits on May 2, 2017

• Fix HTTP-Response-Metadata for wget WARCs. Changes came from https://github.com/commoncrawl/ia-web-commons/commit/58e85a60d75707da55ed499f836e57d49347484a

  ldko

  committed
  4077670

  Copy full SHA for 4077670

• Merge branch 'empty-http-header-field' of https://github.com/sebastian-nagel/webarchive-commons into sebastian-nagel-empty-http-header-field

  ldko

  committed
  898b478

  Copy full SHA for 898b478

• Extract also `property` attributes of HTML meta elements, this fixes #67

  sebastian-nagel

  committed
  bd08143

  Copy full SHA for bd08143

• Do not add value of preceding HTTP header field if there is no value (or

  Show description for cf34a3e
  sebastian-nagel

  committed
  cf34a3e

  Copy full SHA for cf34a3e

Commits on Feb 22, 2017

• Improve HTML link extraction

  Show description for 11579c2
  sebastian-nagel

  committed
  11579c2

  Copy full SHA for 11579c2

Commits on Feb 15, 2017

• Merge pull request #69 from sebastian-nagel/cc-url-parser-ignore-empty-port

  Show description for cb91063
  ldko

  authored
  cb91063

  Copy full SHA for cb91063

• Merge pull request #68 from sebastian-nagel/HTMLResourceFactory_to_use_CharsetDetector

  Show description for ed33bae
  ldko

  authored
  ed33bae

  Copy full SHA for ed33bae

• Merge pull request #65 from dportabella/fix

  Show description for 1ee2529
  ldko

  authored
  1ee2529

  Copy full SHA for 1ee2529

• Merge pull request #62 from iipc/kris-sigur-patch-1

  Show description for c5876b3
  ldko

  authored
  c5876b3

  Copy full SHA for c5876b3

Commits on Jan 27, 2017

• CharsetDetector: remove unnecessary check for empty string (contributed by @ldko)

  sebastian-nagel

  committed
  0384028

  Copy full SHA for 0384028

Commits on Jan 18, 2017

• CSS link extraction: clip also unpaired leading and trailing quotation marks

  sebastian-nagel

  committed
  194a1fa

  Copy full SHA for 194a1fa

Commits on Jan 4, 2017

• Improve clipping of quotation marks in CSS link extraction

  Show description for b918f7f
  sebastian-nagel

  committed
  b918f7f

  Copy full SHA for b918f7f

Commits on Dec 9, 2016

• Strip empty port, do not fail

  sebastian-nagel

  committed
  9e41abc

  Copy full SHA for 9e41abc

Commits on Nov 24, 2016

• Match http-equiv meta elements with unquoted attribute values, e.g.

  Show description for 824dd82
  sebastian-nagel

  committed
  824dd82

  Copy full SHA for 824dd82

• HTML encoding detection: fix errors with empty content or empty charset values

  sebastian-nagel

  committed
  607acaa

  Copy full SHA for 607acaa

• Use CharsetDetector to guess encoding of HTML document

  sebastian-nagel

  committed
  5f223d6

  Copy full SHA for 5f223d6

Commits on Sep 21, 2016

• fix: last header was lost if LF LF (intead of CRLF CRLF)

  dportabella

  committed
  9d7abed

  Copy full SHA for 9d7abed

Commits on Aug 7, 2016

• Make regular expression to extract URLs from CSS more restrictive

  Show description for a23cfeb
  sebastian-nagel

  committed
  a23cfeb

  Copy full SHA for a23cfeb

Commits on Jul 13, 2016

• Remove invalid constant

  Show description for 0cbca57
  

  Kristinn Sigurðsson

  authored
  0cbca57

  Copy full SHA for 0cbca57

Commits on May 26, 2016

• Merge pull request #54 from vonrosen/modify-strip-session-params

  Show description for 1485fdd
  

  Kristinn Sigurðsson

  committed
  1485fdd

  Copy full SHA for 1485fdd

• Merge pull request #50 from nlevitt/her-2089

  Show description for cf5f600
  

  Kristinn Sigurðsson

  committed
  cf5f600

  Copy full SHA for cf5f600

• Merge pull request #51 from nlevitt/fix-warc-sizeOnDisk

  Show description for f5e6541
  

  Kristinn Sigurðsson

  committed
  f5e6541

  Copy full SHA for f5e6541