Does not build due to conflict with Maven

[bbrancar]

Hi,

In the new version of Maven, mvn no longer builds http links. This is the error I receive from mvn package:

[ERROR] Failed to execute goal on project ia-hadoop-tools: Could not resolve dependencies for project org.archive:ia-hadoop-tools:jar:1.0-SNAPSHOT: Failed to collect dependencies at org.archive.access-control:access-control:jar:0.1.0-SNAPSHOT: Failed to read artifact descriptor for org.archive.access-control:access-control:jar:0.1.0-SNAPSHOT: The following artifacts could not be resolved: org.archive.access-control:access-control:pom:0.1.0-SNAPSHOT (absent): Could not transfer artifact org.archive.access-control:access-control:pom:0.1.0-SNAPSHOT from/to maven-default-http-blocker (http://0.0.0.0/): Blocked mirror for repositories: [internetarchive (http://builds.archive.org:8080/maven2, default, releases+snapshots)] -> [Help 1]

Do you have a suggested work around/ is there an alternative repository I should use for Warc to Wet conversion?

Thank you

[sebastian-nagel]

Hi @bbrancar, thanks for reporting! I'll have a look during the next days. In doubt, if there is no other solution, I'll reduce this project to WAT/WET conversion only, stripping all unneeded functionality and dependencies.

[sebastian-nagel]

Hi @bbrancar, a solution is on the way. Please, see #5.

However, I forgot about the simplest work-around. Sorry about that! ia-web-commons's ResourceExtractor can be directly called to produce a WAT or WET file (but not both in one turn):

java -cp target/webarchive-commons-jar-with-dependencies.jar org.archive.extract.ResourceExtractor -wat my.warc.gz my.warc.wat.gz

or

java -cp target/webarchive-commons-jar-with-dependencies.jar org.archive.extract.ResourceExtractor -wet my.warc.gz my.warc.wet.gz

This works only locally, but not on a Hadoop cluster.

[eukaryoting]

FYI I got the build working by running maven wrapper in ia-hadoop-tools and seems to work well with Maven 3.5.4.

mvn wrapper:wrapper
mvn wrapper:wrapper -Dmaven=3.5.4
./mvnw clean package

[sebastian-nagel]

Fixed along with #5. Thanks for reporting, @bbrancar !