Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: commoncrawl/news-crawl
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: 2.x
Choose a base ref
...
head repository: commoncrawl/news-crawl
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: main
Choose a head ref
Checking mergeability… Don’t worry, you can still create the pull request.
  • 9 commits
  • 45 files changed
  • 2 contributors

Commits on Mar 31, 2026

  1. Upgrade code to StormCrawler 3.5.1 

    Upgrade dependencies and Maven plugins to latest available versions

Migrate from Elasticsearch to OpenSearch
    @sebastian-nagel
    sebastian-nagel committed Mar 31, 2026
    Configuration menu
    Copy the full SHA
    c4519b8 View commit details
    Browse the repository at this point in the history

  2. Sync GitHub CI build workflow with upstream StormCrawler 

    Prepare for renaming of development branch to "main".
    @sebastian-nagel
    sebastian-nagel committed Mar 31, 2026
    Configuration menu
    Copy the full SHA
    0bee798 View commit details
    Browse the repository at this point in the history

  3. Replace Dockerfile by docker compose config 

    Make the configuration work.
    @sebastian-nagel
    sebastian-nagel committed Mar 31, 2026
    Configuration menu
    Copy the full SHA
    6bd1e31 View commit details
    Browse the repository at this point in the history

  4. Pin version of Jackson: needs to be same as used by Storm

    @sebastian-nagel
    sebastian-nagel committed Mar 31, 2026
    Configuration menu
    Copy the full SHA
    12a5b50 View commit details
    Browse the repository at this point in the history

Commits on Jun 12, 2026

  1. Upgrade to StormCrawler 3.6.0, Storm 2.8.8

    @sebastian-nagel
    sebastian-nagel committed Jun 12, 2026
    Configuration menu
    Copy the full SHA
    0d68ae5 View commit details
    Browse the repository at this point in the history

Commits on Jun 16, 2026

  1. Consistently name components and containers in docker compose configu… 

    …ration

Add suffix "-news-crawl" to all Storm / Zookeeper container names to avoid
name collisions with other StormCrawler setups running on the same system.
    @sebastian-nagel
    sebastian-nagel committed Jun 16, 2026
    Configuration menu
    Copy the full SHA
    98e531a View commit details
    Browse the repository at this point in the history

  2. Merge pull request #73 from commoncrawl/upgrade-storm-crawler-3.6.0 

    Upgrade to StormCrawler 3.6.0
    @sebastian-nagel
    sebastian-nagel authored Jun 16, 2026
    Configuration menu
    Copy the full SHA
    bb02815 View commit details
    Browse the repository at this point in the history

  3. Add GitHub Dependabot configuration file

    @sebastian-nagel
    sebastian-nagel committed Jun 16, 2026
    Configuration menu
    Copy the full SHA
    f57b83d View commit details
    Browse the repository at this point in the history

  4. Add gitignore and automatic code formatting (#71) 

    - Added Cosium formatter 
- Added .gitignore 
- Fixed Licence headers (start with /* instead of /**)
    @lfoppiano
    lfoppiano authored Jun 16, 2026
    Configuration menu
    Copy the full SHA
    e6e4d74 View commit details
    Browse the repository at this point in the history
Loading