pypi package#55

Draft

damian0815 wants to merge 34 commits into

mainfrom

damian/feat/pypi-package

Commits on Oct 16, 2025

well-formed example passes
Damian Stewart
committed

Commits on Oct 17, 2025

wip (back to basics on pyspark)
Damian Stewart
committed
identical output with different format and sort order
Damian Stewart
committed
typo
Damian Stewart
committed
fix failing tests by spinning up a proper spark session and RDD
Damian Stewart
committed

Commits on Oct 20, 2025

exactly match cc-mrjob output
Damian Stewart
committed
wip content encoding and invalid url tests
Damian Stewart
committed
Add test for >50 sitemaps; add accumulator checks to tests
Damian Stewart
committed
add fastwarc implementation
Damian Stewart
committed
fix logging
Damian Stewart
committed

Commits on Oct 30, 2025

simplify & address review comments
Damian Stewart
committed
add exception handling for urljoin
Damian Stewart
committed
add python unit test github workflow
Damian Stewart
committed
downgrade from uv
Damian Stewart
committed
remove same-host optimization
Damian Stewart
committed
split PySpark into its own requirements.txt; update README
Damian Stewart
committed
typo
Damian Stewart
committed
typos
Damian Stewart
committed
polish
Damian Stewart
committed
cleanup remaining spark-submit invocations
Damian Stewart
committed

Commits on Oct 31, 2025

update docs with expanded PySpark info
Damian Stewart
committed
typo; clarify debugging limitations
Damian Stewart
committed
address review comments
Damian Stewart
committed
wip
Damian Stewart
committed
add pyproject.toml
Damian Stewart
committed

Commits on Nov 3, 2025

wip packaging
Damian Stewart
committed

Commits on Nov 7, 2025

wip package_and_run_job
Damian Stewart
committed
update version
Damian Stewart
committed
add pypi publish workflow
Damian Stewart
committed
Merge branch 'main' into damian/feat/pypi-package
Damian Stewart
committed

Commits on Nov 13, 2025

try and make the workflow run
Damian Stewart
committed
try and make the workflow run 2
Damian Stewart
committed
try and make the workflow run 3
Damian Stewart
committed
fix pypi project url
Damian Stewart
committed