Commits

History for

Commits on Aug 29, 2016

Initial Python 3 prep work
sylvinus
committed

Commits on Aug 26, 2016

Fix lint
sylvinus
committed

Commits on Aug 25, 2016

Exclude nofollow links from the webgraph (#52), rename "datasources" to "dataproviders" to avoid confusion with document sources, and other smaller refactors
sylvinus
committed

Commits on Jul 27, 2016

Upgrade to Spark 2.0.0 and refactor WebGraph + PageRank jobs to use Spark SQL for more performance
sylvinus
committed

Commits on Jul 21, 2016

Plugin refactors + split internal & external links
sylvinus
committed

Commits on Jul 19, 2016

Scale back on the use of DataFrames because of Java Heap issues
sylvinus
committed

Commits on Jul 18, 2016

Do more validation on collected links
sylvinus
committed

Commits on Jul 16, 2016

Improve static data import & add ETA
sylvinus
committed

Commits on Jul 15, 2016

Commits on Jul 12, 2016

Refactor the indexing pipeline by adding support for plugins & many other small fixes.
sylvinus
committed

Commits on Mar 12, 2016

Index subdomain separately from path. Make sure queries with 2 terms have to match both for commonsearch/cosr-results#12
sylvinus
committed

Commits on Feb 27, 2016

Avoid indexing data URIs for images #21
Adrien Di Pasquale
committed

Commits on Feb 24, 2016

Tweaks to relevance & indexing. Should help #17 & #16.
sylvinus
committed

Commits on Feb 17, 2016

First code import!
sylvinus
committed

Loading