Commits

Branch selector

 master

User selector

sylvinus

All time

Commit History

Commits on Nov 14, 2016

• Fix lint

  sylvinus

  committed
  28ca0c1

  Copy full SHA for 28ca0c1

• Merge branch 'master' of github.com:commonsearch/cosr-back

  Show description for 65d71b5
  sylvinus

  committed
  65d71b5

  Copy full SHA for 65d71b5

• Rename "path" parameter to "output" in many plugins. Documentation improvements after feedbacks from PyConFR sprint. Add Python implementation of PageRank for education purposes. Generalize the use…

  Show description for d125c98
  sylvinus

  committed
  d125c98

  Copy full SHA for d125c98

Commits on Nov 7, 2016

• Update to Common Crawl October 2016

  sylvinus

  authored
  1850222

  Copy full SHA for 1850222

• Update import_commoncrawl.sh

  sylvinus

  authored
  1d26be5

  Copy full SHA for 1d26be5

Commits on Oct 13, 2016

• Fix absolute import error. Thanks @MickaelBergem for the report :)

  sylvinus

  committed
  70c38af

  Copy full SHA for 70c38af

Commits on Sep 4, 2016

• Upgrade Protobug (wasn't building properly anymore)

  sylvinus

  committed
  a4dbfd2

  Copy full SHA for a4dbfd2

• PageRank fixes for upcoming tutorial

  sylvinus

  committed
  d80d1dc

  Copy full SHA for d80d1dc

Commits on Aug 29, 2016

• Initial Python 3 prep work

  sylvinus

  committed
  a5b06db

  Copy full SHA for a5b06db

Commits on Aug 27, 2016

• Add a test for the Common Crawl source, rename the dump.DocumentMetadata plugin, rename the old "parquet" source to "metadata"

  sylvinus

  committed
  f069527

  Copy full SHA for f069527

Commits on Aug 26, 2016

• Fix lint

  sylvinus

  committed
  3bee4cf

  Copy full SHA for 3bee4cf

Commits on Aug 25, 2016

• Exclude nofollow links from the webgraph (#52), rename "datasources" to "dataproviders" to avoid confusion with document sources, and other smaller refactors

  sylvinus

  committed
  4f0a62a

  Copy full SHA for 4f0a62a

Commits on Aug 13, 2016

• Fix Travis build

  sylvinus

  committed
  ebdbec9

  Copy full SHA for ebdbec9

• Experimental OAI-PMH source

  sylvinus

  committed
  cc62598

  Copy full SHA for cc62598

• Fix Travis build

  sylvinus

  committed
  613404a

  Copy full SHA for 613404a

• Use openssl for md5

  sylvinus

  committed
  ec2f8fb

  Copy full SHA for ec2f8fb

Commits on Aug 12, 2016

• Travis cache (#50) + Warning when using outdated Docker image

  sylvinus

  committed
  5540fee

  Copy full SHA for 5540fee

Commits on Aug 11, 2016

• Fix createDataFrame call for Spark 2.0.0

  sylvinus

  committed
  998e2b3

  Copy full SHA for 998e2b3

• Usability improvements to help for the new tutorial

  sylvinus

  committed
  9f0cbe3

  Copy full SHA for 9f0cbe3

• Remove hack for ca-certificates-java

  sylvinus

  committed
  9e0154a

  Copy full SHA for 9e0154a

• Use Travis-supplied Docker 1.12

  sylvinus

  committed
  3f018af

  Copy full SHA for 3f018af

• Smarter defaults for filter plugin + Move partition logic to document sources

  sylvinus

  committed
  3d029f2

  Copy full SHA for 3d029f2

Commits on Aug 2, 2016

• Add Travis notifs to Slack

  sylvinus

  committed
  01fec8b

  Copy full SHA for 01fec8b

• Fix slack button URL

  sylvinus

  committed
  742ec55

  Copy full SHA for 742ec55

• Test slack button

  sylvinus

  committed
  0a3df9b

  Copy full SHA for 0a3df9b

• Merge pull request #51 from bakztfuture/master

  Show description for 3ab7176
  sylvinus

  authored
  3ab7176

  Copy full SHA for 3ab7176

Commits on Jul 31, 2016

• Fixes to make PageRank run at scale, first live dump created!

  sylvinus

  committed
  6fe1355

  Copy full SHA for 6fe1355

Commits on Jul 30, 2016

• Add experimental hyperlinks plugin to collect links to a specific domain

  sylvinus

  committed
  45f87e8

  Copy full SHA for 45f87e8

• Convert grep plugin to use Spark SQL

  sylvinus

  committed
  f58d9cd

  Copy full SHA for f58d9cd

• PageRank improvements, with full set of tests

  sylvinus

  committed
  1419666

  Copy full SHA for 1419666

Commits on Jul 29, 2016

• Multiple fixes to get PageRank working at scale

  sylvinus

  committed
  58c3c9b

  Copy full SHA for 58c3c9b

Commits on Jul 27, 2016

• Upgrade to Spark 2.0.0 and refactor WebGraph + PageRank jobs to use Spark SQL for more performance

  sylvinus

  committed
  e9c1e2b

  Copy full SHA for e9c1e2b

Commits on Jul 21, 2016

• Spark configuration management

  sylvinus

  committed
  872c269

  Copy full SHA for 872c269

• Plugin refactors + split internal & external links

  sylvinus

  committed
  700b3fb

  Copy full SHA for 700b3fb

• Introduce intermediate document metadata Parquet dumps. Misc refactorings

  sylvinus

  committed
  4208e21

  Copy full SHA for 4208e21