Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: jquery/jquerymobile.com
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: main
Choose a base ref
...
head repository: jquery/jquerymobile.com
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: add-typesense
Choose a head ref
  • 1 commit
  • 2 files changed
  • 1 contributor

Commits on Oct 12, 2023

  1. Build: Enable typesense scraper 

    This includes the sitemap so that we're sure no content is missed.

Unlike api.jquery.com, api.jquerymobile.com does not start with an
index that links to all content pages. This means the crawler would
have to to rely on category pages to discover all content, except
we don't want the cralwer to index /category/ pages, and thus are
matched by stop_urls, which means they are never crawled.

If there was a variant of `stop_urls` that behaved like `follow,noindex`
instead of `noindex,follow` we could use that, but I'm not aware of
such feature. The sitemap accomplishes the same thing in a more
efficient manner.

Ref jquery/infrastructure-puppet#33
    @Krinkle
    Krinkle committed Oct 12, 2023
    Configuration menu
    Copy the full SHA
    0449a40 View commit details
    Browse the repository at this point in the history
Loading