|
1 | 1 | # Data Sources
|
2 | 2 |
|
3 |
| -This project uses data from various sources that are openly licensed or in the public domain. Below are the sources and their respective information: |
| 3 | +This project uses data from various sources that are openly licensed or in the |
| 4 | +public domain. Below are the sources and their respective information: |
| 5 | + |
4 | 6 |
|
5 | 7 | ## CC Legal Tools
|
6 | 8 |
|
7 |
| -**Description:** _A .txt provided by Timid Robot containing all legal tool paths. The data from Google Custom Search will only cover 50+ general, most significant categories of CC License for data collection quota constraint. As an additional note, the order of precedence of license the collected data's first column is sorted due to intermediate data analysis progress._ |
| 9 | +**Description:** A `.txt` file provided by Timid Robot containing all legal |
| 10 | +tool paths. |
8 | 11 |
|
9 | 12 | **API documentation link:**
|
10 |
| -- [List of all current CC legal tool paths by TimidRobot](https://github.com/creativecommons/quantifying/blob/main/google_custom_search/legal-tool-paths.txt) |
| 13 | +- [`google_custom_search/legal-tool-paths.txt`][tools-paths]: a list of all |
| 14 | + current CC legal tool paths by TimidRobot) |
11 | 15 |
|
12 | 16 | **API information:**
|
13 | 17 | - No API key required
|
14 | 18 | - No query limits
|
15 | 19 |
|
| 20 | +[tools-paths]:google_custom_search/legal-tool-paths.txt |
| 21 | + |
| 22 | + |
16 | 23 | ## Flickr
|
17 | 24 |
|
18 |
| -**Description:** _With over 5 billion photos (many with valuable metadata such as tags, geolocation, |
19 |
| -and Exif data), the Flickr community creates wonderfully rich data. The Flickr API is how you can |
20 |
| -access that data. In fact, almost all the functionality that runs flickr.com is available through |
21 |
| -the API._ ([Flickr: The Flickr Developer Guide](https://www.flickr.com/services/developer/)) |
| 25 | +**Description:** _With over 5 billion photos (many with valuable metadata such |
| 26 | +as tags, geolocation, and Exif data), the Flickr community creates wonderfully |
| 27 | +rich data. The Flickr API is how you can access that data. In fact, almost all |
| 28 | +the functionality that runs flickr.com is available through the API._ ([Flickr: |
| 29 | +The Flickr Developer Guide](https://www.flickr.com/services/developer/)) |
22 | 30 |
|
23 |
| -**API documentation link:** |
| 31 | +**API documentation link:** |
24 | 32 | - [API documentation - Flickr Services](https://www.flickr.com/services/api/)
|
25 | 33 |
|
26 | 34 | **API information:**
|
27 | 35 | - API key required
|
28 | 36 | - Query limit: 3600 requests per hour
|
29 | 37 | - Data available through CSV format
|
30 | 38 |
|
| 39 | + |
31 | 40 | ## Google Custom Search JSON API
|
32 | 41 |
|
33 |
| -**Description:** _The Custom Search JSON API allows user-defined detailed query and access towards related query data using a programmable search engine._ |
| 42 | +**Description:** The Custom Search JSON API allows user-defined detailed query |
| 43 | +and access towards related query data using a programmable search engine. |
34 | 44 |
|
35 | 45 | **API documentation links:**
|
36 |
| -- [Custom Search JSON API Reference | Programmable Search Engine | Google Developers](https://developers.google.com/custom-search/v1/reference/rest) |
37 |
| -- [Method: cse.list | Custom Search JSON API | Google Developers](https://developers.google.com/custom-search/v1/reference/rest/v1/cse/list) |
| 46 | +- [Custom Search JSON API Reference | Programmable Search Engine | Google |
| 47 | + Developers][google-json] |
| 48 | +- [Method: cse.list | Custom Search JSON API | Google Developers][cse-list] |
38 | 49 |
|
39 | 50 | **API information:**
|
40 | 51 | - API key required
|
41 | 52 | - Query limit: 100 queries per day for free version
|
42 | 53 | - Data available through JSON format
|
43 | 54 |
|
| 55 | +**Notes:** |
| 56 | +- The data from Google Custom Search will only cover 50+ general, most |
| 57 | + significant categories of CC License for data collection quota constraint. |
| 58 | + As an additional note, the order of precedence of license the collected |
| 59 | + data's first column is sorted due to intermediate data analysis progress. |
| 60 | + |
| 61 | +[google-json]: https://developers.google.com/custom-search/v1/reference/rest |
| 62 | +[cse-list]: https://developers.google.com/custom-search/v1/reference/rest/v1/cse/list) |
| 63 | + |
| 64 | + |
44 | 65 | ## Internet Archive Python Interface
|
45 | 66 |
|
46 |
| -**Description:** _A python interface to archive.org to achieve API requests towards internet archive._ |
| 67 | +**Description:** A python interface to archive.org to achieve API requests |
| 68 | +towards internet archive. |
47 | 69 |
|
48 |
| -**API documentation link:** |
49 |
| -- [internetarchive.Search - Internetarchive: A Python Interface to archive.org](https://internetarchive.readthedocs.io/en/stable/internetarchive.html#internetarchive.Search) |
| 70 | +**API documentation link:** |
| 71 | +- [internetarchive.Search - Internetarchive: A Python Interface to |
| 72 | + archive.org][ia-search] |
50 | 73 |
|
51 | 74 | **API information:**
|
52 | 75 | - No API key required
|
53 | 76 | - No query limits
|
54 | 77 |
|
| 78 | +[ia-search]: https://internetarchive.readthedocs.io/en/stable/internetarchive.html#internetarchive.Search |
| 79 | + |
| 80 | + |
| 81 | +## MediaWiki Action API |
| 82 | + |
| 83 | +**Description:** _The MediaWiki Action API is a web service that allows access |
| 84 | +to some wiki features like authentication, page operations, and search. It can |
| 85 | +provide meta information about the wiki and the logged-in user._ ([API:Main |
| 86 | +page - MediaWiki](https://www.mediawiki.org/wiki/API:Main_page)) |
| 87 | + |
| 88 | +**API documentation link:** |
| 89 | +- [MediaWiki Action API](https://www.mediawiki.org/wiki/API:Main_page) |
| 90 | + |
| 91 | +**API information:** |
| 92 | + - No API key required |
| 93 | + - Query limit: depends on user status and request type |
| 94 | + - Data available through XML or JSON format |
| 95 | + |
| 96 | + |
55 | 97 | ## The Metropolitan Museum of Art Collection API
|
56 | 98 |
|
57 |
| -**Description:** _An API endpoint for receiving Metropolitan Muesum of Art Collection's CC-Licensed works._ |
| 99 | +**Description:** _The Met’s Open Access datasets are available through our API. |
| 100 | +The API (RESTful web service in JSON format) gives access to all of The Met’s |
| 101 | +Open Access data and to corresponding high resolution images (JPEG format) that |
| 102 | +are in the public domain._ ([The Metropolitan Museum of Art Collection |
| 103 | +API](https://metmuseum.github.io/)) |
58 | 104 |
|
59 |
| -**API documentation link:** |
60 |
| -- [Latest Updates | The Metropolitan Museum of Art Collection API](https://metmuseum.github.io/) |
| 105 | +**API documentation link:** |
| 106 | +- [Latest Updates | The Metropolitan Museum of Art Collection |
| 107 | + API](https://metmuseum.github.io/) |
61 | 108 |
|
62 | 109 | **API information:**
|
63 | 110 | - No API key required
|
64 | 111 | - 80 queries per second
|
65 | 112 |
|
| 113 | + |
66 | 114 | ## Vimeo API
|
67 | 115 |
|
68 |
| - **Description:** _The Vimeo API allows users to perform filtered, advanced search on Vimeo videos._ |
| 116 | +**Description:** The Vimeo API allows users to perform filtered, advanced |
| 117 | +search on Vimeo videos. |
69 | 118 |
|
70 |
| -**API documentation link:** |
| 119 | +**API documentation link:** |
71 | 120 | - [Getting Started with the Vimeo API](https://developer.vimeo.com/api/start)
|
72 | 121 |
|
73 | 122 | **API information:**
|
74 | 123 | - API key required
|
75 | 124 | - Query limit: 5000 authenticated requests per day
|
76 | 125 | - Data available through JSON format
|
77 | 126 |
|
78 |
| -## MediaWiki Action API |
79 | 127 |
|
80 |
| -**Description:** _The MediaWiki Action API is a web service that allows access to some wiki features like authentication, page operations, and search. It can provide meta information about the wiki and the logged-in user._ |
81 |
| - |
82 |
| -**API documentation link:** |
83 |
| -- [MediaWiki Action API](https://www.mediawiki.org/wiki/API:Main_page) |
84 |
| - |
85 |
| -**API information:** |
86 |
| - - No API key required |
87 |
| - - Query limit: depends on user status and request type |
88 |
| - - Data available through XML or JSON format |
89 |
| - |
90 | 128 | ## YouTube Data API
|
91 | 129 |
|
92 |
| -**Description:** _An API from YouTube for platform users to upload videos, adjust video parameters, and obtain search results._ |
| 130 | +**Description:** An API from YouTube for platform users to upload videos, |
| 131 | +adjust video parameters, and obtain search results. |
93 | 132 |
|
94 |
| -**API documentation link:** |
95 |
| -- [Search: list | YouTube Data API | Google Developers](https://developers.google.com/youtube/v3/docs/search/list) |
| 133 | +**API documentation link:** |
| 134 | +- [Search: list | YouTube Data API | Google |
| 135 | + Developers](https://developers.google.com/youtube/v3/docs/search/list) |
96 | 136 |
|
97 | 137 | **API information:**
|
98 | 138 | - API key required
|
|
0 commit comments