|
| 1 | +title: Linked Commons: GSoC'20 Wrap Up |
| 2 | +--- |
| 3 | +categories: |
| 4 | +announcements |
| 5 | +cc-catalog |
| 6 | +product |
| 7 | +gsoc |
| 8 | +gsoc-2020 |
| 9 | +open-source |
| 10 | +cc-dataviz |
| 11 | +--- |
| 12 | +author: subhamX |
| 13 | +--- |
| 14 | +series: gsoc-2020-dataviz |
| 15 | +--- |
| 16 | +pub_date: 2020-08-28 |
| 17 | +--- |
| 18 | + |
| 19 | +body: |
| 20 | +Time flies faster when you are having fun! I didn't believe it back then. But now I do after experiencing it. It couldn't have been more accurate that here I am writing my concluding blog of the **GSoC 2020: The Linked Commons series** when I just started enjoying things. |
| 21 | + |
| 22 | +In this post, I will give a brief overview of the linked commons and my GSoC contributions. It was an exciting journey, and I loved working on this project. |
| 23 | + |
| 24 | +Before I begin any further just for ritual, let me share a one liner on **what The Linked Commons is**, although I highly recommend reading the other posts in this series, who knows you might join our team.😉 |
| 25 | +> The CC catalog data visualization or linked commons is a web application which finds and explores relationships between Creative Commons licensed content on the web. |
| 26 | +
|
| 27 | + |
| 28 | +My primary contributions to the project during the GSoC timeline were threefold. |
| 29 | + |
| 30 | +**Firstly**, revamp the design and migrate the project to react.js for a fast and scalable rendering performance. |
| 31 | + |
| 32 | +**Secondly**, add graph filtering methods and scale the data to enable users to visualize massive data more efficiently. |
| 33 | + |
| 34 | +**At last**, make the developer onboarding easy by dockerizing the project and bring more application portability. |
| 35 | + |
| 36 | + |
| 37 | + |
| 38 | +## GSoC Work Product |
| 39 | + |
| 40 | +The live version of the linked commons can be found [here](http://dataviz.creativecommons.engineering/). You can interact with it and **"explore the creative commons in graphs"**. |
| 41 | + |
| 42 | +If you wish to access the raw or filtered data, then here is a brief documentation of our new API. |
| 43 | + |
| 44 | + |
| 45 | +```yaml |
| 46 | +URL: /api/graph-data |
| 47 | +Method: GET |
| 48 | +Description: Returns a randomized graph having around 500 nodes and links. |
| 49 | +``` |
| 50 | + |
| 51 | +```yaml |
| 52 | +URL: /api/graph-data/?name={node_name} |
| 53 | +Method: GET |
| 54 | +Description: Returns the filtered graph with a set of nodes which are either immediate neighbours to {node_name} in the original graph or the transpose graph. |
| 55 | +``` |
| 56 | + |
| 57 | +```yaml |
| 58 | +URL: /api/suggestions/?q={query} |
| 59 | +Method: GET |
| 60 | +Description: Returns a set of nodes which contains the {query} pattern in their nodeid |
| 61 | +``` |
| 62 | +
|
| 63 | +
|
| 64 | +### Demo |
| 65 | +
|
| 66 | +<div style="text-align: center; width: 90%; margin-left: 5%;"> |
| 67 | + <figure> |
| 68 | + <img src="graph-filtering.gif" alt="demo" style="border: 1px solid black"> |
| 69 | + <figcaption>Linked Commons: Filtering the Graph 🔥</figcaption> |
| 70 | + </figure> |
| 71 | +</div> |
| 72 | +
|
| 73 | +
|
| 74 | +
|
| 75 | +
|
| 76 | +### My Code Contributions |
| 77 | +
|
| 78 | +**Repository:** https://github.com/creativecommons/cccatalog-dataviz/ |
| 79 | +
|
| 80 | +**Commits:** https://github.com/creativecommons/cccatalog-dataviz/commits/master |
| 81 | +
|
| 82 | +**Contributors:** https://github.com/creativecommons/cccatalog-dataviz/graphs/contributors |
| 83 | +
|
| 84 | +
|
| 85 | +[**Migrate frontend to React #28**](https://github.com/creativecommons/cccatalog-dataviz/pull/28) |
| 86 | +- Migrated the frontend to a web application using React.js for smooth rendering performance. |
| 87 | +- Add client-side graph filtering method to enable users to interact with the loaded graph. |
| 88 | +
|
| 89 | +
|
| 90 | +[**Add server-side filtering #29**](https://github.com/creativecommons/cccatalog-dataviz/pull/29) |
| 91 | +- We realized that the client-side graph filtering method is not very scalable. This PR adds the basic structure for the backend server and adds server-side graph filtering logic. |
| 92 | +- Added a parser to convert the input JSON file from `{nodes:[], links:[]}` schema to the distance list format. |
| 93 | + |
| 94 | +<div style="text-align: center; width: 90%; margin-left: 5%;"> |
| 95 | + <figure> |
| 96 | + <img src="api-call.png" alt="API call" style="border: 1px solid black"> |
| 97 | + <figcaption>API call to filter graph data</figcaption> |
| 98 | + </figure> |
| 99 | +</div> |
| 100 | + |
| 101 | + |
| 102 | + |
| 103 | +[**Design upgrade #33**](https://github.com/creativecommons/cccatalog-dataviz/pull/33) |
| 104 | +- It revamped the design of the frontend. |
| 105 | +- Added both primary light theme and secondary dark theme |
| 106 | + |
| 107 | +<div style="text-align: center; font-style: normal; width: 80%; margin-left: 10%;"> |
| 108 | + <figure> |
| 109 | + <img src="design-dark.png" alt="Dark Theme" style="border: 1px solid black"> |
| 110 | + <figcaption><em>Linked Commons: Dark Theme</em></figcaption> |
| 111 | + </figure> |
| 112 | +</div> |
| 113 | + |
| 114 | +[**Add node suggestions feature #35**](https://github.com/creativecommons/cccatalog-dataviz/pull/35) |
| 115 | +- Added query autocomplete feature, to enable users to explore all the nodes in the database. |
| 116 | +- This functionality aims to minimize the number of misspelt filtering tries from the client. |
| 117 | +- Refer to [this blog](/blog/entries/linked-commons-autocomplete-feature/) for the motivation and detailed report on why we added autocomplete aka node suggestions feature. |
| 118 | + |
| 119 | + |
| 120 | +[**Fix filtering module #38**](https://github.com/creativecommons/cccatalog-dataviz/pull/38) |
| 121 | +- Optimizes the build-dB-script to run efficiently on the larger and newer dataset of the cc-catalog. |
| 122 | +- Added the basic form of the randomized graph filtering method. |
| 123 | +- Refer to [this blog](/blog/entries/linked-commons-data-update/) for a piece of detailed information on data update. |
| 124 | + |
| 125 | + |
| 126 | +[**Database upgrade and core enhancements #39**](https://github.com/creativecommons/cccatalog-dataviz/pull/39) |
| 127 | +- It upgrades the primary database from shelve to MongoDB for higher performance. |
| 128 | +- Dockerizes the frontend and backend for both dev and prod environments for higher application portability. |
| 129 | + |
| 130 | + |
| 131 | +[**Frontend enhancements #40**](https://github.com/creativecommons/cccatalog-dataviz/pull/40) |
| 132 | +- Fixes common UI bugs, updates the frontend design and enhances the mobile and smaller devices experience with the linked commons |
| 133 | +- Modularizes and updates the code documentation |
| 134 | + |
| 135 | +<div style="text-align: center; width: 80%; margin-left: 10%;"> |
| 136 | + <figure> |
| 137 | + <img src="design-light.png" alt="Theme Light" style="border: 1px solid black"> |
| 138 | + <figcaption>Linked Commons: Light Theme</figcaption> |
| 139 | + </figure> |
| 140 | +</div> |
| 141 | + |
| 142 | + |
| 143 | +<div style="text-align: center; width: 90%; margin-left: 5%;"> |
| 144 | + <figure> |
| 145 | + <img src="lighthouse-audit.png" alt="Lighthouse Audit" style="border: 1px solid black"> |
| 146 | + <figcaption>Lighthouse Stats of the latest version of the Linked Commons</figcaption> |
| 147 | + </figure> |
| 148 | +</div> |
| 149 | + |
| 150 | + |
| 151 | +### What’s Next? |
| 152 | +Throughout this internship period, we, the Linked Commons team, aimed to make this version the best among all. But there is still scope for improvement. |
| 153 | + |
| 154 | +Just to give you some insights; currently, the complete graph contains 235k nodes and 4.14million links. During the preprocessing, we dropped a lot of the nodes. Additionally, we removed more than 3 million nodes which didn't have cc_licenses information. So, in general, the current version shows only those nodes which are soundly linked with other domains and their licenses information is available. To give a complete picture of the massive "cc-catalog", the linked commons need to "gird up his loins". |
| 155 | + |
| 156 | +After seeing the tremendous potential it has, I will undoubtedly continue working on it and help the linked commons in this quest. ⚡ |
| 157 | + |
| 158 | + |
| 159 | +### Ending Note |
| 160 | +In the end, I would like to thank my mentors Maria and Brent, for their unconditional guidance throughout this internship period. The insights I got from them will truly help me in the days to come. |
| 161 | + |
| 162 | +Special thanks to Francisco, Anna and Kriti for the awesome brainstorming ideas in the UX meet which helped us build an increment superior version of the Linked Commons. |
| 163 | + |
| 164 | +It is not the end, rather a new beginning. Cheers! 🚀🚀🚀 |
0 commit comments