Refactored data pushes to CCOS to use a single script.

Author: kgodey

.github/workflows/community_teams_list_page.yml

@@ -1,10 +1,11 @@
-name: Generate Community Team Members Page
+name: Push data to CC Open Source
 on: 
   schedule:
   - cron: '0 0 * * *'
   push:
     branches:
       - master
+      - refactor_ccos_updates
 jobs:
   build:
     runs-on: ubuntu-latest
@@ -18,12 +19,12 @@ jobs:
       run: |
         python -m pip install --upgrade pip
         python -m pip install pipenv
-        cd search_roadmap_export
+        cd push_data_to_ccos
         pipenv install
     - name: Export token to env and run script
       run: |
-        cd search_roadmap_export
-        pipenv run python search_roadmap_export.py
+        cd push_data_to_ccos
+        pipenv run python sync.py
       env:
         ADMIN_GITHUB_TOKEN: ${{ secrets.ADMIN_GITHUB_TOKEN }}
         ADMIN_ASANA_TOKEN: ${{ secrets.ADMIN_ASANA_TOKEN }}

.github/workflows/generate_projects_page.yml

@@ -12,35 +12,25 @@ information about that script.
 | Directory Name                    | Script Purpose                                                                  |
 | --------------------------------- | ------------------------------------------------------------------------------- |
 | [`normalize_repos`][norm]         | Ensures that all CC repos have standard labels and branch protections           |
-| [`generate_projects_page`][proj]  | Generates the ["Projects" page on CC Open Source][osproj]                       |
-| [`community_teams_list_page`][ct] | Generates the ["Community Teams Members" page on CC Open Source][ctlistpage] |
-| [`search_roadmap_export`][search] | Exports the `CC Search Roadmap` Asana project to opensource.creativecommons.org |
+| [`push_data_to_ccos`][push_to_ccos]  | Generates various pages with dynamic data on [CC Open Source][ccos] |
 
 
 ## Workflows
 | YML File Name                   | Workflow Purpose                                                        |
 | -------------------------------- | --------------------------------------------------------------------- |
 | [`add_community_pr.yml`][community_pr_yml]        |  Runs hourly at 5 minutes past every hour UTC and add community PRs to [Active Sprint – CC Search](https://github.com/orgs/creativecommons/projects/7) |
-| [`community_teams_list_page.yml`][community_teams_yml]        | Runs daily at 00:00 UTC and whenever someone pushes to master branch and uses [`community_teams_list_page`][ct] |
-| [`generate_projects_page.yml`][gen_proj_yml]        | Runs daily at 00:00 UTC and whenever someone pushes to master branch and uses [`generate_projects_page`][proj] |
 | [`normalize_repos.yml`][norm_pr_yml]        |  Runs daily at 00:00 UTC and whenever someone pushes to master branch and uses [`normalize_repos`][norm]   |
-| [`search_roadmap_export.yml`][search_roadmap_yml]        | Runs daily at 00:00 UTC and whenever someone pushes to master branch and uses [`search_roadmap_export`][search] |
+| [`push_data_to_ccos.yml`][push_ccos_yml]        | Runs daily at 00:00 UTC and whenever someone pushes to master branch and uses [`push_data_to_ccos`][push_to_ccos] |
 
 
 [community_pr_yml]:.github/workflows/add_community_pr.yml
-[community_teams_yml]:.github/workflows/community_teams_list_page.yml
-[gen_proj_yml]:.github/workflows/generate_projects_page.yml
 [norm_pr_yml]:.github/workflows/normalize_repos.yml
-[search_roadmap_yml]:.github/workflows/search_roadmap_export.yml
+[push_ccos_yml]:.github/workflows/push_data_to_ccos.yml
 
-[ct]: community_teams_list_page/
 [norm]:normalize_repos/
-[proj]:generate_projects_page/
-[search]:search_roadmap_export/
-
-[ctlistpage]: httpe://opensource.creativecommons.org/community/community-teams/members
-[osproj]:https://opensource.creativecommons.org/contributing-code/projects/
+[push_to_ccos]:push_data_to_ccos/
 
+[ccos]: httpe://opensource.creativecommons.org/
 
 
 ## License

.github/workflows/search_roadmap_export.yml renamed to .github/workflows/push_data_to_ccos.yml

@@ -6,9 +6,11 @@ verify_ssl = true
 [dev-packages]
 
 [packages]
-pygithub = "*"
 asana = "*"
+pygithub = "*"
 gitpython = "*"
+emoji = "*"
+pyyaml = "*"
 
 [requires]
 python_version = "3.7"

README.md

@@ -0,0 +1,45 @@
+# Push Data to CCOS
+
+This folder contains a script that pushes the latest version of dynamically generated data to CC Open Source. This data updates the following pages:
+* [CC Search page on CC Open Source][ccsearch] (roadmap portion)
+* [Community Team Members page on CC Open Source][ctlistpage]
+* [Projects page on CC Open Source][osproj].
+
+[ccsearch]: https://opensource.creativecommons.org/cc-search/
+[ctlistpage]: httpe://opensource.creativecommons.org/community/community-teams/members/
+[osproj]:https://opensource.creativecommons.org/contributing-code/projects/
+
+This script is run via GitHub Actions automatically. 
+
+## Running the Script
+
+1. Install [Pipenv](https://pipenv.readthedocs.io/en/latest/)
+2. Navigate to the `push_data_to_ccos` folder and run `pipenv install`
+3. Set the `ADMIN_GITHUB_TOKEN` environment variable with your GitHub token. You will need a GitHub token with admin permissions to the `creativecommons` GitHub organization.
+4. Set the ADMIN_ASANA_TOKEN environment variable with your Asana token. You will need an Asana token with access to the Creative Commons Asana organization.
+5. Run the script: `python3 sync_data.py`
+
+## About the Code
+
+### Repository data
+
+Repository data is generated in `get_repo_data.py` and pulls data on CC repositories using the GitHub API.
+
+### CC Search Roadmap Data
+
+Repository data is generated in `get_search_roadmap_data.py`. This pulls tasks from the `CC Search Roadmap` Asana project in CC's Asana organization, 
+does some basic cleaning and filtering, and adds the tasks to a template.
+
+### Community Team Data
+
+Community Team member data is generated from the `Community Teams Tracking` Asana project in CC's Asana organization.
+
+### Pushing to GitHub
+
+Data is pushed to the `[creativecommons.github.io-source](https://github.com/creativecommons/creativecommons.github.io-source)` repository in the `push_data_via_git.py` file.
+
+
+## Repository README
+
+See repository README for licensing information, etc:
+- [`../README.md`](../README.md)