Site Reliability Engineering / DevOps SaltStack configuration files

CODE_OF_CONDUCT.md:

The Creative Commons team is committed to fostering a welcoming community. This project and all other Creative Commons open source projects are governed by our Code of Conduct. Please report unacceptable behavior to conduct@creativecommons.org per our reporting guidelines.

See CONTRIBUTING.md.

• Avoid insecure repository clones: This repository includes encrypted secrets. Do not run git-crypt unlock on clones that are not otherwise secured (ex. strong login password, disk encryption).
• Avoid editing the base environment: The base environment is configured to prevent commit and push actions. Please use your development environment and pull the changes to base.
• Sign your commits:
  • The main branch (default branch) has the Require signed commits (Include administrators) GitHub branch protection enabled.
    • About required commit signing - User Documentation
  • Ensure you are using RemoteForward in your SSH configuration to forward your GnuPG agent to salt-prime (see the example configuration, under Setup, below).
  • Ensure you have configured your newly cloned repository to sign commits (see the git config command, under Setup, below).

• SSH connection information: example local/laptop ~/.ssh/config configugration:

  Host bastion-us-east-2
      HostName bastion-us-east-2.creativecommons.org
      User ARTHUR
  
  Host salt-prime
      HostName 10.22.11.11
      ProxyJump bastion-us-east-2
      RemoteForward /run/user/4242/gnupg/S.gpg-agent /Users/ARTHUR/.gnupg/S.gpg-agent.extra
      User ARTHUR
  
  Host *
      ServerAliveCountMax 60
      ServerAliveInterval 30
      TCPKeepAlive no
  

  • Assumes remote username ARTHUR and remote uid 4242. Replace these values in your own local/laptop configuration.
  • ProxyJump allows you to ssh salt-prime from your local/laptop.
  • RemoteForward allows you to sign your commits.
• Setup your development repository on salt-prime:
  1. Clone repository to /srv with your username. For example:

     cd /srv
     git clone git@github.com:creativecommons/sre-salt-prime.git ${USER}

  2. Setup your newly cloned repository.
     1. Configure commit signing:

        cd /srv/${USER}
        git config user.email YOUR_EMAIL
        git config user.signingkey YOUR_GPG_ID
        git config commit.gpgsign true

     2. Unlock encrypted secrets:

        cd /srv/${USER}
        git-crypt unlock

  3. Specify the environment when you test changes. For example:

     sudo salt \* state.highstate saltenv=${USER} test=True

     • use --state-verbose=True to see successes
     • use --state-output=full_id to see full detail of successes
     • use --log-level=debug --log-file-level=warning to see debug messages (without logging those debug messages, which may contain secrets, to the log file)

• Use AWS well, but avoid technologies that create AWS lock-in (ex. Confidant)
• Salt Prime must not contain any exclusive data (use Git)
• Git repository must not contain any unencrypted secrets
• Git repository commits must be signed and applied to the main branch via Pull Requests
• A compromised minion must not be able to escalate access
  • SysAdmins must not forward their SSH agent
  • Must not reuse application passwords (ex. Prod and Dev databases must have different passwords)
  • Pillar data must be restricted by Minion ID based classification
    • The only grain which can be safely used is grains['id'] which contains the Minion ID. (FAQ Q.21)

• Amazon Web Services (AWS)
  • Creative Commons is already using it and staff are familiar with it
  • Features allow security (ex. screened subnets, security groups policies)
  • Features allows Infrastructure as Code
  • us-east-2
    • cost effective
    • avoid conflict/collision over region limited resources (ex. ElasticIPs)
• Debian 11 (Bullseye), Debian 10 (Buster), and Debian 9 (Stretch)
  • Free/Open Source
  • Debian Stable
  • Creative Commons is already using it and staff are familiar with it
• git-crypt - transparent file encryption in git
  • Free/Open Source
  • Performance: files are decrypted in the checked out repository
  • Security: automatic encryption and directory based filters minimize the chance of unencrypted secrets being pushed to GitHub
• SaltStack
  • Free/Open Source
  • Performance
  • Creative Commons is already using it and staff are familiar with it
  • Version: 3005.2
    • For current targeted minion version, see minion_target_version in pillars/salt/init.sls

Minions are added and configured from salt-prime with the following Minion ID schema: HST__POD__LOC (host/role__pod/group__location). These variables are used to determine the state and pillar data.

Show top states example command:

sudo salt \* pillar.item states saltenv=${USER}

See docs/Host_Classification.md for details.

See docs/Orchestration.md.

• For the SaltStack version that this repository is developed on, see Decisions, above.
• This repository attempts to tracks the most current release of SaltStack in the SaltStack Debian repository: https://repo.saltstack.com/apt/debian/
• For current version of SaltStack in Debian proper, see Debian -- Package Search Results -- salt-master

• Hardening Salt
  • The only grain which can be safely used is grains['id'] which contains the Minion ID. (FAQ Q.21)
• Salt Best Practices
• Salt Formulas

• Salt Module Reference
  • state modules
• Requisites and Other Global State Arguments

• README.md
• bootstrap-aws/README.md
• docs/
  • Host_Classification.md
  • Orchestration.md
  • WordPress.md

• saltstack-formulas/mysql-formula: Install the MySQL client and/or server
• saltstack-formulas/php-formula: Formula to set up and configure php

• LICENSE (Expat/MIT License)