How to Authorize Your CI/CD to Checkout Additional Repository

Author: Sanhe Hu, Enquizit Inc

In many cases, you need the content of another private repository to test the current repository. For example, your main repository is a public repo, but you need a secret config file from a private repository, how do you securely setup your CI/CD environment?

Our Goal:

1. Your main repo’s CI/CD environment should be able to check out the private repo.
2. Anyone who has access to the public repo should not able to access anything from the private repo

In this article, I use CircleCI for example.

Solution1. (Recommended)

Create a separate GitHub account as a Machine User, add this Machine User as a collaborator to the private repo you need to access to, and use the Machine User’s personal access token to check out the private repo in your CI/CD system.

1. Sign up a Machine User Github account, let’s say, username = MyORG-Machine
2. Create a personal OAuth access token (Account Settings → Developer Settings) with full access of everything in MyORG-Machine
3. Add MyORG-Machine as a collaborator to the private, and only grant MyORG-Machine read-only access to that repo.

Code:

# check out your main repo
$ git clone "https://github.com/MyOrg/main-project.git"
# check out your private repo
$ git clone "https://${TOKEN}@github.com/MyOrg/main-project.git"

Pro:

• Even the OAuth access token has full access to MyORG-Machine, but MyORG-Machine‘s access to MyORG is limited. In other words, the token can’t access any other repository in MyORG.
• You can disable personal OAuth access token anytime if you doubt that the token is leaked.

Con:

• You can’t use the same Machine User for other projects that needs access to a different private repo. Otherwise, if the token used in the first project is leaked, then the hacker can also access the private repo in the second project. You need many Machine User for many projects.

Reference:

• Machine User: https://developer.github.com/v3/guides/managing-deploy-keys/#machine-users
• Enable Your Project to Check Out Additional Private Repositories: https://circleci.com/docs/2.0/gh-bb-integration/#enable-your-project-to-check-out-additional-private-repositories

Solution2. (Easy but Risky)

You can grant CI/CD GitHub User Key of your GitHub account (in CircleCI, it is at Project Settings → Checkout SSH Keys → Authorize with GitHub). In other words, you grant your CI/CD system equivalent power as your GitHub Account.

Pro:

• Convenient and fast.

Con:

• If your CI/CD environment been hacked, then the hacker can do everything you can do.

Reference:

• Share User Key with CircleCI: https://circleci.com/docs/2.0/gh-bb-integration/#security

Solution3. (Secure, but too much work)

1. manually create an ssh key pair, paste the public key to GitHub Menu → Repository Setting → Deploy Key → Add new Key.
2. use a hacky way to include the private key into CI/CD system, put it at $HOME/.ssh.id_rsa_deploy_key or specify that key for git clone command.

Basically, it creates a point-to-point authentication from the machine having your private key to the repository. But securely store, and transfer the private key becomes another difficult problem. And it takes additional effort to manage those key pairs.

Pro:

• Per repo / per ssh level of grained access.

Con:

• Too much work to set up everything, especially managing and injecting the private key.

Reference:

• Generate a new SSH key and adding to your SSH agent: https://help.github.com/en/articles/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent#generating-a-new-ssh-key
• How to specify the private SSH-key to use when executing shell command on Git?: https://stackoverflow.com/questions/4565700/how-to-specify-the-private-ssh-key-to-use-when-executing-shell-command-on-git