The SecureDrop project uses CircleCI for running automated test suites on code changes.
The relevant files for configuring the CI tests are the
the main repo, the configuration file at
the scripts in
devops/. You may want to consult the
CircleCI Configuration Reference
to interpret the configuration file. Review the
workflows section of the
configuration file to understand which jobs are run by CircleCI.
The files under
devops/ are used to create a libvirt-compatible environment on GCE.
The GCE host is used as the Ansible controller, mimicking a developer’s laptop,
to provision the machines and run the tests against them.
We skip unnecessary jobs, such as the staging run, for pull requests that only
affect the documentation; to do so, we check whether the branch name begins with
docs-. These checks are enforced in different parts of the configuration,
mainly within the
In CI, we rebase branches in PRs on HEAD of the target branch.
This rebase does not occur for branches that are not in PRs.
When a branch is pushed to the shared
freedomofpress remote, CI will run,
a rebase will not occur, and since opening a
PR does not trigger a re-build,
the CI build results are not shown rebased on the latest of the target branch.
This is important to maintain awareness of if your branch is behind the target
branch. Once your branch is in a PR, you can rebuild, push an additional
commit, or manually rebase your branch to update the CI results.
Running the CI Staging Environment¶
The staging environment tests will run automatically in CircleCI, when
changes are submitted by Freedom of the Press Foundation staff (i.e. members
freedomofpress GitHub organization). The tests also perform
basic linting and validation, like checking for formatting errors in the
You will need a Google Cloud Platform account to proceed. See the Google Cloud Platform Getting Started Guide for detailed instructions.
In addition to a GCP account, you will need a working Docker installation in order to run the container that builds the deb packages.
You can verify that your Docker installation is working by running
docker run hello-world and confirming you see “Hello from Docker” in the
output as shown below:
$ docker run hello-world Hello from Docker! This message shows that your installation appears to be working correctly. ...
Setup Environment Parameters¶
Source the setup script using the following command:
You will be prompted for the values of the required environment variables. There
are some defaults set that you may want to change. You will need to export
GOOGLE_CREDENTIALS with authentication details for your GCP account,
which is outside the scope of this guide.
Use Makefile to Provision Hosts¶
make help to see the full list of CI commands in the Makefile:
$ make help Makefile for developing and testing SecureDrop. Subcommands: ci-go Creates, provisions, tests, and destroys GCE host for testing staging environment. ci-go-xenial Creates, provisions, tests, and destroys GCE host for testing staging environment under xenial. ci-lint Runs linting in linting container. ci-teardown Destroys GCE host for testing staging environment.
To run the tests locally:
You can use
./devops/gce-nested/ci-runner.sh to provision the remote hosts
while making changes, including rebuilding the Debian packages used in the
Staging environment. See Virtual Environments: Servers for more information.