Testing: CI

The SecureDrop project uses multiple automated third-party solutions for running automated test suites on code changes:

Travis tests

The Travis test suite provisions the development VM and runs the application test suite against the latest version of the code. It also performs basic linting and validation, e.g. checking for mistakes in the Sphinx documentation (see Documentation Guidelines).

CI test layout

The relevant files for configuring the CI tests are:

├── .circleci <--- folder contains config for CircleCI
├── devops
│   ├── inventory <-- environment specific inventory
│   ├── playbooks <-- playbooks to start CI boxes
│   ├── scripts   <-- shell wrapper scripts
│   ├── templates <-- contains templates for ansible tasks
│   └── vars <-- environment specific variables
├── .travis.yml  <--- config for development tests on travis
└── Makefile  <-- defines make task shortcuts

The files under devops/ are used to create a minimized staging environment on AWS EC2. The CircleCI host is used as the Ansible controller to provision the machines and run the Testing: Configuration Tests against them.

Running the CI staging environment

The staging environment tests will run automatically in CircleCI, when changes are submitted by Freedom of the Press Foundation staff (i.e. members of the freedomofpress GitHub organization).

Tip

You will need an Amazon Web Services EC2 account to proceed. See the AWS Getting Started Guide for detailed instructions.

In addition to an EC2 account, you will need a working Docker installation in order to run the container that builds the deb packages.

You can verify that your Docker installation is working by running docker run hello-world and confirming you see “Hello from Docker” in the output as shown below:

$ docker run hello-world

Hello from Docker!
This message shows that your installation appears to be working correctly.
...

Setup environment parameters

Source the setup script using the following command:

source ./devops/scripts/local-setup.sh

You will be prompted for the values of the required environment variables. There are some defaults set that you may want to change. You will need to determine the value of your VPC ID to use which is outside the scope of this guide.

Use Makefile to provision hosts

Run make help to see the full list of CI commands in the Makefile:

$ make help
Makefile for developing and testing SecureDrop.
Subcommands:
    docs: Build project documentation in live reload for editing.
    docs-lint: Check documentation for common syntax errors.
    ci-spinup: Creates AWS EC2 hosts for testing staging environment.
    ci-teardown: Destroy AWS EC2 hosts for testing staging environment.
    ci-run: Provisions AWS EC2 hosts for testing staging environment.
    ci-test: Tests AWS EC2 hosts for testing staging environment.
    ci-go: Creates, provisions, tests, and destroys AWS EC2 hosts
           for testing staging environment.
    ci-debug: Prevents automatic destruction of AWS EC2 hosts on error.

To run the tests locally:

make ci-debug # hosts will not be destroyed automatically
make ci-go

You can use make ci-run to provision the remote hosts while making changes, including rebuilding the Debian packages used in the Staging environment. See Virtual Environments: Servers for more information.

Note that if you typed make ci-debug above, you will have to manually remove a blank file in ${HOME}/.FPF_CI_DEBUG and then run make ci-teardown to bring down the CI environment. Otherwise, specifically for AWS, you will be charged hourly charges until those machines are terminated.