Investigating Logs

When troubleshooting issues with your SecureDrop instance, be sure to examine all relevant log files on both servers. To work with logs, it is helpful to be familiar with commands like less, tail and grep; to inspect older, archived logs (names end with .gz) you can use commands like zless and zgrep to avoid manually decompressing each file.

Note

You can use the securedrop-admin tool to extract logs to send to Freedom of the Press Foundation for analysis. Run the following command on your Admin Workstation:

cd ~/Persistent/securedrop
./securedrop-admin logs

This command will produce encrypted tarballs containing logs from each server. See the command output for more information.

Logs to examine on both servers

  • /var/log/kern.log: Use this file to investigate kernel-related issues, including warnings or errors specific to AppArmor or grsecurity (a set of patches applied to the kernels for additional security hardening)

  • /var/log/syslog: Use this file to investigate most other system issues, including iptables configuration problems or Tor network issues. Use search patterns, e.g., search for “app Tor” to find log entries specific to Tor.

Application Server Logs

See the directory /var/log/apache2/* for web server access and error logs. In production systems, logging is only enabled for the Journalist Interface to the files journalist-access.log and journalist-error.log, and the logs do not contain IP address information.

When investigating an application error on the Source Interface (e.g., if you see an “Internal Server Error” when submitting a document), it can make sense to temporarily enable error logging. To do so:

  1. Log into your Application Server from your Admin Workstation via ssh app

  2. Edit the file /etc/apache2/sites-enabled/source.conf (requires sudo)

  3. Comment out the old ErrorLog and LogLevel directives, e.g., like so:

# Enabling logging for error investigation, 2020-04-18, ~admin
#
# ErrorLog /dev/null
# LogLevel critical

  1. Add the desired new logging configuration in the same location (inside the <VirtualHost> block), e.g.:

ErrorLog /var/log/apache2/source-error.log
LogLevel debug

  1. Save the file and reload the configuration with sudo service apache2 reload

  2. Visit the Source Interface and reproduce the error

  3. Inspect the log file /var/log/apache2/source-error.log for any details

  4. Remember to set the configuration back to the default values once your investigation is complete.

Note that the debug logging level is highly verbose; if you want to adjust it, see the Apache documentation for more information about the different logging levels.

If you encounter an application error, and you have not modified the application code, please be sure to file an issue or contact us via securedrop@freedom.press (GPG encrypted).

Monitor Server Logs

  • /var/ossec/logs/ossec.log: Examine this file to investigate problems with OSSEC itself not functioning as expected (e.g., you are not seeing alerts when you would expect them to).

  • /var/ossec/logs/alerts/alerts.log: This file contains the most recent alerts generated by OSSEC. If you have correctly configured OSSEC emails, the text of these alerts should correspond to the text of the emails.

  • /var/log/mail.log and /var/log/procmail.log: These files contain information about email delivery and email processing (for encrypting the alerts). Investigate these files if you believe OSSEC is correctly configured, but you are not receiving emails.