Upgrade from 0.5.x to 0.6.x¶
Updating the Tails Workstations¶
All Tails drives should be updated to Tails 3.6, released concurrently with SecureDrop 0.6. For the Secure Viewing Station, Admin Workstation and Journalist Workstation, you need to manually upgrade as explained in the Tails documentation.
For the Journalist Workstations and the Admin Workstation, you will also need to update the SecureDrop code using the following manual method:
cd ~/Persistent/securedrop
git fetch --tags
git checkout 0.6
gpg --recv-key "2224 5C81 E3BA EB41 38B3 6061 310F 5612 00F4 AD77"
git tag -v 0.6 # Output should include "Good signature"
./securedrop-admin setup
From this point forward, you will be able to use the command
securedrop-admin check_for_updates
to check whether updates are available,
and securedrop-admin update
to check for updates and to apply them in a
single step.
Troubleshooting Linux Kernel Issues¶
The latest Linux kernel (version 4.4.115) will be automatically installed on your SecureDrop servers within 24 hours of the 0.6 release (13th of March at 22:00 UTC). If you are using hardware other than our official hardware recommendations, you face moderate risk of a SecureDrop outage.
If you are experiencing an outage as part of the 0.6 release (e.g. you cannot reach the source interface, you cannot SSH into the servers etc.), you will have to perform the steps outlined below.
First, you need to physically access each server. Power down the server (safely if possible), attach required peripherals (keyboard, monitor), and power the server back up. If you did not configure Google Authenticator for console access, you will need to use single user mode in order to login to each server.
Boot into Single-User Mode¶
To access single user mode, you will have to edit the boot options for the new kernel. You can do so using the GRUB bootloader, pictured below:
Press any key quickly just once. You will only have about 2 to 3 seconds before Ubuntu starts booting. If you miss that window, just log in normally and reboot safely, provided you can log in. Do not unplug or forcibly shut down the server.
Once you hit a key, you will be able to interact with the menu with the up (⬆) and down (⬇) keys. Select “Ubuntu” as shown above, and press “e” to edit the boot options. In the line that begins “linux”, replace the word “quiet” with “single”. Note that the word “quiet” may be wrapped, as in the screenshot below:
Press the “F10” key to boot.
Test the New Kernel¶
Observe the boot process. It is possible that the system will fail to boot completely; if so, the log information will help us to understand what is happening.
Provided that you can log in, check if you have network access. Try a
command such as sudo host freedom.press
. If you don’t have network
access, it is most likely due to the upgraded kernel missing a network
driver for your hardware.
If everything appears to be operating normally, the outage may not be kernel-related. In that case, please still follow the steps at the end of this document to send us log information along with an issue report, and we will help you investigate.
If you are experiencing network issues or other kernel problems, we recommend that you roll back to an older kernel, and that you report the issue to us immediately.
Compare the Behavior of the Old Kernel¶
Reboot the server in a safe way with sudo reboot
. After the BIOS screen,
you can select a new kernel from the GRUB boot menu by selecting
Advanced Options for Ubuntu, pictured below.
Under Advanced Options, choose the option with kernel version 3.14.x-grsec
.
As before, you may need to edit the kernel options to enter single user
mode. The boot process should proceed normally. Wait until you get a
login prompt and log in.
Once you are logged in, check to see if you have network access. If you do, then your instance is having an issue with the 4.x kernel. In that case, we need to temporarily set an older kernel as the default.
Roll Back to the Old Kernel¶
Important
The older kernel has reached end of life, so it is of critical importance for the near term security of your instance that we work together to resolve any compatibility issues. Rolling back to an older version is only a stopgap measure to avoid a prolonged outage of your SecureDrop instance.
Inspect the file /boot/grub/grub.cfg
. You should find a menuentry
line
with the same text that you selected during boot, e.g.:
submenu 'Advanced options for Ubuntu'…
menuentry 'Ubuntu, with Linux 3.14.xx-grsec…
Take note of its position among the other submenu entries (it will most likely be third). Then edit the GRUB configuration:
sudo vim /etc/default/grub
Make a backup of the file or take a note of the current value of
GRUB_DEFAULT
somewhere, so you can restore the previous behavior easily at a
later point.
Once you have done so, set the GRUB_DEFAULT
variable to point to the index
of the menu and submenu. Note that the index starts at 0, so for a typical
setup, the line in /etc/default/grub
would look like this:
GRUB_DEFAULT=”1>2”
The “1” means the second entry of the main menu (“Advanced options”), the “2” means the third entry of the submenu. Again, update these numbers consistent with your configuration.
Caution
Ensure that you have chosen the right index for the main menu and the submenu, and double-check that you are beginning the count at 0, not 1; otherwise, you may boot into the wrong kernel.
This change still has to be applied to take effect on the next boot:
sudo update-grub2
Now you can reboot into the old, working kernel.
sudo reboot
The server should come up automatically. From here on, you should be
able to perform all administrative tasks via SSH as usual. If you want
additional confirmation of the kernel version, the command
uname -r
should display 3.14.79-grsec
.
Please notify us of the compatibility issue so we can help you resolve it ASAP.
Report Compatibility Issues¶
If you have encountered issues with the kernel upgrade, it is important that you report them to us so that we may incorporate any necessary changes to our updated kernel, and so that we can work with you to switch back to the new kernel as soon as possible.
Run the following commands via SSH from the Admin Workstation:
cd ~/Persistent/securedrop/
source .venv/bin/activate
cd install_files/ansible-base
ansible all -b -m setup > server-facts.log
Please also send us a copy of /var/log/syslog
and /var/log/dmesg
for
analysis.
You can share server-facts.log
, syslog
and dmesg
with us as follows:
- If you are a member of our Support Portal, please create a new issue and attach the files to it.
- Alternatively, email us at securedrop@freedom.press (GPG encrypted) with the subject “SecureDrop kernel facts” and the files attached.
Once we get your information, we can try to provide assistance to resolve compatibility issues.
Getting Support¶
Should you require further support with your SecureDrop installation or upgrade, we are happy to help!
- Community support is available at https://forum.securedrop.org
- Paid support options are provided by Freedom of the Press Foundation. Contact securedrop@freedom.press for more information.