With gettext, text to be translated is specially marked in source code. A Python example:
if not (msg or fh): flash(gettext("You must enter a message or choose a file to submit."), "error") return redirect(url_for('main.lookup'))
In this code, the string
You must enter a message or choose a file to
submit. can be automatically extracted for translation. The
gettext function to which it is passed is used as a marker by
pybabel or similar tools to collect the
strings to be translated and store them into a .pot file at
securedrop/translations/messages.pot. For instance:
#: source_app/main.py:111 msgid "You must enter a message or choose a file to submit." msgstr ""
The .pot file serves as a template for all the language-specific
.po files, which are where Weblate stores the contributed
translations. For each language to be translated, a directory is
created, such as
securedrop/translations/fr_FR, and populated with
a .po file derived from the template. For instance,
securedrop/translations/fr_FR/LC_MESSAGES/messages.po is almost
securedrop/translations/messages.pot except for the
msgstr fields, which will contain the French translations, e.g.:
#: source_app/main.py:111 msgid "You must enter a message or choose a file to submit." msgstr "Vous devez saisir un message ou sélectionner un fichier à envoyer."
There’s one last type of file in the gettext system, a machine-readable version of the .po translations called a .mo file. Applications use these to get translations at runtime. The .po files are compiled to .mo files when the SecureDrop package is installed.
The Weblate web application is used to translate strings and relies on gettext behind the scenes. It owns the .pot and .po files. When preparing a SecureDrop release, a pull request is created to pull in all the translations that have been updated in Weblate.
The desktop icons installed on SecureDrop workstations are also
translated. The icon templates are in the
Their labels are collected in the
desktop.pot file and translated
in the corresponding .po files in the same directory (
de.po etc.). All translations are merged from the
files into the corresponding
*.j2 file and committed to the
SecureDrop repository. They are then installed when configuring Tails
Most of the work in managing translations within the SecureDrop code
base is supported by
securedrop/i18n_tool.py. It provides
convenient wrappers around pybabel and gettext , and is used to
update strings to be translated; pull translations from Weblate; to
compile translations before running tests and while packaging
Add a new language¶
SecureDrop only supports a subset of all the languages being worked on
in Weblate: some of them are partially translated or not fully
reviewed. The list of fully supported languages is hard-coded in the
i18n_tool.py file, in the
SUPPORTED_LANGUAGES variable. When a
new language is completely translated and reviewed, the
i18n_tool.py file must be manually edited to add this new language
Update strings to be translated¶
After strings are modified in the SecureDrop source code, templates or
desktop labels, the
must also be updated. Individual developers should not do this
whenever changing strings in the code; the translations are updated in
bulk when it’s time to update the Weblate fork.
Translations can be updated with the following command:
$ make translate
i18n_tool.py translate-messages and
translate-desktop. These commands update the .pot files for the
SecureDrop server code and the desktop icons, as well as the .po
files for each language.
The new source strings will only be visible to translators
in Weblate after the
develop branch is merged into the
Merge develop into the Weblate fork¶
Weblate works on a long standing fork of the SecureDrop git
repository and is exclusively responsible for the content of the
*.po files. The content of the
must be merged into the
i18n branch to make updated source strings
available to Weblate.
Translation must be suspended in Weblate, and any uncommitted changes committed and pushed, to avoid conflicts:
- Go to the Weblate repository page for SecureDrop.
- And finally, click
develop branch can now be merged into
$ git clone https://github.com/freedomofpress/securedrop $ cd securedrop $ git remote add i18n email@example.com:freedomofpress/securedrop-i18n.git $ git fetch i18n $ git checkout -b i18n i18n/i18n $ git merge origin/develop $ make translate
translate Makefile target uses the
i18n_tool.py command to
*.po files in sync with the SecureDrop
source code. After running
make translate, carefully
review the output of
git diff. Check
messages.pot first for
updated strings, looking for formatting problems. Then review the
messages.po of one existing translation, with a focus on
translations. There is no need to review other translations because
they are processed in the same way. When you are satisfied with the
result, it can be merged with:
$ git commit -a -m 'l10n: sync with upstream origin/develop' $ git push i18n i18n
- Go to the Weblate commit page for SecureDrop and verify the
commit hash matches the last commit of the
i18nbranch. This must happen instantly after the branch is pushed because Weblate is notified via a webhook. If it is different, ask for help.
Weblate pushes the translations done via the web interface to the develop branch in a fork of the SecureDrop git repository. These commits must be manually cherry-picked and proposed as pull requests for the SecureDrop git repository.
Merge translations back to develop¶
Weblate automatically pushes the translations done via the web
interface as a series of commits to the
i18n branch in the
Weblate SecureDrop branch, which is a fork of the
branch of the SecureDrop git repository. These translations need to
be submitted back to the
develop branch via pull requests. When
you create a branch for this, begin its name with
i18n-, as that
prefix triggers special CI tests for translations.
To fetch the latest translations from the
i18n branch into your
working copy of the SecureDrop repository, run these commands in your
$ git checkout -b i18n-merge origin/develop $ securedrop/bin/dev-shell ./i18n_tool.py --verbose update-from-weblate $ securedrop/bin/dev-shell ./i18n_tool.py --verbose update-docs
You now have the latest translations on your
It is very important to check that each translated string looks like a plausible translation, with no markup. Even if the reviewer does not understand the language, if a translated string looks strange, someone other than the reviewer must be consulted to verify it means something. It is extremely unlikely that a contributor will manipulate a translated string to introduce a vulnerability in SecureDrop, but any suspicious translation should be investigated.
To check the new translations, you’ll need to compile them and verify them by running our automated tests and, ideally, by checking them in the SecureDrop source and journalist interfaces.
$ securedrop/bin/dev-shell ./i18n_tool.py --verbose translate-messages --compile
For the desktop icons of the source and journalist interfaces, compilation updates their template files with all the translations collected from the .po files.
This can be done by running the following command:
$ securedrop/bin/dev-shell ./i18n_tool.py --verbose translate-desktop --compile
SecureDrop web interfaces¶
After a translation is compiled, the web page in which it appears can
be verified visually by starting the SecureDrop development servers
and navigating via
http://localhost:8080 for the source interface
http://localhost:8081 for the journalist interface. You can
start the development servers with:
$ make dev
The translations can be checked automatically by running the SecureDrop page layout tests:
$ export PAGE_LAYOUT_LOCALES="en_US,fr_FR" # may be set to any supported languages $ make test TESTFILES=tests/pageslayout [...] tests/pageslayout/test_journalist.py::TestJournalistLayout::test_account_edit_hotp_secret[en_US] PASSED tests/pageslayout/test_journalist.py::TestJournalistLayout::test_account_edit_hotp_secret[fr_FR] PASSED [...]
if unset, PAGE_LAYOUT_LOCALES defaults to en_US (US English) and ar (Arabic).
After running the tests, screenshots for each locale are available
filenames can be found in the tests that created them, in
The translated templates for the desktop icons are:
Check that each of them contains a
Name line for each of SecureDrop’s supported locales.
Push your branch and create a pull request¶
After you’ve checked the translations, you’re ready to push your
i18n-merge branch and create a pull request to get the
translations merged to the SecureDrop
If there have been multiple commits per language, as can happen if source strings need to be translated again after being changed to correct critical errors, or to incorporate suggestions from the source string feedback period, they should be combined via an interactive rebase. Reorder the commits to group them by language, then squash the commits for each language into one. The goal is to end up with one commit per supported language on the merge branch.
When you’re happy with the state of language commits on your merge branch:
$ git commit -m "l10n: compile desktop icons' translations" # if needed $ git push i18n-merge
The CI job
translation-tests will automatically run the
above page layout tests in all supported languages on
branches named with the prefix
i18n-. If you’ve followed
that naming convention, the translation tests should soon be
run on your pull request.
If you have an abundance of time, you can run all the translation tests locally with:
$ make translation-test
And at long last, you’re done. Go to https://github.com/freedomofpress/securedrop and propose a pull request.
Unlike the SecureDrop application translations, the desktop
icon translations are compiled and merged into the
repository. They need to be available in their translated
securedrop-admin tailsconfig is run, because
the development environment is not available.
Update Weblate screenshots¶
You can use the script
securedrop/upload_screenshots.py to update
UI screenshots that are used to illustrate strings in Weblate. The script
depends on the existence of up-to-date layout test results, which you can
generate using this command in the base directory:
$ LOCALES=en_US make translation-test
Inspect the screenshots in the directory
and make sure that their content corresponds to the expected version of the
Obtain your API key
in Weblate. Export the token to the environment variable
You can now run this command to perform an upload:
If new screenshots were added as part of this run, make sure to associate them with relevant strings in Weblate, which you can do from the screenshots list.
Two weeks before the release: string freeze¶
When features for a new SecureDrop release are frozen, the localization manager for the release will:
- Merge develop into the Weblate fork.
- Update Weblate screenshots so translators can see new or modified source strings in context.
- Update the i18n timeline in the translation section of the forum.
- Post an announcement to the translation section of the forum (see an example).
- Remind all developers about the string freeze in Gitter.
- Add a Weblate announcement with the translation timeline for the release.
- Create a pull request for every source string suggestion coming from translators.
Correct acknowledgment of translators’ contributions is important, so
i18n_tool.py makes it easy to list the translators who have helped
since the last merge of Weblate translations, with
list-translators. A list of everyone who has ever contributed
translations to SecureDrop can be obtained with
list-translators --all. There are
Makefile targets for these,
$ make list-all-translators ar: A. Nonymous Ahmad Gharbeia Ahmed Essam Ali Boshanab [...]
A translation admin has special permissions on Weblate and the repositories. When someone is willing to become an admin, a thread is started in the translation section of the forum. If there is consensus after a week, the permissions of the new admin are elevated. If there is not yet consensus, a public vote is organized among the current admins.
The privileges of an admin who has not been active for six months or more are revoked, but they can apply again at any time.
The community of SecureDrop translators works very closely with the SecureDrop developers and some of them participate in both groups. However, the translator community has a different set of rules and permissions, and therefore independent policies from SecureDrop itself.
The full set of admin permissions can be granted at:
- https://weblate.securedrop.org/admin/weblate_auth/user/ (grant staff and superuser status)
- https://forum.securedrop.org/admin/users/list/active (click on the user and
- https://github.com/freedomofpress/securedrop-i18n (make sure that the user has commit access)
Granting reviewer privileges in Weblate¶
- Visit https://weblate.securedrop.org/admin/weblate_auth/user/.
- Click on the user name.
- In the
Available groupslist and click on the right arrow to move it to the
Chosen groupslist and click on the left arrow to remove it.
- In the
Update the Weblate full text index¶
$ ssh firstname.lastname@example.org $ cd /app/weblate $ sudo docker-compose run weblate rebuild_index --all --clean
Note that the new index may not be used right away. Some workers may still have the old index open. If the index is holding up translators with a release looming, the server can be rebooted.