With gettext, text to be translated is specially marked in source code. A Python example:
if not (msg or fh): flash(gettext("You must enter a message or choose a file to submit."), "error") return redirect(url_for('main.lookup'))
In this code, the string
You must enter a message or choose a file to
submit. can be automatically extracted for translation. The
gettext function to which it is passed is used as a marker by
pybabel or similar tools to collect the
strings to be translated and store them into a .pot file at
securedrop/translations/messages.pot. For instance:
#: source_app/main.py:111 msgid "You must enter a message or choose a file to submit." msgstr ""
The .pot file serves as a template for all the language-specific
.po files, which are where Weblate stores the contributed
translations. For each language to be translated, a directory is
created, such as
securedrop/translations/fr_FR, and populated with
a .po file derived from the template. For instance,
securedrop/translations/fr_FR/LC_MESSAGES/messages.po is almost
securedrop/translations/messages.pot except for the
msgstr fields, which will contain the French translations, e.g.:
#: source_app/main.py:111 msgid "You must enter a message or choose a file to submit." msgstr "Vous devez saisir un message ou sélectionner un fichier à envoyer."
There’s one last type of file in the gettext system, a machine-readable version of the .po translations called a .mo file. Applications use these to get translations at runtime. The .po files are compiled to .mo files when the SecureDrop package is installed.
The desktop icons installed on SecureDrop workstations are also
translated. The icon templates are in the
Their labels are collected in the
desktop.pot file and translated
in the corresponding .po files in the same directory (
de.po etc.). All translations are merged from the
files into the corresponding
*.j2 file and committed to the
SecureDrop repository. They are then installed when configuring Tails
At the start of the release process, the
localization manager collects string changes on the
in the main SecureDrop repository and merges them to the
branch of the securedrop-i18n repository. The changes will then
appear in Weblate, and translation can begin. At the end of the
translation period, the localization manager collects the changes to
the PO files on
securedrop-i18n/i18n in a pull request for
securedrop/develop. Once that pull request is merged, the
translations are backported to the release branch in the main
Most of the work in managing translations within the SecureDrop code
base is supported by
securedrop/i18n_tool.py. It provides
convenient wrappers around pybabel and gettext , and is used to
update strings to be translated; pull translations from Weblate; to
compile translations before running tests and while packaging
Add a new language¶
SecureDrop only supports a subset of all the languages being worked on
in Weblate: some of them are partially translated or not fully
reviewed. The list of fully supported languages is hard-coded in the
i18n_tool.py file, in the
SUPPORTED_LANGUAGES variable. When a
new language is completely translated and reviewed, the
i18n_tool.py file must be manually edited to add this new language
Update strings to be translated¶
Whenever strings are modified in the SecureDrop source, whether in
Python code, HTML templates, or desktop icon labels, the translation
files should also be updated by running
make translate in the root
of the SecureDrop working copy.
translate target runs
i18n_tool.py translate-messages and
i18n_tool.py translate-desktop, which in turn use
extract to gather source strings. These commands update the .pot
files for the SecureDrop server code and the desktop icons, as well as
the .po files for each language.
make translate, carefully review the output of
securedrop/messages.pot first for updated strings,
looking for problems like:
- overly idiomatic English
- fragmented text, such as pieces of a sentence intended to be concatenated together, which can be difficult to translate
- messages that are marked with plain
gettextand contain plurals based on numeric placeholder variables – these should generally be marked with
ngettextso that they can be translated properly in languages with complex plural forms
Then review the
messages.po of one existing translation, with a
focus on new translations, which are often marked fuzzy. There
is no need to review multiple languages’
.po files because they
are processed in the same way.
Once you’ve reviewed the changes, submit them in a pull request for
develop branch in the main SecureDrop repository.
The new source strings will only be visible to translators in
Weblate after they’ve been merged to
that branch has been merged into
localization manager does this at the beginning of our release cycle.
Merge develop into the Weblate fork¶
2) Then, translation must be suspended in Weblate, and any uncommitted changes committed and pushed, to avoid conflicts:
- Go to the Weblate repository page for SecureDrop.
- And finally, click
securedrop/developbranch can now be merged into
$ git clone https://github.com/freedomofpress/securedrop $ cd securedrop $ git remote add i18n email@example.com:freedomofpress/securedrop-i18n.git $ git fetch i18n $ git checkout -b i18n i18n/i18n $ git merge origin/develop $ git commit -a -m 'l10n: sync with upstream origin/develop' $ git push i18n i18n
- Verify that Weblate has the latest changes, and unlock the repository.
- Go to the Weblate commit page for SecureDrop and verify the
commit hash matches the last commit of the
i18nbranch. This must happen instantly after the branch is pushed because Weblate is notified via a webhook. If it is different, ask for help.
Translation can now begin. As translators make progress, Weblate
pushes the translations done via the web interface in commits to the
i18n branch of the securedrop-i18n repository (a fork of the
main SecureDrop repository). When the translation period ends,
these commits will be collected into a pull request for the main
Merge translations back to develop¶
Weblate automatically pushes the translations done via the web
interface as a series of commits to the
i18n branch in the
securedrop-i18n repository, which is a fork of the
branch of the main SecureDrop repository. These translations need
to be submitted back to the
securedrop/develop branch via pull
requests. When you create a branch for this, begin its name with
i18n-, as that prefix triggers special CI tests for translations.
To fetch the latest translations from the
securedrop-i18n/i18n branch into your
working copy of the SecureDrop repository, run these commands in your
$ git checkout -b i18n-merge origin/develop $ securedrop/bin/dev-shell ./i18n_tool.py --verbose update-from-weblate $ securedrop/bin/dev-shell ./i18n_tool.py --verbose update-docs --docs-repo-dir /path/to/documentation
You now have the latest translations on your
It is very important to check that each translated string looks like a plausible translation, with no markup. Even if the reviewer does not understand the language, if a translated string looks strange, someone other than the reviewer must be consulted to verify it means something. It is extremely unlikely that a contributor will manipulate a translated string to introduce a vulnerability in SecureDrop, but any suspicious translation should be investigated.
To check the new translations, you’ll need to compile them and verify them by running our automated tests and, ideally, by checking them in the SecureDrop source and journalist interfaces.
$ securedrop/bin/dev-shell ./i18n_tool.py --verbose translate-messages --compile
For the desktop icons of the source and journalist interfaces, compilation updates their template files with all the translations collected from the .po files.
This can be done by running the following command:
$ securedrop/bin/dev-shell ./i18n_tool.py --verbose translate-desktop --compile
SecureDrop web interfaces¶
After a translation is compiled, the web page in which it appears can
be verified visually by starting the SecureDrop development servers
and navigating via
http://localhost:8080 for the source interface
http://localhost:8081 for the journalist interface. You can
start the development servers with:
$ make dev
The translations can be checked automatically by running the SecureDrop page layout tests:
$ export PAGE_LAYOUT_LOCALES="en_US,fr_FR" # may be set to any supported languages $ make test TESTFILES=tests/pageslayout [...] tests/pageslayout/test_journalist.py::TestJournalistLayout::test_account_edit_hotp_secret[en_US] PASSED tests/pageslayout/test_journalist.py::TestJournalistLayout::test_account_edit_hotp_secret[fr_FR] PASSED [...]
if unset, PAGE_LAYOUT_LOCALES defaults to en_US (US English) and ar (Arabic).
After running the tests, screenshots for each locale are available
filenames can be found in the tests that created them, in
The translated templates for the desktop icons are:
Check that each of them contains a
Name line for each of SecureDrop’s supported locales.
Push your branch and create a pull request¶
After you’ve checked the translations, you’re ready to push your
i18n-merge branch and create a pull request to get the
translations merged to the SecureDrop
If there have been multiple commits per language, as can happen if source strings need to be translated again after being changed to correct critical errors, or to incorporate suggestions from the source string feedback period, they should be combined via an interactive rebase. Reorder the commits to group them by language, then squash the commits for each language into one. The goal is to end up with one commit per supported language on the merge branch.
When you’re happy with the state of language commits on your merge branch:
$ git commit -m "l10n: compile desktop icons' translations" # if needed $ git push i18n-merge
The CI job
translation-tests will automatically run the
above page layout tests in all supported languages on
branches named with the prefix
i18n-. If you’ve followed
that naming convention, the translation tests should soon be
run on your pull request.
If you have an abundance of time, you can run all the translation tests locally with:
$ make translation-test
And at long last, you’re done. Go to https://github.com/freedomofpress/securedrop and propose a pull request.
Unlike the SecureDrop application translations, the desktop
icon translations are compiled and merged into the
repository. They need to be available in their translated
securedrop-admin tailsconfig is run, because
the development environment is not available.
Update Weblate screenshots¶
You can use the script
securedrop/upload_screenshots.py to update
UI screenshots that are used to illustrate strings in Weblate. The script
depends on the existence of up-to-date layout test results, which you can
generate using this command in the base directory:
$ LOCALES=en_US make translation-test
Inspect the screenshots in the directory
and make sure that their content corresponds to the expected version of the
Obtain your API key
in Weblate. Export the token to the environment variable
You can now run this command to perform an upload:
If new screenshots were added as part of this run, make sure to associate them with relevant strings in Weblate, which you can do from the screenshots list.
Two weeks before the release: string freeze¶
When features for a new SecureDrop release are frozen, the localization manager for the release will:
- Merge develop into the Weblate fork.
- Update Weblate screenshots so translators can see new or modified source strings in context.
- Update the i18n timeline in the translation section of the forum.
- Post an announcement to the translation section of the forum (see an example).
- Remind all developers about the string freeze in Gitter.
- Add a Weblate announcement with the translation timeline for the release.
- Create a pull request for every source string suggestion coming from translators.
Correct acknowledgment of translators’ contributions is important, so
i18n_tool.py makes it easy to list the translators who have helped
since the last merge of Weblate translations, with
list-translators. A list of everyone who has ever contributed
translations to SecureDrop can be obtained with
list-translators --all. There are
Makefile targets for these,
$ make list-all-translators ar: A. Nonymous Ahmad Gharbeia Ahmed Essam Ali Boshanab [...]
A translation admin has special permissions on Weblate and the repositories. When someone is willing to become an admin, a thread is started in the translation section of the forum. If there is consensus after a week, the permissions of the new admin are elevated. If there is not yet consensus, a public vote is organized among the current admins.
The privileges of an admin who has not been active for six months or more are revoked, but they can apply again at any time.
The community of SecureDrop translators works very closely with the SecureDrop developers and some of them participate in both groups. However, the translator community has a different set of rules and permissions, and therefore independent policies from SecureDrop itself.
The full set of admin permissions can be granted at:
- https://weblate.securedrop.org/admin/weblate_auth/user/ (grant staff and superuser status)
- https://forum.securedrop.org/admin/users/list/active (click on the user and
- https://github.com/freedomofpress/securedrop-i18n (make sure that the user has commit access)
Granting reviewer privileges in Weblate¶
- Visit https://weblate.securedrop.org/admin/weblate_auth/user/.
- Click on the user name.
- In the
Available groupslist and click on the right arrow to move it to the
Chosen groupslist and click on the left arrow to remove it.
- In the
Update the Weblate full text index¶
$ ssh firstname.lastname@example.org $ cd /app/weblate $ sudo docker-compose run weblate rebuild_index --all --clean
Note that the new index may not be used right away. Some workers may still have the old index open. If the index is holding up translators with a release looming, the server can be rebooted.