Mozilla PO Tools
Reference documentation
This file is work in progress being put together by Javier Solá (www.khmeros.info).
For information about the tools of the Translate project, please visit http://translate.sourceforge.net/
Applies to Translate Tools version 0.8rc. Last changed on 11/23/2004
The Translate project tools are a package of utilities designed to facilitate the translation of the OpenOffice application suite and of applications belonging to the Mozilla family (Mozilla suite, Firefox, Thunderbird, etc.) from English to other languages.
In all these applications, the messages that need to be translated to the target language are in specific files. These tools convert these files into a standard format (PO format) that is normally used for translation. The advantage of this is that there are special editors to translate PO files, making the work of the translators very easy, seeing the original and the translation at the same time, knowing how much work has been done, saving all messages in a database (translation memory) that can be used in other files, etc. Once the files have been translated by the translators, these tools are used again to pack the translated files into a format that Mozilla or OpenOffice can understand.
The tools are divided in three blocks: OpenOffice tools, Mozilla tools and generic support tools.
The Mozilla tools are formed by two programs:
moz2po - Converts a Mozilla English Language Pack (XPI file) to PO format files.
po2moz - Converts PO files (generated by moz2po and translated) back to Mozilla Language Pack format
The equivalent programs for OpenOffice are:
oo2po - Converts OpenOffice GSI (SDF) files to PO format files.
po2oo - Converts PO files (generated by oo2po and translated) back to GSI format.
Support tools are:
pofilter - Finds possible errors in PO files
PoMerge - Merges translated messages from a PO file into another PO file.
This program is part of the Translate tools, a package designed to facilitate the translation of Open Source applications to new languages. Tools within the package are used to convert the Mozilla type files which contain translatable messages (.dtd and .properties files) to PO format (a standard file format used in Open Source). PO files can be easily translated using specialized editors. Then these tools are once again used to pack the translated files into a format that Mozilla can understand (a XPI Language Pack). They can be used with any of the Mozilla family applications (Firefox, Thunderbird, Mozilla, etc.).
The moz2po program unpacks an original Mozilla Language pack (a file with the .xpi extension) into a directory, converting the translatable (.dtd and .properties) files into equivalent standard PO files, while also making available to the translator other files that require changes (in order to localize the Language pack).
As input it takes a Mozilla American English Language Pack. This Language pack is a .XPI file (a zip format compressed file that includes all the files that need to be translated or localized).
As output it produces a whole new tree (a folder with subfolders inside) that contains all the files from the original Language pack (except for .dtd and .properties files, which have been translated to PO format). The folder produced contains the following subfolders:
reg |
contains PO files that need to be translated (which were originally in the US.jar file, inside the .xpi file) and other files to be localised |
lang-reg |
contains PO files that need to be translated (which were originally in the en-US.jar file) and other files to be localised |
win |
contains PO files that need to be translated (originally in the en-win.jar file) and other files to be localised. |
mac |
contains PO files that need to be translated (originally in the en-mac.jar file) and other files to be localised. |
unix |
contains PO files that need to be translated (originally in the en-unix.jar file) and other files to be localised. |
package |
contains other files from the .xpi file that need to be localised. It follows the structure of the original XPI file |
Other folders |
may contains PO files that need to be translated and other files to be localised. |
Template |
contains a copy of the original English .xpi file, used later by po2moz to build the .xpi file for the target language. Do not touch it. |
moz2po can also be used to convert to PO format a language pack that is already translated. As the language pack for your language does not have the English original messages (needed for the PO file), it also needs to use an original English language pack (en-US.xpi) as a template, so that it will have all the information needed for the PO files. This conversion (to PO format) permits the use of the information contained in the Mozilla files with a Translation Memory tool that uses PO files as input.
moz2po other use is to convert any tree of files that includes .dtd files and .properties files into another tree with identical structure that includes all the same files of the original tree, except that all these .dtd and .properties files are converted to PO files with the same name and with the to .dtd.po and .properties.po extensions.
This last application is useful for the new localization packs used in Mozilla products for CVS based localization. The Mozilla Foundation produces ZIP files that include a tree of files identical to the tree used for localization of a given language in the Mozilla-product CVS. Once localized, this packaged can be directly uploaded to CVS, and the Mozilla Foundation will produce the local language executable packages, as well as the installers.
This function must be used only once, before starting the translations. If it is used again once some of the PO files have been translated, it will overwrite the translated files with empty PO files, having to start the translation again.
Note: the format of the generated PO files is UTF-8, which allows this files to be translated to any language that uses Unicode. When translated back to ,dtd and .properties formats, .dtd files are generated in UTF-8 format, and .properties files in Escaped Unicode format, as required by Mozilla applications.
Different syntax is given for the different uses of the program:
Conversion of an original English XPI file into a tree of .po files and other localizable files
usage: moz2po [options] INPUT OUTPUT
Arguments (mandatory): | |
INPUT | INPUTis the filename (xpi extension included)of an original English language pack (an XPI file) belonging to an application of the Mozilla family (Mozilla, Firefox, Thunderbird, Camino). |
OUTPUT | OUTPUT is the name of the directory inside which all the subfolders with the equivalent PO files, and all other information will be placed. If the folder itself does not exist, it will be created by moz2po. If the folder exists already, files inside might be overwritten. |
Options: | |
-P | Produce PO template (.pot) files instead of Po files (.po), but the contents of the file is exactly the same. (note: .pot files are sometimes interpreted by MS Windows as PowerPoint template files, pay no attention to this). |
--version | Show program's version number and exit |
-h | Show a help with the program’s usage syntax and exit |
Examples:
moz2po –i en-US.xpi –o locale
This command creates - based on the en-US.xpi original English pack - the tree described above under the locale name, containing the structure of the package (in the locale\package subfolder), a copy of the en-US.xpi file in the locale\template sub-folder (to be used for repackaging), and other subfolders that contain PO files that need to be translated and other files which might need to be localised (locale\reg, locale\lang-reg, locale\win, etc.).
Conversion (migration) of an already translated language pack into a PO files and other localized files
usage: moz2po [options] -i INPUT -o OUTPUT –t TEMPLATE
Arguments (mandatory): | |
INPUT | INPUT is the filename (xpi extension included) of a language pack of a given Mozilla family application that has already been translated into your language. It must have exactly the same structure as the original English pack used as TEMPLATE. |
OUTPUT | OUTPUT is the name of the directory inside which all the subfolders with the equivalent PO files, and all other information will be placed. If the folder itself does not exist, it will be created by moz2po. If the folder exists already files inside might be overwritten. |
TEMPLATE | TEMPLATE is an original English language pack (an XPI file) for the same Mozilla family application (and same program version) than the package that is used as INPUT. |
Options: | |
--version | Show program's version number and exit |
-h | Show a help with the program’s usage syntax and exit |
Example:
moz2po -i km-KH.xpi -o locale -t en-US.xpi
This program takes the Khmer language pack for a Mozilla application (km-KH.xpi) and the original US English language pack for the same application (and version, en-US.xpi) and creates a directory of PO files that have in the original string (msgid) the English original and in the translation string (msgstr) the Khmer translation.
This command creates - based on the km-KH.xpi file (an XPI language pack for Khmer language) and the en-US.xpi original English pack - the tree described above under the locale name, containing the structure of the km-KH.xpi package (in the locale\package subfolder), a copy of the en-US.xpi file in the locale\template sub-folder (to be used for repackaging), and other subfolders that contain PO files that already contain the translated strings from km-KH.xpi and other files which have already been localised in km-KH.xpi (such as contents.rdf files) These directories are named locale\reg, locale\lang-reg, locale\win, etc.
Conversion a tree that contains .dtd. and .properties files into a similar tree in which these files are turned into PO files.
usage: moz2po [options] INPUT OUTPUT
Arguments (mandatory): | |
INPUT | INPUTis the name of the folder that contains the tree of files that need to be converted to PO. |
OUTPUT | OUTPUT is the name of the folder in which the output tree with all the PO files will be created. If the folder exists already files inside might be overwritten. |
Options: | |
-P | Produce PO template (.pot) files instead of Po files (.po), but the contents of the file is exactly the same. (note: .pot files are sometimes interpreted by MS Windows as PowerPoint template files, pay no attention to this). |
--version | Show program's version number and exit |
-h | Show a help with the program’s usage syntax and exit |
Example:
moz2po –i EnglishTree –o localeTree
This instruction copies all the EnglishTree folder and its contents into localeTree, maintaining the subfolder structure. All files are copied to the second tree, except for .dtd and .properties files, which are replaced in the localeTree folder by their equivalent PO files with extensions .dtd.po and .properties.po
This program is part of the Translate tools, a package designed to facilitate the translation of Open Source applications to new languages. Tools within the package are used to convert the Mozilla type files which contain translatable messages (.dtd and .properties) to PO format (a standard file format used in Open Source). PO files can be easily translated using specialized editors. Then these tools are once again used to pack the translated files into a format that Mozilla can understand (a XPI Language Pack). IT can be used with any of the Mozilla family applications (Firefox, Thunderbird, Mozilla, etc.).
Once the user has created a tree of localizable files using moz2po, and translated and localized everything that was required, po2moz takes all this tree again and converts it to a Language pack for your language.
The name of the locale of a language pack is used often inside the language pack. This name (<LanguageCode-RegionCode>, as en-US, km-KH, af-ZA, etc.) is passed to po2moz as an argument, and po2moz takes care of inserting it in file names or folder names in the destination pack when this is necessary.
po2moz can also be used to convert any tree of files that includes .dtd.po files and .properties.po files to another identical tree that includes the all the same files of the original tree, except that all these PO files are converted to .dtd and .properties files. This is useful for the new localization packs used in Mozilla products for CVS based localization. The Mozilla Foundation produces ZIP files that include o tree of files identical to the tree used for localization of a given language in the Mozilla-product CVS. The tree in these packs is converted to a tree that contains PO files using the moz2po tool, and after translation and localization, the tree is converted again with po2moz into a tree that contains translated .dtd and .properties files... and which can be uploaded directly into CVS.
In order to assure that all the original information from the original .dtd or .properties files is put into the newly generated file, it uses as templates the original .dtd and .properties files that are in the original folder from which the .po files where generated. Files in the target directory are re-written if the program is run more than once, but this does not have any side-effects, as the files are generated always from the PO files.
Note: PO files include the “&” character to specify that a given shortcut can be used (it does not always indicate this, when it followed by a word and a semicolon it indicates a variable name). The “&” indicates that the next character can be used as a shortcut. This shortcut character can be changed when editing to PO file, and a different character can be used for the short. If the “&” character is not present in the translation, po2moz will automatically include the old shortcut in the final .dtd file.
Different syntax is given for the different uses of the program:
Creation of an XPI file from localized PO files:
usage: po2moz [options]-i INPUT -o OUTPUT -l LOCALE
Arguments (mandatory): | |
INPUT | INPUT is the folder that contains all the information that will go into the XPI file. This folder was created by moz2po, and, as reference, is the folder that contains the subfolders reg, reg-lang, package, template, etc. |
OUTPUT | OUTPUT is the name of the XPI file (including the .xpi extension) to be generated with all the translated and localised files. |
LOCALE | LOCALE is the locale used in the new XPI file. It has the format xx-RR, where xx is the two character ISO-639 code for the target language (if there is a two character code, otherwise it would be xxx-RR, where xxx is three letter character code for the language), and RR is the two letter ISO-3166 code for the country or region. See http://www.loc.gov/standards/iso639-2/ and http://www.iso.org/iso/en/prods-services/iso3166ma/index.html if you need more information. |
Options: | |
- x EXCLUDED | EXCLUDED can be a file or a folder name. If it is a file, it is not included in the XPI file, if it is a folder, any files or subfolder that are contained in the EXCLUDED folder are not included in the XPI file. This can be used when there is additional information in the original XPI that is not desired in the final XPI, such as CVS information. |
--version | Show program's version number and exit |
-h | Show a help with the program’s usage syntax and exit |
Example:
po2moz -i locale -o firefox1.0RC.km-KH.xpi -l km-KH
It takes the (already created with moz2po and translated) PO files contained in the locale directory, converts them to the original .dtd and .properties formats and with them it creates a language pack called firefox1.0RC.km-KH.xpi for the target language. In this package, locale sensible dile names and directory names use KH as the region name and km-KH as the full locale name when needed.
Creation of a tree that contains translated .dtd. and .property files from localized PO files:
usage: po2moz [options] -i INPUT -o OUTPUT -t TEMPLATE
Arguments (mandatory): | |
INPUT | INPUT is the folder that contains, directly or in subfolders, translated .dtd.po and .properties.po files, as well as other files needed for localization. |
OUTPUT | OUTPUT is the name of folder in which the program will create a folder identical to the INPUT folder, except that .dtd.po files will be converted to .dtd files and .properties.po files will be converted to .properties files. |
TEMPLATE | TEMPLATE is the original directory from which the INPUT directory was created (by using the moz2po program). It includes the original .dtd and .properties files in English. It is used to assure that the generated .dtd and .properties files are identical in structure to the original ones (to include information that could not be stored in the PO files). |
Options: | |
- x EXCLUDED | EXCLUDED can be a file or a folder name. If it is a file, it is not included in the XPI file, if it is a folder, any files or subfolder that are contained in the EXCLUDED folder are not copied to the OUTPUT folder. This can be used when there is additional information in the original folder that is not desired in the finalone, such as CVS information. |
--version | Show program's version number and exit |
-h | Show a help with the program’s usage syntax and exit |
Example:
po2moz -i locale -o firefox1.0RC-kmKH -t firefox1.0RC-enUS
It takes the (already translated) PO files contained in the locale directory, converts them to the original .dtd and .properties formats (with the help of the files contained in the original firefox1.0RC-enUS folder) and creates a new folder called firefox1.0RC-kmKH, identical in structure to locale and to firefox1.0RC-enUS folders, in which instead of PO files there are translated .dtd and .properties files.
Given a PO file, or a set of PO files, the pofilter program tests if the entries of the(se) file(s) fulfill certain conditions of correctness. The incorrect entries (each entry being composed of a msgid original string and a msgstr translated string) are saved into a new file, so that entries in this file can be corrected and then merged back into the PO file (by using the program pomerge).
For each entry considered as not correct by pofilter, it will include a standard PO entry in the output file, preceded by a comment explaining the type of error detected and by a comment identifying the translation as fuzzy. Such message may something like:
#_whitespace: whitespace at the beginning or end of the string did not match
#, fuzzy
#_test: test description
PO filter can be applied to a single file, to all the PO files in a directory or to all the PO files in a given tree. The output will take the same shape as the input, creating a file, a directory or a tree of files.
There are quite a number of filters that may be used in the program, each one of them testing for a given condition of correctness. The user may select which filters he wishes to use. These filters are:
· blank - Checks whether a translation is totally blank
· doublequoting - Checks whether double quoting is consistent between the two strings
· endpunc - Checks whether punctuation at the end of the strings match
· escapes - Checks whether escaping is consistent between the two strings
· numbers - Checks whether numbers of various forms are consistent between the two strings
· purepunc - Checks that strings that are purely punctuation are not changed
· short - Checks whether a translation is much shorter than the original string
· simplecaps - Checks the capitalisation of two strings isn't wildly different
· singlequoting - Checks whether single quoting is consistent between the two strings
· startpunc - Checks whether punctuation at the beginning of the strings match
· unchanged - Checks whether a translation is basically identical to the original string
· untranslated - Checks whether a string has been translated at all
· variables - Checks whether variables of various forms are consistent between the two strings
· whitespace - Checks whether white spaces at the beginning and end of the strings match
usage: python pofilter [options] -i INPUT -o OUTPUT
Arguments:
-i INPUT INPUT can be either the filename of a PO file that needs to be checked or the name of a directory that contains PO files that need to be checked. If it is the name of a folder and the option –R is not used, INPUT will only refer to the files in that folder. If the –R (recursion) option is used; it will refer to all PO files in that folder and any other folders contained in the folder (in the tree of which this folder is the root).
-o OUTPUT In case INPUT is the name of a file; OUTPUT must be the name of the file to which the errors will be written. The file does not need to exist before. If it exists it will be erased before the filtering process starts. If INPUT is a directory, OUTPUT also needs to be a directory to which error files will be written to. If the –R (recursion) option is used, other folder may be created inside if they are needed but do not exist. Files created by pofilter in OUTPUT when INPUT is a directory will have the same names as the original INPUT files.
Options:
-R Recurse subdirectories. When INPUT is a folder, apply the filters also to the files contained in the subfolders of that folder.
--review Include elements marked for review (re-writing). Entries of PO files that have been translated and submitted can be earmarked for review (re-writing) by a first reviewer. While reviewing translations, reviewers sometimes pick up subtle errors, such as bad word choice or places where the translator has incorrectly maintained English language). If the reviewer thinks that the entry has to be checked, he/she will include in the PO file (before the msgid) a line of the form:
#, review - some comment…
If the review option is used, and such line appears in an entry, the entry will be copied to the OUTPUT file.
--ignorereview Don't include elements marked for review...
--mozilla Use the standard checks for Mozilla translations. This adjusts the filter to work from files generated from Mozilla files, assuring that the & mark is used for shortcuts, etc.
--openoffice Use the standard checks for OpenOffice translations. Same for OpenOffice.
--gnome Use the standard checks for Gnome translations. Same for Gnome.
--excludefilter NOFILTER NOFILTER is either a filter o a list of filters separated by spaces (using the names listed above). All filters, except the ones included in NOFILTER will be used by the program. This can be used several times in the same command.
-t YESFILTER YESFILTER is either a filter o a list of filters separated by spaces (using the names listed above). If this option is used, only filters included in YESFILTER will be used during the test.
--progress=PROGRESS Shows progress of program in different ways. PROGRESS may be either: simple, console or verbose. When set to simple, it only shows a set of dots in the screen, one for each file translated. console produces a status line that shows what percentage of the total task has already been accomplished. verbose shows the above mentioned status bar, but is also shows –one by one- the names of the files being translated. The use of this argument is optional. By default the program uses the console output.
-l List available filters and exit.
--version Show program's version number and exit
-h Show this help message and exit
This function takes translated messages from a PO file and used them to fill the translation field (msgstr) of another PO file that has at least some identical messages in English (which need to be translated the same way). It is useful for updating translations to new version of a product. The PO file for the new product usually has almost all the messages of the old version, but it also has new messages. This utility fill in the old translations in the new file, leaving only empty the fields for translations that did not exist before, and therefore helping translators avoid having to retype the translations.
It is also used – in conjunction with pofilter - to correct errors in PO files. Pofilter will select the entries that have possible errors and place them in a separate file. Once these errors are corrected, pomerge can be use to automatically integrate (replace) the corrected translations in the old document.
For each entry in each target PO file (in the TEMPLATE folder), the program looks if there is an entry with identical original English string (msgid) in the file of the INPUT directory that has the same name. If it finds it, the replacement entry from the INPUT directory file will be used in the file written to the OUTPUT directory. If no entry is found in the INPUT directory file, the entry from the TEMPLATE directory file will be used.
pomerge can be used in two different ways, depending on if you want to create a new directory in which the resulting PO files are stored (without touching either the original PO files or the corrections), or if you want to directly modify the original PO files: if the TEMPLATE and OUTPUT folders are different, then the files in the TEMPLATE directory will not be affected, and new files will written to the OUTPUT directory. If the TEMPLATE and OUTPUT folders are identical, then the files in that FOLDER will be modified and corrected with the corrections made in the INPUT folder.
usage: python pomerge [options]-i INPUT -o OUTPUT -t TEMPLATE
Arguments (mandatory usage):
-i INPUT INPUT is the folder that contains all PO files with correct (or corrected) translations. This data will be used to modify the PO files of the same names that are placed in the TEMPLATE directory, placing the result in the OUTPUT directory.
-o OUTPUT OUTPUT is the name of the folder in which the PO files with all the correct data (correct translations from the INPUT and TEMPLATE folders) will be placed. If OUTPUT is the same directory as TEMPLATE, the files will be overwritten with the corrections.
-t TEMPLATE TEMPLATE is the directory that contains the PO files that either need correction or that have empty msgstr fields that need to be filled. The files in the TEMPLATE directory will not be altered, unless the TEMPLATE and OUTPUT folders are the same
Options:
--version Show program's version number and exit.
-h Show a help message and exit.
Examples :
pomerge -i errors/ -o existing_files/ -t existing_files
After generating an error directory by using the pofilter utility on the existing_files directory; and then correcting the mistakes in the files of the errors directory, this function merges the corrections back into the files of the existing_files directory (overwrites them, as the template and output directories are identical).