News from Robin and Elizabeth - July 2020

Elizabeth writes: Hello everyone! Payap University has officially re-opened, which means Robin is back to work. This month, we thought we’d share with you a project that he worked on recently.

Here in Thailand, one of our next-door neighbors is the nation of Cambodia. Cambodia is a beautiful place, filled with rice paddie and ancient temples, and even though it’s a small country (about the size of Missouri), it has more than 20 different languages! Recently, one of our members in Cambodia was approached by a major university in Phnom Penh, the capital of Cambodia. This university plans to publish an updated and expanded dictionary of the Khmer (Cambodian) language, and they wanted to know if the Language Forge website could help. Language Forge is a website that our organization developed to help create a dictionary for any language — and Robin has done a lot of work on it. Making a dictionary is a long process, but Language Forge has made it a lot easier! The Khmer language already has a Bible translation, but we wanted to help out anyway in order to foster a good relationship with one of the countries where we work. Our partnership with universities in Cambodia gives us access to work with the minority languages of Cambodia which need Bible translation.

To make a dictionary of a written language, you need a lot of text - the more the better. Lexicographers collect books, letters, emails, speeches, stories, sermons, and more, and they make lists of all the words they find. If you have a large variety of sources, you will find a larger variety of words, and you will get a better idea of all the different ways these words can be used. For example, you might already have the word “season” in your dictionary, defined as “one of the four divisions of the year”, but looking at a cookbook might remind you that “season” can also mean to add flavoring to food. Dictionaries also need lots of example sentences to illustrate how a word is used, and they use this collection of texts to find these examples.

The Royal Academy of Cambodia had dozens of Word documents, each with a list of Khmer words. They also had collected 85,000 articles to glean words from - roughly five and a half million words of text in Khmer! But in order to use Language Forge to organize it all into a dictionary, first they had to upload it all to the website. The website is designed to import data from our linguistic analysis software, but not from Word documents — and they didn’t want to have to retype everything into the website!

Fortunately, Robin could help. He quickly wrote a custom program to take the text from all these different Word documents and change it to a format that could be instantly uploaded to the website. It saved them weeks of work! And now the Khmer dictionary team can use Language Forge to create a quality dictionary.

Robin writes: Hello everyone! If you’re not a computer professional, you’ll probably want to skip this section and scroll down to the pictures below. But I know several of our partners are computer professionals, and you might be interested in the crunchy technical details of what I do. So this month, I figured I’d give you those details.

The Language Forge website is running on the Apache web server, and is hosted in an Amazon EC2 server. The backend uses PHP and MongoDB, and the frontend is using AngularJS. (Yes, AngularJS, not Angular; we’re short on developers — right now I’m the only one on the project — so we haven’t done the upgrade yet). The Send/Receive subsystem is actually a Mercurial repository under the hood. It was originally written in C# for the FieldWorks project, and the way Language Forge works with it is to actually have the PHP code run a C# program as a separate process, and the C# program (which was my main work focus for over a year, about 3-4 years ago) does all the work of pulling data out of MongoDB and making a commit in the Mercurial repo, then doing a Mercurial “push” operation to send it to our central repo server. There are about three different projects involved in the process, only one of which is code that I wrote — so when a bug happens in the Send/Receive process, it can take me a long time to figure out where the bug is. I actually spent the entire month of June hunting down one single bug!

That’s probably enough for now. If you read this section and liked having those details, let me know and I’ll include a “crunchy technical bits” section a little more often.