When a language you don't understand appears in your Facebook News Feed, you can touch a button and quickly translate it. Facebook offers a way of communicating not just with the millions of people who speak your language, but with millions of others who speak something else. Or at least it almost does. Like so many other online translation services, Facebook comes with a caveat: Its translations don't always make sense.
But like several other giants of the internet, Facebook is working to eliminate that rather significant caveat. This morning, the company's central artificial-intelligence lab released a paper describing a new technology that could accelerate the evolution of machine translation not only inside Facebook but across the internet. According to Facebook's tests, its technique---borrowed from image recognition---produces better translations than the current state of the art, and it does so far more efficiently than other methods, which could eventually lead to even sharper translations.
Christopher Manning, a Stanford University professor who specialized in machine translation and has reviewed the paper, calls it an "impressive achievement," particularly because it can train translation models more quickly than existing systems. And Facebook indicates that its engineers are now rolling this technique onto the company's social network, which serves more than 1.8 billion people across the globe.
Facebook's approach relies on neural networks, complex mathematical systems that can learn tasks by analyzing vast amounts of data. Over the past several years, this general technique has rapidly reinvented everything from image recognition to speech recognition to online search. Now it's overhauling the field of machine translation. This past fall, Google unveiled a new translation system driven entirely by neural networks that topped existing models, and many other companies and researchers are pushing in the same direction, most notably Microsoft and Chinese web giant Baidu.
"We've seen more improvements over the past two years than we have seen in the past decade," says John Tinsley, the CEO of Iconic Translation Machines, a translation technology company based in Dublin.
But Facebook is taking a slightly different tack from most of the other big players. It's using what are called convolutional neural networks, a technique invented by the venerable deep-learning researcher Yann LeCun, who now oversees Facebook's AI lab. Rather than analyze a sentence sequentially, one piece at a time, a convolutional neural network can analyze many different pieces at once, before organizing those pieces into a logical hierarchy.
The convolutional neural network is an old idea that has already proven enormously effective when recognizing objects in photos. And others have explored such networks as a basic technique for machine translation, including researchers at DeepMind, a Google AI lab based in London. But Manning says Facebook's translation system is the most impressive demonstration to date.
Even if the system is only marginally more accurate than systems like the one Google rolled out in the fall, the company says its technique is about nine times more efficient that other neural network-based methods. Convolutional neural networks are better at processing different pieces of a dataset at the same time. "You can have parallel computation on different parts of a sentence," Manning says. "You don't have to push things along word by word."
As a result, Facebook can train its systems with significantly less computing power. According to Jaime Carbonell, the director of the Language Technologies Institute at Carnegie Mellon University, that means it can do more with its available data center hardware and, in theory, push the technique forward far quicker. "In some cases, this is a small advantage," he says. "But it could be huge advantage."
Others may help push the technique forward as well. Like Google before it, Facebook is not only publishing a paper describing its new system but open-sourcing the software engine that drives the system, freely sharing the code with the world at large. It's even sharing models it has already trained on its own data. This is part of a larger effort across the internet's biggest companies to freely share their AI research. It means that translation will evolve far more quickly across the internet---not just on Facebook.
