Yandex Blog

Jane Zavalishina is the CEO of Yandex Data Factory, a spin-off of Yandex founded in 2014 to provide machine learning solutions to enterprises. In this post, Jane explains how YDF’s business has evolved since its launch, and why industrial AI is now in focus of its strategy.

Since its inception, Yandex Data Factory (YDF) has pioneered an innovative way to create value for companies by applying our expertise in machine learning and artificial intelligence (AI) to help solve their business needs. YDF arose as a solution to the problem many businesses faced at the peak of the big data craze. Essentially, businesses had begun amassing huge amounts of information, but were struggling to extract tangible value from this data.

The solution, of course, is in machine learning. Our parent company, Yandex, was an early leader in machine learning technology and today, machine learning powers 70 percent of Yandex’s products and services. We realised that wherever large stores of data exist, so does the opportunity to use that data to reach measurable business improvements. The same algorithms that power Yandex’s services, can be used to help other businesses improve their operations, revenues and profitability.

Over the past two years, we have worked with a number of companies across multiple industries on various successful projects. Together with our clients, we discovered the best use cases where machine learning can be applied to increase the efficiency of existing processes in a measurable way – be it predicting demand for a retail chain, or using computer vision to cut moderation costs for online service. Along the way, we accumulated a huge amount of expertise on merging data science with business.

One such case included our work with Magnitogorsk Iron & Steel Works (MMK), that marked one of the first ever collaborations of its kind between a technology company and steel company. MMK, one of the world’s largest steel producers, wanted to reduce its production costs while maintaining the same high-quality product. YDF developed a machine learning-based service that recommends the optimal amount of ferroalloys—the ingredients needed to produce specific steel grades. Our predictive system demonstrated the reduction of ferroalloy use by an average of five percent, equating to annual savings of more than $4 million in production costs, while consistently maintaining the same high quality of steel.

Similarly, we are now optimising the operations of a gas fractionation unit for a petrochemical company. Our solution recommends the fractionation unit parameters for maintaining the best performance and energy savings, decreasing costs in the process. Last week, we also signed a collaboration agreement with Gazprom Neft, an integrated oil company. We plan to apply our technologies to well drilling and completion, and other production processes. These successful efforts demonstrate the high potential for collaboration between artificial intelligence and industrial manufacturing.

The industrial sector – responsible for one-third of global GDP – has proven to be the ideal vertical, perfectly positioned for the effective application of our technologies. The industrial sector has become YDF’s focal point through the combination of our own successful application of predictive analytics with industrial data and the fit of the industry. Put simply, manufacturers know the value of optimisation at their hearts. Industrial manufacturing is also a unique cultural fit. They value measurements above opinions, they have perfected integrating new technologies in the existing processes, and they know how to estimate their effect through properly designed experiments.

For decades, the cornerstone of competitiveness in manufacturing has been centered on the optimization of existing processes, reaching for each tenth of a percent of efficiency in each step. And when all traditional optimisation means have been applied, the next efficiency leap of five to ten percent is often prohibitively expensive and equally time-consuming. These improvements typically consist of equipment upgrades with multi-million dollar investments, years spent on construction, rigorous training and implementation, and a lengthy delay before seeing any tangible financial return. Compared to this, receiving the same level of optimization via machine learning in a matter of months with minimal upfront investment is nothing short of revolutionary.

These long-term benefits extend far beyond a simple profit and loss sheet, and can help conserve both human capital and natural resources. By training machines to focus on the mundane, routine decisions that keep a factory running, artificial intelligence and machine learning allow human employees time to tackle more important tasks. By applying these technologies to oil and gas, companies not only achieve time and material savings, they can also reduce their energy consumption by up to 25 percent.

Our AI-enhanced models create endless opportunities to add value to the manufacturing industry. These benefits are especially noticeable in process manufacturing, where materials and mixtures – metals, chemicals, etc. – are produced. Essentially, these are also the industries responsible for the highest resource consumption.

The AI revolution in manufacturing is happening right now, and we are thrilled to be leading the charge. As this future becomes a reality, we’ll be there – at the forefront – blazing new trails in the industrial sector and delivering far-reaching effects for both the companies we work with and the larger communities they serve.

Yandex’s on-demand transportation service Yandex.Taxi unveils its autonomous car project. The prototype of a self-driving car the company has developed is a step towards a comprehensive set of driverless technologies for application across a wide range of industries.

The driverless car incorporates Yandex’s own technologies some of which, such as mapping, real-time navigation, computer vision and object recognition, have been functioning in a range of the company’s services for years. The self-driving vehicle’s ability to ‘make decisions’ in complex environments, such as busy city traffic, is ensured by Yandex’s proprietary computing algorithms, artificial intelligence and machine learning.

“Self-driving cars are set to revolutionalise the way we commute within a matter of a decade,” says Dmitry Polishchuk, head of Yandex.Taxi Self-Driving Project. “At this point in time, there are dozens of companies around the world building their own driverless cars, but only a few of them have components crucial for turning this project into reality. These components include a stack of reliable technologies and algorithms, engineering expertise and resources, and access to the market for self-driving vehicles. Yandex.Taxi, with the backing of Yandex, is one of the few players who can boast of possessing all of the above.”

Yandex.Taxi’s effort in developing the self-driving car technology aims at creating a fully-fledged autopilot functionality, which is described as Level 5, according to the currently universally accepted classification system for automated vehicles. This system classes all self-driving cars into levels from 0 to 5, where Level 0 means a person has full control over the vehicle, and Level 5 involves no human intervention.

Yandex.Taxi will push on with experimenting and honing the self-driving technology, together with improving maps, navigation and route planning implemented in this project. Tests on public roads are expected to kick off next year.

With Yandex.Taxi test-driving the self-driving service, Yandex looks forward to partnering with car manufacturers and other companies interested in taking the autonomous car technology to the road.

Yandex proudly announces the creation of our new Machine Intelligence and Research (MIR) Division. The MIR division will function as a centralized, cross-functional unit to accelerate innovation and unify our core machine learning technologies. The MIR division will also transfer cutting-edge research from our various research teams into Yandex products and services. Yandex has tapped Misha Bilenko to head the new division, which brings together a mix of teams focusing on AI-centered technologies including:

• MatrixNet and DaNet – Machine learning has always been at the core of Yandex consumer products and information services. In 2009, we launched MatrixNet, our proprietary machine learning platform. Today, MatrixNet is used in nearly every product and service Yandex offers. One important feature of MatrixNet is its resistance to overfitting, which takes into account a very large number of factors when ranking the relevancy of search results. DaNet is the deep neural network (DNN) framework developed at Yandex that provides state-of-the-art runtime performance for many tasks that rely on deep learning.
• Computer Vision – People learn to recognize objects at a very young age. Machines, on the other hand, must be trained to recognize objects using vast amounts of labeled and unlabeled data. Yandex’s market-leading image recognition technology uses machine learning to detect similar images in visual search results as well as perform a number of high-end vision tasks, from automotive photo analysis for auto.ru, to predicting weather patterns using satellite imagery.
• Speech – Yandex’s SpeechKit voice recognition technology uses machine learning to help people better communicate with devices and be more productive on the go. SpeechKit technology powers voice commands for Yandex search and is also used in Yandex’s traffic information app, Yandex.Navigator, offering motorists voice activation control. The SpeechKit SDK enables businesses to easily integrate Yandex’s speech technologies in their productivity tools and virtual assistants.
• Translation – With more than 90 languages in production, Yandex is one of very few companies in the world that has access to enough data to meet today’s high machine translation standards. Yandex.Translate uses machine learning throughout its stack, including unique technology for translating rare languages that don’t have enough written data to use classical methods, instead relying on linguistic structures from related popular languages to fill in the gaps.

From speech-to-speech translation to virtual assistants that chat with people and use cameras to see, the MIR division offers amazing opportunities for synthesis and cross-pollination within Yandex’s machine learning, computer vision, speech and translation technologies. By bringing team members from these core technologies together, the MIR division will improve Yandex’s machine and natural processing capabilities, enhancing its products and services and ultimately delivering consumers and businesses a better experience.

Under Misha Bilenko’s guidance, the unified division will be able to integrate its top research findings across all of Yandex products and services. Misha joins Yandex after 10 years of experience working at Microsoft, where he led the Machine Learning Algorithms team in the Cloud and Enterprise division, following a career in the Machine Learning Group for Microsoft Research. Misha brings a unique blend of leadership skills, research expertise and machine learning knowledge to Yandex. His leadership will be instrumental as the MIR division expands Yandex’s research efforts to experiment with new projects and achieve more long-term goals building the next generation of intelligent products and services.

Yandex builds personalised content recommendation technology Zen into Yandex Browser on all platforms in 24 countries and 15 languages. Based on the latest developments in artificial intelligence research, Zen recommendation technology uses the company’s vast global web index to pick stories, images, videos and other content for each individual user and offer it them right in the new tab of Yandex Browser.

The intelligent content discovery feed in Yandex Browser delivers recommendations based on the user’s location, browsing history, their viewing history and preferences in Zen, among hundreds of other factors. Zen uses natural language processing and computer vision to understand the verbal and visual content on the pages the user has viewed, liked or disliked, to offer them the content they are likely to like. Yandex’s recommendation technology Disco, based on the company’s machine-learning algorithm, MatrixNet, helps Zen choose which suggestions to offer to the user at any given point in time. Targeted to identify the user’s personal long-term interests and cater to them, Zen also delivers content not directly related to their immediate preferences. The more the user interacts with Zen, the better are the chances that they will see serendipitously interesting content.

‘With all the vastness of information available on the internet, something genuinely interesting isn’t easy to come by. Zen helps solving this problem,’ says Victor Lamburt, head of Yandex Zen. ‘It points each user to what’s interesting specifically to them. This is the future for all web browsers: providing personal internet experience and helping people discover something new’.

The infinite personally targeted content feed in Yandex Browser gives web users an opportunity to discover something they appreciate, but wouldn’t have found it otherwise. To start exploring this new internet experience, all one needs to do is download Yandex Browser and give Zen some browsing history to work with. Alternatively, liking or disliking a few websites on Zen’s start up page will help it understand your preferences on the outset. Users can also alter the type or topic of content they are offered later on by choosing to view more of similar content, less of it, or block specific sources altogether.

Zen first appeared as an experimental feature in Yandex’s launcher app for Android in Mexico and Brazil in 2015. The average time the users spent viewing Zen’s recommended content has increased since then from only 5 minutes to 20 minutes in May 2016. Zen is currently available both in Yandex Launcher and Yandex Browser for iPhone, Android mobile devices and Windows PC and laptops.

Yandex’ personal content recommendation technology can also be easily integrated into third-party mobile applications, such as browsers or launcher apps, and offers great monetisation potential for OEMs, app developers, and mobile carriers.