TensorFlow, PyTorch, and Keras: These three deep learning frameworks have dominated AI for years, even as new entrants are gaining momentum. But one framework you haven’t heard much about in the West is Chinese PaddlePaddle, the most popular Chinese framework in the world’s most populous country.

It is an easy-to-use, efficient, flexible and scalable deep learning platform originally developed by Baidu, the Chinese AI giant, to apply deep learning to many of its own products. Today it is used by over 4.77 million developers and 180,000 businesses worldwide. While it’s hard to get comparable numbers for other frameworks, suffice it to say that it’s a lot.

Baidu recently announced new updates to PaddlePaddle as well as 10 big deep learning models that span natural language processing, vision, and computational biology. Models include a 100 billion parameter natural language processing (NLP) model called ERNIE 3.0 Zeus, a pretrained model for geography and language called ERNIE-GeoL, and a pretrained composite representation learning model called HELIX-GEM.

The company also created three new large industry models—one for the electric power industry, one for banking, and another for the aerospace industry—by fine-tuning the company’s ERNIE 3.0 Titan model with industry data and expertise in unsupervised learning tasks. .

Software platforms are packages of related helpers, compilers, code libraries, toolkits, and application programming interfaces (APIs) that enable you to develop a project or system. Deep learning frameworks bring together everything you need to design, train, and test deep neural networks through a high-level programming interface. Without these tools, the implementation of deep learning algorithms would take a long time, because otherwise reusable code fragments would have to be written from scratch.

Baidu started developing such tools back in 2012, months after Jeffrey Hinton’s breakthrough in deep learning at an ImageNet competition.

In 2013, a UC Berkeley doctoral student created a framework called Caffe that supports convolutional neural networks used in computer vision research. Baidu used Caffe to develop PaddlePaddle, which supported recurrent neural networks in addition to CNNs, giving it an NLP edge.

The name PaddlePaddle comes from Parallel Distributed Deep Learning, a reference to the platform’s ability to train models on multiple GPUs.

Google’s open source TensorFlow in 2015 and Baidu’s open source PaddlePaddle the following year. When Eric Schmidt introduced TensorFlow in China in 2017, it turned out that China was ahead of him.

While Meta’s TensorFlow and PyTorch, launched in 2017, remain popular in China, PaddlePaddle is more geared toward industrial users.

“We have put in a lot of effort to lower barriers to entry for individuals and companies,” said Ma Yangjun, general manager of AI ecosystem at Baidu.

PyTorch and TensorFlow require users to have a greater deep learning experience compared to PaddlePaddle, whose toolkits are designed for non-experts in production environments.

“In China, many developers are trying to use AI in their work, but they don’t have much AI experience,” Ma explained. “Therefore, in order to expand the use of AI across industries, we have provided PaddlePaddle with many low-threshold toolkits that are easier to use so that the wider community can use them.”

AI engineers usually know little about industries, and industry experts know little about AI. But the easy-to-understand code of PaddlePaddle comes with a lot of tutorials and tools to help users. It is highly scalable and has a full set of APIs to suit different needs.

3 people crowd around the robot in a desert with small trees and bushes.
These developers used the Desert Robot PaddlePaddle to automate the tree planting process.Baidu

It supports large-scale data training and can train hundreds of machines in parallel. It provides a neural machine translation system, recommender systems, image classification, sentiment analysis, and semantic role labeling.

According to Ma, toolkits and libraries are PaddlePaddle’s forte. For example, PaddleSeg can be used to segment images. PaddleDetection can be used to detect objects. “We cover the entire AI development pipeline from data processing to training, compressing models, adapting to different hardware,” Ma said, “and then how to deploy them on different systems, such as Windows or Linux. operating system either on an Intel chip or on an Nvidia chip.

The platform also contains toolkits for advanced research purposes such as Paddle Quantum for quantum computing models and Paddle Graph Learning for graph-based learning models.

“That’s why PaddlePaddle is very popular in China right now,” he said. “Developers use such toolkits, not just the tool itself.”

Since the source code was open, PaddlePaddle has been rapidly developed to have the best performance and user experience in various industries outside of Baidu, as well as in countries outside of China, thanks to the extensive documentation in English. PaddlePaddle currently offers over 500 algorithms and pre-trained models for the rapid development of industrial applications. Baidu has been working to reduce the size of the models so that they can be deployed in real applications. Some of the models are very small and fast and can be deployed on a camera or mobile phone.

Industrial Application PaddlePaddle

  • Transportation companies are using PaddlePaddle to deploy AI models that control traffic lights and improve traffic efficiency.
  • Manufacturing companies use PaddlePaddle to increase productivity and reduce costs.
  • Recycling companies use PaddlePaddle to develop object detection models that can identify different types of garbage for a garbage sorting robot.
  • Shouguang County in Shandong Province is implementing artificial intelligence to monitor the growth of various vegetables, suggesting the best time for farmers to pick and pack them.
  • In Southeast Asia, PaddlePaddle has been used to control artificial intelligence forestry drones to prevent fires.

PaddlePaddle has parameter server technology for training sparse models that can be used in recommender systems and real-time searches. But he also combined the models into even larger systems that are used for scenarios that do not require real-time results, such as text generation or image generation.

Baidu sees large dense models as another way to reduce the barrier to AI adoption, as the so-called base models can be adapted to specific scenarios. Without a foundation model, you have to design everything from scratch.

Ma said the research areas are moving towards cross-model learning of different modalities such as speech and vision. He said that Baidu is also using knowledge graphs in its deep learning process. “Previously, a deep learning system worked on raw texts or raw images without any input, and the system used self-supervised learning to gather rules outside of the data,” Ma said. “But now we’re looking at knowledge graphs as input.”