Interview: The future of AI will be products and services based on lifelong learning

Pankaj Gupta and Korbinian Spann talk about the future of Natural Language Processing

Dr. Korbinian Spann
Dr. Korbinian Spann
27. October 2020
Dr. Pankaj Gupta has earned his PhD in Computer Science with a focus on deep learning and NLP (Natural Language Processing). He spent six years at Siemens working as a Lead Research Scientist and published more than 15 patents and 20 publications. He has founded his own startup, DRIMCo GmbH in Munich. For Insaas, Pankaj works as a technology and science advisor.

Insaas provides a software platform to develop customer centric services and products based on data. Our vision is a world with only custom products and services fit for use instead of mass products: ecological friendly, efficiently and automatically produced, meaningful for employees. To summarise all the voices in different languages and topics Insaas is building an artificial intelligence that is capable of lifelong learning. This solution will improve to understand human sentiment and preference better and better over time.

Korbinian: Why do you think lifelong learning will make the difference for artificial intelligence?

Pankaj: Lifelong learning is crucial in this era of AI. And is one of the current topics that I am working on. As we humans do, this system keeps accumulating knowledge, reuse the past knowledge for future learning and also minimise forgetting of the past learning.

The classical machine learning systems are built in isolation, it means they run on a given dataset for a particular task and therefore, they do not retain and accumulate knowledge they have learned over time. And those systems can not model tasks in sequence due to this catastrophic forgetting.

In comparison, Lifelong Learning systems are designed to retain and reuse interconnected knowledge over lifetime. An example would be: if I have learned to formulate a sentence in English, it does not mean that I forget the characters of the alphabet. I would still remember them and in fact, I use the prior alphabetical knowledge to formulate a sentence. Humans easily do lifelong learning, but it is not trivial for machines, therefore the Lifelong Machine Learning systems are crucial for artificial general intelligence.

Korbinian: What could be a product in this regard?

Pankaj: An AI product with lifelong learning capabilities would offer continuous learning, for example with human-in-the-loop or active learning where humans overrule AI systems. The goal is to provide continuous feedback and the AI keeps learning with these continuous feedbacks. And essentially lifelong learning systems employ transfer learning and domain adaptation techniques to offer better AI experience to users without forgetting the past.

If we do customer review analysis at Insaas, the lifelong learning AI product will continuously learn over streams of customer feedback from social media platforms. And it means the system learns on batches of customer reviewers over time from different domains and also reuse and retain past learnings across domains.

Korbinian: Where do you see the biggest opportunities in Natural Language Processing today?

Pankaj: Since NLP is evolving, I believe the opportunities are endless. As you know, a lot of fundamental research cutting edge research is happening at lightning speed. However, there is a big vacuum on the applied side of it, particularly in the Industrial space to analyse text.

Therefore, I see a big opportunity in turning cutting-edge AI research into building high-quality production-ready NLP applications. And particularly for different industries, it comes with several challenges, for example the lack of training data, domain shifts or affordable infrastructure like GPUs.

Take for example the BERT models. They have gained success in scientific communities, however their application in production environments is still challenging, because their inference is still very slow and they are computationally intensive. And it is challenging to deploy such large NLP models on customer infrastructures.

I see another exciting opportunity in building federated lifelong learning NLP applications. This is what we are setting up at DRIMCo for every AI functionality, because we respect customer data ownership and privacy. So the goal is to build a decentralized lifelong-learning AI system where the customer data remains on their premise and the algorithms and predictive models travel between the clients and the server. The sole purpose is the cross-client transfer learning from a “phantom” pool of real-world data for better customer AIX (AI Experience) without actually moving the customer data.