• Demonstration part of an ongoing defense project funded through DARPA’s Environment-Driven Conceptual Learning (ECOLE) program.
• Using generative AI with computer vision to construct conceptual AI models that can not only recognize objects and actions but provide meaningful explanations for these decisions. 
• Could enable AI- agents, in collaboration with human counterparts, to perform future DoD mission-critical analytical tasks.

NISKAYUNA, NY – MAY 7, 2024 -- Providing a glimpse of the future to come with AI-enabled machines, GE Aerospace researchers have successfully integrated generative AI with computer vision to demonstrate AI with child-like learning capabilities.  GE Aerospace’s AI technology, funded through DARPA’s ECOLE program, is being demonstrated at the Ash Carter Exchange for Innovation and National Security and AI Expo taking place this week in Washington D.C.   

Today, AI can achieve some level of conceptual understanding through pattern recognition that draws inferences from large datasets of millions of images.  Through its work with DARPA, GE Aerospace researchers have demonstrated AI that is capable of both understanding and explaining concepts. This allows for a new form of continuous learning where the system can analyze its mistakes and subsequently improve its performance.   

“Rapid advances in generative AI and computer vision are getting us much closer to creating AI- agents that can enter into new environments, understand what’s happening and do the right thing,” said Peter Tu, Chief AI Scientist, GE Aerospace Research. “The more we can augment machines with this form of intelligence, the more it can help the DoD, businesses and individuals improve the way we live, work, and perform.”

Tu added, “In the aerospace industry, we are already developing AI to improve aircraft utilization, make our process workflows more efficient, and enhance inspection processes to improve safety.  With these examples, it’s all about using AI to help optimize aircraft scheduling, enable our sales workforce to search and seamlessly work through hundreds of pages of a customer service agreements (CSA), or use AI to augment a visual inspection process for parts that helps the service engineer see more and be more productive.” 

GE Aerospace researchers are collaborating with a team of recognized leaders in AI and computer vision technologies, including Language Computing Corporation, Intelligent Retrieval, University of Southern California (USC) and University of California Los Angeles (UCLA).  This has enabled the team to make substantial progress to develop and now demonstrate their AI technology,

How GE Aerospace’s AI Demo Works

Tu says the goal is to develop AI agents that are robust to unforeseen circumstances and are trusted by their human counterparts allowing for collaboration in a wide variety of work-related tasks.  As part of the ECOLE program, Tu is now demonstrating AI that is able to provide an insightful description of an arbitrary object or action in a semantically meaningful manner. 

In the demo Tu is currently running, the AI program GE Aerospace AI researchers have built can interpret and describe observed objects and actions at a conceptual level.  For example, if it is asked to describe an airplane, the AI program can provide details it can see such as, “The object has a wide body with wings and an engine.”

Tu notes that the move to semantic or Third-Wave AI is beginning to accelerate as new advances in generative AI, computer vision, and knowledge representation matures. 

“When it’s all said and done, we want to deploy AI in ways that make our lives better and more productive,” Tu concluded. “In the Aerospace industry, it means integrating AI to help make air travel safer and on-time for passengers. That’s what were ultimately striving to achieve.” 

About GE Aerospace Research

GE Aerospace Research is the central hub of innovation for GE Aerospace and one of the industry’s most diversified and global research and development operations. The lab, with research facilities in the US (Niskayuna, NY) and India (Bengaluru), have more than 650 researchers with expertise across virtually every scientific and engineering discipline. They have teams leading a range of advanced technology developments in materials science, propulsion, power electronics, avionics thermal management, 3D manufacturing and more to expand the frontiers of more sustainable air travel in skies and beyond.