Spinoff of NC State Startup Secures NSF Small Business Innovation Funding

Software developed by an NC State professor’s startup aims to make it easier for real-time AI applications to run on smartphones and other devices. 

Many of these technologies currently rely on highly specialized chips. A new software-based approach could allow billions of existing and future devices — from smartphones to self-driving cars — to reap the benefits of machine learning more efficiently and effectively. 

“Producers may want to rethink their strategy,” says Xipeng Shen, an associate professor in the Department of Computer Science. “Rather than creating special chips, they can use this approach to make their existing hardware be able to achieve the performance they need.”

CoCoPIE, LLC, a startup company co-founded by Shen, was awarded a Small Business Innovation Research Phase I grant from the National Science Foundation in June. 

“Our software can shorten the response time of AI applications and make them much smaller than what state-of-the-art approaches allow,” Shen says. “It opens the door for many AI applications to be deployed on smartphones and IoT devices, which would allow more consumers to enjoy the power of AI.”

Shen says the proprietary compression-compilation co-design software could increase the number of applications people can use on their phones and also improve the performance of existing applications.

CoCoPIE has demonstrated that its software can be used to run apps that apply augmented-reality filters (below), upscale video resolution — and scan lung X-rays for signs of COVID-19.

Deep learning algorithms, by definition, have three or more layers of neural networks. 

In the example above, the video is being fed into what’s known as a deep neural network. Autonomous vehicles, which have many cameras that need to detect objects in real-time, also feed images into a deep neural network, Shen says.

“Our compression makes the deep neural network much smaller, then the transformed deep neural network is able to run much faster,” Shen says. What’s more, the compilation matches “the code generation with the whole compression process.” 

As a result, an empirical comparison to current solutions shows that CoCoPIE’s software can make state-of-the-art neural networks nearly “30 times faster” — while maintaining the same “accuracy and quality.” Meanwhile, Shen says, it also improves energy efficiency, which is key for mobile and IoT devices.

Shen, principal investigator of the NSF award — “Enabling Real-Time AI on End Devices through Compression-Compilation Co-Design” — says the funding will be used to support pilot projects with potential customers in order to better understand the market, as well as to improve the prototype apps CoCoPIE has developed to demonstrate the benefits of its technology.

Shen’s collaborators on the project are CoCoPIE co-founders Bin Ren, an assistant professor of computer science at William & Mary University, and Yanzhi Wang, an assistant professor of electrical and computer engineering at Northeastern University.

CoCoPIE’s software is based on intellectual property licensed from NC State University through Shen’s Ificient Consulting Corporation.