Edge AI promises the benefits of cloud AI, plus responsiveness and privacy, because local model execution eliminates sending personal information to the cloud and waiting for a reply. However, edge systems’ variety challenges developers. They differ in supported data types, processing power, memory capacity, and connected sensors.
The Silicon Valley startup OpenInfer addresses the edge AI challenge with its runtime software and development environment. Models target the runtime’s abstraction, and the company’s software maps this to the underlying hardware. The OpenInfer studio environment allows developers to select parameters such as a specific off-the-shelf model, whether a model should emphasize responsiveness or accuracy, and target hardware capabilities. OpenInfer handles quantization, memory management, task scheduling, and optimizations to improve neural-network performance, helping developers avoid hand tuning.
For greater performance and to access additional data, the runtime supports federation, allowing a wearable to collaborate with a PC, for example, drawing on its greater processing power and long-term data storage.
OpenInfer focuses on AI assistants, envisioning them as real-time companions. Associated with a person, they move to whatever device is in use and react to the current context. Initial target systems include Intel- and Qualcomm-based laptops, and the company plans to address car infotainment next, followed by retail systems. Later, OpenInfer will target robots and drones. The company focuses on enterprise customers and integrators/consultancies, which can bring its technology to other businesses.
Behnam Bastani and Reza Nourai founded OpenInfer in 2024, and the company has secured $8 million in funding. Both have worked at various companies, including in Meta’s Oculus group and for Roblox. The company’s software is in alpha testing with lead customers.