Uptime Industries wants to boost localized AI usage with an AI-in-a-box called Lemony AI

Uptime Industries is taking a bet on on-premise AI infrastructure with an “AI-in-a-box” handheld device called Lemony AI.

The size of a sandwich, Lemony AI can supposedly run a large language model (LLM), AI agents and AI workflows all on one node. Each Lemony requires only 65 watts of power to run, the equivalent of charging a laptop, and can be stacked and connected to form bigger AI clusters.

Sascha Buehrle, co-founder and CEO of Uptime, told TechCrunch that each node can sustain an LLM up to 75 billion parameters in size, and can host open-source models or retrofitted versions of closed models. In a cluster of Lemony nodes, each device can run a different model.

Uptime is launching with partnerships with IBM and JetBrains to provide its customers easier access to AI models, including IBM’s closed models.

Like most tech startups, the idea for Lemony AI stemmed from a side project: Uptime’s co-founders, Buehrle and Ivan Kuleshov, were trying to figure out if they could distribute language models on Raspberry Pi’s micro computers, and while this wasn’t originally related to generative AI, when it worked, they started trying to see what else these Raspberry Pis could handle.

When they realized running models on local devices could prove the key to unlock more AI adoption by organizations — say, enterprises that don’t want to use cloud-based models — they started building a device of their own. They figured going small with an emphasis on data privacy could get companies to adopt AI quicker.

“We need to build something small which can go easily into the teams, and which [does] not require any organization-wide decision making — bringing on-premise generative AI solutions into the business teams, basically,” Buehrle said. Clusters of small and powerful devices can help the system grow according to a customer’s needs, he added.

The company says it has already seen strong demand from business in heavily-regulated industries like finance, healthcare and law.

“Everything stays in your box,” Buehrle said. “So your documents, your files, your emails — the models are hosted in the box, the agents are running locally in the box, and nothing is leaving the box.”

Uptime has so far raised $2 million in a seed funding round led by True Ventures, with participation from Alumni Ventures, JetBrains, and some angel investors. The company will use the funds to further develop its devices.

The startup says the plan now is to get the software it has developed for its micro AI computers, Lemony OS, to work on other companies’ hardware, like the Nvidia DGX Spark. It also wants to expand the software from its current, single-user focus to one that can be used by teams.

Lemony AI costs $499 a month to use, and can be accessed by up to five users.