$30.5M in seed round: CentML – optimizing the learning and performance of AI models

Contrary to popular belief, the era of large seed rounds is not over – at least in the AI field.

Today, CentML, which develops tools to reduce the cost and improve the performance of deploying machine learning models, announced it has raised $27 million in an expanded seed round with participation from Gradient Ventures, TR Ventures, Nvidia and Microsoft Azure AI VP Misha Bilenko.

CentML originally closed the seed round in 2022, but over the past few months, as interest in the product grew, the round was expanded, bringing the total amount raised to $30.5 million.

According to CentML co-founder and CEO Gennady Pekhimenko, the capital raised will be used for CentML’s product development and research, as well as to expand its engineering team and increase its U.S.- and Canada-based staff to 30.

Pekhimenko, an assistant professor at the University of Toronto, co-founded CentML last year with Akbar Nurlybaev and graduate students Shang Wang and Anand Jayarajan. According to Pekhimenko, they were united by the idea of creating a technology that could expand access to computing in the face of a worsening AI chip supply problem.

“The cost of machine learning, talent and chip shortages… Any company working in AI and machine learning faces at least one of these challenges, and most face several at once,” Pekhimenko told TechCrunch. “The most expensive chips are usually unavailable due to high demand from both large companies and startups. This leads companies to sacrifice the size of the model they can deploy or increase inference latency for already deployed models.”

Most companies rely heavily on GPU-based servers to train models, especially generative AI models like ChatGPT and Stable Diffusion. GPUs’ ability to perform many parallel computations makes them well-suited for training today’s most advanced AI.

But chips are in short supply. Microsoft is facing a shortage of server hardware needed to run artificial intelligence so severe that it could lead to service disruptions, the company warned in its summer earnings report. And Nvidia’s highest-performing AI cards are reportedly already sold out through 2024.

That has prompted some companies, including OpenAI, Google, AWS, Meta* and Microsoft, to build – or explore building – their own chips to train models. But this hasn’t been a panacea either. Meta’s work has been fraught with problems, forcing the company to abandon some experimental hardware. And Google is not keeping up with demand for its own cloud-hosted GPU counterpart, the Tensor Processing Unit (TPU), Wired recently reported.

According to Gartner, spending on AI-focused chips will total $53 billion this year and more than double in the next four years, so Pekhimenko believes it’s time to release software that will make models run more efficiently on existing hardware.

“Training AI and machine learning models is becoming increasingly expensive,” said Pekhimenko. “With CentML’s optimization technology, we can reduce costs by up to 80% without sacrificing speed and accuracy.”

That’s a pretty big claim. But at a high level, the CentML software is pretty easy to understand.

The platform attempts to identify bottlenecks in the model training process and predict the overall time and cost of model deployment. In addition, CentML provides access to a compiler – a component that translates programming language source code into machine code understandable by hardware such as GPUs – to automatically optimize model training workloads to achieve the best performance on the target hardware.

Pehimenko says CentML’s software does not degrade the quality of the models and requires “virtually no effort” from engineers.

“For one of our customers, we tripled the speed of the Llama 2 model on Nvidia A10 GPU cards,” he added.

CentML is not the first to take a software-based approach to model optimization. It has competitors in the form of MosaicML, which Databricks acquired in June for $1.3 billion, and OctoML, which received $85 million in cash in November 2021 to develop its machine learning acceleration platform.

However, Pekhimenko claims that CentML’s methods do not result in a loss of model accuracy, as is sometimes the case with MosaicML, and that the CentML compiler is “next generation” and more performant than the OctoML compiler.