Nebullvm – Speeds up AI inference by 5-20x

Nebullvm speeds up AI inference by 5-20x in just a few lines of code rocket.

How Nebullvm Works

This open-source library takes your AI model as input and outputs an optimized version that runs 5-20 times faster on your machine. Nebullvm tests multiple deep learning compilers to identify the best possible way to execute your model on your specific hardware, without impacting the accuracy of your model.

The goal of nebullvm is to let any developer benefit from deep learning (DL) compilers without having to spend tons of hours understanding, installing, testing and debugging this powerful technology.

The library aims to be:

•  Deep learning model agnostic. nebullvm supports all the most popular architectures such as transformers, LSTMs, CNNs and FCNs.
•  Hardware agnostic. The library now works on most CPUs and GPUs and will soon support TPUs and other deep learning-specific ASICs.
•  Framework agnostic. nebullvm supports the most widely used frameworks (PyTorch, TensorFlow and Hugging Face) and will soon support many more.
•  Secure. Everything runs locally on your machine.
•  Easy-to-use. It takes a few lines of code to install the library and optimize your models.
•  Leveraging the best deep learning compilers. There are tons of DL compilers that optimize the way your AI models run on your hardware. It would take tons of hours for a developer to install and test them at every model deployment. The library does it for you!

Nebullvm on GitHub: https://github.com/nebuly-ai/nebullvm
Platform: AI
⭐️: 627