ONNX Model Deployment ML Frameworks

Tools and frameworks for converting, optimizing, and deploying ONNX models across different runtime environments and hardware platforms. Does NOT include general model training, framework-specific inference (e.g., pure TensorFlow or PyTorch inference), or non-ONNX deployment solutions.

There are 115 onnx model deployment frameworks tracked. 6 score above 70 (verified tier). The highest-rated is microsoft/onnxruntime at 100/100 with 19,534 stars and 76,261,123 monthly downloads. 8 of the top 10 are actively maintained.

Get all 115 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=ml-frameworks&subcategory=onnx-model-deployment&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Framework	Score	Tier	Stars	Language
1	microsoft/onnxruntime ONNX Runtime: cross-platform, high performance ML inferencing and training...	100	Verified	19,534	C++
2	onnx/onnx Open standard for machine learning interoperability	98	Verified	20,477	Python
3	NVIDIA/TensorRT NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on...	87	Verified	12,784	C++
4	PINTO0309/onnx2tf Self-Created Tools to convert ONNX files (NCHW) to TensorFlow/TFLite/Keras...	84	Verified	931	Python
5	microsoft/onnxconverter-common Common utilities for ONNX converters	72	Verified	295	Python
6	NVIDIA/DALI A GPU-accelerated library containing highly optimized building blocks and an...	70	Verified	5,643	C++
7	pytorch/TensorRT PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT	67	Established	2,955	Python
8	onnx/onnxmltools ONNXMLTools enables conversion of models to ONNX	66	Established	1,143	Python
9	onnxsim/onnxsim Simplify your onnx model	66	Established	4,307	C++
10	onnx/onnx-tensorrt ONNX-TensorRT: TensorRT backend for ONNX	63	Established	3,194	C++
11	jolibrain/deepdetect Deep Learning API and Server in C++14 support for PyTorch,TensorRT, Dlib,...	61	Established	2,549	C++
12	NVIDIA/MinkowskiEngine Minkowski Engine is an auto-diff neural network library for high-dimensional...	58	Established	2,883	Python
13	nntrainer/nntrainer NNtrainer is Software Framework for Training and Inferencing Neural Network...	56	Established	204	C++
14	sdpython/onnx-extended New operators for the ReferenceEvaluator, new kernels for onnxruntime, CPU, CUDA	56	Established	35	Python
15	onnx/tensorflow-onnx Convert TensorFlow, Keras, Tensorflow.js and Tflite models to ONNX	56	Established	2,519	Jupyter Notebook
16	aphrodite-engine/aphrodite-engine Large-scale LLM inference engine	56	Established	1,673	C++
17	triton-inference-server/dali_backend The Triton backend that allows running GPU-accelerated data pre-processing...	53	Established	141	C++
18	PINTO0309/simple-onnx-processing-tools A set of simple tools for splitting, merging, OP deletion, size compression,...	52	Established	304	Python
19	OAID/Tengine Tengine is a lite, high performance, modular inference engine for embedded device	51	Established	4,510	C++
20	facebookarchive/caffe2 Caffe2 is a lightweight, modular, and scalable deep learning framework.	51	Established	8,398	Shell
21	Quantco/spox Pythonic framework for building ONNX graphs	51	Established	94	Python
22	Tencent/TNN TNN: developed by Tencent Youtu Lab and Guangying Lab, a uniform deep...	51	Established	4,626	C++
23	kraiskil/onnx2c Open Neural Network Exchange to C compiler.	51	Established	368	C
24	emmtrix/emx-onnx-cgen Deterministic ONNX-to-C compiler for embedded and safety-critical systems,...	50	Established	10	Python
25	VeriSilicon/tflite-vx-delegate Tensorflow Lite external delegate based on TIM-VX	50	Established	48	C++
26	zjhellofss/KuiperInfer 校招、秋招、春招、实习好项目！带你从零实现一个高性能的深度学习推理库，支持大模型 llama2...	48	Emerging	3,353	C++
27	AlexanderLutsenko/nobuco Pytorch to Keras/Tensorflow/TFLite conversion made intuitive	47	Emerging	343	Python
28	dotnet/infer Infer.NET is a framework for running Bayesian inference in graphical models	47	Emerging	1,607	C#
29	xboot/libonnx A lightweight, portable pure C99 onnx inference engine for embedded devices...	44	Emerging	647	C
30	onnx/onnx-tensorflow Tensorflow Backend for ONNX	44	Emerging	1,325	Python
31	hailo-ai/hailort An open source light-weight and high performance inference framework for...	44	Emerging	172	C++
32	kibae/onnxruntime-server ONNX Runtime Server: The ONNX Runtime Server is a server that provides TCP...	44	Emerging	181	C++
33	amir-abdi/keras_to_tensorflow General code to convert a trained keras model into an inference tensorflow model	44	Emerging	1,652	Python
34	justinchuby/onnx-shape-inference Symbolic shape inference for ONNX	43	Emerging	6	Python
35	wuba/dl_inference 通用深度学习推理工具，可在生产环境中快速上线由TensorFlow、PyTorch、Caffe框架训练出的深度学习模型。	43	Emerging	415	Java
36	solrex/caffe-mobile Optimized (for size and speed) Caffe lib for iOS and Android with...	43	Emerging	313	C++
37	OAID/Caffe-HRT Heterogeneous Run Time version of Caffe. Added heterogeneous capabilities to...	43	Emerging	269	C++
38	iwatake2222/InferenceHelper C++ Helper Class for Deep Learning Inference Frameworks: TensorFlow Lite,...	42	Emerging	297	C++
39	iwatake2222/play_with_tflite Sample projects for TensorFlow Lite in C++ with delegates such as GPU,...	42	Emerging	381	C++
40	NVIDIA/nvImageCodec A nvImageCodec library of GPU- and CPU- accelerated codecs featuring a...	42	Emerging	143	Jupyter Notebook
41	gizatechxyz/orion ONNX Runtime in Cairo 1.0 for verifiable ML inference using STARK	42	Emerging	176	Cairo
42	gmalivenko/pytorch2keras PyTorch to Keras model convertor	42	Emerging	862	Python
43	NervanaSystems/ngraph nGraph has moved to OpenVINO	42	Emerging	1,343	C++
44	iree-org/onnxruntime-ep-iree IREE based OnnxRuntime Execution Provider	41	Emerging	5	C++
45	microsoft/nn-Meter A DNN inference latency prediction toolkit for accurately modeling and...	41	Emerging	362	Python
46	NVIDIA/tensorrt-laboratory Explore the Capabilities of the TensorRT Platform	41	Emerging	262	C++
47	luxonis/modelconverter Model converter for Luxonis' cameras. Convert your model from ONNX, TF, ......	41	Emerging	13	Python
48	intel/npu-nn-cost-model Library for modelling performance costs of different Neural Network...	40	Emerging	34	C++
49	sdpython/onnx-array-api Array APIs to write ONNX Graphs	40	Emerging	11	Python
50	ysh329/deep-learning-model-convertor The convertor/conversion of deep learning models for different deep learning...	40	Emerging	3,247	—
51	OAID/Tengine-Convert-Tools Tengine Convert Tool supports converting multi framworks' models into tmfile...	39	Emerging	92	C++
52	cyrusbehr/tensorrt-cpp-api TensorRT C++ API Tutorial	39	Emerging	793	C++
53	olilarkin/ort-builder ONNX Runtime static library builder	39	Emerging	74	C++
54	iwatake2222/play_with_tensorrt Sample projects for TensorRT in C++	39	Emerging	197	C++
55	alrevuelta/cONNXr Pure C ONNX runtime with zero dependancies for embedded devices	38	Emerging	216	C
56	AsuharietYgvar/AppleNeuralHash2ONNX Convert Apple NeuralHash model for CSAM Detection to ONNX.	38	Emerging	1,536	Python
57	kibae/pg_onnx pg_onnx: ONNX Runtime integrated with PostgreSQL. Perform ML inference with...	38	Emerging	58	C++
58	NobuoTsukamoto/tensorrt-examples TensorRT Examples (TensorRT, Jetson Nano, Python, C++)	38	Emerging	100	Jupyter Notebook
59	xmba15/onnx_runtime_cpp small c++ library to quickly deploy models using onnxruntime	38	Emerging	385	C++
60	OAID/MXNet-HRT Heterogeneous Run Time version of MXNet. Added heterogeneous capabilities to...	38	Emerging	72	C++
61	Adlik/Adlik Adlik: Toolkit for Accelerating Deep Learning Inference	37	Emerging	807	C++
62	Tencent/Forward A library for high performance deep learning inference on NVIDIA GPUs.	37	Emerging	555	C++
63	Phoenix8215/A-White-Paper-on-Neural-Network-Deployment 模型部署白皮书(CUDA\|ONNX\|TensorRT\|C++)🚀🚀🚀	37	Emerging	244	—
64	merrymercy/tvm-mali Optimizing Mobile Deep Learning on ARM GPU with TVM	36	Emerging	183	C
65	mshr-h/onnx2fx Turn ONNX models into inspectable, transformable, and trainable PyTorch FX...	36	Emerging	10	Python
66	MegEngine/MegFlow Efficient ML solution for long-tailed demands.	35	Emerging	407	Rust
67	RobertBiehl/caffe2-ios ☕️ Caffe2Kit. A simple one step integration of Caffe2 for iOS.	35	Emerging	71	C++
68	hshatti/TONNXRuntime TOnnxRuntime is a Microsoft ONNXRuntime AI and Machine Learning Library for...	35	Emerging	65	Pascal
69	vibeswithkk/ZENITH An open-source bridge for faster ML inference. Supports PyTorch, JAX, and...	34	Emerging	4	Python
70	MaitreChen/openvino-lenet-sample 本仓库包含了完整的深度学习应用开发流程，以经典的手写字符识别为例，基于LeNet网络构建。推理部分使用torch、onnxruntime以及openvino框架💖	34	Emerging	17	Python
71	qbxlvnf11/convert-pytorch-onnx-tensorrt Converting weights of Pytorch models to ONNX & TensorRT engines	34	Emerging	50	Python
72	yuanmu97/InFi InFi is a library for building input filters for resource-efficient inference.	33	Emerging	41	Python
73	yuzawa-san/onnxruntime-java A type-safe, lightweight, modern, and performant binding Java binding of...	33	Emerging	29	Java
74	RunEdgeAI/coreflow Graph-based C++ runtime for building and executing AI, ML, and computer...	33	Emerging	12	C++
75	RidgeRun/r2inference RidgeRun Inference Framework	33	Emerging	27	C++
76	athrva98/polyinfer Unified deployment pipeline	32	Emerging	9	Python
77	zerollzeng/tiny-tensorrt Deploy your model with TensorRT quickly.	32	Emerging	764	C++
78	fateshelled/OnnxGraphQt ONNX model visualizer	31	Emerging	88	Python
79	TeiaCare/TeiaCareInferenceClient TeiaCareInferenceClient is a C++ inference client library that implements...	30	Emerging	3	C++
80	haobosang/TinyTensor TinyTensor is a tool for running already trained NN (Neural Network) models...	30	Emerging	141	C++
81	smarter-project/armnn_tflite_backend TensorFlow Lite backend with ArmNN delegate support for Nvidia Triton	26	Experimental	8	C++
82	becauseofAI/caffe-plus-plus Caffe++: assemble new features to enhance Caffe☕️	26	Experimental	11	C++
83	luchangli03/onnxsim_large_model simplify >2GB large onnx model	26	Experimental	70	Python
84	xiaochus/DeepModelDeploy Deploy deep learning model on difference hardware and framework....	25	Experimental	13	C++
85	victorbadenas/SimpleOnnxInference A simple c++ onnx application for inferencing images	24	Experimental	3	C++
86	loong64/onnxruntime ONNX Runtime: cross-platform, high performance ML inferencing and training...	23	Experimental	1	Shell
87	aws-samples/sagemaker-cv-preprocessing-training-performance SageMaker training implementation for computer vision to offload JPEG...	23	Experimental	21	Python
88	zpye/SimpleInfer A simple neural network inference framework	23	Experimental	25	C++
89	domenicostefani/deep-classf-runtime-wrappers Classification wrappers for 4 Deep Learning Embedded Runtimes for Music...	23	Experimental	6	C++
90	Neumenon/cowrie Multi-language binary JSON codec with tensors, compression, and streaming support.	22	Experimental	—	Go
91	Orange20000922/Filerestore_CLI 一个 NTFS 文件恢复工具，支持 MFT 扫描、签名搜索、智能覆盖检测和高性能文件恢复	22	Experimental	3	C++
92	odeliyach/tiny-ml-runtime Generic neural network inference engine in pure C. 258x faster than PyTorch...	22	Experimental	—	C
93	RyanCCC/Deployment 深度学习应用部署	21	Experimental	8	C++
94	brandonviaje/Infera lightweight inference engine/runtime	20	Experimental	1	C++
95	Dimitrios-Kafetzis/EdgeNN Lightweight, zero-allocation C11 library for neural network inference on ARM...	19	Experimental	—	C
96	jazpeak/NanoNet A lightweight, runtime C++ inference engine for running simple ONNX models.	17	Experimental	2	C++
97	zhehaoxu/deep-deploy deploy deep learning model with MNN	17	Experimental	3	C++
98	veera-adithya-d/hardware-aware-algorithm Inference module of Imagenet	16	Experimental	3	C++
99	torinos-yt/NNOnnx Using CUDA for Faster Machine Learning Inference on Unity	13	Experimental	6	C#
100	DarkStarStrix/Nexa_Inference A inference application to serve Scientific Models	13	Experimental	8	Python
101	brandon-gong/siliconnn Feedforward neural networks in pure ARM64 assembly for Apple Silicon.	12	Experimental	3	Assembly
102	regel/onnxruntime_server An easy way to start an ONNX inferencing server for prediction with GRPC endpoints	12	Experimental	3	CMake
103	MrSyabro/luaort ONNX Runtime binding for Lua	12	Experimental	4	C
104	work-buddy/work-buddy-runtime A custom Linux based OpenCV C++ runtime for powering the Work Buddy AI XR HUD.	12	Experimental	3	C++
105	fancyovo/XingLing-cpp 🚀 High-performance C++/CUDA Inference Engine for XingLing-0.68B....	12	Experimental	1	Cuda
106	Konstantina155/InferONNX Lightweight TEE-based system for secure ONNX model inference using Intel SGX...	11	Experimental	5	C
107	mlomb/onnx2code Convert ONNX models to plain C++ code (without dependencies)	11	Experimental	22	Python
108	glzbcrt/tf-onnx-sample Sample Tensorflow model to classify images and infer using ONNX.	11	Experimental	2	Jupyter Notebook
109	HexaForge-1/highperf-ai-ml-inference High-perf C++ AI/ML inference engine with ONNX Runtime & LibTorch. CPU...	11	Experimental	—	C++
110	Natfii/onnxruntime-gpu-blackwell Pre-built onnxruntime-gpu 1.24.1 with Blackwell sm_120 CUDA kernels (RTX...	11	Experimental	—	—
111	idsia-robotics/navground_onnx Provides a C++ navigation behavior that evaluates a ONNX model.	11	Experimental	—	C++
112	Darth-Kronos/trt-custom-plugins TensorRT plugins for custom operators	11	Experimental	2	C++
113	karthi0804/Pytorch-ResNet-CPP-Inference ResNet based Image classification using Pytorch.	11	Experimental	2	C++
114	selpoG/ort-sample A Minimal Sample to use onnxruntime from C++.	10	Experimental	1	CMake
115	HichemMaiza/onnxruntime-extensions-java Microsoft onnxruntime-extensions builds for Java	10	Experimental	1	—

Comparisons in this category

onnxruntime and onnx (100 vs 98) onnx and onnx-tensorrt (98 vs 63) onnx and tensorflow-onnx (98 vs 56) onnxruntime and onnx-tensorrt (100 vs 63) onnx and onnxmltools (98 vs 66) onnxruntime and onnx2c (100 vs 51) onnxruntime and onnxconverter-common (100 vs 72) TensorRT and onnx-tensorrt (87 vs 63) DALI and dali_backend (70 vs 53) onnx2tf and tensorflow-onnx (84 vs 56)