vllm-mlx and Local_LLM_Training_Apple_Silicon
About vllm-mlx
waybarrios/vllm-mlx
OpenAI and Anthropic compatible server for Apple Silicon. Run LLMs and vision-language models (Llama, Qwen-VL, LLaVA) with continuous batching, MCP tool calling, and multimodal support. Native MLX backend, 400+ tok/s. Works with Claude Code.
This project helps developers and engineers working with AI applications to run large language models and vision-language models on their Apple Silicon Macs much faster. It takes various inputs like text, images, videos, or audio, processes them using different AI models, and produces outputs such as generated text, image descriptions, audio transcriptions, or embeddings. It's designed for anyone building or experimenting with AI solutions who needs to deploy models locally on Apple hardware.
About Local_LLM_Training_Apple_Silicon
GusLovesMath/Local_LLM_Training_Apple_Silicon
Created and enhanced a local LLM training system on Apple Silicon with MLX and Metal API, overcoming the absence of CUDA support. Fine-tuned the Llama3 model on 16 GPUs for streamlined solution of verbose math word problems. Result: a powerful, privacy-preserving chatbot that runs smoothly on-device.
This project offers a specialized chatbot designed to solve complex math word problems. You provide a detailed math problem in plain English, and the chatbot delivers a clear, concise solution. It's ideal for students, educators, or anyone needing quick, private assistance with verbose mathematical reasoning, running directly on your Apple device.
Related comparisons
Scores updated daily from GitHub, PyPI, and npm data. How scores work