luyug/GradCache
Run Effective Large Batch Contrastive Learning Beyond GPU/TPU Memory Constraint
Implements gradient caching to decouple memory constraints from batch size by processing inputs in smaller chunks while maintaining full-batch gradient semantics. Supports both PyTorch and JAX/TPU backends with a flexible API that handles various input formats (tensors, dicts, lists) and integrates seamlessly with Hugging Face Transformers models. Enables tied encoders and distributed training through a customizable loss function interface, allowing cost-effective training on single GPUs or low-RAM systems that previously required high-memory hardware.
429 stars and 22 monthly downloads. No commits in the last 6 months. Available on PyPI.
Stars
429
Forks
27
Language
Python
License
Apache-2.0
Category
Last pushed
Mar 26, 2024
Monthly downloads
22
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/vector-db/luyug/GradCache"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Related tools
RediSearch/RediSearch
A query and indexing engine for Redis, providing secondary indexing, full-text search, vector...
redis/redis-vl-python
Redis Vector Library (RedisVL) -- the AI-native Python client for Redis.
redis-developer/redis-ai-resources
✨ A curated list of awesome community resources, integrations, and examples of Redis in the AI ecosystem.
redis-developer/redis-product-search
Visual and semantic vector similarity with Redis Stack, FastAPI, PyTorch and Huggingface.
messkan/prompt-cache
Cut LLM costs by up to 80% and unlock sub-millisecond responses with intelligent semantic...