yahoo/TensorFlowOnSpark
TensorFlowOnSpark brings TensorFlow programs to Apache Spark clusters.
Enables distributed TensorFlow training and inference across Spark/Hadoop clusters by launching TensorFlow workers on executors with support for both HDFS-native data reads and Spark RDD pushdown via `TFNode.DataFeed`. Supports synchronous/asynchronous training, model/data parallelism, and server-to-server direct communication, requiring minimal code changes to existing TensorFlow programs while integrating seamlessly into Spark data pipelines.
3,859 stars and 11,238 monthly downloads. No commits in the last 6 months. Available on PyPI.
Stars
3,859
Forks
941
Language
Python
License
Apache-2.0
Category
Last pushed
Jul 10, 2023
Monthly downloads
11,238
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/ml-frameworks/yahoo/TensorFlowOnSpark"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Related frameworks
tensorflow/tfx
TFX is an end-to-end platform for deploying production ML pipelines
VowpalWabbit/vowpal_wabbit
Vowpal Wabbit is a machine learning system which pushes the frontier of machine learning with...
projectglow/glow
An open-source toolkit for large-scale genomic analysis
Wei-1/Scala-Machine-Learning
No Dependency Scala Machine Learning Algorithm Gallery
thieu1995/IntelELM
IntelELM: A Python Framework for Intelligent Metaheuristic-based Extreme Learning Machine