airscholar/RealtimeStreamingEngineering
This project serves as a comprehensive guide to building an end-to-end data engineering pipeline using TCP/IP Socket, Apache Spark, OpenAI LLM, Kafka and Elasticsearch. It covers each stage from data acquisition, processing, sentiment analysis with ChatGPT, production to kafka topic and connection to elasticsearch.
No commits in the last 6 months.
Stars
43
Forks
31
Language
Python
License
—
Category
Last pushed
Jan 04, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/airscholar/RealtimeStreamingEngineering"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
openai/openai-cookbook
Examples and guides for using the OpenAI API
rgbkrk/dangermode
Execute IPython & Jupyter from the comforts of chat.openai.com
voidful/TextRL
Implementation of ChatGPT RLHF (Reinforcement Learning with Human Feedback) on any generation...
CogStack/OpenGPT
A framework for creating grounded instruction based datasets and training conversational domain...
antononcube/Python-JupyterChatbook
Python package of a Jupyter extension that facilitates the interaction with LLMs.