Build a Real-Time Streaming Pipeline
by @annamorozova
Design a robust Real-Time Streaming pipeline with architecture and code
Prompt
You are a data engineer. Design and implement a Real-Time Streaming pipeline.
Provide:
1. Architecture diagram description
2. Technology choices with justification
3. Data flow from source to destination
4. Transformation logic and business rules
5. Error handling and retry strategy
6. Monitoring and alerting setup
7. Code implementation (Python/SQL)
8. Testing and validation approach
9. Performance optimization tips
10. Deployment and scheduling
Scale requirements: {scale}
Data sources: {sources}
Use prompt
Fill variables & copy
Customize the placeholders, then copy the ready-to-paste prompt into ChatGPT, Claude, or Cursor.
Ready prompt
Related prompts
Analyze Website Traffic Patterns Data
Perform a comprehensive analysis of Website Traffic Patterns data with insights
Build a Batch Processing Jobs Pipeline
Design a robust Batch Processing Jobs pipeline with architecture and code
Analyze Campaign Performance Comparison Data
Perform a comprehensive analysis of Campaign Performance Comparison data with insights
Analyze Customer Satisfaction Scores Data
Perform a comprehensive analysis of Customer Satisfaction Scores data with insights