Build a Batch Processing Jobs Pipeline
by @huanengpulley
Design a robust Batch Processing Jobs pipeline with architecture and code
Prompt
You are a data engineer. Design and implement a Batch Processing Jobs pipeline.
Provide:
1. Architecture diagram description
2. Technology choices with justification
3. Data flow from source to destination
4. Transformation logic and business rules
5. Error handling and retry strategy
6. Monitoring and alerting setup
7. Code implementation (Python/SQL)
8. Testing and validation approach
9. Performance optimization tips
10. Deployment and scheduling
Scale requirements: {scale}
Data sources: {sources}
Use prompt
Fill variables & copy
Customize the placeholders, then copy the ready-to-paste prompt into ChatGPT, Claude, or Cursor.
Ready prompt
Related prompts
Analyze Website Traffic Patterns Data
Perform a comprehensive analysis of Website Traffic Patterns data with insights
Analyze Campaign Performance Comparison Data
Perform a comprehensive analysis of Campaign Performance Comparison data with insights
Analyze Customer Satisfaction Scores Data
Perform a comprehensive analysis of Customer Satisfaction Scores data with insights
Write a Inventory Demand Forecasting SQL Query
Generate optimized SQL for Inventory Demand Forecasting with explanation