Traffic Prediction
Big data analytics predicting traffic accident severity in Dubai
About
Big data analytics project processing 150K+ real traffic accident records from Dubai to predict accident severity levels. Built on Apache Spark (PySpark 3.5.0) for distributed data processing, implementing K-Means clustering for pattern discovery, Artificial Neural Networks (MLP) for severity classification, and a custom Bagging ensemble method for improved prediction robustness.
The entire pipeline runs in a Dockerized Jupyter environment for reproducibility. Comprehensive exploratory data analysis reveals temporal and spatial patterns in Dubai's traffic incidents, while the multi-model approach ensures robust severity predictions across varying accident conditions.
Key Features
- PySpark distributed processing of 150K+ accident records
- K-Means clustering for traffic pattern discovery
- ANN (MLP) for severity prediction
- Custom Bagging ensemble method
- Docker containerized Jupyter environment
- Comprehensive data visualization and EDA
Architecture
Distributed processing pipeline using PySpark for ETL operations on traffic data. Feature engineering extracts temporal, spatial, and environmental factors. Three modeling approaches run in parallel: K-Means for unsupervised pattern discovery, MLP neural network for classification, and custom Bagging for ensemble robustness. Results are visualized through Matplotlib/Seaborn dashboards.