Task: Design solution for public transport live tracking in the biggest regions in the world.
Bus count by Region
- USA: 1.000.000 buses
- Europe: 900.000 buses
- China: 760.000 buses
Requirements gathering
- The actual location of each bus sending to cloud each 5 sec. With assumption of random distribution, the load is about 200.000 messages per second for USA, 180.000 msg/sec for Europe.
- Users interact with databases in specific predefined way (get actual position of single bus or buses on single line), which eliminate complex queries.
- Traffic is unstable and cyclic over day, with spikes of traffic in rush hour and nighttime low.
- All data should be effectively logged for further statistical analysis.
- Fine-grained access control should be available for field administrators for specific tasks (like device fleet management or log access).
- The solution should be cost-effective.
High-Level Design
Design Rationale
-
IoT Core:
- Each bus sends location data every 5 seconds, generating a large volume of events. IoT Core is designed to process this high-throughput, event-driven data efficiently.
- It supports secure communication with IoT devices (X.509 cert.), ensuring data integrity and privacy.
-
DynamoDB:
- DynamoDB is a NoSQL database that scales horizontally, making it ideal for handling the large datasets generated by cities like Shanghai (20,000 buses).
- The predefined queries (e.g., fetching the location of a single bus or buses on a line) align with DynamoDB's fast key-value lookups.
- Its on-demand capacity mode handles cyclic and unstable traffic patterns effectively.
-
API Gateway:
- Users can query the database to fetch live bus positions, with API Gateway acting as a lightweight, scalable intermediary.
- It supports automatic scaling to handle traffic spikes during rush hours.
-
IAM (Identity and Access Management):
- Delegated administration is possible through IAM, allowing specific tasks (like fleet management) to be securely assigned to administrators.
- Fine-grained access controls ensure secure handling of sensitive data.
-
S3 Bucket:
- Detailed logs can be collected for statistical analysis, such as usage patterns or performance metrics.
- S3 is highly durable and cost-effective for storing large amounts of data.
Low-Level Design
Performance and Future Enchancements
- AWS services are highly reliable, ensuring minimal downtime, which makes it choose #1 for scaling globally.
- Pay-as-you-go pricing for IoT Core, DynamoDB, and API Gateway ensures costs scale with usage means cost effectiveness for cyclic nature of public transport system.
- IoT Core and DynamoDB are built for massive-scale applications, capable of handling millions of messages per second globally.
- The architecture is flexible enough to integrate additional features, such as Cognito access control, WAF firewall for security or Route 53 for improvement of global user experience.