We can't find the internet
Attempting to reconnect
Something went wrong!
Hang in there while we get back on track
Predict Lightning Strikes using Django and AWS with Calvin Hendryx Parker - DjangoCon US 2022
Discover how to predict lightning strikes in real-time using Django, AWS Lambda, and serverless architecture, and learn about the challenges and solutions faced in scaling and deploying the project.
- Using serverless architecture with AWS Lambda and Django, to scale the detection of lightning strikes.
- The Lambda function ingests radar data, runs the prediction algorithm, and makes predictions in under 500 milliseconds.
- The challenge was to reduce the inference time from 90 seconds to make it fast enough for real-time predictions.
- Using Docker images in AWS Lambda to standardize the environment and make it easier to develop and deploy.
- Using Redis to cache data and speed up processing, especially when dealing with large amounts of data.
- Infrastructure as code with Terraform to manage and deploy the infrastructure.
- Using Fargate for hosting Django bits, and Lambdas for ingest and prediction.
- Balancing performance and cost when using AWS services.
- Using open data sources, such as NEXRAD data from NOAA, and storing it in S3 buckets.
- The project uses a multistage build to handle C dependencies and compile them in the Lambda image.
- The Django application uses a custom client library to interact with Redis.
- The project also leverages GitLab configurations with infrastructure as code.
- The next step is to take this architecture to the next level and explore using it for other predictions, such as tornadoes or floods.
- The project uses a code artifact repository to host the ECR images.
- The challenge of developing a truly cloud-native application is simulating local development with private Python package repositories.
- The AWS services used in the project include SNS notifications, ECR containers, and Lambda functions.