DjangoCon 2022 | Handling Django in highly concurrent & scale environment

Django

Handling Django in highly concurrent and scale environments: solutions for concurrent updates, searching large datasets, and optimizing performance without sacrificing accuracy.

Key takeaways

The talk presents several solutions to handling Django in highly concurrent and scale environments, including handling concurrent updates and searching in large datasets.
The speaker uses a real-world example of optimizing a Django admin pagination in an e-commerce system, where they had to improve the search functionality.
The problem is that the search query was taking too long to execute, and they needed to optimize it without sacrificing accuracy.
The first solution tried to use the count property of the Django pagination object, but it was not suitable for concurrent updates.
The speaker also talked about the importance of caching in Django and how to use it efficiently.
The fourth solution proposed using PostgreSQL’s EXPLAIN command to estimate the count of matching records, but it had its own trade-offs.
The speaker also mentioned the need for parallelization and using distributed locks to handle concurrency issues.
The talk also touches on the idea of using a probabilistic count approach, such as HyperLogLog, but did not delve deeper into the implementation details.
Furthermore, the speaker discussed the concept of namespace search, where you can create separate search fields for specific use cases to improve search performance.
In terms of concurrency, the speaker emphasized the need for locking and how to use it effectively in Django.
Finally, the speaker concluded that the solution they implemented was a combination of caching, parallelization, and understanding the database’s internal workings to achieve the best results.

DjangoCon 2022 | Handling Django in highly concurrent & scale environment

More talks