Detecting and Fixing Unbound Concurrency Problems - Ivan Valkov

Learn how to detect and fix unbound concurrency problems in your Go application, including creating a benchmark, profiling, and configuring limits on Go routines to prevent memory issues.

Key takeaways

Detecting and fixing unbound concurrency problems requires understanding the failure scenarios of your use case and bug.
Check metrics, such as the number of Go routines and what they are doing, to identify the issue.
Create a benchmark to reproduce the problem and profile to identify bottlenecks.
Use a simple log listener to simulate slow log ingestion and report issues.
Run a profile, check the metrics, and try to fix the code to bound the number of Go routines.
Use configuration options to set a limit on the number of Go routines.
Try to reproduce the problem to fix it, and then verify the fix works.
Events can be queued outside of the application, as in Knative eventing, to reduce the load on the logging aggregator.
Avoid creating too many Go routines and monitor memory usage to prevent running out of memory.
Consider reducing the number of Go routines and adjusting the log listener to avoid creating millions of Go routines.
Think about how to detect and fix unbound concurrency in your application.
Bound the number of Go routines and monitor memory usage.
Consider adding a timeout to prevent Go routines from building up.
Use debugging tools, such as pprof, to profile and understand the code.
Verify that the fix works by reproducing the problem and checking metrics.
Monitor and adjust memory usage to prevent running out of memory.
Use a single channel for work queuing to simplify the code.

Detecting and Fixing Unbound Concurrency Problems - Ivan Valkov

More talks