Allen Downey - Long-tailed distributions in the natural and engineered world | SciPy 2023

Explore the unexpected ubiquity of long-tailed distributions in natural and engineered systems, and learn how to model and prepare for rare events with a good understanding of underlying physics.

Key takeaways

Long-tailed distributions are common in natural and engineered systems, violating our intuition and defying prediction.
A long-tailed distribution has a large number of extreme values, making it difficult to predict and prepare for rare events.
The log T distribution is a type of long-tailed distribution that is common in many fields, including astronomy and engineering.
Long-tailed distributions can be caused by various mechanisms, including preferential attachment and aggregating processes.
Taming long-tailed distributions requires a good model and a understanding of the underlying physics.
A bad model can seem OK if it is only looking at the normal part of the distribution, but can be disastrous if extrapolated to extreme values.
Extrapolating a model beyond the data can lead to unrealistic predictions, including underestimating the probability of large rare events.
Long-tailed distributions can be seen in many natural systems, including earthquakes, solar flares, and asteroid sizes.
A log X scale can be used to visualize the tail of a distribution, showing the extreme values more clearly.
The Student t-distribution is a type of long-tailed distribution that can be used to model data with heavy tails.
A mixture of normal distributions can also produce a long-tailed distribution.
The concept of a “gray swan” refers to a rare event that is not completely unexpected, but is still difficult to predict.
A black swan is a rare event that is completely unexpected and has a major impact.
Nassim Taleb’s book, “The Black Swan”, discusses the importance of understanding and preparing for rare events.

Allen Downey - Long-tailed distributions in the natural and engineered world | SciPy 2023

More talks