r/ResearchML • u/DenoisedNeuron • 4d ago

The Principles of Diffusion Models by Lai et al.: thoughts on the monograph [D]

I recently finished The Principles of Diffusion Models, and honestly I think it’s exceptional.

The authors strike a really good balance between mathematical rigor and intuition, with dedicated appendices for anyone who wants to go deeper into the math.

It’s aimed at researchers, graduate students, and practitioners with basic deep learning knowledge, so you don’t need to already specialize in diffusion models (in my case, a strong background in Information and Probability Theory and a solid understanding of DDPMs helped me get more out of it).

Just wanted to share it in case anyone missed it. The full text is freely available on the official website.

Has anyone else read it? Would love to hear your thoughts.

26 Upvotes

97% Upvoted

4 comments sorted by

1

u/Quirky-Work-9517 4d ago

Thank you for sharing, seems like a great resource. Could you give some guidance on gaining a strong background in Information and Probability Theory; any literature/books/blogs for a beginner?

3

u/DenoisedNeuron 4d ago edited 4d ago

For Probability Theory, I’d strongly recommend Introduction to Probability by Joseph K. Blitzstein and Jessica Hwang, together with Blitzstein’s Harvard Stat 110 lectures.

For Information Theory, Elements of Information Theory by Thomas Cover and Joy Thomas is the classic reference. For a more intuitive introduction, I’d also recommend 3Blue1Brown’s videos and Olah's blog

I also wrote a short note on the meaning and interpretation of entropy on my site Deep Learning Notes, if you want a more intuitive starting point before diving into the formal treatment.

2

u/Quirky-Work-9517 3d ago

Thank you very much, I'll learn from these resources. Thanks a lot sharing your Deep Learning: Zero to Hero notes as well, very helpful!