(... to be written)
Using Bayes Theorem in Cryptography was Alan Turing's idea, while working at Bletchley Park trying to break the German Enigma. Another important paper to introduce probability theory to the context of cryptography is the semial [Shannon45].
Sandy Zabell gave a talk at Rutgers U. on Alan Turing, and the Applications of Probability to Cryptography (using Bayesian Statistics), referencing [Turing37, Turing38]:
A complete report on how cryptanalysts at Bletchley Park managed to break Tunny, the successor to Enigma is [Tunny]. It is heavy on maths, but it illustrates the power of Bayesian Statistics in breaking classic, albeit complex, ciphers.
People not familiar with Bayes Theorem and its applications to Cryptography have many sources to get started. One of them is Jonathan Katz, who gave a great introduction to a-priori and a-posteriori probabilities and Bayes Theorem in [KatLin15]. He also gave a concrete example in "Perfect Secrecy II" of his Coursera course:
A more thorough introduction to conditional probability, as used in Bayes' Theorem is MIT's John Tsitsiklis "Conditioning and Bayes' Rule", which is part of the course MIT 6.041 Probabilistic Systems Analysis and Applied Probability, Fall 2010 (YouTube playlist):
- [Turing37] Alan M. Turing: The Applications of Probability Theory to Cryptography. (full pdf via arxiv.org)
- [Turing38] Alan M. Turing: Paper on Statistics of Repetitions. (full pdf via arxiv.org)
- [Shannon45] Claude E. Shannon: A Mathematical Theory of Cryptography. (full pdf via iacr.org)
- [KatLin15] Jonathan Katz, Yehuda Lindell: Introduction to Modern Cryptography, 2nd Edition. 2015.
- [Tunny] Donald Michie, Jack Good et al.: General Report on Tunny, with Emphasis on Statistical Methods. (full pdf via www.codesandciphers.org.uk)