Information theory

Basic probability theory

A discrete random variable X consists of a finite set of values

X and a probability distribution defined on X .

How much information is contained in a message?

The probability that X takes the value x X is written

How much does the ciphertext tell about the key, or about the

Pr[X = x] or just Pr[x] if X is obvious.

P
Always, Pr[x] 0 for all x, and
xX Pr[x] = 1.

plaintext?
How certain can we be that decryption is correct, even when

using frequency analysis?

Ex: X a die (trning) X = {1, 2, 3, 4, 5, 6}, Pr[X = xi ] = 1/6 for all
xi X .

Joint and conditional probability

Relating joint and conditional probability

Pr[x, y ] = Pr[x|y ]Pr[y ]

For X and Y random variables on X and Y ,

i.e. x and y at the same time is x given y when y .

the joint probability Pr[X = x, Y = y ] is the probability that X

takes the value x and Y takes the value y (at the same time.)

Since Pr[x, y ] = Pr[y , x],

the conditional probability Pr[X = x|Y = y ] is the probability

Pr[x, y ] = Pr[y |x]Pr[x]

that X takes the value x given that Y takes the value y .

X and Y are independent random variables if

Pr[X = x, Y = y ] = Pr[X = x] Pr[Y = y ]

for all x X and y Y .

Bayes theorem: if Pr[y ] > 0, then

Pr[x|y ] =

Pr[y |x]Pr[x]
Pr[y ]

(Directly from the equations above.)

3

Bayes and dice

Perfect secrecy?
Setting: crypto system hM, C, K, E, Di where M, C, K are
considered random variables.
Assume K and M are independent; C depends on both.

Bayes theorem: if Pr[y ] > 0, then

Pr[x|y ] =

Pr[y |x]Pr[x]
Pr[y ]

Probability distribution of ciphertexts (many m and k may give

same c):
X
Pr[C = c] =
Pr[K = k]Pr[M = Dk (c)]

Example: X represents the sum of two fair dice (trningar): values

{2, . . . , 12}, and Y gets value d when the dice have the same value
(double) and n otherwise.
Exercise:
1

write down the probability distributions for X and Y

calculate Pr[d |4] and Pr[4|d ] directly

verify (by using) Bayes theorem

{k:cC (k)}

where C (k) = {Ek (m) : m M} (the possible ciphertexts when k

is the key).
Probability that c is cipher when m is plaintext (many keys
possible):
X
Pr[C = c|M = m] =
Pr[K = k]

Pr[d |4]Pr[4] = Pr[d ]Pr[4|d ]

{k:m=Dk (c)}
5

Plaintext given cipher

Silly crypto system

Ciphertext-only attack: given a certain ciphertext, what is the

probability that a certain ciphertext matches it?

M = {a, b} with Pr[a] = 1/4, Pr[b] = 3/4,

K = {x, y , z} with Pr[x] = 1/2, Pr[y ] = Pr[z] = 1/4,
C = {1, 2, 3, 4}.
a b
x 1 2
Encryption matrix:
y 2 3
z 3 4

Use Bayes theorem:

Pr[M = m|C = c] =
=

Pr[M = m] Pr[C = c|M = m]

Pr[C = c]
P
Pr[M = m] {k:m=Dk (c)} Pr[K = k]
P
{k:cC (k)} Pr[K = k]Pr[M = Dk (c)]

Exercise: calculate Pr[c] and Pr[m|c]. (Verify maths by examining

matrix, and v.v.)

Perfect secrecy

A system has perfect secrecy if no information about plaintext is

learnt by getting the ciphertext, i.e. Pr[m|c] = Pr[m] for all
m M and c C.
Which systems have perfect secrecy?
Simplest(?): Shift cipher with one-character message, key

equiprobable (equally probable).

Most general: one-time-pad Vernam cipher.