1.1 Randomness versus determinism
Determinism in science and engineering:
▯ Deterministic view in science: provided su▯cient information is available
about the initial state and operating conditions of a natural process or
a man-made system, its future behavior can be predicted exactly.
▯ This operational viewpoint has been the prevailing one in most of your
college and university education (mechanics, circuit theory, etc.)
▯ A typical example is provided by classical mechanics:
- Consider the motion of a particle under the in
uence of various
forces in three-dimensional space.
- If we know the initial position and velocity vectors of the particle,
its mass and the total force ▯eld, Newton’s laws can be used to
calculate (i.e. predict) the future trajectory of the particle.
2003 Ben▯t Champagne Compiled January 11, 2012 1.1 Randomness versus determinism 2
The concept of randomness:
▯ The above view is highly idealistic: In most "real-life" scienti▯c and
engineering problems, as well as many other situations of interest (e.g.
games of chance), we cannot do exact predictions about the phenomena
or systems under consideration.
▯ Two basic reasons for this may be identi▯ed:
- we do not have su▯cient knowledge of the initial state of the system
or the operating conditions (e.g. motion of electrons in a micropro-
- due to fundamental physical limitations, it is impossible to make
exact predictions (e.g. uncertainty principle in quantum physics)
▯ We refer to such phenomena or systems as random, in the sense that
there is uncertainty about their future behavior: a particular result or
situation may or may not occur.
▯ The observation of speci▯c quantities derived from such a random system
or phenomenon is often referred to as a random experiment.
2003 Beno^▯t Champagne Compiled January 11, 2012 1.1 Randomness versus determinism 3
▯ Consider the following game of chance:
- We roll an ordinary six-sided die once and observe the number show-
ing up, also called outcome.
- Possible outcomes are represented by the set of numbers S = f1;2;3;
- We cannot predict what number will show up as a result of this
- Neither can we predict that a related event A, such as obtaining an
even number (represented by A = f2;4;6g), will occur.
▯ Consider a more sophisticated example from communications engineer-
- Using an appropriate modulation scheme, we transmit an analog
speech signal s(t) over a radio channel.
- Due to channel and receiver noise, and other possible disturbances
during the transmission, the received signal r(t) is generally di▯erent
from the transmitted one, i.e. s(t).
- In general, it is not possible for the radio engineer to predict the
exact shape of the error signal n(t) = r(t) ▯ s(t).
2003 Beno^▯t Champagne Compiled January 11, 2012 1.2 The object of probability 4
1.2 The object of probability
Regularity in randomness:
▯ OK, we cannot predict with certainty the particular outcome in a single
realization of a random experiment, but...
▯ In many practical situations of interest (games of chance, digital commu-
nications, etc..), it has been observed that when a random experiment
is repeated a large number of times, the sequence of results so obtained
shows a high degree of regularity.
▯ Let us be more speci▯c: Suppose we repeat a random experiment (e.g.
rolling a die) n times. Let ▯(A;n) be the number of times that a certain
event A occur (e.g. the result is even). It has been observed that
! constant as n ! 1 (1.1)
▯ The ratio ▯(A;n)=n is called the relative frequency.
▯ The constant provides a quantitative measure of the likelihood of A.
2003 Beno▯t Champagne Compiled January 11, 2012 1.2 The object of probability 5
▯ Consider a simple experiment consisting in
ipping a coin. Let A denote
the event that a head shows up.
▯ Now suppose this experiment is repeated n times. The quantity ▯(A;n)
simply represents the number of times that head shows up, out of n
ips of the coin.
▯ Assuming that the coin is fair (unloaded), we expect the ratio ▯(A;n)=n
to approach 1/2 as n gets larger and larger.
▯ Results of a computer simulation experiment:
- Sequence of observed outcomes: THTHHH...
- Relative frequency versus n:
100 10 102 10
number of repetitions