2 Probability
To most people, probability is a loosely defined term employed in everyday’s conversation to indicate the measure of one’s belief in the occurrence of a future event. Say, what is the chance of arriving on time if one takes the bus? takes the taxi? However, for scientific purposes, it is necessary to give the word probability a definitive, clear interpretation. One can think of probability as the language in discussing uncertainty. For instance, in tossing a coin (random experiment), the outcomes can either be {H} or {T }. If we know exactly what the outcome of the next trial will be, then we are certain that, say, a {H} will show up next. However, it is impossible to know exactly what is going to happen in reality - uncertainty. Definition. A random experiment is a process leading to at least two possible outcomes with uncertainty as to which will occur. Then, the possible outcomes of a random experiment are called the basic outcomes, and the set of all basic outcomes is called the sample space, S. Examples of random experiment (i) Tossing a coin Basic outcomes: {H}, {T } Sample space, S = {H, T } (ii) Rolling a dice Basic outcomes: {1}, {2}, {3}, {4}, {5}, {6} Sample space, S = {1, 2, 3, 4, 5, 6} (iii) Daily changes in Hang Sang Index Basic outcomes: {up}, {down}, {no change} Sample space, S = {up, down, no change} Definition. An event is a set of basic outcomes, or a collection of some basic outcomes from the sample space, and it is said to occur if the random experiment gives rise to one of its constituent basic outcomes. Example 2.1. In a random experiment of throwing two dices, we are interested in the sum of the numbers turned up by the two dices. Basic outcomes: {(1, 1)}, {(1, 2)}, . . ., {(6, 5)}, {(6, 6)} HKU STAT1603 (2012-13, Semester 1) 1

STAT1603 Introductory Statistics Sample space, S = {(1, 1), (1, 2), . . . , (6, 5), (6, 6)} Suppose that we are interested in

2. Probability

(i) A = Event in which the sum is even. Then, A contains the combinations of numbers whose sums equal 2, 4, 6, 8, 10 or 12. (ii) B = Event in which the sum is odd. Then, B contains the combinations of numbers whose sums equal 3, 5, 7, 9 or 11. (iii) C = Event in which the sum is 8. Then, C = {(2, 6), (3, 5), (4, 4), (5, 3), (6, 2)}. Basic Set Operations (i) Belongs to x 2 A means that object (element) x is a member of A. In other words, object (element) x can be found in the set A. (ii) Subset B⇢A,x2B)x2A (iiii) Equal B = A , B ⇢ A and A ⇢ B. (iv) Intersection The intersection of sets A and B, denoted as A \ B, is the set of objects (elements) that belong to both A and B, i.e., x 2 A \ B ) x 2 A and x 2 B. (v) Union The union of sets A and B, denoted as A [ B, is the set of objects (elements) that belong to either A or B, i.e., x 2 A [ B )x 2 A or x 2 B. (vi) Complement The complement of A, denoted as A0 , is a set of all objects (elements) that are not in A, i.e., x 2 A0 ) x 2 A. / Definition. The probability P (E) of an event E 2 S is a set function defined on a class of all events satisfying the axioms of probability which will be discussed later. Theorem 2.1. If an experiment can result in any one of n different equally likely outcomes, and if r of these outcomes together constitute event A, then P (A) = r/n. HKU STAT1603 (2012-13, Semester 1) 2

STAT1603 Introductory Statistics

2. Probability

If the random experiment can be repeated over and over again with n = total number of trials and r = number of trials in which event A actually happened out of the n trials. Then, P (A) is simply the proportion or rate of occurrences of event A in the long run; or P (A) = r = relative frequency of event A. n 1 P (getting a head) = . 2 Question: How can we get this probability? Answer: Flip a fair coin by 10,000 times and count the number of heads, 5,000, say. Then, 5000 1 P (getting a head) = = . 10000 2 If the number of flipping, n, is sufficiently large, P (A) becomes close to 0.5. Hence, probability is an abstraction of relative frequency; or the relative frequency is a realization of probability. To illustrate the probability of occurrence of an event graphically, we can make use of the Venn diagram. Now, the area of a circle indicates the occurrence of an event E.

For example, a fair coin is tossed and we want the probability getting a head is

Axioms (Postulates) of Probability 1. If A is any event in the sample space S, then 0  P (A)  1. 2. P (S) = 1. 3. For a sequence of mutually exclusive events A1 , A2 , . . ., (i.e. Ai \ Aj = ; for i 6= j), 1 X P (A1 [ A2 [ · · · ) = P (Ai ). i=1 HKU STAT1603 (2012-13, Semester 1)

3

STAT1603 Introductory Statistics

2. Probability

Based on the above axioms of probability, we can obtain the following equalities. 1. P (;) = 0 where ; ⌘ { } is an empty set. 2. P (E1 [ E2 ) = P (E1 ) + P (E2 ) if E1 \ E2 = ;.

3. P (E 0 ) = 1

P (E) where E 0 is the complement of E.

In an example of dice rolling, the sample space S = {1, 2, 3, 4, 5, 6}. Let E = {1, 2, 3}. Then, E 0 = {4, 5, 6}. 3 1 1 P (E) = = and P (E 0 ) = 1 P (E 0 ) = . 6 2 2 4. If A ✓ B, then P (A)  P (B).

In an example of dice rolling, the sample space S = {1, 2, 3, 4, 5, 6}. Let A = {2, 3, 5} and B = {2, 3, 4, 5, 6}. 3 5 P (A) = and P (B) = . 6 6 5. P (A [ B) = P (A) + P (B) P (A \ B). 4

HKU STAT1603 (2012-13, Semester 1)

STAT1603 Introductory Statistics

2. Probability

In an example of dice rolling, the sample space S = {1, 2, 3, 4, 5, 6}.

Let A = {1, 2, 3} and B = {2, 3, 4}. Then, A [ B = {1, 2, 3, 4} and A \ B = {2, 3}. 4 3 3 2 P (A [ B) = = P (A) + P (B) P (A \ B) = + . 6 6 6 6 6. P (A0 \ B) = P (B) P (A \ B).

Let A = {1, 2, 3} and B = {2, 3, 4}. Then, A0 = {4, 5, 6} and A0 \ B = {4}. 1 3 2 P (A0 \ B) = = P (B) P (A \ B) = . 6 6 6 P PP 7. P (E1 [ E2 [ · · · [ Ek ) = k P (Ei ) i=1 1i