ࡱ > i k f g h H f y bjbjEE c ' ' "K & % % ) ) ) = = = 8 u = t %: " G: G: : E E E $ e : q ) G D @ E G G : % % G: : '
Q{ Q{ Q{ G % G: p ) : Q{ G Q{ Q{ h J X : dВ = _L 0 \ X ) X E E h Q{ F T lF Q E E E : : k E E E G G G G E E E E E E E E E : Chapter 2 Bernoulli Random Variables in n Dimensions
1. Introduction
This chapter is dedicated to my STAT 305C students at Iowa State University in the Fall 2006 semester. It is due, in no small part, to their thoughtful questions throughout the course, but especially in relation to histogram uncertainty, that has convinced me to address the issues in this chapter in a rigorous way, and in a format that I believe is accessible to those who have a general interest in randomness.
There are many phenomena that involve only two possible recordable or measurable outcomes. Decisions ranging from the yes/no type to the success/failure type abound in every day life. Will I get to work on time today, or wont I? Will I pass my exam, or wont I? Will the candidate get elected, or not? Will my friend succeed in her business, or wont she? Will my house withstand an earth quake of 6+ magnitude, or wont it? Will I meet an interesting woman at the club tonight, or wont I? Will my sisters cancer go into remission, or wont it. And the list of examples could go on for volumes. They all entail an element of uncertainty; else why would one ask the question. With enough knowledge, this uncertainty can be captured by an assigned probability for one of the outcomes. It doesnt matter which outcome is assigned the said probability, since the other outcome will hence have a probability that is one minus the assigned probability. The act of asking any of the above questions, and then recording the outcome is the essence of what is in the realm of probability and statistics termed a Bernoulli random variable, as now defined.
Definition 1.1 Let X denote a random variable (i.e. an action, operation, observation, etc.) the result of which is a recorded zero or one. Let the probability that the recorded outcome is one be specified as p. Then X is said to be a Bernoulli(p) random variable.
This definition specifically avoided the use of any real mathematical notation, in order to allow the reader to not be unduly distracted from the conceptual meaning of a Ber(p) random variable. While this works for a single random variable, when we address larger collections of them, then it is extremely helpful to have a more compact notation. For this reason, we now give a more mathematical version of the above definition.
Definition 1.2 Let X be a random variable whose sample space is EMBED Equation.3 , and let p denote the probability of the set {1}. In compact notation, this is often written as EMBED Equation.3 . Then X is said to be a Bernoulli(p), or, simply, a Ber(p) random variable.
Since this author feels that many people grasp concepts better with visuals, the probability structure of a Ber(p) random variable is shown in Figure 1.
At one level, Figure 1 is very simple. The values that X can take on are included in the horizontal axis, and the probabilities associated with them are included on the vertical axis. However, conceptually, the implications of Figure 1 are deep.
Figure 1. The probability structure for a Ber(p=0.7) random variable.
X is a 1-dimensional (1-D) random variable, since the values that it can take on are its sample space EMBED Equation.3 , which includes simply numbers, or scalars. So, these numbers can be identified as a subset of the real line, which in Figure 1 is the horizontal axis. Since probabilities are also just numbers, they require only one axis, which in Figure 1 is the vertical line. But what if X were a 2-D random variable; that is, its sample space was a collection of ordered pairs? As we will see presently, then we would need to use a plane (i.e. an area associated with, say, a horizontal line and a vertical line). In that case, the probabilities would have to be associated with a third line (e.g. a line coming out of the page). To summarize this concept, the probability description for any random variable requires that one first identify its sample space. In the case of Figure 1, that entailed drawing a line, and then marking the values zero and one on that line. Second, one then associates probability information associated with the sample space. In the case of Figure 1, that entailed drawing a line perpendicular to the first line, and including numerical probabilities associated with zero and one.
Another conceptually deep element of Figure 1 is an element that Figure 1 (as does almost any probability figure in any text book in the area) fails to highlight. It is the fact that, in Figure 1, the probability 0.7 is not, I repeat, NOT the probability associated with the number 1. Rather, it is the probability associated with the set {1}. While many might argue that this distinction is overly pedantic, I can assure you that ignoring this distinction is, in my opinion, one of the most significant sources of confusion for students taking a first course in probability and statistics (and even for some students in graduate level courses I have taught). Ignoring this distinction in the 1-D case shown in Figure 1 might well cause no problems. But ignoring it for higher dimensional case can result in big problems. So, lets get it straight here and now.
Definition 1.3 The probability entity Pr() is a measure of the size of a set.
In view of this definition, Pr(1) makes no sense, since 1 is a number, not a set. However, Pr({1}) makes perfect sense, since {1} is a set (as defined using { }, and this set contains only the number 1 in it. Since Pr(A) measures the size of a set A, we can immediately apply natural reasoning to arrive at what some books term axioms of probability. These include the following:
Axiom 1. Pr( EMBED Equation.3 .
Axiom 2. EMBED Equation.3 , where EMBED Equation.3 ; that is, EMBED Equation.3 is the empty set.
Axiom 3. Let A and B be two subset of EMBED Equation.3 . EMBED Equation.3 .
The first axiom simply says that when one performs the action and records a resulting number, the probability that the number is in EMBED Equation.3 must equal one. When you think about it, by definition, it cannot be a number that is not in EMBED Equation.3 . The second axiom simply states that the probability that you get no number when you perform the action and record a number must be zero. To appreciate the reasonableness of the third axiom, we will use the visual aid of the Venn diagram shown in Figure 2.
Figure 2. The yellow rectangle corresponds to the entire sample space, EMBED Equation.3 . The size (i.e. probability) of this set equals one. The blue and red circles are clearly subsets of EMBED Equation.3 . The probability of A is the area in blue. The probability of B is the area in red. The black area where A and B intersect is equal to EMBED Equation.3 .
Since Pr() is a measure of size, it can be visualized as area, as is done in Figure 2. Imagining the sample space, EMBED Equation.3 , to be the interior of the rectangle, it follows that the area shown in yellow must be assigned a value of one. The circle in red has an area whose size is Pr(A), and the circle in blue has a size that is Pr(B). These two circles have a common area, as shown in black, and that area has a size that is EMBED Equation.3 . Finally, it should be mentioned that the union of two sets is, itself, a set. And that set includes all the elements that are in either set. If there are elements that are common to both of those sets, it is a mistake to misinterpret that to mean that those elements are repeated twice (once in each set). They are not repeated. They are simply common to both sets. Clearly, if sets A and B have no common elements, then EMBED Equation.3 . Hence, from Axiom 2, the rightmost term on Axiom 3 is zero. In relation to Figure 2 above, that would mean that the blue and red circles did not intersect. Hence, the area associated with their union would simply be the sum of their areas. We will encounter this situation often in this chapter. For this reason, we now formally state this as a special case of Axiom 3.
Axiom 3- A Special Case: Let A and B be two subsets of EMBED Equation.3 . If EMBED Equation.3 , then EMBED Equation.3 .
We are now in a position to apply address the above axioms and underlying concepts in relation to the Ber(p) random variable, X, whose sample space is EMBED Equation.3 . To this end, lets begin by identifying all the possible subsets of EMBED Equation.3 . Since EMBED Equation.3 has only two elements in it, there are four possible subsets of this set. These include {0}, {1}, EMBED Equation.3 and EMBED Equation.3 . The first two sets here are clearly subsets of EMBED Equation.3 . However, the set EMBED Equation.3 is also, formally speaking, a subset of itself. However, since this subset is, in fact, the set itself, it is sometimes called an improper subset. Nonetheless, it is a subset of EMBED Equation.3 . The last subset of EMBED Equation.3 , namely the empty set, EMBED Equation.3 , is simply, by definition, a subset of any set. Even so, it has a real significance, as we will presently describe. And so, the collection of all the possible subsets of EMBED Equation.3 is the following set:
EMBED Equation.3 .
It is crucially important to understand that EMBED Equation.3 is, itself a set. And the elements of this set are, themselves sets. Why is this of such conceptual importance? It is because Pr() is a measure of the size of a set. Hence, Pr() measures the size of the elements of EMBED Equation.3 . It does not measure the size of the elements of EMBED Equation.3 , since the elements of this set are numbers, and not sets.
In relation to Figure 2, we have the following results:
(i) EMBED Equation.3 ;
(ii) EMBED Equation.3 ;
(iii) Since EMBED Equation.3 , we have EMBED Equation.3
(iv) Since EMBED Equation.3 , we could also arrive at the rightmost value, 1.0, in (iii)
via Axiom 2; namely, EMBED Equation.3 .
The practical beauty of the set EMBED Equation.3 is that any question one could fathom in relation to X can be identified as one of the elements of EMBED Equation.3 . Here are some examples:
Question 1: What is the probability that you either fail ( {0} ) or you succeed ( {1} ) in understanding this material? Well, since or represents a union set operation, the event that you either fail or succeed is simply EMBED Equation.3 , which is an element of EMBED Equation.3 .
Question 2: What is the probability that you fail? Since here, failure has been identified with the number, 0, the event that you fail is a set that includes only the number 0; that is, {0}. And, of course, this set is in EMBED Equation.3 .
Question 3: What is the probability that you only partially succeed in understanding this material? Well, our chosen sample space does not recognize partial success. It has only two elements in it: 0 = failure, and 1 = success. And so, while this is a valid question for one to ask, the element in EMBED Equation.3 that corresponds to this event of partial success is the empty set, EMBED Equation.3 . So, the probability that you partially succeed in this setting is zero.
2. Two-Dimensional Bernoulli Random Variables.
It might seem to some (especially those who have some background in probability and statistics) that the developments in the last section were belabored and overly pedantic or complicated. If that is the case, wonderful! Those individuals should then have no trouble in following this and subsequent sections. If, on the other hand, some troubles are encountered, then it is suggested that these individuals return to the last section and review it. For, all of the basic concepts covered there are simply repeated in this and future sections; albeit simply in two dimensions. However, in fairness, it should be mentioned that the richness of this topic is most readily exposed in the context of not one, but two random variables. It is far more common to encounter situations where the relationship between two variables is of primary interest; as opposed to the nature of a single variable. In this respect, this section is distinct from the last. It requires that the reader take a different perspective on the material.
Definition 2.1. Let EMBED Equation.3 and EMBED Equation.3 be Bernoulli random variables. Then the 2-dimensional (2-D) random variable EMBED Equation.3 is said to be a 2-D Bernoulli random variable.
The first item to address in relation to any random variable is its sample space. The possible values that the 2-D variable EMBED Equation.3 can take on are not numbers, but, rather ordered pairs of numbers. Hence, the sample space for X is
EMBED Equation.3 . (2.1)
Key things to note here include the fact that since X is 2-D, its sample space is contained in the plane, and not the line. Hence, to visualize its probability description will require three dimensions. Also, since now, EMBED Equation.3 has 4 elements (as opposed to 2 elements for the 1-D case), its probability description will require the specification of 3 probabilities (not only one, as in the 1-D case). Define the following probabilities:
EMBED Equation.3 (2.2)
Even though (2.1) defines four probabilities ( EMBED Equation.3 ), in view of Axiom 2 above, only three of these four quantities need be specified, since the fourth must be one minus the sum of the other three.
0
0
1
Figure 3. Visual description of the probability structure of a 2-D Bernoulli random variable.
Having defined the sample space for X, and having a general idea of what its probability description is, the next natural step is to identify all the possible subsets of (2.1). Why? Because remember, any question one can fathom to ask in relation to X corresponds to one of these subset. And so, having all possible subset of X in hand can give confidence in answering any question that one might pose. It can also illuminate questions that one might not otherwise contemplate asking. Since this set contains 4 elements, the total number of subsets of this set will be 24 = 16. Lets carefully develop this collection, since it will include a procedure that can be used for higher dimensional variables, as well.
A procedure for determining the collection, EMBED Equation.3 of all the subsets of (2.1):
(i) All sets containing only a single element: {(0,0}, {(1,0)}, {(0,1)}, {(1,1)}
(ii) All sets containing two elements:
-pair (0,0) with each of 3 elements to its right elements: {00, 10}, (00, 01}, {00, 11}
-pair (1,0) with each of the two elements to its right: {10, 01}, {10, 11}
-pair (0,1) with the one remaining element to its right: {10 , 11}
[Notation: for simplicity we use 10 to mean the element (1,0), etc.]
(iii) All sets containing 3 elements:
-pair {00 10} with the first element to the right: {00 10 01}
-pair {00 10} with the second element to the right: {00 10 11}
-pair {00 01} with the element to the right of 01: {00 01 11}
-pair {10 01} with the element to the right: {10 01 11}
(iv) EMBED Equation.3 and EMBED Equation.3
If you count the total number of set in (i) (iv) you will find there are 16. Specifically,
EMBED Equation.3 { {00}, {10}, {01}, {11}, {00,10}, {00,01},{00,11},{10,01},{10,11},
{01,11}, {00,10,01} , {00,10,11}, {00,01,11} , {10,01,11} , EMBED Equation.3 , EMBED Equation.3 } (2.3)
It is important to note that the four singleton sets {(0,0)}, {(1,0)}, {(0,1)} and {(1,1)} have no elements in common with one another. Since they are each a 1-element set, to say that two of them have an element in common would be to say that they each have one and the same element. While the ordered pairs (0,0) and (0,1) do, indeed, have the same first coordinate, their second coordinates are different. As shown in Figure 3, they are two distinctly separate points in the plane. Thus, the intersection of the sets {(0,0)} and {(0,1)} is the empty set.
A second point to note is that any element (i.e. set) in the collection (2.3) can be expressed as a union of two or more of these disjoint singleton sets. For example,
{(0,0), (1,1) } = {(0,0)} EMBED Equation.3 {(1,1)}.
Hence, from Axiom 3 above,
EMBED Equation.3 .
It follows that if we know the probabilities of the singleton sets, then we can compute the probability of any set in EMBED Equation.3 . We now state this in a formal way.
Fact: The probability structure of a 2-D Bernoulli random variable is completely specified when 3 of the 4 probabilities EMBED Equation.3 are specified.
In view of this fact, and the above Definition 2.1, it should be apparent that Definition 2.1 is incomplete, in the sense that it does not define a unique 2-D Bernoulli random variable. This is because in that definition only two parameters were specified; namely, EMBED Equation.3 and EMBED Equation.3 . Even so, the given definition is a natural extension of the definition of a 1-D Bernoulli random variable. We now offer an alternative to Definition 2.1 that does completely and unambiguously define a 2-D Bernoulli random variable.
Definition 2.1 The random variable EMBED Equation.3 is said to be a completely defined 2-D Bernoulli random variable if its sample space is EMBED Equation.3 and if any three of the four singleton set probabilities EMBED Equation.3 are specified.
This alternative definition eliminates the lack of the complete specification of the 2-D Bernoulli random variable, but at the expense of not seeming to be a natural extension of the 1-D random variable.
Now, lets address the question of how the specification of EMBED Equation.3 leads to the specification of EMBED Equation.3 and EMBED Equation.3 . To this end, it is of crucial conceptual importance to understand what is meant when one refers to the event that EMBED Equation.3 equals one, within the
2-D framework. Remember: ANY question one can ask, in relation to EMBED Equation.3 can be identified as one unique set in the collection of sets given by (2.3). This includes questions such as: what is the probability that EMBED Equation.3 equals one? In the 2-D sample space for X, this event is:
The event that EMBED Equation.3 equals one (often written as [ EMBED Equation.3 =1] ) is the set {(1,0), (1,1)}.
This set includes all elements whose first coordinate is 1, but whose second coordinate can be anything. Why? Because there was no mention of EMBED Equation.3 ; only EMBED Equation.3 . If you are having difficulty with this, then consider when you were first learning about x, y and graphing in high school math. If there is no y, then you would identify the relation x=1 as just the point 1.0 on the x-axis. However, in the x-y plane, the relation x=1 is a vertical line that intersects the x-axis at the location 1.0. You are allowing y to be anything, because no information about y was given.
And so, we have the following relation between EMBED Equation.3 and EMBED Equation.3 :
EMBED Equation.3 (2.4a)
Similarly,
EMBED Equation.3 (2.4b)
From (2.4) we observe more of the missing details when one specifies only EMBED Equation.3 and EMBED Equation.3 in relation to a 2-D Bernoulli random variable. If these parameters are specified, then one still needs to specify one of the four parameters EMBED Equation.3 for a complete, unambiguous description of the probability structure of EMBED Equation.3 .
There is one common situation where specification of only EMBED Equation.3 and EMBED Equation.3 is sufficient to completely specify the probability structure of EMBED Equation.3 . It is the situation where EMBED Equation.3 and EMBED Equation.3 are statistically independent. In more simple terms, this situation is one wherein knowledge of the value EMBED Equation.3 has no influence on the probability that EMBED Equation.3 equals any specified value. For example, if you toss a coin and you read heads (map heads to 1), then that result in no way changes the probability that you will get a heads on a second flip, does it? If we agree that it does not, then the 2-coin flip is an example of a 2-D Bernoulli random variable, where the two components of EMBED Equation.3 are statistically independent. As another example, consider parts inspection at the delivery dock of a company. If a randomly selected part passes inspection, it is natural to assume that the probability that the next selected part passes is not influenced by knowledge that the first part passed. While this is a natural assumption in parts inspection protocols, it may not necessarily be true. If the parts were manufactured in the presence of an unknown systematic manufacturing malfunction, then the fact that the first part passed may well influence the probability that the second part passes; for example, if there is only one good part, then the probability that the second part passes, given the first part passed will be zero.
We will presently address the mathematical details of what is required for EMBED Equation.3 and EMBED Equation.3 to be assumed to be statistically independent. However, in order to expedite that investigation, it is appropriate to first address yet another source of major confusion to novices in this area.
Unions and Intersections, Ands and Ors, and Commas and Commas-
It should be apparent by now that probability is intimately related to sets. As noted above, it is, in fact, a well-defined measure of the size of a set. Yet, as was noted above, the vast majority of text books dealing with probability and statistics use a notation that, at the very least, de-emphasizes the notion of sets. For example, in the case of the 2-D Bernoulli random variable EMBED Equation.3 , most books will use notation such as
EMBED Equation.3 (2.5a)
If one realizes that Pr(A) measures the size (i.e. probability) of the set A, then it must be that, depending on how you read (2.5a), either EMBED Equation.3 is a set, or EMBED Equation.3 is a set. In either case, it is very likely that a student who has had some prior exposure to sets has never seen either one of the above expressions for a set. To this point, we have been using a more common notation for a set; namely the {} notation. Lets rewrite (2.5a) in this more common notation.
EMBED Equation.3 (2.5b)
There is nothing ambiguous or vague about (2.5b). The set in question has two elements in it; namely the element (1,0) and the element (1,1). In particular, (1,0) is an element, and not a set. Whereas, { (1,0) } is a set, and that set contains the element (1,0). One might argue that (2.5a) is clear, and that (2.5b) is involves too many unnecessary symbols that can cause confusion. However, lets consider the following probability:
EMBED Equation.3 (2.5c)
This expression includes a set operation symbol, namely the intersection symbol EMBED Equation.3 . This suggests that EMBED Equation.3 and EMBED Equation.3 are sets. Moreover, (2.5c) suggests that these two sets may have elements in common. But what exactly is the set EMBED Equation.3 ? Well, if we ignore EMBED Equation.3 , then we have only a 1-D Bernoulli random variable, whose sample space is {0,1}. In that case, the expression EMBED Equation.3 means the set{1}. However, if we include EMBED Equation.3 in our investigation, then the expression EMBED Equation.3 means { (1,0) , (1,1) }. These are two distinctly different sets, and yet each set is expressed as EMBED Equation.3 . A seasoned student of probability might argue that one must keep in mind the entire setting when interpreting the meaning of EMBED Equation.3 . However, for a student who has no prior background in the field, it is often not so easy to keep the entire setting in mind. Before we can reconcile this ambiguity, we need to first address the set notion of a union ( EMBED Equation.3 ).
The union of two sets is a set whose elements include all of the elements of each set; but where elements in common in these two sets are not counted twice. For example
EMBED Equation.3 (2.5d)
Notice that the common element (1,1) is not counted twice in this union of the sets. Unfortunately, the same type of notation used in (2.5c) for intersections is commonly also used for unions. Specifically, we have the expression
EMBED Equation.3 (2.5e)
Since (2.5c) and (2.5e) each involve a set operation, the above discussion related to the ambiguity and vagueness of expressions such as EMBED Equation.3 applies to (2.5e).
Now lets reconcile these two types of expressions. In doing so, we will discover that there are commas and there are commas. To this end, we will express (2.5c) in the unambiguous notation associated with a 2-D Bernoulli random variable.
EMBED Equation.3 (2.5f)
The leftmost expression in (2.5f) is ambiguous when not accompanied by a note that we are dealing only with a 2-D random variable. (What if we actually had a 3-D random variable?) The middle expression is unambiguous. Furthermore, any student with even a cursory exposure to sets would be able to identify the single element, (1,1), that is common to both sets. The equality of the leftmost and rightmost expressions reveals that in this 2-D framework, we can refer to the element (1,1) as the element whose first component is one and whose second component is one. Hence the comma that separates these two components of the element (1,1) may be read as an and comma.
Similarly, rewriting (2.5e) and referring to (2.5c) gives
EMBED Equation.3 (2.5g)
Hence, the commas that separate the elements (1,0), (0,1) and (1,1) may be read as or commas. After a bit of reflection, the reader may find all of this to be obvious. In that event, this brief digression will have served its purpose. In that case, lets proceed to the following examples to further assess the readers grasp of this topic.
Example 2.1 Let EMBED Equation.3 ~Ber( EMBED Equation.3 ). Notice that there is no assumption of independence here.
Clearly state the sets corresponding to the following events:
(i) EMBED Equation.3 : Answer: EMBED Equation.3 .
(ii): EMBED Equation.3 : Answer: EMBED Equation.3 .
(iii) EMBED Equation.3 : Answer: EMBED Equation.3 .
(iv) EMBED Equation.3 : Answer: EMBED Equation.3 .
Compute the probabilities of the events in (a), in terms of EMBED Equation.3 .
(i) EMBED Equation.3 : Answer: EMBED Equation.3 .
(ii): EMBED Equation.3 : Answer: EMBED Equation.3 .
(iii) EMBED Equation.3 : Answer: EMBED Equation.3 .
(iv) EMBED Equation.3 : Answer: EMBED Equation.3 .
Hopefully, the reader felt that the answers in the above example were self-evident, once the sets in question were clearly described as such. The next example is similar to the last one. However, it extends the conceptual understanding of this topic to arrive at a very important and useful quantity; namely the cumulative distribution function (CDF).
Example 2.2 Again, let EMBED Equation.3 ~Ber( EMBED Equation.3 ). Now, let EMBED Equation.3 be any pair of real numbers (i.e. any point in the plane). Notice here that EMBED Equation.3 is not constrained to be an element of EMBED Equation.3 .
(a) Develop an expression for EMBED Equation.3 as a function of EMBED Equation.3 while ignoring X2 .
Solution: If we want to, we can approach this problem in exactly the same manner as in the above example; namely, by clearly describing the set corresponding to the expressed event EMBED Equation.3 ; namely,
EMBED Equation.3 .
However, since our interest here is only in the random variable EMBED Equation.3 , whose sample space is extremely simple (i.e. {0,1} ) we will choose this approach. The p-value for this random variable in terms of the 2-D probabilities is given above in (2.4a). The probability description for EMBED Equation.3 was shown in Figure 1 above. But that figure only utilized x-values in the range [0,2]. The expression we are to develop here should consider any value of x1 . The following expression is hopefully clear from Figure 1:
EMBED Equation.3 (2.6)
This expression is plotted below for the value EMBED Equation.3
Figure 4. Graph of EMBED Equation.3 given by (2.6).
So, how exactly did Figure 4 arise from Figure 1? Well, Figure 1 shows where the lumps of probability are, and also gives the values of these lumps. For example, the lump at location EMBED Equation.3 is the probability EMBED Equation.3 . If the reader is confused by the fact that Figure 1 is for the Ber(p) random variable, X, while Figure 2 is for the Ber( EMBED Equation.3 ) random variable, EMBED Equation.3 , it should be remembered that when we discussed Figure 1 there was only one random variable. However, now there are two. And so, now, we need to have some way to distinguish one from the other. Nonetheless, both are Bernoulli random variables. And so, both will have the general probability structure illustrated in Figure 1; albeit with possibly differing p-values.
So, again- how did Figure 4 arise from Figure 1? The key to answering this question is to observe that EMBED Equation.3 is the totality of the probability associated with the interval EMBED Equation.3 . Hence, as long as EMBED Equation.3 , the value of EMBED Equation.3 will be zero, since the first lump of probability is at EMBED Equation.3 . So, at this location, EMBED Equation.3 will experience an increase in probability, in the amount EMBED Equation.3 . This increase, or jump in EMBED Equation.3 is shown in Figure 4. As we allow EMBED Equation.3 to continue its travel to the right of zero, since there are no lumps of probability in the interval (0,1) the value of EMBED Equation.3 will remain at the value EMBED Equation.3 throughout this region. When EMBED Equation.3 , the value of EMBED Equation.3 will increase by an amount EMBED Equation.3 , since that is the value of the lump of probability at this location: EMBED Equation.3 . Hence, when EMBED Equation.3 we have EMBED Equation.3 . In words, the probability that the random variable EMBED Equation.3 equals any number less than or equal to one is 1.0. It follows that there are no more lumps of probability to be accumulated as EMBED Equation.3 continues it travel to the right beyond the number 1. This is the reason that EMBED Equation.3 remains flat to the right of EMBED Equation.3 in Figure 4. It is the accumulating feature of EMBED Equation.3 as EMBED Equation.3 travels from left to right that is responsible for the following definition.
Definition 2.3 Let X be any 1-D random variable. Then EMBED Equation.3 is called the cumulative distribution function (CDF) for X.
Example 2.2 continued:
(b) Develop an expression for EMBED Equation.3 as a function of EMBED Equation.3 while not ignoring X2.
Solution: Again, as in (a), we write
EMBED Equation.3 .
As we compute the probability of this set, lets actually identify the actual set that corresponds to a given value for u:
for EMBED Equation.3 ,
for EMBED Equation.3
for EMBED Equation.3 .
j
N o t i c e t h a t t h e r i g h t m o s t e x p r e s s i o n s i n ( i i ) a n d ( i i i ) a b o v e a r e s u m m a t i o n s . I t i s f a i r t o a r g u e t h a t t h e s u m m a t i o n n o t a t i o n i s u n d u l y h e a v y , i n t h e s e n s e t h a t ( i i ) , f o r e x a m p l e , c o u l d h a v e b e e n w r i t t e n m o r e s i m p l y a s E M B E D E q u a t i o n . 3 . N o t o n l y is this a fair argument, it points to yet another example where the biggest stumbling block to a novice might be the notation, and not the concept. However, in this particular situation the summation notation was chosen (at the risk of frustrating some novices) in order to highlight a concept that is central in dealing with two (or more) random variables. We now state this concept for the more general case of two random variables, say, X and Y, whose joint probability structure is specified by a collection of joint probabilities, say, EMBED Equation.3 .
Fact 2.1 Consider a 2-D random variable, (X,Y) having a discrete 2-D sample space EMBED Equation.3 , and corresponding joint probabilities EMBED Equation.3 . Then EMBED Equation.3 .
In many books on the subject, Fact 2.1 is stated as a theorem, and often it is accompanied by a proof. However, we do not believe that this fact is worthy of the theorem label. It is an immediate consequence of the realization that the set EMBED Equation.3
A reader who has had a course in integral calculus might recognize that integration is synonymous with accumulation. The above Fact 2.1 says, in words: To obtain EMBED Equation.3 integrate the joint probabilities over the values of y. For the benefit of such readers, consider the following example.
Example 2.3 Consider a random variable, say, U, whose sample space is EMBED Equation.3 (i.e. the closed interval with left endpoint 0, and with right endpoint 1). Furthermore, assume that U has a uniform probability distribution on this interval. Call this distribution EMBED Equation.3 . The meaning of the term uniform here, is that the probability of any sub-interval of EMBED Equation.3 depends only on the width of that interval, and not on its location. For a sub-interval of width 0.1 (be it the interval (0,0.1) or (0.2,0.3), or [0.8,0.9]) the probability that U falls in the interval is 0.1. This distribution is shown in Figure 5 below. It follows that the probability the U falls in the interval [0,u] is equal to u. Another way of expressing this is EMBED Equation.3 . But this is exactly the definition of the CDF for U. And
so EMBED Equation.3 . This CDF is also shown in Figure 5 below. Notice that this CDF is linear in u, and has a slope equal to 1.0. The derivative of this CDF is, therefore, just its slope, which is exactly EMBED Equation.3 . Hence, here, we can conclude that EMBED Equation.3 is the derivative of EMBED Equation.3 ; or, equivalently, E M B E D E q u a t i o n . 3 i s t h e i n t e g r a l o f E M B E D E q u a t i o n . 3 . j
F i g u r e 5 G r a p h s o f t h e C D F , E M B E D E q u a t i o n . 3 ( t h i c k l i n e ) , a n d i t s d e r i v a t i v e E M B E D E q u a t i o n . 3 ( t h i n l i n e ) .
T h e a b o v e e x a m p l e i s a d e m o n s t r a t i o n o f t h e f o l l o w i n g general definition that holds for any random variable.
Definition 2.4 Let W be any random variable, and let EMBED Equation.3 be its cumulative distribution function (CDF). Then the (possibly generalized) derivative of EMBED Equation.3 is EMBED Equation.3 , which is called the (possibly generalized) probability density function (PDF) for W.
In Example 3.2 above, indeed, the derivative of the CDF EMBED Equation.3 is exactly the PDF EMBED Equation.3 . However, in the case of EMBED Equation.3 with a CDF having the general structure illustrated in Figure 4 above, we see that the CDF has a slope equal to zero, except at the jump locations. And at these locations the slope is infinite (or, if you like, undefined). What is the derivative of such a function? Well, properly speaking, the derivative does not exist at the jump locations. Hence, properly speaking, EMBED Equation.3 does not have a PDF. However, generally speaking (i.e. in the generalized sense) we can say that it s d e r i v a t i v e h a s t h e f o r m i l l u s t r a t e d i n F i g u r e 1 a b o v e . S p e c i f i c a l l y , t h e P D F i s i d e n t i c a l l y z e r o , e x c e p t a t t h e j u m p l o c a t i o n s w h e r e i t c o n t a i n s l u m p s o f p r o b a b i l i t y . [ F o r t h o s e r e a d e r s w h o a r e f a m i l i a r w i t h D i r a c - f u n c t i o n s , t h e s e l u m p s a r e , i n f a c t , w e i g h t e d - f u n c t i o n s , w h o s e w e i g h t s a r e t h e p r o b a b i l i t y v a l u e s ] . T h e k e y p o i n t s h e r e a r e t w o :
K e y P o i n t # 1 : E v e r y r a n d o m v a r i a b l e h a s a w e l l - d e f i n e d C D F , a n d
K e y P o i n t # 2 : I f t h e C D F i s n o t d i f f e r e n t i a b l e , t h e n , p r o p e r l y s p e a k i n g , t h e P D F d o e s n o t e x i s t. Nonetheless, if we allow generalized derivatives, then it does exist everywhere, except at a discrete number of locations.
In the next chapter we will discuss the relation between the CDF and PFD of a wide variety of random variables. However, for the time being, lets return to Bernoulli random variables. In particular, there are two topics that still need to be addressed before we move on to n-D Bernoulli random variables. One is the topic of statistical independence,and the second is the topic of conditional probability. As we shall see shortly, these two topics are strongly connected.
Definition 2.5 Let (X,Y) be a 2-D random variable with sample space EMBED Equation.3 . Let A be a subset of this space that relates only to X, and let B be a subset that relates only to Y. Then the subsets (i.e. events) A and B are said to be (statistically) independent events if EMBED Equation.3 . If all events relating to X are independent of all events relating to Y, then the random variables X and Y are said to be (statistically) independent.
Before we investigate just exactly how the notion of statistical independence relates to a 2-D Bernoulli random variable, lets demonstrate its practical implications in an example.
Example 2.4 Consider the act of tossing a fair coin twice. Let EMBED Equation.3 correspond to the action that is the kth toss, and let a heads correspond to one, and a tails correspond to a zero. Then, EMBED Equation.3 is a 2-D Bernoulli random variable. Since the coin is assumed to be a fair coin, we have
EMBED Equation.3 and EMBED Equation.3 .
But because the coin is fair, each of the four possible outcomes, {(0,0)}, {(1,0)}, {(0,1)}, {(1,1)} should have the same probability. Hence, EMBED Equation.3 . Rewriting this probability in the usual notation gives
EMBED Equation.3 .
So, we see that the events EMBED Equation.3 and EMBED Equation.3 are statistically independent. In exactly the same manner, one can show that all of the events related to EMBED Equation.3 (i.e. EMBED Equation.3 and EMBED Equation.3 ) are independent of all the events related to EMBED Equation.3 (i.e. EMBED Equation.3 and EMBED Equation.3 ). We can conclude that the assumption of a fair coin, and in particular, that the above four outcomes have equal probability, is equivalent to the assumption that EMBED Equation.3 and EMBED Equation.3 are statistically independent. j
N o w , l e t s l o o k m o r e c l o s e l y a t a 2 - D B e r n o u l l i r a n d o m v a r i a b l e E M B E D E q u a t i o n . 3 w i t h s p e c i f i e d p r o b a b i l i t i e s E M B E D E q u a t i o n . 3 . W i t h o u t l o s s o f g e n e r a l i t y , l e t s a s s u m e t h e f i r s t t h r e e p r o b a b i l i t i e s h a v e b e e n s p e c i f i e d . T h e n E M B E D E q u a t i o n.3 . We now address the question:
UNDER WHAT CONDITIONS ARE THE EVENTS EMBED Equation.3 AND EMBED Equation.3 INDEPENDENT ?
ANSWER: Lets first express the condition for independence in terms of the usual notation. Then we will translate the condition in terms of sets. These events are independent if:
EMBED Equation.3 . (2.7a)
In terms of sets, (2.7a) becomes
EMBED Equation.3 . (2.7b)
In terms of the specified probabilities, (2.7b) becomes
EMBED Equation.3 (2.7c)
Even though (2.7c) is the condition on the specified probabilities for these events to be independent, we can arrive at a more simple expression by using the fact that EMBED Equation.3 . First, lets rewrite (2.7c) as
EMBED Equation.3 . (2.7d)
Subtracting EMBED Equation.3 from each side of (2.7d), and rearranging terms, gives
EMBED Equation.3 . (2.7e)
Equation (2.7e) is the condition needed to assume that the events EMBED Equation.3 and EMBED Equation.3 are independent.
Using exactly the same procedure, one can show that the condition (2.7e) is the condition. We state this formally in the following fact.
Fact 2.2 The components of the 2-D Bernoulli random variable EMBED Equation.3 with specified probabilities EMBED Equation.3 are statistically independent if and only if the condition EMBED Equation.3 holds.
Example 2.4 above is a special case of this fact. Since we assumed EMBED Equation.3 , clearly, the above condition holds. This equality of the elemental probabilities is a sufficient condition for independence, but it is not necessary. Consider the following example.
Example 2.5. Suppose the person has very good control over the number of rotations the coin makes while in the air. In particular, suppose the following numerical probabilities:
EMBED Equation.3 . Now, we need to find the numerical value of EMBED Equation.3 (if there is one) such that the relation EMBED Equation.3 holds. To this end, express this condition as:
EMBED Equation.3 (2.8a)
This equation can be rewritten as a quadratic equation in the unknown EMBED Equation.3 .
EMBED Equation.3 . (2.8b)
Applying the quadratic formula to (2.8b) gives
EMBED Equation.3 . (2.8c)
Inserting the above numerical information into (2.8c) gives
EMBED Equation.3 (2.8d)
Notice that for the chosen values EMBED Equation.3 there are two possible choices for EMBED Equation.3 . Furthermore, they add up to 0.6. Hence, if we choose the first for EMBED Equation.3 , then the second is exactly EMBED Equation.3 . It should also be noted that (2.8c) indicates that for certain choices of EMBED Equation.3 and EMBED Equation.3 there will be no value of EMBED Equation.3 that makes the components of X independent. Specifica l l y , i f b o t h E M B E D E q u a t i o n . 3 a n d E M B E D E q u a t i o n . 3 a r e l a r g e e n o u g h s o t h a t t h e t e r m i n s i d e t h e s q u a r e r o o t i s n e g a t i v e , t h e n t h e r e i s n o r e a l - v a l u e d s o l u t i o n f o r E M B E D E q u a t i o n . 3 . j
W e n o w a d d r e s s t h e c o n c e p t o f c o n d i t i o n a l p r o b a b i l i t y in relation to a 2-D Bernoulli random variable. First, however, we give the following definition of conditional probability in the general setting.
Definition 2.6. Let A and B be two subsets of a sample space, EMBED Equation.3 , and suppose that EMBED Equation.3 . Then, the probability of A given B, written as EMBED Equation.3 is defined as
EMBED Equation.3 . (2.9)
To understand (2.9) we refer to the Venn diagram in Figure 2. What given B means is that our sample space is now restricted to the set B. Stated another way, nothing outside of the set B exists. So, in Figure 2, only the red circle exists now. Equation (2.9) is the probability of that portion of the set A that is in the set B. The EMBED Equation.3 , which is the black area in Figure 2, is the size of the intersection relative to the entire sample space. Since our new sample space is the smaller one, B, the probability of this intersection relative to B, demands that we scale it by dividing that probability by the probability of B, as is done in (2.9).
Now that we have Definition 2.6, we can make an alternative definition of statistical independence defined per Definition 2.5. Specifically,
Definition 2.5 Events A and B (where it is assumed that EMBED Equation.3 ) are said to be statistically independent events if EMBED Equation.3 .
Remark In relation to Figure 2, this means that if A is contained entirely in B, then restricting our sample space to B does not alter the probability of A. In other words, if under the condition B, the probability of A is not changed, then A and B are statistically independent. However, while that condition that A is entirely contained in B is a sufficient condition for independence, it is not necessary. Again, referring to Figure 2, all that is necessary is that the overlap of A and B be just enough so that the black intersection area equals the product of the blue and read areas.
We now proceed to relate the concept of conditional probability to a 2-D Bernoulli random variable. Because the sample space for this random variable is so simple, it offers a clear picture of both the meaning and value of conditional probability.
Example 2.6 Again, let EMBED Equation.3 ~Ber( EMBED Equation.3 ). Develop the expression for EMBED Equation.3 .
Solution:
EMBED Equation.3 . (2.10a)
In particular,
EMBED Equation.3 (2.10b)
and
EMBED Equation.3 . (2.10c)
The probabilities (2.10b) and (2.10c) are t h e p - v a l u e s f o r E M B E D E q u a t i o n . 3 , c o n d i t i o n e d o n t h e e v e n t s E M B E D E q u a t i o n . 3 a n d E M B E D E q u a t i o n . 3 , r e s p e c t i v e l y . j
A s s i m p l e a s i t w a s t o o b t a i n ( 2 . 1 0 ) , i t c a n b e a n e x t r e m e l y v a l u a b l e t o o l . S p e c i f i c a l l y , i f o n e h a s r e l i a b l e n u m e r ical values for EMBED Equation.3 , then (2.10) is a prediction model, in the sense that, if we have obtained a numerical value for EMBED Equation.3 , it allows us to predict the probability that EMBED Equation.3 will equal zero or one. Remember, if EMBED Equation.3 and EMBED Equation.3 are independent, then the numerical information associated with EMBED Equation.3 is irrelevant, in the sense that it does not alter the probability that EMBED Equation.3 will equal zero or one. But there are many situations where these random variables are not independent.
3. n-Dimensional Bernoulli Random Variables
Definition 2.7 Let EMBED Equation.3 where each EMBED Equation.3 . Then X is said to be an n-D Bernoulli random variable.
The p-values EMBED Equation.3 in the above definition are not generally sufficient to describe X unambiguously. The reason lies in the fact that the sample space for X includes 2n distinct elements. Hence, to completely describe the probability structure of X requires the specification of EMBED Equation.3 probabilities. Specifically, we need to specify all but one of EMBED Equation.3 . There is a situation wherein the n p-values EMBED Equation.3 are sufficient to completely describe X; namely when the n random variables comprising X are mutually independent.
However, in doing so, we will demonstrate the value of the uniform random variable considered in Example 3 above.
Using a Random Number Generator to Simulate n-D iid Bernoulli Random Variables
In this section we address the problem of simulating data associated with a Bernoulli random variable. This simulation will utilize a uniform random number generator. And so, first, we will formally define what we mean by a uniform random number generator.
Definition 2.8 A uniform random number generator is a program that, when called, produces a random number that lies in the interval [0,1].
In fact, the above definition is not very formal. But it describes in simple terms the gist of a uniform random number generator. The following definition is formal, and allows the generation of n numbers at a time.
Definition 2.8Define the n-D random variable EMBED Equation.3 where each EMBED Equation.3 is a random variable that has a uniform distribution on the interval [0,1], and where these n random variables are mutually independent. The two assumptions that these variables each have the same distribution and that they are mutually independent is typically phrased as the assumption that they are independent and identically distributed (iid). Then U is an n-D uniform random number generator.
The following example uses the uniform random number generator in Matlab to demonstrate this definition.
Example 2.7 Here, we give examples of an n-D uniform random variable, U, using the Matlab command rand, for n=1,2 and 25:
U = rand(1,1) is a 1-D uniform random variable. Each time this command is executed, the result is a randomly selected number in the interval [0,1]. For example:
>> rand(1,1)
ans =
0.9501
U=rand(1,2) is a 2-D uniform random variable. For example,
>> rand(1,2)
ans =
0.2311 0.6068
U=rand(5,5) is a 25-D uniform random variable. For example,
>> rand(5,5)
ans =
0.3340 0.5298 0.6808 0.6029 0.0150
0.4329 0.6405 0.4611 0.0503 0.7680
0.2259 0.2091 0.5678 0.4154 0.9708
0.5798 0.3798 0.7942 0.3050 0.9901
0.7604 0.7833 0.0592 0.8744 0.7889
It is important to note that the command rand(m,n) is the EMBED Equation.3 -D random variable. The numbers are a result of the command. They are not random variables. They are numbers. A random variable is an action, algorithm, or operation that when conducted yields numbers. j
W e n o w p r o c e e d t o s h o w h o w t h e u n i f o r m r a n d o m n u m b e r g e n e r a t o r c a n b e u s e d t o s i m u l a t e m e a s u r e m e n t s o f a B e r n o u l l i r a n d o m v a r i a b l e . L e t s b e g i n w i t h a 1 - D r a n d o m v a r i a b l e . A g a i n , w e w i l l u s e M a t l a b c o m m a n d s t o t h i s e n d .
U s i n g U t o a r r i v e a t E M B E D Equation.3 : For EMBED Equation.3 , define the random variable, X, in the following way: Map the interval EMBED Equation.3 to the event [X=0], and map the event [ EMBED Equation.3 to the event [X=1]. Recall from Example 3 above that EMBED Equation.3 . Hence, it follows that EMBED Equation.3 . Therefore, since X can take on only the value zero or one, we have EMBED Equation.3 ; that it, X is a Ber(p) random variable. Here is a Matlab code that corresponds to EMBED Equation.3 :
p=0.7;
u=rand(1,1);
if u <=1-p
x=0
else
x=1
end
For example:
>> p=0.7;
u=rand(1,1);
if u <=1-p
x=0
else
x=1
end
x =
1
Now, suppose that we want to simulate multiple measurements associated with this random variable EMBED Equation.3 associated with the above code. Well, we could simple embed the code in a DO loop, and repeat the above operation the desired number of times. Well, it turns out that Matlab is a programming language that is not well-suited to DO loops. If the loop count is small, it works fine. But if you wanted to simulate, say one million values associated with X, then it would take a long time. In fact, Matlab was designed in a way that makes it a very fast code for batch or vector operations. With this in mind, the code below is offered. It includes no IF/ELSE commands, and it requires no DO loop for multiple measurements. We will give the code for the case of one million values associated with X.
p=0.7;
m=1000000;
u=rand(1,m);
u=u-(1-p);
x=ceil(u);
The command u=rand(1,m) results in a 1x1000000 vector of numbers between 0 and 1. The command u=u-(1-p), shifts every number to the left by an amount 1-p. Thus, since here p=0.7, every number that was in the interval [0,0.3] has been shifted to a number in the interval[-0.3,0]. In particular, not only is every such number now a negative number, but the closest integer to the right of it is zero. The ceil command rounds numbers to the next higher integer. The command ceil is short for ceiling, or round up to the nearest integer. Similarly, numbers originally in the interval (0.3,1] are moved to the interval (0,0.7]. Since they are still positive, the next highest integer associated with them is one. Here is an example of running the above code. Rather than showing x, which contains one million zeros/ones, we included a command that adds these numbers. This sum is the number of ones, since zeros contribute nothing to a sum.
>> p=0.7;
m=1000000;
u=rand(1,m);
u=u-(1-p);
x=ceil(u);
>> sum(x)
ans =
700202
Notice that the relative frequency of ones is 700202/1000000, which is pretty close to the 0.7 p-value for X. In fact, if we were to pretend that these numbers were collected from an experiment, then we would estimate the p-value for X by this relative frequency value. The value of running a simulation is that you know the truth. The truth in the simulation is that the p-value is 0.7. And so, the simulation using 1000000 measurements appears to give a pretty accurate estimate of the true p-value. We will next pursue more carefully what this example has just demonstrated.
Using EMBED Equation.3 to Simulate EMBED Equation.3 Independent and Identically Distributed (iid) Ber(p) Random Variables, and then, from these, investigating the probability structure of the random variable EMBED Equation.3 .
In this subsection we are interested in using simulations to gain some idea of how many subjects, n, would be required to obtain a good estimate of the p-value of a typical subject. The experiment is based on the question: What is the probability that a typical American believes that we should withdraw from Iraq. We will identify the set {1} with an affirmative, and the set {0} with opposition. We will ask this question to n independent subject and record their responses. Let EMBED Equation.3 be the response of the kth subject. Notice that we are assuming that each subject has the same probability, p, of believing that we should withdraw. Thus, EMBED Equation.3 is an n-D random variable whose components are iid Ber(p) random variables. After we conduct the survey, our next action will be to estimate p using the estimator
EMBED Equation.3 . (2.7)
Notice that (2.7) is a random variable that is a composite action that includes first recording the responses of n subjects, and then taking an average of these responses.
But suppose we were considering conducting an experiment where only 100 measurements were being considered. Well, if we run the above code for this value of m, we get a sum equal to 74. Running it a second time gives a sum equal to 67. And if we run the code 500 times, we could plot a histogram of the sum data, to get a better idea of the amount of uncertainty of the p-value estimator for m=100. Here is the Matlab code that allows us to conduct this investigation of how good an estimate of the true p-value we can expect:
p=0.7;
n=500;
m=100;
u=rand(m,n);
u=u-(1-p);
x=ceil(u);
phat=0.01*sum(x);
hist(phat)
Figure 6. Histogram of the p-value estimator (2.7) associated with m=100 subjects, using 500 simulations.
Notice that the histogram is reasonably well centered about the true p-value of 0.7. Based on the 500 estimates of (2.7) for n=100, the sample mean and standard deviation of the estimator (2.7) are 0.7001 and 0.0442, respectively. W e r e w e t o u s e a 2 - r e p o r t i n g e r r o r f o r o u r e s t i m a t e o f ( 2 . 7 ) f o r m = 1 0 0 , i t w o u l d b e ~ 0 . 0 9 ( o r 9 % ) .
T o g e t a n i d e a o f h o w t h e r e p o r t i n g e r r o r m a y b e i n f l u e n c e d b y t h e n u m b e r o f c h o s e n s u b j e c t s , n , u s e d i n ( 2 . 7 ) , w e e m b e d d e d t h e a b o v e c o d e i n a n m - D O L OOP, for values of m=100, 1000, and 10,000. For each value of m we computed the sample standard deviation. The code and results are given below.
>> %PROGRAM NAME: phatstd
m=[100 1000 10000];
phatstdvec=[];
p=0.7;
n=500;
for i=1:3
u=rand(m(i),n);
u=u-(1-p);
x=ceil(u);
phat=(1/m(i))*sum(x);
phatstd=std(phat);
phatstdvec=[phatstdvec phatstd];
end
phatstdvec
phatstdvec =
0.0469 0.0140 0.0046
Closer examination of these 3 numbers associated with the chosen 3 values of m, would reveal that the standard deviation of (2.7) appears to be inversely proportional to EMBED Equation.3 .
In the next example, we demonstrate the power of knowledge of the conditional PDF of a 2-D Bernoulli random variable, in relation to a process that is a time-indexed collection of random variables. In general, such a process is known as a random process:
Definition 2.9 A time-indexed collection of random variables, EMBED Equation.3 is known as a random process. If the joint PDF of any subset EMBED Equation.3 does not depend on t, then the process is said to be a stationary random process.
The universe is rife with time-dependent variables that take on only one of two possible values. Consider just a few such processes from a wide range of settings:
Whether a person is breathing normally or not.
Whether a drop in air temperature causes a chemical phase change or not.
Whether farmers will get more that 2 inches of rain in July or not.
Whether your cell phone receives a correct bit of information or not.
Whether a cooling pump performs as designed or not.
Whether you get married in any given year or not.
Whether a black hole exists in a sector of the galaxy or not.
All of these examples are time-dependent. In the following example we address what might be termed a Bernoulli or a binary random process.
Example 2.8 Samples of a mixing process are taken once every hour. If the chemical composition is not within required limits, a value of one is entered into the data log. Otherwise, a value of zero is entered. The Figure 7 below shows two randomly selected 200-hout segments of the data log for a process that is deemed to be pretty much under control.
From these data, we see that, for the most part, the process is in control. However, when it goes out of control, there seems to be a tendency to remain out of control for more than one hour.
Under federal regulations, the mixture associated with an out-of-control period must be discarded. Management would like to have a computer model for simulating this control data log. It should be a random model that captures key information, such as the mean and standard deviation of a simulated data log. Pursue the design of such a model.
Well, having had Professor Shermans STAT 305C course, you immediately recall the notion of a Bernoulli random variable. And so, your first thought is to define the events [X=0] and [X=1] to correspond to in and out of control, respectively. To estimate the p-value for X, you add up all the ones in the lower segment given in Figure 7, and divide this number by 200. This yields the p-value, p=12/200=0.06. You then proceed to simulate a data log segment by using the following Matlab commands:
>> u = rand(1,200);
>> u= u 0.94;
>> y = ceil(u);
>> stem(y)
The stem plot is shown in Figure 8 below.
Figure 7. Two 200-hour segments of the control data log for a mixing process.
Even though Figure 8 has general similarities to Figure 7, it lacks the grouping tendency of the ones. Hence, management feels the model is inadequate.
Figure 8. Simulation of a data log 200-hour segment using a Ber(0.6) random variable.
Use the concept of a 2-D Bernoulli random variable whose components are not assumed to be statistically independent, as the basis for your model. Specifically, X1 is the process control state at any time, t, and X2 is the state at time t+1.
To this end, you need configure the data to correspond to EMBED Equation.3 . You do this in the following way: For simplicity, consider the following measurements associated with a 10-hour segment: [0 0 0 1 0 0 0 0 10]. This array represents 10 measurements of X1. Now, for each measurement of EMBED Equation.3 then measurement that follows it is the corresponding measurement of EMBED Equation.3 . Since you have no measurement following the 10th measurement of EMBED Equation.3 , this means that you have only 9 measurements of EMBED Equation.3 ; namely
0 0 0 1 0 0 0 0 1
0 0 1 0 0 0 0 1 0
Of these 9 ordered pairs, you have 5 (0,0) elements. And so, your estimate for EMBED Equation.3 would be 5/9.
Using this procedure on the second data set in Figure 7, you arrive at the following numerical estimates: EMBED Equation.3 and EMBED Equation.3 . It follows that EMBED Equation.3 .
Since EMBED Equation.3 , your first measurement of your 200-hour simulation is that of a Ber(0.06) random variable. You simulate a numerical value for this variable in exactly the way you did in part (a). If the number is 0, your p-value for simulating the second number is obtained using (2.7b), and if your first number was a 1, then you use a p-value given by (2.7b). Specifically,
EMBED Equation.3
EMBED Equation.3 .
The Matlab code for this simulation is shown below.
%PROGRAM NAME: berprocess.m
% This program generates a realization
% of a Ber(p00, p10, p01,p11) correlated process
npts = 200;
y=zeros(npts+1,1);
% Stationary Joint Probabilities between Y(k) and Y(k-1)
p01=0.05;
p10=p01;
p11=0.05
p00 = 1 - (p11 + p10 + p01);
pvec = [p00 p10 p01 p11]
%Marginal p for any Y(k)
p=p11 + p10
% -------------------------
x = rand(npts+1,1);
y(1)= ceil(x(1)- (1-p)); % Initial condition
for k = 2:npts+1
if y(k-1)== 0
pk = p10/(p00 + p10);
y(k)=ceil(x(k) - (1-pk));
else
pk = p11/(p11 + p10);
y(k)=ceil(x(k) - (1-pk));
end
end
stem(y(1:200))
xlabel('Time')
ylabel('y(t)')
title('Time Series for Process Control State')
Running this code twice, gives the simulation segments in Figure 9 below.
Figure 9. Two 200-hour data log simulations using a 2-D Bernoulli random variable with probabilities EMBED Equation.3 , EMBED Equation.3 , and hence, EMBED Equation.3 .
Management feels that this model captures the grouping tendency of the ones, and so your model is approved. Congratulations!!!
Before we leave this example, lets think about the reasonableness of the ones grouping tendency. What this says is that when the process does go out of control, it has a tendency to remain out of control for more than one hour. In fact, the above conditional probability EMBED Equation.3
States that if it is out of control during one hour, then there is an 83% chance that it will remain out of control the next hour. This can point to a number of possible sources responsible for the process going out of control. Specifically, if the time constant associated with either transient nonhomogeneities in the chemicals, or with a partially blocked mixing valve is on the order of an hour, then one might have reason to investigate these sources. If either of these sources has a time constant on the order of hours, then the above model can be used for early detection of the source. Specifically, we can use a sliding window to collect overlapping data segments, and estimate the probabilities associated with EMBED Equation.3 . If a blockage in the mixing valve takes hours to dissolve, then one might expect the above probability value 0.8333 to increase. We can use this logic to construct a hypothesis test for determining whether we think the valve is blocked or not. We will discuss hypothesis testing presently. Perhaps this commentary will help to motivate the reader to look forward to that topic. j
4 . F u n c t i o n s o f n - D B e r n o u l l i R a n d o m V a r i a b l e s
H a v i n g a s o l i d g r a s p o f t h e s a m p l e s p a c e a n d p r o b a b i l i t y d e s c r i p t i o n o f a n n - D B e r n o u l l i r a n d o m v a r i a b l e i s c r u c i a l i n o r d e r t o a p p r e c i a t e t h e s i m p l i c i t y o f t h e m a t e r i a l i n t h i s s e c t i o n . I f t h e r e a d e r f i n ds this material difficult, then it is suggested that the previous sections be reviewed. Again, the concepts are (i) a random variable, which is an action that, if repeated could lead to different results, (ii) the sample space associated with the variable, which is the set of all measurable values that the variable could take on, and (iii) the probabilities associated with subsets of the sample space. The reader should place the primary focus on the nature of the action and on the set of measurable values that could result. In a sense, the computation of probabilities is after the fact; that is, once the events of concern have been clearly identified as subsets of the sample space, the probability of those events is almost trivial to compute. If the reader can accept and appreciate this view, then this section will be simple. Furthermore, as we arrive at some of the more classic random variables in most textbooks, the reader will not only understand their origins better, but will be able to readily relax the assumptions upon which they are based, if need be.
4.1 Functions of a 1-D Bernoulli Random Variable
A function is an operation, an algorithm, or an action. Due to the extremely simple nature of EMBED Equation.3 , there are not many operations that one can perform on X. One is the following:
Example 2.9 For EMBED Equation.3 , perform the following operation on X:
EMBED Equation.3 .
Since X is a random variable, and Y is a function of X, it follows that Y is also a random variable. In this case, it is the operation of multiplying X by the constant a, and then adding the constant b to it. The first step in understanding Y is to identify its sample space. To this end, perform the above operation on each element in EMBED Equation.3 , and the reader should arrive at EMBED Equation.3 . It should be apparent that the following sets are equivalent: EMBED Equation.3 and EMBED Equation.3 . Since they are equivalent, they must have the same probabilities: EMBED Equation.3 and EMBED Equation.3 .
Since a and b are any constants the readers chooses, it follows that any random variable Y that can take on only one of two possible numerical values is ,basically, just a veiled Bernoulli random variable. m
E x a m p l e 2 . 1 0 F o r E M B E D E q u a t i o n . 3 , p e r f o r m t h e f o l l o w i n g o p e r a t i o n o n X :
E M B E D E q u a t i o n . 3 .
E v e n t h o u g h t h i s o p e r a t i o n i s m o r e c o m p l i c a t e d t h a n t h a t o f t h e l a s t e x a m p l e , t h e r e a d e r s h o u l d n ot feel intimidated. Simply, proceed to identify the sample space for Y, in exactly the same manner as was done in the last example: EMBED Equation.3 and EMBED Equation.3 . Hence, EMBED Equation.3 , and so, again, we see that Y is simply a veiled Bernoulli random variable. m
4 . 2 F u n c t i o n s o f a 2 - D B e r n o u l l i R a n d o m V a r i a b l e
L e t E M B E D E q u a t i o n . 3 . S i n c e X i s c o m p l e t e l y a n d u n a m b i g u o u s l y d e f i n e d b y i t s a m p l e s p a c e , E M B E D E q u a t i o n . 3 a n d t h e a s s o c i a t e d p r o b a b i l i t i e s , E M B E D E q u a t i o n . 3 , t h e r e a d e r s h o u l d f e e l c onfident that he/she can easily accommodate any function of X. Consider the following examples.
Example 2.11 Perform the following operation on X:
EMBED Equation.3 .
We then have the following equivalent sets, or events, and theire associated probabilities:
EMBED Equation.3
Hence, the sample space for Y is EMBED Equation.3 , and the elemental subsets of this set have the above probabilities. Armed with this complete description of Y, the reader should feel competent and unafraid to answer any question one might pose in relation to Y.
For chosen numerical values, EMBED Equation.3 , we have EMBED Equation.3 . Notice that we do not have EMBED Equation.3 . Why? Because EMBED Equation.3 is the set of possible values that Y can take on. And so, it makes no sense to include the number 1 twice. In this case, the subset {1} of EMBED Equation.3 is equivalent to the subset {(1,0),(0,1)} of EMBED Equation.3 . With this awareness of the equivalence of sets, it is almost trivial to compute the probability
EMBED Equation.3
If the reader feels that the above equation is unduly belabored, good. Then the material is becoming so conceptually clear and simple that we are succeeding in conveyance of the same. If the reader is confused or unsure as to the reasons for the equalities in the above equation, then the reader should return to the previous sections and fill in any gaps in understanding.
Before we leave this example, consider the application where EMBED Equation.3 corresponds to the measurement of significant (1), versus insignificant (0) rainfall on two consecutive days. Suppose that on any given day, the probability of significant rainfall is p. Then, EMBED Equation.3 . If we assume that EMBED Equation.3 and EMBED Equation.3 are independent, then we arrive at the following probabilities for Y:
EMBED Equation.3
Notice that the rightmost quantities in these three equations are a consequence of the assumption of independence; whereas the middle quantities entail no such assumption. This leads to the question: Is it reasonable to assume that if a region experiences a significant amount of rainfall on any given day, then it might be more likely to experiences a significant amount on the next day? If the region is prone to experiencing longer weather fronts, where storms linger for more than on day, then the answer to this question would be yes. In that case, EMBED Equation.3 and EMBED Equation.3 are not independent. Hence, the rightmost expressions in the above equation are wrong; whereas the middle expressions are still correct. The caveat here is that one must have reliable numerical estimates of these probabilities. If one only has information about any given day, and not about two consecutive days, then one might resort to assuming the variables are independent. This is not necessarily a wrong assumption. But it is one that sho u l d b e c l e a r l y n o t e d i n p r e s e n t i n g p r o b a b i l i t y i n f o r m a t i o n t o f a r m e r s . m
E x a m p l e 2 . 1 2 P e r f o r m t h e f o l l o w i n g o p e r a t i o n o n X :
E M B E D E q u a t i o n . 3 .
T h i s o p e r a t i o n i s n o t a s f a b r i c a t e d a s i t m i g h t s e e m . Consider sending a text message to your friend. Most communications networks convert text messages into a sequence of zeros and ones. Each 0/1 is called an information bit. Now, let the event that you send a 1 correspond to EMBED Equation.3 , and let the event that your friend correctly receives it be EMBED Equation.3 .
Then, here, the event EMBED Equation.3 corresponds to a bit error in the transmission. We then have the following equivalent sets, or events, and their associated probabilities:
EMBED Equation.3 .
Hence, EMBED Equation.3 . Even though we have the joint probability information in the parameters EMBED Equation.3 , it is more useful to compute the conditional probability information. After all, what you are really concerned with is the event that your friend correctly receives a 0/1, given that you sent a 0/1. As in Example 2.8, these conditional probabilities are given by
EMBED Equation.3
Usually, it is presumed that each bit you send to your friend is as likely to be a zero as it is a one; that is, EMBED Equation.3 . In this case, the above error probabilities become
EMBED Equation.3
If it is further assumed that EMBED Equation.3 , then we arrive at the usual expression for the bit transmission error:
EMBED Equation.3 . m
4 . 2 F u n c t i o n s o f a n n - D B e r n o u l l i R a n d o m V a r i a b l e
R e c a l l t h a t a c o m p l e t e a n d u n a m b i g u o u s d e s c r i p t i o n o f a n n - D B e r n o u l l i r a n d o m v a r i a b l e r e q u i r e s s p e c i f i c a t i o n o f t h e p r o b a b i l i t i e s a s s o c i a t e d w i t h t h e E M B E D E q u a t i o n . 3 e l e m e n t a l s e t s i n the sample space for X. This sample space can be expressed as:
EMBED Equation.3 .
Denote the probability associated with the elemental set EMBED Equation.3 as EMBED Equation.3 . If one has access to m numerical measurements EMBED Equation.3 of X, then one can estimate EMBED Equation.3 by the relative number of occurrences of the element EMBED Equation.3 in relation to the number of measurements, m. In this section we will restrict our attention to the more common setting, wherein the components of X are mutually independent. It follows that
EMBED Equation.3 . (2.8a)
Now, since EMBED Equation.3 , we have
EMBED Equation.3 (2.8b)
We are now in a position to consider some classic functions of EMBED Equation.3 .
Example 2.13 Define the random variable Y, which is the smallest index, k, such that EMBED Equation.3 . For example, suppose n=5. Then, in relation to the elements (0,1,0,0,1) and (0,0,1,1,0), the associated values for Y are 2, and 3, respectively. More generally, the sample space for Y is EMBED Equation.3 . Before we compute the probabilities associated with the elemental subsets of this sample space, lets find the equivalent events in EMBED Equation.3 . Specifically,
EMBED Equation.3 .
Hence, in view of the assumption that the components of EMBED Equation.3 are mutually independent, we have
EMBED Equation.3 . (2.9a)
If we assume that, not only are the components of X mutually independent, but that they all have exactly the same p-value, then we obtain the following well known geometric probability model for Y:
EMBED Equation.3 (2.9b)
Of course, the expression (2.9b) is simpler than (2.9a). It should be, since all the p-values are assumed to be the same. However, (2.9a) is often a more realistic situation than (2.9b). All too often, the assumption of equal p-values is born of convenience or of lack of understanding. We will see this same assumption in the next example.
Example 2.14 In Example 2.7 we used Matlab to investigate the probability description for the p-value estimator associated with the assumedly independent and identically distributed (iid) Ber(p) components of EMBED Equation.3 , given by (2.7), and repeated here:
EMBED Equation.3 .
In this example, we will obtain the actual probability model for EMBED Equation.3 . Furthermore, we will obtain it in the more general (and often more realistic) case, where the components are independent, but they do not have one and the same p-value. To this end, we first give the sample space for the following random variable:
EMBED Equation.3
The sample space for Y is
EMBED Equation.3 .
Notice that this sample space has n+1 elements in it. With this, we are in a position to identify the subset of EMBED Equation.3 that corresponds to the elemental subset {k} of EMBED Equation.3 . We begin with the two simplest subsets:
EMBED Equation.3 and EMBED Equation.3 .
The key point here is that the only way that Y can take on the value zero (n) is if every component of X takes on the value zero (one). In the more general setting wherein the components are assumed to be mutually independent, but with not necessarily the same p-value, the probabilities of these two events are simply
EMBED Equation.3
and
EMBED Equation.3 .
Next, we consider the slightly more challenging events EMBED Equation.3 and EMBED Equation.3 . In particular, the only way that Y can take on the value one is if one and only one of the components of X equals one. Similarly, the only way that Y can take on the value EMBED Equation.3 is if one and only one of the components of X equals zero. Hence, we have the following equivalent events:
EMBED Equation.3 only one component of X equals 1 EMBED Equation.3
and
EMBED Equation.3 only one component of X equals 0 EMBED Equation.3
Notice that the elements making up these sets are distinct. For example, the element EMBED Equation.3 is a point in n-D space that is distinctly different than the point EMBED Equation.3 . Sure, many of the components of these two elements are the same. But the elements are distinctly separate points in n-D space. Their intersection is, therefore, the empty set. Hence, the probability of the event EMBED Equation.3 is simply the sum of the elemental probabilities associated with the set EMBED Equation.3 ; that is:
EMBED Equation.3
Similarly,
EMBED Equation.3
While to many readers, these expressions may seem formidable, if not down right ugly, such readers should carefully assess whether their queasiness is due to the unfamiliar notation, or to a lack of conceptual underst a n d i n g . T h e f i r s t p o r t i o n s o f e a c h o f t h e s e e q u a t i o n s w e r e l o n g h a n d e x p r e s s i o n s , w h e r e i n e a c h p r o b a b i l i t y i s n o t e d i n d i v i d u a l l y . T h e s e c o n d p o r t i o n s o f t h e s e e q u a t i o n s u s e s u m m a t i o n ( ") a n d p r o d u c t ( ") n o t a t i o n , i n o r d e r t o m a k e t h e e x p r e s s i o n s m o r e c o mpact. The reader who is unfamiliar with this type of notation should not misconstrue discomfort over that, with lack of conceptual understanding.
Now, lets consider the most general event EMBED Equation.3 . This event is the event that k of the n components of X take on the value one, while the others take on the value zero, right? Well, one way corresponds to the event EMBED Equation.3 . Notice that we have included subscripts on the 0s and 1s simply to make it clear as to their positions in the ordering of the n components of this element. It should be clear that the probability of this event is simply
EMBED Equation.3 . (2.10)
The question now is: How many ways can one position k ones and (n-k) zeros in n slots?
Each way will have a corresponding probability; just as the way of positioning all the ones first, (followed by all the zeros), resulted in the above probability. The answer to this question begins by answering a similar question:
How many ways can one order n distinctly different objects in n slots?
Well, in the first slot we can place any one of the n distinct objects. Once we have chosen one of them, then in the second slot we have only EMBED Equation.3 distinct objects to choose from. Having chosen one of them, we are left to choose from EMBED Equation.3 distinct objects to place in the third slot, and so on. Think of this as a tree, where each branch has branches, and each of those branches has branches, and so on. The question is to figure out how many total branches there are. Lets identify the first slot with the biggest diameter branches. Then we have n of these, corresponding to the n distinct objects. Now, each one of these main branches has EMBED Equation.3 branches of slightly smaller diameter. And each one of those branches has EMBED Equation.3 slightly smaller branches, and so on. So, the total number of branches is
EMBED Equation.3 (read as n factorial)
Now, if each of the k + , 8 9 H I J
& ' o u
e f g r u v z { 8 9 @ A D R [ \ ] _ p q r ! źź|r| h4V hG<} 56 h4V hB 56 hB 56 hl h4V hB 6 h4V hB hB 5hB hB 5>* hB hB 56>* h4V hG<} h$= h4V h " hMa hMa hMa 5 hMa 5 h " 5h " 56CJ aJ h " 5CJ aJ hMa 5CJ aJ , 8 9 I J f g q r = > C D
< gdHV
< gdMa
< gd! f ! + . / 3 4 ` a t u v w # $ ' ( ) + < = > Ҭzunununib h4V h%g h%g 5 h! f 56 h! f 5jP h4V h! f EHU#jH
h4V h! f CJ UVaJ j h4V h! f Uh4V h! f 6 h4V h! f j h4V h! f EHU#jH
h4V h! f CJ UVaJ j h4V hB Uh4V hB 6 hB 5>* hB 56>* hB 5 h4V hB % ! / s 0 x { 6 3 6 н~~zvvqvq hHV 6hHV hv29 hb 6jn
h4V h$= EHU#jH
h4V h$= CJ UVaJ h4V h$= j h4V h$= Uh$= h$= 6hb h4V hv29 6 h4V hv29 hv29 5j h%g hv29 5U hMa 6hMa h%g 5 h4V h%g h4V h%g 6 ,6 B C D O R j o 4 " # 6 7 8 9 < B E F Y Z [ \ d e x y z мЭĖćxмi\j> hM h]N EHU jH
h]N CJ UVaJ j hM h]N 5EHU jH
h]N CJ UVaJ j hM 5U j hM hM EHU jH
hM CJ UVaJ j hM U hM 5 hM 56 hM hMa hHV 6hM hHV 6 hM hHV hHV 5 hHV 56 hb hHV $D ; < ! ! ,! -! " " ' ' ;( <( h, i, , , p. q. .
< gd! f
< gdHV z { u j h]N h]N EHUjg h]N hm EHU jH
hm CJ UVaJ jb h]N h]N EHU jH
h]N CJ UVaJ h]N 5 h]N 56 hM h]N 6jq h]N h]N EHU jH
h]N CJ UVaJ j h]N Uh]N j hM U * [ ] ! ! ! ! *! +! ,! -! 6! t! u! ! ! ! ! ! ! " " "
" " " " I" J" t" u" z" {" " " " аСаСЃ~~~~v j hLZ U hLZ 6hLZ j> h_ h_ EHUj< h_ h_ EHU jH
h_ CJ UVaJ j h_ U h_ 5hb hy hy j h_ Uh_ hMa hy 6hy h]N 6h]N j h]N Uj h]N h]N EHU *" " " " " " # !# 4# 5# 6# 7# # # $ $ f$ g$ z$ {$ |$ }$ $ $ % % & & $& %& 8& 9& :& ;& +' ϸϦϦϞ~yyyj] jXG h) h) EHU jH
h) CJ UVaJ h) 6j E hLZ h) EHU j"H
h) CJ UVaJ h) j h) U hm 6jC h]N hm EHU jH
hm CJ UVaJ j hm Uhm h_ hLZ j hLZ Uj@ hLZ hLZ EHU j"H
hLZ CJ UVaJ "+' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ( ( ( ( ( ( "( #( 6( 7( 8( 9( ( ( ( ( ( ( ( ( ( ( ( ( 0) 1) D) E) |o j?P hn hn EHU jH
hn CJ UVaJ hn 6jM hn hn EHU j
H
hn CJ UVaJ jK h) hn EHU jH
hn CJ UVaJ jI h]N hn EHU jH
hn CJ UVaJ j hn U hn 5 hn 56 hm hn +E) F) G) O) P) c) d) e) f) ) ) ) ) ) ) ) ) ) ) ) ) +* ,* ?* @* A* B* D* U* V* i* j* k* l* * * + + 2+ 3+ 殡ה搈ylgy h?= 6j\ h]N h?= EHU jH
h?= CJ UVaJ j h?= Uh?= jZ h]N hn EHUjX hn h?= EHU jbH
h?= CJ UVaJ jV h]N hn EHUjT h]N hn EHU jH
hn CJ UVaJ hn j hn UjR h]N hn EHU '3+ 4+ 5+ J+ K+ ^+ _+ `+ a+ y+ z+ + + + + + + ;, <, O, P, Q, R, c, f, g, h, i, , , , , , , , , 滮暍zk\z jf h?= h?= 5EHU j H
h?= CJ UVaJ j h?= 5U h?= 5hn jd hn h?= EHU jH
h?= CJ UVaJ h?= 6jb hn h?= EHU jbH
h?= CJ UVaJ j` h]N h?= EHU jH
h?= CJ UVaJ h?= j h?= Uj^ h]N h?= EHU #, , , , , , , , , - I- - - - - - - - - - - . . 0. 1. 2. 3. e. h. o. p. . . . . . . . . þöçþöË~þöobö jo hud hud EHU j
H
hud CJ UVaJ jm hn hud EHU jH
hud CJ UVaJ jk h?= hud EHU jH
hud CJ UVaJ j hud U hud 6hud h?= 6jui h?= h?= EHU jH
h?= CJ UVaJ j h?= Uh?= hE h?= B*ph &. . . . ,/ / / / 0 0 1 1 2 2 4 4 4 4 4 8 8 9 9 : : : : < <
< gd! f . . . . . . . / / /
/ / / (/ )/ */ +/ 7/ 8/ K/ L/ M/ N/ / / / / / / / / / / / / ɼxi\xx j{ h'p6 h'p6 EHU jH
h'p6 CJ UVaJ j h'p6 Uh'p6 jty hud hud EHU jH
hud CJ UVaJ jv hud hud EHU jH
hud CJ UVaJ jyt hud hud EHU jvH
hud CJ UVaJ j hud Uj;r hud hud EHU j4H
hud CJ UVaJ hud "/ / / / 0 0 .0 /0 E0 H0 \0 ]0 p0 q0 r0 s0 0 0 0 0 $1 )1 o1 p1 1 1 1 1 1 1 1 1 1 1 1 1 N2 2 2 2 2 t jH
hE5 CJ UVaJ j hE5 UhE5 h'p6 >*j h?= h'p6 EHUj h'p6 h'p6 EHU j^H
h'p6 CJ UVaJ hE5 h'p6 >*j h?= h'p6 EHU h'p6 6h'p6 j h'p6 Ujg~ h?= h'p6 EHU jH
h'p6 CJ UVaJ (2 2 2 2 2 2 2 3 3 3 3 3 3 :4 ;4 N4 O4 P4 Q4 4 4 4 4 4 4 4 ]5 )6 ,6 6 6 6 7 7 7 i8 k8 8 8 8 8 沥枚|w|pi h? 56 h>E hl h>E 6h>E h? 6h? hMa hl 5CJ aJ hl hm hE5 hE5 j hE5 hE5 EHU jH
hE5 CJ UVaJ j܈ h?= hE5 EHU jH
hE5 CJ UVaJ hE5 >*h'p6 hE5 j hE5 Uj h?= hE5 EHU (8 8 8 8 8 8 8 8 8 8 9 9 9 9 b9 c9 v9 w9 x9 y9 9 9 9 9 9 9 9 9 9 %: &: 9: :: ;: <: e: r: özk^z j h? h EHU jPH
h CJ UVaJ j h U h 6h>E h 56 h h? 6je h? h? EHU jPH
h? CJ UVaJ j& h? h? EHU jH
h? CJ UVaJ j h? h? EHU jH
h? CJ UVaJ j h? Uh? h? 5 $r: : : : : : : : : : : : : : ; ; ; ; ; ; ; ; ; ; < < < < < < < < < < < < < /= 0= C= D= 履t j
H
ha% CJ UVaJ j ha% ha% EHU j H
ha% CJ UVaJ j ha% Uha% j h'p6 h EHU j^H
h CJ UVaJ h>E j
h h EHU jH
h CJ UVaJ j h Uhh} h? h 6h (< = = = = = = = = = = = = = = = = = > > "> $> V> > ? a? b? +B gdR gdv
< gd! f D= E= F= = = = = = = = > > !> "> #> $> U> V> ? ? E? N? `? a? b? ? ? @ *@ -@ \@ ^@ @ @ GA LA A A *B +B ,B XB YB lB ͻɶɱ歩~v j hh} U hh} 56 hh} hh} H* h>E hh} h>E 6 h>E h>E h>E hh} 6hh} hR h|. 6 h|. 5j h|. UmH nH u h|. hv hv j hv UmH nH u hv ha% j ha% Ujw ha% ha% EHU ++B ,B B B B B C C `C aC C C C C 6D 7D ]D ^D D D D D E E TE UE E E E E gdR lB mB nB oB B B B B B B B C C 5D =D [D E ZE [E nE oE pE qE uE vE E E E E E E F üzk^ j h"6 h"6 EHU jH
h"6 CJ UVaJ jܡ h"6 h"6 EHU jH
h"6 CJ UVaJ j h"6 Uh"6 h o hzL 5B*
ph p hzL >*hzL hh} hh} hh} hh} >* hzL hh} hh} 5 hh} 56 j hh} Uj h?= hh} EHU jH
hh} CJ UVaJ F F F F F F F F F F F F F F F F F H H I I ?I _I I I I I I I 0J 1J DJ EJ Ȼכ{n_ j H
hf> CJ UVaJ j ho(% ho(% EHU jH
ho(% CJ UVaJ j ho(% U ho(% 6h>E ho(% j h"6 h"6 EHU jH
h"6 CJ UVaJ j h"6 h"6 EHU jH
h"6 CJ UVaJ h"6 j h"6 UjХ h"6 h"6 EHU jBH
h"6 CJ UVaJ E HF F F F H H I I I I J J IJ JJ J J K K M M N N O O P R
< gdQ
gd"6
gd"6 EJ FJ GJ HJ IJ J J J J J J J J J K K vK wK K K K K K K K K K K K L L L L L öڢړ{wrwj[ j H
hQ CJ UVaJ j h0 U h\}s 6h\}s h0 6h0 j. hf> h0 6EHU j H
h0 CJ UVaJ j hf> hf> 6U hf> 56 j hf> hf> EHU jN H
hf> CJ UVaJ j hf> Uhf> hf> 6ho(% hf> j ho(% Uj߭ ho(% hf> EHU !L L L L L L L L L L *M 3M dM nM xM |M M M M M M M M M M M M N !N VN WN jN kN lN ƺ}sdU j hQ hQ 6EHU jJ H
hQ CJ UVaJ j hQ 6U j hQ hQ 6EHU &jPH
hQ hQ 6CJ UVaJ j hQ hQ 6U hQ hQ 6 hQ 5 hQ 56 hQ 6hQ j h0 hQ EHU j: H
hQ CJ UVaJ h0 j h0 Uj h0 hQ EHU !lN mN N N N N N N N &O /O O O O O O O O P
P P !P "P #P 'P (P ;P ht 6EHU j H
ht CJ UVaJ hf> ht 6j hf> ht 6U ht hQ j hf> hQ 6EHU j H
hQ CJ UVaJ hf> hQ 6j hf> hQ 6U hQ 6j hQ 6U
P P P P P P P P P P %Q &Q 9Q :Q ;Q R ?R @R AR `R aR tR uR ˾⬤tg j h s h s EHUj h s h s EHU j[ hQ h s 6EHU &jPH
hQ h s 6CJ UVaJ hQ h s 6j hQ h s 6U h s 6jQ h s h s EHU j H
h s CJ UVaJ j h s Uh s ht j ht UjF h0 ht EHU %R R R R T T `U aU U U U U U 0W 1W L] M] ^ ^ ^ ^ ` ` ` ` b b *c +c
< gdQ uR vR wR (S )S S ?S @S FS GS ZS [S \S ]S S S S S S S S S T T 7T 9T OT ST gT iT T T T T T T T T ,U -U @U AU BU CU 淪楡杘桐t j ho ho EHU jj$H
ho CJ UVaJ j ho U hm 6hm ho h s 6j h s h s EHU j H
h s CJ UVaJ h\}s j h s h s EHU j"H
h s CJ UVaJ h s j h s Uj h s h s EHU -CU GU HU [U \U ]U ^U aU bU uU vU wU xU U U U U U U U U U U U U
V V V
V V V %V }yj] jX h% h% EHU j8H
h% CJ UVaJ h\}s j h% h% EHU j7H
h% CJ UVaJ j h% Uh% j ho hGi EHU j$H
hGi CJ UVaJ j ho U j6 hf> ho 6EHU j H
ho CJ UVaJ hf> ho 6j hf> ho 6U ho %V &V 'V (V >V GV mV pV V V V V V V W W *W +W ,W -W 0W kW lW W W Žגvgc[cL j8H
h
IP CJ UVaJ j h
IP Uh
IP j hQ h% 6EHU &jPH
hQ h% 6CJ UVaJ hQ h% 6j hQ h% 6U jl hf> h% 6EHU j H
h% CJ UVaJ hf> h% 6j hf> h% 6U h% 6h% j h% Ujb h% h% EHU j8H
h% CJ UVaJ W W W W W W W W W W W W W W W W X X %X &X 'X (X ,X -X @X AX BX CX GX `X X X X 渰~qbU j h
IP h
IP EHU j:H
h
IP CJ UVaJ j h
IP h
IP EHU j:H
h
IP CJ UVaJ j. hQ h
IP 6EHU &jPH
hQ h
IP 6CJ UVaJ hQ h
IP 6j hQ h
IP 6U h
IP 6j$ h% h
IP EHU j8H
h
IP CJ UVaJ h
IP j h
IP Uj h% h
IP EHU X X X X X X Y Y Y Y DY Y Y Z &Z TZ UZ hZ iZ jZ kZ Z Z Z '[ K] L] M] k] ] ] ] ] ȻvrrnnnfnW jJ;H
hc!> CJ UVaJ j hc!> Uhc!> h" j hQ h> 6EHU &jPH
hQ h> 6CJ UVaJ hQ h> 6j hQ h> 6U h> 6h\}s h> j h
IP h
IP EHU j;H
h
IP CJ UVaJ h
IP j h
IP Uj h
IP h
IP EHU jJ;H
h
IP CJ UVaJ ] ] ] ] ] ] ] ] ] ^ ^ l_ m_ <` E` V` W` j` k` l` m` ` ` ` ` ` ` ` ` a a 5a 6a wa ÿ濺zk^zV ha ha 6j ha ha EHU jBH
ha CJ UVaJ j ha U jw hQ ha 6EHU &jPH
hQ ha 6CJ UVaJ hQ ha 6j hQ ha 6U ha 6ha hc!> 56 jo h
IP hc!> EHU j;H
hc!> CJ UVaJ hc!> j hc!> Uje h
IP hc!> EHU !wa xa a a a a a a a a a a a a a b b c c c c !c "c c c c d Nd Pd Qd d d
e e e e e !e fe Ȼsf j ha ha EHU jEH
ha CJ UVaJ j ha Uha j hNRP hNRP EHU jDH
hNRP CJ UVaJ j hNRP U hNRP 6hNRP jV ha hNRP EHU jBH
hNRP CJ UVaJ j" ha hNRP EHU jBH
hNRP CJ UVaJ ha j ha U &+c d d )e *e i i Cj Dj pj qj Xk Yk k k Xl Yl Hm Im hm im p p Ap Bp ap bp q q
< gdQ fe se ze {e e e e e e e e e e e e e e e e e >f @f If Jf ]f ^f _f `f uf vf f f f f f f f f g g öÍ~q j ha ha EHU jGGH
ha CJ UVaJ j ha ha EHUj ha ha EHU jFH
ha CJ UVaJ js ha ha EHU jBH
ha CJ UVaJ j
ha ha EHU jYFH
ha CJ UVaJ j ha Uha ha 6 'g g g ;g hZB 6EHU j H
hZB CJ UVaJ hf> hZB 6j hf> hZB 6U hZB j hQ hZB 6U j2 hQ hZB 6EHU q 8r 9r wr xr r r r r 8s 9s xs ys s s
t t Nt Ot Pt t t
< h^hgd73
< h^hgdz
& F
< gdz
< h^hgdZB
& F
< gdZB
< gdQ r r r r r r r r r r r r r r r r s s s s s s s s 3s 4s 5s 6s >s ?s Rs Ss Ts Us Ws öײףׇzk^ jF hZB hz EHU j$:H
hz CJ UVaJ jC hZB hz EHU j:H
hz CJ UVaJ jA hZB hR`l EHU ju9H
hR`l CJ UVaJ hZB j> hZB hR`l EHU j7H
hR`l CJ UVaJ hR`l 6hR`l j hR`l Ujv< hZB hR`l EHU j7H
hR`l CJ UVaJ "Ws ]s _s `s ss ts us vs ws xs ys s s s s s s s s s s s s s s s s s s t t
t t t t (t ɼzk^ jP hz hz EHU j hz 6EHU j H
hz CJ UVaJ hf> hz 6j hf> hz 6U hz 5hR`l j I hZB hz EHU j:H
hz CJ UVaJ j hz Uhz hz 6 #(t )t *t +t -t 3t 5t 6t It Jt Kt Lt Mt Nt Pt Vt Wt jt kt lt mt ot ut wt xt t t t t t t t t t t t öײן׃vgZ j\ hZB hJ: EHU j`=H
hJ: CJ UVaJ jWZ hJ: hJ: EHU jv Iv Jv ]v ^v _v `v ev fv yv zv ½uh`Q j H
hM CJ UVaJ hf> hM 6j hf> hM 6U jb hQ hM 6EHU &jPH
hQ hM 6CJ UVaJ hQ hM 6j hQ hM 6U hM hM 5 hM 56 hZB 5 hJ: hJ: hJ: 6hJ: j]_ hJ: hJ: EHU j=H
hJ: CJ UVaJ j hz Uhz hz 6 t t t 1v 2v Dw Ew w w x x x x z z { Z{ [{ \{ ]{ _{ { {
gd"6
< gdQ
< h^hgdZB
< gdM
< gdJ:
< h^hgdJ: zv {v |v v v v v v v v v v v v v w w w w w w *w +w >w ?w @w Aw Bw Cw Dw Ew cw dw ww xw ɼਛ{{vlv] jGAH
h]5 CJ UVaJ j h,k 5U h,k 5h,k jk hM h,k EHU j@H
h,k CJ UVaJ jh hM hM EHU ja@H
hM CJ UVaJ hM 6jf hM h,k EHU j@H
h,k CJ UVaJ j hM UhM j hf> hM 6U jWd hf> hM 6EHU"xw yw zw w w w w w w w w w w w w w w w w w w w w lx mx x x x x x x ĵذӗqb]V hne( hne( hne( 5 jq h]5 h]5 5EHU jsBH
h]5 CJ UVaJ hne( 6hne( h]5 h]5 6 hZB 5h,k h,k 5hoP hoP 5H* hoP 56 hoP 5 jo h]5 h]5 5EHU jAH
h]5 CJ UVaJ h]5 5j h]5 5U h,k 5j h,k 5U jm h,k h]5 5EHU x x x x x x x x x x x x y y (y )y *y +y y y z z z z z z Sz Uz z z z z z z z z rmeaa h h h H* h 6jx hne( hne( EHU j>uH
hne( CJ UVaJ hne( 6jv hne( hne( EHU jsH
hne( CJ UVaJ j hne( Uhne( hne( 5 hZB 5 jt h]5 h~ 5EHU jBH
h~ CJ UVaJ j h]5 5U h]5 5 h]5 h]5 hne( h]5 #z z {
{ { { { B{ C{ V{ W{ X{ Y{ Z{ [{ ]{ ^{ _{ h{ i{ r{ s{ { { { { { { { { X| Y| l| ķsdӌ`X j h UhQ j, h,k h 5EHU jGAH
h CJ UVaJ j h 5U h h 5 h 5 hKJ h jt hKJ hKJ Uh&k h h j6~ h h EHU jwH
h CJ UVaJ hD1 j1{ h h EHU jvH
h CJ UVaJ h j h U l| m| n| o| v| | | | | | | | | | | | | } } } } &} '} (} )} <} =} P} Q} R} S} ,~ 5~ ~ ~ ~ ~ ~ ~ ~ ~ , ķ۳{nӳ׳ j hY{ hY{ EHU jH
hY{ CJ UVaJ j hY{ hY{ EHU jҖH
hY{ CJ UVaJ j hY{ U hY{ 6hY{ j h hY{ EHU jEH
hY{ CJ UVaJ h'_ h j h Ujϓ hY{ hY{ EHU jYH
hY{ CJ UVaJ ){ ~ ~ I J ԅ Յ ` 3 4 Q m 6 7 ; <
& F
8h hL^h`Lgdp
gd"6 , - @ A B C D & ' : ; < ϸϜπsogoXK j! hY{ hk EHU jH
hk CJ UVaJ j hk Uhk j hY{ hk EHU jEH
hk CJ UVaJ j hY{ hk EHU jH
hk CJ UVaJ jg hY{ hY{ EHU jJH
hY{ CJ UVaJ j hY{ UhY{ jĜ h,k hY{ 5EHU jGAH
hY{ CJ UVaJ hY{ 5j hY{ 5U < = U V i j k l ـ ڀ & ' ( ) ; O с ҁ Ȼre j hk hk EHU jH
hk CJ UVaJ jG hY{ hk EHUh'_ j= hk hk EHU jH
hk CJ UVaJ j hY{ hk EHUj hk hk EHU jH
hk CJ UVaJ jV hY{ hk EHU jsH
hk CJ UVaJ hk j hk U ' * + > ? @ A U [ \ o p q r ς Ђ т ҂ ŸteX j hk h|&j EHU jH
h|&j CJ UVaJ j4 h|&j h|&j EHU jwH
h|&j CJ UVaJ j h|&j Uh|&j j* hk h|&j EHU j4H
h|&j CJ UVaJ j hY{ hk EHU jsH
hk CJ UVaJ j hk hk EHU jH
hk CJ UVaJ j hk Uh'_ hk " L M ` a b c L M ` a b c Ȅ Ʉ ܄ Ȼtgc[c j h Uh ju hk h|&j EHU jH
h|&j CJ UVaJ j4 hY{ h|&j EHU jsH
h|&j CJ UVaJ j* h|&j h|&j EHU jH
h|&j CJ UVaJ j h|&j h|&j EHU j6H
h|&j CJ UVaJ j h|&j h|&j EHU jH
h|&j CJ UVaJ h|&j j h|&j U !܄ ݄ ބ ߄ H I J T X ] ^ b e ƅ х Ӆ ԅ Յ ݅
Ȼװצצחװצ׆zzuzkz j hoP 5U h'_ 5 hoP 5 hoP 56 h'_ j h hw EHU j͞H
hw CJ UVaJ h 6 h 5 h 56 hY{ j h|&j h EHU jH
h CJ UVaJ h j h Uj hY{ h EHU jsH
h CJ UVaJ & ! 2 3 F G H I O R S [ \ ] ^ _ ` h 0 1 8 9 ôҩҢҚteaY j hoP Uh j hoP hoP 5EHU jSH
hoP CJ UVaJ h hoP hoP 6hoP hoP 5H* hoP hoP 5 hoP 56 hoP hoP 56>* j h]5 hoP 5EHU jAH
hoP CJ UVaJ hoP 5j hoP 5U jq h,k hoP 5EHU jGAH
hoP CJ UVaJ "9 L M N O P Q U V i j k l q r Խԡԍԅ}na}]X] h~ 6h~ j h h EHU j H
h CJ UVaJ j h Uh hp hoP h j hp hp EHU j%H
hp CJ UVaJ j h h EHU j H
h CJ UVaJ j h Uh j hoP Uj h h EHU j H
h CJ UVaJ hoP " 1 2 3 4 5 6 7 ? a d Ȍ Ɍ ܌ ތ ߌ = νήνΒνvie]e j hkX UhkX j h~ hp EHU j&H
hp CJ UVaJ jB h hkX EHU j"H
hkX CJ UVaJ ja h~ h~ EHU j!H
h~ CJ UVaJ j h~ U h~ 5hoP h~ j h h~ EHU jO H
h~ CJ UVaJ j h Uh h~ 6 # ~ < C H i j
P Q d ùê̏̅}na}̅̅} j0 h$ h$ EHU jH
h$ CJ UVaJ j h$ U h$ 6 h$ 5 h$ 56 h[ 6 j hp h[ 6EHU j'H
h[ CJ UVaJ j hp 6U hp 6hp h$ hoP h~ h~ j hkX Ujp hkX hp EHU jA$H
hp CJ UVaJ d e f g Ґ Ӑ Ԑ Ր $ ' I J ] ^ _ ` ηΥΞΥΥΥΏΥ~ob j5 hZ hOP2 EHU jH
hOP2 CJ UVaJ h< j hZ hZ EHU jH
hZ CJ UVaJ hZ hZ hZ 6j h$ hZ EHU jH
hZ CJ UVaJ j hZ UhZ h$ 6h$ j h$ Uj h$ hZ EHU jH
hZ CJ UVaJ & p q ɓ ӓ ד ؓ , . 0 2 @ P X Z yuqfjp h [_ h [_ Uh [_ h$ jK h$ hOP2 EHUj& hOP2 hOP2 EHU jBH
hOP2 CJ UVaJ j hOP2 hOP2 EHU j#H
hOP2 CJ UVaJ hOP2 hOP2 6j h$ hOP2 EHUj h$ hOP2 EHU jH
hOP2 CJ UVaJ j hOP2 U hOP2 6hOP2 ( 7 8 i j B D Z [ * + &
gd"6 ̔ Δ < > d f h j " & 8 C F G K L P S m n 𩥡yld j h9 Uj6 hOP2 hOP2 EHU jnH
hOP2 CJ UVaJ j hOP2 U hOP2 5 hOP2 56 hOP2 6hOP2 h9 h [_ hOP2 j h$ h [_ EHU jH
h [_ CJ UVaJ j hOP2 h [_ EHU jBH
h [_ CJ UVaJ j h [_ Uh [_ h [_ 5 h [_ h [_ & ? [ \ f h i m u ˗ ̗ ߗ Ȼױי}pldl j h [_ Uh [_ j@! h9 h9 EHU j$H
h9 CJ UVaJ j h9 h9 EHU jH
h9 CJ UVaJ hOP2 h9 6h9 h9 56 j h9 h9 EHU jcH
h9 CJ UVaJ h9 j h9 Uj h9 h9 EHU jH
h9 CJ UVaJ ! ˘ Ә t ڜ ޜ ~ 9 I Ɵ ޟ ߟ Y Z [ g h i o r } j)H
hr CJ UVaJ j hr U hr 56 h5 56 hr 6 h" 6h" hr h73 h9 h?P 6h?P j% h [_ h [_ EHU h [_ 6h [_ j h [_ Uj# h [_ h [_ EHU j7H
h [_ CJ UVaJ . = > C D T U b d o p v z { ¡ š ١ ڡ ) * + ! " 5 捆y j hr U hr 5 hr 56 hr h5 56 h5 6j * h5 h5 EHU j*H
h5 CJ UVaJ j h5 Uh5 h5 h5 56 h5 hr 56 h5 hr 5 hr 6hr j hr Uj' hr hr EHU /5 6 7 8 ] ^ ʣ ˣ ̣ ͣ & . 1 2 E F G H K O R S f g h i k
öײף۲ײׇzײrcV jK6 hR< hR< EHU jmH
hR< CJ UVaJ j hR< Uj3 hr hR< EHU jlH
hR< CJ UVaJ j0 hr hR< EHU jlH
hR< CJ UVaJ hR< jf. hr hr EHU jkH
hr CJ UVaJ hr 6hr j hr Uj_, hr hr EHU j#kH
hr CJ UVaJ !& k I J k l ' ( B C z { ګ ۫ S T 4 5 6 7 f
gd"6 H I R S f g h i إ # $ 7 8 9 : @ A T U V W ķsdWjBB hR< h EHU joH
h CJ UVaJ j8@ h h EHU jhoH
h CJ UVaJ j h Uh j= hR< h EHU joH
h CJ UVaJ j; hR< h EHU jnH
h CJ UVaJ j8 hR< hR< EHU jnH
hR< CJ UVaJ hr hR< j hR< U "W \ ] p q r s Ԧ զ ֦ צ ܦ ݦ Ƨ ǧ ȧ ɼuh[ jZO h h EHUjPM h h EHU jhoH
h CJ UVaJ j
K hR< h EHU j"pH
h CJ UVaJ jH h h EHU jpH
h CJ UVaJ jF h h EHU joH
h CJ UVaJ jD hR< h EHU jnH
h CJ UVaJ j h Uh #ȧ ɧ & ' M N a b c d h i | ûìûÐûtgû jX hR< h)N EHU jnH
h)N CJ UVaJ jV h)N h)N EHU jmrH
h)N CJ UVaJ jS h)N h)N EHU jqH
h)N CJ UVaJ j h)N Uh)N jcQ hr hcP EHU jkH
hcP CJ UVaJ j hcP UhcP hR< h j h U "| } ~ A B Z [ n o p q Ϋ ϫ Ы ѫ 4 5 H I J K R S ӸӜӋ|oNjӋgX jmrH
hR CJ UVaJ j hR Ujab hD1 hR EHU juH
hR CJ UVaJ hR j_ hD1 hD1 EHU j`uH
hD1 CJ UVaJ j] hD1 hD1 EHU jtH
hD1 CJ UVaJ j hD1 UhR< hD1 h)N j h)N UjZ hR< h)N EHU joH
h)N CJ UVaJ 5 6 F G Z [ \ ] g s t K L _ ` a b f g z º«ººsfº jn hR< hQ } EHU jnH
hQ } CJ UVaJ jdl hD1 hQ } EHU jfyH
hQ } CJ UVaJ jUj hQ } hQ } EHU j*yH
hQ } CJ UVaJ j hQ } UhQ } jqg hD1 hQ } EHU jxH
hQ } CJ UVaJ h/} hR j hR Ujd h)N hR EHU $f g ı β ϲ . / e f
gdQ }
gd"6 z { | } Ʈ ܮ " # W X k l m n ݯ ޯ yjbUb j hz hz 6U hz hz 6 ju hz hQ } 6EHU &jqH
hz hQ } 6CJ UVaJ jCs hz hQ } 6EHU &jkH
hz hQ } 6CJ UVaJ j hz hQ } 6U hz hQ } 6 hQ } 5hD1 hC`, 6hC`, hQ } j hQ } Ujp hR< hQ } EHU joH
hQ } CJ UVaJ A B U V W X ñ ı ű ر ٱ ڱ ۱ ݱ Ȥг~qiZKqGC h( hd j| hz hd 6EHU j}H
hd CJ UVaJ hz hTO 6j hz hTO 6U hTO hz hTO hTO 5 hTO 56 jKz hz hz 6EHU j(|H
hz CJ UVaJ hz hz 6hR hz hQ } 6hz hz 6j hz hz 6U jw hz hz 6EHU &jfyH
hz hz 6CJ UVaJ
I J ] ^ _ ` ò IJ Ų Ʋ ) * + , / E F Y Z ®qd`Q jH
h`_ CJ UVaJ h`_ jN h( h( EHU j hz h( 6EHU j=H
h( CJ UVaJ h( 6h67- h( hR j6 hz h( 6EHU &jfyH
hz h( 6CJ UVaJ hz h( 6j hz h( 6U j( h( h( EHU jƁH
h( CJ UVaJ h( j h( U Z [ \ d e < = P Q R S v w ˾毢ڞ枑zkcT jH
hf5 CJ UVaJ j hf5 U j hz hf5 6EHU j}H
hf5 CJ UVaJ hz hf5 6j hz hf5 6U hf5 j$ h`_ hf5 EHU jH
hf5 CJ UVaJ j h`_ h`_ EHU jIH
h`_ CJ UVaJ j h`_ Uh( h`_ j h( Uj\ h( h`_ EHU l m Z [ ֿ Q R \ ] ~
gd"6 Ҵ Ӵ i j } ~ ɵ ʵ ˵ ̵ D F 滮柒vid hf5 6j hf5 hf5 EHUj hf5 hf5 EHU jH
hf5 CJ UVaJ j hf5 hf5 EHU jH
hf5 CJ UVaJ j hf5 hf5 EHU jH
hf5 CJ UVaJ j hf5 hf5 EHU jH
hf5 CJ UVaJ hf5 j hf5 Uj hf5 hf5 EHU &F H J R T z | ~ ` b Է ո ָ 滮槣wj j2 h/} h/} EHU jH
h/} CJ UVaJ j h/} U h/} 5 h/} 56 h/} 6h/} h( hf5 hf5 j) hf5 hf5 EHU jH
hf5 CJ UVaJ j hf5 hf5 EHU jH
hf5 CJ UVaJ hf5 j hf5 Uj hf5 hf5 EHU % " : < G H [ \ ] ^ B D t u A נsfj4 h h EHU jVH
h CJ UVaJ j h U h 6h j h/} h/} EHU jIH
h/} CJ UVaJ jp h/} h/} EHU j&H
h/} CJ UVaJ h/} 5 h/} 56 h/} j h/} Uj7 h/} h/} EHU jϊH
h/} CJ UVaJ (A B Y Z l w { | ʼ ռ " # 6 7 8 9 J j n o ֽ " $ L M b c y z þ ľ ߾ l m r ƹ h%KA 6h%KA h%KA 5j h h EHU jVH
h CJ UVaJ jo h/} h EHU jϊH
h CJ UVaJ j h U h 5 h 56 h 6h h/} h h 6 8r s տ ֿ 8 9 L M N O R Z ] ^ q wo`SoNo h~ 6j h~ h~ EHU jH
h~ CJ UVaJ j h~ U jO hf> h~ 6EHU j H
h~ CJ UVaJ hf> h~ 6j hf> h~ 6U j hQ h~ 6EHU &jPH
hQ h~ 6CJ UVaJ hQ h~ 6j hQ h~ 6U h~ 5 h~ 56 h~ h h%KA h%KA 6 q r s t { | } ~
$ & L N P R ӣӔӂӣsfӣ j. hL3b hL3b EHU jH
hL3b CJ UVaJ hL3b 6j? hL3b hL3b EHU j}H
hL3b CJ UVaJ j hL3b UjJ h~ hL3b EHU jH
hL3b CJ UVaJ j hg Uh~ hL3b hg j h~ Uj h~ hg EHU jH
hg CJ UVaJ " \ ] ^ _ ` # $ % & ' (
gd"6 $ % & ' Ȼ|m`||Q jH
hg CJ UVaJ j hg hg EHU jrH
hg CJ UVaJ j hg U j hf> hg 6EHU j H
hg CJ UVaJ hf> hg 6j hf> hg 6U hg jv hL3b hdQ EHU jUH
hdQ CJ UVaJ hL3b j hL3b Uj6 hL3b hdQ EHU j9H
hdQ CJ UVaJ
! " & ' : ; < = } ~ J N [ \ ^ a b 涩Ҝ捀yupipe h67- h67- 56 h67- 5h( hg hg jA hg hg EHU jH
hg CJ UVaJ j7 hg hg EHUj. hg hg EHU jݔH
hg CJ UVaJ j$ hg hg EHU jrH
hg CJ UVaJ hg 6hg j hg Uj hg hg EHU ' 2 3 4 5 v w 9 : M 븫뗊놁wo j hs Uhs hs 6H* hs 6hs j h67- hs EHU jH
hs CJ UVaJ h67- 6j h67- h67- EHU jH
h67- CJ UVaJ jJ h67- h67- EHU jH
h67- CJ UVaJ j h67- Uh67- h67- 5 h! 5 h67- 56'M N O P % & C D ^ _ ` # Ȼ۷|xsxkf_k hh*r 56 hh*r 5hh*r h73 5 h73 6h73 h67- h" h67- h" 56 j h67- h" EHU jH
h" CJ UVaJ j h" U h" 6h" j h" h" EHU jZH
h" CJ UVaJ hs j hs Uje hs hs EHU jH
hs CJ UVaJ ## $ ' ( 3 5 6 9 X l R T z { S T % M N Q S ټ٭xsojojo hh*r 6hh*r h8 6j h8 h8 EHU jH
h8 CJ UVaJ j h8 Uh8 j h6a h8 EHU jH
h8 CJ UVaJ j h6a U h6a 5 h6a 56 h6a 6h?P h6a hT7J 5 h73 5 h73 56 h73 hz h73 hh*r h73 )S Y Z a 7 8 D J _ a k n r { 2 4 n ' ( ; < = > V ] d ȷ j hakF ht EHU jH
ht CJ UVaJ j hakF U hakF 6htw hakF htw 6h% h% 6 h% 6 h8 h8 h% hT7J 5 h8 5 h8 56 h6a h8 h8 56 h8 6h8 4 m n & 1 2 m n { M
h^hgd%
& F
gd%
gd"6
$ / 3 8 < @ A N O Y f q u z ~
gdnh
gdakF
h^hgd% d
* + , - K L u v ²rnfn j h Uh j htw h EHU jH
h CJ UVaJ j htw htw EHU jH
htw CJ UVaJ j htw U j htw htw 56EHU j H
htw CJ UVaJ j htw 56U htw 56 htw htw htw 6hakF htw ht 6ht & / 0 C D E F Y Z
öק~sl h 56 j h 56UjN h h EHU jrH
h CJ UVaJ j h h EHU jH
h CJ UVaJ j htw h EHU jH
h CJ UVaJ h 6h j h Uj_ h h EHU j[H
h CJ UVaJ &
@ M N e f ` z q { c f źŏŏŏŏŋ{ŋtpkpkp h' 6h' hX]5 h4E! h4E! hX]5 hft h4E! 6h4E! hX]5 6 j htw hX]5 56EHU j H
hX]5 CJ UVaJ hX]5 56 j hX]5 56UhX]5 hnh htw h j h 56U j htw h 56EHU j H
h CJ UVaJ * : ;
P gd4E!
P gd`
gdakF
gdnh 9 ; < B C V W X Y e f y z { | , - . / 0 Y & ' ( ) @ C xi\xj hCi hCi EHU jZH
hCi CJ UVaJ j hCi U hCi 6hCi j hft hCi 56EHU jOH
hCi CJ UVaJ jV hft hCi 56EHU j@H
hCi CJ UVaJ j hft hCi 56EHU j,H
hCi CJ UVaJ j hft 56U hft 56 h' hft $; < 1 X Y
gd
P gd`
h i ~ H I # $ y ! ' | ਖ਼| hC`, hL 6hL h' 6h' h8Y h8Y jB hft h8Y 56EHU jOH
h8Y CJ UVaJ hft h8Y 6 j hft h8Y 56EHU j@H
h8Y CJ UVaJ h8Y 56 j h8Y 56Uh8Y hCi hCi 6 0| } 1 2 3 W X Y > ? ? @ S T U V W X Y d Y ԻԻԷ hT7J h< j h]e h< EHU j H
h< CJ UVaJ j h]e Uh9e h4 h]e hb 6hb h% h hL h 6 h]e 6h h 5 h h j h h Uh' hL h 2
# 6 W [ f g t u X Y X Y T U
gd]e Y d g 9 R S T U : > + , 4 8 O m : > O Q U 𧣟|t| h% h% 6h% hL h1y= he h9e 5 h9e 56 hT7J h9e h hL; h9e hT7J 6jb hT7J h9e EHU j2.H
h9e CJ UVaJ j hT7J hT7J EHU jw-H
hT7J CJ UVaJ j hT7J UhT7J hT7J 5 hT7J 56- ( q / a + , P Q
h^hgd%
& F
gd%
gd"6
gd]e
gd
& F
gdL; U W _ a 7 9 F
\ ] S ( ) * A C D ⸭⣜⸌} jRH
h CJ UVaJ j h Uh h H* h 6>* h 6 hw 5jH hw hw Uh h1y= 5j5 h1y= h1y= U he 6j" he h1y= 6U h1y= hw h8 h8 h8 6h8 h% h% 6 -
\ ] R S D E ~ F
h^hgd
& F
gd
gd
gd"6
h^hgd% Q R S U r s ' ( ; < = > p q վբՍվqd] h h1 jxd h h1 EHU jRH
h1 CJ UVaJ jnb h1 h1 EHUh1 h1 H*je` h1 h1 EHU jH
h1 CJ UVaJ j[^ h1 h1 EHU jYH
h1 CJ UVaJ j h1 Uh1 h h H* h 6h j h Uj\ h h EHU $ X Y l m n o } ~ - . A B C D F L M ` a b c Ÿ|t|eXt jo h h EHU jH
h CJ UVaJ j h Uh jnm h1 h EHU jH
h CJ UVaJ j*k h1 h EHU j[H
h CJ UVaJ jh h1 h1 EHU jH
h1 CJ UVaJ jf h1 h1 EHU jH
h1 CJ UVaJ j h1 Uh1 hw "c 6 Q R x y
< = U V n o z { Ÿ #h B*CJ OJ QJ ^J aJ ph h OJ QJ #h B* CJ OJ QJ ^J aJ ph"" h 6h ju hL3b h EHU jٴH
h CJ UVaJ jDr h~ h EHU j4H
h CJ UVaJ j h Uhw h 3 5 6 R y = V o { ; 7$ 8$ H$ gd
gd : ; ? C D a b ; < > ? @ J 嫠 j h Uh h 5j h h Ujy h h Uh #h B*CJ OJ QJ ^J aJ ph #h B*CJ OJ QJ ^J aJ ph #h B*CJ OJ QJ ^J aJ ph h OJ QJ #h B* CJ OJ QJ ^J aJ ph"" 0; D b 7 8 9 : ; = > @ } ~
gd
gd 7$ 8$ H$ gd |
>
{ | Ȼכ{ngcc[c j h%F Uh%F hgH hgH j hL3b hgH EHU jٴH
hgH CJ UVaJ j hgH U hgH 6hgH h$ jN h1 h EHU jH
h CJ UVaJ j
h1 h EHU j[H
h CJ UVaJ h j h Uj h1 h EHU jH
h CJ UVaJ ( * d 8 < h j / 0 1 8 = A B U V ƿ}vqi j h= U h= 5 h= 56 h= 6jP h#q h= EHU jKH
h= CJ UVaJ j h#q Uh= h#q 56 h#q 5h#q hU 6 hU 56 hU 5hgH h$ h%F hU j h%F Uj h h%F EHU jRH
h%F CJ UVaJ % f h 7 8 i j 0 1 [ > h j ` a
gd (
gd
gd V W X } ~ d e v x 3 4 5 6 X Y l m n o 涩暍ꉄ| j h& U h& 6h& j$ h= h& EHU jH
h& CJ UVaJ jӰ h= h& EHU jH
h& CJ UVaJ j h= h= EHU jdH
h= CJ UVaJ h= 6h= j h= Ujw h#q h= EHU + % & 9 : ; < A B U V W X Z [ a b g h ( * P Ȼאzzzzsnjbj j h Uh h 5 h 56 h& 6h= j h& h& EHU jTH
h& CJ UVaJ j h& h& EHU j$H
h& CJ UVaJ j h& h& EHU j:H
h& CJ UVaJ h& j h& Uj h& h& EHU jH
h& CJ UVaJ %P R T V 4 6 8 : F G L S öק~ob j h= h EHU jTH
h CJ UVaJ j h& h EHU j)H
h CJ UVaJ jh h& h EHU jH
h CJ UVaJ j h= h EHU jiH
h CJ UVaJ h 6h j h Uj h#q h EHU jKH
h CJ UVaJ $ f r t , . T V X Z < ? _ ` a h l m n }vqlq}d j h ( U h]W 5 h ( 5 h ( 56 h ( 6h ( jf h h ( EHU jH
h ( CJ UVaJ j h h EHU jXH
h CJ UVaJ h 6jd h h EHU jH
h CJ UVaJ j h U h 56 h 5 hlN 5h h& ' X> e? ? ? A RA lA
gdlN 8 8 8 8 8 8 8 9 9 9 H9 I9 \9 ]9 ^9 _9 c9 k9 p9 9 9 9 9 9 9 9 9 9 9 9 9 9 D: E: : : : : : : |m jH
h$/ CJ UVaJ j5 hp hp EHU jH
hp CJ UVaJ hp 6 hp hp hp 5 hp 56 j]3 hp hp EHU jH
hp CJ UVaJ j hp Uj0 h}! hp EHU jH
hp CJ UVaJ j h}! Uhp h}! ': : : ); *; =; >; ?; @; O; P; c; d; w; x; y; z; ; ; ; ; ; ; ; ; < < < < < < B< C< < < < ˾ںګڏsfbb]b]b hAim 6hAim jA h$/ hAim EHU jH
hAim CJ UVaJ jz? h$/ h[W EHU j H
h[W CJ UVaJ j< h$/ h$/ EHU jH
h$/ CJ UVaJ hp j: h$/ h$/ EHU jH
h$/ CJ UVaJ h$/ j h$/ Uh: j hp Uj 8 h$/ h$/ EHU #< < < < < < < < < < >= D= W= Y= = V> W> X> `> e> h> p> |> > > > > > >
? ? ? ? ? )? *? =? >? ?? @? ? ? ? ? ȷȷȷȷȷȷȷȯȠȈy jOH
h[W CJ UVaJ j h[W 56UjlH h$/ h[W EHU j H
h[W CJ UVaJ j h[W U h[W 6 h[W 5 h[W 56 h[W jTE h$/ hAim EHU jH
hAim CJ UVaJ j hAim U hAim 6hAim hAim 56+? ? ? ? ? ? @ @ @
@ @ @ A A :A ;A NA OA PA QA gA hA lA mA A A A A A A A A A A A %B Ÿܳܨܳ{wh[ܳw j R hv` hv` EHU j"H
hv` CJ UVaJ hv` hK hK jO hft hK 56EHU j!H
hK CJ UVaJ hK 56 j hK 56U hK 6jM hK hK EHU j H
hK CJ UVaJ j hK UhK h[W j h[W 56U jJ hft h[W 56EHU #lA mA A A B B B C ?D @D iD nD D D !F "F tF xF F F H H I I I I (I L
gd'
gdlN %B &B 9B :B ;B D ?D PD özvqvv h' 6h' jM[ hv` hv` EHU j#H
hv` CJ UVaJ jX hv` hv` EHU jW#H
hv` CJ UVaJ h[W jV hv` hv` EHU j"H
hv` CJ UVaJ hv` 6jT hv` hv` EHU j"H
hv` CJ UVaJ hv` j hv` U )PD QD dD eD fD gD nD oD D D D D D D D D D D D D D D D D E
E E E \E ]E yE }E E E E E E E E Ȼ~~~~~ob jF PF QF \F ]F pF qF rF sF xF yF F F F F F F F F F F F F F F F F 겥ꖉzmf h}u h}u jr h}u h}u EHU jo(H
h}u CJ UVaJ jXp h' h}u EHU jS(H
h}u CJ UVaJ jm h}u h}u EHU j'H
h}u CJ UVaJ h}u 6j`k h' h}u EHU j'H
h}u CJ UVaJ j h}u Uh}u h' 6h' j h' U (F G !G 4G 5G 6G 7G EG FG {G |G G G G G H
H hH iH |H }H ~H H H H H H H H H H H H H I ԸԜԀso` j,H
hU1 CJ UVaJ h' j| h}u h} EHU j'H
h} CJ UVaJ jy h' h} EHU j3&H
h} CJ UVaJ jw h} h} EHU j)H
h} CJ UVaJ j h} U h} 6h} j`u h}u h} EHU jO)H
h} CJ UVaJ j h}u Uh}u " I I I I I I I I I $I %I &I 'I (I 1I 5I K FL IL L L L L L L L L L L L M M M OM M M M ؽ̩ؤؠsogo j h] Uh] j6 h[w h[w EHU j0H
h[w CJ UVaJ j h[w Uhp h[w 6h[w hU1 6 h$/ hU1 j hU1 hU1 EHU j -H
hU1 CJ UVaJ j hU1 UhAim hU1 h[W h[W h} h} j h} Uj~ hU1 hU1 EHU $L L vN N N O O
O O O 9P :P S S S S ڃ ۃ
gd_
gdU
gdlN M M M M N N uN vN N N N N N N N N N N N O O
O O O O P $P %P 8P nP oP P P P P P P Q 6Q צ{nj h@ j^ hU h@ EHU jU6H
h@ CJ UVaJ j hU U hU 6>* hU h] 6 hU 6hU hp hH1 j h] h] EHU j2H
h] CJ UVaJ h[w h] 6h] j h] Ujj h] h] EHU j1H
h] CJ UVaJ &6Q 7Q JQ KQ LQ MQ R R R R R R R R R R R R .S 3S AS BS US VS WS XS S S S S S S S S S S S S S S S S öyw U h h h 6j h@ h EHU j8H
h CJ UVaJ h hU jɖ h@ h@ EHUj hU h@ EHU jU6H
h@ CJ UVaJ h@ 6j h@ h@ EHU j6H
h@ CJ UVaJ h@ j h@ U + ones and the EMBED Equation.3 zeros were distinctly different (say, different colors, or different aromas!), then there would be n! different possible ways to order them. However, all the ones look alike, and all the zeros look alike. And so there will be fewer than n! ways to order them. How many fewer? Well, how many ways can one order the k ones, assuming they are distinct? The answer is k! ways. And how many ways can one order EMBED Equation.3 zeros, assuming they are distinct? The answer is EMBED Equation.3 ways. But since the ones and zeros are not distinct, the number of ways we can position them into the n slots is:
EMBED Equation.3 (read as n choose k ) (2.10b)
Now, each way we position the k ones in the n slots has a corresponding probability, as demonstrated above. And so, there can be a lot of probabilities to compute, even for a modest value of n. For example, if n=10, then for k=5 we have to compute
EMBED Equation.3
different probabilities, and then add them up to get EMBED Equation.3 . And so, even if we were to now proceed to develop a general expression for this probability, it would be really ugly! Furthermore, given specified p-values EMBED Equation.3 corresponding to the n Ber(pj) components of X, we would still need a calculator, if not a computer, to compute this probability for the events [Y=k] for all of the n+1 possible values of k: k=0, 1, , n. For these reasons, we will focus our attention on an algorithm for computing the elemental probabilities associated first with EMBED Equation.3 . Then, having numerical values for these probabilities, we will develop an algorithm that uses them to compute the probability of each [Y=k] event. We do this in the Appendix to this chapter, since the development is mainly one of writing a program. We will use Matlab as our programming language. However, those familiar with other languages (e.g. C++) may prefer to write their own code.
Recall, that the only assumption, thus far, has been that the n components of X are independent. We will now make the further assumption that they all have one and the same p-value. In this case (2.10) becomes
EMBED Equation.3 (2.12)
However, since the p-values are identical, then so long as an element of EMBED Equation.3 has k 1s (and, consequently, n-k 0s), that element has the probability (2.12). And the number of distinct elements of this type is given by (2.11). Hence, under the assumption that the components of X are iid , we have the following probability description for their sum, Y:
EMBED Equation.3 (2.13)
The probability model (2.13) is known as the Binomial probability model. Notice that is entails two parameters: n, the number of Bernoulli random variables being added, and p, the p-value associated with each and every Bernoulli variable.
Having (2.13), it is trivial to arrive at the probability model for our p-value estimator
EMBED Equation.3 .
The sample space for EMBED Equation.3 is obtained by dividing each element in EMBED Equation.3 by n. Hence, we have the following equivalent elemental events:
EMBED Equation.3 .
It follows that the probability description for EMBED Equation.3 is
EMBED Equation.3 . (2.14)
The following is a Matlab code to compute this theoretical probability model for EMBED Equation.3
n=100;
p = 0.7;
xvec = 0;
pvec = (1-p)^n;
for k = 1 : n;
xvec = [xvec , k/n];
pval = ( factorial(n)/(factorial(k)*factorial(n-k)) ) * p^k * (1-p)^(n-k);
pvec = [pvec , pval ];
end
stem(xvec, pvec)
This code gives the following figure.
Figure 10. Graph of the Binomial distribution (2.13) for n=100 and p=0.7.
A comparison of Figure 7 and Figure 10 shows that Figure 7 has a shape reasonably similar to the theoretical model in Figure 10.
APPENDIX
To begin, we should figure out just how many elemental events are associated with EMBED Equation.3 . Well, in the first slot there can be a zero or a one. Then for each of these branches, the second slot can be a zero or a one, and so on. It turns out that there are a total of EMBED Equation.3 branches ; that is, there are EMBED Equation.3 elements in EMBED Equation.3 . For example, if n=10, then EMBED Equation.3 has EMBED Equation.3 elements in it. Now lets compute the following numbers:
EMBED Equation.3 the number of elements that contain no 1s = 1
EMBED Equation.3 the number of elements that contain one 1s = EMBED Equation.3
EMBED Equation.3 the number of elements that contain one 1s = EMBED Equation.3
In general,
EMBED Equation.3 the number of elements that contain one 1s = EMBED Equation.3 .
PAGE
PAGE 24
EMBED Equation.3
EMBED Equation.3
EMBED Equation.3
EMBED Equation.3
EMBED Equation.3
EMBED Equation.3
" # $ % _ ` ρ Ё с A D ʂ ˂ ӂ Ԃ ׂ ߂ | hH1 hvl 6j h hvl EHU j ;H
hvl CJ UVaJ hvl hlN j h h EHU j:H
h CJ UVaJ jŝ h h EHU h 6j h h EHU j9H
h CJ UVaJ h j h U 0
d g Ń ك ڃ . / B C D E ڄ ܄ ( ) ˴ˢˢ˓ˢˢˢˢˢ h 6H* j h h EHU j@H
h CJ UVaJ h 6jS h h EHU j>H
h CJ UVaJ j h Uh jפ h'
h EHU j_=H
h CJ UVaJ j h'
Uhvl h'
6h'
2 Ņ ƅ ȅ G H [ \ ] ^ e k % & 5 6 Έ ψ Ј ш # $ 7 8 9 : > ? X [ ƽƵƦƵƊ}y h jc hH1 hH1 EHU jZJH
hH1 CJ UVaJ j h] hH1 EHU jIH
hH1 CJ UVaJ j hH1 Uh'
hH1 6hH1 h{ 6jެ h{ h{ EHU jYAH
h{ CJ UVaJ j h{ Uh{ h 6h . و ڈ O P t u Ћ ы ʌ ˌ a b + : O
gd_ L M N O d e x y z { 1 2 8 : ϋ Ћ ы 3 4 G H I J r þ÷Åvi jY hg hg EHU jMH
hg CJ UVaJ j hg U ju hft hg 56EHU jHMH
hg CJ UVaJ j hg 56U hg 56 hg 6hg h 56 jh h h EHU j"KH
h CJ UVaJ j h UhH1 h h 6 )r s E F Y Z [ \ w x ö~zvqvivZ jMH
h CJ UVaJ j h U h 6h h> j hg hg EHU jNH
hg CJ UVaJ j hg hg EHU jMH
hg CJ UVaJ jd hg hg EHU j&NH
hg CJ UVaJ hg 6j] hg hg EHU jMH
hg CJ UVaJ hg j hg U " Ŏ Ǝ ǎ ) + . / 3 5 9 : ɏ ʏ ˏ 1 2 3 4 {n j h{ h{ EHU jYAH
h{ CJ UVaJ j h{ Uh{ hH1 hH1 5 h{ 5 hH1 5 h 5 hH1 hH1 h> 6 h> 56 h> 5 h> h> j h> h> UhH1 h h h> h j h Uj hg h EHU 'O Ǝ ǎ : ; ʏ ˏ , -
gd}^
gda
gd_ ! 4 5 6 7 C D W X Y Z \ l n w x ȻכrcVכ j- ha ha EHU jCH
ha CJ UVaJ j( h{ ha EHU jYAH
ha CJ UVaJ j ha U ha 6ha j# h{ h{ EHU jYAH
h{ CJ UVaJ j h{ h{ EHU jBH
h{ CJ UVaJ h{ j h{ Uj h{ h{ EHU jrBH
h{ CJ UVaJ ! & - . A B C D K p q t u Ԓ Ւ ķӯӠӯӄwӯh[ j h}^ h}^ EHU jFH
h}^ CJ UVaJ jE ha h}^ EHU j|FH
h}^ CJ UVaJ j h}^ h}^ EHU jLFH
h}^ CJ UVaJ j h}^ Uj ha h}^ EHU jqFH
h}^ CJ UVaJ h}^ jb ha h}^ EHU jFH
h}^ CJ UVaJ ha j ha U # A B U V W X ] ^ _ ` a c d f g i j l m s t u w x ~ Ÿ{s j h UhC`, 0J mH nH uh
h 0J j h 0J UhK j hK U h'
h'
h'
ha h}^ ju h}^ h}^ EHU j%GH
h}^ CJ UVaJ j0 ha h}^ EHU jGH
h}^ CJ UVaJ j h}^ Uha h}^ ) Z [ \ ] ^ _ ` b c e f h i k l u v w h]hgdl &`#$gdKZ
gd_
gd}^ ̓ ͓ Γ ϓ ѓ ғ ȻאtgX jH
h CJ UVaJ j hQ h EHU jtH
h CJ UVaJ j h( h EHU j@H
h CJ UVaJ j hEQ h EHU jH
h CJ UVaJ j h9 h EHU jH
h CJ UVaJ h j h Uj hE h EHU jH
h CJ UVaJ Г ѓ
gd_ h'
h'
hKZ h j h Ujh h(zI h EHU , 1h / =!"#$% `!( KSZ'rN R XJ xcdd``> @c112BYL%bpu @c112BYL%bpu bޱ Us i#?@Z*!5Hy F \Ps}v& ` p021)W2 ԡ -_`n LN`!- @%N Hd ` h 0 xcdd`` @c112BYL%bpu# 1dBA%7\`
; FLLJ% I s: /2 N`!- ,[4 ` ` 0 0 xcdd`` @c112BYL%bpu# 1dBA%7\`
; FLLJ% A2 u(2tA4 A!_ v120e C?N`!- l r7\ v\ ` XJ 0 xcdd`` @c112BYL%bpu# 1dBA%7\`
; FLLJ% s: /2 5L P D d
$T b
c $ A ? ? 3 " ` ? 2 >ԢNAf
ַ v D `!n >ԢNAf
ַȂ HD
XJ < xcdd``vdd``ba V d,FYzP1n: &lB@?b
ㆪaM,,He` @201d++&1X +|-KSe F\
YAZk+a|u0` WWbq
W&00s9| F]L #l*H?!2B}6!ý^ \(8Zq%F&&\U @ ]
@Y` yAE D d
x@ b
c $ A ? ? 3 " ` ? 2 yE"Ne c k `!c yE"Ne c,
1 xcdd``cd``ba V d,FYzP1n: & B@?b u 8
ㆪaM,,He` 01d++&1X +|-̛6Nh8W:vJ߂ ďD7 5 Ma`:@${@F9 c$@aS
2"n%߰ʷb`S{P``cI)$5a fE.jO D d
u 0
# A " U [;sÎEDVÊ` ї~.;$kQ2HiX ]k)CX=TN{DQl)X.,T +mtBYN5>r`No.hh!`]
D&l
@6X ,]
`c4+ )oB>䃐_<] !B9AN9ANS)CCBN9Anwfnזպh5X
R44\ {=} } ՓOO' "|ʡo =tݎ0]lN2Cq,?FrY^!\^p%v8nW]ZI+P| XMשMyƾ>&q< xyX'bQB_Ug-gΗz)^fC`S
aWyəF r@ùiU&aLY^x[0 }d1&B+3a&縨k=
t@ֲaϑ360g ڭ*X-C9:W92^VD̚+ I _sx<_Y>>F^*wrIv=sE! Z XJyQ>W) bx klL>1SCǧZ;>#|GN|Z
YċOk6[ٷDکhYi^f-:gfzĀbbXXkU֙Z
갞QX`} 뻂g`!>X-a"(׆5ik[a 8pi(ȷVo· ZM
_]6Y^97"sV[uLuLa 3::`]ju1g]ºh}_Mu &:buAaκЅu#M"'I$g=Iu|$º9ַa7-d}| d= 1zDz qr7\X_%\u&~uXku g=º ;nM
n5Yw+
Ӝg.O{dWedTX'
NjN*X7d}~ׅC~YWhPXW`]ɺBa]a.u G/ d UXG
j*u}2VVYkeE
ww wwyCIkaL+Cty=}
YX>x9vd~%D GX~3=}nŭnʶiz/\Wό5{~U̶mz? xmR81:8~9<.a 5Fo~ڂ ¬ ss_=D^b-^Te{NjZnV]wy=Z9k#-tOy
vq\Հ1\uRHsB)G p-\ec- [ R e]4A%웮ƣyt=ӬdP D d
$T b
c $ A ? ? 3 " ` ? 2 >ԢNAf
ַ v
`!n >ԢNAf
ַȂ HD
XJ < xcdd``vdd``ba V d,FYzP1n: &lB@?b
ㆪaM,,He` @201d++&1X +|-KSe F\
YAZk+a|u0` WWbq
W&00s9| F]L #l*H?!2B}6!ý^ \(8Zq%F&&\U @ ]
@Y` yA> D d
T
! " # $ % & ' ( ) * + , - . / 0 1 2 3 4 5 6 7 8 9 : ; < = > ? @ A B C D E F G H I J K L M N O P Q R S T U V W X Y Z [ \ ] ^ _ ` a b c d e f g h i j k l m n o p q r s t u v w x y z { | } ~
! " # $ % & ' ( ) * + , - . / 0 1 2 3 4 5 6 7 8 9 : ; < = > ? @ A B C D E F G H I J K L M N O P Q R S T U V W X Y Z [ \ ] _ ` a b c d e j m n o q p r t s u w v x z y { | } ~ R o o t E n t r y w F +Вl @ D a t a
^ E W o r d D o c u m e n t v c O b j e c t P o o l y 1В+В _ 1 2 2 2 6 2 9 6 2 9 F 1В1В O l e
C o m p O b j f O b j I n f o $ ) , - 0 5 8 ; @ C F I L Q T U X ] ` c f i l o r s t w z }
F Microsoft Equation 3.0 DS Equation Equation.3 9q 3 0C Z
S X
= { 0 , 1 }
F Microsoft Equation 3.0 DS EqE q u a t i o n N a t i v e O _ 1 2 2 2 6 2 9 7 8 8 F 1В1В O l e
C o m p O b j
f uation Equation.3 9q - pE [
P r [ X = 1 ] = p
F Microsoft Equation 3.0 DS Equation Equation.3 9q O b j I n f o
E q u a t i o n N a t i v e
I _ 1 2 2 2 6 8 5 9 5 1 F 1В1В O l e
C o m p O b j
f O b j I n f o E q u a t i o n N a t i v e C _ 1 2 2 2 6 8 6 1 1 8 , F 1В1В ' V [
S X
) = 1
F Microsoft Equation 3.0 DS Equation Equation.3 9q % V [
P r ( ) = 0 O l e
C o m p O b j f O b j I n f o E q u a t i o n N a t i v e A _ 1 2 2 2 6 8 6 1 9 2 F 1В1В O l e
C o m p O b j f O b j I n f o
F Microsoft Equation 3.0 DS Equation Equation.3 9q :
= { }
F Microsoft Equation 3.0 DS Equation Equation.3 9q E q u a t i o n N a t i v e 9 _ 1 2 2 2 6 8 6 2 3 1 " F 1В1В O l e
C o m p O b j f O b j I n f o ! E q u a t i o n N a t i v e ! ) _ 1 2 2 2 6 8 6 3 6 1 $ F d4Вd4В O l e
"
D
F Microsoft Equation 3.0 DS Equation Equation.3 9q V [
S X C o m p O b j # % # f O b j I n f o & % E q u a t i o n N a t i v e & 6 _ 1 2 2 2 7 0 1 9 5 3 @ ) F 1В1В O l e
' C o m p O b j ( * ( f O b j I n f o + * E q u a t i o n N a t i v e +
F Microsoft Equation 3.0 DS Equation Equation.3 9q u V [
P r ( A *"B ) = P r ( A ) + P r ( B ) "P r ( A )"B )
F Microsoft Equation 3.0 DS Eq_ 1 2 2 2 6 8 8 2 3 4 1 . F 1В1В O l e
. C o m p O b j - / / f O b j I n f o 0 1 uation Equation.3 9q V [
S X
F Microsoft Equation 3.0 DS Equation Equation.3 9q % V [
P r ( A )"B ) E q u a t i o n N a t i v e 2 6 _ 1 2 2 2 6 8 8 5 4 6 3 F 1В1В O l e
3 C o m p O b j 2 4 4 f O b j I n f o 5 6 E q u a t i o n N a t i v e 7 A _ 1 2 2 2 7 0 2 9 9 9 8 F 1В1В O l e
9
F Microsoft Equation 3.0 DS Equation Equation.3 9q V [
A )"B =
F Microsoft Equation 3.0 DS Eq g C o m p O b j 7 9 : f O b j I n f o : < E q u a t i o n N a t i v e = 9 _ 1 2 2 2 7 0 3 3 7 3 6 = F 1В1В O l e
> C o m p O b j < > ? f O b j I n f o ? A E q u a t i o n N a t i v e B q uation Equation.3 9q U V [
P r ( A *"B ) = P r ( A ) + P r ( B )
F Microsoft Equation 3.0 DS Equation Equation.3 9q _ 1 2 2 2 7 0 3 5 1 0 ; T B F d4Вd4В O l e
D C o m p O b j A C E f O b j I n f o D G E q u a t i o n N a t i v e H O _ 1 2 2 2 7 0 3 7 1 4 J G F d4Вd4В O l e
J C o m p O b j F H K f 3 V [
S X
= { 0 , 1 }
F Microsoft Equation 3.0 DS Equation Equation.3 9q
V [
O b j I n f o I M E q u a t i o n N a t i v e N ) _ 1 2 2 2 7 0 4 1 4 0 O L F d4Вd4В O l e
O C o m p O b j K M P f O b j I n f o N R E q u a t i o n N a t i v e S _ 1 2 2 2 7 0 4 3 2 4 Q F d4Вd4В
F Microsoft Equation 3.0 DS Equation Equation.3 9q u 02 [
! X
= { { 0 } , { 1 } , S X
, }
F Microsoft Equation 3.0 DS Eq O l e
V C o m p O b j P R W f O b j I n f o S Y E q u a t i o n N a t i v e Z 6 uation Equation.3 9q V [
! X
F Microsoft Equation 3.0 DS Equation Equation.3 9q 5 V [
P r ( { 0 } ) _ 1 2 2 2 7 0 4 6 5 3 E Y V F d4Вd4В O l e
[ C o m p O b j U W \ f O b j I n f o X ^ E q u a t i o n N a t i v e _ Q _ 1 2 2 2 7 0 4 6 9 2 ^ [ F d4Вd4В O l e
a C o m p O b j Z \ b f = 0 . 3
F Microsoft Equation 3.0 DS Equation Equation.3 9q 5 lF
P r ( { 1 } ) = 0 . 7 O b j I n f o ] d E q u a t i o n N a t i v e e Q _ 1 2 2 2 7 0 4 7 5 8 ` F d4Вd4В O l e
g C o m p O b j _ a h f O b j I n f o b j E q u a t i o n N a t i v e k I _ 1 2 2 2 7 0 4 8 1 8 ' e F d4Вd4В
F Microsoft Equation 3.0 DS Equation Equation.3 9q - M N
{ 0 } )"{ 1 } =
F Microsoft Equation 3.0 DS Equation Equation.3 9q O l e
m C o m p O b j d f n f O b j I n f o g p E q u a t i o n N a t i v e q M
P r ( { 0 } *"{ 1 } ) = P r ( { 0 } ) + P r ( { 1 } ) = 0 . 3 + 0 . 7 = 1 . 0
F Microsoft Equation 3.0 DS Equation Equation.3 9q _ 1 2 2 2 7 0 4 9 1 8 j F d4Вd4В O l e
u C o m p O b j i k v f O b j I n f o l x R M D1
{ 0 } *"{ 1 } = { 0 , 1 } = S X
F Microsoft Equation 3.0 DS Equation Equation.3 9q c V [
P r ( { 0 , 1 E q u a t i o n N a t i v e y n _ 1 2 2 2 7 0 5 0 4 9 h r o F d4Вd4В O l e
{ C o m p O b j n p | f O b j I n f o q ~ E q u a t i o n N a t i v e _ 1 2 2 2 7 0 5 5 0 2 t F d4Вd4В O l e
} ) = P r ( S X
) = 1 . 0
F Microsoft Equation 3.0 DS Equation Equation.3 9q V [
S X
F Microsoft Equation 3.0 DS Eq b
c $ A ? ? 3 " ` ? 2 Լi! d `!\ Լi!R `\ XJ * xcdd``dd``ba V d,FYzP1n: &,B@?b X
ㆪaM,,He` @201d++&1X +|-Kʠ T3V_b>
W&00dq.(|FɞL #l*H!2B}(6!ý^ \8a/)#RpeqIj.,\E.B ~` uB D d
@ b
c $ A ? ? 3 " ` ? 2 r2v 7Ä; h @ `!` r2v 7Ä; . xcdd``fcd``ba V d,FYzP1n: &B@?b u
UXRY 7S?&,e`abM-VK-WMcرsA V0ZɗT T߰TϘ@|JN(ߊ
(Hb# I9 x;|a##d!fE悂B>e78Ҋ[p ACM
F&&\$ @ ]`
" Xd3 D d
H@ b
c $ A ? ? 3 " ` ? 2 } ֖FR"! Y `!Q ֖FR"! @ " xcdd`` @c112BYL%bpuurEid3 O.Ad-Fmo6 ~+`c f8-oҟm9 @c112BYL%bpu @c112BYL%bpu @c112BYL%bpu ~?Yjz3)S ;! n ~?Yjz3)S PNG
IHDR Fr sRGB pHYs + IDATx^V8μ;3 vYKw >?_ @;w: @ > @ ݔ @@ @@@ `{ tS @ l @ \nJ P MI =@
p) @ @@ .@7% ($@ @ ݔ @@ @@@ `{ tS @ l @ \nJ P MI =@
p) @ @@ .@7% ($@ @ ݔ @@ @@@ `{ tS @ l @ \nJ P MI =@
p) @ @@ .@7% ($@ @ ݔ @@ @@@ `{ tS @ l @ \nJ P MI =@
p) @ @@ .@7% ($@ 7
4 & wk =$ " : DpDM
H
$r 55 @ ) @@D@ G @ N*'@ QSC 8 DpDM
H
$r 55 @ ) @@D@ G @ N*'@ QSC 8 DpDM
H
$r 55 @ ) @@D@ G @ N*'@ QSC 8 DpDM
H
$r 55 @ ) @@D@ G @ N*'@ QSC 8 DpDM
H
$r 55 @ ) @@D@ G @ N*'@ QSC 8 DpDM
H
$r 55 @ ) @@D@ G @ N*'@ QSC 8 DpDM
H
$r 55 @ ) @@D@ G @ N*'@ QSC 8 DpDM
H
$r 55 @ ) @@D@ G @ N*'@ QSC 8 DpDM
H
$r 55 @ ) @@D@ G @ N*'@ QSC 8 DpDM
H
$r 55 @ ) @@D@ G @ N*'@ QSC 8 DpDM
H
$r 55 @ ) @@D@ G @ -9aX. oV+nN=(> P '/"c1 2i -c' =8vs Y9' =8R#?Pzxe ]!w:鶮O;¹~MwA
p"Na,2S yw]s&~6-5_[B .z5 tq`~ؗ7]xJvyF,SDuÏ{ܯo!Gme>}!p~} a+{\'a,mD ]o8 s{~ی9 M .366` ]$~O@8nVt:
q Ý$mpX v{鸿~"qjb;i >O'ٲ/V(! ^çY;Уgp8'cxZ B zVb(<Y,|tg8rH 2hV;@wK@ %ieMepYǸ=o?ǦﱣjCa HH\3)VZ% 4h
X\kyGfʉTU;WɛwG# @VUj sMR7;a[=ޒ ^ *4k 6
8nr6 ML^`.xnO'lꌷ$g{"@ p-^8 v; l[\iw"mξ|ʚ'@]@ 4 ?4xλoxNrQ'6`Mԇ[psݭ# uNo/"vk\k;vvQ1}v / 7 rA;d, @*XWܦ$@x9U@ :[SYIb u !{j8 _|^Ql~yYz:K7]b-ෳg
sO3$<źbN?Ϋ R@ ٶ^>~}\ݳ\nVX\@ />w @x
@F@ ?\ π+ /|T@ w"0:(! -rer; 2zRC)ꦎG֏ð{p
:g|cۨ&6- @@ Y!L( 'l["@ p=
A:yc!K#0 [J iZFNy^F1Xt:~3Ogf{!p($@ x=)tc;}qO Q(# 3zj @@P@ @ Q)tc{݂AOMa: @ . v* @@X@ F*[ =h)%{\nҘ "@ )!xfݗ;bFe.p}@xfe"X_AWV . [ Ww"$ [4u̚\\/ {`
kgrq a pnyMյ_jlp4U2~Jߞcmv;bjJu[ nŽn>/"}\ܛ #ӡX)D`x<| 7 ;P7:6qCaS[ l}x8 >[ Bj#uZ
`{ [' W
w|{]HmY'lC\??$SH+O@w,Hoc#nqyYlKo4S֗ wٿ@=% >OJ7rV*z`ⓀC
lUNvd
u; * x+lʫm]m"mqc uje7cn:7ry2x~C6\a.u|+t9~2xݦ`tцī 6}[4:~3Sī
?.LWx`Gߐ{9?>[2.;z.3Hu&'6^>B _։aѻ{d+pSN8ZG/ݥbX~kr wJ .J?}x2^8Y
b+;I ! F8%;
݉}W: wF .:da}W2 @0,z~oP ]T0x?bxopzңů
p~@ΓX (p<ԡ}zW@ [w6@I,wSX#h/ RI7ܬ
=T .2B7n}w\ ηE .2Bqa/. v+ "# ηܛF @ L ༡ @@ n&S@ 8oh , @ 4 f2 @ / F @ L ༡ @@ n&S@ 8oh , @ 4 f2 @ / F @ L ༡ @@ n&S@ 8oh , @ 4 f2 @ / F @ L ༡ @@ n&S@ 8oh , @ 4 f2 @ / F @ L ༡ @@ n&S@ 8oh , @ 4 f2 @ / F @ L ༡ @@ n&S@ 8oh , @ 4 f2 @ /py{{ˏ2e[~, >8 D @c112BYL%bpu 1, ЄJH D d
TT b
c $ A ? ? 3 " ` ? 2 O h FX @c112BYL%bpu 1, ЄJH; D d
@ b
c $ A ? ? 3 " ` ? 2 uĐ Ql ? a !A `!Y uĐ Ql ?
' xcdd``fcd``ba V d,FYzP1n: &6! KA?H1 :l ǀqC0&dT20 KXB2sSRsv,\~ HT TϱjVrBV @V@F*
$ ? tdSo;a 0l+ssZ|=p{AF /₆8l
5v0o86+KRse Pdk {> 1 l0fW D d
TT b
c $ A ? ? 3 " ` ? 2 O Z2