ࡱ> mE@ 6bjbj "u,m/  888889DU`;rz????HHH_______$AbRd_2JH@H2J2J_?? `QQQ2J??_Q2J_QQR6XZJY?: @5v8LZYZ%`0U`Y@KevPBKe8YKeYH0I"Q5IQIHHH__38Q"8 Information as a Measure Several books were written in the late 50s to formally define the concept of information as a measure of surprise (or lack of) or as the uncertainty of outcomes. These books were inspired by earlier work by Shannon and Wiener, who independently arrived to the same expression for average information. Let X be a random variable associated to a space ( having n mutually exclusive events, such that |E| = [e1, e2, e3en] so  EMBED Equation.3  |P| = [p1, p2, p3, pn] so  EMBED Equation.3  Let E(X) be some function such that, if experiments are conducted many times, the averages of X will approach E(X). Shannon and Wiener suggested the expression below to quantify the average uncertainty (or chaos, or disorder, or entropy) associated to a complete sample space (:  EMBED Equation.3  For each event ek there is a value, or quantity, xk, such that  EMBED Equation.3  The term log (pk) is called the amount of self-information associated to the event ek. The unit of information, called a bit (binary unit) is equivalent to the amount of information associated with selecting one event from the set. The average amount of information, called Entropy, is defined for a sample space ( of equally probable events Ek as:  EMBED Equation.3  If we have a fair coin, such that p(H) = p(T) = , then  EMBED Equation.3  bit Note that I(E1) = I(E2) = -log(1/2) = 1 bit. Extending this example, if we have a sample ( with 2N equally probable events, Ek (k=1,2,,2N), then  EMBED Equation.3  bits Example: Ea=[A1, A2] P=[1/256, 255/256] => EMBED Equation.3  = 0.0369 bit Eb=[B1, B2] P=[, ] => EMBED Equation.3 = 1 bit Suggesting that it is easier to guess the value of Ak than to guess the value of Bk. The measure H complies with the following axioms: Continuity: If the probabilities of events change, the entropy associated to the system changes accordingly. Symmetry: H is invariant to the order of events, i.e., H(p1,p2,pn) = H(p2,p1,pn). Extremal Value: The value of H is largest when all events are equally likely, because it is most uncertain which event could occur. Additivity: H2 = H1 + pmHm when the mth event is a composition of other events. The Shannon-Wiener formulation for Entropy gained popularity due to its simplicity and its axiomatic properties. To illustrate, consider an earlier definition of information, due to R. A. Fisher, who essentially defined it as an average second moment in a sample distribution with density f(x) and mean m:  EMBED Equation.3  Thus, for example, expressing the normal distribution  EMBED Equation.3  in logarithmic form, deriving with respect to the mean, and taking the integral:  EMBED Equation.3   EMBED Equation.3   EMBED Equation.3  The Shannon-Wiener expression for information can be generalized for 2-dimensional probability schemes, and by induction, to any n-dimensional probability schemes. Let (1 and (2 be two discrete sample spaces with sets {E} =[E1,E2,,En] and {F} =[F1,F2,,Fm]. We can have three complete sets of probability schemes: P{E} = [P{Ek}] P{F} = [P{Fj}] P{EF}= [P{EkFj}] The joint probability matrix is given by:  EMBED Equation.3  We can obtain the marginal probabilities for each variable as in:  EMBED Equation.3   EMBED Equation.3 =  EMBED Equation.3   EMBED Equation.3   EMBED Equation.3  and  EMBED Equation.3  From the matrix we can also compute the total and marginal entropies, H(X), H(Y), H(X,Y)  EMBED Equation.3   EMBED Equation.3   EMBED Equation.3  Note that to obtain H we must find the corresponding p(xk) and p(yj) first. To better understand the calculations involved in H(X,Y) versus H(X), and H(Y), let m=n=3. Then  EMBED Equation.3   EMBED Equation.3   EMBED Equation.3   EMBED Equation.3  while  EMBED Equation.3   EMBED Equation.3 - [p(1,1)+p(1,2)+p(1,3)][log(p(1,1)+p(1,2)+p(1,3)] [p(2,1)+p(2,2)+p(2,3)][log(p(2,1)+p(2,2)+p(2,3)] [p(3,1)+p(3,2)+p(3,3)][log(p(3,1)+p(3,2)+p(3,3)] We can also compute the conditional entropies. Due to the addition theorem of probability, the union of Ek is  EMBED Equation.3  Therefore, marginalizing Ek  EMBED Equation.3  from Bayes theorem:  EMBED Equation.3 , therefore:  EMBED Equation.3   EMBED Equation.3  where p{yj} is the jth marginal; and so  EMBED Equation.3   EMBED Equation.3  Note again that the two equations imply that you have to compute the marginals first. From Bayes theorem EMBED Equation.3  we can write: H(X,Y) = H(X|Y) + H(Y) = H(Y|X) + H(X) Example: Two honest dice, X and Y, are thrown. Compute H(X,Y), H(X), H(Y), H(X|Y) and H(Y|X). The joint probability table is: Y\X123456e(x)11/361/361/361/361/361/361/621/361/361/361/361/361/361/631/361/361/361/361/361/361/641/361/361/361/361/361/361/651/361/361/361/361/361/361/661/361/361/361/361/361/361/6f(y)1/61/61/61/61/61/61/1 The entropies can be calculated from the table:  EMBED Equation.3  bits  EMBED Equation.3  bits  EMBED Equation.3  bits A Measure of Mutual Information We would like to formulate a measure for the mutual information between two symbols, (xi,yj). Solomon Kullback wrote in 1958 a book on the study of logarithmic measures of information and their application to the testing of statistical hypotheses such as determining if two independent, random samples, were drawn from the same population, or if the samples are conditionally independent, etc. Let Hi (i=1,2) be the hypothesis that X is from a population with a probability measure  EMBED Equation.3 . Applying Bayes theorem:  EMBED Equation.3  for i=1,2 Expanding P(Hi|x) for i=1,2 solving for f1 and f2 and simplifying  EMBED Equation.3  taking the log we obtain  EMBED Equation.3  The right side of the equation is a measure of the difference between the odds in favor of H1 after the observation X=x and before the observation. Kullback defined this expression as the information in X=x for discriminating in favor of H1 against H2. The mean information is the integral of the expression, which is written as  EMBED Equation.3  for  EMBED Equation.3  Generalizing for k-dimensional Euclidean spaces of two dimensions with elements {X,Y}, the mutual information between {X,Y} is given by  EMBED Equation.3  We can think of the pair {X,Y} as the signals that a transmitter X sends to a receiver Y. At the transmitter, p(xi) conveys the priors for each signal being sent, while at the receiver, p(xi|yj) is the probability that xi was sent given that yj was received. Therefore the gain in information has to involve the ratio of the final and initial ignorance, or p(xi|yj) / p(xi). Let {X} =[x1,x2,,xn]  EMBED Equation.3  and {Y} =[y1,y2,,ym]  EMBED Equation.3  We can re-write the mutual information I(X:Y) for the discrete case as:  EMBED Equation.3  Using  EMBED Equation.3 , we can also write I(X:Y) as  EMBED Equation.3  We can also write I(X:Y) as expressions involving entropy: I(X:Y) = H(X) + H(Y) H(X,Y) I(X:Y) = H(X) H(X|Y) I(Y:X) = H(Y) H(Y|X) Example: Compute I(X:Y) for a transmitter with an alphabet of 5 signals, [x1, x2, x3, x4, x5] and a receiver with 4 signals [y1, y2, y3, y4]. The Joint Probability Table (JPT) and a system graph are:  y1y2y3y4x10.25000x20.100.3000x300.050.100x4000.050.10x5000.050 f(x1) = 0.25 g(y1) = 0.25 + 0.10 = 0.35 f(x2) = 0.10 + 0.30 = 0.40 g(y2) = 0.30 + 0.05 = 0.35 f(x3) = 0.05 + 0.10 = 0.15 g(y3) = 0.10 + 0.05 + 0.05 = 0.20 f(x4) = 0.05 + 0.10 = 0.15 g(y4) = 0.10 f(x5) = 0.05 p(x1|y1) = p(x1,y1)/g(y1) = .25/.35 = 5/7 p(y1|x1) = p(x1,y1)/f(x1)=.25/.25=1.0 p(x2|y2) = .3/.35=6/7 p(y2|x2)=p(x2,y2)/f(x2)=.3/.4 = .75 p(x3|y3) = 0.5 p(y3|x3) = 2/3 p(x4|y4) = 1.0 p(y4|x4) = 2/3 p(x2|y1) = 2/7 p(y1|x2) = 1/4 p(x3|y2) = 1/7 p(y2|x3) = 1/3 p(x4|y3) = p(y3|x4) = 1/3 p(x5|y2) = 0.05/0.20 = p(y3|x5) = 0.05/0.05 = 1.0  EMBED Equation.3 H(X,Y) = 2.665 etc Note:  EMBED Equation.3  Likewise, the calculations for H(X), H(Y), H(X|Y) and H(Y|X) can be performed. Given these, we can assess whether X and Y are independent variables. Another interesting question is where do probabilities come from and how can we use them to create a Bayesian network? To answer these questions, let's consider the two sets below, S, and C, which were sampled from a database related to the famous "Chest Clinic" example. The variables S and C represent instantiations of Smoking (Y/N) and Cancer (Y/N). s={1110110100110101100000101011010010111010000111011000000111011000000111000000100101111000101010001100} c={0000000100000000000000001000000000000000000000000000000001001000000000000000000000000000000000000000} The joint probability table for the sample is approximately: C0C1S00.550.0S10.410.04 P(S,C) = 0.55 + 0.41 + 0.0 + 0.04 = 1.0 H(S,C) = -0.41log(0.41) - 0.04log(0.04) - 055log(0.55) = 1.1834  EMBED Equation.3  = -0.55log(0.55)-0.41log(0.45)-0.04log(0.45) = 0.99277  EMBED Equation.3  = -0.55log(0.96) - 0.41log(0.96) - 0.0 - 0.04log(0.04) = 0.243244  EMBED Equation.3  H(S|C) = -0.55log(0.55/0.96) - 0.41log(0.41/0.96) - 0.0 - 0.04log(0.04/0.04) = 0.9453  EMBED Equation.3  H(C|S) = -0.55log(0.55/0.55) - 0.41log(0.41/0.45) - 0.04log(0.04/0.45) = 0.19467 H(S,C) = H(S) + H(C|S) = H(C) + H(S|C) 0.99277 + 0.19467 = 0.24324 + 0.9453 1.18744 ( 1.188 I(S:C) = H(S) + H(C) - H(S,C) = 0.99277 + 0.2432 - 1.188 ( 0.048 I(S:C) = H(S) - H(S|C) = 0.99277 - 0.9453 ( 0.048 I(C:S) = H(S) + (HC) - (H(S,C) = 0.99277 + 0.2432 - 1.188 ( 0.048 I(C:S) = H(C) - H(C|S) = 0.24324 - 0.19467 ( 0.048 Note that we could have also calculated I(C:S) by inverting the order of i and j in the summations. All we want is to assess conditional dependency. The fact that H(S|C) = 0.9453 >> H(C|S)= 0.19467 indicates that there is less uncertainty (surprise) regarding C when S is known, and therefore a Bayesian network involving the two variables carries more information when this relationship is represented as  Therefore the conditional probability table associated to the edge should be: C0C1C0C1S055/550/55OrS01.00.0S141/4504/45S10.9110.089 And the next question is: If we have a data base with N variables, should we compute the mutual information for each of the ((N-1) pairs? For N=5 variables, we only need 4+3+2+1 = 10 calculations of mutual information. However when the number of variables is much larger, as for example, N=103, we would need to find ways to reduce the number of computations. Kullback also defined the divergence, J(1,2), as the mean observation from  EMBED Equation.3  for discriminating in favor of H2 against H1, as  EMBED Equation.3  and  EMBED Equation.3   EMBED Equation.3  J(1,2) is a measure of the divergence between H1 and H2, i.e., a measure of how difficult it is to discriminate between them. Kullback studied properties about these measures (additivity, convexity, invariance, sufficiency, minimum discrimination information and others) and made.  A measure is a rather precise definition (involving such things as (-algebras) which makes it difficult to understand for non-mathematicians such as myself. All we need to know here is that this and other definitions form the basis for much of what is known as Mathematical Analysis. For example, every definition of an integral is based on a particular measure. The study of measures and their application to integration is known as Measure Theory. CSCE 822 Data Mining & Knowledge Discovery University of South Carolina  PAGE 1 Juan E. Vargas- CSCE X1 X2 X3 X4 X5 Y1 Y2 Y3 Y4 C S j pbpO$jUv9 h87CJUVmHnHujh87CJOJQJU hlh87CJOJQJmHsHhlh876CJH*mHsHhlh876CJEHmHsH"hlh876CJEHH*mHsHhlh876CJmHsH jWh87 h876]h87CJOJQJ(jh870JB*CJ(OJQJUphh87B*CJ(OJQJphh87M N   / G H   Q n (pdh$dha$dh$dha$]u4=666        u v , - / 0 °°°°Ѝ}shs]shsPs jWh87CJOJQJh876CJOJQJh876CJOJQJh87CJOJQJj;h87CJEHOJQJU$jw9 h87CJUVmHnHuhlh876CJEHmHsH"hlh876CJEHH*mHsHhlh876CJmHsH hlh87CJOJQJmHsHjh87CJOJQJUjh87CJEHOJQJU0 C D E F W X Y y z |  {k^^Th%OCJOJQJh8756CJOJQJh8756CJOJQJ\]h876CJEHH*OJQJjh87CJEHOJQJUjcH? h87CJUVaJh876CJEHOJQJh876CJOJQJjh87CJOJQJUj`h87CJEHOJQJU,jFx9 h87CJOJQJUVmHnHuh87CJOJQJ       Q R e f g h { | ߽ߘ߅uߘfVKKh87CJH*OJQJj h87CJEHOJQJUj6< h87CJUVaJj h87CJEHOJQJU$jÏ9 h87CJUVmHnHujh87CJOJQJUhHCJOJQJh876CJEHOJQJh876CJOJQJ jWh87CJOJQJh%OCJOJQJh87CJOJQJh8756CJOJQJhlCJOJQJ ()*-.12KL_`abqruvyzfXzj1h876CJEHU'j9 h876CJUVmHnHujh876CJUh876CJEHh> h> 6CJH*aJ h876CJjh87CJEHOJQJUjK? h87CJUVaJjh87CJOJQJUh876CJEHOJQJh876CJEHOJQJh876CJOJQJh87CJOJQJ"yz3={﷬﷒}h87CJEHOJQJh876CJH*h875CJOJQJ\h876CJEHOJQJh876CJOJQJh87CJOJQJj<h876CJEHU'j99 h876CJUVmHnHujh876CJU h876CJh876CJEH1p3{Rjk F$ & Fdh^`a$$dha$dhRSfghiʺtaQjh87CJEHOJQJU$j9 h87CJUVmHnHujh87CJEHOJQJU$j[9 h87CJUVmHnHujh87CJEHOJQJU$j9 h87CJUVmHnHujEh87CJEHOJQJU$jҌ9 h87CJUVmHnHujh87CJOJQJUh876CJOJQJh87CJOJQJ FSTcdrstuw~ŻŮŮśśśłsj C hh5CJUVaJh8756CJOJQJ\]h876CJEH h876CJh87CJEHOJQJ jWh87CJOJQJh-ACJOJQJh87CJOJQJjh87CJOJQJUjh87CJEHOJQJU$j,9 h87CJUVmHnHu+FGWgxyIay} !":;dhdhgd`x$dha$+,-.12EFԶ{qbO{q{q@j C hdKCJUVaJ%j\)hh5huCJEHOJQJUj+ C huCJUVaJhdKCJOJQJjhdKCJOJQJUj&hdlCJEHOJQJU$j9 hdlCJUVmHnHuhdlCJOJQJjhdlCJOJQJUh8756CJOJQJ\]h87CJOJQJjh87CJOJQJU%jJ#hh5hh5CJEHOJQJUFGHIJ]^_`abuvwx|}~ƼƼo_UFj C hh5CJUVaJhdlCJOJQJj`2h87CJEHOJQJU$j 9 h87CJUVmHnHuh87CJOJQJjh87CJOJQJU%jg/h/?hh/?hCJEHOJQJUj C h/?hCJUVaJh/?hCJOJQJjh/?hCJOJQJUhdKCJOJQJjhdKCJOJQJU%j -hh5hdKCJEHOJQJU   !"#67ԵԗxhU$j9 h87CJUVmHnHuj:h87CJEHOJQJUj76< h87CJUVaJjQ8h87CJEHOJQJU$j`9 h87CJUVmHnHuhI6CJOJQJh876CJOJQJhd!CJOJQJhICJOJQJh87CJOJQJjh87CJOJQJU%jV5h/?hhh5CJEHOJQJU789;pstuz}~./0ͿͿך{k\LjxGh87CJEHOJQJUj97< h87CJUVaJjgDh87CJEHOJQJUj.7< h87CJUVaJjAh87CJEHOJQJU$j`9 h87CJUVmHnHu"h\h\6CJH*OJQJaJh\h\6CJOJQJh\CJOJQJh87CJOJQJjh87CJOJQJUjj>h87CJEHOJQJU2JQij&>g%&MN & F dh8dh^8dh$dha$0123FGHIQRefghjk~繩眓ueUUJh87CJH*OJQJhnh876B*CJ]phhHLh876B*CJ]phjPh876CJEHU]j@7< h87CJUVaJh876CJ]jh876CJU]j2Mh87CJEHOJQJUj76< h87CJUVaJjRJh87CJEHOJQJUjO7< h87CJUVaJh87CJOJQJjh87CJOJQJU"#scP$j19 h87CJUVmHnHuj#Wh87CJEHOJQJUj:8< h87CJUVaJh.CJOJQJjTh87CJEHOJQJU$j9 h87CJUVmHnHuhUhUCJH*OJQJaJhUCJOJQJjRh87CJEHOJQJUj-98< h87CJUVaJjh87CJOJQJUh87CJOJQJ#$%&':;<=GHRTgh{|}~Ĵשמ׋{hXNh3HCJOJQJjlah87CJEHOJQJU$jF9 h87CJUVmHnHuj^h87CJEHOJQJU$j9 h87CJUVmHnHuh87CJH*OJQJh87CJH*OJQJj9\h87CJEHOJQJU$j9 h87CJUVmHnHuh87CJOJQJjh87CJOJQJUjYh87CJEHOJQJU%&MNklqr35XZ}˻޴ގގގގynfnfnfnfnfnfnfyh87OJQJh875CJOJQJh87CJOJQJh876CJOJQJ h876CJ h"?h87CJOJQJmH sH h"?h876CJmH sH h"?6CJj2dh87CJEHOJQJU$j9 h87CJUVmHnHuh87CJOJQJjh87CJOJQJUh_CJOJQJ& dh$Ifdh   dh$Ifkdf$$Iflִ \ dl<*E0    4 la   dh$If   dh$Ifkd*h$$Iflִ \ dl<*E0    4 la$).2 dh$If235  dh$IfkdSi$$Iflִ \ dl<*E0    4 la5:?DINSW dh$IfWXZ  dh$Ifkd|j$$Iflִ \ dl<*E0    4 laZ_dinsx| dh$If|}  dh$Ifkdk$$Iflִ \ dl<*E0    4 la dh$If  dh$Ifkdl$$Iflִ \ dl<*E0    4 la dh$If  dh$Ifkdm$$Iflִ \ dl<*E0    4 la dh$If dhkd.o$$Iflִ \ dl<*E0    4 la8Urs!"12KLde t!$dha$$]a$$dha$dh/01289LMNOUVijkls ˻ިޅukee]e]eSh87CJOJQJh87CJEH h87CJh^hh87CJ\jKvh87CJEHOJQJU$j+9 h87CJUVmHnHuj}sh87CJEHOJQJU$j=9 h87CJUVmHnHujph87CJEHOJQJU$j9 h87CJUVmHnHuh87CJOJQJjh87CJOJQJUhMCJOJQJ "&'(0z{-.컫예}ssdjyC h8`gCJUVaJhu CJOJQJh876CJOJQJj{h87CJEHOJQJU$jշ9 h87CJUVmHnHuj'yh87CJEHOJQJU$j9 h87CJUVmHnHujh87CJOJQJUh876CJEH h876CJh87CJOJQJh^hCJOJQJ./0LM`abc# c j z :!?!D!ԠԕԂr_Oԕjh87CJEHOJQJU$j9 h87CJUVmHnHujh87CJEHOJQJU$j9 h87CJUVmHnHuh876CJOJQJhtFh876CJOJQJ%jʀh5h5CJEHOJQJU$jع9 h87CJUVmHnHuh87CJOJQJjh87CJOJQJU%j}hu h8`gCJEHOJQJUD!V!_!d!k!s!u!v!!!!!!!!!!!!!!"""I"L"M"O"P"Q"j"k"l"v"""""""鳣陎~~~qqeh87CJEHOJQJh876CJEHOJQJh876CJEH h876CJh876CJOJQJh87CJOJQJjh87CJEHOJQJU$jB9 h87CJUVmHnHujh87CJOJQJUh8`gCJOJQJh876CJOJQJh87CJOJQJh8756CJOJQJ't!u!!!##<#g#####$$$$Y$Z$x$$$$8%r%s%u%v%$dha$dh$dha$""""""### ######### #3#4#5#6#<#C#D#F#G#K#L#M#O#P#c#d#e#f#######q$j9 h87CJUVmHnHujvh87CJEHOJQJU$jl9 h87CJUVmHnHujGh87CJEHOJQJU$j 9 h87CJUVmHnHujh87CJOJQJUh87CJEH h87CJh876CJEH h876CJh87CJOJQJ*#########$$$$$$0$6$Y$Z$x$$$$$$$$$$$$$%%%%$%&%'%Ĵס׆qkkckckckckckkch87CJEH h87CJhlh876CJmH sH h876CJh876CJOJQJjh87CJEHOJQJU$j9 h87CJUVmHnHujh87CJEHOJQJU$j9 h87CJUVmHnHuh87CJOJQJjh87CJOJQJUjh87CJEHOJQJU&'%*%+%.%/%2%3%4%s%t%u%v%w%x%y%z%{%|%}%~%%%%%%%%%%%%%%%%%%%%%%%%%%%(((((((ýýýýýñññññʪjoh87CJEHUje<8< h87CJUVaJjh87CJU h876CJh87h87CJEH h87CJ h875CJh87CJOJQJ&jh87CJOJQJUmHnHuh87CJOJQJh87CJEH h87CJ4v%w%z%}%%%%%%%Pkd͗$$Iflr \ ,*04 la. dh$If %%%%%%%%%Pkd$$Iflr \ ,*04 la. dh$If%%%%%%%YPPPPP dh$Ifkdә$$Iflr \ ,*04 la.%%%%%%%YPPPPP dh$Ifkd$$Iflr \ ,*04 la.%%%%%%%YPPPPP dh$Ifkd$$Iflr \ ,*04 la.%%%&<&z&&&&'YQQQQQQQQ$dha$kd$$Iflr \ ,*04 la. '':'I'X'g'v''''''''((*(+(1(O(P(((I*J***$a$`dh^`$dha$(*(7(8(K(L(M(N(()I*M**+]+^+_+`+a+b+c+e+f+g+h+s+t+u+++++++,,Ͽ𩛩}jZjh87CJEHOJQJU$j9 h87CJUVmHnHu h876CJh87CJEHOJQJh875CJOJQJh87CJOJQJ^JaJh87CJOJQJ^Jh%"CJOJQJj֠h87CJEHOJQJU$j 9 h87CJUVmHnHujh87CJOJQJUh87CJOJQJ h87CJ#*++\+]+^+a+d+e+h+m+q+ikd4$$IflFs Cm*{0    4 la dh$If$dha$$a$ q+r+u+z++vvv dh$IfkdC$$IflFs Cm*{0    4 la +++++9,,,--k-l---wwwwwwwwwwww$dha$kd$$IflFs Cm*{0    4 la ,9,:,M,N,O,P,S,,,,,,,--------. .>.?........ ////Z/ͽચwmmmmmbbh876CJOJQJ jh876CJjh87CJEHOJQJU$j9 h87CJUVmHnHuj˪h87CJEHOJQJU$j69 h87CJUVmHnHuj]h87CJEHOJQJU$j9 h87CJUVmHnHuh87CJOJQJjh87CJOJQJU h876CJ%---.F.G....j/k///f0g0i0000000000 dh$If$dha$Z/[/i/k/z/}//////g0h0000000000000000000000000000011-21222222쳿쳿쳿쳿쳿쳿쳿쳿jh87CJOJQJUh8756CJOJQJh87CJH*OJQJ jSh87CJOJQJh87CJEHOJQJh875CJOJQJ&jh87CJOJQJUmHnHu hD^6CJ h876CJh87CJOJQJh1'CJOJQJ1000003*** dh$Ifkd<$$Ifl֞d4* \ 4l2d804 la00000 dh$If $dh$Ifa$000003*** dh$Ifkd$$Ifl֞d4* \ 4l2d804 la00001 dh$If111s2t23+++$dha$kd$$Ifl֞d4* \ 4l2d804 la2222 3 3333 3&3'3:3;3<3=3?3@3S3T3U3V3X3shQAhjkh87CJEHOJQJU,j~9 h87CJOJQJUVmHnHuh876CJOJQJjh87CJEHOJQJU,j9 h87CJOJQJUVmHnHuj h87CJEHOJQJU,j9 h87CJOJQJUVmHnHuh87CJOJQJjh87CJOJQJUj9h87CJEHOJQJU,j9 h87CJOJQJUVmHnHut23 3%3>3?3W3X3s4t4u4<6=6i6j6666666666666$a$dh$a$$a$X3^3s4t4u4v444j6666666666øhz0JmHnHujh870JU h870J jsh87h87jh870JUh! h!CJ h87CJ h876CJ66666666666666666666dh 1h/ =!"#$%;Dd B  S A? 2lNĩO:6rD`!ylNĩO:6r  dGxRJP=Tm,ABuر"6PlZ .N*88\ nn^(&\={=7y e@) 8d2,ˤVR%eUwF%6f̐1~cm9F;> acD 5~qmlr}mwH;0M}L~V0M0(ᅿߋ]o_JsS]|b#RI_>ORVE'\{Ѩ!T/>Dz$:uIvY`;z~twu"Vìg3h0BwړO-b3%Dd [B  S A? 2SgzaF07H-TB3B'0ׅ?0CEsHh^Hfnj_jBP~nb e,RF0?tዢ͢8$6_̎{{`D?a!pNsCN~:ܕфK!M\ =Ĥ\Y\ː~f:suYDd  hP  S A? "2~ʙ@ceYGH`!~ʙ@ceYGH @p|Wxcdd``fd``baV d,FYzP1n:&&! KA?H1Z ㆪaM,,He`HI? 01d++&1-bF: pi# J‡< @* *a ~*>`wT`DCE<@VN~z*&qN||C,naBܤrl.IbĕBn0o k3dsAS8AlMNwLLJ% H  'f86wDd < B  S A? 2t|fS `!t|f`Hb@2xK@߽IKJAtPNRPAM[hA9(289㽻 -QL}.{@f E:oȘaYa9SDd'qK!5kGD`q~Q7C_ '[Av^ |TЕ,094إN-s/۪ܽV-wxfPHCPzO1)ocԂoȧ?CαrMp:N(G<τy HQN#k~Mx-qYN8-vٮ2sLbdHCE!Vۈ6mXE Be3fC!Z n;?AlmVtX3s|1 O 2GO$3]8IqSQEژũ,L;:e(pdOٵBgjX/E;*ʬck%܊%RRr)ݤnl卩 {$j>sSismlDY&.z8om' uߊ{5&ݝu0/:3_nnV{PuI'3<)\ H92B,p)u?_E?߻균ϰ)z{1!|Ѭ8Ogp3Q*Dd P  S A? "2K&Gb`!K&Gb+ hxcdd``$d@9`,&FF(`T逈 A?dMznĒʂT/&  !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`acdefghilopqsrtuvxwyz{|}~Root Entry1 FJvnzData b?WordDocument0"ObjectPool3vJv_972125781FvvOle CompObjfObjInfo  #&'(),/0369<?BCFIJMPQTWZ]^_`cfghijklorsvyz{|}~ FMicrosoft Equation 3.0 DS Equation Equation.39qDIwI #E k =U k=1u *" FMicrosoft Equation 3.0 DS EqEquation Native `_972125959 FvvOle CompObj fuation Equation.39qDIwI p k =1 k=1n " FMicrosoft Equation 3.0 DS Equation Equation.39qObjInfo Equation Native  `_972126278FvvOle  CompObj fObjInfoEquation Native _1061708731|JFvvpI8I H(X)="p i logp ii=1n " FMicrosoft Equation 3.0 DS Equation Equation.39qrp x k ="lOle CompObjfObjInfoEquation Native ogP{e k }="logp k FMicrosoft Equation 3.0 DS Equation Equation.39qפIwI H(X)=I(E k )="p k_972132291,FvvOle CompObjfObjInfoEquation Native _1010217354FvvOle !CompObj "f logp kk=1n " FMicrosoft Equation 3.0 DS Equation Equation.39qII H(E k )=H(E 1 +E 2 )="12log(12)"ObjInfo!$Equation Native %$_1061881836$FvvOle *12log(12)="log)12=1 FMicrosoft Equation 3.0 DS Equation Equation.39qx> I(E k )="logp k ="loCompObj#%+fObjInfo&-Equation Native ._972132859)Fvvg(2 "N )=N FMicrosoft Equation 3.0 DS Equation Equation.39q0IwI I(A k )Ole 1CompObj(*2fObjInfo+4Equation Native 5L_972132921'1.FvvOle 7CompObj-/8fObjInfo0: FMicrosoft Equation 3.0 DS Equation Equation.39q0II I(B k )I FMicrosoft Equation 3.0 DS Equation Equation.39qEquation Native ;L_9721970743FvvOle =CompObj24>fObjInfo5@Equation Native A_972197309O8FvvOle DJٌIwI I=["lnf(x)"m """ +" ] 2 f(x)dx FMicrosoft Equation 3.0 DS Equation Equation.39qCompObj79EfObjInfo:GEquation Native H_972197467=Fv`vJ٘pII f(x)=1 2  exp["(x"m 2  ) 2 ] FMicrosoft Equation 3.0 DS Equation Equation.39qJ٠IlI logf(x)Ole KCompObj<>LfObjInfo?NEquation Native O="12ln2"ln"(x"m 2  ) 2 FMicrosoft Equation 3.0 DS Equation Equation.39qJTII "lnf"m=x"m 2_972197623;EBF`v`vOle RCompObjACSfObjInfoDUEquation Native Vp_972198188cGF`v`vOle XCompObjFHYf FMicrosoft Equation 3.0 DS Equation Equation.39qJII I=x"m 2 () 2""" +" 1 2  exp["x"m 2  () ObjInfoI[Equation Native \8_1124600727^TLF`v`vOle a2 ]dx=1 2 FMicrosoft Equation 3.0 DS Equation Equation.39qy©b, [P{X,Y}]=p{1,1}p{1,2}....p{1,m}p{CompObjKMbfObjInfoNdEquation Native e_972217770@wQF`v`v2,1}p{2,2}...p{2.m}............p{n,1}p{n,2}...p{n,m}[] FMicrosoft Equation 3.0 DS Equation Equation.39qOle mCompObjPRnfObjInfoSpEquation Native qJـmII P{x k }=p{x k ,y j } j=1m " FMicrosoft Equation 3.0 DS Equation Equation.39q_1124600875YVF`v`vOle tCompObjUWufObjInfoXwEquation Native x_1124600855[F`v`vOle CompObjZ\fy—o  [P{X,Y}]=p{1,1}p{1,2}....p{1,m}p{2,1}p{2,2}...p{2.m}............p{n,1}p{n,2}...p{n,m}[] FMicrosoft Equation 3.0 DS Equation Equation.39qykt_ p(x1)p(x2)p(x3)p(x4) FMicrosoft Equation 3.0 DS EqObjInfo]Equation Native _1124600493"h`F`v`vOle CompObj_afObjInfobEquation Native _972217610eF`v`vuation Equation.39qyX, P{x k }=p(x,y) {x,y}\y " =p{x k ,y j } j=1m " FMicrosoft Equation 3.0 DS EqOle CompObjdffObjInfogEquation Native luation Equation.39qJPIwI P{x 1 }=P{E 1 }=P{E 1 F 1 *"E 1 F 2 *"...*"E 1 F m }=p{1,1}+p{1,2}+...+p{1,m}_1124600587jF`v`vOle CompObjikfObjInfol FMicrosoft Equation 3.0 DS Equation Equation.39qy|a P{y j }=p(x,y)= {x,y}\x " p{x k ,y j } k=1n "Equation Native _972218464oF`v`vOle CompObjnpf FMicrosoft Equation 3.0 DS Equation Equation.39qJ٨IwI H(X,Y)="p{k,j}logp{k,j} j=1m " k=1n "ObjInfoqEquation Native _1010221623tFvvOle  FMicrosoft Equation 3.0 DS Equation Equation.39qTII H(X)="p{k,j} j=1m " ()logp{k,j} j=1m " [] k=1n " =CompObjsufObjInfovEquation Native p_972219115myF`v`v"p{x k }logp{x k } k=1n " FMicrosoft Equation 3.0 DS Equation Equation.39qJT\II H(Y)="p{k,j} k=1n Ole CompObjxzfObjInfo{Equation Native p" ()logp{k,j} k=1n " [] j=1m " ="p{y j }logp{y j } j=1m " FMicrosoft Equation 3.0 DS Equation Equation.39q_1010241582~F`v`vOle CompObj}fObjInfoEquation Native P_1010241593FvvOle CompObjf4II H(X,Y)="p(1,1)log(p(1,1)))(+p(1,2)log(p(1,2))()+p(1,3)log(p(1,3))() FMicrosoft Equation 3.0 DS Equation Equation.39q8II "p(2,1)log(p(2,1)))(+p(2,2)log(p(2,2)))(+p(2,3)log(p(2,3))()ObjInfoEquation Native 4_1010241615FvvOle CompObjfObjInfoEquation Native 4_1010238784Fvv FMicrosoft Equation 3.0 DS Equation Equation.39q$I@I "p(3,1)log(p(3,1)))(+p(3,2)log(p(3,2)))(+p(3,3)log(p(3,3))()Ole CompObjfObjInfoEquation Native < FMicrosoft Equation 3.0 DS Equation Equation.39q II H(X)= FMicrosoft Equation 3.0 DS Equation Equation.39q_1010317613FvvOle CompObjfObjInfo&HtnIpzI * k E k=1m " =1 FMicrosoft Equation 3.0 DS Equation Equation.39qJdIwI F j =#Equation Native d_972219604FvvOle CompObjfObjInfoEquation Native _1010317824FvvOle E k F jk=1n *" FMicrosoft Equation 3.0 DS Equation Equation.39q&|tnIpzI p(x,y)=p(x|y)p(y)=p(y|x)p(x)CompObjfObjInfoEquation Native _9722196976rFvvOle CompObjfObjInfoEquation Native  FMicrosoft Equation 3.0 DS Equation Equation.39qJ4ILI P{X=x k |Y=y j }=P{X=x k )"Y=y j }P{Y=y j }_972219848FvvOle CompObjfObjInfo FMicrosoft Equation 3.0 DS Equation Equation.39qJـ~II p{x k |y j }=p{k,j}p{y j }Equation Native _972220133FvvOle CompObjf    #$%(+,-0589:;>ABCFIJKLORSTUX[^abcfiloruvwx{~ FMicrosoft Equation 3.0 DS Equation Equation.39qJII H(X|Y)="p{x k ,y j }logp{x k |y j ) k=1n " j=1m "ObjInfoEquation Native _972310143FvvOle CompObjfObjInfo Equation Native  _972296934F@v@v FMicrosoft Equation 3.0 DS Equation Equation.39qzI?@ABCDEFGHIJKLMNOPQVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~`b]F"L `05L *'] ZZ< f22:# 9@8 l+{VBWOL`"P` Z&ߠBܤrع7^&%@*lku.TyVTyPvĆq~=XKR]k0@LT3P?]&g3 IALbwѓ r\{^=`dbR ,.IeCw-3XI Dd DhB   S A ? 2sy,sO`!Gy,s@|xuQJA<6c@=-9 $><Â" HOO<3kb cOd@z骭%<І;RuvhCCl j)L7xkr -蠲}X'B,KЇԃf"l&.5n\W7w{t]wϾffă,%W[r sT &YTT;&vHk#=+XͶ?w2S5HsOܹi|5ѹ?zIwDd $ DB   S A ?  2Kzl`!Kzlx?KAgfQ@E€"hEȝ(x1IGXXXX~`%;s&v!P-A\Et#DA}2O :.,q4vݜM 85ƾxo},^NB}Bf0ɡ@t,Ћ ke;)OW i,}R23㝐ݽLZ;?f>|gQ'FPFS07WPQ^'Yur27ے/iLA rRu\(uI {_Dd lB   S A ?  2Cg'`!g4(xS1KQw{{$ a !E6IN,[Ճ;Z*"iR[B@jo}gac73! 11Sk8"Da,fG%*yU?rcqDс^{|Q:jxm&/pI& 8#T ]/Wj1:z݋ypyyp.bq"0\EŞfގ=waZD=T󞵐O('C-I+~l ћ? ?xK*)9bi.oK3x,xUX樎gyXJ}<Mh`%F tu|זE'M66Z:g2Վ7,x3'g tE<8_nAEЉc$XI nDDd lB   S A ?  21$MQ9wOp% `!$MQ9wOp%p xS;KA&9O@D%ZbaЀ/ FTVVDl؉ XZޒSDXnoy-( )OXQVH,d< =#%Ofwg3T\X#= Gরb5g\h ;+j %kwTL'MVD|!VGOx!T?zoc[wp* 9L{*!8+n^(*Ft%YW(ϿQSBe4筘RmGQOmyd{e7AZ}lp y*_TIRM'ޫx庻Y3#!7Rf2u%q,/eEvC\hvG&-*݃:+k:q LS`eѭH@#S8\\('}IDd <XB   S A ?  2uQ0`B_`!uQ0`B_` UxRJQ=375|4K Z$7/,-mF,,~׹7 FΙs!!aveeFL85G9`rK2A SW(~pXRl9%b8::H-sM' xyˉRW|/m~~y֟ O'F?#YgˍpFϟh7ˡd-~Dd B  S A?  2pI+<ꮘ `!pI+<ꮘ<F)(+xkAȏ,t6s[A<ZSQ`@+&؃A$`obG/^="[$Bęy3E c x*W\e!t0()$%::EMG[ψa ⽞"ʤtsb^ؕ> 8cB\U-71wLB^vS'\寴E:h3+iH.6;FC6k>tX3_MnhvChnFsorv!/w[Ja|pV--CCOBR|O3l ܃ͱ gt23̋#b?Lk"SʻVy8JyRs.e%_ *9ɓʍn-rŊ~7a/47j;b:+v&Ra~6ՇG'ҡ#23ڋ?[nQ:c:΄ff#}T"Pu2UW2@mڗRBT)JI|gx]r!+Y+yCV쳚M+[#v|9a|&!+iM\jʗQ+.SoQ(>_:_]Ԯ;.WS3o]PiQEwMjb;zL>xx1[_ӞK)~,qN>'`!! 3 >qNx @XhnxJ@g'WR0S*zxT+GB w#*}@N|xw8_e5a̅D'&'kƞs 5gҙAVoqe❀Iu'z2=iӜK_cInB "wԩuduv<0 _ հh&8sH}Dd @ h > s *A;? ?3"`?2#$i&3(R)*)`!#$i&3(R)*@8 xڕMhA߼YI CZ!]joU""TYT0mtƫ(,ZobIHכ.JݷM 8ߓv׸;au="|1YaR3U(4N}S"sM; >\٫`44Ks^L*:ƏH .+5$x%b~hAge?ᗶI-e7,Q ?& /Q|ԑܤdo'+=ue w 2Ae ,jIuNQ0Ou5y=Y޹CV챖G+V4cc5VزhGF2ۤLj?JV\*8Մ/W\ފ1|uvĻ]wB*_&He_xǢa vp4>%#{0D7$=:SX*xVmG9Hڐ7CKA] #pK:{ f3}i2%\Dd b < c $A9? ?3"`?250 ՔF(O-`!z50 ՔF(xtL HxڕJ@;܁U"BPP M#Xxp-SYYXDPN| K R!n6I°g. Ԏ `,B&82mмֲ\xVsX@`tNND[icyNIfdQM!<(榯Ѓܺgw.Ե/VCU)B%>J}Օ+ק'oZcIÞpe>}Fv.bX'\J> cn/@_#:Ab^>ǡ(` 9,WiTm>;k]Dd h 9 s *A6? ?3"`?2=7сZBN /`!7сZBN `@G xڥK@]&QCuupEXWh! 8MAGQ' ƻL-{%(@`P nD q"B-¨:si4QSbqf<:#^Ѡ?I נw޾a %L)ǒ>FOtk?)8;5: ۸:] IF{vS~SZK'x>y>=N^~[n;/q22@> Iꙕ@ƈ,E[אu(@*| `SnaU?گcHJ2Vg%Uٰ_xy:%ȭ# xXV* j^_F o仰_AnK/Χ8' n9#bPtg1i2 `\GsDd @B  S A? 2`U-2܄{{ <2`!4U-2܄{{ L(]? xK[Ag6W&%j* Mn=؛޴^U@Oͱ =BP[=ֻg=SA& Ώ"__af]rDZA>dqVrg gr Fa| 'rv Gm@n4ml_y\))kQG@) [.v[z>CQp8gÔ}=q1Pee{*^u ?>ŗ Q?C_+@{߱#tsPpF뮸}]c\׹]֣\?9W#/)y7]T\2msue:gSL\2x𫇊O !S@%赢Ɂ#zPRe{A& Dd h : s *A7? ?3"`?2?ɭOx'95`!ɭOx'9@@ xڥK@߻_IZDTPQEP:*XZi!F`'Gn.RHEqqAx&9Z0!.O罧| @gW:#J="4Mf{NM16 4@ \ QqxazD`l>0gUAWz-67N hI6O`Wܸ@b=LbtwV]EVk'x>eAa.?}UV]b5ZȪ3u|g%3VsqUO3`z6$xOq?/ӒY$Lzymڍ} Pܾ9g^^GDGV0~_  g1n>Λ[s#5HI||F  V=%dIp 1=)=WӜdh"S O*Zu>|9Dd B  S A? 2\~nTbܴ!Z8`!\~nTbܴ!Z: @2xJ@LKFckABB,bmV(^ڴ \ ]ntm̙t4*& oΜ8~(lPBpPqpeJW0]TMa&6g% WBc?׬:RxDo@mfn,2J {wpI[uP}>;үܠ߂&X#n|.4-y'b}UbZ/YUP3}3̴"3M {3V$v~:S;"z\sΙk ë y;Wpޖ<ٖ $#B_A΃kA'XG1=k (Wdx6^.WP9'%O(^}n؛x46!n&&zhC~ >u#mDd L P  S A? "2ڪr¸tӼNɟOfѯn? ?}[OϘ{~ʾ5zغ>^K-=1[,Tҹ…RqjwlHuz;YW;lšCjg1J {-;i練{"edƗvT6ye ~e ՗cT"JTIQ|ɞȓo&)fdͻ2ҴP=$^ű~q\4*.X׶Jއh|5a-{o79]uګ.jePitSdQDd B  S A? 2k"6/>`!k"6/l"P6]xMhSAg/_ZM1b"l%*œЋ=@"lI JPPo/b-AɛAP4j;2OggggQ !Q#*UC +B6M\ªr/!rjc#WHr6qId6SXF劄xHGkʴ9QZ@<ի%o"zMlq~)ު 7"xMyxr~_i\Hiiykx0SHz-r$5|Srv] o["ήk~K.1BӫT8)^ȞpM%cd*u\y$ʃiߍp{_BsdqgKiYߍvoEVԈ]5X!Ž :'SV70!A$Kq\G||/q10:/3:lXrB㸬Dd B  S A? 2\~nTbܴ!ZA`!\~nTbܴ!Z: @2xJ@LKFckABB,bmV(^ڴ \ ]ntm̙t4*& oΜ8~(lPBpPqpeJW0]TMa&6g% WBc?׬:RxDo@mfn,2J {wpI[uP}>;үܠ߂&X#n|.4-y'b}UbZ/YUP3}3̴"3M {3V$v~:S;"z\sΙk ë y;Wpޖ<ٖ $#B_A΃kA'XG1=k (Wdx6^.WP9'%O(^}n؛x46!n&&zhC~ >u#Dd TP  S A? "2mL8{ @b1##X`=F !#T57L! KA?HZ : ǀjx|K2B* R*3{ vL@(\_(OhH?e ꀔjC7S?~br<.i` E&?Od1\ׁWhp'P] 90*ʨ6*ШjyGdQ}" 7ϓA5IEҨ>0@A8GPmC( ɢ62]01KUQQM<΍fQnT7s`΅jlNTykNPȂj^,3~FzH EZu8?L] ?C·/@<(wc=y%o'ʃjP@|]8?+o-ٸ8+C LLJ%Ðf#Dd TP  S A? "26V@3]k;G`! V@3]k;' =XJxڥTKAf\]E@,L@C݅.@*@<cu֩cP:t/:t :'̮~cP.Λy{ogg :/7k*=5J!0 12IzHMFKF;]vxdBJe#_), f")7sji (O.Lk$K8e:"5_`&Ω uZkB9X؍fKŕmD?"S6qt|Oa>dͬMj;, `~!A<qų9[7uЩ}?vș5H95)*eV ΜQH֙"?)yw3lk4%鉉Ff8u^$h٢GL݈P:+պ.СDd TP  S A? "2< ֋ya\*wZ pńnzF/hYHr (%= 6die,NRGDOK:?ŗ; 3}f#im'ϭl\Dja& "3NrxFC6k2NFM4+ c׆C_[}1kkc g~Pb/vHyCMg(1ea+npg'gq愌v%5k쫓9`AwbtbF ~]m%rϩ>`g"LO*PZҋZ $ mDd L P  S A? "2ڪr¸tӼNɟOfѯn? ?}[OϘ{~ʾ5zغ>^K-=1[,Tҹ…RqjwlHuz;YW;lšCjg1J {-;i練{"edƗvT6ye ~e ՗cT"JTIQ|ɞȓo&)fdͻ2ҴP=$^ű~q\4*.X׶Jއh|5a-{o79]uګ.jePitSdDd [@P  S A? "2^]Afw7:P`!2]Afw7`S xcdd``~ @bD"L1JE `xX,56~) M @ +j䆪aM,,He` `g11W&00ri#F[1d++&10t@L`II Mp~?#a2#&3M2S{0pAC J`'C/+#RpeqIj.ld3@7Dd J  C A? "2Tl ma T0>5uR`!mTl ma T0>5@@8 ;xڕR1KP^m,4$tqpT Tll͵E\3tq-*5ZwPTF )Sfky0D٣DL.{ 7i-h Bho`\N'QbY|S`B/x7Y?\Vɮn%p ^לi)c ja49 r%]Ϧ:t;X`S_~I?'{aJr _UB-ks]vۊ@6G.!ĚxkļJ8ɸWNzlKDd B  S A? 2ĒU`!Ē  dWxR1KP޽` c`+8&BZ NஃC꤈߻ w ЀJʪ=K AIpdS,sITSƫcM*K[//`&VAy (^w|XT)aBp `N":plj7*X$cfv#gx!Vv|>ЗKɸo=5>a{~[TdYR%tr;xdiq i839s,_x&B'R)~PKd;K"6xcxF= Z댇 ,j\DOgmgDd $@J  C A? " 2 &ѱʐ<7\gW`! &ѱʐ<7\4# kxcdd`` @b1##X`=F !#T57L1! KA?H: b ǀjx|K2B* R. `W01d++&10\ F\ wEA5^_F_&QAU͍*ƅfyP`FU_=8ߔ= #8N> &8$*C`!B@J8߈U%.Ty_NT7gFUU}DfZ.ܤ&X /H-炦8.p'- `p:fdbR ,.Iͅ`uf1Dd B  S A? !2IhHaY`!IhHa@@h $xuK/Q9V_iUCj#$%%$ ]D &Kk ;@I|k¸^?;!!@wD7!$vʕABQ.N@#ͣPErэ/hMgJz]xR؏{2& ^څO:V# r(o&k*t0E7ÌfI8nx"vUP8*rüvHn;s|6+xU-FT﬚9rBfF ʙ>I.%0m w Vӆcv?Faݏh~@s5WAp)!62Y2r¶ь{r 滔Nv] B׊{(vVL|Q~%YfO"+|+[ۛ^~ѕxȞƃט*et^]z\aMoc 3.=̃uM! |@0*j Dd B  S A? #2(KPFڋE4˒^`!KPFڋE4˒z@@h $x/QmKЪVũD  "Ưn[i$E =Г+8im'6}7ߙ7;Sn-8A^Nq\9 a֍;j/oc2~zN|/Vג"o15>M7|%}5Qnxy.|?*Vqq>k=a8[wg^*3WKJc{N/ܔuxM=]Υw 9=Lp ٗ1,߱2&뻾:vԧJek[)=jI> ^ Jޗ|#:)ؿCWEhzbRXNǜ[69hA/Q*:d맡 !Dd B  S A? $20K@ێ$ a`!K@ێ$ z@$xcdd``eb``beV dX,XĐ Ɂ倸 A?dqw@=P5< %! *31;a& MρH(0]n!k~n*n8N*xY뫢KZ/Lp?rod~Ukފs"#N)_ILSgZ;Z_xw(P1.mboLοy}yo/z0kQןN&Nqu%VrYRCcbv{~tqGAb6$$If!vh55*5E55555#v#v*#vE#v#v:V l055*5E55/  / / / /  / / /  / / /  / 4a'$$If!vh55*5E55555#v#v*#vE#v#v:V l055*5E55/ / / /  4a'$$If!vh55*5E55555#v#v*#vE#v#v:V l055*5E55/ / / /  4a'$$If!vh55*5E55555#v#v*#vE#v#v:V l055*5E55/ / / /  4a'$$If!vh55*5E55555#v#v*#vE#v#v:V l055*5E55/ / / /  4a'$$If!vh55*5E55555#v#v*#vE#v#v:V l055*5E55/ / / /  4a5$$If!vh55*5E55555#v#v*#vE#v#v:V l055*5E55/ / / /  / 4a_$$If!vh55*5E55555#v#v*#vE#v#v:V l055*5E55/  / / / / / / /  4aDd DB ! S A? &2XH"Έ z}x<4p`!,H"Έ z}x<@ *xKA߼lըRQAP<(R㡞PSHЛo^[7"gμ TZCjEԍBp@ug\sOݎ. .nM$[j;7АJfmg5pG? x1G{nSNw߷$xǂ} r^KUʛ{ʫY@T=[9lFfMO?I9KӚx=Pn J=Bt9F(c[IiQxX@-LN_q+~&)@#qn.z]ptP>L7>ak-(~vntU&6 2|Lf;ꜷ`&*3iAW<i _ӳf3D2x ݸ !y/ݿ^B{tTUqVۂ|˚UĮ@38']-%VBB R!HS 7e8!|a$7a'p~(%ip5py| ~U:~!Tِ1GXJi0!8c\Y;0m`DVQGB WmcL(&_30@penR~ΐ wi+oͳ\| NLP;=I)$5!`uZDd |h ? s *A<? ?3"`?+24}Od~`!}Odr` xڝS1KQsVC-D*#(*"XZQD4B$' v!ba-ZYV!`q)M4@N{G,]fۙ( >~jd)10dc_5g;- j2)D +~ciɅ`p}h~RjaF[gUYGVkw h>AX*ͬ矺_Së]qCNNz { e|AMRO.J=파-CompObjEfObjInfoGEquation Native H0_971692202FvvzI@I logf 1 (x)f 2 (x)=logP(H 1 |x)P(H 2 |x)"logP(H 1 )P(H 2 ) FMicrosoft Equation 3.0 DS Equation Equation.39qOle MCompObjNfObjInfoPEquation Native Q IwI I(1:2)=logP(H 1 |x)P(H 2 |x)  +" d 1 "logP(H 1 )P(H 2 ) FMicrosoft Equation 3.0 DS Eq_971692441FvvOle VCompObjWfObjInfoYuation Equation.39q0I I  1 = 2 FMicrosoft Equation 3.0 DS Equation Equation.39qEquation Native ZL_972297794FvvOle \CompObj]fObjInfo_Equation Native `_972295946FvvOle dJٰI@I I(X:Y)=f(x,y)logf(x,y)g(x)h(y)  +"+" dxdy FMicrosoft Equation 3.0 DS Equation Equation.39qCompObjefObjInfogEquation Native hl_972296300FvvJPIwI x ii=1N " =N 1 FMicrosoft Equation 3.0 DS Equation Equation.39qJPDII y jj=1M " =N 2Ole jCompObjkfObjInfomEquation Native nl_972296913Fv@vOle pCompObjqfObjInfos FMicrosoft Equation 3.0 DS Equation Equation.39qJIwև{gtI_AxG%d >L?ޟMWKI_)Bů=RB(r~aἓzsAymv_ v:|@+6 H5׋% tp~=xۯDd t!B & S A$? ,2FK-BȌYT"`!K-BȌYTz!@2xT=K$AUgvcA d1  349p.\ \bpE01{zkwիzozz\1JҠ.#!R:҇m:Ohץtu4B;|/X ir,dIly%p^(PWvH1Kj"gDcC qlպ^-Ҭ^?T̯f&2ӥbnB|xSU%X׬pc|'w5J\*J VO=W\H);;siojx(_; 8bL]>5`ho$緒O ׊av7#ەU3ni“+Yx)}Zq=rSc~os~}m~ IL8W*|ScݠR{-PxtQ$]TG{wl ЛP\pzWA#{7`Lpc~|f#zZ?7Dd lB ) S A'? /2&69ཀྵyψ`!69ཀྵy!xuS+DQ>VB(?Y?̨-ĖFaccc,eIy9fMsν1 Z1(!J} `d 5;Ae]^떴jR>VZEjE-ќ$L-,>)P)+cP𴞣&=y_tp!sbtvc[ðR(c3If:ideWGM`"i*ڔJvWe8nMg{KkFjH5lcfJslO_8x #!q;(p[d_=k'j2)N?TkI\OiI4y-S)Uz-+''i w0cѿߎʴY>ySGvgg{ZM˂yAļӎG0 II'Ra1Dd B + S A)? 12i3 t"_d<w`!oi3 t"_d<@ =xRJ@}3^<0X ;\e#^*N6~X[FV?4N"F& ;ov۷' 5t4d& qR٦5L{wKwEg2֑|uvJslO (Jb`*fb$.~t{d0D$N^?7LsLdpho?TZEuZ*c8D#NkLVW0nBIyW._~NQF];?oc*8?S/(=y2TcAP&/7x'V=)˧dDd B , S A*? 22X|/"puҵe4`!,|/"puҵen@P%xAKA߼iP j+*X D[ƃBk!hI~B C!{;Ra}6I9 @WIYL WA+SbWQhu7\]j6!)yTqr}pM/ t'E}f5;Γpj?S9>b + Ϝ/ ɥxaA_|>uO1_.WJ;7WUaTfB%&AOyr1C\WY;r7MsxKU?`[%i v'[ao?>~>tUIjb/;ܖ<=1|vrvUsOcuv\;7DcQ}OK>gn>3)x8R*P{ y_Dd ,B - S A? 32/9fk"3ْ`!/9fk"34(kxuK@߻4iTkEq BJĭB_i -ր888;?pp\K<|wwlLzL9lf9"VpV&+~/KE9[ 0 *Og9Q)]n!k~n*n8N*xY뫢KZ/Lp?rod~Ukފs"#N)_ILSgZ;Z_xw(P1.mboLοy}yo/z0kQןN&Nqu%VrYRCcbv{~tqGAb6Dd 8B . S A+? 42Ca1|xz8`!a1|xz@ D#xK[AggD ib-cP-DJ!FQ ؀= x`/^CO ugvǾ7μ}+  jĐ( gU75=b&Z}YZ G1d(k7K_B堺SKrkLzz_%ɥوf<+'a|Џِ4b}'_n5k8W3ԕc1Y5|.F^҈??͙[>Ti^e?MpYqiW_aM ]y料*Y=T-/;ɖɝLY'm'3,ٿkT=Vl >M[}|/zϜ-n 0u Rg]on$$If.!vh55*555#v#v*#v:V l055*5/  / / / /  / / 4a.$$If.!vh55*555#v#v*#v:V l055*5/ / / 4a.$$If.!vh55*555#v#v*#v:V l055*5/ / / 4a.$$If.!vh55*555#v#v*#v:V l055*5/ / / 4a.$$If.!vh55*555#v#v*#v:V l055*5/ / / 4a.$$If.!vh55*555#v#v*#v:V l055*5/ / / 4a.gDd -0P / S A,? "52nmX]Cѧ);`!nmX]Cѧ);I+qkexڕOA̶jAME( [hU@A8M(mRn1 z?=Q\:ovw-67&3 ZQ~ r3&Go6r$r̉s7/5(tB &!6( vb"0rwު{2o`.9tV }g U\ l{d?{Wg_'0tXq1#|1G [~.K6p'W2kZz Θ)sLQms+-ge>8m˹#+ٻ}m4~+B0Ґo*uxC t9GuvvȘ˟P#ea2.LҌϗ r}_\/Qby^vfTstV<3J׸j ݠ;ZQP#iF`O+~OOz4KR)jЌj\;F+A3:8#(&w֐gq䴪Pz+QÃC`Dl|4F! !4_n5 A@[`YwCqX^Dd DB 0 S A-? 62o=Lx&/`!o=Lx&/ 8>jxR=OA.~$bu0ZHCcbr 5 &%@_X+* ,,v֚x1eM潛7+v8JMAH2%ܒLTg)hjy5 UYEj;'☚Ի"q/E(GAh]EM]{3kW^svӴ.m+l ƽHʏ~[ vsCyW'+'iaNc;{Rۚ>өO38#bh` z[Dd4B,hI/^ $$If !vh55*5{#v#v*#v{:V l055*5{/  / / / /  / / 4a $$If !vh55*5{#v#v*#v{:V l055*5{/ / / 4a $$If !vh55*5{#v#v*#v{:V l055*5{/ / / 4a lDd B 1 S A.? 72.XuRO|?5`!.XuRO|?0  xxJ@g6imjSAOi$ Ћھ@BZ(G  UA^ $YRĄef ,])9 A+]are&0V["ٗj%%iؒZ pS'iEqY# 'LKƉv}|نT0rIrҺ7N;%\qGI| *e&8 _Ig`զF)~E@Gv)JʡRV\?[c';ionDd @ B 2 S A/? 82v-Eɻhv~О`!v-Eɻhv~О0 zx=KP9WbcA -XA; lGA?hB ҭ?E_p讃Pprp`$ZEĄ˽Ϲ}rLc3OR xPTpV*˄:K9 j|lCRc8 pHFkwK8zZafԼbj%OGC?|PWK˚sEnfLNq5Gq{40N:c@gtMf9&oy)7y_|?:0oH >(R̻(\P,+JYԼc1Фu%)ѣ!-t-=D^z~߆>Jr16Dd B 3 S A0? 92goLjh|`!tgoLjh̐%@\9BxkA߼ݤ$ŴilL-nTB{"ELnZ VHB%a/z 7AgzY<'(Λ "Eܰ|&#p^,4rBa"fQuy)\tavLNǏABJm9z*"GM -W DAWA9 8OŊ[kvP~ҕNcY+`X>op'NOÜ|#Y<7|7~(4l>]߷?{xq3q?pu<~4zT7kP֕Yiǩ QN{K&xENdz>=9S ow*~iL`dTdT`9_Ǹ(/Q7\5='Z_"/: {N~ʖNlΞh٫)zTtYdt?棎>AS.yEk(wqc8Oc,9rpǶ,NcDS:tr2JBR( 898tvq.⠃l:E]}Q= tQ-TC!0 q#'QXu {yi<kDT9V|~ɠ'}xƧްy%RD5Twnh΂WS7!Fώ✟|-o<WDۖsw,?\ Ϲk_\u~O8~7B<8-sz~/zjw_m6jsP 飜 JWyUo7\<]:uO`9ᕼ s .}`!`9ᕼ s .Z x=OK1n jp z/TP0*( ޅ ^x/cIwRM wJR5v]ѡD[kRGg95|tr=X'B(SLj%L9 Lחm` Y _0ޙ͒k8]M#U#ͷ܁bL渇בޯ~E8As]x-1,>2kDd P B 6 S A3? <2uFH[H|2Q`!uFH[H|2@2xS;KA;>@-4-"R0bK4?R2 6 9w6]";no-BBqPWP()+ !mۼ%8u3sDZ$L^Z--ChN6 qYD+3d7\+kTZal{3}d\$U UxE' BDcT7T‹oq 9ةxTqRMp+oOC*hW\iN"ElkTj!:T {3 5jcz z'٣]|ګU3v?ófV줲aSrgYWxߝݪ_k_P' +|]+W6傊7f 1}Vb|9M3D|([P`/+0Dd B 7 S A4? =2_zT\1`!_zT\1@@2xS;KP>6Vq*E-b}`cBDA?@AApvMĩޓېBMHrws)PW|⤬|W\9yBGAD I+ӛqpUX?I @ݐ1'vuP;*^I^&b#>e Kk6!L`b%V2>NOW@=9E`!NOW@=9E5R@2xkA߼mjI5i!m4m(, x1j5?"X/ ZЋ4zkŋ^xKkE)3a_* 7N.E_q8B1'2&# +M2̪҇`unj:W|4Oc`y/8;:y! (~9CQf?m7jz[LTcKjlh@k#SkCGRHbX1?Ve|][@">  Footnote TextCJaJ4@24 Header  !4 @B4 Footer  !.)@Q. Page NumberJP@bJ Body Text 2$dha$ CJOJQJ.&'(+..&'(+.1.MN/GHQn(p3{    R j k    F G W g x y  I a y }  !":;2JQij&>g%&MN  $).235:?DINSWXZ_dinsx|}8Urs!"12KLdetu<gYZx8rsuvwz}<z:IXgv  * + 1 O P I"J"""##\#]#^#a#d#e#h#m#q#r#u#z######9$$$%%k%l%%%%%&F&G&&&&j'k'''f(g(i((((((((((((((((((((((((()))s*t*+ +%+>+?+W+X+s,t,u,<.=.i.j...............................000000000000000000000000000000 0 0 0 000000000000000000000000000000p0p00000p000p0000000p0p0p0p0p0p0 0p 00p0p00p00p0p0p00p0000p0p0p0p0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00000p0000p00p0p0p0p00p00p0p0p00p0p0p0p0p00p0p00p000p0p00p0p0p0p0p0p00p0p0p000p0p0p0p0p0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00p00p0p0p0p00p00000000000p0p0p0p0p0p0p00p0p0p00p0p00 0p0 0p0 0 0 0 0 0 0 0 0 0 0 0 00p00p00p0p0p0p0p0 0p0p0p0p0p0000p0 0p0 0 0p0p0(00p0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00p0p00p00p0p000(0p0pz000 0@0z00@0z000?@AFJKLMNOPRSTVXYZ[\^`6/CEQegK_a   R f h      + - 1 E G I ] _ a u w }  "68.02FHQegj~"$&:<g{}/18LNUikz-/L`bu35Oce   7 K M ###9$M$O$$$$%%%*** +++&+:+<+?+S+U+.:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::KRTo!8@\(  @ pU 2) ~  6?pU  ~  6?&U 2) ~B  BD? ^' ~B  BD? Z'~B  BD?@'=~B  BD?B^'0~B   BD?^'~B   BD?^'~B   BD?'~B   BD?^'  0@&0( C ~ `tt0 G`T`TAM`TM`TGS GT0TPAMC"~2  6?0@&p( ~B  BD?p`'`'~2  6?@&0( B S  ?sg(.h*"Ft| |[4viw$ixdij.j.x..t....=*urn:schemas-microsoft-com:office:smarttags PlaceName=*urn:schemas-microsoft-com:office:smarttags PlaceType9*urn:schemas-microsoft-com:office:smarttagsplace pr{rt|~FHQT&M'(*+LMklZx '' ,,u,=....&,;=jmxz*,rtk m G I W Y g i y | prJO>C&L37PTmq-/28;=@B35Zw{}<>z|:<IKXZgivx + . { } K"M"""####$$Q$U$$$%%o%q%%%&&J&L&&&&&k'm'''.*3***!+$+X+Z+u, --=....33333333333333333333333333333333333333333333333333333333333333333333333333333333333333  I y } JQ%8s2LwP ^###$$%k''() +&+u,=.h...........u,=....vargasjevargasjevargasjevargasjevargasjevargasjevargasjevargasjevargasjeJuan E. Vargas a6 `pjx"yr*ZS6B+,/(0Da0 2TL>> %oXoo Yzhh^h`o(.hh^h`o(()hh^h`o(.hh^h`o(. ^` o(. ^` o(.tt^t`OJPJQJ^Jo(- DD^D`OJQJo(o   ^ `OJQJo(   ^ `OJQJo( ^`OJQJo(o ^`OJQJo( TT^T`OJQJo( $$^$`OJQJo(o ^`OJQJo(hh^h`) ^` o(. hh^h`OJQJo( ^` o(. >>6 Yz%oX0Da06B+x"yooaj 2 T        216> >L PP%"'#1'h587;JFdKPL%OD^8`g/?hdl mOzd!!"?vu y.H7L_lM^h nz\HL5h`xTU -AuItF3H  $).235:?DINSWXZ_dinsx|}xsuvwz}I"J"\#]#^#a#d#e#h#m#q#r#u#z###k'i(((((((((((((((((((((((())r,u,<..y9a̙0@\\\\.`@UnknownGz Times New Roman5Symbol3& z ArialoCentury SchoolbookNewCenturySchlbkINewCenturySchlbk?5 z Courier New;Wingdings"qhҪA7JyFA&%P%PQ%24_,_, 3QH(?l#CSCE 822: Information as a Measure College of EngineeringJuan E. Vargas8           FMicrosoft Word Document MSWordDocWord.Document.89qRoot Entry1 F1vn|Data b?WordDocument0"ObjectPool3vJvl1      "!#$%&'()+*,-./023456789:;<=>?@ABCDEFGHIJKLMNOPQVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~  !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`acdefghiopqsrtuvxwyz{|}~1TableUeSummaryInformation(2DocumentSummaryInformation8CompObjjMicrosoft Word 10.0@u7@֚f@D"d@jv%՜.+,D՜.+,l( hp  University of South CarolinaoP_, $CSCE 822: Information as a Measure Titlep@  FMicrosoft Word Document MSWordDocWord.Document.89q L_AdHocReviewCycleID_EmailSubject _AuthorEmail_AuthorEmailDisplayName )>tvargasje@engr.sc.edu Vargas, Juan