ࡱ> ;=:'` rHbjbj bbD""""l#L**t#P$P$P$P$+%+%+%)))))))$+h&.t)&+%+%&&)P$P$N*)))&P$P$))&))))P$h# "&))d*0*).{(.).)p+%"M%)e%y%+%+%+%))) +%+%+%*&&&& Multiple Regression % Often we have data on several independent variables that can be used to predict / estimate the response. Example: To predict Y = teacher salary, we may use: Example: Y = sales at music store may be related to: % A linear regression model with more than one independent variable is a multiple linear regression (MLR) model: % In general, we have m independent variables and m + 1 unknown regression parameters. Purposes of the MLR model (1) Estimate the mean response E(Y | X) for a given set of X1, X2, & , Xm values. (2) Predict the response for a given set of X1, X2, & , Xm values. (3) Evaluate the relationship between Y and the independent variables by interpreting the partial regression coefficients b0, b1, & , bm (or their estimates). Interpretations: % (Estimated intercept): the (estimated) mean response if all independent variables are zero (may not make sense) % bi (or  EMBED Equation.3 ): The (estimated) change in mean response for a one-unit increase in Xi , holding constant all other independent variables. % May not be possible: What if X1 = home runs and X2 = runs scored? % Note: The partial effects of each independent variable in a MLR model do not equal the effect of each variable in separate SLR models. % Why? The independent variables tend to be correlated to some degree. % Partial effect: interpreted as the effect of an independent variable  in the presence of the other variables in the model. % Finding least-squares estimates of b0, b1, & , bm is typically done using matrices:  EMBED Equation.3  = (XTX)-1 XTY where: Y = vector of the n observed Y values in data set X = matrix containing the observed values of the independent variables (see sec. 8.2)  EMBED Equation.3  = a vector of the least squares estimates  EMBED Equation.3  % We will use software to find the estimates of the regression coefficients in the MLR model. Example: Data gathered for 30 California cities. Y = annual precipitation (in inches) X1 = altitude (in feet) X2 = latitude (in degrees) X3 = distance from Pacific (in miles) Estimated model is:  EMBED Equation.3  From computer: Interpretation of EMBED Equation.3 ? Interpretation of EMBED Equation.3 ? Interpretation of EMBED Equation.3 ? Inference with the MLR model % Again, we don t know s2 (the error variance), so we must estimate it. % Again, we use as our estimate of s2: % As in SLR, the total variation in the sample Y values can be separated: TSS = SSR + SSE. % SS formulas given in book  for MLR, we will use software. Rain example: SSR = SSE = Error df = MSE = % Most values in ANOVA table similar as for SLR. % m d.f. associated with SSR % n  m  1 d.f. associated with SSE Overall F-test % Tests whether the model as a whole is use&(Zh   , . x D F    < > @ D F H R T V < > ޹޹޹޹޹ު޹޹޹޹޹޹h.hT~56CJ$aJ$hT~hT~56CJ$H*aJ$hT~hT~5CJ$H*aJ$hT~hT~56>*CJ$aJ$hT~hT~56CJ$aJ$hV5CJ$aJ$hT~hT~5>*CJ$aJ$hT~5CJ$aJ$hg$hT~5>*CJ$aJ$hT~5>*CJ$aJ$/(*  l n p r t v x D gdT~$a$gdT~rH h j ,.P6rRrt0$a$gd]$a$gdT~ ,.L:<>HJprtv õõåÌ}j\P\h `56CJ$H*aJ$hT~h `56CJ$aJ$$jh `h `5CJ$EHUaJ$jFM h `CJUVaJjh `5CJ$UaJ$hU5CJ$H*aJ$hT~h `5CJ$OJQJaJ$h `h `5>*CJ$aJ$h `5CJ$aJ$hT~hT~56CJ$H*aJ$hT~5CJ$aJ$hT~5CJ$H*aJ$hT~hT~5CJ$OJQJaJ$8:^  RTz|~ĴĴĴČ}j\\\h]h]5CJ$H*aJ$$j'h]h]5CJ$EHUaJ$j0GM h]CJUVaJjh]5CJ$UaJ$hT~h]56CJ$H*aJ$h]5CJ$H*aJ$hT~h]5CJ$OJQJaJ$h]5CJ$aJ$h `h `5>*CJ$aJ$hT~h `56CJ$aJ$h `5CJ$aJ$hT~h `5CJ$H*aJ$ BDjlnp2@  #$%I_`s龫霉{{{rdrjhA^5CJ$UaJ$hA^5CJ$aJ$h]h]5CJ$H*aJ$$jh `hA^5CJ$EHUaJ$jGM hA^CJUVaJ$jWh]h]5CJ$EHUaJ$j0GM h]CJUVaJjh]5CJ$UaJ$h]h]56CJ$aJ$h]5CJ$aJ$h]h]5>*CJ$aJ$$02#IJw   ()$a$gd.gdA^stuv  (.ƾƾƾsfZLh.hA^5>*CJ$aJ$hT~hA^5CJ$aJ$jhA^hA^EHUj)GM hA^CJUVaJj hA^hA^EHUjGM hA^CJUVaJj h2WhA^EHUjGM hA^CJUVaJhA^jhA^UhA^5CJ$aJ$jhA^5CJ$UaJ$$j hA^hA^5CJ$EHUaJ$jDGM hA^CJUVaJ.02BD28:>@FH4444444ؼخإإ~pgYghfhf5CJ$H*aJ$hf5CJ$aJ$h.h.56CJ$aJ$Uh.h.5>*CJ$aJ$h.5CJ$aJ$h]h]56CJ$aJ$h]5CJ$aJ$h]hA^56CJ$aJ$hA^hA^5>*CJ$aJ$hA^hA^56CJ$aJ$hA^5CJ$aJ$hA^hA^5CJ$H*aJ$hA^hA^5CJ$OJQJaJ$`l: 44444456555N6P6t6v6$a$gd.gdA^less. % Null hypothesis: none of the independent variables are useful for predicting Y. H0: b1 = b2 = & = bm = 0 Ha: At least one of these is not zero % Again, test statistic is F* = MSR / MSE % If F* > Fa(m, n  m  1), then reject H0 and conclude at least one of the variables is useful. Rain data: F* = Testing about Individual Coefficients % Most easily done with t-tests. % The j-th estimate,  EMBED Equation.3  , is (approximately) normal with mean bj and standard deviation  EMBED Equation.3 , where cjj = j-th diagonal element of (XTX)-1 matrix. % Replace s2 with its estimate, MSE: % To test H0: bj = 0, note: % For each coefficient, computer gives:  EMBED Equation.3 ,  EMBED Equation.3 , and t statistic. Ha Reject H0 if: Software gives P-value for the (two-tailed) test about each bj separately. Rain data: F-tests about sets of independent variables % We can also test whether certain sets of independent variables are useless, in the presence of the other variables in the model. Example: Suppose variables under consideration are X1, X2, X3, X4, X5, X6, X7, X8. Question: Are X2, X4, X7 needed, if the others are in the model? % We want our model to have  large SSR and  small SSE. Why? % If  full model has much lower SSE than the  reduced model (without X2, X4, X7), then at least one of X2, X4, X7 is needed. ! conclude b2, b4, b7 not all zero. % To test: H0: b2 = b4 = b7 = 0 vs. Ha: b2, b4, b7 not all zero Use: Reject H0 if Example above: numerator d.f. = % Can test about more than one (but not all) coefficients within computer package (TEST statement in SAS or anova function in R) Example: Inferences for the Response Variable in MLR As in SLR, we can find: % CI for the mean response for a given set of values of X1, X2, & , Xm. % PI for the response of a new observation with a given set of values of X1, X2, & , Xm. Examples: % Find a 90% CI for the mean precipitation for all cities with altitude 100 feet, latitude 40 degrees, and 70 miles from the coast. % Find a 90% prediction interval for the precipitation of a new city having altitude 100 feet, latitude 40 degrees, and 70 miles from the coast. Interpretations: % The coefficient of determination in MLR is denoted R2. % It is the proportion of variability in Y explained by the linear relationship between Y and all the independent variables (Note: 0 d" R2 d" 1). % The higher R2, the better the linear model explains the variation in Y. % No exact rule about what a  good R2 is. Rain example: Interpretation: 44444444444n5p55555555555j6l6~667787:7`7b7d7f7h7777ؽددءدؙymbhf5CJ$H*aJ$hfhf5CJ$aJ$j,hfhfEHUjHM hfCJUVaJhfjhfUhfhf5>*CJ$aJ$hfhf56CJ$aJ$"hfhf5CJ$H*OJQJaJ$h$5CJ$aJ$hf5CJ$aJ$hfhf5CJ$H*aJ$hfhf5CJ$OJQJaJ$&v6x6z6|6~66678888 9 99999999:::::: :gd(gdf$a$gdfgdA^7778888(8,82848h8j8n8r88888888888"9f9h9~scXOGjh86Uh865CJ$aJ$hf5CJ$H*aJ$hfhf5CJ$OJQJaJ$hfhfCJ$aJ$hA^hf5CJ$H*aJ$hA^hf5CJ$OJQJaJ$h]hf5CJ$H*aJ$hfhf56CJ$aJ$hfhf5CJ$H*aJ$$jUhfh865CJ$EHUaJ$jHM hfCJUVaJjhf5CJ$UaJ$hf5CJ$aJ$h99999999999999999: ::::::oaoaSChfh(5CJ$OJQJaJ$h(h(5>*CJ$aJ$hs|h(5>*CJ$aJ$"hs|h(5CJ$H*OJQJaJ$hs|h(5CJ$aJ$h(5CJ$aJ$hf5CJ$aJ$$jhfh(5CJ$EHUaJ$jHM h(CJUVaJh865CJ$aJ$jh865CJ$UaJ$jh86Ujhfh86EHUjHM h86CJUVaJh86 :"::::::::::::8;:;@<B<<<p=r===>??gdqr$a$gd(gd(:::8;;;<<><@<<<<<<<<<<<<<<<<< ======>>>>>>>>>>>> ? ????????^?`?d?f?h?n?p?̳̳̳̳̳̳̳̳̳̳̳̳̳̳̳̣̣̣̣̣̾̾̕hfhqr5CJ$H*aJ$hfhqr5CJ$OJQJaJ$hqr5CJ$H*aJ$hqrhqr5CJ$H*aJ$hqr5CJ$aJ$hE#5CJ$aJ$h(h(5>*CJ$aJ$h(5CJ$aJ$h(5CJ$H*aJ$:p?r?x?z?|??????????????<@@@@ AAAAbAA^B`BbBfBhBjB췮|seWseWhT~hLC25CJ$H*aJ$hT~hLC256CJ$aJ$hLC25CJ$aJ$hLC2hT>o5>*CJ$aJ$#hT>ohT>o5CJ$OJQJ^JaJ$#hT>ohkD5CJ$OJQJ^JaJ$hT>o5CJ$aJ$hkD5CJ$aJ$hqrhqr5CJ$H*aJ$hfhqr5CJ$H*aJ$hfhqr5CJ$OJQJaJ$hqr5CJ$aJ$hqr5CJ$H*aJ$"?????????:@<@>A@ARATAVAXAZA\A^A`AbAAAAA|B$a$gdLC2gdqr ^`gdqrjBtBvBxB|B~BCCCCCC&C(C*C,C.C0CBCDCFFrFtFFFFF0G2G^G`GGG H"H,HNHRHpH;;ypphZ 5CJ$aJ$hLC2hLC25>*CJ$aJ$hLC2hLC256CJ$aJ$"hLC2hLC25CJ$H*OJQJaJ$hT>o5CJ$aJ$hT~hLC25CJ$H*aJ$hLC2hLC256CJ$H*aJ$hLC256CJ$H*aJ$hT~hLC256CJ$H*aJ$hT~hLC256CJ$aJ$hLC25CJ$aJ$'|B~B.C0CDCFCNDpErEtEvEEEEEEEEEEEEF F@GBGG,H.HgdLC2gdqr.H0H2HNHPHRHrHgdA^gdqrpHrHhT~hf5CJ$aJ$,1h/ =!"#$% 5 01h:pLC2/ =!"#$% 9 01h:pLC2/ =!"#$% P 5 01h:pLC2/ =!"#$% 'Dd lb  c $A? ?3"`?2q"y W)dTMD`!E"y W)dTHhxڕ1KALbWXvXZR s81!+ʟ`ik+)')!:;Y(.ͷCc vHx)bc}pwDvֱ c%oDH:<]ǵ }c-A_m3tX$/QyVfqo^zˤ-=1y|j?U1]I=E|]rrg .9nJOʽ|7_'X7MuļӯƓ@bmkݦ?OK[0Dd b  c $A? ?3"`?2z>d818KYzVk`!N>d818KYzRxcdd``$d@9`,&FF(`TiS A?dbAQ@P5< %!  @_L ĺE,a ˌL *'\ \UC@6GLF]F\@~KD7l?мbr<1nn \PrCl,.v0o8+KRs8@2u(2t5=`n-ME0Dd b  c $A? ?3"`?2z>d818KYzV`!N>d818KYzRxcdd``$d@9`,&FF(`TiS A?dbAQ@P5< %!  @_L ĺE,a ˌL *'\ \UC@6GLF]F\@~KD7l?мbr<1nn \PrCl,.v0o8+KRs8@2u(2t5=`n-MEDd lb  c $A? ?3"`?2N@|EG-hw `!N@|EG-hw 4 hnxcdd``ed``baV d,FYzP1n:B@?b 30XsC0&dT20$ͤ `[YB2sSRs"6~.ObyYt9 ;Hgn%ļH1 M rZQy 2l s*a|O&Tn/7loܞ.V_ΟŌa~;#r 1m_o)Dd ,|b ! c $A ? ?3"`?2ss12^1)O`!Gs12^1)`0xcdd`` @c112BYL%bpuData 2=WordDocument6bObjectPool9 x#_1296481409F  Ole CompObjfObjInfo $).1479:;<=>@ABCDF FMicrosoft Equation 3.0 DS Equation Equation.39qet "  i FMicrosoft Equation 3.0 DS Equation Equation.39qEquation Native :_1296548912 F  Ole CompObj fObjInfo Equation Native  4_1296558000 F  Ole  l~ "  FMicrosoft Equation 3.0 DS Equation Equation.39qZ0J\J "  0 ,"  1 ,& ,"  mCompObj fObjInfo Equation Native v_1296557892F  Ole CompObjfObjInfoEquation Native  FMicrosoft Equation 3.0 DS Equation Equation.39q$o 2Y ="  0 +"  1 X 1 +"  2 X 2 +"  3 X 3_1296558016F  Ole CompObjfObjInfo FMicrosoft Equation 3.0 DS Equation Equation.39q "  0 FMicrosoft Equation 3.0 DS Equation Equation.39qEquation Native :_1296558111'F  Ole CompObj fObjInfo! Equation Native !:_1296558121$F  Ole "xD) "  2 FMicrosoft Equation 3.0 DS Equation Equation.39qdL "  3CompObj#%#fObjInfo&%Equation Native &:_1296564695",)F  Ole 'CompObj(*(fObjInfo+*Equation Native +: FMicrosoft Equation 3.0 DS Equation Equation.39q~ "  j FMicrosoft Equation 3.0 DS Equation Equation.39q_12965647301.F  Ole ,CompObj-/-fObjInfo0/Equation Native 0T_12965649973F  Ole 2CompObj243f88 !  c jj  2 FMicrosoft Equation 3.0 DS Equation Equation.39q4H$n  c jj MSE ObjInfo55Equation Native 6P1TableZ.SummaryInformation(88^)Dd ,b  c $A? ?3"`?2s ,K@بدOG`!G ,K@بدhxcdd`` @c112BYL%bpuOIb"<7h6W[ꜱ*7ʫh"brH^5z"'}|]<K%}Ɯy޺h גOg4di|bK~msXoAFsuCu/3KUd ~ݓqY!Vmf~w`hk_H6L/(Y<VOh+'0  < H T`hpxMultiple Regression hitchcockNormal hitchcock5Microsoft Office Word@G@|Q@11՜.+,0 hDocumentSummaryInformation8?HCompObjEqp   University of South Carolina& X' Multiple Regression Title  FMicrosoft Office Word Document MSWordDocWord.Document.89q@@@ T~NormalCJ_HaJmH sH tH DAD Default Paragraph FontRiR  Table Normal4 l4a (k(No Listbb.bebbjklmno45wx(9m  ST)O9:#IJw  ( ) q 5 6 Y _ v w / 0 P Q c d e f g h k l  3456789:IJ!"  #$FG ghP89?@ABPQRd000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000003456789:IJ!"dZ00;Z00Z00 @0Z00Z00Z00Z00X00 X00X0000X0 0X0 000X00X00X00X00 00 s.47h9:p?jBpHrH%'(*+-0 0v6 :?|B.HrH&),./rH)=?!57_su   1 3  b::::::::::::8@0(  B S  ?JX9|KX9dd9*urn:schemas-microsoft-com:office:smarttagsState9*urn:schemas-microsoft-com:office:smarttagsplace |n)+lne g < ? D F >Aced qs$(gkOT<> e g y } $&IL^bO;=d3333333333333333333333333333333333333!8 ^ _ ?ORadd.Z VE#LC286^ROA^T>oXuT~qr$]kD `Us(.f]@~~