ࡱ> vxuEq bjbjt+t+ "AAPE]  8d -^t"f-h-h-h-h-h-h-$.0- -WWWWT f- f-WWg,  f-@%d_ xP-NC 2569 Under Review in Neural Computation, Revised June 10, 2002 Selecting the Loss Function for Robust Linear Regression Vladimir Cherkassky Yunqian Ma Department of Electrical and Computer Engineering, University of Minnesota, Minneapolis, MN 55455, U.S.A. {cherkass, myq}@ece.umn.edu Abstract This paper addresses selection of the loss function for regression problems with finite data. It is well-known (under standard regression formulation) that for a known noise density there exist an optimal loss function under an asymptotic setting (large number of samples), i.e. squared loss is optimal for Gaussian noise density. However, in real-life applications the noise density is unknown and the number of training samples is finite. Robust statistics provides prescriptions for choosing the loss function using only general information about noise density; however robust statistics is based on asymptotic arguments and may not work well for finite sample problems. For such practical situations, we suggest using Vapniks EMBED Equation.3-insensitive loss function. We propose a practical method for setting the value of EMBED Equation.3 as a function of known number of samples and (known or estimated) noise variance. First we consider commonly used unimodal noise densities (such as Gaussian and Laplacian). Empirical comparisons for several representative linear regression problems indicate that the proposed loss function yields more robust performance and improved prediction accuracy, in comparison with commonly used squared loss and least-modulus loss, especially for noisy high-dimensional data sets. We also performed comparisons for symmetric bimodal noise densities (where large errors occur more likely than small errors). For such (bimodal) noise, the proposed EMBED Equation.3-insensitive loss consistently provides improved prediction accuracy (in comparison with other loss functions), for both low-dimensional and high-dimensional problems. Introduction _________________________________________________________ Estimation of a real-valued function from finite set of (noisy) samples is a central problem in applied statistics. Even though the main solution approaches have been proposed centuries ago (i.e., the least-squares method by Gauss and the least-modulus method by Laplace), statistical justification of these methods is based on restrictive assumptions. That is, statistical optimality of these methods is based on two main assumptions: Asymptotic setting, i.e. good statistical properties such as asymptotic unbiasedness of the least-squares method for linear regression, holds for large number of training samples. It is not at all clear whether this property (unbiasedness) is necessary for finite-sample problems; Knowledge of the noise density, i.e., the least squares method has the smallest variance for linear regression with normal additive noise. However, in most applications, the noise density is not known. In this paper, we consider regression problems under practical settings when the number of samples is finite, and the noise density is unknown. There are two main issues that need to be addressed by any method, that is: model selection (complexity control); choice of parameter estimation (optimization) procedure. Model selection is outside the scope of this paper; however see (Cherkassky and Mulier, 1998; Cherkassky, 2002) for discussion of model complexity control under the framework of statistical learning theory (or VC-theory). Here we only focus on the parameter estimation procedure, or equivalently, on the choice of the loss function for regression estimation. Moreover, we only consider linear regression problems where the number of terms (in a linear model) is given a priori. For such problems, we consider three representative loss functions, i.e. standard squared loss, least-modulus loss (commonly used for robust regression), and EMBED Equation.3-insensitive loss function recently proposed by Vapnik (Vapnik, 1995) for Support Vector Machine (SVM) regression. Our goal is to investigate (via empirical comparisons) appropriateness of these loss functions for finite-sample estimation problems with unknown noise density. Statistical characterization of different types of loss functions in terms of their robustness to different noise models is addressed in robust statistics (Huber, 1964); and robust loss functions have been extensively used for modeling real-life noisy data (i.e. least-modulus loss has been successfully used for forecasting of economic and political data in (Werbos and Titus, 1978)). However, characterization of different loss functions for finite-sample problems remains sketchy and unsatisfactory, because robust statistics provides only asymptotic characterization of unbiased estimators under maximum-likelihood approach. On the other hand, it is well known that with finite-sample estimation problems it may be preferable to use biased estimators (e.g. recall the bias-variance dilemma in statistics). The paper is organized as follows. Section 2 reviews standard regression problem statement, describes representative loss functions, and reviews assumptions underlying each loss function. Section 3 presents empirical comparisons for finite-sample settings, for symmetric unimodal noise densities (commonly used in statistical literature) and for bimodal noise densities. Empirical results show that for unimodal noise, EMBED Equation.3-insensitive loss (with appropriately chosen EMBED Equation.3-value) provides superior robustness and prediction accuracy for high-dimensional sparse data sets. However, for large-sample settings (with unimodal noise) standard squared-loss provides superior prediction accuracy. For bimodal noise, the proposed EMBED Equation.3-insensitive loss consistently yields best prediction accuracy for large-sample and small-sample settings alike. Summary and discussion are given in Section 4. 2 Loss functions for regression _____________________________________________ We consider standard regression formulation under general setting for predictive learning (Vapnik, 1995; Cherkassky and Mulier, 1998; Hastie et al, 2001). The goal is to estimate unknown real-valued function in the relationship: EMBED Equation.3 \* MERGEFORMAT (1) where EMBED Equation.3  is independent and identically distributed (i.i.d.) zero mean random error (noise),  EMBED Equation.3  is a multivariate input and EMBED Equation.3 \* MERGEFORMATis a scalar output. The estimation is made based on a finite number (n) of samples (training data): EMBED Equation.3 . The training data are i.i.d. samples generated according to some (unknown) joint probability density function (pdf),  EMBED Equation.3  (2) The unknown function in (1) is the mean of the output conditional probability (aka regression function)  EMBED Equation.3 . (3) A learning method (or estimation procedure) selects the 'best' model  EMBED Equation.3  from a set of approximating functions (or possible models)  EMBED Equation.3 parameterized by a set of parameters EMBED Equation.3 . The quality of an approximation is measured by the loss or discrepancy measure EMBED Equation.3 , and the goal of learning is to select the best model minimizing (unknown) prediction risk:  EMBED Equation.3  (4) It is known that the regression function (3) is the one minimizing prediction risk (4) with the squared loss function loss:  EMBED Equation.3  (5) Note that the set of functions EMBED Equation.3 , EMBED Equation.3  supported by a learning method may or may not contain the regression function (3). Thus the problem of regression estimation is the problem of finding the function  EMBED Equation.3  (regressor) that minimizes the prediction risk functional  EMBED Equation.3  (6) using only the training data. This risk functional measures the accuracy of the learning method's predictions of unknown target function EMBED Equation.3 . Since the prediction risk functional (6) is not known, we need to estimate the best model  EMBED Equation.3 using only available (training) data. For a given parametric model, its parameters are estimated by minimizing the empirical risk:  EMBED Equation.2  (7) For example, with commonly used squared loss, the best model is found via minimization of average squared error:  EMBED Equation.2  (8) In this paper we assume that in regression formulation (1) the target function is linear and its parametric form is known, i.e. EMBED Equation.3 \* MERGEFORMAT (9) The goal is to estimate parameters of a linear regression (9) when the number of samples is finite, and the noise density in (1) is not known. The key problem for regression is the choice of a loss function used to minimize the empirical risk functional (7). For example, it is well-known that using quadratic loss function for regression problems with normal additive noise provides an asymptotically efficient (best unbiased) estimator for regression. Using standard statistical arguments (Smola and Schlkopf, 1998; Hastie et al, 2001), one can find the optimal loss function (in a maximum-likelihood sense) for a known noise density EMBED Equation.3 , that is:  EMBED Equation.3  (10) Two commonly used types of noise and corresponding optimal loss functions are shown in Figure 1a, 1b. We note, however, that this optimality is based on asymptotic arguments and it may not hold well for finite-sample settings. In most cases the noise distribution is unknown and robust methods (Huber, 1964) provide various loss functions using only general information about the model of noise. For example, if one only knows that the noise density is a unimodal symmetric smooth function, then the best minimax strategy for regression estimation is provided by the least-modulus (or absolute-value) loss:  EMBED Equation.3  (11) We emphasize here that statistical optimality properties for both the least-modulus and squared loss (under respective assumptions about the noise density) should be understood under an asymptotic setting, i.e. when the number of training samples is large. Several empirical results (presented later in this paper) clearly demonstrate that statistical optimality of these loss functions does not hold for finite-sample settings. Statistical analysis of various noise models typically considers unimodal and smooth noise densities, reflecting an assumption that small errors are much more likely than large errors. In practice, however, we are often faced with symmetric bimodal distributions (that may also contain discontinuities). Figure 1c, 1d shows representative examples, i.e. uniform noise model and bimodal uniform noise density:  EMBED Equation.3  (12) In this paper, we shall use noise densities shown in Figure 1 as representative examples of unimodal and bimodal types of noise. Recently, a new loss function called EMBED Equation.3-insensitive loss has been proposed by Vapnik (1995): EMBED Equation.3 (13) This loss function had been proposed in the context of Support Vector Machine (SVM) regression. SVM approach to linear regression amounts to (simultaneous) minimization of EMBED Equation.3-insensitive loss and minimization of the norm of linear parameters (EMBED Equation.3). This can be formally described by introducing (non-negative) slack variablesEMBED Equation.3 EMBED Equation.3, to measure the deviation of training samples outside EMBED Equation.3-insensitive zone. Thus SVM regression can be formulated as minimization of the following functional: EMBED Equation.3 (14) Subject to EMBED Equation.3 This optimization problem can be transformed into the dual problem (Vapnik, 1995), and its solution is given by EMBED Equation.3 (15) with coefficient values in the range EMBED Equation.3,EMBED Equation.3. And EMBED Equation.3denotes the dot product in the input space. In representation (15), typically only a fraction of training samples appear with non-zero coefficients and such training samples are called support vectors. For most applications, the number of support vectors (SVs)  EMBED Equation.2 is usually much smaller than the number of training samples. For nonlinear regression problem, SVM approach performs first a mapping from the input space onto a high-dimensional feature space, and then performs linear regression in the high-dimensional feature space using EMBED Equation.3-insensitive loss (Vapnik, 1995; Cherkassky and Mulier, 1998; Schoelkopf et al, 1999). SVM approach enables efficient model complexity control using a special structure on a set of high-dimensional linear models in the feature space. SVM model complexity control and its attractive computational formulation have resulted in many successful applications for nonlinear regression (Schlkopf et al, 1999). However, in this paper we do not consider nonlinear regression problems, since our goal is a better understanding of different loss functions, rather than efficient complexity control. The primary motivation (for using EMBED Equation.3-insensitive loss) appears to be computationally efficient SVM regression formulation resulting in a sparse form of SVM solution. Namely, the value of EMBED Equation.3 controls the number of support vectors (the sparseness of SVM solutions) and thus indirectly controls model complexity and generalization. More recently, several researchers used SVM-like optimization formulation with different loss functions, i.e., squared loss, squared loss with EMBED Equation.3-insensitive zone, and Huber loss (Vapnik, 1998; Smola et al, 1998; Suykens et al, 2001). In fact, one can use any convex loss function under SVM regression formulation; however only EMBED Equation.3-insensitive loss yields sparse SVM solutions (Vapnik, 1998). This paper presents a different interpretation of Vapniks EMBED Equation.3-insensitive loss, i.e. its advantages for finite-sample estimation problems. The main idea of EMBED Equation.3-insensitive loss is to ignore small errors during minimization of empirical risk (7); whereas large errors are assigned absolute-value loss appropriate for unknown noise density. The conceptual difference between EMBED Equation.3-insensitive loss and other types of loss functions (Huber loss, squared loss, and least modulus loss) is that the latter do not make qualitative distinction between small errors and large errors, which is crucial for finite-sample estimation problems. Unfortunately, SVM framework does not provide clear guidelines on how to select the value of EMBED Equation.3-insensitive zone for a given data set, so in practical studies its selection depends on user experience and/or on computationally intensive data-driven techniques (Schoelkopf et al, 1999). SVM estimates depend on both parameter C and the value of EMBED Equation.3 in formulation (14). Hence, obtaining optimal SVM solutions for linear regression requires setting optimal values for both parameters. Recently, Cherkassky and Ma (2002) proposed a practical method for selecting the value of EMBED Equation.3 and the value of regularization parameter C for SVM regression directly from the training data. Namely, the value of C is chosen as:  EMBED Equation.2  (16) where  EMBED Equation.2  is the mean of the training responses (outputs), and  EMBED Equation.2  is the standard deviation of the training response values. Prescription (16) can effectively handle outliers in the training data. In practice, the response values of training data are often scaled so that  EMBED Equation.2 =0; then the optimal C is  EMBED Equation.2 . The value of EMBED Equation.3 is selected as  EMBED Equation.2  (17) where  EMBED Equation.2  is the standard deviation of additive noise and n is the number of training samples, and  EMBED Equation.2  is an empirically determined constant. The value  EMBED Equation.2  was suggested by Cherkassky and Ma (2002). Thus, expression (17) with  EMBED Equation.2  is used throughout this paper for setting the value of EMBED Equation.3-insensitive zone. Note that using prescription (17) requires estimation of noise level  EMBED Equation.2 ; this can be accomplished using standard noise estimation approaches see Cherkassky and Ma (2002) for details. In the remainder of the paper we assume that standard deviation of noise  EMBED Equation.2  is known (or can be accurately estimated from data). However, we point out here that in most practical applications the noise density is unknown (and can not be accurately estimated), even when its standard deviation can be reliably estimated from the training data. Further, it can be shown (Cherkassky and Ma, 2002) that for linear regression settings the choice of EMBED Equation.3 affects prediction accuracy much more than the choice of C parameter (provided that C parameter is larger than the value given by (16)). In other words, one can use very large values of C parameter in SVM formulation (14) without any degradation in prediction risk. For example, Fig. 2 shows SVM prediction accuracy as a function of both parameters, for a representative linear regression data set. As evident from Fig. 2, one can use any large C-value (say over 40) and then select EMBED Equation.3-value to optimize prediction risk. Conceptually, using large C (infinite) values in SVM formulation (14) means that such formulation becomes equivalent to minimization of EMBED Equation.3-insensitive loss for the training data, without penalization (regularization) term. This enables fair comparisons between SVM and other formulations / loss functions (e.g. least squares) which do not use regularization term. In the next section we present empirical comparisons for several linear regression estimation using three representative loss functions: squared loss, least-modulus and EMBED Equation.3-insensitive loss with selection of EMBED Equation.3 given by (17). Our goal is to investigate the effect of a loss function on the prediction accuracy of linear regression with finite samples. Even though SVM regression has been extensively used for regression applications (Scholkopf et al, 1999), its success is mainly due to remarkable ability of SVM models to handle nonlinear high-dimensional problems. However, there is little consensus and understanding of the importance of EMBED Equation.3-insensitive loss itself for standard linear regression estimation. The only existing study (Drucker, et al, 1997) showing empirical comparisons between SVM and ordinary least squares (OLS) for linear regression makes rather indefinite conclusions. This study applies SVM and OLS to a linear regression problem with 30 input variables, where regression estimates are obtained from 60 noisy training samples, and concludes that at high noise levels SVM is better than OLS, but at low noise levels OLS is better than SVM. This study is rather sketchy since it uses a single data set for regression comparisons, and does not describe any systematic procedure for selecting the value of EMBED Equation.3 for SVM regression. Empirical Comparison ________________________________________________ This section presents systematic comparisons of several loss functions/methods for linear regression. Specifically, we are interested in the following factors that are likely to affect generalization performance of SVM, OLS and LM methods, such as: low vs. high-dimensional data sets (target functions); sparse vs. non-sparse settings (where sparseness can be defined as the ratio of the number of samples to the number of input variables); percentage of 'important or significant input variables for high-dimensional problems; the amount of additive noise (measured as SNR) type of noise. First we describe experimental procedure and then present empirical results. Comparisons are organized into 2 groups, that is: large-sample setting for low-dimensional problems (when the number of samples is significantly larger than the number of parameters in linear regression), and high-dimensional or sparse problems. For high-dimensional problems (which is the case of practical interest) we also show comparisons for various settings corresponding to large/small percentage of significant input variables, high/low noise levels etc. All comparisons for different methods are shown for three representative unimodal noise densities: Gaussian, Laplacian and Uniform. In addition, we show comparisons for bimodal noise density. The goal (of comparisons) is to gain better understanding of relative advantages/limitations of different methods for linear regression: optimal least squares (OLS), least modulus (LM) and SVM regression. Note that SVM method has a tunable parameter EMBED Equation.3 selected via analytical prescription (17) for all comparisons presented in this paper. Alternatively, optimal selection of EMBED Equation.3 can be done using resampling methods. We empirically compared the resampling approach (via cross-validation) and analytical approach for selecting the value of EMBED Equation.3, and found no significant difference in terms of prediction accuracy of SVM estimates. Training data: we use simulated training data EMBED Equation.2  with random  EMBED Equation.2 -values uniformly distributed in the input space, and  EMBED Equation.2 -values generated according to EMBED Equation.2 . Target functions. Several low-dimensional and high-dimensional data sets are generated using the following target functions: Low-dimensional  EMBED Equation.2  (18) High-dimensional function of 20 variables:  EMBED Equation.2 ,  EMBED Equation.2  (19) We also considered variations of a high-dimensional function (19) with only a fraction of important (or significant) variables, as shown in Table 1. Table 1 Target function10% EMBED Equation.3 25% EMBED Equation.3 50% EMBED Equation.3 75% EMBED Equation.3  Sample size. Various training sample sizes (n= 30,40,50) are used to contrast relative performance of different methods under large sample settings and sparse sample settings. The distinction can be quantified using the ratio of the number of samples (sample size) to the number of input variables. Additive noise. The following types of unimodal noise were used: Gaussian noise, uniform noise and Laplacian noise. Notice that squared loss is (asymptotically) optimal for Gaussian noise and least-modulus loss is (asymptotically) optimal for Laplacian noise density. In addition, we used bimodal noise (see Figure 1d) that is expected to be most appropriate for  EMBED Equation.2 -insensitive loss. We also varied the noise level (as indicated by different SNR values) for high-dimensional data, in order to understand the effect of noise level on methods performance. Signal-to-noise ratio (SNR) is defined as the ratio of the standard deviation of the true (target function) output values over the standard deviation of the additive noise. Given a large number of experimental design factors (as defined above), we conducted many experiments with different target functions, noise levels etc. This paper presents only a small but representative subset of comparisons. However, we emphasize that all qualitative conclusions presented in this paper are consistent with a wider set of comparison results not shown here due to space limitations. Experimental protocol. For a given training sample with specified statistical properties (sample size, noise level/type etc. as defined above) we estimate parameters of regression via minimization of the empirical risk (7) using 3 different loss functions, i.e., standard square loss, modulus loss and  EMBED Equation.2 -insensitive loss (with proposed selection of  EMBED Equation.2 -value). The quality of each model is evaluated as its prediction accuracy, measured as mean-squared error (MSE) between (estimated) model and the true target function. This quantity is measured using large number of independent test samples uniformly distributed in the input space. Specifically, for low-dimensional problems we used 200 test samples, and for high-dimensional problems 2,000 test samples were used to estimate the prediction risk. Since the model itself depends on a particular (random) realization of training sample (of fixed size), its (measured) prediction accuracy is also a random variable. Hence, we repeat the experimental procedure (described above) with many different realizations of training data (100 runs) and show average prediction accuracy (risk) for methods comparison. Graphical presentation of prediction accuracy (risk) for three estimation methods uses the following labels: OLS (for ordinary least-squares method), LM (for least-modulus method) and SVM (for SVM with  EMBED Equation.2  insensitive loss using proposed optimal selection of  EMBED Equation.2 ). Notice that LM method is a special case of SVM with  EMBED Equation.2 -insensitive loss (with  EMBED Equation.2 =0). Comparison results. Empirical comparisons for low-dimensional target function (18) are summarized in Table 2. The results show average prediction accuracy (MSE) for various methods/loss functions observed for different types of noise (with the same SNR value). As expected, OLS shows superior performance for Gaussian noise, whereas LM is the best for Laplacian noise. For unimodal noise OLS shows best performance overall, whereas SVM clearly provides superior prediction performance for bimodal noise. Good overall performance of standard OLS can be explained by noting that this data set is an example of large-sample setting, i.e. we use 30 samples to estimate 3 parameters of regression function. Hence standard statistical arguments underlying asymptotic optimality of different loss functions hold well in this case. Table 2: Prediction accuracy for low-dimension data EMBED Equation.3 , n=30, SNR=2 Results show the average MSE error (for 100 realizations) for different types of noise. UnimodalBimodalGaussianUniformLaplacianOLS0.42100.43380.40020.4807LM0.65601.12380.33272.0734SVM0.59810.44870.65030.1993 As mentioned earlier, SVM method selects the value of  EMBED Equation.2  using proposed analytic expression (17). Empirical results in Fig. 3 are shown to support our claim that expression (17) yields (near) optimal selection of  EMBED Equation.2 -values. Namely, Fig. 3 shows comparisons of prediction risk obtained for the low-dimensional data set using SVM with  EMBED Equation.2 -values chosen around the optimal value selected by (17), that is: 50% smaller than selected value (denoted as 50% in Fig. 3); 50% larger than selected value (denoted as +50%); Twice the selected value (denoted as 2*Sel). Comparisons of prediction risk shown in Fig.3 (with different  EMBED Equation.2 -values) show that expression (17) selects the value of  EMBED Equation.2 -parameter quite well (close to optimal) for different types of noise. Next we show comparisons for a high-dimensional target function (19). Results shown in Fig. 4 are intended to illustrate how methods prediction performance depends on the sparseness of training data. This is accomplished by comparing prediction risk (MSE) for data sets with different sample sizes (n=30, 40 and 50) under the same SNR=2. Results in Fig.4 indicate that SVM method consistently (for all types of noise) outperforms other methods under sparse settings, i.e. for 30 samples when the ratio n/d is smaller than 2. However, for 50 samples, when this ratio is larger than 2, we approach large-sample settings, and the methods performance becomes similar. The distinction between sparse setting and large-sample setting is not very straightforward as it also depends on the noise level. That is why comparisons in Fig. 4 are shown for a given (fixed) SNR value for all data sets. Next we show comparisons for the same high-dimensional target function (19) under sparse setting (n=30 samples) for different noise levels (SNR=1,3,5,7) in order to understand the effect of noise level on methods performance. Results in Fig. 5 clearly show superiority of SVM method for large noise levels; however for small noise levels SVM does not provide any significant advantages over OLS. Note that MSE results in Fig. 5 are shown on a logarithmic scale, so that the difference in prediction performance (MSE) for different methods at high noise levels (SNR=1) is quite significant (i.e., of the order of 100% or more). Additional experiments with high-dimensional data use high-dimensional target functions with a small number of important input variables (shown in Table 1). Here the goal is to understand how the methods performance is affected by the percentage of important variables, under sparse settings. Results presented in Fig. 6 use the same sample size n=30 and noise level SNR=2 for all comparisons. Examination of results in Fig. 6 suggests that SVM provides superior performance for all target functions due to sparseness of training data and relatively high noise level. However, relative performance of SVM is much better when the percentage of important variables is high. When the percentage of important variables is low (say 10%), SVM does not yield significant advantages vs OLS; this can be explained by noting that data sets with small number of important input variables effectively represent relatively large-sample settings. 4. Summary and Discussion _______________________________________________ This paper presents empirical comparisons between several loss functions (methods) for linear regression with finite samples. Previous SVM regression comparison studies focused mainly on nonlinear regression settings where one needs to select several hyper-parameters (such as SVM kernel,  EMBED Equation.2 -parameter and regularization parameter) in order to obtain good SVM model; and such parameter selection is usually performed by expert users and/or using resampling methods (Scholkopf, 1999). In contrast, this paper focuses on linear regression and provides an analytic prescription for selecting the value of  EMBED Equation.2 -insensitive loss. Empirical comparisons presented in this paper indicate that SVM approach clearly provides better prediction accuracy than OLS and LM for linear regression with sparse and noisy data. Our findings contradict a common opinion in the statistical literature (Smola and Schlkopf, 1998; Hastie et al, 2001) that for a given (known) noise density there exist an optimal method (loss function). Results presented in this paper have a number of practical and conceptual implications as discussed next. First, we have demonstrated practical advantages of using  EMBED Equation.2 -insensitive loss (with proposed selection of EMBED Equation.2 -value) for finite-sample linear regression problems, when the training data is noisy and sparse. For low-dimensional problems (i.e. non-sparse settings) using SVM does not offer any improvement over standard OLS method, as can be expected from asymptotic statistical theory. However, advantages of SVM approach become apparent for sparse high-dimensional data sets where it outperforms ordinary least squares (OLS) even for Gaussian noise. Second, this superiority of  EMBED Equation.2 insensitive loss for settings where OLS method is known to be statistically optimal has an important conceptual implication: statistical results developed under asymptotic settings do not hold for finite-sample high-dimensional data sets. Moreover, our results suggest that for finite-sample settings one only needs to know (or estimate) the standard deviation of noise, rather than its distribution. Third, our empirical results help to explain the empirical success of SVM approach for nonlinear regression, as explained next. Under standard SVM approach to nonlinear regression one first needs to perform a nonlinear mapping from an input space onto high-dimensional feature space and then perform linear regression (with special constraints) in this feature space according to SVM formulation. Minimization of regularized functional (14) for nonlinear regression should always result in sparse settings, where the number of parameters (in linear regression) is comparable to the number of training samples. According to findings presented in this paper, for such sparse data sets linear SVM should use  EMBED Equation.2 insensitive loss (rather than alternative loss functions). It may be interesting to perform empirical comparisons between several nonlinear SVM regression formulations using different loss functions in (14). In contrast to a popular opinion that the loss function in (14) should be tailored to particular noise density (Smola and Sholkopf, 1998; Hastie et al, 2001) we expect an overall superiority of  EMBED Equation.2 insensitive loss for sparse high-dimensional problems. Further, our results help to understand better relative importance of  EMBED Equation.2 insensitive loss vs regularization term in SVM formulation (14). Whereas both factors are important for nonlinear SVM regression, our experience suggests that the choice of  EMBED Equation.2 insensitive loss parameter is much more important for linear regression. Even for nonlinear SVM, the quality of SVM solutions is typically much more sensitive to proper setting of  EMBED Equation.2 parameter rather than regularization parameter (Cherkassky and Ma, 2002). Finally, we have shown that SVM approach with  EMBED Equation.2 insensitive loss can be successfully used for symmetric bimodal noise distributions. This finding may be of practical importance in applications where bimodal noise models preclude application of standard OLS and LM methods. Conceptually, Vapniks loss function with insensitive zone can be viewed as a new approach for providing regularization for ill-posed estimation problems with sparse data. Our empirical comparisons indicate that such an approach can be quite useful for sparse high-dimensional data sets, provided that one can control the degree of regularization (via the choice of  EMBED Equation.2 -value). This view also leads to novel interpretation of SVM formulation (14) where regularization can be performed in two ways: Standard regularization via the choice of parameter C; Regularization via the choice of  EMBED Equation.2 -value. In this paper we only considered the second approach (by intentionally setting the value of C very high). This approach works well for linear regression settings. However, for nonlinear SVM (with kernels) we may need to explore more advanced strategies for joint regularization (optimization) using both parameters C and  EMBED Equation.2 . This opens new directions for SVM research. Acknowledgements_______________________________________________________ The authors would like to thank Dr. Vladimir Vapnik for several stimulating discussions. We also acknowledge several constructive suggestions by Referee 1 that helped to improve technical quality of the paper. This work was supported, in part, by NSF grant ECS-0099906. References _____________________________________________________________ Cherkassky, V., & Ma, Y.(2002). Practical Selection of SVM Parameters and Noise Estimation for SVM Regression, Neurocomputing (under review). Cherkassky, V., & Ma, Y.(2002). Selection of Meta-Parameters for Support Vector Regression, Proc. ICANN-2002 (to appear). Cherkassky, V.(2002). Model Complexity Control and Statistical Learning Theory, Natural Computing, 1, Kluwer, 109-133 Cherkassky, V., & Mulier, F.(1998). Learning from Data: Concepts, Theory and Methods, Wiley. Drucker, H., Burges, C., Kaufman, L., Smola, A., & Vapnik, V.(1997). Support Vector Regression Machines, Neural Information Processing Systems 9, Eds. M. Moser, J. Jordan and T. Petsche, 155-161, MIT Press. Hastie, T., Tibshirani, R., & Friedman, J.(2001). The Elements of Statistical Learning: Data Mining, Inference and Prediction, Springer. Huber, P., (1964). Robust estimation of a location parameter, Ann. Math. Stat., 35, 73-101. Scholkopf, B., Burges, C., & Smola, A.(1999). Advances in Kernel Methods: Support Vector Machine, MIT Press. Smola, A., & Schlkopf, B.(1998). A Tutorial on Support Vector Regression, NeuroCOLT Technical Report NC-TR-98-030, Royal Holloway College, University of London, UK. Suykens, J., Vandewalle, J., & De Moor, B.(2001). Optimal control by least squares support vector machines, Neural Networks 14, Pergamon, 23-35. Vapnik, V. (1995). The Nature of Statistical Learning Theory. New York: Springer Vapnik, V. (1998). Statistical Learning Theory, New York: Wiley. Werbos, P., & Titus, J., (1978). An empirical test of new forecasting methods derived from a theory of intelligence: the prediction of conflict in Latin America, IEEE Trans. Systems, Man & Cybernetics. PAGE  PAGE 1 C}" op       ]nWXhijk@NLM]^_`O jv U j U jU jUj @ UV jU6 j5U j5Uj @ 5UV j5U566CJOJQJ5CJOJQJ5ABC}"#,  {O $hx $ & Fx  & Fx$x$x $x$xBC}"#,  {OEv$!^!!"b##<$0%M%%%z&&I))t*+4,-|//H000"3>3h33358<?AAaCCCGKBQCQQRRDSS                     JIJ]^_`}~EFYZ[\|u jYEHUj;/@ UVnH j)EHUj],@ PJUV jEHUjlI@ UVnH jEHUj*@ PJUV jZEHUjу*@ PJUV jEHUjY}@@ PJUVmH jU j_ EHUj|@@ PJUVmHEH jEHU,Ev$!^!!"b##<$0%M%%%z&&I))t*+4, $px $hx $px4$ $x$x$x  " # $ % I J ] ^ _ ` $!%!8!9!:!;!!!!!!!<"="P"Q"R"S"T"U"h"i"j"k"##$#%#߉΂ j*EHU j(EHU j/&EHUj?@@ PJUVmH j#EHUj*=@ PJUVmH j!EHUj^,@ PJUV jEHUjb,@ PJUV jEHUj^,@ PJUV jU jEHUj^,@ PJUV1%#&#'#b#c#v#w#x#y##$#$$$7$8$9$:$<$$$$$$$0%1%C%D%E%F%%%%%%%%z&{&&&&&&))')()»x j:EHUji'@ CJUVnHo(EH jEHU j7EHUjUN@ UVnH j4EHUjM@ PJUVmH j2EHUj^,@ PJUVEH j0EHUjd/@ UVnH6 jt.EHUjb,@ PJUV jU je,EHU/();)<)=)>)I)J)])^)_)`)X+`+++,,,,//////m0n0~00000000001111111111ƿwj @ UV jKUj @ UV jHEHUjM@ PJUVmH jGUj @ UV jWDEHUj,Q@ PJUVmH jAEHUjM@ PJUVmH6 jk?EHUj(M@ PJUVmH jU jn=EHUjM@ PJUVmH.4,-|//H000"3>3h33358<?AAaCC$x$hx $xx$nx $px$x$x$x$x4$ $hx111H2I2Y2Z2[2\2]2^2n2o2p2q2222222"3#333435363S3T3d3e3f3g333333344,4-4.4/40414A4 j^EHUjD@ UVnH j \EHUj,M@ PJUVmH jXEHUjM@ PJUVmH j+VEHUjTD@ UVnH jBTUj @ UV j REHUj @ UV jPEHUj @ UV jU jMEHU/A4B4C4D4J4K4[4\4]4^4c5d5v5w5x5y5666666 9999 9!9999999::::::;;;;;;?<@<P<Q<R<S<<<<<<<==== jrU jpU jnU jlU jjU jiU jgUj @ UV j8eEHUjo@ UVnH j4cEHUjhI@ UVnH jU jaEHUjD@ UVnH====??? ?!?"?@@+@,@-@.@AA!A"A#A$AAAAAAAAAAAAA4B5BGBHBIBJBCC,C-C.C/CICJC\C]Czjx"@ UVnHo( jEHUj!@ UVnHo( jʀEHUj!@ UVnHo( j~EHUj&)@ CJUVnHo( j|EHUj(@ CJUVnHo( j6zU jMxUj @ UV jdvUj @ UV jU j{tU0]C^C_CnCoCCCCCCCCCCCCCCCCCNDODaDbDcDdDDDDDDDDDEEE EAEBERESETEUEEEEEEE~FFFFˏ jU jU jEHU jEHUjE@ PJUVmH jEHUjE@ PJUVmH j4Uj(uY? UV jEHUjE@ PJUVmH jUj @ UV jU jEHU6CCGKBQCQQRRDSSSSScYLZZ%[P[[\$\$x$x4$x$ & Fx$x $ & Fx $hx$x$xFFFHHHHHHIIJJJJJJJJJJ^L_LoLpLqLrLLLLLLLMMYNZNjNkNlNmNNNQQ)Q*Q+Q,QCQQ[TgTUUWWWWWWBXCXSXTXUXVXXXY Yִֹ jdU j{U5 jU jU6 jU jםUj @ UV jU jU jUj @ UV jU jZUFSSSScYLZZ%[P[[\$\%\&\6\7\;\S\T\X\p\q\u\\\\\\\]`VbhkDlllllllllllllllllllllmmmmmm!m(m/m0m1mo@otoopzv#z$znz}~"Y                  P Y Y YcYpYYYYYYYYYYYYYYZZZZZ3Z4ZFZGZHZIZLZ]ZZZZZZZP[Q[c[d[e[f[h[i[{[|u j2EHUjy@ CJPJUVmH j˳EHUj y@ CJPJUVmH j{EHUj*@ CJUVnHo( jEHUj*@ CJUVnHo( jEHUj*@ CJUVnHo( j6EHUj*@ CJUVnHo(6 jU jMU-{[|[}[~[\$\%\;\<\O\P\Q\R\X\Y\l\m\n\o\u\v\\\\\\\\\\\\\]]]^^_C_D_V_W_޲ޛބ{yyyyojy@&@ UVnH6jCJEHUjI@ CJUVnHo(j,CJEHUj@ CJUVnHo(jlCJEHUj@ CJUVnHo(jCJEHUj'@ CJUVnHo( jCJUCJ5CJ jU jŸEHUjwz@ CJPJUVmH*$\%\&\6\7\;\S\T\X\p\q\u\\\\\\\]`tttt $hx$x$x$2$$Tl t0,"$x$$xW_X_Y_Vbkbccccccccccccgggggghh0h1h2h3hjhkh}h~hhhhhhhhhhhkkl l3l4l5l6lgmhmzm{m|m}mnn-n.n jEHU jEHUj*@ CJUVnHo(5 jEHU jEHU j0EHU jdEHU jEHU jEHUjy@&@ UVnH6 jU jEHU<`VbhkDllllllllllllx:$$l4F@,"`c`$x$x$x$x $hxllllllllllmYN$$lrj U@,"c$x$P$$l4rj U@,"  mmmmmm!m(m/m0m1mo@otoopzv $ & Fx $hx$xN$$lrj U@,"c$x$.n/n0nnnnnnnoooooo/p0pBpCpDpEp$znz{{{{{{||||||~~/0BCDErsAG]^pqrs ȅɅۅ܅݅ jEHU jEHU j9EHU jmEHU6 jEHU jEHU5 j EHU j=EHU jqEHUjy@&@ UVnH jU jEHUAzv#z$znz}~"YftNȑ>kPcE$xdx $ & Fx$x $hx݅ޅqrɈʈ܈݈ވ߈#$6789rz{َڎfst,:.4b+ұCJ5 j1EHU jeEHU jEHU jEHU6 jEHU j5EHU jiEHU jEHUjy@&@ UVnH jUFftNȑ>kPcEP[cdfghi 2G~ϕޕ0Ws(OPQWXY[\bcdef0JmH0J j0JU6PYZ[fghijklmnopqrstuvwxyz{|hh&`#$$x|}~$x 1h/ =!"#$%Dd9J  c &A?2K?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklnopqrstwz|{}~Root Entry4 F@_Gd_yData mWordDocument3"ObjectPool6$@__Gd__1075186869F!a_!a_Ole CompObjeObjInfo !"%(+.147<ADGJKNQRUXY\adefilmnquvy~ FMicrosoft Equation 3.0 Equation.2 Equation.39q `AJ e FMicrosoft Equation 3.0 DS Equation Equation.39qEquation Native <_1077968088B F____Ole CompObj fObjInfoOle10Native  ,Equation Native  L_1077968217F____( y=rx()+es0 y=gx()+ FMicrosoft Equation 3.0 DS Equation Equation.39qOle  CompObj fObjInfoEquation Native )s H  FMicrosoft Equation 3.0 DS Equation Equation.39qHuI$I x FMicrosoft Equation 3.0 DS Eq_1076528081)F____Ole CompObjfObjInfo>Equation Native ,_1076528156F__D__Ole CompObjfuation Equation.39q( y=rx()+eHȆI I y FMicrosoft Equation 3.0 DS Equation Equation.39qObjInfoOle10Native,Equation Native ,_1078553758)!FD__` `_Ole CompObj "fObjInfo#Equation Native  i (x i ,y i ),(i=1,...,n) FMicrosoft Equation 3.0 DS Equation Equation.39qP(ItI p(x,y)=_1076649380&F` `_`w`_Ole #CompObj%'$fObjInfo(&Equation Native 'l_10768372928+F`w`_`w`_Ole )CompObj*,*fp(x)p(y|x) FMicrosoft Equation 3.0 DS Equation Equation.39qQ<t g(x)=y  +" p(y|x)dyObjInfo-,Equation Native -m_10766496080F`_`_Ole /CompObj/10fObjInfo22Equation Native 3L_1076649665.5F,`_,`_ FMicrosoft Equation 3.0 DS Equation Equation.39q0`I,I f(x, 0 ) FMicrosoft Equation 3.0 DS Equation Equation.39qOle 5CompObj466fObjInfo78Equation Native 9@$|#I,)I f(x,)x FMicrosoft Equation 3.0 DS Equation Equation.39q(ItI "_1076650659=L:F,`_,`_Ole :CompObj9;;fObjInfo<=Equation Native >4_10766496833?F '`_P`_Ole ?CompObj>@@f FMicrosoft Equation 3.0 DS Equation Equation.39q8XI$I L(y,f(x,))X FMicrosoft Equation 3.0 DS Equation Equation.39qObjInfoABEquation Native CT_1077800746QGDFP`_j`_Ole ECompObjCEFfObjInfoFHEquation Native I_1077952509IFj`_,`_s$ R()=L(y,f(x,))p(x,y  +" )dxdy FMicrosoft Equation 3.0 DS Equation Equation.39qOle LCompObjHJMfObjInfoKOEquation Native PsnP! L(y,f(x,))=(y"f(x,)) 2 FMicrosoft Equation 3.0 DS Equation Equation.39qLII R()=(_1076650739NF`T`_@`_Ole SCompObjMOTfObjInfoPVEquation Native W_1076847835SF@`_@`_Ole ZCompObjRT[fy"f(x,)) 2 p(x,y  +" )dxdy FMicrosoft Equation 3.0 DS Equation Equation.39q0 g(x)ObjInfoU]Equation Native ^5_1078823956XF`_`_Ole _CompObjWY`fObjInfoZbEquation Native c_1078895701ku]F`_`_ FMicrosoft Equation 3.0 DS Equation Equation.39qKx&ܕ R emp ()=1nL(y i ,f(x i ,)) i=1n "Ole gCompObj\^hfObjInfo_jEquation Native k FMicrosoft Equation 3.0 DS Equation Equation.39q@ R emp ()=1n(y i "f(x i ,)) 2i=1n "_1085220713bF P`_ `_Ole oCompObjadpfObjInfor FMicrosoft Equation 3.0 DS Equation Equation.39q( y=rx()+e18 fx,()= 0 + 1 x 1 +....+ d x dOle10Nativeces,Equation Native t_1078826270p[hF `_ `_Ole wCompObjgixfObjInfojzEquation Native {5_1078826536mF `_ `_ FMicrosoft Equation 3.0 DS Equation Equation.39qK+ p() FMicrosoft Equation 3.0 DS Equation Equation.39qOle |CompObjln}fObjInfooEquation Native Ku+ L(y,f(x,))="logp(y"f(x,)) FMicrosoft Equation 3.0 DS Equation Equation.39qKa9  L(y,f(x_1078824069VrF`_<a_Ole CompObjqsfObjInfotEquation Native }_1079083308wF!a_!a_Ole CompObjvxf,))=|y"f(x,)| FMicrosoft Equation 3.0 DS Equation Equation.39qp p()=0.25"3<<"10.251<<30otherwiObjInfoyEquation Native _1078824095|F!a_!a_Ole se{ FMicrosoft Equation 3.0 DS Equation Equation.39q% y .   "&!@ & MathTypeCompObj{~fObjInfoOlePres000}Equation Native Symbolw@ LSwUSw0-2 "] y2 ] y2 ] y2 j-y2  -y2 y2 -y2 ! =yTimes New RomanLSwUSw0- 2 = otherwise2 xy2 Afy2  yy2 xy2 #fy2 yy2 hif2 kxf2  ff2 zyf2 FLfSymbolw@ LSwUSw0-2 5ef2 wf2 !ef2 wf2 wf Symbolw@' ,LSwUSw0-2 efTimes New RomanLSwUSw0-2 |f2 )f2 Q,f2 (f2  |f2 |f2 )f2 3,f2 (f2 |f2 0f2 ))2 ,)2 ()2 .,)2 () & "System 0-K$ L  (y,f(x,))=0if|y"f(x,)|d"|y"f(x,)|"otherwise{_1075186861F b_ b_Ole CompObjeObjInfo FMicrosoft Equation 3.0 Equation.2 Equation.39q `AJ e FMicrosoft Equation 3.0 DS Equation Equation.39qEquation Native <_1075186860F@8Ta_@8Ta_Ole CompObjfObjInfoOlePres000"Equation Native F_1075186859F@8Ta_(na_F{  .  @& & MathTypeP Times New RomanLSwUSw0-2 2yTimes New RomanLSwUSw0-2 r||2 "||Symbolw@ LSwUSw0-2 w| & "System 0-*(Խ |||| 2 FMicrosoft Equation 3.0 DS Equation Equation.39qb 1 Ole CompObjfObjInfoOlePres000.  `&@ & MathType` Times New RomanLSwUSw0-2 *yTimes New RomanLSwUSw0-2 o,y Times New RomanLSwUSw0-2 iy2 iySymbolw@ LSwUSw0-2 xy2 xy & "System 0-5(  i , i* FMicrosoft Equation 3.0 DS Equation Equation.39qEquation Native Q_1075186858F(na_ua_Ole CompObjfObjInfoOlePres0004Equation Native E_1075186857F5b_5b_4   .  @& & MathTypePTimes New RomanLSwUSw0-2 `-ny2 `.iyTimes New RomanLSwUSw0- 2 `,...2 `1.Symbolw@ LSwUSw0-2 ` =. & "System 0-)hoD i=1,...n FMicrosoft Equation 3.0 Equation.2 Equation.39q `AJ eOle CompObjeObjInfoEquation Native <_1078230868 Fua_a_Ole CompObjfObjInfo FMicrosoft Equation 3.0 DS Equation Equation.39q a .  @`&  & MathType- @ (  OlePres000Equation Native _1078824180zF$a_@a_Ole Symbolw@$ bLSwUSw0-2 !y Symbolw@m LSwUSw0-2 =ySymbolw@$ cLSwUSw0-2  +y2 +y Times New RomanLSwUSw0-2 ny2 \iy2  iy2 2 iyTimes New RomanLSwUSw0-2 ny2 Cy Times New RomanLSwUSw0-2  1y2 *y2 2yTimes New RomanLSwUSw0-2 )y2  (y2 ||2 ||2 Z2|2 T1|Symbolw@m LSwUSw0-2  x|2 X x|2 ^w| & "System 0-      !"#$'*+,-./0123456789:;<=?@ADGHIJKLMNOPQRSUX[\]^_`abcdefhknopqrstuvwxyz{|}~؞ 12|||| 2 +C( i + i* ) i=1n " FMicrosoft Equation 3.0 DS Equation Equation.39q  CompObjfObjInfoOlePres000Equation Native W.  @`&  & MathTypeSymbolw@ yLSwUSw0-2 :y2 :y2 :y2 B:y2 :y2 @=y2 @y2 n +y2 ! y2 B-y2 t +y2 ' y2 -yTimes New RomanLSwUSw0-2 @ ny2 @iy2 yy2 xy2 Bfy2 vxy2 fy2 $yy Times New RomanLSwUSw0-2 xiy2 aiy2 `S iy2 `Ciy2 `Xiy2 Y iy2 *iy2 iyTimes New RomanLSwUSw0- 2 @ ,...,2 @ 1.2 @,.2 @0.2 @*,.2 r).2 ,.2 (.2 D).2 ,.2 (. Times New RomanLSwUSw0-2 *.2 ~ *.Symbolw@M LSwUSw0-2 @x.2 @x.2 z x.2 K e.2 Fw.2  x.2 Q e.2 w. & "System 0-K;lz y i "f(x i ,)d"+ i* f(x i ,)"y i d"+ i  i , i* e"0,i=1,...,n{ FMicrosoft Equation 3.0 DS Equation Equation.39q_1078824236F@a_@a_Ole %CompObj&fObjInfo((   .  ``&   & MathTypeTimes New RomanLSwUSw0-2 )y2 L,y2  (y2  )y2 (y2 )y2 H(y Times New RomanLSwUSw0-2 " *y2 OlePres000)2Equation Native >_1078230950F@a_ Ha_Ole B1yTimes New RomanLSwUSw0-2 xy2 xy2 W Ky2 xy2 fy Times New RomanLSwUSw0-2 iy2  iy2 iy2 ny2 iy`Times New RomanLSwUSw0-2 7QSVSymbolw@ ILSwUSw0-2  aV2 aVSymbolw@ LSwUSw0-2 -V2 =VSymbolw@ JLSwUSw0-2 V Symbolw@ LSwUSw0-2 G=V & "System 0-K@ܕ f(x)=( i=1n SV "  i " i* )<x i ,x> FMicrosoft Equation 3.0 DS Equation Equation.39qF>  .  @&CompObjCfObjInfoEOlePres000FfEquation Native TL & MathType-Times New RomanLSwUSw0-2 ny2 lCy Times New RomanLSwUSw0-2 yiySymbolw@ jLSwUSw0-2 `y2 `Jy Times New RomanLSwUSw0-2 *yTimes New RomanLSwUSw0-2 `40ySymbolw@ LSwUSw0-2 `Yay & "System 0-0P 0d" i* d"C FMicrosoft Equation 3.0 DS Eq_1078230962F Ha_ Ha_Ole VCompObjWfObjInfoYuation Equation.39qF s .  @& & MathType-Times New RomanLSwUSw0-2 ny2 lCy Times New RomanLSwUSw0-OlePres000ZEquation Native gH_1078552770F Ha_ b_Ole i2 qiySymbolw@ LSwUSw0-2 `y2 `FySymbolw@q LSwUSw0-2 `RayTimes New RomanLSwUSw0-2 `40y & "System 0-,< 0d" i d"C FMicrosoft Equation 3.0 DS Equation Equation.39q(   .  ``&   & MathTypeCompObjjfObjInfolOlePres000m2Equation Native 9Times New RomanLSwUSw0-2 )y2 L,y2  (y2  )y2 (y2 )y2 H(y Times New RomanLSwUSw0-2 " *y2 1yTimes New RomanLSwUSw0-2 xy2 xy2 W Ky2 xy2 fy Times New RomanLSwUSw0-2 iy2  iy2 iy2 ny2 iy`Times New RomanLSwUSw0-2 7QSVSymbolw@ ILSwUSw0-2  aV2 aVSymbolw@ LSwUSw0-2 -V2 =VSymbolw@ JLSwUSw0-2 V Symbolw@ LSwUSw0-2 G=V & "System 0- <","> FMicrosoft Equation 3.0 DS Equation Equation.39q4 n SV_1074667375Fb_k.b_Ole CompObjfObjInfoEquation Native :_1085221004F@\Hb_@\Hb_Ole CompObjf FMicrosoft Equation 3.0 DS Equation Equation.39q1 C=max(|y+3 y |,|y"3 y |)ObjInfoEquation Native _1085221158F@\Hb_ib_Ole CompObjfObjInfoEquation Native 0_1074667937F qb_Xb_ FMicrosoft Equation 3.0 DS Equation Equation.39q1 k y FMicrosoft Equation 3.0 DS Equation Equation.39q     F "!#%$&(')+*,.-/1023456879:;<=?@ABDCGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~? 2.YϧaҺ' V `!YϧaҺ' V:`!Hxcdd`` @c112BYL%bpuztWI;gr/4I%ʼDd B   S A?  2&Oլn%(-U`!Oլn%(-`!x5O @ ZAOŃ7EԳCAU/CD6vkCB2-g@W*c0lvAড়ĞI9 @B\ߛp0w&'.d䂦*.pʃ%VLLJ% H X, SyDd hB  S A ? 2yWFEh)}U`!MWFEh)}X@|xcdd``ad``beV dX,XĐ Ɂi A?d-g@P5< %! `fjvF+B2sSRs&\ F\ /X@5` M6}VpM Ma`^fr4a"% ~5F&4.hpc8P 0#I)$5bAZIDd @B  S A ? 2h ზE!/D`!< ზE!/8  xڅOjQ ƿd;  R ETaBp<{i^fAxI^_BVFli8ʹ׾"L,cGxHћcJp!ym:)S܈5K&~R:0NzJۖ_#}Pb|Jwn~`zf]ںo]Z4lFrz!UҺ?P9LKDd B  S A ? 22S٤w馄9W60 `!S٤w馄9W60` dHxmP; P_0XX* 鬬=J`3x{vEv>B " LX:[X"e$QCUezLٴo15li˨!|c]L,hn6a0͇ztu4tmwioHLJsC{8Y:G7 uI=}ugbnv7[%ROs/E2Dd <@B  S A ? 2 Sk͗QA\!`!T Sk͗QAN` "xڅQMJa }ɴA\ p"h+0ΕkMzO"n$3E?%yK>B`C_UUĘ"YY6,rTT7yk1__oweȔ@xSL,jZ8o{c\~5!x%Szwq4 ϣ$ƃMt$/pi"NJ>pNE/?Cy>S`4ރd*y^Aމ\wSp ZiQӍk?G /·wP+At4\uYgu.RܑDd 3 B  S A ? 2@HjY#`!@HjY"  @CxJ@g6WZmZD=b/ ACZ|GP"xQQ"w6Xv̰AH(9`bA"A.-u@Gc, Kb9;<jW&%R2+5YX3B^BfB7*NrHFmĶ$DX#.J^F%z(u\Iq,}a^<ٷwSː>OwBy/ j+i|z-~{u:u2~eU?kRfw//w5ڿ+ ]YvۊzD_-ٗKX&Mq.fˬW*_5)b﨤pDd h hB  S A? 2xSNOUH9Ls&`!xSNOUH9L@@0J||xcdd`` @c112BYL%bpu8׀pyQ籢WaAUƄ&G(@ Ma(wgN1eRQ+j&JÏ^p{ֱ@XCRgHE?.ΏcFĈ*` 9k5E?o \1b;R321)Wx ], 0+Lz1;Dd @B  S A ? 2h ზE!/D(`!< ზE!/8  xڅOjQ ƿd;  R ETaBp<{i^fAxI^_BVFli8ʹ׾"L,cGxHћcJp!ym:)S܈5K&~R:0NzJۖ_#}Pb|Jwn~`zf]ںo]Z4lFrz!UҺ?P9LKDd B  S A ? 22S٤w馄9W60*`!S٤w馄9W60` dHxmP; P_0XX* 鬬=J`3x{vEv>B " LX:[X"e$QCUezLٴo15li˨!|c]L,hn6a0͇ztu4tmwioHLJsC{8Y:G7 uI=}ugbnv7[%ROs/E2Dd hB  S A ? 2yWFEh)}U,`!MWFEh)}X@|xcdd``ad``beV dX,XĐ Ɂi A?d-g@P5< %! `fjvF+B2sSRs&\ F\ /X@5` M6}VpM Ma`^fr4a"% ~5F&4.hpc8P 0#I)$5bAZIDd B  S A? 2/ʕy.`!/ʕy!@CxڍJQ9~)D+i h-Z/`8)),!zW"2m.{r߽q3!A # Dd۶\%R tq%ȢLJVIX@l(I~s\1[X #K ARo͖)wQ,jDCckj\`6_3F~.0:8>!J=^,rs_I?oi{zTBiSzS<Йk >N.7Oͯ!??6:I',sT_ҐoO&^6Nsn)ai^myg"ԏ8wߠÚWhϑF5tx8$Z)pr{{H/2Dd @N  s *A? ?2Ss ۣ9iM"/?1`!'s ۣ9iM"@  xcdd`` @c112BYL%bpuXGBD1 (%jX%d *FƏ0a#;;ss 8@WXq-dL[ =ϓV?kӾڀQsIXݑhȐ\X'b"d=Ҵ\CLx_P)q+BVx|7ܡQɌV0*QnҙŒXL&KC1bh/uVS-bT^Bw:˝=U>e+_KVkjZ3q5k{^ fpZc "; c@x9*&`ʉ 2'\UAR>ޗ E%Y4<BWe _@SD3IBgP8 d=16R%;jʕhI.ؙqsy]RROcWDd eN  s *A? ?2@ ;TE٢=8`! ;TE٢=(@PtxڥT/A~v*-u(qhDEApPmGH$84qrqw!8ּRf7o2  F/BTu]eXB5oQo0 ]byt[>j|D`rnsaY3*{QΠG-ͤv&q;HQGK!W%XޱWG!#1gycUL4. D#wK?Gz(m"ư-;RnH;TO:3h)NNZs+70$8znDgaӌo3֚sb,EfG#m/(c@xڪ T^ʩ<UgZoX*a[T~LJ-}\WV:ѻBɯO=OoO3I\O[)C)w,+\ 'll!C3t,[&ƿDd 0 hh r s *A2? ?3"`?2oJ48\} :`!oJ48\} ʼ@:|xcdd``cb``baV d,FYzP1n:&a! KA?H1Z πqC0&dT20$1 @2@penR~CP(OP f3``@6ctW󌹸`7L@2WBj Q! ~ Ay 1j5B )0 y3܊J(*Of/ fT['޻w33 ?SbP6#t$J6J4ۍ1;3=UHʂ?hm')Սf7ٿhGJ'|(܉[; rNw~7@" (9i|&R~k}Z`^dr̯4mQd>ϫ~b)"w/ <㝾x9nVM{v%o;-Z%5$5_uB~dOP{x~2sg:icʹ ӊ.7Ţ3_U\P;>(C8sKy瘗58wyfox16x1^^"_}Z{je+2fߣVٵL%”b_Ƒ"Ui)CЂDN4z!{%ͅDd9J   c &A?2K6y⥆){7vOooZa_|2$N,wT#Ggv@ya{L{M n)h4=5Av[Dd  8B * S A? *2*@2jli1sH4P\`!@2jli1sH4B`RPx=,A߼;AG ՝XqCMHFaD#jN$UH5ofwrwf75 ;DF/F= s-Hջsb۵uj@8$ aH\j0^L[7e KAQܱ^ܐsnR &u] Bx6nhex*Jܯ2DܡyG<`o?zL>u/xIq;?uU]/sVna^>pw"5cf.>M/Ѭ!ŚNU1_3eqK:fUs&@ܢyׇZ/  RY^d߻?aY(`!3v%Ղ뎫uWyt^"mINz|Pl򏤼1|; )a-Y"zw}UOB݃:Dd `|$N + s *A? ?+2vB`R{ۋ΄H<'t_`!lvB`R{ۋ΄H<'`X 0:xcdd``ndd``baV d,FYzP1n:&v! KA?H1ā깡jx|K2B* Rvfjv,L ! ~ Ay >_C΀H9@ڈk,Vs%<?e`eȬDw'(P 27)?(%˝s-fX\ {2Bg_~& X⏉-\X8~L \NQ]=4Ĥ\Y\ t0.?0~.Dd `|8N , s *A? ?,2WO߾6%hJa`!`WO߾6%`X 0.xcdd``ed``baV d,FYzP1n:&v! KA?H1ā깡jx|K2B* Rvfjv,L ! ~ Ay >_C΀H9@ڈk"Vs%<?e`fȬDw'(P 27)?(%˝saX\ {2BXƒ͞~..vܿ1 "I gt0TTrAc ]=Ĥ\Y\ te:n'Dd X8N - s *A ? ?-2b @$ըEtgF>xc`!6 @$ըEtgFxcdd``~ @c112BYL%bpu @c112BYL%bpu% #RpeqIj.n P>Dd9J   c &A? 2K$d@9`,&FF(`Ti!! KA?H1Z @IznĒʂT/&`b]F"L1 l 0e[+7S? `3B2sSRs|1XFF\A^фT<( сgspA@nMШaQ?̍$ LUiHEe"_ᣇ#a &y98 ;c dopenR~CPtw1n9'%%CTbM䂦\.pꆘ q;+KRsA<.hj9͝SDd |oh t s *A4? ?3"`?;2D'bVz `!'bVzf``!0xcdd``$d@9`,&FF(Ole CompObjfObjInfoEquation Native 6HI  y FMicrosoft Equation 3.0 DS Equation Equation.39q3t 'y " FMicrosoft Equation 3.0 DS Eq_1074667917FXb_Xb_Ole CompObjfObjInfoEquation Native 8_1074668152FXb_b_Ole CompObjfuation Equation.39qp 3 y FMicrosoft Equation 3.0 DS Equation Equation.39qkO  (,n)=ObjInfoEquation Native :_1078297275Fb_b_Ole CompObjfObjInfoEquation Native k_1062827304F`Db_`Db_ lnnn  FMicrosoft Equation 2.0 DS Equation Equation.29q // s FMicrosoft Equation 3.0 DS EqOle CompObjfObjInfoEquation Native <_1078297569Fb_b_Ole CompObjfObjInfouation Equation.39qk -  FMicrosoft Equation 3.0 DS Equation Equation.39qk =3Equation Native )_1078297531F{b_{b_Ole CompObjfObjInfoEquation Native 1_1085221526F b_c_Ole  FMicrosoft Equation 3.0 DS Equation Equation.39q1e؞ (x i ,y i ),(i=1,...n)CompObjfObjInfoEquation Native _1085221541Fc_"c_Ole CompObjfObjInfoEquation Native ) FMicrosoft Equation 3.0 DS Equation Equation.39q1  x FMicrosoft Equation 3.0 DS Equation Equation.39q_1085221569F0*c_:c_Ole CompObjfObjInfoEquation Native )_1085221554FBc_Bc_Ole CompObjf1 kb y FMicrosoft Equation 3.0 DS Equation Equation.39q1)b y=g(x)+ObjInfoEquation Native E_1084782880 FBc_,dc_Ole CompObj fObjInfo Equation Native _1084783036 F,dc_`tc_ FMicrosoft Equation 3.0 DS Equation Equation.39q;~ g(x)=12x 1 +7x 2 ,x"[0,1] 2 FMicrosoft Equation 3.0 DS EqOle CompObj fObjInfoEquation Native uation Equation.39q; g(x)=4x 1 +4x 2 +3x 3 +3x 4 +2x 5 +x 6 """+x 20 FMicrosoft Equation 3.0 DS Eq_1084783223F`tc_c_Ole CompObjfObjInfouation Equation.39q;6=l x"[0,1] 20 FMicrosoft Equation 3.0 DS Equation Equation.39qEquation Native R_1084613927f`Fc_டc_Ole CompObjfObjInfoEquation Native _1084614314FPc_Pc_Ole 10 g(x)=4x 1 +4x 2 +0.01x 3 +...+0.01x 20 FMicrosoft Equation 3.0 DS Equation Equation.39qCompObjfObjInfoEquation Native _1084614334$!FPc_Pc_18 g(x)=4x 1 +4x 2 +3x 3 +3x 4 +2x 5 +0.01x 6 ...+0.01x 20 FMicrosoft Equation 3.0 DS Equation Equation.39qOle CompObj "fObjInfo#Equation Native V1:H[ g(x)=4x 1 +4x 2 +3x 3 +3x 4 +2x 5 +x 6 +...+x 10 +0.01x 11 ...+0.01x 20_1084614473.&FPc_c_Ole CompObj%'fObjInfo(  !"#$& FMicrosoft Equation 3.0 DS Equation Equation.39q1:8L g(x)=4x 1 +4x 2 +3x 3 +3x 4 +2x 5 +x 6 +...+x 15 +0.01x 16 ...+0.01Equation Native V_1076248697+F dd_ dd_Ole  CompObj*, fx 20 FMicrosoft Equation 3.0 DS Equation Equation.39q p  FMicrosoft Equation 3.0 DS EqObjInfo- Equation Native  )_10846972460F`c_sc_Ole CompObj/1fObjInfo2Equation Native l1Table1uation Equation.39q1Pp  g(x)=12x 1 +7x 2Oh+'00 HT p |  6Comparison of SVM Regression with Least `T))QRcgbR 6 znĒʂT/&`b]F"L `0ro.dabM-VK-WMc Z Č`uP3vq`cTVrA]v5D5v0o؟LLJ% ԕ H`v0@5QDd @|9N : s *A"? ?<2U7W.İ7T1`!)7W.İ7T` 0xcdd``> @c112BYL%bpu% #RpeqIj.n .M>HDd oN ; s *A#? ?=2VTЉ ȥ2`!*TЉ ȥ`!xcdd``> @c112BYL%bpu2a`*^4J=`!5`*^4J`@C0xcdd`` @c112BYL%bpu S A%? "`@2q3FJ3JNSM`!q3FJ3JNSMN `HPxR=KA$A-DR">s-6Fllr&$˽7fH  I!! P2cbv`uI0 CV@l@:X":C>BDמYM uS:vT'&J(Mk6e6\[ [|c+amu+=qҭPͶYZ~{d]zľ=l"WS|)!H^F̍J7.O}P$d-?G&㏘aȷſG&>7v,핅b7j \%[+7$7#U~Dd 4uB ? S A&? A2,ֵQ6/]'yx`!ֵQ6/]'y4@Yxcdd`` @c112BYL%bpu .| z- `+KRsA<.,BF0( J-fDd 4vB @ S A'? B2+"#7 5kF:`!"#7 5kF:@`!xEN1 @$j`H/*, B:+k?I6Vqoz3s7Kpх9_D%".BрڥVM3Dd 04uB B S A(? D2fh&Y'HPB`!:h&Y'HPkHxuP=KA}3~dvh-wFM3Dd9J  c &A?2K .| z- `+KRsA<.,BF0( J-fDd 4uB E S A&? G2,ֵQ6/]'y`!ֵQ6/]'y4@Yxcdd`` @c112BYL%bpu .| z- `+KRsA<.,BF0( J-fDd9J  c &A?2K ;kwEĵ.\ ƹi(  w枹g V_A+& 2,tвY+弲\Rzq+Ș OnUxU f\D闶+-5qԫEq7nvp~ NThjոeJJ5q ~tA0a8x#7U'\s gӯߦ:vﴏVQۚ5$nu1'GZ{3 J>.}iޕook"!¤׏bFU21K !Dd Lh w s *A7? ?3"`?S2*F|X/a֑Δſ`!F|X/a֑Δſ:@@xMN P =U 'ঈS?@~@UAA#$XspnN@цyEA?rF>5Zs'º:ZH7a]#EKA-8U"S'_zb:4:zmn0? nK$C4268%vb%:pQ <dfoAR,h .IDd Ch u s *A5? ?3"`?T2-_$ןx”y.h ֯`!_$ןx”y.h:`!xEN P = 'ঈߡ">VѡCUhtqp{\knz!7$'9! e"&qTjOW3-&]Ԉ jKdM4 Wz}`O>bMȏ.Xm1>K Zt\lx(*ܚ&a)KiK17/rPDd @wh x s *A8? ?3"`?U2VG97jp`!hVG97j4 6xcdd``Ned``baV d,FYzP1n:&B@?b u 30 UXRY7S?&lebabMa(w>< X.dR*yͼR.'1HNbЮ3bC2sSRspb#ܞK, p{A| 4{0fxTTb wܿ "8 \1b;+KRsA<.,#c3XmtgDd P he B P S A)? V27nѸ?}`!7nѸ?}R@|sxJ@gGj'EQE,_NYq}$?I!>KVrU{!+Aִ@V+6(G)9 ^~Dd XheB Q S A*? W2lH=vW;ZVP&6yv`!lH=vW;ZVP&6y@X+|xӻO@Rq >|l&WDH A`?`8:dBNG>_{r|0xXC=JHGeY7K"1 mBp! ` {S|eG >[<ठj?Z]c@:e@.)l+oᜊK>"Bv^_Dnz1Mo~ُ&9v>I)_r?g;7[XqkHv",<(;]^wnY@oV+lav=q ?AԀzlMt襤= aW߮IBV+W rBiM|JDd hfB R S A+? X2JE+T"Lia `!YJE+T"Li @~ |'xcdd``vdd``baV d,FYzP1n:&&6! KA?H1ʾ ㆪaM,,He`I? @201d++&1-b pi#/ Y@N22< X@*b `3 !?ׁ1p~0]L@(\PZ˟w1ݥs2R_ .hlqc b;+KRsA<.f8wfoDd hN S s *A,? ?Y2㮗+7 ł~Ż`!㮗+7 ł~ڤ@P%|xcdd`` @c112BYL%bpup׈(1jƸyftq ȡ|Ą 0,cQ]$1 xz~r^ɾ/y֓C>v aV4Ov (CXDŽb|cRlг5ƤtU8jFU =]CW&]|y^]U^#&Bܶq͐=asM_ԩ\;0:GKrccz2~ߧ>CݺEs¤$DWGzo.~V^E`qqbĔ] tzazu*IePE_Џm#8 yJn5#y0sn(|+ 7_ZvDd 4vN W s *A0? ?]2*nxYJDHM i`!nxYJDHM :@`!xENA}3wJ!r>\ $$rJ4*5;{BMS*1CE=Y始xJ jhX1bWAm+R0|f?p34_KaOHu |BJ!ccK u#Pv݉]̺|}.COo*]-.Dd 4vN X s *A0? ?^2*nxYJDHM 5`!nxYJDHM :@`!xENA}3wJ!r>\ $$rJ4*5;{BMS*1CE=Y始xJ jhX1bWAm+R0|f?p34_KaOHu |BJ!ccK u#Pv݉]̺|}.COo*]-.Dd 4vN Y s *A0? ?_2*nxYJDHM `!nxYJDHM :@`!xENA}3wJ!r>\ $$rJ4*5;{BMS*1CE=Y始xJ jhX1bWAm+R0|f?p34_KaOHu |BJ!ccK u#Pv݉]̺|}.COo*]-.Dd 4vN Z s *A0? ?`2*nxYJDHM `!nxYJDHM :@`!xENA}3wJ!r>\ $$rJ4*5;{BMS*1CE=Y始xJ jhX1bWAm+R0|f?p34_KaOHu |BJ!ccK u#Pv݉]̺|}.COo*]-.Dd 4vN [ s *A0? ?a2*nxYJDHM `!nxYJDHM :@`!xENA}3wJ!r>\ $$rJ4*5;{BMS*1CE=Y始xJ jhX1bWAm+R0|f?p34_KaOHu |BJ!ccK u#Pv݉]̺|}.COo*]-.Dd 4vN \ s *A0? ?b2*nxYJDHM e`!nxYJDHM :@`!xENA}3wJ!r>\ $$rJ4*5;{BMS*1CE=Y始xJ jhX1bWAm+R0|f?p34_KaOHu |BJ!ccK u#Pv݉]̺|}.COo*]-.Dd 4vN ] s *A0? ?c2*nxYJDHM 1`!nxYJDHM :@`!xENA}3wJ!r>\ $$rJ4*5;{BMS*1CE=Y始xJ jhX1bWAm+R0|f?p34_KaOHu |BJ!ccK u#Pv݉]̺|}.COo*]-.EDd TN ^ s *A1? ?d2G ŸZ䰠^`!wG ŸZ䰠^ XJExcdd``Ndd``baV d,FYzP1n:&\B@?b p10 UXRY7&,e`abM-VK-WMc Z `  ,@RF\͹@2F0b6߰b~?g40.0Y@AahW25|Mp8jDBwG'HEf6TQFTn%

\ $$rJ4*5;{BMS*1CE=Y始xJ jhX1bWAm+R0|f?p34_KaOHu |BJ!ccK u#Pv݉]̺|}.COo*]-.Dd 4vN ` s *A0? ?f2*nxYJDHM `!nxYJDHM :@`!xENA}3wJ!r>\ $$rJ4*5;{BMS*1CE=Y始xJ jhX1bWAm+R0|f?p34_KaOHu |BJ!ccK u#Pv݉]̺|}.COo*]-.Dd 4vN a s *A0? ?g2*nxYJDHM `!nxYJDHM :@`!xENA}3wJ!r>\ $$rJ4*5;{BMS*1CE=Y始xJ jhX1bWAm+R0|f?p34_KaOHu |BJ!ccK u#Pv݉]̺|}.COo*]-.Dd 4vN b s *A0? ?h2*nxYJDHM W`!nxYJDHM :@`!xENA}3wJ!r>\ $$rJ4*5;{BMS*1CE=Y始xJ jhX1bWAm+R0|f?p34_KaOHu |BJ!ccK u#Pv݉]̺|}.COo*]-.Dd 4vN c s *A0? ?i2*nxYJDHM #`!nxYJDHM :@`!xENA}3wJ!r>\ $$rJ4*5;{BMS*1CE=Y始xJ jhX1bWAm+R0|f?p34_KaOHu |BJ!ccK u#Pv݉]̺|}.COo*]-.Dd 4vN d s *A0? ?j2*nxYJDHM `!nxYJDHM :@`!xENA}3wJ!r>\ $$rJ4*5;{BMS*1CE=Y始xJ jhX1bWAm+R0|f?p34_KaOHu |BJ!ccK u#Pv݉]̺|}.COo*]-.Dd 4vN e s *A0? ?k2*nxYJDHM `!nxYJDHM :@`!xENA}3wJ!r>\ $$rJ4*5;{BMS*1CE=Y始xJ jhX1bWAm+R0|f?p34_KaOHu |BJ!ccK u#Pv݉]̺|}.COo*]-.Dd 4vN f s *A0? ?l2*nxYJDHM `!nxYJDHM :@`!xENA}3wJ!r>\ $$rJ4*5;{BMS*1CE=Y始xJ jhX1bWAm+R0|f?p34_KaOHu |BJ!ccK u#Pv݉]̺|}.COo*]-.Dd 4vN g s *A0? ?m2*nxYJDHM S`!nxYJDHM :@`!xENA}3wJ!r>\ $$rJ4*5;{BMS*1CE=Y始xJ jhX1bWAm+R0|f?p34_KaOHu |BJ!ccK u#Pv݉]̺|}.COo*]-.Dd 4vN h s *A0? ?n2*nxYJDHM `!nxYJDHM :@`!xENA}3wJ!r>\ $$rJ4*5;{BMS*1CE=Y始xJ jhX1bWAm+R0|f?p34_KaOHu |BJ!ccK u#Pv݉]̺|}.COo*]-.Dd4vN  s *A? ?2*nxYJDHM 4`!nxYJDHM :@`!xENA}3wJ!r>\ $$rJ4*5;{BMS*1CE=Y始xJ jhX1bWAm+R0|f?p34_KaOHu |BJ!ccK u#Pv݉]̺|}.COo*]-.Dd4vN  s *A? ?2*nxYJDHM 4`!nxYJDHM :@`!xENA}3wJ!r>\ $$rJ4*5;{BMS*1CE=Y始xJ jhX1bWAm+R0|f?p34_KaOHu |BJ!ccK u#Pv݉]̺|}.COo*]-.Dd4vN  s *A? ?2*nxYJDHM 4`!nxYJDHM :@`!xENA}3wJ!r>\ $$rJ4*5;{BMS*1CE=Y始xJ jhX1bWAm+R0|f?p34_KaOHu |BJ!ccK u#Pv݉]̺|}.COo*]-.Dd4vN   s *A? ?2*nxYJDHM y4`!nxYJDHM :@`!xENA}3wJ!r>\ $$rJ4*5;{BMS*1CE=Y始xJ jhX1bWAm+R0|f?p34_KaOHu |BJ!ccK u#Pv݉]̺|}.COo*]-.Dd4vN  s *A? ?2*nxYJDHM E4`!nxYJDHM :@`!xENA}3wJ!r>\ $$rJ4*5;{BMS*1CE=Y始xJ jhX1bWAm+R0|f?p34_KaOHu |BJ!ccK u#Pv݉]̺|}.COo*]-.Dd4vN  s *A? ?2*nxYJDHM 4`!nxYJDHM :@`!xENA}3wJ!r>\ $$rJ4*5;{BMS*1CE=Y始xJ jhX1bWAm+R0|f?p34_KaOHu |BJ!ccK u#Pv݉]̺|}.COo*]-.Dd 4vN o s *A0? ?u2*nxYJDHM `!nxYJDHM :@`!xENA}3wJ!r>\ $$rJ4*5;{BMS*1CE=Y始xJ jhX1bWAm+R0|f?p34_KaOHu |BJ!ccK u#Pv݉]̺|}.COo*]-.Dd4vN  s *A? ?2*nxYJDHM 4`!nxYJDHM :@`!xENA}3wJ!r>\ $$rJ4*5;{BMS*1CE=Y始xJ jhX1bWAm+R0|f?p34_KaOHu |BJ!ccK u#Pv݉]̺|}.COo*]-.Dd4vN  s *A? ?2*nxYJDHM u4`!nxYJDHM :@`!xENA}3wJ!r>\ $$rJ4*5;{BMS*1CE=Y始xJ jhX1bWAm+R0|f?p34_KaOHu |BJ!ccK u#Pv݉]̺|}.COo*]-.SummaryInformation(5DocumentSummaryInformation8 CompObj%jSquare Methodtomp Unknown User SVnknnkn Normal.dotrHoneywell UserV3neMicrosoft Word 8.0e@G@6{@LQa@P=_Oxy՜.+,D՜.+,t0 hp  DellComputerCorporation>,1 6Comparison of SVM Regression with Least Square Method Title 6> _PID_GUIDAN{B034572B-E81A-47A8-8C39-A890AC1C7B7F}  FMicrosoft Word Document MSWordDocWord.Document.89q [8@8 NormalCJ_HaJmH sH tH @@@ Heading 1$$dh@&a$5EH<A@< Default Paragraph Font:O: Normal (Web)dd[$\$&X@& Emphasis6]:B: Body Text$7$8$H$a$PJ8P"8 Body Text 2$a$5PJ8"8 Caption$dha$ 5PJ\(OA( HTML Cite6]4R4  Balloon TextCJaJNOrN author$`a$CJOJQJaJnHtHuLOL address$`a$CJOJQJaJnHtHuHOH email$`a$CJOJQJaJnHtHu>Q> Body Text 3$dh4$a$CJ6 @6 Footer 9r G$CJaJ&)@& Page Number.U@. Hyperlink >*B*ph  !"#$%&'()*+,-./0123456789:;<=>?@ABCD      A789:;<=>?@A !"#$%&'()* +!,"-#B/CC&1'2(3)4*5+6,7-8.9/:0;1<2=3>4?5@6.$B%0D  !"#$%&'()*+,-./0123456789:;<=>?@ABCDG      !"#$%&'()*+,-./0123456789:;<=>?@ABC %#()1A4=]CF Y{[W_.n݅MPRSTVWXY[]^`dfh4,C$\`lmzv|NQUZ_abceijkSO\goWhjL]_I]_}EY["$I]_$8:<PRThj$&bvx# 7 9 0!C!E!!!!z"""'%;%=%I%]%_%'((+++m,~,,,,,------H.Y.[.].n.p...."/3/5/S/d/f////0,0.000A0C0J0[0]0c1v1x1222 55 5555666777?8P8R8888999;;!;<+<-<=!=#=======4>G>I>?,?.?I?\?^?n?????????N@a@c@@@@@AAAARATAAAA~BBBDDDEFFFFF^HoHqHHHHYJjJlJM)M+MSSSBTSTUTTU UUUUUUUUVV3VFVHVVVVPWcWeWhW{W}W;XOXQXXXlXnXuXXXXXXC[V[X[______cccd0d2djd}dddddh3h5hgizi|ij-j/jjjjkkk/lBlDlwwwxxx/{B{D{r{{{]}p}r}ȁہ݁qɄ܄ބ#68zي:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: !!O2$,7F7띂^""$XÇ6 "$u81WR "$L{ & @0(  B S  ?+ 1 !&38,/)2$$$$$$''^1a122;;uuz zzz z&z-/)1.4Phi //==??OOVW~WWYYV^l^_`Ԍ֌݌JegnǍՍ׍Y[׎َ<uw֐ؐ1EWPhi cherkassky cherkassky cherkassky cherkassky cherkassky cherkassky cherkassky mayunqian cherkasskyjC:\ECE Faculty (A)\vlad\research\papers\loss fct comparisons\Regressionlossfunction_text_revised_06_10.docHoneywell User C:\b_cherkass\Regressionloss.doc3CU*O6lh$Go(O\.F0[ZAs?#ld;^`o(.^`.pLp^p`L.@ @ ^@ `.^`.L^`L.^`.^`.PLP^P`L.hh^h`o( H\H^H`\hH) \^`\hH. \^`\hH. 4\4^4`\hH)  \ ^ `\hH. | \| ^| `\hH.  \ ^ `\hH) \^`\hH.hh^h`o(-^`o(()^`o(.^`.pLp^p`L.@ @ ^@ `.^`.L^`L.^`.^`.PLP^P`L.h^`.h^`.hpLp^p`L.h@ @ ^@ `.h^`.hL^`L.h^`.h^`.hPLP^P`L.,<,^,`T}@ !20d,P5Comparison of SVM Regression with Least Square Method Unknown UserHoneywell User