回歸模型評估

Evaluation metrics;

評估指標；

Mean/Median of prediction

預測的均值/中位數

Standard Deviation of prediction

預測標準差

Range of prediction

預測范圍

Coefficient of Determination (R2)

測定系數(R2)

Relative Standard Deviation/Coefficient of Variation (RSD)

相對標準偏差/變異系數(RSD)

Relative Squared Error (RSE)

相對平方誤差(RSE)

Mean Absolute Error (MAE)

平均絕對誤差(MAE)

Relative Absolute Error (RAE)

相對絕對誤差(RAE)

Mean Squared Error (MSE)

均方誤差(MSE)

Root Mean Squared Error on Prediction (RMSE/RMSEP)

預測的均方根誤差(RMSE / RMSEP)

Normalized Root Mean Squared Error (Norm RMSEP)

歸一化均方根誤差(范數RMSEP)

Relative Root Mean Squared Error (RRMSEP)

相對均方根誤差(RRMSEP)

Let us consider an example of predicting Active Pharmaceutical Ingredients (API) concentration in a tablet. Using absorbance units from NIR spectroscopy we predict the API level in the tablet. The API concentration in a tablet can be 0.0, 0.1, 0.3, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0. We apply PLS (Partial Least Square) and SVR (Support Vector Regressor) for the prediction of API level.

讓我們考慮一個預測片劑中活性藥物成分(API)濃度的示例。 使用NIR光譜中的吸光度單位，我們可以預測片劑中的API含量。 片劑中的API濃度可以是0.0、0.1、0.3、0.5、1.0、1.5、2.0、2.5、3.0。 我們將PLS(偏最小二乘)和SVR(支持向量回歸)用于API級別的預測。

NOTE: The metrics can be used to compare multiple models or one model with different models

注意：指標可用于比較多個模型或一個模型與不同模型

預測的均值/中位數 (Mean/Median of prediction)

We can understand the bias in prediction between two models using the arithmetic mean of the predicted values.

我們可以使用預測值的算術平均值來理解兩個模型之間的預測偏差。

For example, The mean of predicted values of 0.5 API is calculated by taking the sum of the predicted values for 0.5 API divided by the total number of samples having 0.5 API.

例如，通過將0.5 API的預測值的總和除以具有0.5 API的樣本總數來計算0.5 API的預測值的平均值。

np.mean(predictedArray)

In Fig.1, We can understand how PLS and SVR have performed wrt mean. SVR predicted 0.0 API much better than PLS, whereas, PLS predicted 3.0 API better than SVR. We can choose the models based on the interest of the API level.

在圖1中，我們可以了解PLS和SVR如何執行wrt意思。 SVR預測0.0 API優于PLS，而PLS預測3.0 API優于SVR。 我們可以根據API級別的興趣來選擇模型。

Disadvantage: Mean is affected by outliers. Use Median when you have outliers in your predicted values

缺點：中位數受異常值的影響。 當預測值中有異常值時，請使用“中位數”

Fig.1. Comparing the mean of predicted values between the two models圖。1。 比較兩個模型之間的預測值的平均值

預測標準差 (Standard Deviation of prediction)

The standard deviation (SD) is a measure of the amount of variation or dispersion of a set of values. A low standard deviation indicates that the values tend to be close to the mean (also called the expected value) of the set,. In contrast, a high standard deviation indicates that the values are spread out over a broader range. The SD of predicted values helps in understanding the dispersion of values in different models.

標準偏差(SD)是一組值的變化或離散量的度量。 低標準偏差表示這些值趨于接近集合的平均值(也稱為期望值)。 相反，高標準偏差表示這些值分布在較寬的范圍內。 預測值的SD有助于理解不同模型中值的分散性。

Standard Deviation Formula標準偏差公式 np.std(predictedArray)

In Fig.2, The dispersion of predicted values is less in SVR compared to PLS. So, SVR performs better when we consider the SD metrics.

在圖2中，與PLS相比，SVR中預測值的離散度較小。 因此，在考慮SD指標時，SVR的性能更好。

Fig.1. Comparing the standard deviation of predicted values between the two models圖。1。 比較兩個模型之間預測值的標準偏差

預測范圍 (Range of prediction)

The range of the prediction is the maximum and minimum value in the predicted values. Even range helps us to understand the dispersion between models.

預測范圍是預測值中的最大值和最小值。 偶數范圍可以幫助我們了解模型之間的差異。

測定系數(R2) (Coefficient of Determination (R2))

R-squared (R2) is a statistical measure that represents the proportion of the variance for a dependent variable that’s explained by an independent variable or variables in a regression model. Whereas correlation explains the strength of the relationship between an independent and dependent variable, R-squared explains to what extent the variance of one variable explains the variance of the second variable. So, if the R2 of a model is 0.50, then approximately half of the observed variation can be explained by the model’s inputs.

R平方(R2)是一種統計量度，代表因變量的方差比例，該因變量由回歸模型中的一個或多個自變量解釋。 相關性說明了獨立變量和因變量之間關系的強度，而R平方說明了一個變量的方差在多大程度上解釋了第二個變量的方差。 因此，如果模型的R2為0.50，則可以通過模型的輸入解釋觀察到的變化的大約一半。

R Squared formulaR平方公式 R (Correlation) (source: http://www.mathsisfun.com/data/correlation.html)R(相關)(來源： http : //www.mathsisfun.com/data/correlation.html ) from sklearn.metrics import r2_score
r2_score(Actual, Predicted)

Disadvantage: R2 doesn’t consider overfitting. For more details.

缺點：R2不會考慮過度擬合。 有關更多詳細信息 。

相對標準偏差(RSD)/變異系數(CV) (Relative Standard Deviation (RSD) / Coefficient of Variation (CV))

There is a saying that apples shouldn’t be compared with oranges or in other words, don’t compare two items or group of items that are practically incomparable. But the lack of comparability can be overcome if the two items or groups are somehow standardized or brought on the same scale. For instance, when comparing the variances of two groups that are overall very different, such as the variance in the size of bluefin tuna and blue whales, the coefficient of variation (CV) is the method of choice: the CV simply represents the variance of each group standardized by its group mean

有一種說法是，不應將蘋果與桔子相提并論，換句話說，不要將實際上無法比擬的兩個項目或一組項目進行比較。 但是，如果將這兩個項目或組以某種方式標準化或以相同的規模進行，則可以克服可比性不足的問題。 例如，當比較總體差異很大的兩組的方差時，例如藍鰭金槍魚和藍鯨的大小方差，變異系數(CV)是選擇的方法：CV簡單地代表了每個組均以其組均值標準化

The coefficient of variation (CV), also known as relative standard deviation (RSD), is a standardized measure of the dispersion of a probability distribution or frequency distribution. It helps us in understanding how the spread is the data in two different tests

變異系數(CV)，也稱為相對標準偏差(RSD)，是概率分布或頻率分布的離散度的標準化度量。 它有助于我們了解兩種不同測試中數據的傳播方式

Standard deviation is the most common measure of variability for a single data set. But why do we need yet another measure, such as the coefficient of variation? Well, comparing the standard deviations of two different data sets is meaningless, but comparing coefficients of variation is not.

標準差是單個數據集變異性的最常見度量。 但是，為什么我們還需要另一種測量方法，例如變異系數？ 好吧，比較兩個不同數據集的標準偏差是沒有意義的，但是比較變異系數不是。

Coefficient of Variation (CV) Formula變異系數(CV)公式 from scipy.stats import variation
variation(data)

For example, If we consider two different data;

例如，如果我們考慮兩個不同的數據；

Data 1: Mean1 = 120000 : SD1 = 2000

數據1：平均值1 = 120000：SD1 = 2000

Data 2: Mean2 = 900000 : SD2 = 10000

數據2：Mean2 = 900000：SD2 = 10000

Let us calculate CV for both datasets

讓我們計算兩個數據集的簡歷

CV1 = SD1/Mean1 = 1.6%

CV1 = SD1 /平均值1 = 1.6％

CV2 = SD2/Mean2 = 1.1%

CV2 = SD2 /平均值2 = 1.1％

We can conclude Data 1 is more spread out than Data 2

我們可以得出結論，數據1比數據2更分散

相對平方誤差(RSE) (Relative Squared Error (RSE))

The relative squared error (RSE) is relative to what it would have been if a simple predictor had been used. More specifically, this simple predictor is just the average of the actual values. Thus, the relative squared error takes the total squared error and normalizes it by dividing by the total squared error of the simple predictor. It can be compared between models whose errors are measured in the different units.

相對平方誤差(RSE)與使用簡單的預測變量時的誤差有關。 更具體地說，這個簡單的預測變量只是實際值的平均值。 因此，相對平方誤差取總平方誤差并通過除以簡單預測變量的總平方誤差對其進行歸一化。 可以在以不同單位計量誤差的模型之間進行比較。

Mathematically, the relative squared error, Ei of an individual model i is evaluated by the equation:

在數學上，單個模型i的相對平方誤差Ei由以下公式估算：

Relative Squared Error (RSE) Formula相對平方誤差(RSE)公式

where P(ij) is the value predicted by the individual model i for record j (out of n records); Tj is the target value for record j, and Tbar is given by the formula:

其中P ( ij )是單個模型i對記錄j (在n條記錄中)預測的值； Tj是記錄j的目標值， Tbar由以下公式給出：

For a perfect fit, the numerator is equal to 0 and Ei = 0. So, the Ei index ranges from 0 to infinity, with 0 correspondings to the ideal.

為了完美擬合，分子等于0且Ei =0。因此， Ei索引的范圍是0到無窮大，其中0對應于理想值。

平均絕對誤差(MAE) (Mean Absolute Error (MAE))

In statistics, mean absolute error (MAE) is a measure of errors between paired observations expressing the same phenomenon. Examples of Y versus X include comparisons of predicted versus observed, subsequent time versus initial time, and one technique of measurement versus an alternative technique of measurement. It has the same unit as the original data, and it can only be compared between models whose errors are measured in the same units. It is usually similar in magnitude to RMSE, but slightly smaller. MAE is calculated as:

在統計中，平均絕對誤差(MAE)是表示同一現象的成對觀測值之間誤差的度量。 Y與X的示例包括比較預測值與觀察值，后續時間與初始時間，一種測量技術與另一種測量技術的比較。 它具有與原始數據相同的單位，并且只能在以相同單位測量誤差的模型之間進行比較。 它的大小通常與RMSE相似，但略小。 MAE的計算公式為：

Mean Absolute Error (MAE) Formula平均絕對誤差(MAE)公式 from sklearn.metrics import mean_absolute_error
mean_absolute_error(actual, predicted)

It is thus an arithmetic average of the absolute errors, where yi is the prediction and xi the actual value. Note that alternative formulations may include relative frequencies as weight factors. The mean absolute error uses the same scale as the data being measured. This is known as a scale-dependent accuracy measure and, therefore cannot be used to make comparisons between series using different scales.

因此，它是絕對誤差的算術平均值，其中yi是預測值，xi是實際值。 注意，替代公式可以包括相對頻率作為權重因子。 平均絕對誤差使用與被測數據相同的標度。 這被稱為與比例有關的精度量度，因此不能用于在使用不同比例的系列之間進行比較。

Note: As you see, all the statistics compare true values to their estimates, but do it in a slightly different way. They all tell you “how far away” are your estimated values from the true value. Sometimes square roots are used and occasionally absolute values — this is because when using square roots, the extreme values have more influence on the result (see Why to square the difference instead of taking the absolute value in standard deviation? or on Mathoverflow).

注意：如您所見，所有統計信息都將真實值與其估計值進行比較，但是以略有不同的方式進行。 它們都告訴您估計值與真實值有多遠。 有時使用平方根，有時使用絕對值-這是因為使用平方根時，極值會對結果產生更大的影響(請參閱 為什么對差值求平方而不是將絕對值 求 標準偏差？ 或對 Mathoverflow )。

In MAE and RMSE, you simply look at the “average difference” between those two values. So you interpret them comparing to the scale of your variable (i.e., MSE of 1 point is a difference of 1 point of actual between predicted and actual).

在MAE和RMSE中，您只需查看這兩個值之間的“平均差”即可。 因此，您可以將它們與變量的范圍進行比較來進行解釋(即，MSE為1分與預測值和實際值之差為1分)。

In RAE and Relative RSE, you divide those differences by the variation of actual, so they have a scale from 0 to 1, and if you multiply this value by 100, you get similarity in 0–100 scale (i.e. percentage).

在RAE和相對RSE中，您將這些差異除以實際的差異，因此它們的標度從0到1，如果將此值乘以100，您將獲得0-100的相似度(即百分比)。

The values of ∑(MeanofActual — actual)2 or ∑|MeanofActual — actual| tell you how much actual differs from its mean value — so you could tell what it is about how much actual differs from itself (compare to variance). Because of that, the measures are named “relative” — they give you results related to the scale of actual.

∑(MeanofActual —實際)2或∑ | MeanofActual —實際|的值。 告訴您實際值與平均值之間有多少不同-因此您可以知道實際值與自身有多少不同(比較 方差 )。 因此，這些度量被稱為“相對” —它們為您提供與實際規模有關的結果。

相對絕對誤差(RAE) (Relative Absolute Error (RAE))

Relative Absolute Error (RAE) is a way to measure the performance of a predictive model. RAE is not to be confused with relative error, which is a general measure of precision or accuracy for instruments like clocks, rulers, or scales. It is expressed as a ratio, comparing a mean error (residual) to errors produced by a trivial or naive model. A good forecasting model will produce a ratio close to zero; A poor model (one that’s worse than the naive model) will produce a ratio greater than one.

相對絕對誤差(RAE)是一種衡量預測模型性能的方法。 RAE不應與相對誤差相混淆，相對誤差是相對精度的一種通用度量，用于測量鐘表，標尺或刻度尺等儀器。 它表示為比率，將平均誤差(殘差)與普通模型或幼稚模型產生的誤差進行比較。 一個好的預測模型將產生接近于零的比率。 較差的模型(比原始模型差的模型)將產生大于1的比率。

It is very similar to the relative squared error in the sense that it is also relative to a simple predictor, which is just the average of the actual values. In this case, though, the error is just the total absolute error instead of the total squared error. Thus, the relative absolute error takes the total absolute error and normalizes it by dividing by the total absolute error of the simple predictor.

它與相對平方誤差非常相似，因為它也與簡單的預測變量有關，后者只是實際值的平均值。 但是，在這種情況下，誤差只是總的絕對誤差，而不是總的平方誤差。 因此，相對絕對誤差取總絕對誤差并通過除以簡單預測變量的總絕對誤差對其進行歸一化。

Mathematically, the relative absolute error, Ei of an individual model i is evaluated by the equation:

在數學上，單個模型i的相對絕對誤差Ei由以下公式估算：

Relative Absolute Error (RAE) Formula相對絕對誤差(RAE)公式

where P(ij) is the value predicted by the individual model i for record j (out of n records); Tj is the target value for record j, and Tbar is given by the formula:

其中P ( ij )是單個模型i對記錄j (在n條記錄中)預測的值； Tj是記錄j的目標值， Tbar由以下公式給出：

For a perfect fit, the numerator is equal to 0 and Ei = 0. So, the Ei index ranges from 0 to infinity, with 0 correspondings to the ideal.

對于完美擬合，分子等于0且Ei =0。因此， Ei索引的范圍是0到無窮大，其中0對應于理想值。

均方誤差(MSE) (Mean Squared Error (MSE))

Mean Squared Error (MSE) or Mean Squared Deviation (MSD) of an estimator (of a procedure for estimating an unobserved quantity) measures the average of the squares of the errors — that is, the average squared difference between the estimated values and the actual value. MSE is a risk function, corresponding to the expected value of the squared error loss. The fact that MSE is almost always strictly positive (and not zero) is because of randomness or because the estimator does not account for information that could produce a more accurate estimate.

估計器的均方誤差(MSE)或均方偏差(MSD)(用于估計未觀察到的量的過程)測量誤差平方的平均值-即，估算值與實際值之間的均方差值。 MSE是一個風險函數，對應于平方誤差損失的期望值。 MSE幾乎始終嚴格為正數(而不是零)的事實是由于隨機性，或者是因為估算器沒有考慮可能產生更準確估算的信息。

The MSE assesses the quality of a predictor (i.e., a function mapping arbitrary inputs to a sample of values of some random variable), or an estimator (i.e., a mathematical function mapping a sample of data to an estimate of a parameter of the population from which the data is sampled). The definition of an MSE differs according to whether one is describing a predictor or an estimator.

MSE評估預測器(即，將任意輸入映射到某個隨機變量的值的樣本的函數)或估計器(即，將數據的樣本映射到總體參數的估計的數學函數)的質量從中采樣數據)。 MSE的定義因描述的是預測變量還是估計變量而異。

The MSE is a measure of the quality of an estimator — it is always non-negative, and values closer to zero are better.

MSE是對估算器質量的度量，它始終是非負的，并且接近零的值更好。

Mean Squared Error (MSE) Formula均方誤差(MSE)公式 from sklearn.metrics import mean_squared_error
mean_squared_error(actual, predicted)

Let’s analyze what this equation actually means.

讓我們分析一下這個方程的實際含義。

• In mathematics, the character that looks like weird E is called summation (Greek sigma). It is the sum of a sequence of numbers, from i=1 to n. Let’s imagine this like an array of points, where we go through all the points, from the first (i=1) to the last (i=n).
  在數學中，看起來像奇怪的E的字符稱為求和(Greek sigma)。 它是從i = 1到n的數字序列的總和。 讓我們想象這就像一個點數組，我們從頭一個點(i = 1)到最后一個點(i = n)遍歷所有點。
• For each point, we take the y-coordinate of the point, and the y’-coordinate. We subtract the y-coordinate value from the y’-coordinate value and calculate the square of the result.
  對于每個點，我們采用該點的y坐標和y'坐標。 我們從y'坐標值中減去y坐標值，然后計算結果的平方。
• The third part is to take the sum of all the (y-y’)2 values and divide it by n, which will give the mean.
  第三部分是取所有(y-y')2值的總和，然后除以n，得到平均值。

Our goal is to minimize this mean, which will provide us with the best line that goes through all the points. For more information.

我們的目標是盡量減少這種均值，這將為我們提供貫穿所有要點的最佳路線。 有關更多信息 。

預測的均方根誤差(RMSE / RMSEP) (Root Mean Squared Error on Prediction (RMSE / RMSEP))

In statistical modeling and particularly regression analyses, a common way of measuring the quality of the fit of the model is the RMSE (also called Root Mean Square Deviation), given by

在統計建模(尤其是回歸分析)中，衡量模型擬合質量的常用方法是RMSE(也稱為均方根偏差)，由

RMSE FormulaRMSE公式 from sklearn.metrics import mean_squared_error
mse = mean_squared_error(actual, predicted)
rmse = sqrt(mse)

where yi is the ith observation of y and ? the predicted y value given the model. If the predicted responses are very close to the true responses the RMSE will be small. If the predicted and true responses differ substantially — at least for some observations — the RMSE will be large. A value of zero would indicate a perfect fit to the data. Since the RMSE is measured on the same scale, with the same units as y, one can expect 68% of the y values to be within 1 RMSE — given the data is normally distributed.

其中yi是y的第i個觀測值， ?是給定模型的y預測值。 如果預測的響應非常接近真實響應，則RMSE將很小。 如果預測的響應與真實的響應大不相同(至少對于某些觀察而言)，則RMSE將很大。 零值表示對數據的完美擬合。 由于RMSE是在與y相同的比例尺和相同的單位下進行測量的，因此，假設數據是正態分布的，則可以預期y值的68％在1 RMSE之內。

NOTE: RMSE is concerned with deviations from the true value whereas S is concerned with deviations from the mean.

注意：RMSE與實際值的偏差有關，而S與平均值的偏差有關。

So calculating the MSE helps to compare different models that are based on the same y observations. But what if

因此，計算MSE有助于比較基于相同y觀測值的不同模型。 但是如果

one wants to compare model fits of different response variables?

是否想比較不同響應變量的模型擬合？

the response variable y is modified in some models, e.g. standardized or sqrt- or log-transformed?

響應變量y是否在某些模型中進行了修改，例如標準化或sqrt轉換或對數轉換？

And does the splitting of data into a training and test dataset (after the modification) and the RMSE calculation based on the test data an effect on point 1. and 2.?

并將數據拆分為訓練和測試數據集(修改后)以及基于測試數據的RMSE計算是否會對點1和點2產生影響？

The first two points are typical issues when comparing ecological indicator performances and the latter, so-called validation set approach, is pretty common in statistical and machine learning. One solution to overcome these barriers is to calculate the Normalized RMSE.

前兩點是比較生態指標性能時的典型問題，而后者，即所謂的驗證集方法 ，在統計和機器學習中非常普遍。 克服這些障礙的一種方法是計算歸一化的 RMSE。

歸一化均方根誤差(范數RMSEP) (Normalized Root Mean Squared Error (Norm RMSEP))

Normalizing the RMSE facilitates the comparison between datasets or models with different scales. You will find, however, various different methods of RMSE normalizations in the literature:

標準化RMSE有助于比較不同比例的數據集或模型。 但是，您會在文獻中找到各種不同的RMSE歸一化方法：

You can normalize by

您可以通過

If the response variables have few extreme values, choosing the interquartile range is a good option as it is less sensitive to outliers.

如果響應變量的極值很少，則選擇四分位數范圍是一個不錯的選擇，因為它對異常值不太敏感。

RMSEP/standard deviation is called Relative Root Mean Squared Error (RRMSEP)

RMSEP /標準偏差稱為相對均方根誤差(RRMSEP)

1/RRMSEP is also a metric. A value greater than 2 is considered to be a good.

1 / RRMSEP也是一個度量。 大于2的值被認為是好的。

There are also terms like, Standard Error of Prediction(SEP) and Ratio of the Standard Error of Prediction to Standard Deviation (RPD) which are mainly used in chemometrics.

還有一些術語，例如預測標準誤差(SEP)和預測標準誤差與標準偏差(RPD)的比率，它們主要用于化學計量學。

I hope this blog helped you to understand different metrics to evaluate your regression model. I have used multiple sources to understand and write this article. Thank you for your time.

我希望這個博客可以幫助您了解不同的指標，以評估您的回歸模型。 我使用了多種資源來理解和撰寫本文。 感謝您的時間。

翻譯自: https://towardsdatascience.com/ways-to-evaluate-regression-models-77a3ff45ba70

回歸模型評估

總結

以上是生活随笔為你收集整理的回归模型评估_评估回归模型的方法的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。