We can take this equation one step further: In order to remedy this situation, one solution is to take the square of this distance (which will always be a positive number), then calculate the sum of these squared distances for all data points and finally take the square root of this sum to obtain the Root Mean Square Error (RMSE): The simplest way to quantify how far the data points are from the regression line, is to calculate the average distance from this line:īut, because some of the distances are positive and some are negative (certain points are above the regression line and others are below it), these distances will cancel each other out - meaning that the average distance will be biased low. Residual standard deviation vs residual standard error vs RMSE Now that we developed a basic intuition, next we will try to come up with a statistic that quantifies this goodness of fit. Mathematically, the error of the i th point on the x-axis is given by the equation: (Y i – Ŷ i), which is the difference between the true value of Y (Y i) and the value predicted by the linear model (Ŷ i) - this difference determines the length of the gray vertical lines in the plots above.

In the plots above, the gray vertical lines represent the error terms - the difference between the model and the true value of Y. Therefore, using a linear regression model to approximate the true values of these points will yield smaller errors than “example 1”. This is because in “example 2” the points are closer to the regression line. Just by looking at these plots we can say that the linear regression model in “example 2” fits the data better than that of “example 1”. Plotted below are examples of 2 of these regression lines modeling 2 different datasets:

(The other measure to assess this goodness of fit is R 2).īut before we discuss the residual standard deviation, let’s try to assess the goodness of fit graphically.Ĭonsider the following linear regression model: The residual standard deviation (or residual standard error) is a measure used to assess how well a linear regression model fits the data.

Standard error of the mean