A comparison of Linear Regression Models for Heteroscedastic and Non-Normal Data

Abstract

In common practices, heteroscedasticity and non-normality are frequently encountered when fitting linear regression models. Several methods have been proposed to handle these problems. In this research, we compared four different estimation methods: ordinary least squares (OLS), transform both sides (TBS), power of the mean function (POM) and exponential variance function (VEXP), dealing with three different forms of the non-constant variances under four symmetric distributions. In order to study the performance of the four methods in estimating the studied model parameters, a simulation study with various sample sizes of 20, 50, 100, and 200 was conducted. To determine the models with the best fit, relative bias, mean squared error (MSE) and coverage probability of the nominal 95% confidence interval were applied. The simulation results and application to real life data suggest that each estimation method performed differently on different variance structures and different distributions whereas the sample size did not give much effect on each estimation method except in the case of extreme heteroscedasticity. In overall, the TBS method performed best in terms of smallest bias and MSE, especially under extreme heteroscedasticity. On the other hand, the OLS method was very accurate in maintaining the nominal coverage probabilities although it had relatively poor performance in terms of bias.