Añadir esta página al libro
Eliminar esta página del libro
Fixed, Random OR within-between BW model effects?
First, if the purpose of an analysis is prediction (as opposed to inference on marginal effects), then FE estimator is the unambiguously preferred estimator. Figure 6 and row 3 of Figures 7, 8, 9, 10, 11 show that the FE predictions always have a smaller RMSE. Because the FE estimator cannot predict outcomes for groups that are not included in the original estimation, the WB approach is the clear second-best estimator in these situations.
FE for prediction
Second, when the purpose of an analysis is inference on marginal effects, simulation suggests the WB approach is preferred over traditional FE estimation. The simulation confirms that the within-group marginal effects estimates are asymptotically identical, but the WB approach provides researchers with additional information regarding the between-group marginal effects (). For the small samples included in our analyses, the WB approach consistently has (essentially) equivalent or smaller MSE than the FE estimation. Outside of the variation that can be attributed to noise, there are no scenarios when FE is MSE-preferred over the WB approach.
WB for inference of marginal effects
Third, as a general rule, the larger the sample size, the more a practitioner should avoid traditional RE estimation. Applying FE estimation on all simulated samples with greater than 500 observations led to a median absolute error of 4% of the true marginal effect. RE estimation led to a median absolute error of 8% of the true marginal effect. In simulations with more than 1,000 observations, RE estimation was only MSE-preferred beyond a trivial threshold (0.005) in a very few cases where 90% of variation of y could not be explained by the model (). It is important to note that this rule contradicts researchers' tendency to use RE estimation for problems with large J. Presumably researchers are hesitant to “waste” degrees-of-freedom for each of the J groups when J is quite large. While it is true that more degrees of freedom are used by choosing FE estimation, simulation shows that using that FE estimation is MSE-preferred (relative to RE estimation) in most large-J scenarios. This is a case where the WB approach is a valuable compromise between the two traditional estimators. The WB approach generates practically equivalent estimates as FE but uses only a fraction of the degrees-of-freedom.
Larger sample sizes → FE
Fourth, small samples mark the circumstances under which a practitioner might consistently choose precision over bias. Our simulations show that, when combined with small samples (of less than or equal to several hundred observations), two observable data characteristics make it especially likely that RE estimation would be MSE-preferred. One scenario is when the estimated model explains a very small portion of the variation in the outcome measurement. When small sample size is combined with a poorly-fit model, the imprecision of FE and WB estimation tends to mislead the researcher more than the bias of RE estimation, even at large ρ. The goodness-of-fit of the clustered model can be explored by examining the R2 statistic associated with LSDV estimation. Considering only simulations with R2<0.5 and less than 500 observations, the traditional RE estimator had a smaller absolute error than the FE estimator 57% of the time.
Another unique scenario when the RE estimator is consistently MSE-preferred and should be considered is for small samples that have relatively small within-group variation for the variable of interest. Again, in these cases, the imprecision of the FE and WB estimators might be more caustic than the RE estimator's bias. In simulation cases with less than 500 observations and within-group variation less than 20% of the total variation, RE estimation leads to a smaller absolute error 53% of the time. However, with small samples, both the WB and FE estimators are less reliable as they draw inference from too few observations and likely too little variation. If choosing a RE model, there may be a large but imprecise distinction compared to FE and WB. The practitioner may have inadequate power to adequately adjust for cluster-level confounding.
The fifth rule garnered from this simulation is that the Hausman test is most insightful with large samples. Unfortunately, these are the same cases when the test is needed the least. In simulations with greater than or equal to 1,000 observations, the Hausman test recommends the estimator with the smaller error 67% of the time. However, when the FE estimator is blindly applied to all these same cases, it generates the same median absolute error and is the “better” estimator (when compared to the traditional RE estimator) in 70% of the simulations.
These rules of thumb do not apply to all situations that the practitioner may encounter. For example, different estimation techniques are required for binary response models and complex survey data. When the outcome variable is binary, specifying RE, FE, or WB model in this framework produces new challenges for correct model specification and interpretation of marginal effects. The RE binary outcome model is a special form of the population average model. Instead of estimating (as in the linear case), it assumes uj has a parametric distribution and is independent from . The marginal effect from a RE binary response model is the population average effect for an individual at = 0 .