An Empirical Comparison of Information-Theoretic Selection.pdf

An Empirical Comparison of Information-Theoretic Selection Criteria for Multivariate Behavior Genetic Models Kristian E. Markon1,2 and Robert F. Krueger1 Received 1 Sept. 2003—Final 10 Aug. 2004 Information theory provides an attractive basis for statistical inference and model selection. However, little is known about the relative performance of different information-theoretic criteria in covariance structure modeling, especially in behavioral genetic contexts. To explore these issues, information-theoretic fit criteria were compared with regard to their ability to discriminate between multivariate behavioral genetic models under various model, distribution, and sample size conditions. Results indicate that performance depends on sam- ple size, model complexity, and distributional specification. The Bayesian Information Crite- rion (BIC) is more robust to distributional misspecification than Akaike’s Information Criterion (AIC) under certain conditions, and outperforms AIC in larger samples and when comparing more complex models. An approximation to the Minimum Description Length (MDL; Rissanen, J. (1996). IEEE Transactions on Information Theory 42:40–47, Rissanen, J. (2001). IEEE Transactions on Information Theory 47:1712–1717) criterion, involving the empirical Fisher information matrix, exhibits variable patterns of performance due to the complexity of estimating Fisher information matrices. Results indicate that a relatively new information-theoretic criterion, Draper’s Information Criterion (DIC; Draper, 1995), which shares features of the Bayesian and MDL criteria, performs similarly to or better than BIC. Results emphasize the importance of further research into theory and computa- tion of information-theoretic criteria. KEY WORDS: Akaike’s Information Criterion (AIC); Bayesian Information Criterion (BIC); Minimum Description Length (MDL); model selection; Monte Carlo. INTRODUCTION Numerous studies have examined the performance of goodness-of-fit statistics