Evaluation of the BFI10 Personality Inventory with New Zealand University Students: Failing Psychometric Tests
citation: Brown, G. T. L. & Sotardi, V. (2019, March). Evaluation of the BFI10 Personality Inventory with New Zealand University Students: Failing Psychometric Tests. Poster presented at the 2019 International Congress of Psychological Science (ICPS), Paris, France.
Abstract: The 10-item Big Five inventory (BFI10) is a brief solution for measuring five personality traits. The BFI10 inventory was administered in two independent samples of New Zealand university students. The factor structure of the responses was analysed independently and jointly in three analytic studies. Each study is reported separately because sampling and response administration was different. Joint analysis overcomes any possible limitations associated with sample size.
In Study 1 (n = 330), confirmatory factor analysis of the 5-factor model had acceptable fit characteristics, but scale estimates of reliability were weak and four items had negative error variance and item loadings for four items were very low. Exploratory factor analysis with oblique rotation had superior fit but deleted the Openness factor and two items; further, Agreeableness and Conscientiousness had different item combinations.
In Study 2, (n = 231), CFA with MLE of the 5-factor model resulted in poor fit characteristics to data. Item loadings for two factors (Agreeableness – reverse item, and Openness – reverse item) had statistically non-significant values. Scale estimates of reliability were poor (ranging from α = .19 to .70; M = .47). EFA with MLE and oblique rotation yielded four factors, but item loadings were acceptable only for the Neuroticism and Extraversion factors. Similar to Study 1, the factors for Openness, Agreeableness, and Conscientiousness had different item combinations.
Because the expected BFI10 model was not recoverable in these two samples, it was decided to merge the data sets to overcome any possible limitations due to small sample size. In Study 3, a number of techniques were applied to the two data sets. The best fitting model in Study 1 and Study 2 was tested for invariance on the sample in the other study. A joint common EFA (MLE, oblique rotation) followed by CFA was used to ascertain a plausible model structure and test it for fit to the data. Principal component analysis of the joint data was conducted and then submitted to CFA to test for fit to the data. The Study 2 data were inadmissible for the original BFI10 model because of covariance matrix not positive definite. Likewise, the best fitting model from Study 1 (4 factors, 8 items) was also positive not definite for the Study 2 data. The best fitting model from Study 2 (4 factors, 10 items) was inadmissible for Study 1 data for the same reason. The joint solution based on PCA, had 3 factors and 9 items, two of which loaded on two factors. The dominant component had 6 items (one from each personality trait and two from agreeableness), while component two had 3 items, and component 3 had 2 items. Fit was good, but invariance was limited to configural equivalence.
All modelling resulted in inability to recover the intended 10 item, 5-factor BFI10 or generate a model that was invariant across both samples. These analyses raise serious doubts about the reliability and generalisability of the BFI10. Users are cautioned against using this brief Big Five inventory without thorough examination of model-data fit.
