Jonas Moss

Behavior Research Methods 58(107) Doi: https://doi.org/10.3758/s13428-026-02968-4

Testing goodness-of-fit and multi-group nested models in confirmatory factor analysis under non-normality is foundational in psychometrics and related fields. Recently, a penalized eigenvalue block averaging (pEBA) procedure was proposed for testing goodness-of-fit, showing promise in a restricted type I error control simulation study. In this study, we extend the simulation conditions to higher dimensions for latent and observed vectors and evaluate type I error control and power for many pEBA variants and traditional test statistics. All statistics are evaluated in four versions, by crossing the base statistic (ML or Browne’s RLS), and whether the asymptotic covariance matrix estimator was bias-corrected or not. We develop pEBA methods in the new setting of nested model comparison, accompanied by extensive Monte Carlo evaluation of their performance in weak invariance testing, including type I error control and power. The best-performing procedure for goodness-of-fit testing was pEBA with four blocks, based on the RLS statistic, using the asymptotic covariance matrix estimator without bias correction. For measurement invariance, pEBA with singleton blocks, using the standard ML statistic and the unbiased estimator for the asymptotic covariance matrix, performed best. The pEBA procedures are available in the newly developed R package semTests.

Applied Psychological Measurement Doi: https://doi.org/10.1177/01466216261440511

When only summary statistics from published studies are available, the Hunter–Schmidt interval is the standard tool for inference on Spearman’s disattenuated correlation, but it treats reliability estimates as known constants and ignores their sampling variability. We derive a simple delta method variance that accounts for the uncertainty of all estimates while requiring nothing beyond the summaries already at hand. Under bivariate normality of scores and coefficient alpha from a normal parallel model, the corrected interval is asymptotically valid. In simulations it achieves coverage near nominal, while Hunter–Schmidt can undercover substantially when reliability is imprecisely estimated.

Psychometrika 89(2) p. 517-541 Doi: https://doi.org/10.1007/s11336-023-09945-2

Most measures of agreement are chance-corrected. They differ in three dimensions: their definition of chance agreement, their choice of disagreement function, and how they handle multiple raters. Chance agreement is usually defined in a pairwise manner, following either Cohen’s kappa or Fleiss’s kappa. The disagreement function is usually a nominal, quadratic, or absolute value function. But how to handle multiple raters is contentious, with the main contenders being Fleiss’s kappa, Conger’s kappa, and Hubert’s kappa, the variant of Fleiss’s kappa where agreement is said to occur only if every rater agrees. More generally, multi-rater agreement coefficients can be defined in a g-wise way, where the disagreement weighting function uses g raters instead of two. This paper contains two main contributions. (a) We propose using Fréchet variances to handle the case of multiple raters. The Fréchet variances are intuitive disagreement measures and turn out to generalize the nominal, quadratic, and absolute value functions to the case of more than two raters. (b) We derive the limit theory of g-wise weighted agreement coefficients, with chance agreement of the Cohen-type or Fleiss-type, for the case where every item is rated by the same number of raters. Trying out three confidence interval constructions, we end up recommending calculating confidence intervals using the arcsine transform or the Fisher transform.

Structural Equation Modeling p. 1-13 Doi: https://doi.org/10.1080/10705511.2024.2372028

We propose new ways of robustifying goodness-of-fit tests for structural equation modeling under non-normality. These test statistics have limit distributions characterized by eigenvalues whose estimates are highly unstable and biased in known directions. To take this into account, we design model-based trend predictions to approximate the population eigenvalues. We evaluate the new procedures in a large-scale simulation study with three confirmatory factor models of varying size (10, 20, or 40 manifest variables) and six non-normal data conditions. The eigenvalues in each simulated dataset are available in a database. Some of the new procedures markedly outperform presently available methods. We demonstrate how the new tests are calculated with a new R package and provide practical recommendations.

Psychometrika 88 p. 241-252 Doi: https://doi.org/10.1007/s11336-022-09898-y

The polychoric correlation is a popular measure of association for ordinal data. It estimates a latent correlation, i.e., the correlation of a latent vector. This vector is assumed to be bivariate normal, an assumption that cannot always be justified. When bivariate normality does not hold, the polychoric correlation will not necessarily approximate the true latent correlation, even when the observed variables have many categories. We calculate the sets of possible values of the latent correlation when latent bivariate normality is not necessarily true, but at least the latent marginals are known. The resulting sets are called partial identification sets, and are shown to shrink to the true latent correlation as the number of categories increase. Moreover, we investigate partial identification under the additional assumption that the latent copula is symmetric, and calculate the partial identification set when one variable is ordinal and another is continuous. We show that little can be said about latent correlations, unless we have impractically many categories or we know a great deal about the distribution of the latent vector. An open-source R package is available for applying our results.

Psychometric gathering

Frontiers in Pharmacology Doi: https://doi.org/10.3389/fphar.2023.1152305

Psykologisk tidsskrift p. 46-52