| Indicator | Description | Unit | |
|---|---|---|---|
| \(y_1\) | HbA1c | 3-month avg. blood glucose | % |
| \(y_2\) | FPG | Fasting plasma glucose | mmol/L |
| \(y_3\) | Insulin | Fasting insulin level | µU/mL |
| \(y_4\) | PCr | Plasma creatinine | µmol/L |
| \(y_5\) | ACR | Albumin–creatinine ratio | mg/g |
| \(y_6\) | BUN | Blood urea nitrogen | mmol/L |
Bias-reduced estimation of structural equation models
October 16, 2025
Universiteit Gent | R/
{lavaan}
University of Warwick
University of Warwick
Jamil, H., Rosseel, Y., Kemp, O., & Kosmidis, I. (2025). Bias-Reduced Estimation of Structural Equation Models. Manuscript in Submission.
arXiv:2509.25419.
poll
Context
In the social sciences, latent variables are used to represent constructs—the theoretical, unobserved concepts of interest.
Personality traits
Quality of life
Social trust
Competencies
Outline
- Brief overview of SEMs
- Motivating example
- ML estimation and inference
- Examples of SEMs
- Two-factor SEM
- Latent growth models
- Bias reducing methods
- What is bias?
- A review of bias reduction methods
- Reduced-Bias \(M\)-estimation (RBM)
- Implicit correction
- Explicit correction
- Simulation studies and results
\[ \definecolor{kaustorg}{RGB}{241, 143, 0} \definecolor{kausttur}{RGB}{0, 166, 170} \definecolor{kaustmer}{RGB}{177, 15, 46} \definecolor{kaustgrn}{RGB}{173, 191, 4} \definecolor{kaustblu}{RGB}{82, 132, 196} \definecolor{kaustpur}{RGB}{156, 111, 174} \]
Covariance-based approach
Sample correlation matrix looks like this1:
- The data suggests clustering of variables
- \(y_1\),\(y_2\),\(y_3\) measure glycemic control (\(\texttt{GlyCon}\))
- \(y_4\),\(y_5\),\(y_6\) measure kidney health (\(\texttt{KdnHlt}\))
- There is an element of dimension-reduction; much needed for analysing (correlated) multivariate data.
Easier to hypothesize relationships, e.g. \[ {\color{kaustmer}\texttt{KdnHlt}} = \alpha + \beta \, {\color{kaustblu}\texttt{GlyCon}} + \texttt{error} \]
SEM is about modelling the covariance structure of the data, \[ \boldsymbol\Sigma = \boldsymbol\Sigma(\vartheta). \]
SEM equations
\[ \small \begin{gathered} \begin{pmatrix} y_1 \\ y_2 \\ y_3 \\ y_4 \\ y_5 \\ y_6 \end{pmatrix} = \begin{pmatrix} \textcolor{kaustpur}{\lambda_{11}} & 0 \\ \textcolor{kaustpur}{\lambda_{21}} & 0 \\ \textcolor{kaustpur}{\lambda_{31}} & 0 \\ 0 & \textcolor{kaustpur}{\lambda_{42}} \\ 0 & \textcolor{kaustpur}{\lambda_{52}} \\ 0 & \textcolor{kaustpur}{\lambda_{62}} \\ \end{pmatrix} \begin{pmatrix} \eta_1 \\ \eta_2 \end{pmatrix} + \begin{pmatrix} \epsilon_1 \\ \epsilon_2 \\ \epsilon_3 \\ \epsilon_4 \\ \epsilon_5 \\ \epsilon_6 \end{pmatrix} \\[1em] \begin{pmatrix} \eta_1 \\ \eta_2 \end{pmatrix} = \begin{pmatrix} 0 & 0 \\ {\color{kaustgrn}\beta} & 0 \\ \end{pmatrix} \begin{pmatrix} \eta_1 \\ \eta_2 \end{pmatrix} + \begin{pmatrix} \zeta_1 \\ \zeta_2 \end{pmatrix} \end{gathered} \]
Or, more compactly as
\[ \begin{gathered} \mathbf y = {\color{Gray}\boldsymbol\nu +\,} {\color{kaustpur}\boldsymbol\Lambda} \boldsymbol\eta + \boldsymbol\epsilon \\ \boldsymbol\eta = {\color{Gray}\boldsymbol\alpha +\,} {\color{kaustgrn}\mathbf B} \boldsymbol\eta + \boldsymbol\zeta \end{gathered} \]
with assumptions \(\boldsymbol\epsilon\sim\operatorname{N}_p(\mathbf 0, {\color{kaustorg}\boldsymbol\Theta})\), \(\boldsymbol\zeta\sim\operatorname{N}_q(\mathbf 0, {\color{kausttur}\boldsymbol\Psi})\), and \(\operatorname{cov}(\boldsymbol\epsilon, \boldsymbol\zeta)=\mathbf 0\).
SEM parameters include the free entries of \(\color{Gray}\boldsymbol\nu\), \(\color{kaustpur}\boldsymbol\Lambda\), \(\color{kaustorg}\boldsymbol\Theta\), \(\color{Gray}\boldsymbol\alpha\), \(\color{kaustgrn}\mathbf B\), and \(\color{kausttur}\boldsymbol\Psi\).
Dump all in \(\vartheta \in\mathbb R^m\), where \(m < p(p+1)/2 {\color{Gray} \,+\, p}\).
Sometimes, not interested in mean structure, so \(\color{Gray}\boldsymbol\nu\) and \(\color{Gray}\boldsymbol\alpha\) are dropped.
Latent GCM as SEM
\[ \begin{gathered} \boldsymbol\Lambda = \begin{bmatrix} 1 & 0 \\ 1 & 1 \\ \vdots & \vdots \\ 1 & 9 \\ \end{bmatrix} \\ \\ \mathbf y = \boldsymbol\Lambda \boldsymbol\eta + \boldsymbol\epsilon \\ \boldsymbol\epsilon \sim \operatorname{N}_{10}(\mathbf 0, \theta \mathbf I) \\ \boldsymbol\eta \sim \operatorname{N}_2 (\boldsymbol\alpha, \boldsymbol\Psi) \end{gathered} \]
Illustration
Biased MLE estimator for \(\sigma^2\)
Consider \(X_1,\dots,X_n \sim \text N(0, \sigma^2)\). The MLE for \(\sigma^2\) is \(\hat\sigma^2 = \frac{1}{n}\sum_{i=1}^n X_i^2\).
Illustration (cont.)
Score functions are random too
The score function is \(s(\sigma^2)=\ell'(\sigma^2) = -\frac{n}{2\sigma^2} + \frac{1}{2\sigma^4}\sum_{i=1}^n X_i^2\).
A review
| Method | Model | \(\mathcal{B}(\bar\vartheta)\) | Type |
Requirements
|
|||
|---|---|---|---|---|---|---|---|
| \(\mathbb{E}(\cdot)\) | \(\hspace{2pt} \partial \cdot \hspace{2pt}\) | \(\hspace{4pt} \hat\vartheta \hspace{4pt}\) | |||||
| 1 | Asymptotic bias correction | full | analytical | explicit | ✓ | ✓ | ✓ |
| 2 | Adjusted score functions | full | analytical | implicit | ✓ | ✓ | ✗ |
| 3 | Bootstrap | partial | simulation | explicit | ✗ | ✗ | ✓ |
| 4 | Jackknife | partial | simulation | explicit | ✗ | ✗ | ✓ |
| 5 | Indirect inference | full | simulation | implicit | ✗ | ✗ | ✓ |
| 6 | Explicit RBM | partial | analytical | explicit | ✗ | ✓ | ✓ |
| 7 | Implicit RBM | partial | analytical | implicit | ✗ | ✓ | ✗ |
Firth’s adjusted scores methods
Instead of solving \(s(\vartheta)=0\), solve \(s(\vartheta) + \overbrace{A(\vartheta)}^{\mathcal B(\vartheta) I(\vartheta)}=0\).
Explicit RBM estimator
Intuitively, by thinking in terms of a Newton-style update, an explicit estimator is obtained via
\[ \vartheta^*= \hat\vartheta + j(\hat\vartheta)^{-1} A(\hat\vartheta). \]
This moves \(\hat\theta\) in the direction \(A(\hat\vartheta)\) away from the bias, with step length governed by the curvature \(j(\hat\vartheta)^{-1}\).
Operationally, eRBM is simpler and quicker to compute than iRBM—no re-optimisation needed if \(\hat\theta\) is available.
However, unlike iRBM, no guarantees that bias correction stays inside the parameter space.
Simulation design
Sample size: \(n\in \{15,20,50,100,1000\}\)
Item reliability: Low or High (\(\operatorname{Rel} = p^{-1}\sum_{j=1}^p \boldsymbol\Sigma_{jj}^* / \boldsymbol\Sigma_{jj}\))
Distributional assumption: Normal or Non-normal
Results: Two-factor SEM
Results: Two-factor SEM (cont.)
Results: Latent GCM
