Introduction

In educational and psychological measurement, the use of multiple test forms that assess the same construct is a common practice, particularly to prevent item exposure, ensure test security, and administer large-scale tests. When different forms are administered to different groups of examinees, it is necessary to place the resulting scores on a common scale to enable meaningful comparisons. In psychological assessment, this issue of comparability goes beyond a simple technical concern. When the same construct is measured across different settings or groups, a lack of score consistency can seriously weaken the validity of any clinical or diagnostic interpretations based on those results. This process is referred to in the literature as test equating [1] and can also be understood as expressing examinees’ ability parameters on the same latent trait metric [2]. Through equating, it is possible to monitor individuals’ performance across administrations and to compare item parameter estimates obtained from different samples. In psychological assessment contexts, such comparability is essential, as test scores are often used to inform decisions about individuals’ cognitive, emotional, or behavioral functioning.

Item Response Theory (IRT) based equating approaches are widely used in measurement practice because they model item and person parameters separately and are relatively less sensitive to sampling fluctuations [3–5]. This is particularly important in psychological measurement, where ensuring comparability across administrations supports valid inferences about latent traits. One of the most employed designs in this context is the nonequivalent groups with anchor test (NEAT) design [1], in which a set of common items is used to link a new form to a base form and place both on the same scale [6]. This approach is supported by evidence showing that characteristic-curve methods based on separate calibration yield more stable results than moment methods under the Common-Item Equating Design [7].

In IRT-based equating, anchor items play a central role in establishing the linking, and the accuracy of the linking constants largely depends on the stability of their item parameters. Accordingly, the validity of the equating results rests on the assumption that anchor items function equivalently across forms. Item parameter drift (IPD) refers to the phenomenon in which the difficulty or discrimination parameters of anchor items shift across administrations [8,9], thereby posing one of the most fundamental threats to the assumption of parameter invariance. The primary causes of IPD include curriculum changes, item exposure, and cultural, educational, or technological developments that may render anchor items easier or harder for examinees over time [10,11]. When item parameter drift occurs, scale transformations may become biased, which can in turn affect ability estimates and compromise the comparability of scores across test forms [12]. Among the various forms of IPD, drift in the difficulty (b) parameter represents the most encountered and practically consequential form in large-scale assessments [13,14]. Although drift in the discrimination (a) and pseudo-guessing (c) parameters also occurs in practice, b-parameter drift appears to be more prevalent in operational testing contexts and has been shown to have a more direct impact on equating accuracy. Drift in a parameter tends to be less frequent and has comparatively more localized effects on scale transformation, while c-parameter drift is particularly difficult to detect reliably due to the inherently lower precision of its estimates [13,14]. For these reasons, the present study focuses specifically on b-parameter drift, reflecting conditions most likely to arise in operational testing contexts. In psychological testing contexts, such distortions may compromise the accuracy of individual-level interpretations, especially when test scores are used for diagnostic or evaluative purposes.

Another major challenge in equating studies is missing data. Missing responses are ubiquitous in social and educational research and directly affect the validity and reliability of statistical analyses. Examinees may omit items for various reasons, such as difficulty in choosing among response options, lack of motivation to respond, concerns about data security, or time limitations that prevent them from reaching all items [15]. The impact of missing data is determined not only by the proportion of missingness but also by its pattern and underlying mechanism [16]. As the amount of missing data increases, the representativeness of the sample decreases, generalizability is compromised, and statistical bias tends to increase [17]. In psychological research and assessment, such biases may disproportionately affect certain groups of individuals, thereby raising concerns about fairness and validity. Following Rubin [18], missing data mechanisms are categorized as missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR). Under MCAR, missingness is independent of both observed and unobserved variables; under MAR, it depends only on observed variables such as ability level [19]. In this study, only MCAR and MAR were simulated, since MNAR introduces complex methodological challenges in large-scale test equating [20,21]. MCAR represents completely random missingness, whereas MAR reflects missingness dependent on observed variables, consistent with patterns observed in empirical data [15].

To address missing data, a variety of traditional and modern approaches have been proposed. Traditional methods such as listwise and pairwise deletion may lead to biased estimates and inflated standard errors, particularly when the proportion of missing data is high [22]. Among modern techniques, multiple imputation (MI) is a flexible and efficient approach that involves generating multiple plausible versions of the incomplete dataset and combining results from these completed datasets. Allison [23] noted that MI is applicable across a wide range of statistical models, making it one of the most practical imputation methods. In recent years, MI has been increasingly recommended in the IRT context to reduce bias due to missing data and to provide more realistic representations of estimation uncertainty [24]. In the present study, MICE was applied to generate multiple plausible datasets and to combine estimates using Rubin’s rules [23,25]. The number of imputations for each condition is reported in the Method section, where the rationale for the selected imputation strategy is described in detail.

A review of the literature indicates that, although test equating has been examined in relation to missing data [26–28] and item parameter drift [29–33] separately, to our knowledge, no study has yet examined their interactive effects within a single controlled framework, a gap that has direct implications for operational equating practice.

Given that missing responses are unavoidable in operational testing programs, understanding how missing data and anchor item parameter drift jointly affect equating accuracy is of both theoretical and practical importance. In particular, it remains unclear how different missing data mechanisms and rates influence the standard errors of the equating constants in the presence of parameter drift, to what extent MI can reduce this uncertainty, and how the performance of the commonly used equating method is affected under these non-ideal conditions.

The current simulation study was designed to systematically manipulate missing data mechanism (MCAR, MAR), missing data rate (0%, 10%, 20%), IPD rate (0%, 10%, 20%), and IPD magnitude (0, 0.25, 0.50), allowing for a controlled examination of equating accuracy under realistic operational scenarios using the Stocking–Lord (SL) method and IRT true score equating. The SL method was selected over competing linking methods, notably the Mean/Sigma and Haebara approaches, due to its demonstrated robustness in the presence of IPD. Unlike the Mean/Sigma method, which relies on the mean and standard deviation of item parameter estimates and may therefore be more sensitive to parameter drift, the SL method minimizes discrepancies across the full test characteristic curve, thereby reducing sensitivity to localized parameter instability. The Haebara method similarly operates at the item level but has, in some simulation studies, shown sensitivity to outlying item parameters [29]. Previous studies have also suggested that the SL method can exhibit relatively greater stability under conditions such as small sample sizes and noisy data [34,35]. In combination with IRT true score equating, the SL method facilitates the transformation of item parameters and ability estimates across test forms. This is particularly important when the 3PL model is used, as non-zero guessing parameters (c) require accurate scale alignment.

In sum, the study integrates NEAT design, multiple imputation for missing data, and the SL scaling approach to investigate the joint effects of IPD and missing responses on equating constants (A and B), where A represents the scale transformation constant, and B represents the location transformation constant used to place the new form on the base-form metric [1].

To the authors’ knowledge, research examining the joint effects of item parameter drift and missing data within a unified simulation framework in IRT-based equating remains limited. The findings are intended to provide evidence-based guidance to practitioners and researchers who must make methodological decisions regarding anchor item management and missing data handling in operational testing programs.

Method

Simulation design structure

The simulation design in this study included the factors of IPD rate (0%, 10%, 20%), IPD magnitude (0, 0.25, 0.50), missing data mechanism (MCAR, MAR), and missing data rate (0%, 10%, 20%).

However, the design was not fully crossed. Certain combinations were excluded based on logical constraints. For example, when the missing data rate was 0%, the missing data mechanism (MCAR/MAR) was not applicable. Similarly, when the IPD rate was 0%, the IPD magnitude factor was not implemented. Therefore, instead of generating all possible combinations, only theoretically meaningful and interpretable conditions were included. In total, 25 experimental data sets were created. Detailed information on these conditions is provided in Table 1.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Simulation conditions (N = 25 conditions × 500 replications).

https://doi.org/10.1371/journal.pone.0353665.t001

Evaluation metrics

The estimation of the A and B transformation constants obtained from the SL method was evaluated using Bias and RMSE. These metrics are widely used as standard performance indicators in IRT equating simulation studies [53]. Bias and RMSE were calculated as follows:

where λ̂_r is the estimated parameter in replication r, λ is the true parameter value, and R is the total number of replications.

In addition to absolute Bias and RMSE, Relative Bias (RB) and Relative RMSE (RRMSE) were also computed to assess the magnitude of deviation relative to the true parameter values [37,54]:

In calculating relative metrics, conditions with |λ| < 0.05 were excluded to avoid division-by-zero problems; for these cases, only absolute Bias and RMSE were reported. The threshold of |λ| < 0.05 was adopted as a practical cutoff to prevent numerically unstable relative metrics when the true parameter value approaches zero; this decision is consistent with common practice in equating simulation studies [54]. The criterion |RB| > 5%, proposed by Hoogland and Boomsma [55], was used as a threshold for identifying practically significant bias.

Results

This section presents the bias and RMSE values for the A and B equating constants obtained using the SL method across four research questions. The values for all conditions are reported in Table 2, and the differences relative to the reference condition (K1) are presented in Table 3. Relative bias (RB_B) is reported in Table 2 only for conditions in which the true B constant meets the |B_true| ≥ 0.05 threshold; conditions falling below this threshold are marked with a dash (—) to avoid numerically unreliable results, and for these conditions only absolute bias and RMSE are interpreted.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Bias, RMSE, and relative bias of the SL equating constants across 25 simulation conditions.

https://doi.org/10.1371/journal.pone.0353665.t002

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Differences in bias and RMSE relative to the reference condition (K1) across 25 simulation conditions.

https://doi.org/10.1371/journal.pone.0353665.t003

Effect of item parameter drift on bias and RMSE of equating constants

The first research question examines how the rate and magnitude of anchor-item parameter drift affect the accuracy of the A and B constants in the absence of missing data. For this purpose, conditions K1 (control), K2, K3, K4, and K5 were analyzed.

For the A constant (see Table 2), bias values ranged between −0.0043 and 0.0013, and no meaningful bias was observed in any condition. RMSE of constant A increased from 0.0512 in K1 to 0.0634 in K5; however, this increase remained limited. Relative bias values for A did not exceed the 5% threshold in any condition.

A similar pattern was observed for the B constant. Bias values were quite small (−0.0011 to 0.0060), and no statistically significant bias was detected. However, RMSE increased as the magnitude of IPD increased. The value, which was 0.0493 in K1, reached 0.0580 in K5 (IPD rate = 20%, Δb = 0.50). When the delta values in Table 3 are examined, ΔRMSE_B in K5 is 0.0087, corresponding to an increase of approximately 18% compared to K1. Relative bias (RB_B) could only be computed in conditions where the true value of B was meaningful; in those cases, values ranged from −2.25 to −4.78 and did not exceed the 5% threshold. This pattern is also visible in Fig 1. When the missing data rate is 0%, the starting points of all IPD conditions are very close, but the lines begin to diverge as the magnitude of drift increases.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. RMSE of the B Constant Across Missing Data Rates and IPD Conditions.

RMSE values for the B equating constant under MCAR and MAR conditions across varying levels of missing data and IPD.

https://doi.org/10.1371/journal.pone.0353665.g001

Taken together, these findings indicate that item parameter drift alone, at least at the levels examined in this study, does not introduce substantial bias in the A and B constants, but it does increase estimation variability, particularly under moderate drift conditions (Δb = 0.50).

Effect of missing data mechanism and rate on bias and RMSE of equating constants

The second research question focuses on the effects of the missing-data mechanism and rate in the absence of IPD. For this purpose, conditions K6, K7, K8, and K9 were compared with K1. The A constant remained largely stable in the presence of missing data. Bias values ranged from −0.0125 to 0.0018, and RMSE increased only slightly from 0.0512 in K1 to a maximum of 0.0550 in K7 (MCAR 20%) (see Table 2). No clear difference was observed between MCAR and MAR conditions in terms of the A constant.

For the B constant, however, the two mechanisms show a clear divergence. Under MCAR conditions (K6–K7), bias values (0.0027 and 0.0039) remained very close to that of K1 (−0.0007), and RMSE values stayed within a narrow range (0.0490–0.0491). Under MAR conditions, the pattern changes; bias increased to 0.0094 in K8 (MAR 10%) and to 0.0251 in K9 (MAR 20%). The delta values in Table 3 further highlight this difference, with ΔBias = 0.0258 and ΔRMSE = 0.0073 in K9. Since there is no true location difference between forms in conditions without IPD, the true population value of B is zero; therefore, relative bias could not be computed due to division by zero, and interpretations were based on absolute Bias_B values. Fig 1 also visually supports this distinction. In the MCAR panel, the line for the no-IPD condition remains nearly flat as the missing rate increases, whereas in the MAR panel, a clear upward trend is observed.

A similar pattern can be seen in Fig 2, where K7 (MCAR 20%) remains close to the reference line, while K8 and K9 (MAR) show a noticeable increase.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Bias in the B Constant Across Selected MCAR and MAR Conditions.

Bias values of the B equating constant under selected missing data and IPD conditions across MCAR and MAR mechanisms.

https://doi.org/10.1371/journal.pone.0353665.g002

These findings suggest that the MCAR mechanism does not systematically affect the B constant, whereas the MAR mechanism, especially at a 20% missing rate, introduces a notable positive bias. This pattern likely reflects the systematic loss of lower-ability individuals under MAR conditions.

Discussion and conclusion

In this study, the effects of IPD, missing data mechanisms (MCAR, MAR), and missing data rates (10% and 20%) on the accuracy of the SL transformation constants (A and B) in IRT-based test equating were examined through simulation. When the findings were considered in light of the four research questions, it became clear that the A constant remained largely stable across all conditions. The main pattern, however, emerged in the B constant, and was primarily driven by the missing-data mechanism.

With respect to the first research question, no meaningful bias was detected in either the A or B constants under conditions that included only IPD (K1–K5). Although RMSE values increased slightly as the magnitude of IPD increased, this increase did not reach a level of practical concern. This result partially aligns with the patterns reported by Wells et al. [14] and Han et al. [30]. Those studies similarly indicated that IPD alone does not substantially distort equating constants, although estimation errors tend to accumulate as the magnitude and proportion of drift increase. In the present study, the relatively moderate levels of IPD (10% and 20%) and drift magnitudes (0.2–0.5) suggest that deterioration in equating accuracy is largely a function of drift size, with stronger effects likely to emerge at higher levels [32].

Regarding the second research question, MCAR and MAR mechanisms produced clearly different effects on the B constant. Under MCAR conditions, bias values for B remained close to zero. In contrast, under MAR conditions, particularly at the 20% missing rate, a noticeable positive bias was observed. This finding indicates that when missingness is completely random, parameter estimation is largely preserved. However, when missingness is systematically related to ability, the B constant becomes measurably distorted.

Missing data mechanisms are among the key factors influencing equating accuracy. MAR is more complex than MCAR because the probability of missingness depends on the observed variables. When appropriate methods are not employed, this complexity can lead to higher levels of error and bias in parameter estimation. Indeed, the missing-data literature emphasizes that, under MAR conditions, violations of model assumptions or the use of inadequate methods can increase estimation error [19,56]. In this respect, correctly identifying the missing-data mechanism and handling it with appropriate techniques is critical for accurate equating results. The findings of the present study confirm this pattern within the context of the 3PL model and the SL method. Under MAR conditions, the dependence of missingness on individuals’ ability levels can bias both item and ability parameter estimates, ultimately reducing equating accuracy [21].

For the third research question, the combined effect of IPD and missing data produced greater bias than either factor alone. This interaction was especially pronounced in combinations involving MAR and moderate levels of IPD, whereas it remained much more limited in conditions that included MCAR. The highest levels of error were observed in condition K25 (IPD 20%, b = 0.50 × MAR 20%), where the bias for the B constant reached 0.029 and the RMSE value was 0.065. This pattern suggests that uncertainty stemming from two sources may reinforce each other, particularly when missingness is concentrated among lower-performing examinee groups, thereby amplifying the effect of IPD. To the best of our knowledge, no study in the literature directly examines the joint effect of these two factors. Therefore, the present findings underscore the importance of considering sources threatening equating accuracy in combination rather than in isolation.

Regarding the fourth research question, the effectiveness of MI can be indirectly evaluated by comparing MCAR and MAR conditions. Under MCAR, MI largely compensated for the B constant. Under MAR, however, bias continued to increase to a notable extent, especially when moderate levels of missingness and IPD co-occurred. MI, based on Rubin’s [57] rules, is known to improve parameter estimation under ignorable missing-data conditions, yet it does not fully eliminate the effects of systematic missingness patterns. While MCAR does not systematically distort parameter estimates, MAR conditions are known to produce noticeable bias. The present study shows that this general pattern also holds in the context of equating and highlights cases in which MAR-related bias cannot be fully corrected through MICE. These findings suggest that MICE was more effective under MCAR conditions than under MAR conditions, particularly when moderate levels of missingness (20%) and IPD (Δb = 0.50) co-occurred.

Overall, the findings of this study point to three main conclusions. First, the missing-data mechanism, particularly MAR, can introduce systematic bias into equating constants. Second, estimation error increases markedly as the rate of missing data rises. Third, although the effect of IPD alone is limited, it becomes considerably stronger when combined with MAR.

These findings also have several practical implications for operational test equating. First, missing-data patterns in operational assessments should be systematically monitored and documented prior to equating, as the mechanism, not only the rate, determines the extent of equating error. Therefore, in large-scale assessments, missing-data patterns should be systematically monitored and reported. Second, although MICE reduces error under MAR conditions, it does not eliminate it entirely. In the present study, imputation was conducted using item responses only, without incorporating person-level covariates. Drawing on general MI theory, including performance-related auxiliary variables, such as total score, in the imputation model may help reduce bias under MAR conditions by better capturing the cause of missingness [58]; however, the extent to which this approach improves equating accuracy warrants empirical investigation in future studies. Third, in the presence of IPD, the quality control of the anchor item set is critically important. Identifying and removing drifting anchor items prior to equating appears essential for limiting equating error under both MCAR and MAR conditions.

Several limitations of the present study should be acknowledged. First, the MAR mechanism was operationalized by assigning differential missingness probabilities based on examinees’ latent ability (θ). Although this approach is commonly used in IRT simulation studies and reflects realistic omission behavior in testing contexts [15,21], the resulting condition is more appropriately interpreted as a near-MAR mechanism rather than a strictly observed-variable MAR process. Future studies may extend this framework by incorporating directly observed auxiliary variables or by examining fully MNAR conditions.

A second limitation concerns the imputation strategy. In the present study, MICE was implemented using item response data only, without including person-level auxiliary variables such as total score or subgroup membership. Including such variables may improve the plausibility of the MAR assumption and further reduce residual bias under missing data conditions [56,58]. Future research may also consider comparing MICE with no-imputation or complete-case approaches to evaluate the relative effectiveness of different missing data handling strategies in IRT equating contexts.

Additional design assumptions should also be considered when interpreting the generalizability of the findings. First, the guessing parameter (c) was fixed at 0.20 across all items, which simplified estimation but does not reflect the item-level variability in pseudo-guessing typically observed in operational testing programs. Second, item parameter drift was simulated exclusively in the positive direction to facilitate the detectability of equating error and isolate the effect of drift magnitude. Accordingly, the findings may be most applicable to testing conditions in which drift occurs predominantly in a single direction. Future studies should examine how item-level variation in c parameters and bidirectional drift patterns interact with missing data mechanisms to influence equating accuracy.

The present study examined the joint effects of item parameter drift and missing data on IRT-based test equating accuracy within a unified simulation framework. Three main conclusions emerge from the findings. First, the missing data mechanism, rather than the rate of missingness alone, is the primary driver of equating error, with MAR conditions introducing substantial positive bias in the B constant that cannot be fully mitigated by MICE. Second, item parameter drift alone produced relatively limited bias at the levels examined, but amplified equating error considerably when combined with MAR. Third, while MICE effectively reduces error under MCAR, its performance is limited under systematic missingness conditions. Together, these findings underscore the importance of diagnosing missing data mechanisms prior to equating, monitoring anchor item stability, and developing more robust imputation strategies for operational testing programs, particularly in psychological assessment contexts where equating errors may affect individual-level decisions.