Guide: Adaptive Linking

Linking helps you compare writing samples that were originally ranked in separate sets. For example, you might have ranked last year’s samples and this year’s samples separately. Each ranking describes differences within its own set. Comparisons between the two sets are needed to put their scores on a common scale.

The reference set is called the hub; each set you want to connect to it is a spoke. Phase A means ranking each set separately. Phase B means using comparisons between sets to link their scores.

In version 1.6.0, you supply the between-set comparisons yourself and choose one of three linking methods. Automatic selection of Phase B comparisons is unavailable while it awaits separate validation. Ordinary adaptive ranking within a set remains available. The linking sessions guide walks through a complete example using invented results, without an API key or provider request.

Choose what to retain from Phase A

Keep the original Phase A results for both sets, including sample IDs and the settings used to fit them. The three methods need different parts of those results:

Method	What you need from each set	What the method does
E1: fixed_shape_offset	Estimated scores	Keeps within-set score differences fixed and estimates how far to shift the spoke
E2: gaussian_posterior_bridge	Saved joint posterior draws	Uses the saved uncertainty to update both sets and their separation
E3: joint_offset	Original comparison outcomes and compatible original model/prior settings	Fits both sets and their separation together, using each comparison once

Posterior draws are saved sets of plausible scores from a Bayesian fit. E2 needs them together because uncertainty in one sample’s score is related to uncertainty in other scores. A table of scores and separate standard errors is not enough. E3 needs the original outcomes because it rebuilds the fit from those comparisons.

There is no default linking method yet. The study will determine that choice. The examples choose a method to show how to use it, without recommending a winner. The design guide explains the statistical differences.

Prepare your comparisons

Choose one writing trait, such as organization, and use compatible Phase A models for both sets. Keep the same shared judge settings for linking: beta describes presentation-order bias and epsilon describes random lapses. These settings come from Phase A and are held fixed during linking.

Record each between-set judgment with the IDs of the samples shown as A and B, the set each belongs to, and the winner. Give every judgment its own observation_id. If a pair was judged more than once, retain separate rows and separate judgment IDs. Do not accidentally import the same judgment twice.

Keep comparisons reserved for checking accuracy in a separate table. Only pass comparisons intended for fitting in cross. A session label such as "probe" does not exclude observations from the fit.

Fit and inspect the link

1. Use prepare_link_input() to supply both sets, their Phase A information, the between-set comparisons and your chosen method.
2. Use fit_link() for a single fit, or start_link_session() if you want to save the work and add comparisons later.
3. Inspect the offset, scores, uncertainty and fit diagnostics before using results.

The offset is the estimated shift from the hub to the spoke. Positive values place the spoke higher on the common scale. Without any between-set comparisons, the offset reflects only the starting assumption, not an established link. By default that assumption is a Normal distribution with mean 0 and SD 5.

E1’s uncertainty describes the shift while treating the earlier scores as fixed. It does not include uncertainty in those earlier rankings. E2 and E3 include uncertainty in both the scores and the shift, using approximations explained in the design guide. Missing uncertainty is reported as NA; a failed fit should not be treated as a completed link.

Save, add comparisons, and reuse a reference

Use save_link_session() and load_link_session() to save and restore your work. To add comparisons, keep all previous rows in their original order, append the new rows, prepare the updated input, and call resume_link_session(). Reusing unchanged input returns the saved fit. Changing the method, Phase A results, judge settings or numerical settings requires a new session.

Several spokes can share one hub. Each link is fitted separately, and E2/E3 may update the hub scores differently for each spoke. Reports identify the spoke so you can keep those results separate.

To reuse human rubric scores from the hub, see the rubric calibration guide. The saved calibration stays attached to the original reference set; new target labels are not needed to apply it.

Restart a session from an older version

Version 1.6.0 removes the anchored-joint linking method. Saved Phase B sessions from that method cannot be resumed. The error message asks you to restart from the original Phase A results.

Choose E1, E2 or E3, recover the Phase A information it requires, and start a fresh session. You can reuse compatible historical Phase A artifacts by passing list(artifact = artifact) for each set. Supply the original between-set judgments intentionally as cross. An already-linked Phase B result cannot replace an original Phase A result. Missing draws or original comparisons must be recovered from their source.

Citation

Mercer, S. H. (2026). Guide: Adaptive Linking [R package vignette]. Comprehensive R Archive Network. https://doi.org/10.32614/CRAN.package.pairwiseLLM