Build OpenAI batch JSONL lines for paired comparisons

This helper constructs one JSON object per pair of writing samples, suitable for use with the OpenAI batch API. It supports both /v1/chat/completions and /v1/responses endpoints.

Usage

build_openai_batch_requests(
  pairs,
  model,
  trait_name,
  trait_description,
  prompt_template = set_prompt_template(),
  endpoint = c("chat.completions", "responses"),
  temperature = NULL,
  top_p = NULL,
  logprobs = NULL,
  reasoning = NULL,
  include_thoughts = FALSE,
  request_id_prefix = "EXP",
  store = NULL,
  max_output_tokens = NULL,
  prompt_caching = NULL,
  ...
)

Arguments

pairs: A data frame or tibble with columns ID1, text1, ID2, and text2.
model: Character scalar giving the OpenAI model name. Supports standard names (e.g. "gpt-4.1", "gpt-5.6-sol") and date-stamped versions (e.g. "gpt-5.4-2026-01-15").
trait_name: Short label for the trait (e.g., "Overall Quality").
trait_description: Full-text definition of the trait.
prompt_template: Character template containing the placeholders {TRAIT_NAME}, {TRAIT_DESCRIPTION}, {SAMPLE_1}, and {SAMPLE_2}. Defaults to set_prompt_template().
endpoint: Which OpenAI endpoint to target. One of "chat.completions" (default) or "responses".
temperature: Optional temperature parameter. If NULL, it is omitted so the model/provider default applies. Must be NULL for reasoning modes that do not support it.
top_p: Optional top-p parameter. If NULL, it is omitted so the model/provider default applies.
logprobs: Optional logprobs parameter.
reasoning: Optional reasoning effort for GPT-5 series when using the /v1/responses endpoint. For "gpt-5" and "gpt-5-mini", "none" is normalized to "minimal". For later GPT-5.x reasoning models, use model-supported efforts such as "none", "low", "medium", "high", "xhigh", or "max".
include_thoughts: Logical; if TRUE and using responses endpoint with reasoning, requests a summary. Defaults reasoning to "low" for GPT-5 series models if not specified.
request_id_prefix: String prefix for custom_id; the full ID takes the form "<prefix>_<ID1>_vs_<ID2>".
store: Optional logical scalar for either endpoint. TRUE and FALSE are forwarded unchanged; omitted or NULL leaves the field absent.
max_output_tokens: Optional positive whole numeric scalar, at most .Machine$integer.max, for the Responses endpoint only. Includes visible output and reasoning tokens. Omitted or NULL leaves the field absent.
prompt_caching: Batch caching policy: NULL (model-aware default), "disabled", or "implicit". See the compatibility rules below.
...: Reserved; must be empty. Raw provider caching controls are not accepted. Use prompt_caching instead.

Value

A tibble with one row per pair and columns:

custom_id: ID string used by the batch API.
method: HTTP method ("POST").
url: Endpoint path ("/v1/chat/completions" or "/v1/responses").
body: List column containing the request body.

Details

Invalid controls fail even for empty pairs. Responses are stored for later API retrieval by default; set store = FALSE to disable response storage. For Chat Completions, store controls storage for distillation or evaluations. Omitting it preserves that endpoint's provider default. This parameter does not control Batch input/output/error file retention and does not imply zero data retention. See https://developers.openai.com/api/docs/guides/your-data.

Batch prompt caching

The default disables prompt caching for these verified model IDs: gpt-5.6-luna, gpt-5.6-terra, gpt-5.6-sol, gpt-6-luna, gpt-6-sol, gpt-6-astra, and gpt-6.1-sol. Each request body gets prompt_cache_options = list(mode = "explicit"), with no breakpoints, on both endpoints. Set prompt_caching = "implicit" to omit that field and retain provider caching when prefix reuse is expected to be worthwhile.

Recognized earlier OpenAI models retain their historical request bodies when the policy is omitted or NULL; explicitly disabling caching on those models errors. Unknown IDs (including unlisted snapshots, future models and custom aliases) require an explicit "implicit" opt-in or a verified model ID. Support is an offline allowlist; version numbers and snapshot suffixes never establish support for disabling caching. Older GPT-3.5/4, GPT-5 through GPT-5.5 and o-series naming forms, including recognized legacy date-suffixed names, retain provider defaults.

Invalid policies and manual cache controls fail before submission, even with zero pairs. Raw prompt_cache_options, prompt_cache_breakpoint, prompt_cache_key, retention and TTL controls are not accepted by these builders. Keys affect cache routing/accounting; retention and TTL affect lifetime; store = FALSE affects response storage. None substitutes for disabling implicit cache breakpoints. Prompt text and keys are not rewritten. This policy affects future package-built Batch requests only, not live requests or already submitted jobs. See the prompt-caching guide and vignette("advanced-batch-workflows") for usage-based cost comparisons.

Examples

data("example_writing_samples", package = "pairwiseLLM")

pairs <- example_writing_samples |>
  make_pairs() |>
  sample_pairs(n_pairs = 3, seed = 123) |>
  randomize_pair_order(seed = 456)

td <- trait_description("overall_quality")
tmpl <- set_prompt_template()

# 1. Basic chat.completions batch with no thoughts
batch_tbl_chat <- build_openai_batch_requests(
  pairs             = pairs,
  model             = "gpt-4.1",
  trait_name        = td$name,
  trait_description = td$description,
  prompt_template   = tmpl,
  endpoint          = "chat.completions"
)

# 2. GPT-5.6 Sol Responses Batch with Reasoning
batch_tbl_resp <- build_openai_batch_requests(
  pairs = pairs,
  model = "gpt-5.6-sol",
  trait_name = td$name,
  trait_description = td$description,
  prompt_template = tmpl,
  endpoint = "responses",
  include_thoughts = TRUE, # implies reasoning="low" if not set
  reasoning = "medium",
  store = FALSE,
  max_output_tokens = 1024
)

batch_tbl_chat
#> # A tibble: 3 × 4
#>   custom_id      method url                  body            
#>   <chr>          <chr>  <chr>                <list>          
#> 1 EXP_S17_vs_S12 POST   /v1/chat/completions <named list [2]>
#> 2 EXP_S19_vs_S15 POST   /v1/chat/completions <named list [2]>
#> 3 EXP_S01_vs_S15 POST   /v1/chat/completions <named list [2]>
batch_tbl_resp
#> # A tibble: 3 × 4
#>   custom_id      method url           body            
#>   <chr>          <chr>  <chr>         <list>          
#> 1 EXP_S17_vs_S12 POST   /v1/responses <named list [6]>
#> 2 EXP_S19_vs_S15 POST   /v1/responses <named list [6]>
#> 3 EXP_S01_vs_S15 POST   /v1/responses <named list [6]>