Skip to contents

Resume polling and download results for multiple batch jobs

Source: R/llm_multi_batch.R
llm_resume_multi_batches.Rd

This function takes the output of llm_submit_pairs_multi_batch() (or a previously written registry CSV) and polls each batch until completion, downloading and parsing results as they finish. It implements a conservative polling loop with a configurable interval between rounds and a small delay between individual jobs to reduce the risk of API rate‑limit errors. Each retrieval GET has a bounded HTTP retry budget. Exhausted transient retrievals leave the job unfinished for the next polling round; permanent HTTP errors, parsing errors and local file errors propagate immediately. There is no overall round limit. Retrieval retries do not resubmit comparisons or add scientific failed-attempt rows.

Usage

llm_resume_multi_batches(
  jobs = NULL,
  output_dir = NULL,
  interval_seconds = 60,
  per_job_delay = 2,
  write_results_csv = FALSE,
  keep_jsonl = TRUE,
  write_registry = FALSE,
  tag_prefix = "<BETTER_SAMPLE>",
  tag_suffix = "</BETTER_SAMPLE>",
  verbose = FALSE,
  write_combined_csv = FALSE,
  combined_csv_path = NULL,
  openai_max_retries = 3
)

Arguments

jobs

A list of job objects returned by llm_submit_pairs_multi_batch(). If NULL, a registry CSV is loaded from output_dir and converted into an internal jobs structure.

output_dir

Directory containing the batch files and (optionally) the registry CSV. If jobs is NULL, this directory must be supplied so that the registry can be loaded. When jobs is provided and output_dir is NULL, the directory is inferred from the first job’s batch_output_path. When writing results CSVs or updating the registry, this directory is used.

interval_seconds

Number of seconds to wait between rounds of polling unfinished batches. The default (60) mirrors the example in the advanced vignette.

per_job_delay

Number of seconds to wait between polling individual jobs within a single round. A small delay (e.g. 2) can help prevent 429 (Too Many Requests) responses.

write_results_csv

Logical; if TRUE, each batch’s parsed results are written to a CSV file (csv_path) in output_dir as soon as they are available. If FALSE (the default), results are kept in memory.

keep_jsonl

Logical; if FALSE, the .jsonl input and output files will be deleted after the job results have been parsed. Defaults to TRUE.

write_registry

Logical; if TRUE, a CSV registry of batch jobs will be written (or updated) at the end of polling. When reading jobs from a saved registry via output_dir, this argument can be used to control whether the registry is refreshed on disk as job statuses change. Defaults to FALSE. See llm_submit_pairs_multi_batch() for additional details on the registry format.

tag_prefix, tag_suffix

Character strings passed to parse_anthropic_batch_output() and parse_gemini_batch_output(). These tags mark the start and end of the “better” sample in each provider’s output. The defaults match those used in the vignette.

verbose

Logical; if TRUE, prints progress messages during polling and result processing. Messages include the batch ID, provider, and current state on each polling round, as well as summary messages when combined results are written to disk. Defaults to FALSE.

write_combined_csv

Logical; if TRUE, the combined results tibble returned by the function will also be written to a CSV file. The path to write this file is determined by combined_csv_path. Defaults to FALSE.

combined_csv_path

Optional file path for the combined results CSV. If write_combined_csv = TRUE and combined_csv_path is NULL, the combined results will be written to file.path(output_dir, "combined_results.csv"). When a non‑NULL value is supplied, it is treated as an absolute path if it begins with “/”, “~/”, or a Windows drive letter (e.g. “C:”), or if it contains a directory component (i.e. dirname(combined_csv_path) != "."). In that case it will be used exactly as given. Otherwise the file name is assumed to be relative to output_dir. This argument is ignored when write_combined_csv = FALSE.

openai_max_retries

Positive integer giving the total HTTP attempt budget per GET when downloading OpenAI batch output, including its metadata lookup. Defaults to 3. Retries honor Retry-After or use exponential backoff with jitter; per_job_delay controls spacing between jobs, not HTTP retries. There is no additional outer download retry loop. Status polling and other providers' retrievals use three HTTP attempts per GET.

Value

A list with four elements: jobs, the updated jobs list with each element containing parsed results and a done flag; combined, a tibble obtained by binding all completed results (NULL if no batches completed); failed_attempts, a tibble of failed attempts captured during normalization; and batch_failures, a tibble describing batches that reached a terminal non-success status. If write_results_csv is TRUE, the combined tibble is still returned in memory. If write_combined_csv is TRUE, the combined tibble is also written to a CSV file on disk (see combined_csv_path for details) but is still returned in memory.

See also

llm_submit_pairs_batch(), llm_download_batch_results()

Other batch backends: anthropic_create_batch(), anthropic_download_batch_results(), anthropic_get_batch(), anthropic_poll_batch_until_complete(), build_anthropic_batch_requests(), build_gemini_batch_requests(), build_openai_batch_requests(), gemini_create_batch(), gemini_download_batch_results(), gemini_get_batch(), gemini_poll_batch_until_complete(), llm_download_batch_results(), llm_submit_pairs_batch(), llm_submit_pairs_multi_batch(), openai_create_batch(), openai_download_batch_errors(), openai_download_batch_output(), openai_get_batch(), openai_poll_batch_until_complete(), openai_upload_batch_file(), run_anthropic_batch_pipeline(), run_gemini_batch_pipeline(), run_openai_batch_pipeline(), write_openai_batch_file()

Examples

# Continuing the example from llm_submit_pairs_multi_batch():
# After submitting multiple batches, resume polling and combine the results.
if (FALSE) { # \dontrun{
# Suppose `outdir` is the directory where batch files were written and
# `jobs` is the list of job metadata returned by llm_submit_pairs_multi_batch().

results <- llm_resume_multi_batches(
  jobs               = jobs,
  output_dir         = outdir,
  interval_seconds   = 60,
  per_job_delay      = 2,
  write_results_csv  = TRUE,
  keep_jsonl         = FALSE,
  write_registry     = TRUE,
  verbose            = TRUE,
  write_combined_csv = TRUE
)

# The combined results are available in the `combined` element
print(results$combined)
} # }