Scoping a Frontier Post-Training Pilot: Rubric Schema, Canaries & Sandbox Gate

Never sign a multi-hundred-thousand-dollar post-training SOW based on slide-deck PhD headcounts or cherry-picked marketing examples. Execute a blinded 50-trajectory pilot across 2 to 3 independent labs scored against identical Docker test suites, eloquent-wrong DPO canaries, and keystroke telemetry schemas.

Interactive RLHF Rubric Weight & Pilot Telemetry Schema Builder

Configure your post-training pipeline stage (SFT, DPO, PRM, or RLVR), length-penalty coefficient, and specialist tier to generate a normative pilot evaluation specification.

Post-Training Pipeline Architecture

Verification Domain

Reward Signal HygieneEnforce Length-Bias Decoupling Penalty (−0.35 on padded completions)

SCHEMA_PREVIEW // dpo_pairwise.jsonl

{
  "schema_version": "rlhf-std-2026.11",
  "pipeline_type": "dpo_pairwise",
  "task_domain": "repo_swe",
  "annotator_attestation": {
    "credential_verified": "senior_swe_l6",
    "paste_events_count": 0,
    "llm_entropy_check": "PASS_HUMAN_ORIGIN"
  },
  "prompt": "Refactor async connection pool to prevent cancellation leak under timeout.",
  "chosen": {
    "model_id": "policy_ckpt_a",
    "unit_tests_passed": 14,
    "token_count": 412
  },
  "rejected": {
    "model_id": "policy_ckpt_b",
    "unit_tests_passed": 14,
    "token_count": 940
  },
  "rubric_vector": {
    "correctness_gate": 1,
    "edge_case_coverage": 5,
    "code_idiomaticity": 5,
    "verbosity_penalty_applied": -0.35,
    "preference_margin": "STRONG_CHOSEN (+2)"
  }
}

[01]

PILOT_STEP_01 // ADVERSARIAL_PILOT_SEEDING

1. Seed 50 Frontier-Cliff Prompts Plus 5 Blind Eloquent-Wrong Canaries

Construct your 50-item pilot strictly from failure modes where your current production checkpoint scores `pass@8 <= 0.25`—such as multi-file async race bugs, multi-hop statutory conflicts, or graduate electrodynamics derivations. Easy undergraduate prompts allow unvetted generalist vendors to mask shallow comprehension behind fluent prose.

Embed at least 5 unmarked canary trajectories inside the pilot batch: counterfactual local API specifications for SFT tasks and 'eloquent-wrong vs. terse-correct' pairs for DPO tasks. Require every bidding vendor to return raw per-rater judgments prior to internal QA manager filtering so you measure authentic contributor floor quality.

Demand raw per-annotator multi-rater matrices (3 independent specialists per pilot item) to verify ordinal Krippendorff's alpha >= 0.84 directly.
Inspect `chosen / rejected` token-length ratios across returned DPO pairs; reject vendors whose chosen completions systematically exceed rejected length by > 25%.
Require signed confirmation of 100% cap-table independence from competing foundation model labs prior to transmitting proprietary failure prompts.

[02]

PILOT_STEP_02 // SANDBOX_AND_LOSS_MASKING

2. Require Reproducible OCI Digests and Explicit Assistant Loss Masks

For coding, bash, browser DOM, and SQL/API agent pilots, reject static markdown transcripts that cannot be replayed deterministically. Require every delivered trajectory to reference an immutable Docker image SHA-256 digest, pinned base Git commit, exact tool-call JSON payloads, and out-of-band `FAIL_TO_PASS` test exit codes.

Verify that multi-turn JSONL records explicitly tag token loss masks (`loss_mask: 1` strictly on assistant reasoning and action tokens; `loss_mask: 0` on user prompts, system instructions, and tool stdout/stderr observations) so downstream TRL or Verl training scripts never backpropagate cross-entropy gradients over mock environment outputs.

[03]

PILOT_STEP_03 // TELEMETRY_AND_LINEAGE_SLA

3. Audit Raw Keystroke Telemetry Fields and Enforce Whole-Lineage Rollback

Inspect the `telemetry_attestation` object attached to every pilot row: verify non-degenerate inter-keystroke interval variance (`iki_cv >= 0.52`), authentic human editing ratios (`0.06 <= revision_ratio <= 0.28`), zero untrusted external paste characters, and salted `annotator_id_hash` continuity.

Bind pilot acceptance thresholds directly into your production SOW: any delivery shard dropping below ordinal alpha 0.84, failing > 6% of blind golden canaries, or exhibiting confirmed synthetic LLM assistance triggers automatic whole-contributor lineage replacement at zero buyer cost.

[04]

SAMPLE SPECIFICATION ARTIFACT

Copyable Pilot Specification (rlhf-pilot-trajectory.jsonl)

Issue the identical specification artifact below to 2–3 independent candidates simultaneously so turnaround latency, edge-case adherence, and defect rates are measured on a controlled baseline.

ARTIFACT // rlhf-pilot-trajectory.jsonlJSONL

{"pilot_spec_id":"rlhf-50-trajectory-pilot-v2026.11","task_id":"pilot_swe_rlvr_001","pipeline_mix":["proctored_sft_warmstart","on_policy_dpo_pair","docker_rlvr_verifier"],"annotator_floor":{"credential":"Staff_SWE_or_ICPC_Finalist","live_proctored_onboarding":true,"shadow_dom_telemetry_required":true},"sandbox_spec":{"oci_digest":"ghcr.io/org/repo-eval-env@sha256:4a92f01c88e3b712","network_namespace":"CLONE_NEWNET_AIRGAPPED","reset_sla_p95_ms":140,"out_of_band_tests":["tests/regression/test_pool_cancel.py::test_waiter_wakeup_race"]},"acceptance_gates":{"fail_to_pass_exit_code":0,"pass_to_pass_regression_count":0,"krippendorff_alpha_ordinal_min":0.85,"eloquent_wrong_canary_accuracy_min":0.95,"max_chosen_to_rejected_token_ratio":1.25}}
{"pilot_spec_id":"rlhf-50-trajectory-pilot-v2026.11","task_id":"pilot_stem_prm_002","pipeline_mix":["phd_stepwise_prm","lean4_autoformalization_check"],"annotator_floor":{"credential":"PhD_Mathematics_or_Theoretical_Physics","live_proctored_onboarding":true,"shadow_dom_telemetry_required":true},"sandbox_spec":{"lean_toolchain":"leanprover/lean4:v4.12.0","mathlib_rev":"8b31a92c4e10f7d5","forbid_axioms":["sorryAx","Classical.choice_unverified_override"]},"acceptance_gates":{"kernel_typecheck_exit_code":0,"step_boundary_f1_agreement_min":0.88,"spurious_outcome_rejection_rate":1,"untrusted_clipboard_char_ceiling":0}}

Transmit Post-Training Pilot Specification & Rubric Brief

Submit your target post-training pipeline (SFT, DPO, PRM, RLVR), domain credential floor, container sandbox requirements, and trajectory volume to request calibrated lab pilot quotes.