Post-Training, Reward Modeling & RLVR Engineering Specification Library
Eight deep alignment engineering specifications authored for post-training researchers—covering SFT vs. DPO/KTO scaling dynamics, atomic length-penalized reward rubrics, containerized SWE-bench repository trajectories, stepwise STEM PRMs and Lean 4 formal verification, stateful RLVR multi-turn gyms, anti-LLM keystroke telemetry, multi-turn CBRN/cyber red-teaming with benign contrast twins, and multilingual tokenizer-fertility alignment.
SFT, DPO, PRM & RLVR Engineering Guides
SFT Expert Demonstrations vs. Pairwise Preference Optimization (DPO, KTO & SimPO)
Alignment engineering specification comparing PhD/SWE SFT demonstrations, DPO, KTO, and SimPO across KL budgets, on-policy sampling, and token scaling.
[OPEN_SPECIFICATION →]Designing Atomic RLHF Grading Rubrics and Calibrated Reward Signals
Normative rubric specification decomposing RLHF preference pairs into atomic binary axes, lexicographic vetoes, length-bias penalties, and PRM rewards.
[OPEN_SPECIFICATION →]Repository-Level Coding and SWE-bench Training Trajectories
Engineering guide to containerized SWE-bench training trajectories, fail-to-pass unit test harnesses, Git commit mining, and multi-file patch rubrics.
[OPEN_SPECIFICATION →]STEM Reasoning, Process Reward Models, and Lean 4 Formal Verification Datasets
Normative specification for step-level Process Reward Models (PRMs), Lean 4 formal autoformalization corpora, and competition STEM RLVR verifiers.
[OPEN_SPECIFICATION →]Verifiable RL Environments and Multi-Turn Agent Trajectories
Normative engineering guide to stateful RLVR tool-use gyms, deterministic reward functions for GRPO and PPO, environment resets, and trajectory pruning.
[OPEN_SPECIFICATION →]Preventing LLM-Assisted Cheating in Human RLHF Annotation
Technical specification for blocking LLM contamination in RLHF workflows using keystroke telemetry, DOM isolation, prompt canaries, and perplexity audits.
[OPEN_SPECIFICATION →]Adversarial Red-Teaming and Safety Alignment Datasets
Engineering multi-turn adversarial attack trees, expert CBRN and cyber red-teaming corpora, benign over-refusal calibration, and Pareto safety rubrics.
[OPEN_SPECIFICATION →]Multilingual and Low-Resource RLHF Alignment
Why machine-translated preference pairs degrade fluency and pragmatics, plus native specialist curation, reward transfer traps, and tokenizer fertility.
[OPEN_SPECIFICATION →]