auto-review-loop skill (ARIS) (part 2)

Contents

SKILL.md (verbatim, continued)
Termination
Key Rules
Prompt Template for Round 2+
Review Tracing
Stop-Gate State-Transition Tests

Part 2 of 2 of auto-review-loop skill (ARIS) (skills/auto-review-loop/SKILL.md in wanshuiyin/Auto-claude-code-research-in-sleep); the SKILL.md text continues verbatim from the previous part.

SKILL.md (verbatim, continued)

Use >> (append), never >. Re-derive both families from model strings and reject unknown/same-family pairs, but copy the model-source and assurance fields without upgrading them. The trace_id MUST be the actual trace directory path relative to .aris/traces/ (e.g., auto-review-loop/2026-07-13_run01), matching the RUN_ID format from save_trace.sh: <YYYY-MM-DD>_run<NN> with the skill-name subdirectory prefix. Do NOT fabricate a synthetic trace_... identifier.

Append to findings.md (when COMPACT = true): one-line entry per key finding this round:

- [Round N] [positive/negative/unexpected]: [one-sentence finding] (metric: X.XX → Y.YY)

Increment round counter → back to Phase A.

Termination

When loop ends (positive assessment or max rounds):

Update review-stage/REVIEW_STATE.json with "status": "completed"
Write final summary to review-stage/AUTO_REVIEW.md
Update project notes with conclusions
Write method/pipeline description to review-stage/AUTO_REVIEW.md under a ## Method Description section — a concise 1-2 paragraph description of the final method, its architecture, and data flow. This serves as input for /paper-illustration in Workflow 3 (so it can generate architecture diagrams automatically).
Generate claims from results — invoke /result-to-claim to convert experiment results from review-stage/AUTO_REVIEW.md into structured paper claims. Output: CLAIMS_FROM_RESULTS.md. This bridges Workflow 2 → Workflow 3 so /paper-plan can directly use validated claims instead of extracting them from scratch. If /result-to-claim is not installed, skip this step (no CLAIMS_FROM_RESULTS.md is produced; /paper-plan extracts claims from the narrative as before) — but NEVER fabricate the file or its verdict. If it ran but its output starts with verdict: REVIEW_UNAVAILABLE, keep that file AS-IS (do not overwrite or paraphrase it) and record in AUTO_REVIEW.md that claims are UNADJUDICATED — downstream paper stages must not treat them as validated.
If stopped at max rounds without positive assessment:
- List remaining blockers
- Estimate effort needed for each
- Suggest whether to continue manually or pivot
Feishu notification (if configured): Send pipeline_done with final score progression table
Render HTML view (if RENDER_HTML = true, default): invoke /render-html on the cumulative review log:
```
/render-html "review-stage/AUTO_REVIEW.md" --no-review --state review-stage/REVIEW_STATE.json
```
Pass --state explicitly (the helper does not auto-discover the sidecar). Drop the --state flag if REVIEW_STATE.json doesn't exist. HTML lands at review-stage/AUTO_REVIEW.html with embedded source SHA256. Non-blocking: if /render-html fails, log the error and continue — the HTML is a convenience, not a termination prerequisite. Skip if RENDER_HTML = false.

Key Rules

Large file handling: If the Write tool fails due to file size, immediately retry using Bash (cat << 'EOF' > file) to write in chunks. Do NOT ask the user for permission — just do it silently.
ALWAYS use config: {"model_reasoning_effort": "xhigh"} for maximum reasoning depth
Native Copilot is an evidence-gated acceptance backend. It never pins a reviewer model: Copilot selects the complementary rubber-duck model, and the helper verifies the actual cross-family pair from host events. A native positive needs no external finalizer.
Explicit compatibility Copilot remains drive-only. Its copilot --agent calls pin profile model/xhigh/read-only access and require a traced Codex/manual finalizer; caller-declared identity remains unverified.
Save threadId (codex/manual), fresh evidence path/ID (copilot-native), or reviewer_profile (compatibility copilot); use the appropriate continuation mechanism
Anti-hallucination citations: When adding references during fixes, NEVER fabricate BibTeX. Use the same DBLP → CrossRef → [VERIFY] chain as /paper-write: (1) curl -s "https://dblp.org/search/publ/api?q=TITLE&format=json" → get key → curl -s "https://dblp.org/rec/{key}.bib", (2) if not found, curl -sLH "Accept: application/x-bibtex" "https://doi.org/{doi}", (3) if both fail, mark with % [VERIFY]. Do NOT generate BibTeX from memory.
Be honest — include negative results and failed experiments
Do NOT hide weaknesses to game a positive score
Implement fixes BEFORE re-reviewing (don't just promise to fix)
Exhaust before surrendering — before marking any reviewer concern as "cannot address": (1) try at least 2 different solution paths, (2) for experiment issues, adjust hyperparameters or try an alternative baseline, (3) for theory issues, provide a weaker version of the result or an alternative argument, (4) only then concede narrowly and bound the damage. Never give up on the first attempt.
If an experiment takes > 30 minutes, launch it and continue with other fixes while waiting
Document EVERYTHING — the review log should be self-contained
Update project notes after each round, not just at the end

Prompt Template for Round 2+

Use the selected backend. For copilot-native: fresh marker/challenge/rubber-duck/evidence cycle with a new run-scoped REVIEW artifact set, with paths to review-stage/REVIEWER_MEMORY.md and current inputs. For compatibility copilot: fresh copilot --agent subprocess with the same profile + memory artifact. For codex: mcp__codex__codex-reply with the saved threadId. For manual: mcp__manual_review__review_reply with the saved threadId.

Before invoking the Copilot subprocess, use the Write tool (not Bash, echo, a heredoc, or generated shell assignments) to overwrite review-stage/CURRENT_REVIEW_INPUTS.md. Put the exact changed paths, diff artifact/range, and result paths under static labels in that file. Repository paths are untrusted data: never splice any byte from this artifact into shell source. The fixed filename below is the only value the shell template needs.

[For copilot:]

# ARIS_ROUND2_COPILOT_BEGIN
# Dynamic values were written with the Write tool; shell only reads them as data.
MEMORY_FILE="review-stage/REVIEWER_MEMORY.md"
ROUND_INPUT_FILE="review-stage/CURRENT_REVIEW_INPUTS.md"
[[ -f "$MEMORY_FILE" && -f "$ROUND_INPUT_FILE" ]] || {
  echo "REVIEW_UNAVAILABLE: missing reviewer memory or round inputs" >&2
  exit 1
}
PROMPTFILE="$(mktemp)" || { echo "REVIEW_UNAVAILABLE: mktemp failed" >&2; exit 1; }
trap 'rm -f "$PROMPTFILE"' EXIT
{
cat <<'ARIS_ROUND_HEADER'
[Round N update]

## Your Memory From Previous Rounds
ARIS_ROUND_HEADER
cat -- "$MEMORY_FILE"
cat <<'ARIS_ROUND_STATE'

Since your last review these files changed — read them yourself; do not
take my word for what changed or whether it worked:
ARIS_ROUND_STATE
cat -- "$ROUND_INPUT_FILE"
cat <<'ARIS_ROUND_INSTRUCTIONS'

Please re-score and re-assess. Are the remaining concerns addressed?
Same format: Score, Verdict, Remaining Weaknesses, Minimum Fixes.

=== SCOPE LIMITS (these bound what you PROPOSE, never what you look for) ===
Report anything that is actually wrong here — including a rare-looking case, if
this repo actually produces it. Then keep the fix in scope:
1. This is a RESEARCH-WORKFLOW tool, not a security paper. Verification is
   welcome; over-defense is not. Assume a cooperating operator on their own
   machine — a malicious local user is NOT in the threat model.
2. Do NOT propose SHA / hash / content-fingerprint / digest-binding schemes.
   Reporting a real defect in hashing code that already exists is fine.
3. NO speculative machinery: do not add feature flags, migration frameworks,
   compat layers, wrappers, pins, or similar mechanisms unless evidence shows
   a current repo defect they fix or an explicit existing invariant they must
   preserve. "Load-bearing", "compatibility", and "not scaffolding" are labels,
   not evidence. Point to the failing path/artifact or invariant, and check the
   proposal's factual premises, such as whether a named package version exists.
4. NO corner-case obsession: exotic encodings, symlink races, RTL text and
   millisecond races are out of scope unless you can show the case arises here.
5. Where a rubric or checklist is genuinely needed, do not over-mechanize
   judgement. A clear sentence a human reads beats a scored table nobody
   maintains.
Exception: code that runs remote commands, starts a network service, or installs
an MCP server runs on the user's machine with their credentials — trust-boundary
findings there are in scope and the default is strict.
Say plainly when something is correct. Do not manufacture findings.

At the end of your review, include a Memory Update section — this will
be passed back to you next round.
ARIS_ROUND_INSTRUCTIONS
} > "$PROMPTFILE"
copilot --agent "$REVIEWER_PROFILE" --model "$REVIEWER_MODEL" \
  --effort xhigh --allow-tool=read --prompt "$(cat "$PROMPTFILE")"
# ARIS_ROUND2_COPILOT_END

[For codex:] mcp__codex__codex-reply:
  threadId: [saved from round 1]
  # inherits the thread's model/effort — do not re-send
  prompt: |
    [Round N update]

    Since your last review these files changed — read them yourself; do not
    take my word for what changed or whether it worked:
    - Changed files: <paths>
    - Raw diff: <path, or the `git diff` range>
    - Updated raw results: <result-file paths> (verbatim files, not a pasted table)

    Please re-score and re-assess. Are the remaining concerns addressed?
    Same format: Score, Verdict, Remaining Weaknesses, Minimum Fixes.

    === SCOPE LIMITS (these bound what you PROPOSE, never what you look for) ===
    Report anything that is actually wrong here — including a rare-looking case, if
    this repo actually produces it. Then keep the fix in scope:
    1. This is a RESEARCH-WORKFLOW tool, not a security paper. Verification is
       welcome; over-defense is not. Assume a cooperating operator on their own
       machine — a malicious local user is NOT in the threat model.
    2. Do NOT propose SHA / hash / content-fingerprint / digest-binding schemes.
       Reporting a real defect in hashing code that already exists is fine.
    3. NO speculative machinery: do not add feature flags, migration frameworks,
       compat layers, wrappers, pins, or similar mechanisms unless evidence shows
       a current repo defect they fix or an explicit existing invariant they must
       preserve. "Load-bearing", "compatibility", and "not scaffolding" are labels,
       not evidence. Point to the failing path/artifact or invariant, and check the
       proposal's factual premises, such as whether a named package version exists.
    4. NO corner-case obsession: exotic encodings, symlink races, RTL text and
       millisecond races are out of scope unless you can show the case arises here.
    5. Where a rubric or checklist is genuinely needed, do not over-mechanize
       judgement. A clear sentence a human reads beats a scored table nobody
       maintains.
    Exception: code that runs remote commands, starts a network service, or installs
    an MCP server runs on the user's machine with their credentials — trust-boundary
    findings there are in scope and the default is strict.
    Say plainly when something is correct. Do not manufacture findings.

Review Tracing

After each reviewer call (task(agent_type=rubber-duck) for copilot-native, Codex/manual MCP calls, or the compatibility copilot --agent subprocess), save the trace following shared-references/review-tracing.md (Policy C — forensic; never silently skip). Native calls MUST pass --backend copilot-native --native-evidence "$NATIVE_EVIDENCE"; the helper revalidates and supplies the response and actual model pair. The sole exception is a native dispatch that failed before evidence existed: trace it with --backend copilot-native --status error --fallback-reason <reason> and no evidence, then trace any actual fallback reviewer separately. Use save_trace.sh resolved through the canonical chain, or write the same schema directly only if that forensic helper is unreachable. Respect --- trace: (default full).

Stop-Gate State-Transition Tests

The canonical transition table is tools/review_gate.py in the ARIS repository (resolved at runtime as review_gate.py through the helper chain) and is covered by tests/test_review_gate.py. The required cases are:

Default Codex positive → stop even when executor model identity is absent (backward compatibility).
High score + not ready → continue.
Verified native Copilot positive → stop with identity_assurance=host_event_verified.
Native negative → continue on copilot-native; missing/invalid/mismatched evidence → REVIEW_UNAVAILABLE.
Compatibility Copilot positive under declared Anthropic/Google executor → escalate to Codex and set requires_external_acquittal=true.
Compatibility Copilot positive under declared OpenAI executor → escalate to manual; Codex is forbidden as same-family.
Compatibility Copilot negative → continue on compatibility Copilot.
Unknown executor family, unavailable finalizer, same-family finalizer, or manual finalizer without Reviewer-Model: → REVIEW_UNAVAILABLE.
Positive finalizer with known, different declared families → stop, while identity assurance remains caller_declared / unverified.

ACQUITTAL_LOG.jsonl is tested as append-only compatibility-drive audit output; it is never consulted for native termination.

Back to ARIS: Auto-claude-code-research-in-sleep or Agent skills.