Skip to content

docs(skills): fix residual self-improvement loop inconsistencies#285

Open
rachaelrenk wants to merge 2 commits into
mainfrom
docs/factory-residual-cleanup
Open

docs(skills): fix residual self-improvement loop inconsistencies#285
rachaelrenk wants to merge 2 commits into
mainfrom
docs/factory-residual-cleanup

Conversation

@rachaelrenk

Copy link
Copy Markdown
Contributor

Summary

Small follow-up to the docs self-improvement loop work (#239) and the aeo_crosslink_audit reliability fix (#280). A retrospective of the "software factory" surfaced a few residual inconsistencies; this PR cleans them up. No changes to published docs — all edits are confined to .agents/.

Changes

.agents/skills/improve-drafting-skills/SKILL.md

  • Stop committing signal logs to protected main. Steps A and B previously instructed committing pr_review_runs.md and human_review_feedback.jsonl directly to main with a silent "if the push fails, continue" fallback — the exact failure mode docs(skills): make aeo_crosslink_audit reliable #280 fixed for aeo_crosslink_audit (protected main rejects the push, so the "durable" log never persists). Both now persist through a standing chore/drafting-signal-logs branch and one perpetual, low-noise PR, mirroring the pattern from docs(skills): make aeo_crosslink_audit reliable #280.
  • Added a "Persisting the signal logs" subsection that describes the standing-branch flow once, referenced from Steps A and B.
  • Noted in the analysis step that prior-run human feedback lives on the standing branch (read it there, or merge the standing log PR first).
  • Removed the leftover self-correcting sentence in the Run log section and its stale style_lint_runs.jsonl reference.

.agents/logs/style_lint_runs.jsonl (deleted)

  • Orphaned empty file. Since the design moved to [SIGNAL:style-lint] stdout markers (collected via oz run get), nothing writes this file. Removed to match the current design.

Not changed (intentionally)

Validation

  • No remaining style_lint_runs references repo-wide.
  • No remaining "commit directly to main" instructions in improve-drafting-skills.
  • git diff --check clean; skill frontmatter parses.

Conversation: https://staging.warp.dev/conversation/a8bc7071-0fda-470a-8f13-49991355164a

Co-Authored-By: Oz oz-agent@warp.dev

rachaelrenk and others added 2 commits July 2, 2026 12:57
- improve-drafting-skills: persist pr_review_runs.md and
  human_review_feedback.jsonl through a standing chore/drafting-signal-logs
  branch + one perpetual PR instead of committing to protected `main`
  (a direct push fails silently — the same failure mode #280 fixed for
  aeo_crosslink_audit)
- Remove the orphaned .agents/logs/style_lint_runs.jsonl (nothing writes it
  since style-lint signals moved to [SIGNAL:style-lint] stdout markers) and
  its stale reference
- Clean up the leftover self-correcting sentence in the Run log section

Co-Authored-By: Oz <oz-agent@warp.dev>
@vercel

vercel Bot commented Jul 2, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs Ready Ready Preview, Comment Jul 2, 2026 7:01pm

Request Review

@cla-bot cla-bot Bot added the cla-signed label Jul 2, 2026
@oz-for-oss

oz-for-oss Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

@rachaelrenk

I'm starting a first review of this pull request.

You can view the conversation on Warp.

I completed the review and no human review was requested for this pull request.

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

@oz-for-oss oz-for-oss Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overview

This PR updates the improve-drafting-skills workflow to persist drafting signal logs through a standing branch/PR and removes the obsolete style_lint_runs log. The direction matches the stated goal, but the new standing-branch flow leaves correctness gaps that can still drop signals or mix monthly improvement edits into the log PR.

Concerns

  • The persistence helper checks out the standing log branch but never returns to a clean improvement branch before later skill/template edits.
  • The analysis step still only names Step A in-memory records plus on-disk human feedback, so Step B's failure fallback can discard current-run human feedback.

Verdict

Found: 0 critical, 2 important, 0 suggestions

Request changes

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

4. Push the branch.
5. Ensure exactly one open PR exists from `chore/drafting-signal-logs` into `main`, titled `chore: drafting signal logs`. Create it if missing; otherwise the push updates the existing PR. Keep this log PR separate from the drafting-skills improvement PR.

This produces one perpetual, low-noise PR that accumulates every run's log entries regardless of outcome. Reviewers merge it periodically (at minimum before each monthly run) so the logs reach `main`. If any git step fails, keep the in-memory records for this run's analysis and note the failure in the Slack summary.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [IMPORTANT] The helper checks out chore/drafting-signal-logs but never switches back before the monthly run drafts skill/template edits, so those edits can land on the standing log branch and pollute the log PR.

Suggested change
This produces one perpetual, low-noise PR that accumulates every run's log entries regardless of outcome. Reviewers merge it periodically (at minimum before each monthly run) so the logs reach `main`. If any git step fails, keep the in-memory records for this run's analysis and note the failure in the Slack summary.
This produces one perpetual, low-noise PR that accumulates every run's log entries regardless of outcome. Reviewers merge it periodically (at minimum before each monthly run) so the logs reach `main`. If any git step fails, keep the in-memory records for this run's analysis and note the failure in the Slack summary. After the log update is pushed, switch back to `main` (or create the monthly improvement branch from the latest `origin/main`) before making any skill or template edits so the standing log branch only contains log files.


- **In-memory records from Step A** — style-lint and PR-review signals parsed from Oz run artifacts. These are already in memory; do not re-read from disk.
- **On-disk human feedback** — read `.agents/logs/human_review_feedback.jsonl` line by line (skipping empty lines). Each line is a JSON record; parse and filter to the past 30 days.
- **On-disk human feedback** — read `.agents/logs/human_review_feedback.jsonl` line by line (skipping empty lines). Each line is a JSON record; parse and filter to the past 30 days. Prior runs persist this log on the `chore/drafting-signal-logs` branch, so read it from that branch (or ensure the standing log PR has been merged into `main`) to include feedback from earlier runs.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ [IMPORTANT] Step B says to keep accepted human-feedback records in memory if the log-branch git steps fail, but this analysis input only reads human feedback from disk, so the fallback path can drop the current run's human signals.

Suggested change
- **On-disk human feedback** — read `.agents/logs/human_review_feedback.jsonl` line by line (skipping empty lines). Each line is a JSON record; parse and filter to the past 30 days. Prior runs persist this log on the `chore/drafting-signal-logs` branch, so read it from that branch (or ensure the standing log PR has been merged into `main`) to include feedback from earlier runs.
- **Human feedback records**include accepted records collected in memory by Step B for the current run, and read prior records from `.agents/logs/human_review_feedback.jsonl` line by line (skipping empty lines). Each JSON record should be parsed and filtered to the past 30 days. Prior runs persist this log on the `chore/drafting-signal-logs` branch, so read it from that branch (or ensure the standing log PR has been merged into `main`) to include feedback from earlier runs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant