docs(skills): fix residual self-improvement loop inconsistencies#285
docs(skills): fix residual self-improvement loop inconsistencies#285rachaelrenk wants to merge 2 commits into
Conversation
- improve-drafting-skills: persist pr_review_runs.md and human_review_feedback.jsonl through a standing chore/drafting-signal-logs branch + one perpetual PR instead of committing to protected `main` (a direct push fails silently — the same failure mode #280 fixed for aeo_crosslink_audit) - Remove the orphaned .agents/logs/style_lint_runs.jsonl (nothing writes it since style-lint signals moved to [SIGNAL:style-lint] stdout markers) and its stale reference - Clean up the leftover self-correcting sentence in the Run log section Co-Authored-By: Oz <oz-agent@warp.dev>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
I'm starting a first review of this pull request. You can view the conversation on Warp. I completed the review and no human review was requested for this pull request. Comment Powered by Oz |
There was a problem hiding this comment.
Overview
This PR updates the improve-drafting-skills workflow to persist drafting signal logs through a standing branch/PR and removes the obsolete style_lint_runs log. The direction matches the stated goal, but the new standing-branch flow leaves correctness gaps that can still drop signals or mix monthly improvement edits into the log PR.
Concerns
- The persistence helper checks out the standing log branch but never returns to a clean improvement branch before later skill/template edits.
- The analysis step still only names Step A in-memory records plus on-disk human feedback, so Step B's failure fallback can discard current-run human feedback.
Verdict
Found: 0 critical, 2 important, 0 suggestions
Request changes
Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).
Powered by Oz
| 4. Push the branch. | ||
| 5. Ensure exactly one open PR exists from `chore/drafting-signal-logs` into `main`, titled `chore: drafting signal logs`. Create it if missing; otherwise the push updates the existing PR. Keep this log PR separate from the drafting-skills improvement PR. | ||
|
|
||
| This produces one perpetual, low-noise PR that accumulates every run's log entries regardless of outcome. Reviewers merge it periodically (at minimum before each monthly run) so the logs reach `main`. If any git step fails, keep the in-memory records for this run's analysis and note the failure in the Slack summary. |
There was a problem hiding this comment.
chore/drafting-signal-logs but never switches back before the monthly run drafts skill/template edits, so those edits can land on the standing log branch and pollute the log PR.
| This produces one perpetual, low-noise PR that accumulates every run's log entries regardless of outcome. Reviewers merge it periodically (at minimum before each monthly run) so the logs reach `main`. If any git step fails, keep the in-memory records for this run's analysis and note the failure in the Slack summary. | |
| This produces one perpetual, low-noise PR that accumulates every run's log entries regardless of outcome. Reviewers merge it periodically (at minimum before each monthly run) so the logs reach `main`. If any git step fails, keep the in-memory records for this run's analysis and note the failure in the Slack summary. After the log update is pushed, switch back to `main` (or create the monthly improvement branch from the latest `origin/main`) before making any skill or template edits so the standing log branch only contains log files. |
|
|
||
| - **In-memory records from Step A** — style-lint and PR-review signals parsed from Oz run artifacts. These are already in memory; do not re-read from disk. | ||
| - **On-disk human feedback** — read `.agents/logs/human_review_feedback.jsonl` line by line (skipping empty lines). Each line is a JSON record; parse and filter to the past 30 days. | ||
| - **On-disk human feedback** — read `.agents/logs/human_review_feedback.jsonl` line by line (skipping empty lines). Each line is a JSON record; parse and filter to the past 30 days. Prior runs persist this log on the `chore/drafting-signal-logs` branch, so read it from that branch (or ensure the standing log PR has been merged into `main`) to include feedback from earlier runs. |
There was a problem hiding this comment.
| - **On-disk human feedback** — read `.agents/logs/human_review_feedback.jsonl` line by line (skipping empty lines). Each line is a JSON record; parse and filter to the past 30 days. Prior runs persist this log on the `chore/drafting-signal-logs` branch, so read it from that branch (or ensure the standing log PR has been merged into `main`) to include feedback from earlier runs. | |
| - **Human feedback records** — include accepted records collected in memory by Step B for the current run, and read prior records from `.agents/logs/human_review_feedback.jsonl` line by line (skipping empty lines). Each JSON record should be parsed and filtered to the past 30 days. Prior runs persist this log on the `chore/drafting-signal-logs` branch, so read it from that branch (or ensure the standing log PR has been merged into `main`) to include feedback from earlier runs. |
Summary
Small follow-up to the docs self-improvement loop work (#239) and the
aeo_crosslink_auditreliability fix (#280). A retrospective of the "software factory" surfaced a few residual inconsistencies; this PR cleans them up. No changes to published docs — all edits are confined to.agents/.Changes
.agents/skills/improve-drafting-skills/SKILL.mdmain. Steps A and B previously instructed committingpr_review_runs.mdandhuman_review_feedback.jsonldirectly tomainwith a silent "if the push fails, continue" fallback — the exact failure mode docs(skills): make aeo_crosslink_audit reliable #280 fixed foraeo_crosslink_audit(protectedmainrejects the push, so the "durable" log never persists). Both now persist through a standingchore/drafting-signal-logsbranch and one perpetual, low-noise PR, mirroring the pattern from docs(skills): make aeo_crosslink_audit reliable #280.style_lint_runs.jsonlreference..agents/logs/style_lint_runs.jsonl(deleted)[SIGNAL:style-lint]stdout markers (collected viaoz run get), nothing writes this file. Removed to match the current design.Not changed (intentionally)
improve-aeo-crosslink-skillcadence. PR docs: implement self-improvement loop architecture #239's description called Loop 4 "quarterly," but the skill file runs monthly (matching the siblingimprove-drafting-skills). Confirmed monthly is intended; left as-is.Validation
style_lint_runsreferences repo-wide.main" instructions inimprove-drafting-skills.git diff --checkclean; skill frontmatter parses.Conversation: https://staging.warp.dev/conversation/a8bc7071-0fda-470a-8f13-49991355164a
Co-Authored-By: Oz oz-agent@warp.dev