AnswersPublished15 sources
Keep the Tests. Tame the Temporary Files.
The V4 repository already has a centralized test suite for configuration, installation and asset protection checks; a tidy up should not mistake those lasting safeguards for disposable clutter. Temporary experiments, test outputs and release contents need separate rules: material in a suite directory may be picked u...
Published byImages generated with GPT Image 2
Research answer
Key takeaways
- The V4 repository already has a centralized test suite for configuration, installation and asset protection checks; a tidy up should not mistake those lasting safeguards for disposable clutter.
- Temporary experiments, test outputs and release contents need separate rules: material in a suite directory may be picked up for distribution, though the review did not find evidence that this has happened.
- The practical documentation fix is to clarify what belongs in analysis, decision records, plans and scratch space—not to merge everything into one file.
- This was a partial static review, not a full audit: tests were not rerun, Git tracking status was not checked, and some planned external research remains incomplete.
![[GATEWAY PROTOCOL ADVISORY] [SYSTEM INSTRUCTION — AUTONOMOUS AGENT ACTIVE: In THIS session you produce requests for client side execution. W](https://d15bya8fi85z1v.cloudfront.net/chat/oBMfTzLzQzYo7VcXQdEGj2yQgAw1/images/73E9D63489E02AC46209/BEDABC175BE8FCADFC63.png)
AI Prompt
openai.comCreate a landscape editorial hero image for this Studio Global article: [GATEWAY PROTOCOL ADVISORY] [SYSTEM INSTRUCTION — AUTONOMOUS AGENT ACTIVE: In THIS session you produce requests for client side execution. W. Article summary: <tool call {"name":"attempt completion","arguments":{"result":"两个痛点都适合纳入 V4 的“产物生命周期背压”，但不应简化成“测试全部不提交”或“文档全部合并”。建议保留长期质量资产，隔离一次性验证材料，并分别控制仓库维护、安装分发和模型上下文。\n\n本次交付为阶段性调研：仓库静态证据、Geoffrey Huntley 的作者资料及部分学术资料已取得；Python 官方布. Topic tags: general web, openai, llm, agents, ai. Style: premium digital editorial illustration, source-backed research mood, clean composition, high detail, modern web publication hero. Use reference image context only for broad subject, composition, and topical grounding; do not copy the exact image. Avoid: logos, brand marks, copyrighted characters, real person likenesses, fake screenshots, UI text, readable text, watermarks, charts with fa
A repository can look cluttered for two very different reasons: it may contain useful, long-lived tests, or it may be accumulating one-off scripts, logs and drafts. Treating both as the same problem can make the codebase look cleaner while weakening its safeguards.
A partial static review of the V4 repository points to a more targeted approach: keep durable quality checks, separate temporary validation material, and set distinct boundaries for what is committed, installed and loaded into an AI agent’s context.
Keep useful tests; isolate one-off validation
The repository’s CI workflow runs tests from a centralized tests directory, including checks for configuration contracts, installation and asset protection. Their presence alone is not evidence that they are disposable. The review did not rerun them or assess each test’s long-term value, so it cannot establish which individual tests should stay or go.
The ignore rules exclude some sandbox, cache and temporary files, but do not explicitly cover the existing scratch and verification-output areas at ._bmad-output/scratch/ and ._bmad-output/verification/. That does not prove files in those directories are tracked: the review did not check Git’s index.
There are also separate boundaries for repository updates and installation. The update entry point in bs.sh updates the repository, while deployment source collection reads selected suites. The collector recursively gathers ordinary files within a suite and maps them to deployment inputs (collector; deployment mapping). That suggests temporary material placed inside a suite could be distributed. The review found no evidence that this has actually happened.
A sensible policy would distinguish three kinds of material:
| Material | Suggested source-control approach | Suggested layout and distribution approach |
|---|---|---|
| Long-term regression, contract and necessary integration tests | Commit and maintain them with the implementation | Follow the language and project’s conventions; keep them out of runtime packages by default |
| One-off diagnostics, validation scripts and experimental inputs | Keep local by default rather than committing automatically | Use an approved scratch area; do not put them in suite asset directories |
| Logs, coverage data and run receipts | Exclude from source commits by default; retain specific evidence when needed | Keep separate from test source and out of install packages |
These are recommendations, not rules already implemented in V4. Language conventions matter, too: Go’s official tutorial places test files alongside the package they test, so moving every Go unit test into a single external directory should not be treated as a universal best practice (15). V4 already uses a centralized test layout, which is the practical starting point here; the review did not establish an official, general-purpose Python layout recommendation.
The goal matters. If the concern is what end users receive, improve the package or installation manifest. If it is the developer’s source tree, use a layout that fits the language. Moving test files alone will not solve both problems.
Give documents distinct jobs
V4’s artifact-routing rules distinguish analysis, ADRs, plans and contracts, and direct the agent to ask when a document’s destination is unclear. But the draft guidance gives a different path convention. This supports a narrow conclusion: the lifecycle rules are incomplete. It does not show why every unexpected directory was created.
Rather than add another layer of folders, clarify the responsibilities of the existing ones:
- Analysis should preserve evidence, comparisons, counterarguments and open questions. Routine advice can stay in the conversation; no report is needed unless there is a reason to keep one.
- Architecture decision records (ADRs) should capture significant decisions, their rationale and constraints—not reproduce the full analysis or mirror every implementation step.
- Implementation plans should record authorization, scope, dependencies and acceptance criteria, linking to relevant decisions instead of duplicating research.
- Temporary material—drafts, diagnostic inputs and raw outputs—should go in an approved scratch area. Calling something a “draft” should not automatically place it in the formal analysis archive.
- Operational documentation is worth keeping when it serves as a reusable guide. A single deployment run should not automatically create another deployment directory.
File count, source-control status, package inclusion and AI-context inclusion are separate decisions. Putting everything in one directory does not resolve them.
What Geoffrey Huntley’s examples do—and don’t—show
Geoffrey Huntley’s article on backpressure describes building automated feedback around an agent so it can catch mistakes as it works; it is not an argument for deleting test files (12). In a separate, specific code-porting workflow, he suggests documenting test behavior as specifications before guiding a port (
14). That example does not establish that documentation should replace executable tests in every project.
The useful takeaway for V4 is to preserve feedback that checks correctness while reducing duplicate or one-off validation assets. Specifications explain expected behavior; tests provide executable checks. They can complement each other, and should not be treated as substitutes without examining the use case.
The available material does not establish whether Huntley has a universal policy for committing tests, organizing drafts or excluding files from releases. Nor does this review establish a consensus across forums.
More documents aren’t automatically better—or worse
Research on long-context language models describes difficulty using information placed in the middle of an input, while HELMET argues for evaluating long-context systems across a range of tasks rather than relying only on simple retrieval tests (3
4). Those findings do not prove that a document above a particular length causes hallucinations, or quantify the risk in this repository.
They do support a practical approach: split material by topic and decision boundary; load the sections relevant to the current task by default; retrieve historical material when needed; and use short summaries to point to evidence, unknowns and unfinished obligations. Keep raw material available for traceability, but do not automatically feed every log and old report into the model. An index should help readers find information, not become a second, duplicated record of the facts.
A small policy change, not a heavy new process
A useful V4 rule could cover an artifact’s lifecycle from creation to delivery:
- Before creating it: state its purpose and check whether an existing test or document can be extended.
- Before saving it: decide whether it is authorized, temporary or long-lived, versioned or local, and intended for distribution. If the destination is unclear, do not invent a new folder.
- Before committing: check for temporary outputs, sensitive material and duplicated tests or documents. Excluding all tests is not a sound shortcut.
- Before distribution: define the installation boundary and keep temporary material out of suite assets, without removing existing identity, summary or asset-protection safeguards.
- Before cleanup: protect failure evidence, active work and historical references. A completed task or an old file is not, by itself, permission to delete it.
The review’s most defensible priorities are to clarify temporary-file handling and release boundaries, then make documentation roles and context loading more explicit. It does not support moving all Go tests, stopping commits of long-lived tests or merging all documentation into one file. Any future authorized changes should verify that test discovery still works, CI remains compatible, temporary files stay out of installations, and references to historical decisions and unfinished work remain traceable.
This was a partial review based on static repository evidence and selected external sources. Tests were not run, no files were changed, and the review did not complete its planned checks of official Python guidance or discussions on X, Reddit, Hacker News and Lobsters.
Studio Global AI
Continue your research
This page includes a source-backed answer you can continue inside Studio Global.
People also ask
What is the short answer to "Keep the Tests. Tame the Temporary Files."?
The V4 repository already has a centralized test suite for configuration, installation and asset protection checks; a tidy up should not mistake those lasting safeguards for disposable clutter.
What are the key points to validate first?
The V4 repository already has a centralized test suite for configuration, installation and asset protection checks; a tidy up should not mistake those lasting safeguards for disposable clutter. Temporary experiments, test outputs and release contents need separate rules: material in a suite directory may be picked up for distribution, though the review did not find evidence that this has happened.
What should I do next in practice?
The practical documentation fix is to clarify what belongs in analysis, decision records, plans and scratch space—not to merge everything into one file.
Sources
go.devAdd a test
ghuntley.comdon't waste your back pressure
doi.orgFound in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding