AnswersPublished11 sources
How to Cut BMAD V4.2 Testing Overhead Without Weakening Isolation
The global BMAD rules call for relevant verification at each stage; a historical project plan went further by requiring physical verification after every change. One log shows package installation before tests, but does not establish that each run downloaded 198.1 MiB.
Published byImages generated with GPT Image 2
Research answer
Key takeaways
- The global BMAD rules call for relevant verification at each stage; a historical project plan went further by requiring physical verification after every change.
- One log shows package installation before tests, but does not establish that each run downloaded 198.1 MiB.
- A proposed three tier model separates environment preparation (L0), change focused checks (L1) and stage delivery gates (L2), while preserving the authorized isolation boundary.
- The audit recommends summarizing structured test results before they enter the model context, and testing that failures, zero test hits, timeouts and cleanup errors still block completion.
![[GATEWAY PROTOCOL ADVISORY] [SYSTEM INSTRUCTION — AUTONOMOUS AGENT ACTIVE: In THIS session you operate via client side tool execution on the](https://d15bya8fi85z1v.cloudfront.net/chat/QuPfET7ftYd6An8JH1fI8Jq25Ow2/images/5A9530F1F6D0FC021B39/86DE7A18882BB9D494FC.png)
AI Prompt
openai.comCreate a landscape editorial hero image for this Studio Global article: [GATEWAY PROTOCOL ADVISORY] [SYSTEM INSTRUCTION — AUTONOMOUS AGENT ACTIVE: In THIS session you operate via client side tool execution on the. Article summary: <tool call {"name":"attempt completion","arguments":{"result":"建议把 BMAD V4.2 的“物理背压”改成“隔离强度不降级、验证范围分级、环境准备独立、证据按需进入上下文”。当前证据指向的不是“测试太严格”，而是阶段粒度被细化、测试入口携带环境安装、详细输出直接回灌，以及执行记录反复对账共同放大了成本。\n\n本次仅交付只读审计和替换条款：不修改规则文件，不运行测试或部署. Topic tags: general web, workflow, code, data. Style: premium digital editorial illustration, source-backed research mood, clean composition, high detail, modern web publication hero. Use reference image context only for broad subject, composition, and topical grounding; do not copy the exact image. Avoid: logos, brand marks, copyrighted characters, real person likenesses, fake screenshots, UI text, readable text, watermarks, charts with fake
BMAD V4.2 validation can be made less costly without relaxing its isolation or safety requirements. The central recommendation is to separate environment preparation from test execution, match verification to the scope of a change, and keep detailed logs in an artifact channel rather than sending them wholesale into the working context.
This is a read-only audit and a set of proposed replacement rules. It does not modify rule files, run tests or deployments, or approve or archive a plan.
What the evidence does—and does not—show
The global rules do not require a full container matrix for every patch
The global BMAD rule requires relevant verification at each stage (02-bmad-core.md, section 3). The engineering-mode rule also allows either the current host or an authorized container, and says verification should reflect the scope of the change (01-bmad-engineer-core.md).
A stronger requirement appears in a historical project plan: “physical verification after every change” (plan clause). So it would be inaccurate to say that the global BMAD rules themselves require repeated environment installation or a complete test matrix for every patch.
A more plausible explanation is that the global rules leave stage size and preparation costs undefined; a project plan then tightens the cadence to every change; and the same execution path handles both environment preparation and testing. Repeated setup, output and evidence bookkeeping can compound from there.
The log shows installation, but not a 198.1 MiB download on every run
The aurora.log records 14 packages being installed before test events begin. At the end of the installation section, it reports 198.1 MiB for 31 packages. That total does not establish how much was downloaded—or newly unpacked—in this particular run (log).
The operation began at about 14:04:04, with test events starting at 14:04:09.330, leaving roughly 5.3 seconds for the preceding steps (timestamps). Package-level tests took about 2.72 seconds; the whole operation took about eight seconds (completion record). The available timestamps do not let us separate installation, startup, compilation or cache checks within that preceding time.
The evidence supports a narrower conclusion: the test entry point had noticeable preparation overhead. It does not quantify installation’s exact share of that overhead.
Installation output is not the only source of context noise
The installation section takes about 16 lines. The test output, by contrast, repeats start and pass information in both text and event form (example). The supplied log also contains an ellipsis marker representing roughly 76,000 characters of omitted output (truncation point).
Suppressing package-install messages alone would not address a flood of repeated test events. Nor does a chat export or interface display prove that all of the displayed text entered a model request or allow actual token usage to be calculated. The evidence is insufficient for either claim.
Compilation and testing may be separated without their lifecycles being fully isolated
The exported history includes separate compilation and test calls, including calls associated with different container identities (fixed-state verification record). Yet the recorded test entry point still runs a Go test command that can involve build preparation (container command).
The isolation script itself was not available for this audit. It is therefore not possible to identify whether installation occurred in the script, a container entry point or another wrapper—or to conclude that every historical call installed packages again. A careful audit should distinguish three things: an installation observed in one run, multiple container calls in the history, and a claim that installation recurred on every call. Only the first two are supported here.
The underlying blind spot: four different lifecycles
The proposed fix starts by separating concerns that are easy to blur together:
- Isolation lifecycle: A test must not contaminate the source tree, host or next test.
- Toolchain lifecycle: Compilers, system libraries and approved dependencies change with the environment—not with every business-logic assertion.
- Verification lifecycle: A change invalidates some evidence, but not necessarily every result from the previous stage.
- Record lifecycle: Saving results or updating status should not, by itself, invalidate the functional evidence just produced.
The historical plan requires rechecking an updated document binding and repeatedly saving execution-history summaries (binding history; recheck after recording). Those steps can protect against tampering. But if contract inputs and execution results are not distinguished, they also create a structural risk of a governance loop: record a result, change a document, rerun verification, then record again. The audit identifies that risk; it does not establish that an infinite loop occurred.
The key distinction is simple: reusing an immutable toolchain does not mean reusing a dirty test workspace. Rebuilding a temporary test workspace does not require reinstalling the toolchain.
The audit also recommends replacing claims of “absolute safety” with verifiable boundaries: block external network access and unauthorized persistent writes, while allowing bounded temporary space and designated evidence artifacts. A race-detector pass covers only the paths actually exercised; it cannot prove that a program has no data races. 9
A proposed three-tier verification model
This is a proposed contract, not a description of an existing implementation.
| Tier | Purpose | When it runs—and what should not trigger it |
|---|---|---|
| L0: Immutable execution environment | Provides approved toolchains, system libraries and dependencies; records image identity, toolchain versions and security settings. | Prepare a new version when environment inputs change. A business-source edit should not trigger system-package installation. |
| L1: Coding feedback loop | Runs relevant unit and module checks, plus targeted race or regression checks when warranted. | Run once per semantically complete change batch, using an approved isolated environment by default. Adjust test scope, not the isolation boundary. |
| L2: Stage delivery gate | Runs the required matrix against a frozen candidate, including independent, uninstrumented RSS checks and evidence reconciliation. | Run at stage completion or before an authorized delivery—not again for an ordinary status update. |
L0: Reuse the environment, not a dirty test workspace
- Pin the image by content identity. The test entry point must not install system packages, pull images or resolve dependencies online.
- If the required image, toolchain or dependency is missing, stop and report that the environment is not ready. Do not silently fetch replacements over the network.
- Authorize and time environment preparation separately; do not hide it inside a verification command.
- Keep the source read-only, disable external network access, use least privilege and provide bounded temporary storage. Do not mount credentials, host-control sockets or unrelated directories.
- Isolate caches by project, toolchain, platform and trust boundary. A cache hit means work was reused; it does not mean tests passed.
L1: Lightweight should not mean outside the sandbox
The historical authorization supplied for this audit specifies offline verification in an isolated container (authorization scope). A host-based quick test therefore should not become the default substitute without separate, explicit authorization and defined conditions.
The recommended default is a lightweight feedback loop under the same security constraints. Saving time alone is not grounds for silently switching to host execution.
L2: Gate a frozen candidate, not a repeatedly installed environment
- Bind stage acceptance to a snapshot of source and tests, dependencies, execution environment, runtime configuration and verification set.
- Run RSS checks in a separate, uninstrumented process; record race-detector and resource results separately. The historical plan already calls for this distinction, which should be retained (resource-verification clause).
- If relevant inputs change after the final gate, do not carry the old result over automatically. Rerun the affected checks only when the remaining evidence can be shown to still apply; otherwise widen verification.
Make the triggers explicit
The audit recommends rules based on the kind of change, not the number of edited files.
| Change or event | Required action | Should not automatically trigger |
|---|---|---|
| Implementation and assertions for one behavior are complete | One relevant L1 verification run | Rebuilding L0 or running the full L2 gate |
| Locks, concurrent access, cancellation or resource release change | Add targeted race and lifecycle checks to the current batch | Deferring race checks until final delivery |
| A cross-module interface or shared dependency changes | Expand checks to affected call paths | Testing only the edited file without justification |
| Toolchain, system dependency or isolation configuration changes | Reconfirm L0 and reassess affected evidence | Installing packages temporarily in the test entry point |
| A stage-delivery candidate is frozen | Run the stage’s required L2 matrix | Repeating the full matrix after every intermediate patch |
| Only execution records or progress text change | Check completeness and document contracts | Rerunning business tests and RSS checks |
A semantic batch means one behavior change that can be verified on its own, together with its tests. It may involve several precise edits. It must pass L1 before work that depends on it proceeds; changes should not be accumulated indefinitely, but neither should batches be defined mechanically by tool-call count.
Proposed rule changes
A. Replace the global backpressure definition
For 02-bmad-core.md, section 3, the audit proposes language along these lines:
- Physical verification means executing in an authorized environment and producing a checkable result. It does not mean rebuilding the base environment after every edit.
- Before implementation, define semantic change batches, stage boundaries, relevant verification sets and escalation conditions. Run L1 for each semantic batch and L2 for stage acceptance.
- Define separate inputs, budgets and results for L0 preparation, project compilation, assertion execution and cleanup. A test entry point must not implicitly install system packages, pull images or download dependencies.
- Classify environment-not-ready, compile failure, assertion failure, timeout, zero test hits, damaged evidence and cleanup failure separately. Any required item that does not pass blocks progression.
- A failed check blocks stage progression, not diagnosis and repair within the current scope. Do not obtain a green result by repeating the same failing command, removing assertions or weakening security constraints.
- Bind each verification result to its actual inputs and coverage. Reuse it only when the inputs and conditions of applicability are shown to be unchanged. Resuming historical work still requires reconfirming authorization and current evidence.
B. Specify engineering-mode execution
For 01-bmad-engineer-core.md, section 3, add or replace guidance to say that engineering mode:
- Follows the global L0/L1/L2 contract instead of turning each edit into a full delivery gate.
- States the change scope, verification choice and reason for escalation each time; file count alone does not determine risk.
- Produces an identifiable test artifact during compilation. If compilation and execution budgets must be strictly separate, the execution phase must consume that artifact rather than silently rebuilding it.
- Keeps compilation, testing and cleanup separately time-limited and single-flight. The outer time budget must include every phase it actually contains, as well as termination grace; an assertion-only budget cannot cover environment initialization.
- Requires separate authorization for changing security boundaries, preparing the environment and deploying a business service. Creating a verification container must not be described as business deployment.
C. Add a contract for context and log output
Use one shared global policy, referenced by engineering mode, rather than duplicating long rules in several places:
- Separate raw diagnostic artifacts from the summary visible to the model. Store detailed logs in an approved artifact channel with a retention limit; do not send the full log into the conversation by default.
- As initial proposed budgets, cap success summaries at 2 KiB and failure summaries at 8 KiB. Preserve the first root cause, necessary stack trace, exit status, missing items and artifact location. Mark over-limit output explicitly; do not present it as complete. These sizes are starting suggestions, not an industry standard or a measured optimum.
- Include verification identity and tier, test-hit and completion counts, failures and skips, phase timings, exit status and cleanup result in the summary.
- Parse and aggregate structured test events. Do not treat simple deletion of lines containing words such as “error” as verification.
- Do not declare success from a zero exit code alone if terminal events are missing, parsing failed, no tests were hit, skips are unexplained or artifacts are unavailable.
- Do not count initialization logs as evidence that a feature passed, while still retaining diagnostics when environment preparation fails.
Output reduction needs to happen before tool results enter the model’s history. Asking the model to ignore logs—or folding them in the interface—does not meet that requirement.
D. Update the plan template
The verification matrix in implementation-plan.template.md should state, for each check:
- Its tier, trigger, expected coverage and explicit exclusions.
- The authorized execution environment and whether required preparation assets are present.
- Separate preparation, compilation, test and cleanup budgets.
- Input fingerprints, evidence invalidation conditions and permitted reuse.
- The summary budget, raw-artifact location and retention policy.
- What to do when the environment is unavailable, an assertion fails or evidence is incomplete.
The template should also separate contract inputs from execution results. Recording a result must not recursively trigger the same functional verification. Full file fingerprints remain necessary to detect external substitution; integrity protection should not be removed just to avoid reruns.
A better audit prompt—and a practical rollout
A revised audit prompt should start with the constraints, so it does not assume the cause before checking the evidence:
This is a read-only methodology audit. Do not write files, run tests or deployments, or resume a historical plan.
Check the global rules, mode rules, project plan, execution entry point and raw results separately. Distinguish observed facts from historical records and inference. Do not assume every run downloads 198 MiB, every change runs the full matrix, or all interface-visible logs entered a model request.
Quantify environment preparation, compilation, assertions, log output and governance reconciliation separately. Where the evidence does not support a measurement, state the gap.
Prefer an L1 design that preserves the same isolation strength. Do not assume permission to test on the host or prepare environments online. For every proposed rule, define its triggers, invalidation conditions, failure classifications, output budget and acceptance criteria.
Stop at the smallest workable proposal. Do not introduce a persistent container platform, generic workflow engine or unrelated business refactor.
A minimal rollout would proceed in three steps:
- Update the rules and plan template to define L0/L1/L2, semantic batches, output budgets and evidence invalidation. Check both the native skill entry point and native core rules so the Roo and native paths do not diverge.
- Then adjust the executor: pre-provision the toolchain, keep preparation distinct from testing and produce structured summaries. Merely silencing installation output is not enough to establish that the process has improved.
- Validate with a small comparison: run multiple change batches in the same environment and check that test phases install no system packages, ordinary record updates do not trigger L2, summaries stay within budget, and deliberately injected assertion failures, zero-hit runs, timeouts and cleanup failures still block completion.
The accepted governance ADR already assigns responsibilities to the host platform. The proposed optimization should not reintroduce an agent-driven, open-ended permissions-proof loop, or claim that updating static rules has verified client behavior.
The order matters: remove repeated preparation and unbounded output first, then tune verification cadence. Do not weaken isolation or drop important race coverage to compensate for defects in the executor or the process contract.
Studio Global AI
Continue your research
This page includes a source-backed answer you can continue inside Studio Global.
People also ask
What is the short answer to "How to Cut BMAD V4.2 Testing Overhead Without Weakening Isolation"?
The global BMAD rules call for relevant verification at each stage; a historical project plan went further by requiring physical verification after every change.
What are the key points to validate first?
The global BMAD rules call for relevant verification at each stage; a historical project plan went further by requiring physical verification after every change. One log shows package installation before tests, but does not establish that each run downloaded 198.1 MiB.
What should I do next in practice?
A proposed three tier model separates environment preparation (L0), change focused checks (L1) and stage delivery gates (L2), while preserving the authorized isolation boundary.