How to Read a Verification Result
You've run a verification tool on an evidence bundle. It produces an evidence packet, a report, and perhaps an index of verified artifacts. Now what?
Understanding what these verification results actually tell you—and what they don't—is critical. Misreading verification outputs is one of the fastest ways to create false confidence or miss real problems.
This guide explains how to interpret verification results correctly, what they assert, what they don't imply, and how to combine them with other evidence during audits.
The Three-Part Output
Most evidence verification systems produce three related artifacts:
1. Evidence (pass/fail verdict)
A verification attestation that the tool ran and produced a verdict, suitable for archival and later replay.
2. Report (detailed findings)
A human-readable explanation of what was checked, what passed, what failed, and why. This is what auditors read to understand the verification process.
3. Index (artifact inventory)
An inventory of verified artifacts to support traceability.
All three serve different purposes. The evidence supports archival and later replay. The report is for understanding. The index is for traceability.
What Verification Asserts
When verification succeeds, the result asserts structural integrity and contract conformance:
1. Completeness
All required artifacts according to the declared contract are present. Nothing mandatory is missing.
What this means: The bundle contains the minimum set of files needed to understand what happened.
What this doesn't mean: The artifacts are correct, meaningful, or scientifically valid.
2. Integrity
Declared hashes match observed content. No artifacts were corrupted, modified, or substituted after the manifest was created.
What this means: The artifacts you're looking at are the same ones that were present when the bundle was sealed.
What this doesn't mean: The artifacts were correct when created, or that the process that created them was valid.
3. Contract Conformance
The bundle structure matches the declared contract version. Required metadata is present and valid under the declared contract checks.
What this means: The bundle follows the declared format that can be independently interpreted.
What this doesn't mean: The contract itself is appropriate for this use case, or that following the contract guarantees correctness.
4. Deterministic Verification
Given the same bundle and verification tool version, re-running verification should produce the same result under the same compatibility profile.
What this means: The verification process is reproducible and doesn't depend on time, network, or hidden state.
What this doesn't mean: The original computation that produced the bundle was deterministic.
What Verification Does NOT Assert
This is equally important. Verification results deliberately do not make claims about:
1. Scientific Correctness
Verification checks structure, not science. A verified bundle might contain:
Incorrect algorithms
Miscalibrated sensors
Biased training data
Invalid statistical assumptions
All of these would pass verification if the bundle structure is intact.
2. Clinical Safety or Effectiveness
Verification doesn't evaluate whether outputs are safe to use in clinical contexts, whether they meet accuracy requirements, or whether they're fit for purpose.
3. Regulatory Compliance
A verification evidence packet is not a claim of FDA clearance, CE marking, or compliance with any specific standard. It's evidence you can use in regulatory submissions, not a substitute for compliance work.
4. Operational Correctness
Verification doesn't check whether the right data was collected, whether sensors were properly calibrated, or whether operators followed procedures correctly.
5. Result Quality
High-quality results can fail verification (if the bundle is malformed). Low-quality results can pass verification (if the bundle is structurally sound). Quality and verification are orthogonal.
Reading a Success Result
A successful verification result tells you:
"This bundle is structurally intact, conforms to the declared contract version, and all declared hashes match observed content. The bundle can be independently reviewed without requiring access to the original infrastructure."
How to use this:
Trust the custody chain: artifacts haven't been tampered with
Proceed to content review: examine what the artifacts actually contain
Combine with validation evidence: does the science/engineering hold up?
Include in audit packages: verification evidence packet proves integrity for regulators
What NOT to conclude:
"The results are correct" (verification doesn't check correctness)
"The system is compliant" (verification doesn't assess compliance)
"The outputs are safe to use" (verification doesn't evaluate safety)
Reading a Failure Result
Verification failures fall into two broad categories: structural problems and policy requirements not met.
Structural Problems
These indicate issues with bundle integrity or completeness:
Missing artifacts:
Required files according to the contract are absent. This could mean:
The bundle assembly process failed
Files were deleted or lost during transfer
The bundle was created under a different contract version than declared
Hash mismatches:
Declared hashes don't match observed content. This could mean:
Artifacts were modified after manifest creation
Corruption during storage or transfer
The manifest and artifacts are from different runs
Malformed metadata:
Required metadata fields are missing, have wrong types, or don't meet contract requirements. This could mean:
A bug in bundle assembly
Manual editing that broke structure
Version mismatch between bundle and verification tool
Structural problems are hard stops. They indicate the bundle cannot be trusted as an intact record of what happened. Investigation is required before proceeding.
Policy Requirements Not Met
These indicate the bundle doesn't meet specific requirements, even if structurally intact:
Unsupported contract version:
The bundle declares a contract version the verification tool doesn't recognize or support.
Prohibited artifact types:
The bundle contains artifact types that aren't allowed under declared verification policy.
Missing required reports:
The bundle structure is valid, but declared policy requirements specify additional artifacts (e.g., reference workload reports for interpretive outputs) that aren't present.
Policy issues may or may not be showstoppers, depending on context. A bundle verified under an older contract might still be acceptable for some audits. But policy issues require explanation and justification.
Combining Verification with Validation
Verification and validation work together but serve different purposes:
┌─────────────────────────────────────────┐
│ Was the computation scientifically │
│ valid and appropriate? │ ← VALIDATION
│ (Domain experts review algorithms, │
│ methods, and scientific reasoning) │
└─────────────────────────────────────────┘
↑
│ Uses
↓
┌─────────────────────────────────────────┐
│ Is this evidence bundle intact and │
│ trustworthy as a record? │ ← VERIFICATION
│ (Verification tool checks structure, │
│ hashes, and contract conformance) │
└─────────────────────────────────────────┘
Typical audit workflow:
Verification first: Confirm the bundle is intact and hasn't been tampered with
Content review: Examine what the artifacts actually contain
Validation: Domain experts assess whether the methods, algorithms, and results are scientifically sound
Compliance check: Legal/QMS teams confirm regulatory requirements are met
Approval: All layers must pass before the evidence is accepted
If verification fails, you don't proceed to validation—there's no point validating a bundle that might be corrupted or incomplete.
If verification passes but validation fails, you have intact evidence of an incorrect or inappropriate computation. That's still valuable for root cause analysis.
Red Flags in Verification Results
Certain patterns in verification results should trigger deeper investigation:
1. Frequent Hash Mismatches
If you regularly see hash mismatches, it suggests:
Artifacts are being modified after manifest creation
Your bundle assembly process is broken
Storage/transfer is corrupting data
Investigate your workflow before continuing.
2. Contract Version Confusion
If different bundles from the same system declare different contract versions unexpectedly, it suggests:
Inconsistent bundle assembly tooling
Manual overrides of version declarations
Multiple systems producing bundles without coordination
Standardize on a single contract version for a given system version.
3. Missing Required Artifacts, Always the Same Ones
If verification consistently fails because specific artifacts are missing, it suggests:
Your bundle assembly template is incomplete
Those artifacts are being generated but not included
Contract requirements changed but assembly logic didn't
Fix the assembly process, don't work around it.
4. Verification Passes but Auditors Reject the Bundle
If verification succeeds but human reviewers find problems, it means:
Verification is checking structure, but content has issues
The contract doesn't capture all requirements
Declared policy requirements need to be tightened
This is expected: verification is necessary but not sufficient.
How Auditors Should Use Verification Results
If you're an auditor reviewing verification results:
Do:
Verify independently: Re-run verification yourself with the provided tool and bundle. Results should match under the same compatibility profile.
Check tool version: Ensure the verification tool version is documented and matches what you're using.
Review the report: Understand exactly what was checked and what passed.
Combine with other evidence: Verification proves structure; you still need to validate content.
Ask about failures: If any verification failures occurred during development, ask why and how they were resolved.
Don't:
Assume verification means correctness: It doesn't. Verification checks custody, not quality.
Skip independent verification: Don't just trust the evidence; reproduce it.
Ignore policy issues: Even if structurally valid, policy deviations need explanation.
Conflate verification with compliance: Verification is one piece of evidence, not a compliance certification.
The Verification Evidence as Audit Artifact
The verification evidence itself becomes part of the audit trail. It should:
Be retained alongside the evidence bundle
Include enough metadata to reproduce verification later
Be bound to the bundle through cryptographic means
Reference the exact verification tool version used
Years later, an auditor should be able to:
Retrieve the original bundle
Retrieve the original evidence
Obtain the verification tool (same version)
Re-run verification
Confirm the new evidence matches the original under the same compatibility profile
If any step fails, the audit trail is incomplete.
What to Look For in Verification Reports
Useful verification reports often include:
Identity and binding:
Bundle identifier
Evidence reference
Verification tool version
Timestamp (metadata only, not affecting verification outcome)
What was checked:
Contract version declared
Number of artifacts verified
Declared policy requirements applied (if any)
Results summary:
Pass/fail verdict
Count of structural checks performed
Count of policy checks performed
Failure details (if applicable):
Specific artifacts that failed
Specific checks that failed
Failure reasons with enough detail to locate the issue
Reproducibility note:
Confirmation that verification is deterministic
Instructions for independent verification
Common Misinterpretations
Misinterpretation: "Verification passed, so the results are correct."
Reality: Verification confirms structure; correctness requires validation.
Misinterpretation: "Verification failed, so the results are wrong."
Reality: Verification failure means the bundle is incomplete or corrupted; results might still be correct but can't be trusted.
Misinterpretation: "I have a verification evidence packet, so I'm compliant."
Reality: Verification is one piece of evidence in a compliance case, not the whole case.
Misinterpretation: "Verification is just a checkbox for auditors."
Reality: Verification establishes the foundation of trust; without it, validation is built on sand.
Conclusion: Verification as Foundation, Not Ceiling
Verification results tell you whether your evidence bundle is trustworthy as a record. They don't tell you whether the record describes something correct, safe, or compliant.
Think of verification like a notary seal on a document:
The seal proves the document is authentic and unaltered
The seal doesn't prove the document's contents are true or legal
But without the seal, you can't trust anything about the document
How to read verification results correctly:
Verification proves the evidence is intact. Validation proves the evidence is correct. Compliance proves the evidence meets regulatory requirements. You need all three.
Use verification as the foundation it is—solid, essential, and limited in scope. Build validation and compliance on top of it.
And when someone asks, "What does this verification evidence packet mean?" the answer is simple:
"It means this evidence bundle can be trusted as an intact record of what happened. Now let's examine whether what happened was correct, appropriate, and compliant."
For inquiries about verification result interpretation, contact the OEP team.
