The differences between validation and verification

Introduction

In the ISO/IEC 27037, 27041, 27042 and 27043 group of standards for incident investigations there is a requirement to show that methods are valid. This parallels the same requirement in ISO 17025 which is applied, in a law-enforcement context, to forensic sciences.

The terms validation and verification are well-defined in the standards, but are often misused in conversation and can cause some confusion. This guide aims to help clarify the differences between them

Verification

Verification, simply put, is the process of showing that something conforms to its specification. i.e. it "does what it says on the tin". This is something that a manufacturer or vendor can do, and potentially do very well, because they know exactly what their product is designed to do. Proper verification includes consideration of out of scope or out of bounds conditions and error handling to identify the limitations of the product under test and show that these also match the specification.

However, a vendor or manufacturer cannot show that a particular person or organisation can use that product effectively in order to solve a particular problem.

Validation

Validation, on the other hand, shows that a method or procedure is fit for purpose. In other words, validation shows that the way that someone has chosen to solve a particular problem, or achieve a particular outcome, is capable of producing the required result. It sounds similar to verification, but is subtly different.

Validation is about the implementation of a solution to a problem, from end to end, and includes consideration of every aspect of the process, including any manual steps and means by which tools are combined.

Tools which participate in a method do not have to be used in a way that conforms to their specification. This can be particularly useful in an investigative context as it is common for us to adopt "non-forensic" tools in order to produce results. These tools are not designed or tested for investigations - so are not verified because the use exceeds the specification - but methods which use non-verified tools can still be validated.

Example of validation vs. verification

An investigator has developed a method for freeing stuck heads on hard discs. This involves tapping the drive with a hammer. The hammer is one which has been purchased from a local hardware store and is designed for hammering nails into wood. This usage has been tested many thousands of times and the performance of the hammer has been verified as conforming to the specification.

The hammer is a tool which is, however, not verified as a data recovery assistive tool, but the method of tapping the drive (documented as a standard operating procedure or work instruction) can be validated by the person or organisation using it as long as they can devise sufficient tests to show that their method is repeatable and reliable, and what the limitations (failure points and extent of scope) of the method are.

(N.B. - we don't use this as a way of getting drives to work again, and we don't recommend you do either, unless you really validate it first)

Using verification to aid validation.

ISO/IEC 27041 allows an organisation to use verification data provided by a third party (e.g. the vendor) as part of their validation process. If a tool is only ever used in a manner which conforms to the original specification and the provided verification data is complete, then there may be no need to carry out additional testing on the tool. However, if a method uses a tool in a way which is beyond its original specification, then a full validation of that method must be carried out.

If you have any thoughts on this process, or would like help with preparing for or conducting investigations, please contact n-gate ltd. now.