OOS Investigation Phase II (MHRA): Retesting and Averaging

In short: Phase II of the MHRA out-of-specification (OOS) approach starts when the Phase I laboratory investigation finds no assignable laboratory error. It combines a manufacturing investigation with written, pre-approved hypothesis testing, and retesting or resampling only under a defined plan. This note is for QC analysts, supervisors and QA reviewers.

See also: OOS Phase Ia and Ib (MHRA) · OOS Phase I (US FDA) · OOS Phase II (US FDA)

When Phase II applies

Phase II is carried out when the Phase I investigation does not reveal an assignable laboratory error. It is driven by written, approved instructions that test a stated hypothesis.

• Start a manufacturing investigation before any further testing, to find out whether there is a possible manufacturing root cause.
• If not already done, notify the contract giver, marketing authorisation holder (MAH) or QP (as set out in the technical agreement), along with production and QA at a manufacturing site.
• Any additional testing must follow a predefined retesting plan. Include retests by an analyst other than the original one, who should be at least as experienced and qualified in the method.
• If analyst error is confirmed, review all analyses using the same technique by that analyst.

Key terms

Phase II when no cause is found

Hypothesis testing

Hypothesis testing applies to Phase Ia and Phase II. It should begin in Phase Ia and continue into Phase II if no assignable cause is found. QA (or the contract giver or equivalent) must approve a written description before any investigational testing starts. The description must fully document:

• the hypothesis, that is the root cause being tested;
• which samples will be tested;
• exactly how the testing will be carried out;
• how the data will be evaluated.

Hypothesis testing may continue from re-measurement of the original preparations. It cannot replace the original suspect result. It can only confirm or discount a probable cause.

If neither the manufacturing investigation nor the assay failure investigation identifies an assignable cause, retesting may be considered.

Retesting

• Perform it on the original sample, not a different one. A second aliquot of the sample that gave the failing result is acceptable.
• If too little original sample remains, the resampling procedure must be discussed and agreed with QA or the contract giver, and recorded in the laboratory investigation.
• The decision to retest must rest on sound scientific judgement, and the test plan must be approved before retesting begins.
• The minimum number of retests should be written in the procedure and be scientifically justified. Any review of %RSD and repeatability should relate to the values from method validation (accuracy, precision, intermediate precision).
• The number of retests must be specified in advance in a written SOP, may vary with the variability of the test method, and must rest on scientifically sound principles. It must not be adjusted depending on the results obtained, and the procedure should state the point at which retesting ends and the batch is evaluated. Repeating tests until a passing result appears (“testing into compliance”) is unscientific and objectionable under CGMP (US FDA Guidance for Industry: Investigating Out-of-Specification (OOS) Test Results for Pharmaceutical Production, Revision 1, May 2022, section on retesting).
• Use a different analyst where possible, at least as experienced and qualified in the method as the original analyst.

Averaging

• Whether averaging is valid depends on the sample and the purpose of the test. It can give a more accurate result where the test system is inherently variable, for example microbiological assays, kinetic scans of individual wells, endotoxin data from consecutive measurements, or consecutive replicate injections from one HPLC preparation (treated as one test giving one result). Unexpected variation between replicates should still trigger investigation and documentation.
• Averaging must be specified by the test method.
• Averaging cannot be used where the test is meant to measure variability within the product, such as powder blend uniformity or dosage form content uniformity.
• Averaging can hide variability between individual results. Report individual results as separate values unless the method specifies averaging; then a single averaged result may be reported as the final result. Sometimes the variability is also reported, for example the standard deviation (or RSD) along with individual unit results in content uniformity.
• During an OOS investigation, do not average the original result with retest or resample results. This hides variability and misleads, especially when some results are OOS and others are within specification. The laboratory must give QA (contract giver/QP) all individual results.
• A low assay result that is still within specification should also raise concern, because one cause may be that the batch was not formulated properly. Batches must be formulated with the intent to provide not less than 100 percent of the labelled or established amount of active ingredient (21 CFR 211.101(a)), so caution is needed in the release or reject decision (US FDA OOS Guidance, Revision 1, May 2022, section on borderline results within specification).

Confidence limits of the mean

The 95% confidence limits (CL95%) of the mean show the variability when averaging is used:

CL = mean ± t(95%) × s / √n

where s is the sample standard deviation, n is the number of results and t is taken from tables.

Values are the standard two-sided 95% Student t values for n − 1 degrees of freedom (checked: df 1 = 12.71, df 2 = 4.30, df 3 = 3.18, df 4 = 2.78, df 5 = 2.57, df 6 = 2.45, df 9 = 2.26, df 19 = 2.09, df 119 = 1.98, infinite = 1.96).

Resampling

Resampling should rarely occur. It is considered when too little original sample remains, the original sample was not representative of the batch, there was a documented and traceable laboratory error in its preparation, or evidence shows the sample is compromised or invalid. The procedure must be agreed with QA or the contract giver and recorded in the laboratory investigation. Use the same qualified sampling methods as for the initial sample. If the investigation shows the original sampling method was in error, develop, qualify and document a new one. Sound scientific justification is needed.

Outlier tests

An outlier may come from a deviation from the test method or from real variability in the sample. Never assume it is a testing error rather than inherent sample variability. Statistical outlier analysis can be part of the investigation. For validated chemical tests with small variance and a sample considered homogeneous, however, an outlier test cannot be used to justify rejecting data.

The OOS guidance is not directly intended for bioassays, but it can be a starting point. Compendia such as BP, Ph. Eur. and USP give outlier guidance for these tests.

Frequently asked questions

When does Phase II start? When Phase I finds no assignable laboratory error. Start the manufacturing investigation before further testing.

Can I average the original OOS result with retest results? No. Report all individual results to QA, because averaging hides variability.

Can an outlier test invalidate an OOS result? Not for a validated chemical test with small variance on a homogeneous sample. It may be part of the investigation only.

How many retests are needed? The MHRA approach moves away from a fixed number. Define the number in an approved, scientifically justified plan.

Who should do the retest? Where possible a different analyst who is at least as experienced and qualified in the method.

References

1. MHRA (UK), Out-of-specification investigations, GOV.UK publication page, published 28 August 2013, last updated 26 February 2018. Guidance attachment: “Out of Specification & Out of Trend Investigations” slides (file MHRA_OOS_OOT_Oct17). https://www.gov.uk/government/publications/out-of-specification-investigations
2. US FDA, Guidance for Industry: Investigating Out-of-Specification (OOS) Test Results for Pharmaceutical Production, Revision 1, May 2022 (for comparison with the FDA approach).