An out-of-trend (OOT) result that does not follow the expected trend, either in comparison with previous results collected from past history. This article discusses the possible statistical approaches and implementation challenges to the identification of OOT results.

It is intended to begin a conversation toward achieving clarity about how to address the identification of out-of-trend results. It is noted that the identification of OOT results is a complicated issue and that further research and discussion is needed. This article is not a detailed proposal but is meant to begin the discussion toward achieving more clarity about how to address the identification of out-of-trend results.

Regulatory Basis

A review of recent Establishment Inspection Reports (EIRs),FDA 483s, and FDA
Warning Letters indicates the identification of OOT data is becoming a regulatory issue for marketed products. Several companies have received 483 observations requesting the development of procedures documenting how OOT results will be identified and investigated. It is important to distinguish between OOS and OOT results. FDA issued a OOS guidance in the scientific literature and discussed at many scientific conferences about OOS results.

Although the FDA guidance indicates in a footnote that much of the guidance presented for OOS can be used to examine OOT results, there is no clearly established legal or regulatory basis to require consideration of data within specification but not following expected trends.

Identification of Out-of-Trend Results

Avoiding potential issues with marketed product, as well as avoid potential regulatory issues apply of OOT control in the analysis is a best practice in the industry. In summary, the issue of OOT is an important topic both from a regulatory and business point of view. Despite this, little has been discussed in the scientific literature or in regulatory guidance on this topic. This article will introduce some approaches that might be used to identify OOT data and discuss some issues that companies will likely need to address before implementation and during use of an OOT identification procedure.

Statistical Approach Background

There is a need for efficient and practical statistical approach to identify OOT results to detect when a batch is not behaving as expected. To judge whether a particular result is OOT, one must first decide what is expected and in particular what data comparisons are appropriate.

Methodology [3 sigma approach]

• A minimum of 25 – 30 batches data shall be compiled for fixing the Trend range.
• Results that shall be obtained from the 25 batches tabulated, average value,
  minimum and maximum values are noted.
• Standard deviation will be calculated for these 25 batches. Excel spread sheet
  shall be used for Standard deviation calculation.
• Standard deviation will be multiplied by 3 to get the 3 sigma (3 )) value.
• Maximum limit will be arrived by adding the 3 1 value to the Average value of
  25 batches.
• Minimum limit will be arrived by subtracting the 3 1 value from the Average
  value of 25 batches. Minimum value may come in negative also at times.
• The above maximum and minimum limits in 4.1.5 and 4.1.6 shall be taken as the
  Trend range for upper and lower limits.
• Any value that shall be out of this range will be considered as Out of Trend
  (OOT) value or Outlier value.
• Wherever specification has only Not more than, then only Maximum limit for
  trend can be considered. Minimum limit should be excluded.
• Wherever specification has range then both the Maximum and Minimum limits
  for trend should be considered.

Limitations

One advantage of this approach is that as long as the assumptions are met, the rate of false positives can be set when one calculates the limits. However, a disadvantage is the products with limited data, the appropriate limits may be difficult to determine.This can lead to wrongly centered, too narrow, or too wide OOT limits.

Implementation challenges

The purpose of developing a criterion for OOT assessments is to identify the quantitative analytical results during a study that are atypical enough to warrant a
follow-up investigation. Numerous challenges exist that a company must overcome to implement an OOT procedure for commercial batches are …

• What statistical approaches are used to determine OOT criterion? What data are used to determine OOT limits?
• What are the minimum data requirements? What evaluation is performed if the minimum data requirement is not met?
• What data should be used to update limits?
• The investigation requirements (i.e., who is responsible, what is the timeline, how is it documented, who should be notified must be clearly defined.
• Who is responsible for comparing the result with the OOT criterion?
• How is an OOT result confirmed? What additional analytical testing or statistical analyses are appropriate?
• What actions should be taken if an OOT result is confirmed as an unusual result?
• How are OOT investigations incorporated into the annual product review?

Conclusion

Identifying OOT results is a growing concern for FDA and the pharmaceutical industry. Ideally, the method to determine an OOT alarm should not be too complex.

References

• A.M.Hoinowski et al., “Investigation of Out-of-Specification Results,”
  Pharm. Technol. 26 (1), 40–50 (2002).
• FDA Guidance Document, Investigating Out of Specification (OOS)
  Test Results for Pharmaceutical Production.