Nonparametric Tolerance Interval Approach to Immunogenicity Assay Cut Point Determination
This content discusses the importance of determining the cut point for immunogenicity assays to detect anti-drug antibodies (ADA) in large-molecule therapeutics. It highlights the need for well-developed and validated assays to ensure drug efficacy and safety. The approach involves multiple-tiered validation, including screening and confirmatory assays, as well as characterizing immune responses. The screening assay cut point is crucial and should be statistically determined to minimize false positive rates. The FDA recommends specific guidelines for determining the cut point and emphasizes a balanced design for sample testing. The content also covers the assay format example and provides insights into the methodology for cut point design.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
A Nonparametric Tolerance Interval Approach to Immunogenicity Assay Cut Point Determination Jason Zhang, Senior Principal Statistician Joint work with Wenjia Li, Harry Yang and Lorin Roskos Nonclinical Biostatistics Conference, 2017 Rutgers University
Immunogenicity Immunogenicity (IM) Unwanted immune response to large-molecule therapeutics, called anti-drug antibody (ADA) Potentially affect drug efficacy, safety Clinical effect: from no effect at all to extreme harmful effects A concern for clinicians, manufacturers and regulatory agencies Well developed and validated assay is needed for ADA detection, characterization 2
A Multiple-Tiered Approach for Assay Validation Patient Samples Positive Negative Screening Assay Report Negative Confirmatory Assay Positive Characterization of immune response (titer, NAb, isotype, etc.) 3
Assay Format Example Universal assay capable of detecting most immunoglobulin isotypes Species and matrix independent High sensitivity and good tolerance for drug Fairly good ability of detecting low affinity antibodies ECL Signal ECL-Tag Drug Drug Anti-Drug Antibody Biotin MSD Streptavidin-Coated Plate
Cut Point Design At least 50 samples be tested on 3 different days by at least two analyst (FDA, 2016). Balanced design is usually adopted to minimize the impact of possible confounding factors (see, Shankar, et al. 2008; Yang, et al. (2016)) 5
Screening Assay Cut Point Is the level of assay signal at or above which a sample is defined to be potentially positive (reactive) and below which it is defined to be negative Is determined statistically from drug-na ve samples FDA (2009 guidance) recommends that the cut point have an upper negative limit of approximately 95 percent to yield 5% false positive rate on average 6
Screening Assay Cut Point(Contd) In 2016 guidance, FDA recommends that the cut point for screening assays be determined by a 90% one-sided lower confidence interval for the 95th percentile of the negative control population . Under the new guidance, the lower 90% confidence limit of 95% quantile is nothing but the lower 5%-content, 90%-confidence tolerance limit. So that with 90% confidence, 5% false positive rate is guaranteed. 7
Screening Assay Cut Point(Contd) The idea is initially proposed by Shen, et al. (2015) with a motivation to reserve a certain percentage of false positives and minimize the false negative rate There is a potential impact that the proposed cut point is too low such that the low-signal positives can affect the relationship between ADA positivity and other variables such as PK, efficacy, etc. It is therefore critical to use the appropriate statistical method for cut point determination under the new FDA guidance and evaluate its impact. 8
Statistical Models In a typical cut point evaluation, the vast majority of variability comes from samples(subjects), not from the analysts, or plates. A one-component random effect model is the simplest model to capture the data structure of the cut point design. Traditional random effect model often assumes that the random effect (and error term) has normal distribution. The patient population from which the samples are obtained, however, are heterogeneous and typically has non-normal distribution, especially for screening assays. 9
Statistical Models (contd) Consider the balanced one-way random effect model, where it is assumed that there are I subjects with each subjects being tested J times, possibly by different analysts on different days. Let ???= ? + ??+ ???, ? = 1, ,?;? = 1, ,? (1) where ???? are the (normalized) sample responses, which are identically distributed with a continuous cumulative distribution function ?(?). 10
Statistical Models (Contd) When the random effect model and error term are independently normally distributed, this is the conventional one-way random effect model. Methods for finding tolerance interval under normally distributed one-way random effect model are well-studied in the literature, see the book by Krishnamoorthy and Mathew (2004). 11
Non-normal Parametric Models When the random effect is non-normal, Zhang, et al. (2015) proposed some non-normal random effect models for cut point determination. These models include a skew-t random effect and a log-gamma random effect model Tolerance interval for these models have not been developed but is doable However, in practice, it is often difficult to verify whether the assay validation data does follow a specified parametric model. 12
Nonparametric Tolerance Limit Olsson and Rootzen (1996) proposed quantile estimators and their asymptotic properties for nonparametric one-way random effect model These results can be used to construct confidence limit for quantile estimator and be directly applied to cut point determination In this presentation, we considered the balanced one-way random effect model Unbalanced one-way random effect model can be developed similarly, see, Olsson and Rootzen (1996). 13
Assumption and Notation Denote ???? = 1(??? ?), where 1( ) is the indicator function; ? ? be the correlation function between any two indicators ???? and ???? ,? ?;?2? = ? ? 1 ?(?) be the common variance function. For balanced design, the cumulative distribution function ?(?) can be estimated as ? ? = 1 ?? ?,????? . The variance function ?2? can be 2 .And the correlation function ? ? can be estimated as 1 ?? ?,????? ? ? estimated as ?2? = 1 1 ???? ? ? ???? ? ? ? ? = ? ?2(?) ?(? 1) . ? ? ? 14
Cut Point /Tolerance Limit Estimation Define the qth quantile by ??= inf ?:? ? ? . The quantile can be estimated by ??= inf ?: ? ? ? . ?? has asymptotic normal distribution as I goes to infinity, ?? ?? ??(??)?(??) The screening cut point can be estimated as ?0.95 ?0.9 ?? ? 1 ??1 + ? 1 ? ?? is the density function at ??, ??is the estimator of ????, and ? is the estimator of ?, using kernel smoothing method. ? ?(0,1) ? 1 ? is the variance of ? ?? and ?(??) where ???? = 15
Cut Point Estimator/Tolerance Limit(Cont d) An alternative estimator is ?(0.95 ?0.9 ??), which is shown to have coverage probability of 0.9 asymptotically. We denote ?0.95 ?0.9 ?? ? as NonPar1 and ?(0.95 ?0.9 ??) as NonPar2. The advantage of NonPar2 is the avoidance of estimation of ?. Estimation of ? at tail of distribution is imprecise. 16
Existing Methods for Comparison Shen et al. (2015) considered exact lower confidence limit of normal percentile for IID data: ? ? + ???? ?? 1 + ??? 2?2 1 / ? The method is derived from the pivotal quantity Thus, the resulting tolerance limit is exact in terms of coverage probability under IID normal. 17
Tolerance Limit for One-way Random Effect Model Hoffman and Berger (2011) considered to use confidence-interval type of cut point and proposed to use ? + effect model ??,??,? ?? ?? under one-way random The coverage probability appears to be satisfactory under correct assumption There are actually more than a dozen of tolerance interval methods for normally-distributed one-way random effect model. 18
Nonparametric method for IID data Non-parametric tolerance limit can be used when the data are IID. Assume ?1,?2, ,?? are IID random sample from a continuous distribution and ?(1),?(2), ,?(?) are the corresponding order statistics Suppose a random variable Z follows a Binomial distribution Z Bin(n, 1-p) and let k be the largest integer such that Pr(Z k) 1- , then ?(?) is the desired p-content, 1- confidence lower tolerance limit. Young and Mathew (2014) proposed an improved version of nonparametric tolerance limit by using linear interpolation. We denote Young and Mathew s method as Nonpar3. 19
Simulation 1: Coverage Probability under Normal Distribution 20
Simulation 2: Coverage Probability under Student s t-Distribution 21
Simulation 3: Coverage Probability under Gamma Distribution 22
Discrete Nature of Coverage Probability Coverage Probabilities of Screening Cut Point for Different Number of Subjects under Student s t- Distribution 23
Discrete Nature of Coverage Probability Coverage Probabilities of Confirmatory Cut Point for Different Number of Subjects under Student-t Distribution 24
Conclusion/Remarks The nonparametric method is showed to be more appropriate compared to other methods for one-way random effect model with non-normal distribution Is computationally simple compared with smoothing/bootstrapping method Nonpar 2 method is simpler and performs well for cut point validation data where biological variability is usually dominant. The impact of adopting confidence-interval type of cut point should be carefully assessed using appropriate statistical method 25
References FDA. (2009). Guidance for industry: Assay development for immunogenicity testing of therapeutic proteins. FDA. (2016). Draft Guidance for industry: Assay development for immunogenicity testing of therapeutic proteins. Hoffman, D., & Berger, M. (2011). Statistical considerations for calculation of immunogenicity screening assay cut points. Journal of immunological methods, 373(1), 200-208. Olsson, J., & Rootz n, H. (1996). Quantile estimation from repeated measurements. Journal of the American Statistical Association, 91(436), 1560-1565. Shankar, G., Devanarayan, V., Amaravadi, L., Barrett, Y.C., Bowsher, R., Finco-Kent, D., Fiscella, M., Gorovits, B., Kirschner, S., Moxness, M. and Parish, T. (2008). Recommendations for the validation of immunoassays used for detection of host antibodies against biotechnology products. Journal of pharmaceutical and biomedical analysis, 48(5), pp.1267-1281. Shen, M., Dong, X., & Tsong, Y. (2015). Statistical Evaluation of Several Methods for Cut-Point Determination of Immunogenicity Screening Assay. Journal of biopharmaceutical statistics, 25(2), 269-279. Yang, H., Zhang, J., Yu, B., & Zhao, W. (2015). Statistical Methods for Immunogenicity Assessment (Vol. 85). CRC Press. Zhang, J., Yu, B., Zhang, L., Roskos, L., Richman, L., & Yang, H. (2015). Non-normal random effects models for immunogenicity assay cut point determination. Journal of biopharmaceutical statistics, 25(2), 295-306. Young, D.S. and Mathew, T., 2014. Improved nonparametric tolerance intervals based on interpolated and extrapolated order statistics. Journal of Nonparametric Statistics, 26(3), pp.415-432. 26
