From: "Nick Holford" n.holford@auckland.ac.nz
Subject: [NMusers] End of semester MCQ and short answer question 
Date: Thu, July 14, 2005 8:54 pm

Q 1. LLQ is the lower limit of quantitification. Do regulatory Pharmacometric groups
endorse?
a. Ignoring LLQ values
b. Imputing LLQ values as 0
c. Imputing LLQ values as 0.5*LLQ
d. Using the actual measurement i.e. ask the chemical analyst to tell the truth
e. None of the above

Q 2. Beal S. Ways to fit a pharmacokinetic model with some data below the
quantification limit. Journal of Pharmacokinetics and Pharmacodynamics
2001;28(5):481-504
Has there been any advance on this in the last 4 years?

Nick

--
Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand
email:n.holford@auckland.ac.nz tel:+64(9)373-7599x86730 fax:373-7556
http://www.health.auckland.ac.nz/pharmacology/staff/nholford/
_______________________________________________________

From: mmunsaka@tgrd.com 
Subject: RE: [NMusers] End of semester MCQ and short answer question 
Date: Fri, July 15, 2005 8:50 am

Hello,

I am also interested in this question and I a currently looking into this question.
I have the following additional references (simple latex syntax) if this may be of
use to you (and you have not already seen these). I don't think I have seen any
regulatory recommendations on this issue and would be curious as to what the
agencies think. I am also interested in other references that other people may have.

Melvin Munsaka, PhD
Takeda Global Research & Development Center, Inc.

=======================================\begin{thebibliography}{99}

\bibitem{Cox} Cox, C. (2005). Limits of quantitation for laboratory assays.
\emph{Appl. Statist.}, \textbf{54}, 63-76.

\bibitem{Clausen} Clausen, W. H. O., Tabanera, R., and Dalgaard (200X?).
Solving the bias problem in censored pharmacokinetic data.
\emph{http://pubhealth.ku.dk/upload/application/pdf/f51d6748/rr-05-5.pdf}.

\bibitem{Duval} Duval, V. and Karlsson, M. O. (2002). Impact of omission
of data below the limit of quantification on parameter estimates in
a two-compartment model. \emph{Pharm. Res.}, \textbf{19}, 1835-1840.

\bibitem{Graves} Graves, D. A., Locke, C. S., Muir, K. T., and Miller, R. P.
(1989). The influence of assay variability on pharmacokinetic
parameter estimates. \emph{J. Pharmacok. Pharmacod.} \textbf{17},
571-592.

\bibitem{Jackson} Jackson,A. J. (1992). Inappropriate inclusion of
non-quantifiable plasma concentrations in the estimation of exyent
of absorption. \emph{Biopharm. Drug Disp.}, \textbf{15}, 629-634.

\bibitem{Hing} Hing, J. P., Woolfrey, S. G., Greenslade, D. and Wright, P. M. C.
(2001). Analysis of toxicokinetics data using {NONMEM}: {I}mpact of
quantification limit and replacement strategies for censored data.
\emph{J. Pharmacok. Pharmacod.} \textbf{28}, 465-479.

\bibitem{Hoffman} Hoffman, W. P., Heathman, M. A., Chou, J. Z., and
Allen, D. L. (200X?). Analysis of toxicokinetic and pharmacokinetic
data from anaimal studies. In S. P. Millard and A. Krause (eds),
\emph{Applied Statistics in the Pharmaceutical Industry With Case
Studies Using S-Plus}. Chapter 4.

\bibitem{Humbert} Humbert, H., Cabiac, M. D., Barradas, J., and Gerbeau, C.
(1996). Evaluation of pharmacokinetic studies: {Is} it useful to
take into account concentrations below the limit of quantification?.
\emph{Pharm. Res.}, \textbf{13}, 839-845.

\bibitem{Sadler} Sadler, W. A. and Smith, M. H. (1990). Use and
abuse of imprecsion profiles: some pitfalls illustrated bu computing
and plotting confidence intervals. \emph{Clin. Chem.}, \textbf{36},
1346-1350.

\bibitem{Tod} Tod, M (2005). Handling concentrations below quantification
limit in population. Presentation at the 2005 PAGE Meeting.
_______________________________________________________

From:  "Nick Holford" n.holford@auckland.ac.nz
Subject: Re: [NMusers] End of semester MCQ and short answer question 
Date: Sun, July 17, 2005 11:34 am

Thanks to those who took the test and responded to me. Not all responses were sent
back to the open lists so I have anonymised all the responses.

Question 1
==========
Extra marks were given to the PharmPK user who pointed out I should have used BLQ
instead of LLQ. 
"I think what you meant to say in items 1(a) to (c) is "BLQ" (below the limit of
quantitation) instead of "LLQ", since a value that is at the LLQ can just be
reported as such as it is contained within the validated range."
Quite correct! What I meant to ask would have been better expressed as follows. I
have changed "LLQ" to "LLOQ" and use "quantification" (not "quantitation") for
consistency with the FDA Bioanalytical guidance (see below).

Q 1. LLOQ is the lower limit of quantification. Measured concentrations less than
LLOQ are said to be below the limit of quantification (BLQ). Do regulatory
Pharmacometric groups endorse?
a. Ignoring BLQ values
b. Imputing BLQ values as 0
c. Imputing BLQ values as 0.5*LLOQ
d. Using the actual measurement i.e. ask the chemical analyst to tell the truth
e. None of the above

The most definitive material pertinent to Q1 came from the following quote from the
Code of Federal Regulations. It was provided by a ex senior FDA person who would
have dealt commonly with this kind of issue. This person and another current senior
FDA person said they knew of no FDA guidance requiring the use of LLQ to modify data
used for PK analysis.

"Sec. 320.29  Analytical methods for an in vivo bioavailability or 
bioequivalence study.

    (a) The analytical method used in an in vivo bioavailability or 
bioequivalence study to measure the concentration of the active drug 
ingredient or therapeutic moiety, or its active metabolite(s), in body 
fluids or excretory products, or the method used to measure an acute 
pharmacological effect shall be demonstrated to be accurate and of 
sufficient sensitivity to measure, with appropriate precision, the 
actual concentration of the active drug ingredient or therapeutic 
moiety, or its active metabolite(s), achieved in the body."

There is a 2001 Bioanalytical Method Validation guidance that defines LLOQ as
concentrations with 20% CV (http://www.fda.gov/cder/guidance/4252fnl.pdf). It says
nothing that I can see about whether LLOQ should be applied when doing a PK
analysis. Note that Bioanalytical Method validation statistics such as LLOQ are used
to describe the properties of the assay. This guidance does not define how the
concentrations are to be used.
A subsequent 2003 Bioavailability and Bioequivalence document describes PK
procedures but does not mention the use of LLOQ
(http://www.fda.gov/cder/guidance/5356fnl.pdf).

There seems to be a common mis-perception that FDA requires the use of LLOQ in a PK
analysis however no-one has provided any written evidence of this policy so far.

My interpretation of these responses is that the closest answer to Q1 should be:
 "d. Using the actual measurement i.e. ask the chemical analyst to tell the truth"


Question 2
==========
Almost full marks to the nmusers peson who gave a list of references of various
opinions on dealing with BLQ values. Some credit was lost for using an obscure
(Latex <grin>) character based formatting convention and providing a non-retrievable
reference to recent work attributed to M. Tod.

A prominent user of NONMEM responded:
"The only advances have been 
I) my increasing conviction that whenever the pattern of BLQ values is consistent
with the observed PK, that (a) [ignoring BLQ values], along with the adjustment
described by Beal (2001) to account for the bias, is the way to go, and
(II) the addition of a feature in NONMEM version VI that allows this sort of thing
to be easily done."

Nick
_______________________________________________________

From:  "Leonid Gibiansky" leonidg@metrumrg.com
Subject: Re: [NMusers] End of semester MCQ and short answer question 
Date: Sun, July 17, 2005 12:26 pm

Nick,
Your option
d. Using the actual measurement i.e. ask the chemical analyst to tell the truth" should
help to limit LLOQ somehow but at the end you will still get some zero values that
are measurements 
below LLOAATDCFZ (lower  limit of assay ability to distinguish concentration from
zero). Then, you 
will need to choose among 3 other options:
  a. Ignoring 0 values
  b. Using 0 values as 0
  c. Imputing 0 values as 0.5*LLOAATDCFZ
Remembering the advice of "a prominent user of NONMEM" (I am not sure whether this
is the same user 
as your adviser, but I found on more than one occasion that the advices were good),
I usually use 
(a). Do you have any examples where this would lead to the incorrect model? Also,
have you found any 
examples where option (d) was better (resulted in a more precise or more stable
model) than option (a)?

Unfortunately, information concerning NONMEM VI is not relevant to the most of
mortals except those 
few who were granted this tool (I guess, as a recognition of the special
contribution to the NONMEM 
development).
Thanks
Leonid
_______________________________________________________

From: "Nick Holford" n.holford@auckland.ac.nz
Subject: Re: [NMusers] End of semester MCQ and short answer question 
Date:  Sun, July 17, 2005 12:58 pm 

Leonid,

LL0AATDCFZ and its bigger brother LLOD (lower limit of detection) are just as
arbitrary and capricious as LLOQ when it comes to PK analyis. I accept they can be
helpful statistics for those involved in the care and feeding of bioanalytical
methods. 

However, if the chemical analyst (or the computer connected to the measuring device)
was required to report the truth then if the concentration was really zero it should
report a random variable with mean 0 (assuming the measurement process does not get
truncated at zero). The variance of this random variable is a component of the
additive residual error we estimate every day for PK models. So I don't see any need
to apply LL0AATDCFZ. Just give me the true measurement value.

One thing is sure about the true concentration -- until sufficient time has passed
for less than one molecule to be left in the body then the concentration is not 0.
This is longer than most people live...

Stuart Beal has offered some examples of what happens if answer (a) is used (treat
BLQ values as missing). You can also find some more examples in Duval V, Karlsson
MO. Impact of omission or replacement of data below the limit of quantification on
parameter estimates in a two-compartment model. Pharm Res 2002;19(12):1835-40.

I'm afraid I don't have any personal experience comparing the true measurement with
the obscured values reported by chemical analysts.

Nick
_______________________________________________________

From: "Leonid Gibiansky" leonidg@metrumrg.com
Subject: Re: [NMusers] End of semester MCQ and short answer question 
Date: Sun, July 17, 2005 2:14 pm 

Nick,

Thanks for the reference, results seems very reasonable: if you ignore zeros,
predicted 
concentrations may decay not as fast as they should (leading to lower CL, higher V).
However, this 
might be strongly related to the design and sampling. In my examples, fraction of
BQLs was 
relatively small (less than 5%), most zeros were very suspicious (related either to
the 
non-compliance or data errors) because LLOQ was 3-4 orders of magnitude less than
Cmax while 
sampling points were not that far from the dose to warrant zeros. With the good
design, fraction of 
BQLs in the data set is small, and efforts that are needed to include those are not
warranted by the 
gain that you may get from inclusion of those points.

As to the true values, you may be forced to use special segment of the error model
to account for 
the BQL measurements. This can be very similar to LLOQ/2 imputation with BQL
variance fixed at 
(LLOQ/2)^2.

My justification of the idea to ignore zeros (or not to use "true" values) is that
we are not 
interested in the very fine details of the PK behavior (the deaper you look, the
more compartments 
you may descover) and restrict the model to the range of concentrations that are
relevant to your 
problem. Sampling should also be consistent with the goal of the study: samples
should be located at 
characteristic points of the profiles (while BQLs are definitely not in that
range/position).

One can imagine situations when zeros are important: for example, if there is a
sub-population with 
much higher CL: ignoring zeros may hide the fact that concentration decay for a
sub-population is 
much faster than was expected, especially if the sample timing was not designed for
the 
sup-population with the high CL. But even in this case, BQL values may help you to
discover the 
problem with your design, but will not help to build the correct model: it is
difficult to build 
correct model based on the very noisy measurement. If you do not have sufficient
non-BQL time points 
to define the terminal phase and impute some BQL values (or use the true value that
will be a 
reflection of the random noise generated by the instrument), the model for the
subpopulation will be 
defined by the timing of the sampling point rather than by the actual CL. If you
have sufficient 
number of non-BQL time points to define the terminal phase, ignoring the BQLs should
not adversely 
affect the model.

Leonid
_______________________________________________________

From: "Nick Holford" n.holford@auckland.ac.nz
Subject: Re: [NMusers] End of semester MCQ and short answer question 
Date: Sun, July 17, 2005 3:25 pm 

Leonid,

Your discussion agrees with what Stuart Beal wrote. In most cases treating BLQ
values as missing is fine. In some special cases there is a small amount of useful
info from knowing that it is BLQ (but not missing). Imputation of BLQ with LLOQ/2 is
a quick and dirty way to partially recover some of this information.

Nick
_______________________________________________________