From: Bonate, Peter pbonate@ilexonc.com
Subject: [NMusers] NONMEM Estimation algorithms  
Date: Thu, May 20, 2004 8:59 am  

Dear all, 
I am having some trouble understanding some of the estimation
algorithms used in NONMEM and was hoping someone would take
the time to help me out.  Here are some of my questions:

1.) How does what NONMEM does with FOCE differ from the
Lindstrom and Bates conditional algorithm (1990)? 

2.) Is the Laplacian method in NONMEM the same as the Laplace
method suggested by Wolfinger (1993)?  Some have indicated that
the Laplacian method in NONMEM is a second-order Taylor series
approximation about the nonlinear model itself, like FO-approximation
(Racine-Poon and Wakefield, 1998).  But it looks from the NONMEM manuals
that the -2LL for the Laplace option is a second-order Taylor series
approximation about the likelihood function itself.  Which is it? And
this leads to my third question.

3.) Where does the equation on page 5 in the Part VII manuals come from? 
How is it derived?  I have seen no other place in any paper where this
form of the -2LL is presented.  So on a related note are conditional
algorithms published anywhere outside of the NONMEM manuals.  It seems
that the FO algorithm is described in detail in many papers, but the
algorithms that NONMEM actually uses are not published anywhere. 
It this correct?

Thanks for your time if you can help me. 

Pete bonate 



Peter L. Bonate, PhD, FCP 
Director, Pharmacokinetics 
ILEX Oncology 
4545 Horizon Hill Blvd 
San Antonio, TX  78229 
phone: 210-949-8662 
fax: 210-949-8219 
email: pbonate@ilexonc.com 
 

_______________________________________________________

From: Wang, Yaning WangYA@cder.fda.gov
Subject: RE: [NMusers] NONMEM Estimation algorithms
Date: Fri, May 21, 2004 8:39 am 

Hi, Peter:
 
Please see the following comments (they are just my personal opinions). 

 
1)
On page 174 of Marie Davidian and David Giltina's book, Nonlinear Models for Repeated Measurement data
(Chapman & Hall, 1995), NONMEM's FOCE was described as "a full normal theory maximum likelihood version
of the method of Lindstrom and Bates". I don't quite agree with this statement. In my opinion, L & B's method
can be described as a first order Taylor approximation of the nonlinear model around current estimates of both
theta and eta (I will use NONMEM terminology for fixed-effect parameters and random effect parameters).
NONMEM's FOCE without interaction can be described as a first order Taylor approximation of the nonlinear
model around the posterior mode of eta. FOCE without interaction can also be derived from Laplacian approximation.
But this is not the case for FOCE with interaction (see chapter 7 in Edward Vonesh and Vernon Chinchilli's book,
Linear and Nonlinear Models for the Analysis of Repeated Measures, Marcel Dekker 1997). Similarly, NONMEM's FO
method (expansion of nonlinear model around 0 for eta) can also be derived from Laplacian approximation. I believe
that NONMEM's FOCE is superior to L&B's method in terms of approximation accuracy, but L&B's method can
provide restricted maximum likelihood estimators. In Pinhero and Bates' book (chapter 7), Mixed-Effect Models in
S and S-plus, what they called "modifed Laplacian approximation" is basically FOCE in NONMEM in my opinion. The
accuracy rank of different approximation methods should be Adaptive Gaussian Approximation (SAS PROC NLMIXED)>Laplacian
Approximation (NONMEM Laplacian)>FOCE (NONMEM)>L&B's Method (Splus nlme)>FO(NONMEM and SAS). The superiority of
Laplacian in NONME over FOCE is certain for "without interaction" case. I am not sure about "with interaction" case
since only FOCE has this option while Laplacian doesn't (at least for the current version of NONMEM). Despite its
better approximation accuracy, Laplacian method will force "no interaction".  The impact of this on parameter
estimation is unknown. In fact, most of time our residual models, such as proportional model or combination
model, indicate interaction. 

2)
No, I don't think the two Laplacian methods are the same even though the reinterpretation of Wolfinger's method by Marie
Davidian and David Giltina (chapter 6.3.3) made them look so similar. First, Wolfinger's method treated the marginal
likelihood as an integral with respect to both the random effects and the fixed effects with a tacitly present flat
prior for the fixed effects while the Laplacian method in NONMEM treated the marginal likelihood as an integral with
respect to the random effects only. Second, Wolfinger's method led to the restricted maximum likelihood version of L&B's
LME step while the Laplacian method in NONMEM (also described in details in Pinhero and Bates' book) cannot lead to restricted
maximum likelihood estimators unless another Taylor expansion of the model function was done around the current estimates of
theta to make the fixed effects enter the model linearly. In other words, L&B's method involves further approximation
compared to FOCE in NONMEM but with the advantage of being able to obtain restricted maximum likelihood estimators. 

I think there is some misunderstanding about the second-order Taylor expansion in Laplacian method. It is often compared to
the first-order expansion to show its better accuracy in approximation.  In fact, this second-order Taylor expansion is
referring to the Taylor expansion of the log of the integrand that is used to calculate the marginal likelihood (see my
derivation of NONMEM VII likelihood for details). If derived from Laplacian approximation, Laplacian, FOCE and FO all use
second-order Taylor expansion in the Laplacian approximation step. When there is no interaction, FOCE can be thought of
as a first-order Taylor expansion of the nonlinear model around eta^ just like FO is a first-order Taylor expansion of
the nonlinear model around 0 for eta. But I have never seen any paper that proved the second-order Taylor expansion in
Laplacian method is equivalent to second-order Taylor series approximation of the nonlinear model around eta^.

3)
Please see the nonmem.pdf on the following link for detailed derivation of the equation on page 5 of NONMEM VII. I don't
know how to attach file to this mail list. So I put it on my webpage. 

http://www.geocities.com/wangyaning2004/nonmem.pdf

I hope this helps.
 
Yaning Wang, PhD 
Pharmacometrician 

OCPB/FDA

_______________________________________________________