FW: [NMusers] Predictive Performance

Now I know that the email address I used before was incorrect... :-)
> -----Original Message-----
> From: Kimko, Hui [PRDUS]
> Sent: Tuesday, July 03, 2007 10:48 AM
> To: 'navin1180_at_gmail.com'
> Cc: 'owner-nmusers_at_globomaxnm.com'
> Subject: FW: [NMusers] Predictive Performance
> Dear Navin et al.,
> For external validation, you should use population predictions to =
calculate precision and bias, because you are testing the performance of =
your model, built on a model-building dataset, by predicting a new test =
data using its covariate information only: the IPRED is also based on =
the observed test data set values. As you implied, the diagnostic plot =
of IPRED vs. DV would be good to have. IPRED is used to calculate AIPE.
> Since we are discussing Predictive Performance, I'd like to take this =
opportunity to share what the late Vladmir Piotrovsky was working on, =
before he passed away. He was writing an S+ library, which is called "S =
speaks NONMEM". Below is the S+ code to do external validation, with =
breaking the correlation in residuals within each subject when measures =
of predictive performance are computed. This computation is a =
modification of Beal and Sheiner's MPE and RMSE calculation method. More =
detailed description can be found on page 192 of:
> Vermeulen A, Piotrovsky V, Ludwig E, Population PK of Risperidone and =
9-hydroxyrisperidone in patients with acute episodes associated with =
bipolar I disorder. JPKPD 34(2):183-206, 2007.
> In remembrance of our beloved Vladmir,
> Hui
> Advanced Modeling & Simulation, J&J Pharmaceutical Research & =
Development, LLC
> =
==========================
==========================
=============
> ################# Piotrovsky's resampling residual method for external =
validation
> ################# Modification of the method by Sheiner and Beal:
> ################# Some suggestions for measuring predictive =
performance.
> ################# J. Pharmacokinet. Biopharm. 1981; 9(4):503-512
> ################# Author: V. Piotrovsky
> tab <- read.table("C:\\..\run100.txt",header=T,skip=1)
> attach(tab[tab$WRES!=0 ,]) # your table
> z _ split(RES,ID)
> w _ split(RES^2,ID)
> detach()
> mpe _ numeric(0)
> rmse _ numeric(0)
> n _ 1000 # sample size
> for(i in 1:n) mpe[i] _ median(sapply(z,sample,size=1,repl=T))
> for(i in 1:n) rmse[i] _ sqrt(mean(sapply(w,sample,size=1,repl=T))) =

> # Plotting
> par(mfrow=c(1,2),mar=c(5,5,2,0))
> hist(mpe,col=0,prob=T,xlab="Median population prediction =
error",ylab="Frequency",cex=1)
> lines(density(mpe,wid=.1,n=100))
> abline(v=quantile(mpe,.025),lty=4)
> abline(v=quantile(mpe,.975),lty=4)
> abline(v=median(mpe),lwd=4)
> box()
> hist(rmse,col=0,prob=T,xlab="Root mean squared prediction =
error",ylab="",cex=1)
> lines(density(rmse,wid=.1,n=100))
> abline(v=quantile(rmse,.025),lty=4)
> abline(v=quantile(rmse,.975),lty=4)
> abline(v=median(rmse),lwd=4)
> box()
> median(mpe)
> mean(mpe)
> quantile(mpe,.025)
> quantile(mpe,.975)
> median(rmse)
> mean(rmse)
> quantile(rmse,.025)
> quantile(rmse,.975)
> -----Ursprüngliche Nachricht-----
> Von: "navin goyal" <navin1180_at_gmail.com>
> Gesendet: 02.07.07 20:10:56
> An: nmusers <nmusers_at_globomaxnm.com>
> Betreff: [NMusers] Predictive Performance
> Hi everybody,
> I had a question about the Predictive performance of the POPPK Model.
> When I am estimating the precision and bias with the POPPK model I =
have, am I supposed to use the
> individual predictions or the population predictions ???
> I am using "Some suggestions for Measuring Predictive Performance" by =
Sheiner and Beal : J Pk and Bio Vol (:(4) 1981 :503-512 as reference.
> I guess I should be using the population predictions to calculate the =
precision and bias as I want to use the model to predict the plasma =
concentrations. Or does this choice depend on anything else ??
> If I am using the Population predictions then, where else would I be =
using the individual Predictions apart from plotting them against the DV =
to evaluate the Goodness of Fit?>
> Thanks in advance
> --
> --Navin
>

Received on Thu Jul 19 2007 - 16:47:36 EDT