From: Paul Hutson firstname.lastname@example.org Subject: [NMusers] Filing Date: 10/21/2003 9:55 PM All: Prompted by Nick's suggestion of thousands of fits of iterated data, with comparison of the objective functions and description of the parameters, I am very curious to learn from you PowerUsers out there how you: 1) Manage and keep track of file names (and any variations in initial estimates associated with each run), 2) Extract the parameter estimates (Thetas, Etas, Etcs) for descriptive and statistical analysis. For example, do you have Fortran strip out the parameters of interest with a WRITE command to a file that is appended with subsequent runs, or do you create a $TABLE text file, then parse it into a spreadsheet like EXCEL with one or two macros that strip out the parameters to another spreadsheet page?. Either way, how do you keep track, in a given line of parameters gleaned from a given run, of what run and what initial estimates it was derived? Thanks in advance for your insight. Paul _______________________________________________________ From: bvatul email@example.com Subject: Re: [NMusers] Filing Date: 10/22/2003 1:59 AM Hello Wings for NONMEM developed by Nick is a very useful tool and one can extract the parameter estimates etc by using simple command files. Venkatesh Atul Bhattaram CDER, FDA. _______________________________________________________ From: firstname.lastname@example.org Subject: Re: [NMusers] Filing Date: 10/22/2003 8:54 AM Paul, Re: your last comment, below is Fortran code for this. Put a call to this (CALL WRITERES) right before the STOP in cfiles.for and recompile including this source code file. By putting just before STOP (after all other files are closed) you can be sure not to change anything else. (please forgive my non-standard use of actual comments in fortran code). We have not tested this with any compiler other than G77. SUBROUTINE WRITERES C BY MARK SALE, GLAXOSMITHKLINE, LAST REVISED 8/13/2003 INCLUDE 'NSIZES' COMMON /ROCM9/ IERE,IERC C IERE AND RC AS INTERNAL ERROR CODES FOR ESTIMATION AND COVARIANCE INTEGER IERE,IERC C MIN IS 0,1 MINIMIZED SUCCESSFULLY INTEGER MIN C CORR IS 0,1 PASSED CORRELATION TEST, COV IS 0,1 PASSED COVARIANCE TEST INTEGER I,N,CORR,P,COV C ABSOLUTE VALUE OF OFFDIAGONAL CORRELATION MATRIX REAL ABSVAL COMMON /CM2/ NETA,NEPS,NRE2 COMMON /CM1/ NTHETA,NTH,DB(LTH),LB(LTH),UB(LTH),ITR(LTH) C OBJECTIVE FUNCTION VALUE COMMON /ROCM8/ OBJECT DOUBLE PRECISION OBJECT C CORRELATION MATRIX IS CORRM COMMON /CM12/ COVM(LPAR3),COVINM(LPAR3),STHTA(LTH),SEN(LVR,LVR), 1 CORRM(LPAR3),NVLS,VLS(LPAR) COMMON /CM4/ MAXFN,NSIG,OPRPT,OPINT,OPFLP,OPETA1,IDUM(8),OPTWO MIN = 1 C ERROR CODES 0 = OK, 139 = BOUNDARY, OTHER IS ERROR IF(IERE.EQ.0.OR.IERE.EQ.139) MIN = 0 C IF IT FAILS MINIMZATION, IT FAILS COVARIANCE IF(MIN.EQ.0.AND.IERC.EQ.0) THEN COV=0 ELSE COV = 1 END IF C open OUTPUT file OPEN(UNIT= 44,FILE = 'GSKNOUT') WRITE(44,8) OBJECT 8 FORMAT ('OBJECTIVEFUNCTIONVALUE=',F16.2) WRITE(44,'(A13,I3)'), 'MINIMIZATION=', MIN WRITE(44,'(A11,I3)'), 'COVARIANCE=', COV C AND THE CORRELATION, FIRST HOW MANY EFFECTS (DIMENSION OF MATRIX) NEFF = NTHETA + (NETA*(NETA+1))/2 + (NEPS*(NEPS+1))/2 CORR = 0 P = 0 DO 66 I = 1,NEFF DO 65 N = 1,I P = P + 1 C CORRELATION MATRIX IS IN CORRM, LOWER TRIANGULAR ABSVAL = ABS(CORRM(P)) IF((ABSVAL.GT.0.95).AND.(I.NE.N).AND.(ABSVAL.LE.1.00)) CORR = 1 65 CONTINUE 66 CONTINUE C Correlation? IF(MIN.EQ.0.AND.COV.EQ.0.AND.CORR.EQ.0) THEN WRITE(44,'(A15)') 'CORRELATION= 0' ELSE WRITE(44,'(A15)') 'CORRELATION= 1' END IF CLOSE(44) END Mark _______________________________________________________ From: email@example.com Subject: Re: [NMusers] Filing Date: 10/22/2003 9:43 AM One other issue of thetas, these are documented in ROCM6 as below COMMON /ROCM6/ THETAF(40),OMEGAF(30,30),SIGMAF(30,30) DOUBLE PRECISION THETAF,OMEGAF,SIGMAF and can be written to a file like this: WRITE(44,'(A5)') 'THETA' DO 27 I = 1,NTHETA WRITE(44,45) THETAF(I) 27 CONTINUE 45 FORMAT (E16.7) Similarly if you want OMEGA DO 50 I = 1,NETA DO 49 N = 1,NETA WRITE(44,60) OMEGAF(I,N) 49 CONTINUE WRITE(44,'(A2)') ' ' 50 CONTINUE 60 FORMAT (30E16.7,$) Again, this code is for the G77 compiler, some of the formats are different for digital/compaq. As for keeping track of them, obviously this needs to be done by software, not by hand. And we would of course recommend grid computing for this. Mark _______________________________________________________ From: "Bachman, William"
Subject: RE: [NMusers] grid computing Date: 10/22/2003 11:04 AM Hi, Mark, We, on the other hand, would recommend cluster computing! (yet another shameless plug). On a serious note, now that we've figured out clusters, I'm looking at interfacing PDx-Pop to grids. I've done some reading up on the Sun Grid Engine. The nice feature of SGE is its heterogeneity (Windows/Linux/UNIX) and of course the improved use of existing corporate CPU power. [Internal versions of PDx-Pop have the same heterogeneous abilility to access either Windows or Linux servers]. Can you share with us some details on the grid you've implemented at GSK? Bill ps I tried your code (with Compaq 6.6B and it worked fine, although the output is a bit sparse and is hard coded with "GSK". I guess I'm not the only "shameless plugger" out there ;) The other sources of output summaries available to users are: 1. Alison Boekman's "nmsee" 2. the .sum file produced by PDx-Pop: PDx-Pop 1.1j Release 4 Run Summary File Run No: 101 Date: Sep 26 2003 10:43 Output Extracted from file: c:\pdxpop1.1j\example1\101.res DataFile: 002.CSV MINIMIZATION STATUS MINIMIZATION SUCCESSFUL NO. OF FUNCTION EVALUATIONS USED: 88 NO. OF SIG. DIGITS IN FINAL EST.: 3.2 ETABAR IS THE ARITHMETIC MEAN OF THE ETA-ESTIMATES, AND THE P-VALUE IS GIVEN FOR THE NULL HYPOTHESIS THAT THE TRUE MEAN IS 0. ETABAR: 0.20E-01 -0.17E-01 P VAL.: 0.64E+00 0.76E+00 MINIMUM VALUE OF OBJECTIVE FUNCTION : 725.602 MODEL DEFINITION TVCL=THETA(1) CL=TVCL*EXP(ETA(1)) TVV=THETA(2) V=TVV*EXP(ETA(2)) K=CL/V S1=V TAD=TIME ; for a single dose only SID=ID DEL=0 IF(F.EQ.0) DEL=1 W1=1 W2=F IPRED=F IRES=DV-IPRED IWRES=IRES/(W1+W2) Y=F + W1*EPS(1); + W2*ERR(2) ERR1=EPS(1) 95% CONFIDENCE INTERVAL DESCRIPTOR/ FINAL ESTIMATE %RSE LBOUND UBOUND VARIABILITY ________________________________________________________________________________ THETA 1 0.00589 7.81% 0.00499 0.00679 CL, L/hr 2 1.44 5.78% 1.28 1.60 V, L ________________________________________________________________________________ INTERINDIVIDUAL OMEGA VARIABILITY 1,1 0.198 43.1% 0.0306 0.365 CV = 44.5% 2,2 0.201 24.8% 0.103 0.299 CV = 44.8% ________________________________________________________________________________ RESIDUAL SIGMA VARIABILITY 1,1 7.81 16.6% 5.26 10.4 SD = 2.79 ________________________________________________________________________________ *Indicates 95% confidence interval that includes zero %RSE is percent relative standard error (100% x SE/EST) _______________________________________________________ From: firstname.lastname@example.org Subject: RE: [NMusers] grid computing Date: 10/22/2003 12:05 PM Bill, We use United Devices Metaprocessor. It is a web services based system. Basically we package up everything needed (nonmem .obj and lib .files, compiler files, control file, data set and a small executable) into a zip file (about 3.5 Mbytes) and send it off to a computer where it is unpacked, compiled, run and the required files zipped up and sent back. All the files on the "agent" computer are then deleted. Nothing is required on the agent computer except operating system and the metaprocessor software. It runs at low priority, so the owner of the computer doesn't notice at all. All the computers currently on the grid are at a single physical site (this is not a metaprocessor requirement, but a NONMEM license requirement), and we use the g77 compiler because Compaq wanted $500,000 for enough licenses for their Fortran. As for the file name GSKNOUT, feel free to change it to GLOBOMAXNOUT, note there is no clause about not modifying code without first consulting with us ;). Mark _______________________________________________________ From: Terri J. Fisher email@example.com Subject: RE: [NMusers] Filing Date: 10/22/2003 12:40 PM Hi Paul, CLINapps Inc. has developed a clustered computing solution called SmartPK that addresses all of your concerns below. It allows you to harness the power of multiple CPUs, handles bootstrapping and batch loading, tracks file names and parameter estimates, and much more. We will be showcasing SmartPK at next week's AAPS meeting in Salt Lake City. For a sneak peak at the SmartPK datasheet, please see our Online Expo entry: http://www.aapspharmaceutica.com/onlineExpo/Exhibitor_Detail.asp?ID=156030 Terri Fisher CLINapps Inc. 910 Hampshire Road, Suite G Westlake Village, CA 91361 USA Phone: (805) 494-1777, Ext. 15 Fax: (805) 494-1702 firstname.lastname@example.org www.clinapps.com _______________________________________________________ From: "Bonate, Peter" email@example.com Subject: [NMusers] grid computing Date: 10/22/2003 1:01 PM Okay, so for those of us who still work in the dark ages, the question I have is, are things really faster? I'm not talking about faster because the processor of the computer is faster, but faster because NONMEM can take advantage of the parallel processing or grids or parallel algorithms or whatever you want to call it. Thanks, pete bonate _______________________________________________________ From: Jill Fiedler-Kelly Jill.Fiedler-Kelly@cognigencorp.com Subject: [NMusers] grid computing Date: 10/22/2003 1:23 PM Paul, Pete, and others, Grid computing is an ideal solution for running NONMEM on lots of systems. We at Cognigen have been using Grid computing for NONMEM for almost 2 years now. As Bill mentioned in his response, we also implemented grid computing to optimize use of our existing computer infrastructure. When you run NONMEM on your desktop PC, regardless of how fast it is, not only are you limited by the number of jobs you can run, but you are also tying up your computer while it is processing those jobs. When we utilize either of our internal grids, scientists can submit hundreds of jobs simultaneously (bootstrapping) and still utilize their workstation for other tasks. Cognigen is currently utilizing three types of grids for NONMEM. One is SUN-based using SUN's SPARC processors; the second is Intel-based running Linux; and the third is for clients which has a web front end for remote access via the Internet. We initially used SUN Grid Engine for the grid platform, but have since moved over to GridEngine, which is the opensource product that Sun Grid Engine is based on. Also, Sun Grid Engine/GridEngine only runs on a Unix host. It does not run on Windows PCs, but the opensource product does allow for job submission from a Windows PC. Jill ______________________________________ Jill Fiedler-Kelly Cognigen Corporation 395 Youngs Road Buffalo, NY 14221 (v) 716.633.3463, ext. 228 (f) 716.633.7404 (e) Jill.Fiedler-Kelly@cognigencorp.com http://www.cognigencorp.com/ _______________________________________________________ From: Jill Fiedler-Kelly Jill.Fiedler-Kelly@cognigencorp.com Subject: [NMusers] grid computing Date: 10/22/2003 1:26 PM Paul, Since the flood gates have now been opened ... In response to your questions about file names, parameter estimates, etc. you may be interested in looking at Cognigen's PERSPECTIVE Hypertext Data Analysis Mapping software. The primary function of this software is the organization of analysis results, which includes documenting these things you mentioned. There is a NONMEM component in this application that generates summary tables of results, calculates model-specific sumary statistics and allows you to assign labels to parameter estimates. The output from PERSPECTIVE is published to HTML and the entire output is also convertable to PDF for storage or submission. If you'd like to take a look at this, go to http://www.cognigencorp.com/perspective. Jill ______________________________________ Jill Fiedler-Kelly Cognigen Corporation 395 Youngs Road Buffalo, NY 14221 (v) 716.633.3463, ext. 228 (f) 716.633.7404 (e) Jill.Fiedler-Kelly@cognigencorp.com http://www.cognigencorp.com/ _______________________________________________________ From: "Bachman, William" firstname.lastname@example.org Subject: RE: [NMusers] grid computing Date: 10/22/2003 1:31 PM Pete, In the case of multiple runs sent to a multiple-processor cluster, the speed-up factor is roughly inversely related to the number of processors in the cluster. From my AAPS poster presentation in Salt Lake City: Mathematical expressions for total run-time as a function of the number of runs in a batch and the number of processors in the cluster, where: P = no. of processors in the cluster (equal speed CPU's assumed) R = no. of run in the batch (equal individual run-times assumed) T(P,R) = total run-time as a function of P and R T1 = run-time for one run on one processor TO = network and software overhead time N = 1, 2, 3, ... (integer values) M = the number of integral multiples of runs compared to the number of processors = (R - modulus (R/P))/P For R>P and R is an integral N multiple of P: T(P,R) = N (T1 - T1/P) + TO For R< P: T(P,R) = (T1 - T1/R) + TO For R>P and R is not an integral multiple of P: T(P,R) = M (T1 - T1/P) + (T1 - T1/R) + TO As T1 increases, TO approaches 0 and is negligible for all but very fast runs. William J. Bachman, Ph.D. Manager, Pharmacometrics Research and Development GloboMax® The Strategic Pharmaceutical Development Division of ICON plc 7250 Parkway Drive, Suite 430 Hanover, MD 21076 410-782-2212 email@example.com _______________________________________________________ From: Nick Holford firstname.lastname@example.org Subject: Re: [NMusers] Filing Date: 10/22/2003 2:28 PM Mark, You wrote: >>(please forgive my non-standard use of actual comments in fortran code). The de facto NONMEM coding standard does not preclude inclusion of 'actual' comments. For example, C DO NOT MODIFY CODE C WITHOUT FIRST CONSULTING WITH THE NONMEM PROJECT GROUP It is however part of the NONMEM standard not to include 'informative' comments. You should be seeking forgiveness for including comments that explain what the code is doing Nick >> >> SUBROUTINE WRITERES >> C BY MARK SALE, GLAXOSMITHKLINE, LAST REVISED 8/13/2003 >> INCLUDE 'NSIZES' >> COMMON /ROCM9/ IERE,IERC >> C IERE AND RC AS INTERNAL ERROR CODES FOR ESTIMATION AND COVARIANCE >> INTEGER IERE,IERC >> C MIN IS 0,1 MINIMIZED SUCCESSFULLY >> INTEGER MIN >> C CORR IS 0,1 PASSED CORRELATION TEST, COV IS 0,1 PASSED COVARIANCE TEST >> INTEGER I,N,CORR,P,COV etc... -- Nick Holford, Dept Pharmacology & Clinical Pharmacology University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New Zealand email:email@example.com tel:+64(9)373-7599x86730 fax:373-7556 http://www.health.auckland.ac.nz/pharmacology/staff/nholford/ _______________________________________________________