RE: [NMusers] hardware question

Toufigh,
  Based on a fairly small sample, my own experience:
Differences in the 3rd digit of the parameter are not uncommon.
Differences in the 2nd digit, in my experience, are fairly uncommon. I
personally haven't seen any differences in the 1st digit. Tom Ludden has
at least one model/data set that seems particularly sensitive.
 Perhaps even more common at failure to converge/do covariance step on a
different machine/compiler/compiler options.
 So, I'd suggest it is very uncommon to see differences that would led
to different decisions. But the regulatory implication of not
converging may be a problem. I also suspect that NONMEM VI will, as
Tom and Diane suggest, be better - especially if you use the SORT
option (from the shameless plug department of Next Level Solutions)

Mark Sale MD
Next Level Solutions, LLC
www.NextLevelSolns.com


> -------- Original Message --------
> Subject: RE: [NMusers] hardware question
> From: "Toufigh Gordi" <tgordi_at_Depomedinc.com>
> Date: Tue, January 09, 2007 3:40 pm
> To: <nmusers_at_globomaxnm.com>
>
> Hi,
>
> Could anybody provide some more info on how big the differences in
> parameter estimates are on different machines/compilers? In other words,
> are the differences large enough to be of concern when it comes to
> predicting the outcome of a study design or treatment, or are we talking
> about clinically insignificant differences?
>
> Toufigh Gordi
>
> -----Original Message-----
> From: owner-nmusers_at_globomaxnm.com [mailto:owner-nmusers_at_globomaxnm.com]
> On Behalf Of Gastonguay, Marc
> Sent: Tuesday, January 09, 2007 12:09 PM
> To: Mark Sale - Next Level Solutions
> Cc: nmusers_at_globomaxnm.com
> Subject: Re: [NMusers] hardware question
>
> Mark -
> Thanks for your support of NMQual. You're correct that it doesn't make
> all of the differences go away. It simply documents and in some cases
> implements the details (including all of those you've listed below) of
> the hardware/software configuration used for any given run &
> installation. In order to reproduce results (in those cases where
> configuration makes a difference), a reviewer would have to have to
> ensure that they are running on the same configuration.
>
> Nice try for a solution... Obviously the practical and important
> strategy is to explore the impact that possible configuration-related
> differences in results (as well as other model and parameter
> uncertainties) will have on the inferences we draw from our
> modeling/simulation efforts.
>
> Marc G.
>
>
> Mark Sale - Next Level Solutions wrote:
> > Thanks Marc,
> > While I'm a big fan of NMqual (I use it myself), it isn't clear to
> me
> > that it solves much of this problem. I think (personal opinion
> lacking
> > any data), the order of frequency in causes of differences are:
> > Different compiler (some people using g77)
> > Different compiler options (NMqual addresses this - right?, assuming
> > that both parties use NMqual)
> > Different hardware (some people using UNIX, a few still using Vax??,
> > possibly AMD vs Intel?)
> > Differences in OS (Windows vs Linux - haven't personally documented
> > this)
> > 32 vs 64 bit - even within the Intel compiler 9.0, there can be
> > differences, some of the variable are different sizes.
> > Failure to fix bugs (addressed by NMqual)
> >
> > Having said that, please not that I don't have a solution to the
> others
> > (except everyone should buy 64 bit Intel windows and run Windows XP64
> > and Intel 9.0 compiler, and use NMqual.
> >
> >
> > Mark Sale MD
> > Next Level Solutions, LLC
> > www.NextLevelSolns.com
> >
> >
> >
> >> -------- Original Message --------
> >> Subject: Re: [NMusers] hardware question
> >> From: "Gastonguay, Marc" <marcg_at_metrumrg.com>
> >> Date: Tue, January 09, 2007 8:00 am
> >> To: drmould_at_attglobal.net
> >> Cc: "'Mark Sale - Next Level Solutions'" <mark_at_nextlevelsolns.com>,
> >> nmusers_at_globomaxnm.com
> >>
> >> Diane, Mark and NMusers:
> >>
> >> Thanks for pointing out the potential differences in modeling results
>
> >> that may arise from different compilers, OS, hardware, NONMEM
> >> version/patches and the importance of being transparent about these
> >> details to potential reviewers. These are some of the reasons that
> led
> >> us to develop NMQual, a tool which facilitates NONMEM installation,
> >> tracks multiple installations of NONMEM and documents all of these
> >> configuration options for every model run. In fact NMQual is part of
> the
> >> basis of our (and other users') NONMEM software qualification
> strategy.
> >>
> >> Metrum Institute is working on final steps of validation for NMQual
> 6.0,
> >> which further extends the tool's flexibility and functionality to
> manage
> >> both NONMEM V and VI installations and bug-fixes. NMQual 6.0 will
> >> continue to be a platform-independent (Windows, Linux, UNIX, Mac OS),
>
> >> open-source tool and will be available on the Metrum Institute web
> site
> >> (www.metruminstitute.org) by February 1.
> >>
> >> As always, we appreciate user feedback on any tools/resources
> provided
> >> by Metrum Institute.
> >>
> >> Marc
> >>
> >> --
> >> Marc R. Gastonguay, Ph.D.
> >> President & CEO, Metrum Research Group LLC, www.metrumrg.com
> >> Scientific Director, Metrum Institute, www.metruminstitute.org
> >>
> >>
> >>
> >> Diane R Mould wrote:
> >>
> >>> Hi Mark
> >>>
> >>> Yes, clients often want to be able to reproduce a contractor's
> results by
> >>> re-running the final model on their own system and I would also say
> that the
> >>> regulators do as well. For Nonmem V, therefore, having different
> compilers
> >>> and processors is a valid concern. I think Peter Bonate had a
> poster at
> >>> AAPS some years ago investigating differences in hardware and
> compilers.
> >>>
> >>> However for Nonmem VI, our experience thus far has been that it is
> not as
> >>> sensitive to compilers and hardware. In fact it also appears to be
> less
> >>> sensitive than Nonmem V was to the run termination status as well
> (so even
> >>> if the runs terminate, parameter estimates are generally pretty
> good, as is
> >>> the OBJ), which I think is consistent with what Tom Ludden presented
> when he
> >>> gave a preview of Nonmem VI. So for your hardware, I would guess if
> Nonmem
> >>> VI is the system you will be using for future consulting jobs, you
> should
> >>> not have problems duplicating your results on other hardware.
> >>>
> >>> Good luck finding a buyer :-)
> >>>
> >>> Diane
> >>>
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: owner-nmusers_at_globomaxnm.com
> [mailto:owner-nmusers_at_globomaxnm.com]
> >>>> On Behalf Of Mark Sale - Next Level Solutions
> >>>> Sent: Monday, January 08, 2007 10:18 AM
> >>>> Cc: nmusers_at_globomaxnm.com
> >>>> Subject: [NMusers] hardware question
> >>>>
> >>>> For those of you who don't keep track of such things, Intel is,
> today,
> >>>> releasing the desktop (non-gaming) quad core CPU. I'm thinking of
> >>>> redoing my entire network, buying maybe 8 quad core machines for
> the GA
> >>>> application, as well as parallel NONMEM. My question is, as a
> >>>> consultant, my clients frequently want to re-run final models on
> their
> >>>> hardware - and I assume that the regulators do as well. How much
> of a
> >>>> problem will it be if I set up my system as 64 bit Vista (or XP)
> with
> >>>> the Intel 64 bit compiler? There will be times when the 64 bit
> result
> >>>> will be different than the 32 bit (as there are times when
> different
> >>>> compiler setting, different compiler etc give different results).
> Is
> >>>> anyone out there planning to move to 64 bit Intel? Will anyone be
> able
> >>>> to rerun my results? It seems, as far as I can tell, the 64 bit
> system
> >>>> is maybe 20-40% faster than a comparable 32 bit system. So, quad
> core,
> >>>> 64 bit and NONMEM VI (which is faster than NONMEM V), should make a
> >>>> noticable difference.
> >>>>
> >>>>
> >>>> Thanks
> >>>> (anyone out there want to buy 12 slightly used dual core
> computers?)
> >>>>
> >>>> Mark Sale MD
> >>>> Next Level Solutions, LLC
> >>>> www.NextLevelSolns.com
> >>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>>
> >>>
> >>>
> >>>

Received on Tue Jan 09 2007 - 17:09:17 EST