Re: [NMusers] hardware question

Hi Toufigh,

If NONMEM converges, the differences are usually negligible. However, if the runs fail, they may
fail quite differently on different platforms. Below is the example from my recent projects: similar
but not identical g77 compilers on Linux and windows machines; runs failed with 136 error.

Leonid

OF
71559.566
71540.153

THETA - VECTOR OF FIXED EFFECTS PARAMETERS *********
          6.53E+01 9.52E-01 1.99E+02
          6.54E+01 9.57E-01 2.41E+02

  OMEGA - COV MATRIX FOR RANDOM EFFECTS - ETAS ********
  2.65E-02
  0.00E+00 5.05E-01
  0.00E+00 -8.96E-01 3.01E+00

  2.65E-02
  0.00E+00 4.82E-01
  0.00E+00 -1.06E+00 4.36E+00

  SIGMA - COV MATRIX FOR RANDOM EFFECTS - EPSILONS ****
  2.13E+01
  2.13E+01



> Hi,
>
> Could anybody provide some more info on how big the differences in
> parameter estimates are on different machines/compilers? In other words,
> are the differences large enough to be of concern when it comes to
> predicting the outcome of a study design or treatment, or are we talking
> about clinically insignificant differences?
>
> Toufigh Gordi
>
> -----Original Message-----
> From: owner-nmusers_at_globomaxnm.com [mailto:owner-nmusers_at_globomaxnm.com]
> On Behalf Of Gastonguay, Marc
> Sent: Tuesday, January 09, 2007 12:09 PM
> To: Mark Sale - Next Level Solutions
> Cc: nmusers_at_globomaxnm.com
> Subject: Re: [NMusers] hardware question
>
> Mark -
> Thanks for your support of NMQual. You're correct that it doesn't make
> all of the differences go away. It simply documents and in some cases
> implements the details (including all of those you've listed below) of
> the hardware/software configuration used for any given run &
> installation. In order to reproduce results (in those cases where
> configuration makes a difference), a reviewer would have to have to
> ensure that they are running on the same configuration.
>
> Nice try for a solution... Obviously the practical and important
> strategy is to explore the impact that possible configuration-related
> differences in results (as well as other model and parameter
> uncertainties) will have on the inferences we draw from our
> modeling/simulation efforts.
>
> Marc G.
>
>
> Mark Sale - Next Level Solutions wrote:
>> Thanks Marc,
>> While I'm a big fan of NMqual (I use it myself), it isn't clear to
> me
>> that it solves much of this problem. I think (personal opinion
> lacking
>> any data), the order of frequency in causes of differences are:
>> Different compiler (some people using g77)
>> Different compiler options (NMqual addresses this - right?, assuming
>> that both parties use NMqual)
>> Different hardware (some people using UNIX, a few still using Vax??,
>> possibly AMD vs Intel?)
>> Differences in OS (Windows vs Linux - haven't personally documented
>> this)
>> 32 vs 64 bit - even within the Intel compiler 9.0, there can be
>> differences, some of the variable are different sizes.
>> Failure to fix bugs (addressed by NMqual)
>>
>> Having said that, please not that I don't have a solution to the
> others
>> (except everyone should buy 64 bit Intel windows and run Windows XP64
>> and Intel 9.0 compiler, and use NMqual.
>>
>>
>> Mark Sale MD
>> Next Level Solutions, LLC
>> www.NextLevelSolns.com
>>
>>
>>
>>> -------- Original Message --------
>>> Subject: Re: [NMusers] hardware question
>>> From: "Gastonguay, Marc" <marcg_at_metrumrg.com>
>>> Date: Tue, January 09, 2007 8:00 am
>>> To: drmould_at_attglobal.net
>>> Cc: "'Mark Sale - Next Level Solutions'" <mark_at_nextlevelsolns.com>,
>>> nmusers_at_globomaxnm.com
>>>
>>> Diane, Mark and NMusers:
>>>
>>> Thanks for pointing out the potential differences in modeling results
>
>>> that may arise from different compilers, OS, hardware, NONMEM
>>> version/patches and the importance of being transparent about these
>>> details to potential reviewers. These are some of the reasons that
> led
>>> us to develop NMQual, a tool which facilitates NONMEM installation,
>>> tracks multiple installations of NONMEM and documents all of these
>>> configuration options for every model run. In fact NMQual is part of
> the
>>> basis of our (and other users') NONMEM software qualification
> strategy.
>>> Metrum Institute is working on final steps of validation for NMQual
> 6.0,
>>> which further extends the tool's flexibility and functionality to
> manage
>>> both NONMEM V and VI installations and bug-fixes. NMQual 6.0 will
>>> continue to be a platform-independent (Windows, Linux, UNIX, Mac OS),
>
>>> open-source tool and will be available on the Metrum Institute web
> site
>>> (www.metruminstitute.org) by February 1.
>>>
>>> As always, we appreciate user feedback on any tools/resources
> provided
>>> by Metrum Institute.
>>>
>>> Marc
>>>
>>> --
>>> Marc R. Gastonguay, Ph.D.
>>> President & CEO, Metrum Research Group LLC, www.metrumrg.com
>>> Scientific Director, Metrum Institute, www.metruminstitute.org
>>>
>>>
>>>
>>> Diane R Mould wrote:
>>>
>>>> Hi Mark
>>>>
>>>> Yes, clients often want to be able to reproduce a contractor's
> results by
>>>> re-running the final model on their own system and I would also say
> that the
>>>> regulators do as well. For Nonmem V, therefore, having different
> compilers
>>>> and processors is a valid concern. I think Peter Bonate had a
> poster at
>>>> AAPS some years ago investigating differences in hardware and
> compilers.
>>>> However for Nonmem VI, our experience thus far has been that it is
> not as
>>>> sensitive to compilers and hardware. In fact it also appears to be
> less
>>>> sensitive than Nonmem V was to the run termination status as well
> (so even
>>>> if the runs terminate, parameter estimates are generally pretty
> good, as is
>>>> the OBJ), which I think is consistent with what Tom Ludden presented
> when he
>>>> gave a preview of Nonmem VI. So for your hardware, I would guess if
> Nonmem
>>>> VI is the system you will be using for future consulting jobs, you
> should
>>>> not have problems duplicating your results on other hardware.
>>>>
>>>> Good luck finding a buyer :-)
>>>>
>>>> Diane
>>>>
>>>>
>>>>
>>>>> -----Original Message-----
>>>>> From: owner-nmusers_at_globomaxnm.com
> [mailto:owner-nmusers_at_globomaxnm.com]
>>>>> On Behalf Of Mark Sale - Next Level Solutions
>>>>> Sent: Monday, January 08, 2007 10:18 AM
>>>>> Cc: nmusers_at_globomaxnm.com
>>>>> Subject: [NMusers] hardware question
>>>>>
>>>>> For those of you who don't keep track of such things, Intel is,
> today,
>>>>> releasing the desktop (non-gaming) quad core CPU. I'm thinking of
>>>>> redoing my entire network, buying maybe 8 quad core machines for
> the GA
>>>>> application, as well as parallel NONMEM. My question is, as a
>>>>> consultant, my clients frequently want to re-run final models on
> their
>>>>> hardware - and I assume that the regulators do as well. How much
> of a
>>>>> problem will it be if I set up my system as 64 bit Vista (or XP)
> with
>>>>> the Intel 64 bit compiler? There will be times when the 64 bit
> result
>>>>> will be different than the 32 bit (as there are times when
> different
>>>>> compiler setting, different compiler etc give different results).
> Is
>>>>> anyone out there planning to move to 64 bit Intel? Will anyone be
> able
>>>>> to rerun my results? It seems, as far as I can tell, the 64 bit
> system
>>>>> is maybe 20-40% faster than a comparable 32 bit system. So, quad
> core,
>>>>> 64 bit and NONMEM VI (which is faster than NONMEM V), should make a
>>>>> noticable difference.
>>>>>
>>>>>
>>>>> Thanks
>>>>> (anyone out there want to buy 12 slightly used dual core
> computers?)
>>>>> Mark Sale MD
>>>>> Next Level Solutions, LLC
>>>>> www.NextLevelSolns.com
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>
>
>
Received on Tue Jan 09 2007 - 18:08:11 EST