RE: [NMusers] hardware question

Hi,

Could anybody provide some more info on how big the differences in
parameter estimates are on different machines/compilers? In other words,
are the differences large enough to be of concern when it comes to
predicting the outcome of a study design or treatment, or are we talking
about clinically insignificant differences?

Toufigh Gordi

-----Original Message-----
From: owner-nmusers_at_globomaxnm.com [mailto:owner-nmusers_at_globomaxnm.com]
On Behalf Of Gastonguay, Marc
Sent: Tuesday, January 09, 2007 12:09 PM
To: Mark Sale - Next Level Solutions
Cc: nmusers_at_globomaxnm.com
Subject: Re: [NMusers] hardware question

Mark -
Thanks for your support of NMQual. You're correct that it doesn't make
all of the differences go away. It simply documents and in some cases
implements the details (including all of those you've listed below) of
the hardware/software configuration used for any given run &
installation. In order to reproduce results (in those cases where
configuration makes a difference), a reviewer would have to have to
ensure that they are running on the same configuration.

Nice try for a solution... Obviously the practical and important
strategy is to explore the impact that possible configuration-related
differences in results (as well as other model and parameter
uncertainties) will have on the inferences we draw from our
modeling/simulation efforts.

Marc G.


Mark Sale - Next Level Solutions wrote:
> Thanks Marc,
> While I'm a big fan of NMqual (I use it myself), it isn't clear to
me
> that it solves much of this problem. I think (personal opinion
lacking
> any data), the order of frequency in causes of differences are:
> Different compiler (some people using g77)
> Different compiler options (NMqual addresses this - right?, assuming
> that both parties use NMqual)
> Different hardware (some people using UNIX, a few still using Vax??,
> possibly AMD vs Intel?)
> Differences in OS (Windows vs Linux - haven't personally documented
> this)
> 32 vs 64 bit - even within the Intel compiler 9.0, there can be
> differences, some of the variable are different sizes.
> Failure to fix bugs (addressed by NMqual)
>
> Having said that, please not that I don't have a solution to the
others
> (except everyone should buy 64 bit Intel windows and run Windows XP64
> and Intel 9.0 compiler, and use NMqual.
>
>
> Mark Sale MD
> Next Level Solutions, LLC
> www.NextLevelSolns.com
>
>
>
>> -------- Original Message --------
>> Subject: Re: [NMusers] hardware question
>> From: "Gastonguay, Marc" <marcg_at_metrumrg.com>
>> Date: Tue, January 09, 2007 8:00 am
>> To: drmould_at_attglobal.net
>> Cc: "'Mark Sale - Next Level Solutions'" <mark_at_nextlevelsolns.com>,
>> nmusers_at_globomaxnm.com
>>
>> Diane, Mark and NMusers:
>>
>> Thanks for pointing out the potential differences in modeling results

>> that may arise from different compilers, OS, hardware, NONMEM
>> version/patches and the importance of being transparent about these
>> details to potential reviewers. These are some of the reasons that
led
>> us to develop NMQual, a tool which facilitates NONMEM installation,
>> tracks multiple installations of NONMEM and documents all of these
>> configuration options for every model run. In fact NMQual is part of
the
>> basis of our (and other users') NONMEM software qualification
strategy.
>>
>> Metrum Institute is working on final steps of validation for NMQual
6.0,
>> which further extends the tool's flexibility and functionality to
manage
>> both NONMEM V and VI installations and bug-fixes. NMQual 6.0 will
>> continue to be a platform-independent (Windows, Linux, UNIX, Mac OS),

>> open-source tool and will be available on the Metrum Institute web
site
>> (www.metruminstitute.org) by February 1.
>>
>> As always, we appreciate user feedback on any tools/resources
provided
>> by Metrum Institute.
>>
>> Marc
>>
>> --
>> Marc R. Gastonguay, Ph.D.
>> President & CEO, Metrum Research Group LLC, www.metrumrg.com
>> Scientific Director, Metrum Institute, www.metruminstitute.org
>>
>>
>>
>> Diane R Mould wrote:
>>
>>> Hi Mark
>>>
>>> Yes, clients often want to be able to reproduce a contractor's
results by
>>> re-running the final model on their own system and I would also say
that the
>>> regulators do as well. For Nonmem V, therefore, having different
compilers
>>> and processors is a valid concern. I think Peter Bonate had a
poster at
>>> AAPS some years ago investigating differences in hardware and
compilers.
>>>
>>> However for Nonmem VI, our experience thus far has been that it is
not as
>>> sensitive to compilers and hardware. In fact it also appears to be
less
>>> sensitive than Nonmem V was to the run termination status as well
(so even
>>> if the runs terminate, parameter estimates are generally pretty
good, as is
>>> the OBJ), which I think is consistent with what Tom Ludden presented
when he
>>> gave a preview of Nonmem VI. So for your hardware, I would guess if
Nonmem
>>> VI is the system you will be using for future consulting jobs, you
should
>>> not have problems duplicating your results on other hardware.
>>>
>>> Good luck finding a buyer :-)
>>>
>>> Diane
>>>
>>>
>>>
>>>> -----Original Message-----
>>>> From: owner-nmusers_at_globomaxnm.com
[mailto:owner-nmusers_at_globomaxnm.com]
>>>> On Behalf Of Mark Sale - Next Level Solutions
>>>> Sent: Monday, January 08, 2007 10:18 AM
>>>> Cc: nmusers_at_globomaxnm.com
>>>> Subject: [NMusers] hardware question
>>>>
>>>> For those of you who don't keep track of such things, Intel is,
today,
>>>> releasing the desktop (non-gaming) quad core CPU. I'm thinking of
>>>> redoing my entire network, buying maybe 8 quad core machines for
the GA
>>>> application, as well as parallel NONMEM. My question is, as a
>>>> consultant, my clients frequently want to re-run final models on
their
>>>> hardware - and I assume that the regulators do as well. How much
of a
>>>> problem will it be if I set up my system as 64 bit Vista (or XP)
with
>>>> the Intel 64 bit compiler? There will be times when the 64 bit
result
>>>> will be different than the 32 bit (as there are times when
different
>>>> compiler setting, different compiler etc give different results).
Is
>>>> anyone out there planning to move to 64 bit Intel? Will anyone be
able
>>>> to rerun my results? It seems, as far as I can tell, the 64 bit
system
>>>> is maybe 20-40% faster than a comparable 32 bit system. So, quad
core,
>>>> 64 bit and NONMEM VI (which is faster than NONMEM V), should make a
>>>> noticable difference.
>>>>
>>>>
>>>> Thanks
>>>> (anyone out there want to buy 12 slightly used dual core
computers?)
>>>>
>>>> Mark Sale MD
>>>> Next Level Solutions, LLC
>>>> www.NextLevelSolns.com
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>
>>>

Received on Tue Jan 09 2007 - 15:40:11 EST