AnsweredAssumed Answered

vrf Unusual behaviour

Question asked by rsb on Sep 15, 2003
"Schlieker, Reiner" <reiner.schlieker@siemens.com> wrote:
> of times, all of a sudden zero becomes -0.4440892f!  I understand the
> concept of why this might occur (the precision of holding a real number in
> 64 bits, etc) but why is displayed suffixed with an "f"?  I've not seen this
> before - anybody out there have an answer?

Think international standard metric prefix.

milli, micro, pico, nano, femto ...
                          ^

VEE is metric prefix smart- so reads/write such items as

  38u   1.345m  -0.4440892f



Stan


--------------------------------------------------------------------------
Stan Bischof  Agilent Technologies  707-577-3994  stan_bischof@agilent.com
--------------------------------------------------------------------------

---
You are currently subscribed to vrf as: rsb@soco.agilent.com
To subscribe send a blank email to "join-vrf@it.lists.it.agilent.com".
To unsubscribe send a blank email to "leave-vrf@it.lists.it.agilent.com".
To send messages to this mailing list,  email "vrf@it.lists.it.agilent.com". 
If you need help with the mailing list send a message to "owner-vrf@it.lists.it.agilent.com".

Outcomes