<font size=2>Hi </font>
<font size=2>I wish that the display menu in VEE 8 contain a new object ( Matlab X vs Y plot ) which can be added to a panel of a user function in the same way as ( X vs Y plot ) object.</font>
<font size=2>Majed Ali </font>
---You are currently subscribed to vrf as: rsb@soco.agilent.comTo subscribe send a blank email to "join-vrf@it.lists.it.agilent.com".To unsubscribe send a blank email to "leave-vrf@it.lists.it.agilent.com".To send messages to this mailing list, email "vrf@agilent.com". If you need help with the mailing list send a message to "owner-vrf@it.lists.it.agilent.com".
me why I get .562000000000000001 instead of .562 when I convert the number to
text (see attached VEE Pro 7.0 program)? Also, do you have any idea how to fix
this problem?
Regards,
Lee
Lee Hudson, R&D Scientist
**************
Sure.
You get .562000000000000001 becuase that's the precise answer that
IEEE standards call for. Since this is a digital computer it
only has so much precision and can therefore only approximate
most numbers. That is, since you only have 64 bits to work with
for IEEE 64-bit floating point there's only a fixed subset of
the real numbers that can be represented. Everything else is
approximate- hence the origins of what you are seeing.
Many environments choose to hide this. VEE ( for better or worse)
decided to expose it.
If you want less precision then it is just a case of number fomatting,
and one of the best ways to do that is to use a To String object
so that you can specifiy whatever format/precision you like.
Stan
--------------------------------------------------------------------------
Stan Bischof Agilent Technologies 707-577-3994 stan_bischof@agilent.com
--------------------------------------------------------------------------
---
You are currently subscribed to vrf as: rsb@soco.agilent.com
To subscribe send a blank email to "join-vrf@it.lists.it.agilent.com".
To unsubscribe send a blank email to "leave-vrf@it.lists.it.agilent.com".
To send messages to this mailing list, email "vrf@agilent.com".
If you need help with the mailing list send a message to "owner-vrf@it.lists.it.agilent.com".