AnsweredAssumed Answered

vrf FW: Significant Digits

Question asked by rsb on Dec 11, 2002
Ross,
Tried using this formula (intPart(x*1000))/1000 Real.vee
Eric Vahlkamp

***************

Please note that the original poster specified that he wanted to
"restrict the number of significant digits on the display to four"
                        ^^^^^^^^^^^^^^^^

Rounding will set the number of decimals, not the significant digits
in general.

The easiest way to set significant digits as a string is to use
the formatting capability of the TO STRING object with one transaction:

WRITE TEXT a REAL64 STD:4 EOL

(or similar)

regards

Stan


--------------------------------------------------------------------------
Stan Bischof  Agilent Technologies  707-577-3994  stan_bischof@agilent.com
--------------------------------------------------------------------------

---
You are currently subscribed to vrf as: rsb@soco.agilent.com
To subscribe send a blank email to "join-vrf@it.lists.it.agilent.com".
To unsubscribe send a blank email to "leave-vrf@it.lists.it.agilent.com".
To send messages to this mailing list,  email "vrf@it.lists.it.agilent.com". 
If you need help with the mailing list send a message to "owner-vrf@it.lists.it.agilent.com".

Outcomes