AnsweredAssumed Answered

VRF - Need_help: Strange printout

Question asked by VRFuser on Feb 20, 1996
I am using HP VEE to develop calibration tests for the Brunei Min Def
Calibration Centre (I obtained our first copy of the software 3 months ago).
When printing out a results sheet I am having problems printing some REAL
values of limit data.
For example:
Set a REAL constant to the value 8.9972
Now print it.
The result is 8.997199999999999.
Has anyone had this problem?
I can replace the REAL with a TEXT constant and get the correct print out but in
my actual application the number is input by an operator and is checked against
limits and accidental text input is prevented.
I have also found that by increasing the number of significant digits to the
maximum (under the File -> Edit Default Preferences) it is impossible to input
8.9972.
Is there a fix?
I have written a Userfunction to manipulate the number text string to give me
the required value but is it necessary?
Kevin Roke
Brunei Darussalam


Outcomes