AnsweredAssumed Answered

vrf reals

Question asked by rsb on Dec 6, 2001
> I am exporting data from VEE 6 to Exel. The problem is dat VEE has a
> different (strange) way of handling reals. (e.g. 0.7 becomes
> 0.699999999996).
> Is there a way to overcome this problem; e.g. so to get exact 0.7 in
> exel without making use of text-strings?

What you are seeing is simply computer math at work- no computer
represents numbers exactly since they are digital devices. In this case
IEEE 64-bit math cannot exactly represent 0.7. VEE by default
represents it as accurately as IEEE std allows- 0.699999999996
as you note.

Excell also cannot represent this as exact 0.7 (without
using strings) .

What you want , however, is just a number format issue.

When you write to Excel set formatting to the precision that you want
or over in Excell set the formatting as you want- then you
will see "0.7" on the screen as you want.


PS- note that this is true for ALL programs on all computers-
only the details change, such as number precision and default
This is the "vrf" maillist, managed by Majordomo.  To send messages to
this maillist, just email to "".  Subscriptions and
unsubscriptions are done through the address "".
If you need details, just send a message containing the text "help"
to "".