AnsweredAssumed Answered

Real Number Bug

Question asked by Scott-S on Jul 9, 2010
Latest reply on Jul 12, 2010 by PingWei
Hello All;

I have been using Vee off and on since the 90's. There has always been a bug in Vee that has caused me no end of grief. For some reason, some real64 or real32 numbers vee just can't seem to understand. I am pretty sure it has to do with floating point.. yada yada, but what I don't understand is why when I convert the number to a text value for display, it still can't display it correctly.

Take the real64 number 95.96

This numer shows up in vee as 95.9599999999999999

This really messes up my reports and displays. The only 2 ways I can deal with it is to either turn it to text and truncate it which make the number above 95.95 (which is not the number I really want) or I have to break the number it Int and fractional, multiply the fractional by 100, round it, divide by 100 then turn them to text then concantonate the text back together.

I guess I was just wonder why this hasn't been fixed in the many years. And, if any else has an easier workaround?.

Here is an example in Vee Pro 9
Real Number Bug Example.vee

Outcomes