AnsweredAssumed Answered

AW: vrf Comaprison (=) bug?

Question asked by detlef.baranski on Feb 22, 2006
Latest reply on Feb 22, 2006 by detlef.baranski
Or just use Integer allways for the for range and then devide it by 10. It
works correctly.

detlef




-----Ursprngliche Nachricht-----
Von: Gregg C Levine [mailto:hansolofalcon@worldnet.att.net]
Gesendet: Mittwoch, 22. Februar 2006 15:44
An: VRF
Betreff: RE: [vrf] Comaprison (>=) bug?

Hello from Gregg C Levine
This reminds me of something I had to learn the hard way when I was
first learning how to program.

It seems that what we consider real numbers aren't always the flat
1....9 groups that we are used to. As all of you have just discovered
there is a rounding off error that surfaces. And that presumably is
why VEE includes that setting that tells us to what significant
figures the number will be rounded off. We should do more testing, and
be ready to provide to the company a valid series of programs, so that
they can reproduce this on their machines to confirm for themselves
that its indeed a bug and not a feature.
----
Gregg C Levine hansolofalcon@worldnet.att.net
---
"Remember the Force will be with you. Always." Obi-Wan Kenobi

> -----Original Message-----
> From: Vinod [mailto:papineni_vinod@yahoo.com]
> Sent: Wednesday, February 22, 2006 8:41 AM
> To: VRF
> Subject: Re: [vrf] Comaprison (>=) bug?
>
> Warren,
>
> I think this another of the floating-point phenomena. Just probe the
line for the last
> value from the For Range loop object; you will notice that it is not
6 but 5.999...
>
> -Vinod
>
> --- "Pickles, Warren" <Warren.Pickles@tycoelectronics.com> wrote:
>
> > Hi all,
> >
> > Please take a look at the attached program.  I think it's a bug in
VEE
> > but I just want to check I'm not missing something stupid before I
> > submit it to the defect report thingy.  Shouldn't the last value
in the
> > lower Logging Alphanumeric be a zero?  This is a VEE 6.2 prog but
it's
> > the same in VEE 7.01
> >
> > My workaround is to cast the output of the For Range as Real32 so
I'm
> > guessing it's some kind of rounding error (or "feature").
> >
> > Cheers,
> > Warren
>


---
You are currently subscribed to vrf as: Detlef.Baranski@pilkington.de
To subscribe send a blank email request to "join-vrf@lists.it.agilent.com".
To unsubscribe send a blank email to "leave-vrf@it.lists.it.agilent.com".
To send messages to this mailing list,  email "vrf@agilent.com". 
If you need help with the mailing list send a message to
"owner-vrf@it.lists.it.agilent.com".
Search the "unofficial vrf archive" at "www.oswegosw.com/vrf_archive/".

---
You are currently subscribed to vrf as: rsb@soco.agilent.com
To subscribe send a blank email request to "join-vrf@lists.it.agilent.com".
To unsubscribe send a blank email to "leave-vrf@it.lists.it.agilent.com".
To send messages to this mailing list,  email "vrf@agilent.com". 
If you need help with the mailing list send a message to "owner-vrf@it.lists.it.agilent.com".
Search the "unofficial vrf archive" at "www.oswegosw.com/vrf_archive/".

Outcomes