AnsweredAssumed Answered

vrf is this a bug?

Question asked by VRFuser on Nov 7, 2000
Now we're on the right track. It comes down to the finite precision of the
representation of floating point numbers. The representation of 0.2 is
evidently high by a fraction of an LSB so
4/.2 = 19.9999999999999999999 (+/- a few 9s) and 4 mod .2 =
.19999999999999999 which displays as .2 unless you display enough digits.

FWIW, Excel comes up with the same result as VEE, but in MATLAB the
equivalent expression, rem(4,.2) results in 0. With this clue, I looked up
the MATLAB documentation and implemented their function in VEE:

A-intPart(A/B)*B

This gives 0 when A=4 and B=.2 as expected. I haven't checked the behavior
for other than positive A and B.

Regards,
Bill Ossmann
--
bill_ossmann@agilent.com
Agilent Technologies
Imaging Systems Division

>-----Original Message-----
>From: Paul Rubin [mailto:prubin@mpd.com]
>Sent: Tuesday, November 07, 2000 9:03 AM
>To: Cagri ACAR; vrf@lvld.agilent.com
>Subject: RE: vrf is this a bug?
>
>
>Cagri,
>
>Vee 5.01 suffers from round-off errors as did languages of the 1970's.
>
>.25 is a binary number (1/4), .2 is not! 4 /.25 =16, while to
>Vee, 4/.2 =
>20.00000000000000000000001 (I'm not sure of the exact number of 0s). It
>could have been 19.99999999999999999. I have no idea why 4 mod .2 = .2
>
>I know of no universal way around this in calculations. It's a
>"gotcha" one
>must anticipate.
>
>Paul H. Rubin
>Staff Engineer
>MPD Technologies, Inc.
>49 Wireless Blvd.
>Hauppauge, NY 11788
>
>(631) 231-1400 x613
>Fax (631) 424-1725
>prubin@mpd.com
>
>
>
>> -----Original Message-----
>> From: owner-vrf@lvld.agilent.com
>[mailto:owner-vrf@lvld.agilent.com]On
>> Behalf Of Cagri ACAR
>>
>Sent: Tuesday, November 07, 2000 3:32 AM
>> To: vrf@lvld.agilent.com
>> Subject: vrf is this a bug?
>>
>>
>>
>>
>>
>-----------------------------------------------------------------------
>> Hi everybody,
>> I have simple problem. Try this:
>> 4 MOD .2 and
>> 4 MOD .25.
>> I think in both cases we should obtain a zero but
>unfortunely we obtain
>> 0.2 from the first and a 0 from the second one? I wonder
>whether this is a
>> bug, or I am making a logical error. what is the difference
>between 0.2
>> and 0.25. Note that below 0.2, VEE gives us the unexpected
>results. My vee
>> is 5.01.
>> I have solved the problem by multipyling both sides by 100
>and converting
>> them into integer, but of course I am not satisfied with
>this solution.
>>
>> Thanks in advance.
>>
>-----------------------------------------------------------------------
>>
>> Cagri Acar
>> Mikro Dalga ve Sistem Teknolojileri / Microwave and System
>Technologies
>> Test Muhendisligi Bolumu            / Test Engineering Department
>> ASELSAN A.S.
>> PK 101 Yenimahalle Ankara/Turkey
>>
>> Phone:+903123851900-2701
>>
>>
>> ---------------------------------------------------------------------
>> This is the "vrf" maillist, managed by Majordomo.  To send
>messages to
>> this maillist, just email to "vrf@lvld.agilent.com". 
>Subscriptions and
>> unsubscriptions are done through the address
>> "vrf-request@lvld.agilent.com".
>> If you need details, just send a message containing the text "help"
>> to "vrf-request@lvld.agilent.com".
>> ---------------------------------------------------------------------
>>
>
---------------------------------------------------------------------
This is the "vrf" maillist, managed by Majordomo.  To send messages to
this maillist, just email to "vrf@lvld.agilent.com".  Subscriptions and
unsubscriptions are done through the address "vrf-request@lvld.agilent.com".
If you need details, just send a message containing the text "help"
to "vrf-request@lvld.agilent.com".
---------------------------------------------------------------------

Outcomes