AnsweredAssumed Answered

vrf Problem with Real32 data constant - again

Question asked by VRFuser on Sep 7, 2006
 >  As an example,  the specification for a particular measurement  
>  is >= 1.15 AND <= 1.25. Say the result is 1.25000069.   
>  This could be a failure in some interpretations but the spec only requires 1.25.  
 
You could bump the limit up to 1.2501 -- 2 digits beyond the spec, thus "invisible"
and this would let the internal representation pass the spec.  To get in even
tighter, figure out what one LSB is in whatever float32/64 you are using and add
that to your spec.
 
Les Hammer
VP of Engineering
Smart Sensor Systems
720 14th Street SW
Loveland, CO.  80537-6349
Les.Hammer@SmartSensorSystems.com
Les.Hammer@CompleteTest.com

 

---

To subscribe please send an email to: "vrf-request@lists.it.agilent.com" with the word subscribe in the message body.
To unsubscribe send a blank email to "leave-vrf@it.lists.it.agilent.com".
To send messages to this mailing list, email "vrf@agilent.com".
If you need help with the mailing list send a message to
"owner-vrf@it.lists.it.agilent.com".
Search the "unofficial vrf archive" at "www.oswegosw.com/vrf_archive/".  

Outcomes