Troy Weeks wrote:

> Earlier I sent an out a help me email on the spectrum analyzer ignoring my

> commands to set threshold level and simply setting it to the current

> reference level. Very annoying!

>

> In my program I use the formula box to do some simple arithmetic

> computation. I subtract two values. ( Namely -34.4-30 ). When the

> formula box computes this the result is 64.40000000000001...obscenely

> incorrect...but not the exact problem. This is the value I put directly

> for the threshold value of the spectrum analyzer...the spectrum analyzer on

> seeing this incredibly long number of significant digits abruptly changes

> the threshold to my reference and says nothing to me.

>

> What am I going to do about this?

If you're using Direct I/O, just set the appropriate number of

significant digits using the appropriate REAL format.

BTW, the result above is not incorrect. If you have decimal numbers

in binary representation you are only assured of having an exact match

if the fraction is a sum of reciprocal powers of two. Given that you have

a 64-bit floating-point number representation you won't get the

perfect fractional result otherwise.

This is just as true in C and BASIC as it is in VEE, but it's just more

visible in VEE. There's a document on our website that describes

floating-point formats and restrictions in detail.

[<>] regards -- gvg

> Earlier I sent an out a help me email on the spectrum analyzer ignoring my

> commands to set threshold level and simply setting it to the current

> reference level. Very annoying!

>

> In my program I use the formula box to do some simple arithmetic

> computation. I subtract two values. ( Namely -34.4-30 ). When the

> formula box computes this the result is 64.40000000000001...obscenely

> incorrect...but not the exact problem. This is the value I put directly

> for the threshold value of the spectrum analyzer...the spectrum analyzer on

> seeing this incredibly long number of significant digits abruptly changes

> the threshold to my reference and says nothing to me.

>

> What am I going to do about this?

If you're using Direct I/O, just set the appropriate number of

significant digits using the appropriate REAL format.

BTW, the result above is not incorrect. If you have decimal numbers

in binary representation you are only assured of having an exact match

if the fraction is a sum of reciprocal powers of two. Given that you have

a 64-bit floating-point number representation you won't get the

perfect fractional result otherwise.

This is just as true in C and BASIC as it is in VEE, but it's just more

visible in VEE. There's a document on our website that describes

floating-point formats and restrictions in detail.

[<>] regards -- gvg

commands to set threshold level and simply setting it to the current

reference level. Very annoying!

In my program I use the formula box to do some simple arithmetic

computation. I subtract two values. ( Namely -34.4-30 ). When the

formula box computes this the result is 64.40000000000001...obscenely

incorrect...but not the exact problem. This is the value I put directly

for the threshold value of the spectrum analyzer...the spectrum analyzer on

seeing this incredibly long number of significant digits abruptly changes

the threshold to my reference and says nothing to me.

What am I going to do about this?

Troy Weeks

Aval Communications

510 974 2415

1777 N. California Blvd.

Walnut Creek, CA

94596