AnsweredAssumed Answered

vrf RS-232 Run Time Error

Question asked by adam_kohler on Dec 21, 1999
> Subject: Re: vrf:help with exponantiation
> The problem is that the output of a^b is "a container of the highest
> type" of its inputs; ie if both inputs are int32, so is the output. 0.2
> as an int32 = 0, so the result it gives you is "correct" - but the
> output you require is real. If the input B of the formula box is cast to
> real (see attached example), the output will also be real, and you will
> get the correct result.

If at least one of the operands is of type real, the expression will be
evaluated as a real and will return the desired result.  Casting one or
more inputs as mentioned above is one way to force an operand to be of
real type.  A couple of other possibilities are:

Use a real constant in the expression:

A*10.0^B instead of A*10^B

Or use the asReal function to convert the integer to a real as needed:


Any of the three methods will correct your problem.


Bruce Wenner
Agilent Technologies

This is the "vrf" maillist, managed by Majordomo.  To send messages to
this maillist, just email to "".  Subscriptions and
unsubscriptions are done through the address "".
If you need details, just send a message containing the text "help"
to "".