AnsweredAssumed Answered

Real64 to String Bug (VEE9.3)

Question asked by omar on Nov 8, 2013
Latest reply on Nov 19, 2013 by omar

Like • Show 0 Likes0
Comment • 4

Hi everyone,

When i enter an array of type Real64 in the Enum Values input of a Pop-Up List, some numbers such as 8.2, 8.8 and 9.2 are displayed wrongly and as a consequence of that, the Enum output of these numbers which is stored in a string variable is also wrong and all SCPI commands that use this text variable will not be executed by my Device.

How can i make the selection list in the pop up list shows up correctly .

I would appreciate any help.

Attachments

Pop-Up list with Real64 array.jpg44.6 KB

No one else has this question

Outcomes

4 Replies

dBdetlef Nov 13, 2013 8:49 AM
Mark Correct
Correct Answer
Use Real 32 instead ;-) It should fit for numbers like this ....
Why make life more complex if nbot needed. Reduce your amount of memory needed for this variables.
Bug is confirmed i think- hoping that there will be a fix soon.
Like • Show 0 Likes0
Actions
gatekeepr69 Nov 16, 2013 10:22 PM
Mark Correct
Correct Answer
The problem you are referring to is a floating point issue that's been around for a number of releases and I believe is something that happens with all floating point arithmetic but for some reason VEE does not truncate it properly either internally or when converting to "string"

If you try to use Real32 the issue even gets worse, plus there is no Real32 array constant either.

The fix I use is to run the array output into a "To String" object using real64 with a significant no. of digits set to 6 for example. of course to feed this into a pop-up you will need to have it an array so run it into a "From String" object and use token format and a 1D array set "to end" and you can feed this into your "Pop-Up" list.

An alternative to the above mentioned is to loop through your array using a repeat "for range" object - through the "To String" object then into a collector.

I know it's a nasty band-aid but I am not sure of another way to do it, Since our program uses programmatic labels, they must be fed with txt and I run into the same issue.

- James
Like • Show 0 Likes0
Actions
- dBdetlef @ gatekeepr69 on Nov 18, 2013 1:06 AM
  Mark Correct
  Correct Answer
  I think we are talking about different things. There is a real bug on one side, and there are some misunderstandings in number formats on the other.
  Best way to clarify this is to show an example. Ca n you supply one or more?
  There was a lot of discussion about this in the VRF, mainly about the " real existant bug" .
  
  I have to apologize: there is a Bug with INT64 and trigonometric functions. No Bug with Real 64 ( so far ;-) )
  
  Edited by: dBdetlef on Nov 18, 2013 2:04 AM
  Like • Show 0 Likes0
  Actions
omar Nov 19, 2013 1:00 AM
Mark Correct
Correct Answer
Hi,
I did what gatekeepr69 suggested and it worked (i attached a screenshot to this post).

Hopefully Agilent will resolve this in their next release, since this is only a workaround solution.

Thank u guys for your help.
Attachments
- Real64ToString.png44.2 KB
Like • Show 0 Likes0
Actions

Real64 to String Bug (VEE9.3)

Attachments

Outcomes

Attachments

Related Content

Recommended Content