AnsweredAssumed Answered

Real (float) problems !! with numbers

Question asked by Anders on Jan 17, 2010
Latest reply on Jan 18, 2010 by gojoh

Like • Show 0 Likes0
Comment • 3

Hi All...

Im new to this forum and im not a software engineer..
However, used VEE for some years.
My problem is this:
I need to send a calculated value to the RS232..no problems there, but the calculated value. It very simple.
A number multiplyed with 10 and later on in the program divided with 10 (a number from a counter in a loop)
The result mostly gives eg 1.2, 1.3 and so on, but some times the result is 1.999999999999 and my equipment dont understand that.
i have been told its because its Reel (float) format, but it seems i cant use any other format since i have x,x..
How do i get the program to show the correct number ?

If you need more information, just tell me

Thanks

Anders

No one else has this question

Outcomes

emcdude Jan 17, 2010 2:14 PM

I've had the same problem in the past. I solved this by converting the real number to a string, then limiting the length of the string. Next, convert that string back to a real number.

Regards,

Actions

Anders Jan 18, 2010 4:23 AM

Thanks..

I think i will try that. However, the string must have a difined lenght cause my counter goes from 0 til 100's with one decimal = string from 3 to 5.
I could maybe make and string lenght auto detect where i can use int16 numbers...ill try..thanks again

Regards

Anders

Actions

gojoh Jan 18, 2010 4:59 AM

This is probably easy to fix.

Just specify correct format in the DirectI/O object. Use WRITE TEXT a REAL FORMAT FIXED with 1 fractional digit to always send out the number with one decimal.

Actions

3 Replies

emcdude Jan 17, 2010 2:14 PM
Mark Correct
Correct Answer
I've had the same problem in the past. I solved this by converting the real number to a string, then limiting the length of the string. Next, convert that string back to a real number.

Regards,
Like • Show 0 Likes0
Actions
Anders Jan 18, 2010 4:23 AM
Mark Correct
Correct Answer
Thanks..

I think i will try that. However, the string must have a difined lenght cause my counter goes from 0 til 100's with one decimal = string from 3 to 5.
I could maybe make and string lenght auto detect where i can use int16 numbers...ill try..thanks again

Regards

Anders
Like • Show 0 Likes0
Actions
gojoh Jan 18, 2010 4:59 AM
Mark Correct
Correct Answer
This is probably easy to fix.

Just specify correct format in the DirectI/O object. Use WRITE TEXT a REAL FORMAT FIXED with 1 fractional digit to always send out the number with one decimal.
Like • Show 0 Likes0
Actions

Real (float) problems !! with numbers

Outcomes

Related Content

Recommended Content