AnsweredAssumed Answered

mathematic result precision 21469 EzKit lite

Question asked by Uhendan on Jun 21, 2010
Latest reply on Jun 24, 2010 by Uhendan

hi, everyone!

I`ve got a problem with mathematics in visual DSP 5.0:

 

I simulate a sharc 21469 EzKit lite board and write a code for filter coefficient calculation...


just for example (not a part of my code, but the core of the problem, i guess):

//code ....

float x =2;
float y;
  y=sqrt(x);                   //*** or division, or something else..

printf("%.20f",y);


...code//

result would be 1.41421350000000000000

 

...but, for precise mathematical operations,
the true result should rather be:

1,41421356237309504880

 


how can i fix the problem with the precision of math operations? Another C Compiler, i tested, does not do this..

 

why is there a rounding after 8 digits?

 

thanks

 

-UH

Outcomes