AnsweredAssumed Answered

problem with double precision number

Question asked by Milos on Oct 27, 2009
Latest reply on Feb 16, 2010 by CraigG

Hi everybody,

I've got a problem with operation with double precision number

 

for example in Matlab (with double precision arithmetic)

a = 1.570796326794897 radian  (it is pi/2).

in hex a = 3ff921fb54442d18

cos(a)

6.123233995736766e-017

in hex cos(a) = 3c91a62633145c07

 

in Visual DSP++ v5 update 6

simulator ADSP-21369

 

a = 1.570796326794897 radian  (it is pi/2).

in hex a = 3ff921fb54442d18

everything is the same

 

BUT cos(a) is

6.123233995736757e-017

in hex cos(a)= 3C91A626 33145C00

 

It clear that there is difference in last byte "07" is not "00".

 

Can you explain it or describe goniometric function in math.h in detail?

 

Thanks in Advance,

Regards,

Milos

*******

Settings in VisualDSP

Project option -> Processor -> double precision = 64-bit IEEE double precision

Outcomes