Hi everybody,

I've got a problem with operation with double precision number

for example in Matlab (with double precision arithmetic)

a = 1.570796326794897 radian (it is pi/2).

in hex a = 3ff921fb54442d18

cos(a)

6.123233995736766e-017

in hex cos(a) = 3c91a62633145c07

in Visual DSP++ v5 update 6

simulator ADSP-21369

a = 1.570796326794897 radian (it is pi/2).

in hex a = 3ff921fb54442d18

everything is the same

BUT cos(a) is

6.123233995736757e-017

in hex cos(a)= 3C91A626 33145C00

It clear that there is difference in last byte "07" is not "00".

Can you explain it or describe goniometric function in math.h in detail?

Thanks in Advance,

Regards,

Milos

*******

Settings in VisualDSP

Project option -> Processor -> double precision = 64-bit IEEE double precision

The result returned is correct in the first 14 digits. This is what one would expect from a IEEE 64-bit floating point result. The differences you are seeing will be due to the fact that the SHARC cosine function uses 64-bit floating point emulation software to compute the result while Matlab has access to extended floating point support in hardware. If you wish to inspect the source, you can find it at "Your VisualDSP++ Install"/213xx/lib/src/libc_src/cosl.c.