AD5933 readout drift

I have tried to read out the same value at the same setting 511 times. If I plot the real and imaginary values on "the complex plane" i get the following plot:

I have tried to put 0.1 uF and 10 uF caps accros the VDD and GND pins (all VDD pins are tied together and the same for all GND pins). This did not make this effect disappear. Has anybody seen this effect before or have any idear on what to do about it?

SETUP: I have tied REF and VIN together to make sure that his effect is contained to the AD5933 chip.