AD7492 artifacts

Hello!

I am using the AD7492 ADC to convert an analog signal at about 0.1 MSPS but at irregular intervals I am getting single samples which are way off. The samples always contain many 1-bits near each other e.g. 0x3FF0 or oxE000. I have outputted the result to my computer downsampled to 5 kHz which is what is in the figure below.

The schematic is attached. I have also tried adding a 10 uF capacitor to the REF OUT analog reference voltage to no avail. I am new in this area so please bear with me if I have made some silly mistake.

output.pdf