Post Go back to editing

Transient behavior on first samples of AD7980

I am using the AD7980 where I sample 512 values, delay for 10ms then start again. To test the ADC I just input a fixed voltage. But what I am finding is that about the first 30 samples exhibit some form of transient behavior (see image). After that the data seems OK. The ADC is being driven using a buffer amp. I did also have a low pass filter before the ADC but temporarily removed it as it didn't seem to alter the result. Does anyone have any suggestions on what could be causing this? Thanks