AD8475 introduces too much noise, any suggestions?

I'm driving an LTC2440 24 bit sigma delta ADC with an AD8475 funnel amplifier. If I short the ADC inputs together I measure about 7 counts RMS noise, which is about 2uV RMS. This matches my expectation and the figures quoted on the ADC datasheet for a system with a 10Hz to 1kHz bandwidth. My problem is that when I drive the ADC through the AD8475 I measure 105 counts RMS of noise, which is over 25uV RMS.

Can anyone suggest a way of improving on this?