I am having a problem with the PLL on my AD9958. Looking at the SYNC_CLK output, everything appears fine when I have a small clock multiplier set and correspondingly have the VCO gain control bit unset. However, if I increase the clock multiplier (>10) and switch on the gain control bit (as I should) then the PLL does not lock correctly. I see a signal with the correct fundamental frequency, but huge amplitude modulation (see attached figure, taken with a lot of averaging switched on). If I have a high clock multiplier but don't set the gain control bit then the signal is stable, but at the wrong frequency (VCO must be railed, about 64 MHz). I tried changing the charge pump current and that slightly changes the amplitude modulation envelop in some unpredictable way.
I am guessing I might need to modify the PLL loop filter components to get it to be stable. I am currently using the 680 pF to 1.8V as in the data sheet. I am guessing that the data sheet suggestion to include a 0 ohm series resistor is because they expect you might need to change to something else. But I have no idea how to do this systematically or what direction to go in. I have tried a couple of different capacitor values but no improvement.
My clock source is a good 24.576 MHz TCXO driving a differential PECL line driver, terminated into the DDS with a Y terminator arrangement as per fig 34 of the data sheet.
Any suggestions greatly appreciated!