It appears possible to change the crystal oscillator parameters when using an external clock. This has a substantial effect on integer boundary spurs.
In my application I use a 1:1 transformer to drive the XO1 and XO2 pins. The drive level at the transformer primary is about 1.6V PP HCMOS at 50MHz. The transformer secondary connects to the XO1/2 pins.
I use a transformer in order to isolate the AD9578 ground and the analog ground on the board.
My default for register 4 is 00AAh, ie the outputs are all LVPECL, and the crystal gain and capacitance are the default.
At an output frequency of 899.55MHz, I get integer boundary spurs at -52dBc.
If I change the register 4 value to F0AAh, I get a spur value under the same conditions of -73dBc, a very substantial improvement. This register setting changes the crystal amplifier gain to a minimum ( I deduce this from table 25), and sets the load capacitances to maximum.
There is nothing about this effect in the data sheet.
I would help if I understood the structure of the XO1 and XO2 inputs. This cannot be a true differential input, as the crystal amplifier must be on for the input to work at all. I understand that the xtal sustaining amplifier is always on if either PLL uses XO1/2 as the reference, and bit 26 of register 3 has no effect.
This is not the case for inputs XO3/4, when the xtal amplifier must be switched on using bit 8 of register 10, for the input to work.
Could you please comment on this , and if possible supply a part schematic of the XO1/2 inputs.