AD9914 outputting more than single frequency (unknown spurs?)

As a note, I believe there is some mirroring about 1200 MHz

The mirroring behavior of the spurs is to be expected. What you are seeing are the spurious artifacts associated with a Nyquist system.

The DDS samples at the system clock frequency (Fs). The sampling nature of the DDS means its output spectrum behaves in accordance with a Nyquist system. That is, a sampling frequency of Fs implies a Nyquist frequency of Fs/2. Hence, with Fs = 2400 MHz (80MHz x 30), the Nyquist frequency is 1200 MHz (Fs/2).

In fact, you can expect pairs of spurs mirrored about harmonics of Fs (i.e., Fs, 2Fs, 3Fs, etc.). The spurs will be offset from the Fs harmonics by the frequency programmed into the DDS. For example, if you program the DDS for 100 MHz, you will see spurs offset by 100 MHz about harmonics of Fs.

You can mitigate some of these spurs by using a low pass filter at the output of the DDS. Typically, our Evaluation Boards use an elliptic (Cauer) filter with a cutoff frequency at ~0.4Fs with at least 60dB of stop band attenuation.

Hi KennyG, thanks for the answer. 

My main issue is that if I want to generate a frequency at 1183 MHz, the mirrored frequency is 1217 MHz so the low pass filter isn't really helping.

Is the best way to mitigate this program changing the clock frequency to something higher to change the Nyquist frequency? 

Increasing Fs will shift the Nyquist frequency accordingly, along with the spurs.

However, if you are using the device's internal PLL, the PLL's VCO range is constrained to a 100 MHz band from 2400 MHz to 2500 MHz. With an 80 MHz reference clock this only allows for 2400 MHz (30x) or 2480 MHz (31x).

An alternate solution is to use an external PLL that can provide up to 3500 MHz. Then drive the AD9914 with the higher frequency PLL and bypass the internal PLL.