A customer of mine is evaluating an AD9364 (along with a Xilinx FPGA-Zynq board) for  RF receiver. He notices an instability in the AGC for  signal out of band.

Here is below the configuration on Rx1


 The input signal is a sinewave at 407MHz @ -5dBm

The spectrum is not stable. Below 2 captures of the same configuration.


Can you explain what could be the problem and is there an adjustment to be done in  the AGC configration  to improve things.