Recently we made a hardware change which resulted in slightly increased ripple on the 5V supply. After making this change we started to notice occasional failures in the Tx Quadrature Calibration that would result in high LO feedthrough, degrading RF performance. After failure, the A7 register will contain 0xFC indicating that the algorithm did not converge. Further investigation reveals that the algorithm almost always fails to converge according to A7, although the RF performance is usually ok.
I ran the NCO phase sweep manually as described in https://ez.analog.com/docs/DOC-3143 and I obtained a nice plot similar to Figure 1 on that page. However, even in the region where both the LO and image are minimized, A7 often reports failure. When the code performs this sweep, it sometimes doesn't find any successful settings so it defaults to 0, which results in poor RF performance.
After examining the code and doing some experimenting, we found that forcing the Rx and Tx NCO frequency in registers A0 and A3 to '3' improves the success rate of the calibration. The code normally sets this to '0' for our bandwidth setting of 10 MHz, with a 30.72 MHz sample rate. Presumably the failures are due to power supply spurs close to the LO, so moving the NCO away from LO helps. However even with this change there are still occasional errors, around 1% of the time.
So my question is, are there any other settings I could change to make the error rate go to zero? Or would you suggest hardcoding the NCO phase offset based on my bench test, rather than running a sweep every time as in the ADI codebase?