I have been testing out my FMCDAQ2 with the following;
Vivado 2014.2, ZC706 with No-OS software. hdl-Master and No-OS downloaded from Git on 23rd Jan 2015.
Instead of you demo design transmitting a DDS in the AD9144_core and then looping back via the SMA's into the ADC inputs, I used Vivado IP Integrator block diagram method to connect the axi_ad9680_core outputs adc_data_0(63:0) and adc_data_1(63:0) to the axi_ad9144_core inputs dac_ddata_0(63:0) and dac_ddata_1(63:0) replacing the original inputs of dac_ddata_0(63:0) and dac_ddata_1(63:0) (aka "dma_data" of the selectable source data for the DAC in axi_ad9144_channel.v).
By doing this then what signals you feed to the SMA inputs "In_A" and "In_B" should come out the outputs "Out_A" and "Out_B".
I down-converted my RF signal using an I-Q mixer, feeding I into "In_A" and Q into "In_B". Fed the outputs "Out_A" and "Out_B" into another I-Q mixer for up-conversion.
The image frequency was at the same level as the wanted sideband. Investigations showed that the output IF from "Out_B" was at -12dBm (as expected) but the output from "Out_A" was >30dB less at about -43dBm.
I've tried it on 2 different ZC706's as well as using 2 different FMCDAQ2's so presumably not a hardware issue/breakage.
My assumption is that the problem is in the hdl but I was brought up on VHDL not verilog. One test I did was to connect a single ADC output signal adc_data_0(63:0) to all 4 axi_ad9144_core inputs dac_ddata_0/1/2/3(63:0) with the same resultant outputs from the SMA's. This points to the problem being in the DAC core and not the ADC core.
Although I didn't try it, I assume the "DDS sourced" test as per your original design puts out equal signal levels from both SMA's or you would have noticed by now. The only difference here is that I am using another source of data for the DAC inputs. I thought it might be to do with data packing/unpacking formats i.e. the data out of the axi_ad9680_core was packed differently to the expected format into the axi_ad9144_core but this should have been evident when I varied the input power levels, which would have caused random amplitude jumps on the outputs (LSBs being fed to MSBs etc.).
I've tried following the verilog in the axi_ad9144 pcore but can't see anything obvious and have reached a dead end, hence my plea for help and suggestions from you.