AD9364  constellation

I set 0x3F5=0xC1,loopback data to FPGA,and get constellation diagram:


But when I loopback signal from TX to RX outside of AD9364:

There is some noise,and the BER raised
Does anyone know how it happened?

Note that the constellation were plotted by matlab,and the raw data were exported by FPGA design sofeware.

The following information may be helpful
set_rx_lo_freq 5860000000
set_tx_lo_freq 5860000000
set_rx1_gc_mode 0
set_rx1_rf_gain 11