In the AD9371/AD9375 datasheet, The receiver maximum Recommended Input Power is given as −14 dBm (Input is a CW signal at a 0dB attenuation setting; this level increases decibel for decibel with attenuation Maximum Rx gain, at Rx port, matching losses de-embedded). Also the below note is stated,
Note8: Note that the input signal power limit doesn't correspond to 0 dBFS at the digital output because of the nature of the continuous time Σ-Δ ADCs. Unlike the hard clipping characteristic of pipeline ADCs, these converters exhibit a soft overload behavior when the input approaches the maximum level.
The above max input level is specified at 0dB attn then receiver maximum receiver gain will be around 15dB depends on frequency based on your datasheet. Also the above note says that ADC input power is not 0 dBFS at -14dBm input because of the nature of the continuous time Σ-Δ ADCs.
So considering the 15dB gain your ADC input power level at -14dBm is around +1dBm roughly.
What is the ADC noise floor at sampling frequency of 40.96MSPS (20.48MHz) signal in your receiver or ADC input? This data may not be with min-max, you can provide the typical value in any one frequency?
Also due to the receiver has noise figure of around 12-14dB we know it will reduce your stand-alone receiver/ADC noise floor, Based on this data we will use our front-end with low noise figure and gains to get best instantaneous dynamic range performance without AGC from AD9371/AD9375.
I have attached your datasheet spec and note for your reference.