Post Go back to editing

IIO: RX buffer is chopped and filled with zeros. RX/TX synchronization.

Hi,

First, thank you for Pluto! I think it's great, and it's a very affordable way to learn SDR. 

After some initial tests with Gnu-Radio, I went for IIO and tried to run a simple test of TX + RX with digital loopback enabled. I would like to send a TX  buffer and receive it into RX buffer. Maybe in the end I would like to get a simple VNA scenario to work. I have some issues I can't resolve. So first I will write my questions and then I will explain the details.

- How should I call IIO push/refill functions from a single thread (or I shouldn't do that)? My RX buffer gets "chopped" and filled with zeros.

- Can I synchronize TX/RX in such a way that I will know what is a delay between buffers (in terms of number of samples)?

The scenario I run is:

1. Open RX and TX on the same LO frequency with the same sampling rate.

2. Create channels/buffers needed. 

3. Enable digital loopback.

4. Prepare a sinewave using inverse fft.

5. Push TX buffer, refill RX buffer in a loop. Currently 20 iterations. 

6. Catch 10th iteration RX buffer into an array, so I can analyze it later. The idea is that it should contain TX buffer, OR TX buffer shifted by some constant offset + beginning of next TX buffer. Catching 10th iteration should neglect any effects of first/last buffers.

7. Print TX and 10th RX buffer for comparison.

(Additionally I calculate amplitude+phase of both TX and RX).

Code used: https://github.com/mikuslaw/iio/blob/master/main.c

Link it with v3 FFTW and IIO, obviously.

What I get: 

https://github.com/mikuslaw/iio/blob/master/test.txt

As you may see in the file, the RX is offset by around 10k samples. RX contains samples starting from 10k upto 32k of TX and then 10k zeros. I didn't expect zeros there! Any ideas why is that?

Should I call push/refill from two threads for it to work? I thought that there were some internal buffers that are swapped with the buffer I push/refill so that would be short call. I know that refill/push calls may block, but is that the issue here?

Another thing: Can I synchronize TX/RX so I have a constant phase shift? It changes every time I run the code.

NOTE: In the code I use 1/4th of RX buffer to calculate amplitude/phase to reduce the "zeros" effect on those, but it's not a solution of the root cause.

  • The main issue here is the sample rate you are selecting. You cannot set the sample rate below 2.08 MHz without loading an FIR. The radio is probably running at 30 or 60 MHz, which will cause underflows (starvation) and overflows (dropped data) on the TX and RX respectively. This is where the zeros come from.  We have a special functions for loading filters.  The easiest is here: https://github.com/analogdevicesinc/libad9361-iio/blob/c70ed5fe56624e420df87191d4288c3da82062c4/ad9361_baseband_auto_rate.c#L80

    In general the latency between buffers will be fixed for a given simulation if you can keep up with the data streaming in. Otherwise you will have dropped some data on the floor and have no idea how much was lost. To help with things on the TX side I would recommend using cyclic buffers. That way you don't have to keep passing new data into the TX buffers.

    Also know that you will have precision loss in your data since the part and HDL only use 12bits. Please this page about how data is cast inside the FPGA: https://wiki.analog.com/resources/fpga/docs/axi_ad9361#internal_interface_description

    -Travis

  • Buffer push and fill are both blocking.

    Either you create two processes. Or you pre-push some tx buffers, to fill the pipeline. Then create the rx buffer (since it's filling them once created) and then you enter your loop. 

    -Michael

  • Hello Travis,

    thank you for the book on SDR, great to have a good source of information.

    I tried to run the code with 2MSPS on both RX and TX, only later dropped the sample rate to 500k, just to make sure my PC is not dropping data. I think, as I mentioned, while I use a digital loopback, that may not be a factor here. Radio should be omitted. Still, the info about functions for setting baseband sample rate is great. I'm going to need it when I will use a move to using a TX->RX cable connection.

    If I use a cyclic buffer it works. I stared with such configuration. I would like to be able to send non-cyclic data as well, that is why I try to find a more general solution, and hopefully understand what is correct way to call the API. 

    When I generate samples, I zero out 4 LSBs on TX, and I shift samples up by 4 bits, to have comparable numbers. It's only cosmetics, but it should cover 12bit HDL width.

    No matter what size of buffer I use, no matter what sample rate I use, the number of zero samples at the end of the buffer seem to stay roughly the same. I would lean to what mhennerich wrote, I need to work on the way how I call the buffer API.

    Thank you.

  • Hello Michael,

    I knew that push and refill are both blocking. I assumed that when I call refill, it uses data it already gathered in some internal buffer, and just returns that buffer in a short time. I assumed that only if I called the refill again, that would block until I got enough samples. I see that I didn't understand the idea fully.

    The information that the buffer is being filled right after it was created is an important one. It may be a reason of my phase shift jumping around - I create a RX buffer somewhere in the code without care.

    But even if I use two processes, or pre-fill the TX pipeline, I don't see a good way to more or less synchronize TX and RX. If I understand correctly, whatever I do at the API level is several (not sure how many) buffers away from the AD936X digital interface. 

    It would be nice to have IIO parameter to "hold" buffers on TX and "refilling" of RX buffers if set. That way I could prefill TX without pushing that out, then ask for RX buffer, and kick the parameter to start both RX and TX at the same time. I guess that would need to be done on hdl level?

    I will start with pre-filling TX as you suggested. Any ideas how many buffers should I push on start?

    Thank you a lot for your help,

    Jerzy 

  • Its not really how many buffers, its more about the data. The max buffer is 2^24 by default, which should keep the DMA busy for awhile.

    -Travis

  • Hi, I just wanted to write that I manged to get the example running as expected. I loopback samples from thread TX and receive same (with offset) samples in RX thread. Unfortunately each time I run the example code, I get to see a different phase (possibly due to different offset in RX buffer), which is stable in a given run, but not stable between runs.
    Maybe you have some solution for that (to be able to synchronize RX and TX on fpga level? I would think that it would be great to have a bit, which holds capture into RX and send from TX buffer when high, and clears RX and unblocks TX on falling edge. But I guess I know too little of the implementation to say if that is reasonable.

    Also I tried to run it once and keep it running, and then change frequency every n-seconds (I try to create a simple VNA, so I need to calibrate plutos own response), but at some specific frequency boundaries I get to see a drastic phase shift again. I will analyze that if that means the device is "reinitialized" on some LO frequency boundaries, or there is a digital filter which would behave like that. Now that I think about that, a with tuning up and then down will give me an answer if the phase shift is deterministic. I will address that in a separate thread when I have enough info.
    For example 

    Opening contex.
    Opened contex.
    Opening phy device.
    Opened phy device.
    Opening dds device.
    Opened dds device.
    Opening lpc devices.
    Opened lpc devices.
    Set loopback value. Result: 0.
    Opening tx and tx_lo devices.
    Opening tx and tx_lo devices.
    Opening rx and rx_lo devices.
    Opened rx and rx_lo devices.
    Opening receive channels.
    Opened receive channels.
    Channel is scan element> rx0_q:1, rx0_i:1.
    Channel is enabled> rx0_q:1, rx0_i:1.
    Channel is output> rx0_q:0, rx0_i:0.
    Opening transmit channels.
    Opened transmit channels.
    Channel is scan element> tx0_q:1, tx0_i:1.
    Channel is enabled> tx0_q:1, tx0_i:1.
    Channel is output> tx0_q:1, tx0_i:1.
    Creating receive buffer.
    Created receive buffer.
    Creating transmit buffer.
    Created transmit buffer.
    Preparing tx samples.
    Prepared tx. [Out] Amplitude: 32766.238282, Phase: 1.570796.
    Preparing place for rx samples.
    Prepared place for rx samples.
    Processing start...
    Enter rx.
    Enter tx.
    After wait.
    Start tx.
    Start rx.
    Received rx. [In] Amplitude: 0.000000, Phase: 0.000000.
    Received rx. [In] Amplitude: 0.000000, Phase: 0.000000.
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.147646.
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    RX: frequency shift: 1000000000
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    RX: frequency shift: 1020000000
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    RX: frequency shift: 1040000000
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    RX: frequency shift: 1060000000
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    RX: frequency shift: 1080000000
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    RX: frequency shift: 1100000000
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    Received rx. [In] Amplitude: 32766.238282, Phase: 0.762005.
    RX: frequency shift: 1120000000
    Received rx. [In] Amplitude: 32766.238282, Phase: -0.532100.
    Received rx. [In] Amplitude: 32766.238282, Phase: -0.532100.
    Received rx. [In] Amplitude: 32766.238282, Phase: -0.532100.
    Received rx. [In] Amplitude: 32766.238282, Phase: -0.532100.
    Received rx. [In] Amplitude: 32766.238282, Phase: -0.532100.
    RX: frequency shift: 1140000000
    Received rx. [In] Amplitude: 32766.238282, Phase: -0.532100.
    Received rx. [In] Amplitude: 32766.238282, Phase: -0.532100.
    Received rx. [In] Amplitude: 32766.238282, Phase: -0.532100.
    Received rx. [In] Amplitude: 32766.238282, Phase: -0.532100.
    Received rx. [In] Amplitude: 32766.238282, Phase: -0.532100.
    RX: frequency shift: 1160000000
    Received rx. [In] Amplitude: 32766.238282, Phase: -0.532100.
    Received rx. [In] Amplitude: 32766.238282, Phase: -0.532100.
    Received rx. [In] Amplitude: 32766.238282, Phase: -0.532100.
    Received rx. [In] Amplitude: 32766.238282, Phase: -0.532100.
    Received rx. [In] Amplitude: 32766.238282, Phase: -0.532100.
    RX: frequency shift: 1180000000
    Received rx. [In] Amplitude: 32766.238282, Phase: -0.532100.
    Received rx. [In] Amplitude: 32766.238282, Phase: -0.532100.
    Received rx. [In] Amplitude: 32766.238282, Phase: -0.532100.
    Received rx. [In] Amplitude: 32766.238282, Phase: -0.532100.
    Received rx. [In] Amplitude: 32766.238282, Phase: -0.532100.
    Received rx. [In] Amplitude: 32766.238282, Phase: -0.532100.
    Received rx. [In] Amplitude: 32766.238282, Phase: -0.532100.
    Received rx. [In] Amplitude: 32766.238282, Phase: -0.532100.

    An example code (working in my case on digital loopback) is located at: https://github.com/mikuslaw/iio/tree/52787dbe298cff4eede24cd998ca53250e337ece
    Maybe it will be of some use to somebody.

    Thank you and Travis for Your suggestions and help,

    Jerzy

  • RX and TX have independent LOs, which makes their phases random and will change slightly when they move. When you go beyond 100MHz from an initial value, the VCO will recalibrate and have a completely new phase. There really is no pattern to the phase offsets. Since the VCO is within the part there is nothing we can really do in the FPGA to compensate for this. If you have a AD9361 you can use external LOs but AD9363s do not support this option.

    -Travis

  • First I want to say that I know I try to make cow to run like a horse :). Pluto was not designed to be a VNA, so I fully accept that most probably this will not let me measure phase. Now I'm just trying to get most of knowledge from it and will later focus on measuring just amplitude vs freq. I didn't consider that RX and TX VCOs may be out of phase. You are right that it's a crucial factor.

    I have some followup questions though. I'm still limiting my tests to loopback on digital interface. I see a phase change more or less after tuning 100MHz away from initial value. That would confirm what you wrote, but I don't yet understand why do I see that on digital loopback path. Are the buffers reinitialized as well with VCO?

    I also tested the same way with a TX->RX coax and no loopback. I see a phase change every time I retune by 20MHz. Phase is kept constant if I don't retune, which is expected.

    I looked at Reference for AD9363, clock generation specifically. A reference clock is common to RX and TX, then it may be divided or not separately for RX and TX, and then it enters PLL for each. I will improve my knowledge on PLLs in next few days, but shouldn't that mean that if I set the same freq (same division/same callibration etc) for RX/TX it would be phase locked to common ref clock?

    Fref

    Thank you for your insight,

    Jerzy