Post Go back to editing

Deframer error 0x61

Category: Software
Product Number: AD9371
Software Version: 7ea497b6c24172ee293d6e0a9a387a16973da5ce 2022_R2 https://github.com/analogdevicesinc/hdl/tree/2022_R2

Hello, 

We are having trouble with the deframer/tx path using the AD9371 - getting the error code 0x61, 0x21 etc having the "frame symbol lost" as the common denominator. 

We can close to consequently reproduce the error by resetting the FPGA and then running the initialization for the JESD204B link. Additionally, we see "BADDISP" when reading "MYKONOS_deframerGetIrq". Based on the readouts, we can see that it fluctuate between CGS and ILAS phase, not reaching the DATA state. 

Adding a debug core (ILA-core) to some of the signals reveal some weird behavior during the 2 deframer errors. Especially, the ilas_config_data is all 0's for the 2 errors - what might be the cause of this? 

Waveform of working link:

Waveform with deframer error 0x61 (notice how the data is first sent on the second multiframe after sync?):

Waveform with deframer error 0x21 (notice the sync deassertion during the ILAS phase):

Additionally, the SYNC signal falls close to the LMFC boundary in all 3 waveforms, which in our understanding should fall somewhat in the middle of a LMF. This can controlled via the LMFC_offset, right?

We are using Vivado 2022.2 and ADI-HDL version 2022_r2: 

7ea497b6c24172ee293d6e0a9a387a16973da5ce 2022_R2 https://github.com/analogdevicesinc/hdl/tree/2022_R2

Hope you can give some much needed insight into the HDL and the problem. 

Thanks in advance, 

Thomas

  • Here is part of the mykonos init: 

    AD9528 request sysref
    MYK2 DeframerFifoDepth = 0
    MYK2 readEnLmfcCount = 0
    MYK1 check rx framer status
    MYK2 check rx framer status
    ----Middle of Framer check----
    tx_jesd status:
            Link is enabled
            Measured Link Clock: 149.765 MHz
            Reported Link Clock: 149.760 MHz
            Lane rate: 5990.400 MHz
            Lane rate / 40: 149.760 MHz
            LMFC rate: 4.680 MHz
            SYNC~: asserted
            Link status: CGS
            SYSREF captured: Yes
            SYSREF alignment error: No
    MYK1 check obs rx framer status
    MYK2 check obs rx framer status
    MYK2 DeframerIrq = 0x04
    MYK2 deframerSetIrqMask = 0x00
    MYK2 DeframerFifoDepth = 0
    MYK2 readEnLmfcCount = 0
    MYK2 check deframer status
    MYK2 DeframerStatus = 0x61
    AD9371 initialization failed on try 0

  • The deframer and framer status should be 0x28 and 0x3E respectively .

    From the error log looks like the JESD is stuck at CGS phase, and the deframer status states that you have deframer IRQ and FS lost issues.

    Can you please cross check if the JESD config parameters i.e. lane rate , number of lanes configured on AD9371 and FPGA are same? Because this could be one of the reason for the JESD to be stuck in CGS phase.

  • The parameters are matching for the AD9371 and FPGA as far as i can tell, and are:

    F = 4

    K = 32 

    N = 2

    Lanes = 1 

    Can't comment on the Lane rate since i don't know if/where we set/pass it to the HDL. But is set as 5990.400 MHz for the AD9371. 

  • additional point to notice is this error is fairly intermittent. ie it happens once every 10th reset/reboot on the startup, but if ran again it succeeds... We see it not going beyond the ILAS phase during this first initialization. 

  • The lane rate should match for both AD9371 and FPGA . This would be one of the reason for JESD stuck in CGS phase.

    If you are not clear on how to set the lane rate in FPGA , please post this query in FPGA forum.

  • The lane rate should match for both AD9371 and FPGA . This would be one of the reason for JESD stuck in CGS phase.

    If you are not clear on how to set the lane rate in FPGA , please post this query in FPGA forum.

  • The lane rate should match for both AD9371 and FPGA . This would be one of the reason for JESD stuck in CGS phase.

    If you are not clear on how to set the lane rate in FPGA , please post this query in FPGA forum.

  •     const uint32_t rx_lane_rate_khz = _configuration->mykDevice_1->rx->rxProfile->iqRate_kHz * _configuration->mykDevice_1->rx->framer->M *
                                          (20 / no_os_hweight8(_configuration->mykDevice_1->rx->framer->serializerLanesEnabled));
    
        const uint32_t tx_lane_rate_khz = _configuration->mykDevice_1->tx->txProfile->iqRate_kHz * _configuration->mykDevice_1->tx->deframer->M *
                                          (20 / no_os_hweight8(_configuration->mykDevice_1->tx->deframer->deserializerLanesEnabled));
    
        const uint32_t rx_os_lane_rate_khz = _configuration->mykDevice_1->obsRx->orxProfile->iqRate_kHz * _configuration->mykDevice_1->obsRx->framer->M *
                                             (20 / no_os_hweight8(_configuration->mykDevice_1->obsRx->framer->serializerLanesEnabled));
    
        if (rx_lane_rate_khz != tx_lane_rate_khz || rx_lane_rate_khz != rx_os_lane_rate_khz)
        {
            // All the rates are expected to be identical, so this must be an invalid configuration.
            LOG_ERROR("error: Invalid JESD lane rate configuration");
            return false;
        }
    
        const uint32_t jesd_lane_rate_khz = rx_lane_rate_khz;
        const uint32_t jesd_line_rate_khz = jesd_lane_rate_khz / 40;
    
        struct jesd204_rx_init rx_jesd_init = {
            "rx_jesd", RX_JESD_BASEADDR, 4, 32, 1, jesd_line_rate_khz, jesd_lane_rate_khz,
        };
    
        struct jesd204_tx_init tx_jesd_init = {
            "tx_jesd",                      // Device Name
            TX_JESD_BASEADDR,               // Base address
            4,                              // Octets per frame (F)
            32,                             // Frames per multi-frame (K)
            2,                              // Converters per device (M)
            14,                             // Converter resolution (N)
            16,                             // Bits per sample (N')
            false,                          // High density (HD)
            2,                              // Control bits per conversion sample (CS)
            1,                              // JESD204B subclass
            jesd_line_rate_khz,             // Device Clock in KHz
            jesd_lane_rate_khz,             // Lane clock in KHz
        };

    We set the lane rate correctly for both the AD9371 and FPGA via the axi connection.

    From the waveforms it seems the ilas_config_data is set during the ILAS phase? While when it fails the config doesn't get set?

    When it works, we can see the correct config data being set by "axi_jesd204_tx_set_lane_ilas_legacy". 

    We should probably make a post in the FPGA forum...

  • I have further checked the transceiver settings and set it up using the Ultrascale transceiver Wizard - setting a line rate of 5.9904 Gb/s. (Also confirmed via manual calculations based on the Wizard UG)

    This still gives the same errors on the deframer.

    It seems SYNC often goes low before a full multiframe has been sent by the FPGA, meaning AD9371 is potentially seeing "wrong" multiframes in the sense that it has a shifted "view" - could the LMFC start point be slightly different from power-up to power-up between AD9371 and FPGA?

  • We have furthermore checked the physical TX lane trace with a high speed oscilloscope and seen a good eye-diagram and good data transmission. During this we captured a waveform of it failing in the first multiframe of the ILAS phase during the ramp data (failing as in SYNC was asserted and it went back into CGS). 

    Additionally, we found that Character Replacement was OFF on the FPGA side - turning it on made a statistical difference, but still error coding with 0x61 etc. 

      Would it be possible to get some pointers again, as to what this could be?