Not reaching 8 MACs per cycle in SHARC-FX core processing

Hi Leopoldo,

Thank you for your inquiry. We are checking this query with Internal Development team. We will get back to you once we get a response from them.

Best Regards,
Santhakumari.V

Hi Leopoldo,

The main reasons why the code below does not get 1 PDX_MULA_MXF32 per cycle is because:

1) There is inner loop overhead to set up and shut down the loop with some stalls
2) There is a reduction of the unrolled accumulators to a single output vector, with more stalls
3) A little outer loop overhead

In the big picture, the issue is that the overhead outside the loop is significant compared to what happens inside the loop.

Here's what the code looks like when all in-lined:
// firInlineRun
START_CYCLE_COUNT (var);
for (int i=0; i<20; i++) {
  // macTest
  vectorSum = 0;
  for (int i=0; i<72; i++) {
    PDX_MULA_MXF32(vectorSum, inVectors[inChannel][i], coeffVectors[inChannel][i]);
  }
  outVectors[outChannel][0] = vectorSum;
}
STOP_CYCLE_COUNT (cyc, var);

The cycle counter reports 2031 cycles to do 20*72=1440 MULAs, or 101 cycles per macTest. The inner loop really does take 72 cycles spinning on the MULAs, so there are 29 of overhead.

The overall breakdown of one pass looks like:
| Cyc | Phase |
|-----|-----------------|
|   8 | loop prologue |
|  72 | loop |
|   8 | loop epilogue |
|   8 | unrolled accumulator reduction |
|   4 | outer loop overhead |
| 100 | total |

14 are stalls. There is an extra cycle for an initial Icache miss.

There are tools outside of CCES that can be used to profile this down to the instruction level, but let me describe those separately.

Hi jlredford , any chance you could please elaborate on the tools that can be used to profile the code/instructions?
Many thanks, 
Simon

Hello Leopoldo,

The extra cycles come from two places:

• The loop setup and shutdown.  The loop takes a few cycles to set up the unroll and a few at the end to clean up the result
• The reduction at the end of a summation loop. 

The compiler turns this single dot-product into four interleaved ones in order to hide the pipeline stalls in the float multiply-accumulate instruction.  There are three stalls from writing a register in a float32 instruction to being able to use it by another ALU operation.  Those three slots can be filled by three extra MACs on different result registers.  So the compiler creates four sums and then adds them together at the end.  It then has to add up all the elements of the final 8-way vector, which it does by shifting the final vector by four elements and then adding to itself, then shifting by 2 and adding, and then shifting by 1 and adding.  Those adds take a lot of stalls themselves, because they have to wait for the pipeline to empty before doing the next shift-and-add.  There's no help for this if you have just a single final float result.

Note that SHARC+ needs to do far less reduction because it's only 2-way SIMD and only has 1 pipe stall on floats.  It only needs to be unrolled by 2.

The FIR code can do better by computing more than one sum at a time.  The best case is where it's doing 32 sums at a time, i.e. a FIR window of 32.  Then all the sums accumulate within each element of four 8-way vectors, and no reduction is needed at all.  

The other way to help this is to do something else to the answer, like scaling it or adding an offset.  That can be overlapped with the final summation and so reduce the time spent.  That also saves doing extra loads and stores.

Regards,

John