All GMSL1 parts make a frame of 30 or 40 bits for each pixel. Therefore the GMSL link speed is the internal pixel clock (PCLK) times the number of bits per pixel. The GMSL link speed maximum is the same for 30 or 40 bit frames, so the 30 bit frame mode has the higher maximum PCLK.
For display parts, like the MAX9275, the maximum PCLK is 104MHz which gives a maximum 104MHz x 30bits = 3.12Gbps link rate.
For camera parts, like the MAX96705, the maximum PCLK is 116MHz. However, this is in double clocked mode where the interface clock is twice the internal pixel clock, allowing 2 input samples to be made into one pixel. So the pixel rate is 58MHz maximum which gives a maximum 58MHx x 30 bits = 1.74Gbps.
This explains why camera SerDes seem faster, but are in fact slower (and lower pin count and lower power).