using LTC6268 as a TIA, but has a sinusoidal waveform which is 300Mhz in the output port

Can you provide a link to the photodiode data sheet?

Can you provide a photo of how the layout is realized?

Here is the datasheet for PD and a screenshot of the PCB layout for the devices in the schematic above (4-layer board TOP AVCC AGND BOTTOM).

PDF

Dear sir:

         do you have some advice for this issue?

Best Regards!

Dear sir:

         do you have some advice for this issue?

Best Regards!

This might be a result of the measurement setup.. What's your scope's input impedance? Typical scopes have relatively high input capacitance... I'd also verify that C20 is in fact 0.1pF. Large capacitive paths to ground at the output can lead to a larger feedforward path in the amplifier that, at high enough frequencies, could result in this feedforward path dominating (rather than the feedback path). In other words, you're generating peaking in your transfer function which leads to oscillations.

That's just a guess, but I just posted about this phenomenon - I observed peaking in simulations around 1GHz so I wanted to ask the community if they've experienced something like this and it seems you may have. My suggestion - try higher impedance scope probes first (i.e. connect a 10M resistor in series with your probe). If that resolves the issue (or shifts the oscillation frequency) then that's definitely the problem. If not, then there may be some other issue.

Best,

Andy