ADG884 switch Bi-Directional..?

Hi

We are using the ADG884 switch in an audio application.

The end device has one mono audio input connector, and the ADG884 is being used to select whether it is in input or output mode - see image below.

Question --- are the audio channels in the ADG884 bi-directional ?  Should running D1->S1A be the same as S1A->D1

In our application we seem to see high noise on the "input mode" path D1->S1A but see low noise on "output mode" S1B->D1

Is it possible the switch is not designed to be driven from D1 and this is causing audio issues..?

Thanks