Your optical test makes sense and also explains why it did not help. Optical drops the lip sync metadata and forces compressed audio which often increases buffering. HDMI with Passthrough gives you the shortest path.
Set the console to PCM, set the TV to Passthrough, disable TV sound effects, and run the bar in a basic mode. If it is still behind, add a bit of video delay on the TV for that input and you should be good.
