Bug 211181

Summary: Safari users are not able to hear audio from remote participants
Product: WebKit Reporter: Jaya <jaya.allamsetty>
Component: WebRTCAssignee: Nobody <webkit-unassigned>
Status: NEW ---    
Severity: Major CC: bfulgham, feross, webkit-bug-importer, youennf
Priority: P2 Keywords: InRadar
Version: Safari 13   
Hardware: Mac   
OS: macOS 10.15   
Attachments:
Description Flags
Jitsi logs none

Description Jaya 2020-04-29 07:20:00 PDT 
Created attachment 397959 [details]
Jitsi logs

In a webrtc call with around 5-6 participants, safari browser users are getting zero audio levels for some of the remote participants.
All the non-safari users can hear everyone fine. This starts happening when there are atleast 4 other participants in the call.
The remote tracks signaled by the SFU are added to the peerconnection in the remote sdp, track events are fired by the browser and then these are attached to a container.

Steps to reproduce:
Join a call on https://meet.jit.si with 4 other participants.
Checks if the safari user is able to hear everyone else.

This happens on macOS (10.14 & 10.15) very consistently using Safari 13.1. We have reports from the Jitsi community that iOS users are also experiencing this issue.

Attached are Jitsi logs for one such audio track where the issue was observed.
Comment 1 Jaya 2020-04-29 08:18:35 PDT 
This seems to be easily reproducible when a remote participant leaves and joins the call again. 
When a remote source is removed from conference, we change the direction on the corresponding m=line in the sdp as "inactive" and then re-use any inactive m-lines of the same media type when a new remote source is added to the conference. Is it possible that this is what causing the audio issue ?
Comment 2 youenn fablet 2020-04-29 08:23:59 PDT 
(In reply to Jaya from comment #1)
> This seems to be easily reproducible when a remote participant leaves and
> joins the call again. 
> When a remote source is removed from conference, we change the direction on
> the corresponding m=line in the sdp as "inactive" and then re-use any
> inactive m-lines of the same media type when a new remote source is added to
> the conference. Is it possible that this is what causing the audio issue ?

I think this is the issue here.
Is it Safari specific code?

It seems in that case webkit receives a live stream whose WebRTC backend RemoteAudioSource is ended, so exits early when trying to register an observer.
Comment 3 Jaya 2020-04-29 08:26:16 PDT 
(In reply to youenn fablet from comment #2)
> (In reply to Jaya from comment #1)
> > This seems to be easily reproducible when a remote participant leaves and
> > joins the call again. 
> > When a remote source is removed from conference, we change the direction on
> > the corresponding m=line in the sdp as "inactive" and then re-use any
> > inactive m-lines of the same media type when a new remote source is added to
> > the conference. Is it possible that this is what causing the audio issue ?
> 
> I think this is the issue here.
> Is it Safari specific code?

No, we do the same for Firefox as well and we don't see any issue there.

> It seems in that case webkit receives a live stream whose WebRTC backend
> RemoteAudioSource is ended, so exits early when trying to register an
> observer.
Comment 4 youenn fablet 2020-04-29 09:23:15 PDT 
(In reply to Jaya from comment #3)
> (In reply to youenn fablet from comment #2)
> > (In reply to Jaya from comment #1)
> > > This seems to be easily reproducible when a remote participant leaves and
> > > joins the call again. 
> > > When a remote source is removed from conference, we change the direction on
> > > the corresponding m=line in the sdp as "inactive" and then re-use any
> > > inactive m-lines of the same media type when a new remote source is added to
> > > the conference. Is it possible that this is what causing the audio issue ?
> > 
> > I think this is the issue here.
> > Is it Safari specific code?
> 
> No, we do the same for Firefox as well and we don't see any issue there.

Is there a way to test this code path with Chrome as well?
This might be an issue within libwebrtc code.
Comment 5 Jaya 2020-04-29 10:03:04 PDT 
(In reply to youenn fablet from comment #4)
> (In reply to Jaya from comment #3)
> > (In reply to youenn fablet from comment #2)
> > > (In reply to Jaya from comment #1)
> > > > This seems to be easily reproducible when a remote participant leaves and
> > > > joins the call again. 
> > > > When a remote source is removed from conference, we change the direction on
> > > > the corresponding m=line in the sdp as "inactive" and then re-use any
> > > > inactive m-lines of the same media type when a new remote source is added to
> > > > the conference. Is it possible that this is what causing the audio issue ?
> > > 
> > > I think this is the issue here.
> > > Is it Safari specific code?
> > 
> > No, we do the same for Firefox as well and we don't see any issue there.
> 
> Is there a way to test this code path with Chrome as well?
> This might be an issue within libwebrtc code.

Unfortunately, I don't have a quick way of testing this code path with chrome, we are still running chrome in plan-b mode.
Comment 6 Radar WebKit Bug Importer 2020-05-20 14:17:32 PDT 
<rdar://problem/63463249>
Comment 7 Feross Aboukhadijeh 2020-06-02 16:42:59 PDT 
This issue seems possibly related to: https://bugs.webkit.org/show_bug.cgi?id=212669
Comment 8 youenn fablet 2020-06-11 06:18:17 PDT 
No longer reproduces on meet.jit.si.
A workaround was added by jitsi to not reuse for now the transceiver made inactive.
Comment 9 youenn fablet 2020-06-11 07:47:27 PDT 
Surfacing the state of the remote source at the track level in https://bugs.webkit.org/show_bug.cgi?id=213074.
This also contains a small repro case of the jitsi issue.
Comment 10 youenn fablet 2020-06-11 08:26:23 PDT 
(In reply to youenn fablet from comment #8)
> No longer reproduces on meet.jit.si.
> A workaround was added by jitsi to not reuse for now the transceiver made
> inactive.

Another workaround might be to keep the ssrcs in the offer.
Comment 11 Jaya 2020-06-15 11:31:56 PDT 
(In reply to youenn fablet from comment #10)
> (In reply to youenn fablet from comment #8)
> > No longer reproduces on meet.jit.si.
> > A workaround was added by jitsi to not reuse for now the transceiver made
> > inactive.
> 
> Another workaround might be to keep the ssrcs in the offer.

So just overwrite the ssrcs in the m-line if it needs to be re-used to for a new remote source ?
Comment 12 youenn fablet 2020-06-16 00:45:26 PDT 
(In reply to Jaya from comment #11)
> (In reply to youenn fablet from comment #10)
> > (In reply to youenn fablet from comment #8)
> > > No longer reproduces on meet.jit.si.
> > > A workaround was added by jitsi to not reuse for now the transceiver made
> > > inactive.
> > 
> > Another workaround might be to keep the ssrcs in the offer.
> 
> So just overwrite the ssrcs in the m-line if it needs to be re-used to for a
> new remote source ?

Overwriting will probably not work.
If you can reuse the same ssrcs, keeping them in the offer should work.
I haven't played with an m section with multiple ssrcs.
Comment 13 Jaya 2020-06-16 05:26:16 PDT 
(In reply to youenn fablet from comment #12)
> (In reply to Jaya from comment #11)
> > (In reply to youenn fablet from comment #10)
> > > (In reply to youenn fablet from comment #8)
> > > > No longer reproduces on meet.jit.si.
> > > > A workaround was added by jitsi to not reuse for now the transceiver made
> > > > inactive.
> > > 
> > > Another workaround might be to keep the ssrcs in the offer.
> > 
> > So just overwrite the ssrcs in the m-line if it needs to be re-used to for a
> > new remote source ?
> 
> Overwriting will probably not work.
> If you can reuse the same ssrcs, keeping them in the offer should work.
> I haven't played with an m section with multiple ssrcs.

Oh ok, thanks for clarifying. Re-using the ssrcs will not be possible in our case since these are signaled by the jitsi video bridge and would involve a lot of redesign there. Thanks for the suggestion though.