Bug 217713

The report is filed against Gtk, however the proposed change is cross platform. Not sure who would look at this for Gtk, but CC'ing some Apple folks just in case.

Yes, it's not GTK specific. The issue has been seen on WPE too.

Anyway, the issue can be also reproduced on Webkit GTK 2.30.1 without any patches (the 1 second sleep). It was also reproduced on GNOME Web 3.28.6. To do it, timings in the Javascript code must be adjusted.

The usual flow between appending a buffer and seeking is as follows:

1. SourceBuffer::appendBuffer appending a buffer.
2. AppendPipeline::appsrcEndOfAppendCheckerProbe, posting a task to handle end of buffer.
3. AppendPipeline::handleEndOfAppend is called, it updates buffered ranges.
4. We are setting the current position. There is logic in HTMLMediaElement::seekWithTolerance which posts a task HTMLMediaElement::seekTask to the queue when the seek was from DOM.
5. HTMLMediaElement::seekTask is executed. There are buffered ranges available (because they were updated while executing AppendPipeline::handleEndOfAppend), it sets the m_lastSeekTime to some valid value.
6. HTMLMediaElement::currentMediaTime is called. Because we are still seeking, then the value of m_lastSeekTime is returned as the current time.

To reproduce the issue, HTMLMediaElement::seekTask must be called before AppendPipeline::handleEndOfAppend. It's possible to do it with short timeouts in Javascript code like the seek will happen 1ms after calling SourceBuffer::appendBuffer. Additionaly, after the issue will happen, the controls of HTML audio element are unusable and the audio can't be played.

The updated way to reproduce the issue:
1. Prepare audio media:
gst-launch-1.0 audiotestsrc num-buffers=3000 ! avenc_aac ! mp4mux ! filesink location=audio.mp4
ffmpeg -i audio.mp4 -movflags frag_keyframe+empty_moov -movflags frag_keyframe+empty_moov+default_base_moof output.mp4
2. Use the updated attached HTML file with MSE code.
3. Try to launch the browser a few times or refresh and at some point it should happen.

When the race condition is not hit, then everything works fine. The current time returned is always 0.0 and audio can be played. However, when the issue is reproduced, the current time right after seeking is returned as NaN (it is often also the value of duration if EOS was injected quicker). After injecting EOS, the position is seeked to the end and the audio element becomes unusable during interaction.

I attached screenshots from GNOME Web when the issue was reproduced and also when everything went fine.

The issue can't be reproduced with Chrome 84.

Created attachment 411423 [details]
updated web page to reproduce the issue

Attaching the updated web page to reproduce the issue

Created attachment 411424 [details]
GNOME Web failed test

Attaching a failed test when using GNOME Web

Created attachment 411426 [details]
GNOME Web passed test

Attaching a passed test when using GNOME Web

Another observation is that the issue is more easily reproducible, when the appended buffer to the sourcebuffer is bigger. It makes AppendPipeline to post the task for AppendPipeline::handleEndOfAppend later.

Thank you very much for such a very complete bug report.

(In reply to sycy700 from comment #0)
> There is similar code in Chromium. Not sure if the code below is responsible
> that this issue cannot be reproduced using Chromium, but one difference is
> that when there are no ranges available, then 0.0 is returned instead of
> invalidTime for the 'nearest' function.
> 
> https://github.com/chromium/chromium/blob/
> 2420378bc7c24eee30c48455f0c54aa4c7390aa2/third_party/blink/renderer/core/
> html/media/html_media_element.cc#L2095
> 
> void HTMLMediaElement::Seek(double time) {
> ...
> time = seekable_ranges.Nearest(time, now);

This is not entirely accurate. While Nearest() in Chromium will indeed return 0.0 in this case, that is not actually relevant because the function returns early in the case where there are no seekable ranges:

void HTMLMediaElement::Seek(double time) {
  [...]
  if (seekable_ranges.empty()) {
    seeking_ = false;
    return;
  }
  time = seekable_ranges.Nearest(time, now);

The proposed solution of setting time to zero in the case where there are not seekable ranges is not ideal because it discards the previously requested seek time, and it just happens that it's zero in this case.

I assume the empty seekable ranges is a consequence of not yet knowing the duration of the media (which changes after either setting it manually or when parsing the media). Some things I wonder at the moment are:

I still wonder how come a seek is allowed to happen before there were any seekable ranges.

Thanks for the comment. As it was described previously, there is a short time window between appending a segment and until the processed segment data is visible in seekableRanges->ranges() in the function HTMLMediaElement::seekTask.

Segment processing is done in AppendPipeline, in a different thread than the one where seek is done in HTMLMediaElement.

It's possible to inject a seek request in that time window and then the issue can be reproduced.

I am unable to reproduce this in Safari Technology Preview 187 using `updated web page...` on macOS 14.3 Sonoma.

Is it fixed down the line and now only reproducible on WebKitGTK or other ports? Or we can mark it as 'RESOLVED CONFIGURATION CHANGED'.

*** Safari Technology Preview 187 ***

>>> appended buffer
>>> currentTime: 0
>>> currentTime: 0
>>> currentTime: 0

Same as Chrome Canary 123.

I've tested the updated use case against latest WebKitGTK on desktop and I can't reproduce the issue anymore, even after several tries. I added the AppendPipeline.cpp sleep() call to force things even more, with identical results.

Closing bug as 'RESOLVED CONFIGURATION CHANGED'. sycy700, please feel free to reopen it in case it's still reproducible for you.