Bug 247114 - [GStreamer] Video decoding failed while using nvidia NVDECODE in sandboxed WebProcess
Summary: [GStreamer] Video decoding failed while using nvidia NVDECODE in sandboxed We...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Media (show other bugs)
Version: Other
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords:
: 260422 (view as bug list)
Depends on:
Blocks:
 
Reported: 2022-10-26 23:10 PDT by YanVV
Modified: 2024-05-16 12:06 PDT (History)
4 users (show)

See Also:


Attachments
Screenshot showing GPU video decode usage on Nvidia GPUs (1.11 MB, image/png)
2023-08-20 23:47 PDT, Kdwk
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description YanVV 2022-10-26 23:10:40 PDT
the main output that: 

    0:00:20.920842190     2 0x7f2b3400a800 WARN                 nvcodec plugin.c:94:plugin_init: Failed to init cuda, ret: 0x3e7
    0:00:20.920878921     2 0x7f2b3400a800 WARN     GST_ELEMENT_FACTORY gstelementfactory.c:529:gst_element_factory_create_with_properties:<nvvp9dec> loading plugin returned NULL!
    0:00:20.921227424     2 0x7f2b600115e0 ERROR         webkitmediasrc WebKitMediaSourceGStreamer.cpp:525:webKitMediaSrcLoop:<webkitmediasrc2:src_V0> Pushing buffer returned not-linked


and the full output using GST_DEBUG=2 :

```(epiphany:10736): epiphany-WARNING **: 00:36:49.416: Failed to search secrets in password schema: The name org.freedesktop.secrets was not provided by any .service files
0:00:01.675423740     2 0x7f2d3c004d90 WARN    webkitregistryscanner GStreamerRegistryScanner.cpp:188:hasElementForMediaType: All video encoder elements matching caps video/x-av1 are disallowed
0:00:02.057548450     2 0x7f2bac251920 WARN                 qtdemux qtdemux.c:3121:qtdemux_parse_trex:<qtdemux0> failed to find fragment defaults for stream 1
0:00:02.057600731     2 0x7f2bac251920 WARN                 default qtdemux_tags.c:765:qtdemux_tag_add_revdns:<qtdemux0> This tag com.apple.iTunes:iTunSMPB type:1 is not mapped, file a bug at bugzilla.gnome.org
0:00:02.057608281     2 0x7f2bac251920 WARN                 default qtdemux_tags.c:765:qtdemux_tag_add_revdns:<qtdemux0> This tag com.apple.iTunes:Encoding Params type:0 is not mapped, file a bug at bugzilla.gnome.org
0:00:02.057613611     2 0x7f2bac251920 WARN                 default qtdemux_tags.c:765:qtdemux_tag_add_revdns:<qtdemux0> This tag com.apple.iTunes:iTunNORM type:1 is not mapped, file a bug at bugzilla.gnome.org
0:00:02.080354426     2 0x7f2ba80111e0 WARN         audio-resampler audio-resampler.c:274:convert_taps_gint16_c: can't find exact taps
0:00:02.082221173     2 0x7f2ba8011120 WARN         audio-resampler audio-resampler.c:274:convert_taps_gint16_c: can't find exact taps
0:00:02.083159992     2 0x7f2ba8011120 WARN         audio-resampler audio-resampler.c:274:convert_taps_gint16_c: can't find exact taps
0:00:02.097424027     2 0x7f2bac252240 WARN            audiodecoder gstaudiodecoder.c:1536:gst_audio_decoder_finish_frame_or_subframe:<fdkaacdec0> Can't copy metadata because input buffers disappeared
0:00:02.097438887     2 0x7f2bac252240 WARN            audiodecoder gstaudiodecoder.c:1779:gst_audio_decoder_drain:<fdkaacdec0> audio decoder push buffers failed
0:00:02.386831940     2 0x7f2bc400d120 WARN                 qtdemux qtdemux.c:3121:qtdemux_parse_trex:<qtdemux1> failed to find fragment defaults for stream 1
0:00:02.386876491     2 0x7f2bc400d120 WARN                 default qtdemux_tags.c:765:qtdemux_tag_add_revdns:<qtdemux1> This tag com.apple.iTunes:iTunSMPB type:1 is not mapped, file a bug at bugzilla.gnome.org
0:00:02.386883781     2 0x7f2bc400d120 WARN                 default qtdemux_tags.c:765:qtdemux_tag_add_revdns:<qtdemux1> This tag com.apple.iTunes:Encoding Params type:0 is not mapped, file a bug at bugzilla.gnome.org
0:00:02.386890591     2 0x7f2bc400d120 WARN                 default qtdemux_tags.c:765:qtdemux_tag_add_revdns:<qtdemux1> This tag com.apple.iTunes:iTunNORM type:1 is not mapped, file a bug at bugzilla.gnome.org
0:00:02.405556148     2 0x7f2b8c00d800 WARN         audio-resampler audio-resampler.c:274:convert_taps_gint16_c: can't find exact taps
0:00:02.408107651     2 0x7f2b8c00d300 WARN         audio-resampler audio-resampler.c:274:convert_taps_gint16_c: can't find exact taps
0:00:02.409468884     2 0x7f2b8c00d300 WARN         audio-resampler audio-resampler.c:274:convert_taps_gint16_c: can't find exact taps
0:00:02.428769207     2 0x7f2b9800f400 WARN            audiodecoder gstaudiodecoder.c:1536:gst_audio_decoder_finish_frame_or_subframe:<fdkaacdec1> Can't copy metadata because input buffers disappeared
0:00:02.428787827     2 0x7f2b9800f400 WARN            audiodecoder gstaudiodecoder.c:1779:gst_audio_decoder_drain:<fdkaacdec1> audio decoder push buffers failed
0:00:02.453697382     2 0x7f2bc000f1e0 WARN                 qtdemux qtdemux.c:3121:qtdemux_parse_trex:<qtdemux2> failed to find fragment defaults for stream 1
0:00:02.453738802     2 0x7f2bc000f1e0 WARN                 default qtdemux_tags.c:765:qtdemux_tag_add_revdns:<qtdemux2> This tag com.apple.iTunes:iTunSMPB type:1 is not mapped, file a bug at bugzilla.gnome.org
0:00:02.453758122     2 0x7f2bc000f1e0 WARN                 default qtdemux_tags.c:765:qtdemux_tag_add_revdns:<qtdemux2> This tag com.apple.iTunes:Encoding Params type:0 is not mapped, file a bug at bugzilla.gnome.org
0:00:02.453763892     2 0x7f2bc000f1e0 WARN                 default qtdemux_tags.c:765:qtdemux_tag_add_revdns:<qtdemux2> This tag com.apple.iTunes:iTunNORM type:1 is not mapped, file a bug at bugzilla.gnome.org
0:00:02.467378311     2 0x7f2b78012240 WARN                 qtdemux qtdemux.c:3121:qtdemux_parse_trex:<qtdemux3> failed to find fragment defaults for stream 1
0:00:02.467417762     2 0x7f2b78012240 WARN                 default qtdemux_tags.c:765:qtdemux_tag_add_revdns:<qtdemux3> This tag com.apple.iTunes:iTunSMPB type:1 is not mapped, file a bug at bugzilla.gnome.org
0:00:02.467424342     2 0x7f2b78012240 WARN                 default qtdemux_tags.c:765:qtdemux_tag_add_revdns:<qtdemux3> This tag com.apple.iTunes:Encoding Params type:0 is not mapped, file a bug at bugzilla.gnome.org
0:00:02.467429472     2 0x7f2b78012240 WARN                 default qtdemux_tags.c:765:qtdemux_tag_add_revdns:<qtdemux3> This tag com.apple.iTunes:iTunNORM type:1 is not mapped, file a bug at bugzilla.gnome.org
0:00:02.473697870     2 0x7f2b780120c0 WARN         audio-resampler audio-resampler.c:274:convert_taps_gint16_c: can't find exact taps
0:00:02.476189204     2 0x7f2b78012000 WARN         audio-resampler audio-resampler.c:274:convert_taps_gint16_c: can't find exact taps
0:00:02.477465326     2 0x7f2b78012000 WARN         audio-resampler audio-resampler.c:274:convert_taps_gint16_c: can't find exact taps
0:00:02.483902687     2 0x7f2b5c00d520 WARN         audio-resampler audio-resampler.c:274:convert_taps_gint16_c: can't find exact taps
0:00:02.485763645     2 0x7f2b5c00d460 WARN         audio-resampler audio-resampler.c:274:convert_taps_gint16_c: can't find exact taps
0:00:02.487003086     2 0x7f2b5c00d460 WARN         audio-resampler audio-resampler.c:274:convert_taps_gint16_c: can't find exact taps
0:00:02.497893689     2 0x7f2b7c042cc0 WARN            audiodecoder gstaudiodecoder.c:1536:gst_audio_decoder_finish_frame_or_subframe:<fdkaacdec2> Can't copy metadata because input buffers disappeared
0:00:02.497908749     2 0x7f2b7c042cc0 WARN            audiodecoder gstaudiodecoder.c:1779:gst_audio_decoder_drain:<fdkaacdec2> audio decoder push buffers failed
0:00:02.510140415     2 0x7f2b8003d920 WARN            audiodecoder gstaudiodecoder.c:1536:gst_audio_decoder_finish_frame_or_subframe:<fdkaacdec3> Can't copy metadata because input buffers disappeared
0:00:02.510170355     2 0x7f2b8003d920 WARN            audiodecoder gstaudiodecoder.c:1779:gst_audio_decoder_drain:<fdkaacdec3> audio decoder push buffers failed

(process:10742): GLib-CRITICAL **: 00:37:14.526: Failed to set scheduler settings: Operation not permitted
0:00:20.784822890     2 0x5617266f1520 WARN           matroskademux matroska-demux.c:1143:gst_matroska_demux_parse_stream:<matroskademux5> Unknown TrackVideo subelement 0xec - ignoring
0:00:20.920842190     2 0x7f2b3400a800 WARN                 nvcodec plugin.c:94:plugin_init: Failed to init cuda, ret: 0x3e7
0:00:20.920878921     2 0x7f2b3400a800 WARN     GST_ELEMENT_FACTORY gstelementfactory.c:529:gst_element_factory_create_with_properties:<nvvp9dec> loading plugin returned NULL!
0:00:20.921227424     2 0x7f2b600115e0 ERROR         webkitmediasrc WebKitMediaSourceGStreamer.cpp:525:webKitMediaSrcLoop:<webkitmediasrc2:src_V0> Pushing buffer returned not-linked
0:01:03.050881605     2 0x7f2d3c004d90 WARN          downloadbuffer gstdownloadbuffer.c:973:gst_download_buffer_close_temp_location_file:<downloadbuffer3> Failed to remove temporary file /var/tmp/WebKit-Media-ZJQQU1: No such file or directory
0:01:03.053154446     2 0x7f2d3c004d90 WARN          downloadbuffer gstdownloadbuffer.c:973:gst_download_buffer_close_temp_location_file:<downloadbuffer2> Failed to remove temporary file /var/tmp/WebKit-Media-84KQU1: No such file or directory
0:01:03.059220371     2 0x7f2d3c004d90 WARN          downloadbuffer gstdownloadbuffer.c:973:gst_download_buffer_close_temp_location_file:<downloadbuffer0> Failed to remove temporary file /var/tmp/WebKit-Media-R3WFU1: No such file or directory
0:01:03.061242130     2 0x7f2d3c004d90 WARN          downloadbuffer gstdownloadbuffer.c:973:gst_download_buffer_close_temp_location_file:<downloadbuffer1> Failed to remove temporary file /var/tmp/WebKit-Media-DJWOU1: No such file or directory```
Comment 1 YanVV 2022-10-26 23:16:08 PDT
webkit2gtk-4.1  2.38.1
epiphany 42.4
archlinux
Comment 2 Philippe Normand 2022-12-05 13:00:23 PST
+ Patrick in CC (nvidia user afaik).

YanVV, two questions:

- what is the URL of that website?
- does hw-accelerated playback works with the sandbox disabled?
Comment 3 Patrick Griffis 2022-12-06 11:55:58 PST
(In reply to Philippe Normand from comment #2)
> + Patrick in CC (nvidia user afaik).
> 
> YanVV, two questions:
> 
> - what is the URL of that website?
> - does hw-accelerated playback works with the sandbox disabled?

I got rid of Nvidia. However I do know nvdec+gstreamer worked in regular flatpak applications. I don't think I ever tested it in WebKit. So it should be possible.

Nothing special was required for it to work in flatpak, but the CUDA libraries do have to all be installed.
Comment 4 YanVV 2022-12-15 03:15:12 PST
(In reply to Philippe Normand from comment #2)
> + Patrick in CC (nvidia user afaik).
> 
> YanVV, two questions:
> 
> - what is the URL of that website?
> - does hw-accelerated playback works with the sandbox disabled?

any youtube video

when hw-accelerated work  , and seek forward or seek back ,  video freeze but audio normal.

I test all version :
                                    enable sandbox                                                                disable sandbox
webkit2gtk :          hw-acc not work                                                               hw-acc work but video freeze when seek
webkit2gtk-4.1    hw-acc not work                                                                hw-acc work but video freeze when seek
webkit2gtk-5.1     hw-acc work but video freeze when seek              could not play video at all
Comment 5 YanVV 2022-12-15 03:15:46 PST
(In reply to YanVV from comment #4)
> (In reply to Philippe Normand from comment #2)
> > + Patrick in CC (nvidia user afaik).
> > 
> > YanVV, two questions:
> > 
> > - what is the URL of that website?
> > - does hw-accelerated playback works with the sandbox disabled?
> 
> any youtube video
> 
> when hw-accelerated work  , and seek forward or seek back ,  video freeze
> but audio normal.
> 
> I test all version :
>                                     enable sandbox                          
> disable sandbox
> webkit2gtk :          hw-acc not work                                       
> hw-acc work but video freeze when seek
> webkit2gtk-4.1    hw-acc not work                                           
> hw-acc work but video freeze when seek
> webkit2gtk-5.1     hw-acc work but video freeze when seek              could
> not play video at all

webkit2gtk-5.0  sorry
Comment 6 Philippe Normand 2022-12-15 12:55:21 PST
Which decoder is used when hw acceleration is disabled?

GST_DEBUG="videoenc*:6" might give clues.
Comment 7 Philippe Normand 2022-12-15 12:55:38 PST
GST_DEBUG="videodec*:6" I mean.
Comment 8 YanVV 2023-01-05 01:05:21 PST
when using stateless decoder with sandbox disable , it is work fine.
so , video freeze maybe relate to the decoder

but the hw decode faile with sandbox enable is still not  solved
Comment 9 Philippe Normand 2023-01-05 01:37:15 PST
This won't magically solve itself without anyone working on it. I don't have NVIDIA GPU, so I can't debug this.
Comment 10 Philippe Normand 2023-08-19 03:43:20 PDT
*** Bug 260422 has been marked as a duplicate of this bug. ***
Comment 11 Kdwk 2023-08-20 23:47:28 PDT
Created attachment 467361 [details]
Screenshot showing GPU video decode usage on Nvidia GPUs

I originally filed 260422 based on CPU usage, but now I have found an app that actually reports video decode usage of the GPU, and I am pleased to say that on my machine video decode using nvdec elements work correctly after max-ranking nvdec elements
Comment 12 Philippe Normand 2024-04-07 12:14:50 PDT
(In reply to Kdwk from comment #11)
> Created attachment 467361 [details]
> Screenshot showing GPU video decode usage on Nvidia GPUs
> 
> I originally filed 260422 based on CPU usage, but now I have found an app
> that actually reports video decode usage of the GPU, and I am pleased to say
> that on my machine video decode using nvdec elements work correctly after
> max-ranking nvdec elements

So can this bug be closed?
Comment 13 Philippe Normand 2024-05-16 12:06:29 PDT
Closing.