Bug 240113

Summary: Allow MediaPlayerPrivateMediaStreamAVFObjC to avoid pixel conformers if IOSurfaces are not allowed
Product: WebKit Reporter: youenn fablet <youennf>
Component: WebRTCAssignee: youenn fablet <youennf>
Status: RESOLVED FIXED    
Severity: Normal CC: eric.carlson, ews-watchlist, glenn, jer.noble, philipj, sergio, simon.fraser, webkit-bug-importer, youennf, youssefdevelops, y_soliman
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
Patch
ews-feeder: commit-queue-
Patch
ews-feeder: commit-queue-
Patch
none
Patch
none
Patch for landing
ews-feeder: commit-queue-
Patch for landing
none
Patch for landing none

Description youenn fablet 2022-05-05 01:32:13 PDT
Allow MediaPlayerPrivateMediaStreamAVFObjC to avoid pixel conformers if IOSurfaces are not allowed
Comment 1 youenn fablet 2022-05-05 02:06:06 PDT
<rdar://92629845>
Comment 2 youenn fablet 2022-05-05 02:09:21 PDT
Created attachment 458862 [details]
Patch
Comment 3 youenn fablet 2022-05-05 02:42:28 PDT
Created attachment 458863 [details]
Patch
Comment 4 youenn fablet 2022-05-05 02:52:44 PDT
Created attachment 458864 [details]
Patch
Comment 5 youenn fablet 2022-05-05 04:23:52 PDT
Created attachment 458865 [details]
Patch
Comment 6 Eric Carlson 2022-05-05 05:42:43 PDT
Comment on attachment 458865 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=458865&action=review

r=me once the bots are happy

> Source/WebKit/GPUProcess/media/RemoteVideoFrameObjectHeap.cpp:142
> +        m_connection->send(Messages::RemoteVideoFrameObjectHeapProxyProcessor::NewConvertedVideoFrameBuffer { { } }, 0);

It looks like SharedVideoFrameReader doesn't log errors, so it might be worth logging an error here to help debug unexpected failures.

> Source/WebKit/GPUProcess/media/RemoteVideoFrameObjectHeap.cpp:149
> +        m_connection->send(Messages::RemoteVideoFrameObjectHeapProxyProcessor::NewConvertedVideoFrameBuffer { { } }, 0);

Ditto, PixelBufferConformerCV also doesn't log failures.

> Source/WebKit/GPUProcess/media/RemoteVideoFrameObjectHeap.mm:32
> +

Nit: unneeded white space
Comment 7 youenn fablet 2022-05-05 05:51:52 PDT
> It looks like SharedVideoFrameReader doesn't log errors, so it might be
> worth logging an error here to help debug unexpected failures.

Let's fix this in https://bugs.webkit.org/show_bug.cgi?id=236066, I just uploaded a patch.
Comment 8 youenn fablet 2022-05-05 05:53:17 PDT
> > Source/WebKit/GPUProcess/media/RemoteVideoFrameObjectHeap.cpp:149
> > +        m_connection->send(Messages::RemoteVideoFrameObjectHeapProxyProcessor::NewConvertedVideoFrameBuffer { { } }, 0);
> 
> Ditto, PixelBufferConformerCV also doesn't log failures.

Will add logging for this one.
Comment 9 Simon Fraser (smfr) 2022-05-05 08:47:48 PDT
Comment on attachment 458865 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=458865&action=review

> Source/WebKit/WebProcess/GPU/webrtc/RemoteVideoFrameObjectHeapProxyProcessor.cpp:145
> +    connection.send(Messages::RemoteVideoFrameObjectHeap::ConvertBuffer { WTFMove(*buffer) }, 0);
> +    m_conversionSemaphore.wait();

Why this and not just a sync message? Sync IPC has logic to avoid deadlocking, but this doesn't.
Comment 10 youenn fablet 2022-05-05 09:34:07 PDT
Created attachment 458882 [details]
Patch for landing
Comment 11 youenn fablet 2022-05-05 10:36:58 PDT
Created attachment 458888 [details]
Patch for landing
Comment 12 youenn fablet 2022-05-05 11:22:57 PDT
(In reply to Simon Fraser (smfr) from comment #9)
> Comment on attachment 458865 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=458865&action=review
> 
> > Source/WebKit/WebProcess/GPU/webrtc/RemoteVideoFrameObjectHeapProxyProcessor.cpp:145
> > +    connection.send(Messages::RemoteVideoFrameObjectHeap::ConvertBuffer { WTFMove(*buffer) }, 0);
> > +    m_conversionSemaphore.wait();
> 
> Why this and not just a sync message? Sync IPC has logic to avoid
> deadlocking, but this doesn't.

We send a message from main thread but we receive the answer from a work queue.
Comment 13 youenn fablet 2022-05-06 02:10:46 PDT
Created attachment 458941 [details]
Patch for landing
Comment 14 youenn fablet 2022-05-06 02:31:22 PDT
(In reply to youenn fablet from comment #12)
> (In reply to Simon Fraser (smfr) from comment #9)
> > Comment on attachment 458865 [details]
> > Patch
> > 
> > View in context:
> > https://bugs.webkit.org/attachment.cgi?id=458865&action=review
> > 
> > > Source/WebKit/WebProcess/GPU/webrtc/RemoteVideoFrameObjectHeapProxyProcessor.cpp:145
> > > +    connection.send(Messages::RemoteVideoFrameObjectHeap::ConvertBuffer { WTFMove(*buffer) }, 0);
> > > +    m_conversionSemaphore.wait();
> > 
> > Why this and not just a sync message? Sync IPC has logic to avoid
> > deadlocking, but this doesn't.
> 
> We send a message from main thread but we receive the answer from a work
> queue.

Thinking more about it, this will create a deadlock in case of GPUProcess crash, so we might need to use sync IPC, though I am unsure how sync IPC and work queue message receivers do things consistently.
I'll file a bug to keep track of this.
Comment 15 youenn fablet 2022-05-06 02:32:24 PDT
I filed https://bugs.webkit.org/show_bug.cgi?id=240161.
Comment 16 EWS 2022-05-06 02:53:49 PDT
Committed r293886 (250344@main): <https://commits.webkit.org/250344@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 458941 [details].