Bug 218934

Summary: [GPU Process] Share the NativeImage with GPU Process through a ShareableBitmap
Product: WebKit Reporter: Said Abou-Hallawa <sabouhallawa>
Component: CanvasAssignee: Said Abou-Hallawa <sabouhallawa>
Status: RESOLVED FIXED    
Severity: Normal CC: annulen, dino, ews-watchlist, gyuyoung.kim, ryuan.choi, sergio, simon.fraser, thorton, webkit-bug-importer, wenson_hsieh
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Bug Depends on:    
Bug Blocks: 217342    
Attachments:
Description Flags
Patch
none
Patch
none
Patch
none
Patch
ews-feeder: commit-queue-
Patch
thorton: review+
Patch none

Said Abou-Hallawa
Reported 2020-11-13 20:33:49 PST
We need to replace the decoded CGImageRef with a ShareableBitmap which can be shared and used by Web Process and GPU Process. A CGImageRef can be created from ShareableBitmap when drawing the image is needed.
Attachments
Patch (31.88 KB, patch)
2020-11-13 20:36 PST, Said Abou-Hallawa
no flags
Patch (40.59 KB, patch)
2020-11-13 22:57 PST, Said Abou-Hallawa
no flags
Patch (37.61 KB, patch)
2020-11-20 14:33 PST, Said Abou-Hallawa
no flags
Patch (25.85 KB, patch)
2020-12-03 21:10 PST, Said Abou-Hallawa
ews-feeder: commit-queue-
Patch (26.44 KB, patch)
2020-12-03 21:14 PST, Said Abou-Hallawa
thorton: review+
Patch (26.43 KB, patch)
2020-12-04 10:54 PST, Said Abou-Hallawa
no flags
Said Abou-Hallawa
Comment 1 2020-11-13 20:36:10 PST
Said Abou-Hallawa
Comment 2 2020-11-13 22:57:32 PST
Simon Fraser (smfr)
Comment 3 2020-11-18 13:17:31 PST
Comment on attachment 414125 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=414125&action=review > Source/WebCore/platform/graphics/NativeImage.cpp:67 > + m_observer = &observer; > + m_platformImage = nullptr; It's weird (and very non-obvious from the API) that seeing an Observer clears the platform image. > Source/WebCore/platform/graphics/NativeImage.cpp:81 > + ASSERT(m_observer); > + m_platformImage = m_observer->platformImageOfNativeImage(m_renderingResourceIdentifier); But now we have both an observer AND a native image! > Source/WebKit/GPUProcess/graphics/RemoteResourceCache.cpp:49 > + m_nativeImages.add(renderingResourceIdentifier, NativeImage::create(*this, renderingResourceIdentifier)); Why don't we just create the PlatformImage here and pass it into the NativeImage constructor? > Source/WebKit/GPUProcess/graphics/RemoteResourceCache.h:51 > + WebCore::PlatformImagePtr platformImageOfNativeImage(WebCore::RenderingResourceIdentifier) override; platformImageForIdentifier()
Said Abou-Hallawa
Comment 4 2020-11-20 14:33:41 PST
Said Abou-Hallawa
Comment 5 2020-11-20 14:54:31 PST
Comment on attachment 414125 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=414125&action=review >> Source/WebCore/platform/graphics/NativeImage.cpp:67 >> + m_platformImage = nullptr; > > It's weird (and very non-obvious from the API) that seeing an Observer clears the platform image. I renamed NativeImage::Observer to be NativeImage::ImageSource. So setting the ImageSource of the NativeImage should clear the platform image since it will provide it when it is needed. >> Source/WebCore/platform/graphics/NativeImage.cpp:81 >> + m_platformImage = m_observer->platformImageOfNativeImage(m_renderingResourceIdentifier); > > But now we have both an observer AND a native image! Yes I did not want to break the non GPUP scenario if the same NativeImage is drawn later in WebP. This can be revisited later when all the NativeImages are drawn either in GPUP or WebP but not both. >> Source/WebKit/GPUProcess/graphics/RemoteResourceCache.cpp:49 >> + m_nativeImages.add(renderingResourceIdentifier, NativeImage::create(*this, renderingResourceIdentifier)); > > Why don't we just create the PlatformImage here and pass it into the NativeImage constructor? Done. In fact this function RemoteResourceCache::cacheNativeImage() now takes a NativeImage. This cleans the interface and makes caching the NativeImage look very similar to caching the ImageBuffer. >> Source/WebKit/GPUProcess/graphics/RemoteResourceCache.h:51 >> + WebCore::PlatformImagePtr platformImageOfNativeImage(WebCore::RenderingResourceIdentifier) override; > > platformImageForIdentifier() Name was changed.
Radar WebKit Bug Importer
Comment 6 2020-11-20 20:34:13 PST
Said Abou-Hallawa
Comment 7 2020-12-03 21:10:44 PST
Said Abou-Hallawa
Comment 8 2020-12-03 21:14:13 PST
Tim Horton
Comment 9 2020-12-03 21:56:49 PST
Comment on attachment 415384 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=415384&action=review > Source/WebKit/Shared/ShareableBitmap.h:133 > + WebCore::PlatformImagePtr createPlatfromImage() { return makeCGImageCopy(); } Typo: platfrom. it's in a few places > Source/WebKit/Shared/WebCoreArgumentCoders.cpp:-1103 > -void ArgumentCoder<Ref<NativeImage>>::encode(Encoder& encoder, const Ref<NativeImage>& image) I'm shocked this wasn't used for anything else, I thought for sure it predated this project.
Said Abou-Hallawa
Comment 10 2020-12-04 10:54:56 PST
EWS
Comment 11 2020-12-04 11:48:22 PST
Committed r270444: <https://trac.webkit.org/changeset/270444> All reviewed patches have been landed. Closing bug and clearing flags on attachment 415431 [details].
Note You need to log in before you can comment on or make changes to this bug.