Bug 239527 - [GPU Process] Make WebImage be backed by ImageBuffer
Summary: [GPU Process] Make WebImage be backed by ImageBuffer
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Layout and Rendering (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Said Abou-Hallawa
URL:
Keywords: InRadar
Depends on:
Blocks: 239615
  Show dependency treegraph
 
Reported: 2022-04-19 19:20 PDT by Said Abou-Hallawa
Modified: 2022-05-04 22:38 PDT (History)
5 users (show)

See Also:


Attachments
Patch (64.58 KB, patch)
2022-04-19 19:25 PDT, Said Abou-Hallawa
no flags Details | Formatted Diff | Diff
Patch (66.05 KB, patch)
2022-04-19 23:44 PDT, Said Abou-Hallawa
no flags Details | Formatted Diff | Diff
Patch (66.59 KB, patch)
2022-04-20 13:05 PDT, Said Abou-Hallawa
ews-feeder: commit-queue-
Details | Formatted Diff | Diff
Patch (90.99 KB, patch)
2022-04-20 14:11 PDT, Said Abou-Hallawa
no flags Details | Formatted Diff | Diff
Patch (95.47 KB, patch)
2022-04-21 22:41 PDT, Said Abou-Hallawa
no flags Details | Formatted Diff | Diff
Patch (99.74 KB, patch)
2022-04-24 22:03 PDT, Said Abou-Hallawa
ews-feeder: commit-queue-
Details | Formatted Diff | Diff
Patch (99.90 KB, patch)
2022-04-24 22:47 PDT, Said Abou-Hallawa
ews-feeder: commit-queue-
Details | Formatted Diff | Diff
Patch (99.91 KB, patch)
2022-04-24 22:57 PDT, Said Abou-Hallawa
no flags Details | Formatted Diff | Diff
Patch (99.94 KB, patch)
2022-04-25 22:24 PDT, Said Abou-Hallawa
ews-feeder: commit-queue-
Details | Formatted Diff | Diff
Patch (99.96 KB, patch)
2022-04-25 22:55 PDT, Said Abou-Hallawa
no flags Details | Formatted Diff | Diff
Patch (102.58 KB, patch)
2022-04-26 01:22 PDT, Said Abou-Hallawa
no flags Details | Formatted Diff | Diff
Patch (103.55 KB, patch)
2022-04-27 16:06 PDT, Said Abou-Hallawa
thorton: review+
Details | Formatted Diff | Diff
Patch (104.82 KB, patch)
2022-04-27 18:13 PDT, Said Abou-Hallawa
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Said Abou-Hallawa 2022-04-19 19:20:27 PDT
WebImage is used to take page snapshots. Currently a snapshot can be drawn in WebProcess and used by UIProcess. With enabling GPUProcess for rendering, the snapshot will be recorded in WebProcess, drawn in GPUProcess and used in UIProcess. This will require replacing the ShareableBitmap in WebImage by ImageBuffer.
Comment 1 Said Abou-Hallawa 2022-04-19 19:20:56 PDT
rdar://91113628
Comment 2 Said Abou-Hallawa 2022-04-19 19:25:52 PDT
Created attachment 457949 [details]
Patch
Comment 3 Said Abou-Hallawa 2022-04-19 23:44:38 PDT
Created attachment 457963 [details]
Patch
Comment 4 Said Abou-Hallawa 2022-04-20 13:05:47 PDT
Created attachment 458006 [details]
Patch
Comment 5 Said Abou-Hallawa 2022-04-20 14:11:05 PDT
Created attachment 458014 [details]
Patch
Comment 6 Said Abou-Hallawa 2022-04-21 22:41:32 PDT
Created attachment 458114 [details]
Patch
Comment 7 Said Abou-Hallawa 2022-04-24 22:03:24 PDT
Created attachment 458240 [details]
Patch
Comment 8 Said Abou-Hallawa 2022-04-24 22:47:58 PDT
Created attachment 458243 [details]
Patch
Comment 9 Said Abou-Hallawa 2022-04-24 22:57:21 PDT
Created attachment 458244 [details]
Patch
Comment 10 Said Abou-Hallawa 2022-04-25 22:24:55 PDT
Created attachment 458331 [details]
Patch
Comment 11 Said Abou-Hallawa 2022-04-25 22:55:42 PDT
Created attachment 458333 [details]
Patch
Comment 12 Said Abou-Hallawa 2022-04-26 01:22:47 PDT
Created attachment 458338 [details]
Patch
Comment 13 Said Abou-Hallawa 2022-04-27 16:06:16 PDT
Created attachment 458477 [details]
Patch
Comment 14 Said Abou-Hallawa 2022-04-27 16:09:15 PDT
(In reply to Said Abou-Hallawa from comment #13)
> Created attachment 458477 [details]
> Patch

This patch fixes the failures in the api-mac tests. The change in WebChromeClient::createImageBuffer() fixes the timeout in two api tests.
Comment 15 Tim Horton 2022-04-27 17:38:56 PDT
Comment on attachment 458477 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=458477&action=review

> Source/WebCore/page/FrameSnapshotting.cpp:84
> +    auto& document = *frame.document();

Should this be a strong reference?

> Source/WebCore/platform/graphics/ConcreteImageBuffer.h:296
> +        ref(); // Balanced by deref below.

Could this be a RefPtr that gets captured by the block, instead of this raw ref/deref?

> Source/WebKit/Shared/UserData.cpp:402
> +        result = WebImage::create(*paramters, WTFMove(handle));

Spelling: "paramters"

> Source/WebKit/Shared/WebImage.cpp:38
> +    // Create shareable RemoteImageBuffer if GPU Process is enabled.

I kind of see the need for these comments, but they feel indicative of how weird some of this code has gotten :)

I don't think they're quite worded correctly, though, because e.g. this code has NO IDEA why ChromeClient might or might not give you a imagebuffer (doesn't know that it's about GPUP), and would get stale if that reason changed. So maybe we leave them out? Or word them differently ("allow the chrome client to override the image if it wants to" etc. etc. but then it's fairly obvious from reading the code).

> Source/WebKit/Shared/WebImage.h:48
> +    static RefPtr<WebImage> create(const WebCore::IntSize&, ImageOptions, const WebCore::DestinationColorSpace& = WebCore::DestinationColorSpace::SRGB(), WebCore::ChromeClient* = nullptr);

Wasn't Sam pushing us in the direction of "never default to SRGB, always make clients say it, so you can fix them one-by-one"?
Comment 16 Said Abou-Hallawa 2022-04-27 18:13:09 PDT
Created attachment 458484 [details]
Patch
Comment 17 Said Abou-Hallawa 2022-04-27 18:20:28 PDT
Comment on attachment 458477 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=458477&action=review

>> Source/WebCore/page/FrameSnapshotting.cpp:84
>> +    auto& document = *frame.document();
> 
> Should this be a strong reference?

Fixed. This was changed to
    Ref document = *frame.document();

>> Source/WebCore/platform/graphics/ConcreteImageBuffer.h:296
>> +        ref(); // Balanced by deref below.
> 
> Could this be a RefPtr that gets captured by the block, instead of this raw ref/deref?

Unfortunately we can't do that. A lambda can be used as a pointer to function as long as it doesn't capture variables.

>> Source/WebKit/Shared/UserData.cpp:402
>> +        result = WebImage::create(*paramters, WTFMove(handle));
> 
> Spelling: "paramters"

Fixed.

>> Source/WebKit/Shared/WebImage.cpp:38
>> +    // Create shareable RemoteImageBuffer if GPU Process is enabled.
> 
> I kind of see the need for these comments, but they feel indicative of how weird some of this code has gotten :)
> 
> I don't think they're quite worded correctly, though, because e.g. this code has NO IDEA why ChromeClient might or might not give you a imagebuffer (doesn't know that it's about GPUP), and would get stale if that reason changed. So maybe we leave them out? Or word them differently ("allow the chrome client to override the image if it wants to" etc. etc. but then it's fairly obvious from reading the code).

I agree. Comments were removed.

>> Source/WebKit/Shared/WebImage.h:48
>> +    static RefPtr<WebImage> create(const WebCore::IntSize&, ImageOptions, const WebCore::DestinationColorSpace& = WebCore::DestinationColorSpace::SRGB(), WebCore::ChromeClient* = nullptr);
> 
> Wasn't Sam pushing us in the direction of "never default to SRGB, always make clients say it, so you can fix them one-by-one"?

Fixed. Default color space was removed.
Comment 18 EWS 2022-04-28 01:22:28 PDT
Committed r293570 (250084@main): <https://commits.webkit.org/250084@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 458484 [details].