Bug 206514 - [GTK] WebDriver: screenshot ignores CSS transformations or WebGL content.
Summary: [GTK] WebDriver: screenshot ignores CSS transformations or WebGL content.
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebDriver (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2020-01-20 14:49 PST by Carlos Alberto Lopez Perez
Modified: 2022-09-20 22:53 PDT (History)
6 users (show)

See Also:


Attachments
python script to reproduce the issue with selenium (993 bytes, text/plain)
2020-01-20 14:49 PST, Carlos Alberto Lopez Perez
no flags Details
Patch (30.21 KB, patch)
2020-01-23 08:04 PST, Carlos Garcia Campos
no flags Details | Formatted Diff | Diff
Patch (30.26 KB, patch)
2020-01-23 08:15 PST, Carlos Garcia Campos
clopez: review+
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Carlos Alberto Lopez Perez 2020-01-20 14:49:33 PST
Created attachment 388261 [details]
python script to reproduce the issue with selenium

When using the WebDriver screenshot feature to take a snapshoot of the WebView, the snapshot produced ignores CSS transformations or WebGL content

For example, taking a screenshot of any of this URLs, will produce an incorrect screenshot:

1) http://wpt.live/css/css-transforms/perspective-origin-001.html 
On the screen you only see a green square (test passes), but on the screenshot you see the red square (like if the test has failed)

2) https://webkit.org/blog-files/3d-transforms/poster-circle.html
On the screen you see a cylinder moving, but on the screenshot you see flat squares (no cylinder form)

3) https://webglsamples.org/aquarium/aquarium.html
On the screen you see an aquarium and fishes moving, but on the screenshot you don't see the aquarium or any fish (just a grey area)


This issue affects both Safari Driver and WebKitGTK WebDriver.

When using Chrome driver, the screenshot captures the expected content.

I'm attaching a simple script with python and selenium to reproduce the issue.
Comment 1 Carlos Garcia Campos 2020-01-21 00:25:23 PST
In the case of GTK, WebPage::scaledSnapshotWithOptions() doesn't work when accelerated compositing is enabled. To fix this, we would need to take the screenshot from the UI process, like we do for the layout tests.
Comment 2 Carlos Alberto Lopez Perez 2020-01-21 06:40:52 PST
Due to this bug (at least) the following WPT tests are failing on https://wpt.fyi

css/css-transforms/perspective-origin-001.html
css/css-transforms/perspective-origin-001.html
css/css-transforms/perspective-origin-002.html
css/css-transforms/perspective-origin-003.html
css/css-transforms/perspective-origin-004.html
css/css-transforms/perspective-origin-005.html
css/css-transforms/perspective-origin-006.html
css/css-transforms/perspective-origin-x.html
css/css-transforms/perspective-origin-xy.html
css/css-transforms/perspective-translateZ-0.html
css/css-transforms/perspective-translateZ-negative.html
css/css-transforms/perspective-translateZ-positive.html
css/css-transforms/text-perspective-001.html
css/css-transforms/transform-3d-rotateY-stair-above-001.xht
css/css-transforms/transform3d-matrix3d-005.html
css/css-transforms/transform3d-perspective-006.html
css/css-transforms/transform3d-preserve3d-011.html
css/css-transforms/transform3d-scale-004.html
css/css-transforms/transform3d-sorting-001.html
css/css-transforms/transofrmed-preserve-3d-1.html

This is because the WPT runner uses webdriver to take the screenshot of the test and its reftest in order to later compare the screenshots (png images).
The test generates a correct result, but WebDriver fails to produce a correct screenshot from it.

Example at: https://wpt.fyi/results/css/css-transforms/perspective-origin-001.html?label=experimental&label=master&product=chrome&product=firefox&product=safari&product=webkitgtk&aligned
(click on compare to see the images produced, and then on "Run in your browser on wpt.live" to see how it runs ok on the browser)
Comment 3 Carlos Garcia Campos 2020-01-23 07:51:44 PST
This requires platform specific changes, so I'll use this bug for the GTK port fix.
Comment 4 Carlos Garcia Campos 2020-01-23 08:04:27 PST
Created attachment 388544 [details]
Patch
Comment 5 EWS Watchlist 2020-01-23 08:05:11 PST
Thanks for the patch. If this patch contains new public API please make sure it follows the guidelines for new WebKit2 GTK+ API. See http://trac.webkit.org/wiki/WebKitGTK/AddingNewWebKit2API
Comment 6 Carlos Garcia Campos 2020-01-23 08:15:27 PST
Created attachment 388547 [details]
Patch
Comment 7 Carlos Alberto Lopez Perez 2020-01-28 04:56:51 PST
Comment on attachment 388547 [details]
Patch

r=me
Tested it. It fixes the issue. Thanks!
Comment 8 Carlos Garcia Campos 2020-01-29 01:52:25 PST
Committed r255343: <https://trac.webkit.org/changeset/255343>
Comment 9 Radar WebKit Bug Importer 2020-01-29 01:53:13 PST
<rdar://problem/58985212>
Comment 10 Carlos Alberto Lopez Perez 2020-02-05 08:39:22 PST
(In reply to Carlos Garcia Campos from comment #8)
> Committed r255343: <https://trac.webkit.org/changeset/255343>

New passes on wpt.fyi: https://wpt.fyi/results/css/css-transforms?diff&filter=ADC&run_id=412940006&run_id=430010015 (at least all perspective-origin* ones and likely others)