Bug 206514

Summary:

[GTK] WebDriver: screenshot ignores CSS transformations or WebGL content.

Product:

WebKit

Reporter:

Carlos Alberto Lopez Perez <clopez>

Component:

WebDriver

Assignee:

Nobody <webkit-unassigned>

Status:

RESOLVED FIXED

Severity:

Normal

CC:

bburg, berto, cgarcia, ews-watchlist, gustavo, webkit-bug-importer

Priority:

Keywords:

InRadar

Version:

WebKit Nightly Build

Hardware:

Unspecified

OS:

Unspecified

See Also:

https://bugs.webkit.org/show_bug.cgi?id=207266
https://bugs.webkit.org/show_bug.cgi?id=242214
https://bugs.webkit.org/show_bug.cgi?id=242215
https://bugs.webkit.org/show_bug.cgi?id=245119

Attachments:

Description	Flags
python script to reproduce the issue with selenium	none
Patch	none
Patch	clopez: review+

Description Carlos Alberto Lopez Perez 2020-01-20 14:49:33 PST

Created attachment 388261 [details]
python script to reproduce the issue with selenium

When using the WebDriver screenshot feature to take a snapshoot of the WebView, the snapshot produced ignores CSS transformations or WebGL content

For example, taking a screenshot of any of this URLs, will produce an incorrect screenshot:

1) http://wpt.live/css/css-transforms/perspective-origin-001.html 
On the screen you only see a green square (test passes), but on the screenshot you see the red square (like if the test has failed)

2) https://webkit.org/blog-files/3d-transforms/poster-circle.html
On the screen you see a cylinder moving, but on the screenshot you see flat squares (no cylinder form)

3) https://webglsamples.org/aquarium/aquarium.html
On the screen you see an aquarium and fishes moving, but on the screenshot you don't see the aquarium or any fish (just a grey area)


This issue affects both Safari Driver and WebKitGTK WebDriver.

When using Chrome driver, the screenshot captures the expected content.

I'm attaching a simple script with python and selenium to reproduce the issue.

Comment 1 Carlos Garcia Campos 2020-01-21 00:25:23 PST

In the case of GTK, WebPage::scaledSnapshotWithOptions() doesn't work when accelerated compositing is enabled. To fix this, we would need to take the screenshot from the UI process, like we do for the layout tests.

Comment 2 Carlos Alberto Lopez Perez 2020-01-21 06:40:52 PST

Due to this bug (at least) the following WPT tests are failing on https://wpt.fyi

css/css-transforms/perspective-origin-001.html
css/css-transforms/perspective-origin-001.html
css/css-transforms/perspective-origin-002.html
css/css-transforms/perspective-origin-003.html
css/css-transforms/perspective-origin-004.html
css/css-transforms/perspective-origin-005.html
css/css-transforms/perspective-origin-006.html
css/css-transforms/perspective-origin-x.html
css/css-transforms/perspective-origin-xy.html
css/css-transforms/perspective-translateZ-0.html
css/css-transforms/perspective-translateZ-negative.html
css/css-transforms/perspective-translateZ-positive.html
css/css-transforms/text-perspective-001.html
css/css-transforms/transform-3d-rotateY-stair-above-001.xht
css/css-transforms/transform3d-matrix3d-005.html
css/css-transforms/transform3d-perspective-006.html
css/css-transforms/transform3d-preserve3d-011.html
css/css-transforms/transform3d-scale-004.html
css/css-transforms/transform3d-sorting-001.html
css/css-transforms/transofrmed-preserve-3d-1.html

This is because the WPT runner uses webdriver to take the screenshot of the test and its reftest in order to later compare the screenshots (png images).
The test generates a correct result, but WebDriver fails to produce a correct screenshot from it.

Example at: https://wpt.fyi/results/css/css-transforms/perspective-origin-001.html?label=experimental&label=master&product=chrome&product=firefox&product=safari&product=webkitgtk&aligned
(click on compare to see the images produced, and then on "Run in your browser on wpt.live" to see how it runs ok on the browser)

Comment 3 Carlos Garcia Campos 2020-01-23 07:51:44 PST

This requires platform specific changes, so I'll use this bug for the GTK port fix.

Comment 4 Carlos Garcia Campos 2020-01-23 08:04:27 PST

Created attachment 388544 [details]
Patch

Comment 5 EWS Watchlist 2020-01-23 08:05:11 PST

Thanks for the patch. If this patch contains new public API please make sure it follows the guidelines for new WebKit2 GTK+ API. See http://trac.webkit.org/wiki/WebKitGTK/AddingNewWebKit2API

Comment 6 Carlos Garcia Campos 2020-01-23 08:15:27 PST

Created attachment 388547 [details]
Patch

Comment 7 Carlos Alberto Lopez Perez 2020-01-28 04:56:51 PST

Comment on attachment 388547 [details]
Patch

r=me
Tested it. It fixes the issue. Thanks!

Comment 8 Carlos Garcia Campos 2020-01-29 01:52:25 PST

Committed r255343: <https://trac.webkit.org/changeset/255343>

Comment 9 Radar WebKit Bug Importer 2020-01-29 01:53:13 PST

<rdar://problem/58985212>

Comment 10 Carlos Alberto Lopez Perez 2020-02-05 08:39:22 PST

(In reply to Carlos Garcia Campos from comment #8)
> Committed r255343: <https://trac.webkit.org/changeset/255343>

New passes on wpt.fyi: https://wpt.fyi/results/css/css-transforms?diff&filter=ADC&run_id=412940006&run_id=430010015 (at least all perspective-origin* ones and likely others)