209326 – [iOS] 90+ Web Platform Tests fail because the device scale factor is not 1

Bug 209326 - [iOS] 90+ Web Platform Tests fail because the device scale factor is not 1

Summary: [iOS] 90+ Web Platform Tests fail because the device scale factor is not 1

Status:	NEW

Alias:	None

Product:	WebKit
Classification:	Unclassified
Component:	Layout and Rendering (show other bugs)
Version:	WebKit Nightly Build
Hardware:	Unspecified Unspecified

Importance:	P2 Normal
Assignee:	Tim Horton

URL:
Keywords:	InRadar

Depends on:
Blocks:

Reported:	2020-03-19 20:35 PDT by Myles C. Maxfield
Modified:	2020-03-20 18:52 PDT (History)
CC List:	9 users (show)

See Also:

Attachments
iOS screenshot of the corner of the green square inside the big image (4.12 KB, image/png) 2020-03-19 20:35 PDT, Myles C. Maxfield	no flags	Details
macOS screenshot of the corner of the green square inside the big image (4.05 KB, image/png) 2020-03-19 20:36 PDT, Myles C. Maxfield	no flags	Details
macOS screenshot of the corner of the green square inside the big image (3.96 KB, image/png) 2020-03-19 20:38 PDT, Myles C. Maxfield	no flags	Details
PoC patch (4.34 KB, patch) 2020-03-20 14:21 PDT, Myles C. Maxfield	no flags	Details \| Formatted Diff \| Diff
Show Obsolete (1) View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.

Description Myles C. Maxfield 2020-03-19 20:35:22 PDT

It looks like images are drawn into iOS as if the backing store is a retina backing store, which means images get interpolated.

This is a problem for many Web PlatformTests, like css/css-writing-modes/text-indent-vrl-016.xht, though there are many, many more (at least 90?). These tests show an image which is big and white with a precisely sized green square at some location/size, and compare it to a green div at a particular place. On macOS, the green part of the image matches the div exactly, but on iOS, the contents of the image seem to be antialiased, which means the green square is fuzzy, causing a reftest failure.

Screenshots attached.

Comment 1 Myles C. Maxfield 2020-03-19 20:35:56 PDT

Created attachment 394061 [details]
iOS screenshot of the corner of the green square inside the big image

Comment 2 Myles C. Maxfield 2020-03-19 20:36:33 PDT

Created attachment 394062 [details]
macOS screenshot of the corner of the green square inside the big image

Comment 3 Myles C. Maxfield 2020-03-19 20:38:48 PDT

Created attachment 394063 [details]
macOS screenshot of the corner of the green square inside the big image

Comment 4 Myles C. Maxfield 2020-03-19 20:39:34 PDT

For some reason the macOS image isn't as crisp as it actually is on my screen. I don't know why. It shows a perfectly hard edge on macOS.

Comment 5 Myles C. Maxfield 2020-03-19 20:40:29 PDT

(In reply to Myles C. Maxfield from comment #4)
> For some reason the macOS image isn't as crisp as it actually is on my
> screen. I don't know why. It shows a perfectly hard edge on macOS.

*the macOS image I attached to this bug

Comment 6 Myles C. Maxfield 2020-03-19 21:01:26 PDT

(lldb) p adjustedDestRect
(WebCore::FloatRect) $0 = {
  m_location = (m_x = 0, m_y = 0)
  m_size = (m_width = 160, m_height = 320)
}
(lldb) p CGContextGetCTM(context)
error: <user expression 1>:1:1: 'CGContextGetCTM' has unknown return type; cast the call to its declared return type
CGContextGetCTM(context)
^~~~~~~~~~~~~~~~~~~~~~~~
(lldb) p (CGAffineTransform)CGContextGetCTM(context)
(CGAffineTransform) $1 = (a = 2, b = 0, c = 0, d = 2, tx = 714, ty = 544)
(lldb) p (CGAffineTransform)CGContextGetBaseCTM(context)
(CGAffineTransform) $2 = (a = 1, b = 0, c = 0, d = 1, tx = 0, ty = 0)
(lldb) po subImage.get()
<CGImage 0x7f895213a2f0> (IP)
	<<CGColorSpace 0x7f8952139d00> (kCGColorSpaceICCBased; kCGColorSpaceModelRGB; sRGB IEC61966-2.1)>
		width = 160, height = 320, bpc = 8, bpp = 32, row bytes = 640 
		kCGImageAlphaLast | 0 (default byte order)  | kCGImagePixelFormatPacked 
		is mask? No, has masking color? No, has soft mask? No, has matte? No, should interpolate? Yes

Comment 7 Myles C. Maxfield 2020-03-19 21:35:49 PDT

It looks like the 2x2 scale is coming from:

    frame #3: 0x0000000103ff4231 WebKit`WebKit::RemoteLayerBackingStore::drawInContext(this=0x0000000141b5f1f0, context=0x0000000141bb52b8, backImage=0x0000000000000000) at RemoteLayerBackingStore.mm:344:13
    frame #4: 0x0000000103ff38f2 WebKit`WebKit::RemoteLayerBackingStore::display(this=0x0000000141b5f1f0) at RemoteLayerBackingStore.mm:283:13
    frame #5: 0x0000000104decf2d WebKit`WebKit::PlatformCALayerRemote::recursiveBuildTransaction(this=0x0000000141b63b70, context=0x0000000141b902f0, transaction=0x00007ffeeca34030) at PlatformCALayerRemote.cpp:180:102
    frame #6: 0x0000000104ded1d4 WebKit`WebKit::PlatformCALayerRemote::recursiveBuildTransaction(this=0x0000000141b637a0, context=0x0000000141b902f0, transaction=0x00007ffeeca34030) at PlatformCALayerRemote.cpp:202:15
    frame #7: 0x0000000104ded1d4 WebKit`WebKit::PlatformCALayerRemote::recursiveBuildTransaction(this=0x0000000141b53c00, context=0x0000000141b902f0, transaction=0x00007ffeeca34030) at PlatformCALayerRemote.cpp:202:15
    frame #8: 0x0000000104ded1d4 WebKit`WebKit::PlatformCALayerRemote::recursiveBuildTransaction(this=0x0000000141b631e8, context=0x0000000141b902f0, transaction=0x00007ffeeca34030) at PlatformCALayerRemote.cpp:202:15
    frame #9: 0x0000000104ded1d4 WebKit`WebKit::PlatformCALayerRemote::recursiveBuildTransaction(this=0x0000000141b63000, context=0x0000000141b902f0, transaction=0x00007ffeeca34030) at PlatformCALayerRemote.cpp:202:15
    frame #10: 0x00000001049977e7 WebKit`WebKit::RemoteLayerTreeContext::buildTransaction(this=0x0000000141b902f0, transaction=0x00007ffeeca34030, rootLayer=0x0000000141b63000) at RemoteLayerTreeContext.mm:158:21
    frame #11: 0x0000000103b26e48 WebKit`WebKit::RemoteLayerTreeDrawingArea::updateRendering(this=0x0000000141b65000) at RemoteLayerTreeDrawingArea.mm:363:31
...
    frame #17: 0x0000000103b2bd8e WebKit`WebCore::Timer::fired(this=0x0000000141b65080) at Timer.h:126:9

Comment 8 Myles C. Maxfield 2020-03-19 21:38:47 PDT

The drawing has this backtrace:

...
    frame #30: 0x000000043cb4dc84 WebCore`WebCore::RenderLayer::paintLayerContents(this=0x000000045e15e5c0, context=0x000000045e1bdae0, paintingInfo=0x00007ffeecba0010, paintFlags=(m_storage = 96)) at RenderLayer.cpp:4716:13
    frame #31: 0x000000043cb91529 WebCore`WebCore::RenderLayerBacking::paintIntoLayer(this=0x00007ffeecba00f8, layer=0x000000045e15e5c0, paintFlags=(m_storage = 96))::$_5::operator()(WebCore::RenderLayer&, WTF::OptionSet<WebCore::RenderLayer::PaintLayerFlag>) const at RenderLayerBacking.cpp:2910:19
    frame #32: 0x000000043cb90ddf WebCore`WebCore::RenderLayerBacking::paintIntoLayer(this=0x000000045e1f1540, graphicsLayer=0x000000045e159000, context=0x000000045e1bdae0, paintDirtyRect=0x00007ffeecba02e0, paintBehavior=(m_storage = 0), eventRegionContext=0x0000000000000000) at RenderLayerBacking.cpp:2927:5
    frame #33: 0x000000043cb927df WebCore`WebCore::RenderLayerBacking::paintContents(this=0x000000045e1f1540, graphicsLayer=0x000000045e159000, context=0x000000045e1bdae0, clip=0x00007ffeecba0360, layerPaintBehavior=0) at RenderLayerBacking.cpp:3135:9
    frame #34: 0x000000043c63e064 WebCore`WebCore::GraphicsLayer::paintGraphicsLayerContents(this=0x000000045e159000, context=0x000000045e1bdae0, clip=0x00007ffeecba0708, layerPaintBehavior=0) at GraphicsLayer.cpp:516:14
    frame #35: 0x000000043c6bf495 WebCore`WebCore::GraphicsLayerCA::platformCALayerPaintContents(this=0x000000045e159000, (null)=0x000000045e15b200, context=0x000000045e1bdae0, clip=0x00007ffeecba0708, layerPaintBehavior=0) at GraphicsLayerCA.cpp:1717:5
    frame #36: 0x000000043a862df2 WebCore`WebCore::PlatformCALayer::drawLayerContents(graphicsContext=0x000000045e1bdae0, platformCALayer=0x000000045e15b200, dirtyRects=0x00007ffeecba06f8, layerPaintBehavior=0) at PlatformCALayerCocoa.mm:1217:28
    frame #37: 0x000000043c6dd52b WebCore`WebCore::TileGrid::platformCALayerPaintContents(this=0x000000046a9df000, platformCALayer=0x000000045e1dfb70, context=0x000000045e1bdae0, (null)=0x00007ffeecba0918, layerPaintBehavior=0) at TileGrid.cpp:733:9
    frame #38: 0x00000004208633b1 WebKit`WebKit::RemoteLayerBackingStore::drawInContext(this=0x000000045e1678b8, context=0x000000045e1bdae0, backImage=0x0000000000000000) at RemoteLayerBackingStore.mm:353:27
    frame #39: 0x00000004208628f2 WebKit`WebKit::RemoteLayerBackingStore::display(this=0x000000045e1678b8) at RemoteLayerBackingStore.mm:283:13
    frame #40: 0x000000042165bf2d WebKit`WebKit::PlatformCALayerRemote::recursiveBuildTransaction(this=0x000000045e1dfb70, context=0x000000045e1982f0, transaction=0x00007ffeecba14c0) at PlatformCALayerRemote.cpp:180:102
    frame #41: 0x000000042165c1d4 WebKit`WebKit::PlatformCALayerRemote::recursiveBuildTransaction(this=0x000000045e1df7a0, context=0x000000045e1982f0, transaction=0x00007ffeecba14c0) at PlatformCALayerRemote.cpp:202:15
    frame #42: 0x000000042165c1d4 WebKit`WebKit::PlatformCALayerRemote::recursiveBuildTransaction(this=0x000000045e15b200, context=0x000000045e1982f0, transaction=0x00007ffeecba14c0) at PlatformCALayerRemote.cpp:202:15
    frame #43: 0x000000042165c1d4 WebKit`WebKit::PlatformCALayerRemote::recursiveBuildTransaction(this=0x000000045e1df5b8, context=0x000000045e1982f0, transaction=0x00007ffeecba14c0) at PlatformCALayerRemote.cpp:202:15
    frame #44: 0x000000042165c1d4 WebKit`WebKit::PlatformCALayerRemote::recursiveBuildTransaction(this=0x000000045e16b000, context=0x000000045e1982f0, transaction=0x00007ffeecba14c0) at PlatformCALayerRemote.cpp:202:15
    frame #45: 0x00000004212067e7 WebKit`WebKit::RemoteLayerTreeContext::buildTransaction(this=0x000000045e1982f0, transaction=0x00007ffeecba14c0, rootLayer=0x000000045e16b000) at RemoteLayerTreeContext.mm:158:21
    frame #46: 0x0000000420395e48 WebKit`WebKit::RemoteLayerTreeDrawingArea::updateRendering(this=0x000000045e16d000) at RemoteLayerTreeDrawingArea.mm:363:31
...

Comment 9 Myles C. Maxfield 2020-03-19 21:41:27 PDT

So it looks like the source of the "2" is RemoteLayerBackingStore::m_scale.

Comment 10 Myles C. Maxfield 2020-03-19 22:25:06 PDT

Looks like this is getting set from WebCore::Page::m_deviceScaleFactor

Comment 11 Tim Horton 2020-03-19 23:24:01 PDT

We generally try to run tests at 1x, but we don't have support on iOS:

void PlatformWebView::changeWindowScaleIfNeeded(float)
{
    // Retina only surface.
}

Comment 12 Tim Horton 2020-03-19 23:27:38 PDT

See also this code in TestController.cpp:

// Generally, the tests should default to running at 1x. updateWindowScaleForTest() will adjust the scale to
// something else for specific tests that need to run at a different window scale.
m_mainWebView->changeWindowScaleIfNeeded(1);

Comment 13 Simon Fraser (smfr) 2020-03-20 08:22:52 PDT

Also, in GraphicsContext::drawNativeImage():

#if PLATFORM(IOS_FAMILY)
    bool wasAntialiased = CGContextGetShouldAntialias(context);
    // Anti-aliasing is on by default on the iPhone. Need to turn it off when drawing images.
    CGContextSetShouldAntialias(context, false);

    // Align to pixel boundaries
    adjustedDestRect = roundToDevicePixels(adjustedDestRect);
#endif

Comment 14 Myles C. Maxfield 2020-03-20 10:21:19 PDT

_page->setIntrinsicDeviceScaleFactor(WebCore::screenScaleFactor([UIScreen mainScreen]));

Comment 15 Myles C. Maxfield 2020-03-20 14:21:54 PDT

Created attachment 394126 [details]
PoC patch

If you apply the attached patch, at least one of the failing WPT tests starts passing. I have no idea, though, whether this is a good patch, or an incomplete one.

Comment 16 Myles C. Maxfield 2020-03-20 14:31:12 PDT

These are the tests this bug should fix:

imported/w3c/web-platform-tests/css/css-writing-modes/text-indent-vrl-016.xht
imported/w3c/web-platform-tests/css/css-writing-modes/text-indent-vrl-014.xht
imported/w3c/web-platform-tests/css/css-writing-modes/text-indent-vrl-012.xht
imported/w3c/web-platform-tests/css/css-writing-modes/text-indent-vrl-010.xht
imported/w3c/web-platform-tests/css/css-writing-modes/text-indent-vlr-017.xht
imported/w3c/web-platform-tests/css/css-writing-modes/text-indent-vlr-015.xht
imported/w3c/web-platform-tests/css/css-writing-modes/text-indent-vlr-013.xht
imported/w3c/web-platform-tests/css/css-writing-modes/text-indent-vlr-011.xht
imported/w3c/web-platform-tests/css/css-writing-modes/percent-padding-vrl-004.xht
imported/w3c/web-platform-tests/css/css-writing-modes/percent-padding-vrl-002.xht
imported/w3c/web-platform-tests/css/css-writing-modes/percent-padding-vlr-005.xht
imported/w3c/web-platform-tests/css/css-writing-modes/percent-padding-vlr-003.xht
imported/w3c/web-platform-tests/css/css-writing-modes/padding-vrl-004.xht
imported/w3c/web-platform-tests/css/css-writing-modes/padding-vlr-005.xht
imported/w3c/web-platform-tests/css/css-writing-modes/overconstrained-rel-pos-ltr-top-bottom-vlr-003.xht
imported/w3c/web-platform-tests/css/css-writing-modes/overconstrained-rel-pos-ltr-left-right-vlr-005.xht
imported/w3c/web-platform-tests/css/css-writing-modes/normal-flow-overconstrained-vlr-003.xht
imported/w3c/web-platform-tests/css/css-writing-modes/clip-rect-vrl-016.xht
imported/w3c/web-platform-tests/css/css-writing-modes/clip-rect-vrl-014.xht
imported/w3c/web-platform-tests/css/css-writing-modes/clip-rect-vrl-012.xht
imported/w3c/web-platform-tests/css/css-writing-modes/clip-rect-vrl-010.xht
imported/w3c/web-platform-tests/css/css-writing-modes/clip-rect-vlr-017.xht
imported/w3c/web-platform-tests/css/css-writing-modes/clip-rect-vlr-015.xht
imported/w3c/web-platform-tests/css/css-writing-modes/clip-rect-vlr-013.xht
imported/w3c/web-platform-tests/css/css-writing-modes/clip-rect-vlr-011.xht
imported/w3c/web-platform-tests/css/css-writing-modes/box-offsets-rel-pos-vrl-004.xht
imported/w3c/web-platform-tests/css/css-writing-modes/box-offsets-rel-pos-vlr-005.xht
imported/w3c/web-platform-tests/css/css-writing-modes/border-vrl-006.xht
imported/w3c/web-platform-tests/css/css-writing-modes/border-vlr-007.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-226.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-218.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-210.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-202.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-194.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-186.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-178.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-170.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-162.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-154.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-146.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-138.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-130.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-122.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-114.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-106.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-096.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-090.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-084.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-078.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-072.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-066.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-060.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-054.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-048.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-042.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-036.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-030.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-024.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-018.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-012.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vrl-006.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-227.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-219.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-211.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-203.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-195.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-187.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-179.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-171.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-163.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-155.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-147.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-139.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-131.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-123.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-115.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-107.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-097.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-091.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-085.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-079.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-073.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-067.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-061.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-055.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-049.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-043.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-037.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-031.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-025.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-019.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-013.xht
imported/w3c/web-platform-tests/css/css-writing-modes/abs-pos-non-replaced-vlr-007.xht

Comment 17 Radar WebKit Bug Importer 2020-03-20 14:31:47 PDT

<rdar://problem/60704836>