Bug 236699

Summary: REGRESSION (iOS 15.4b3): Catastrophic (20x) performance regression in texImage2D from mp4 video
Product: WebKit Reporter: Simon Taylor <simontaylor1>
Component: WebGLAssignee: Kimmo Kinnunen <kkinnunen>
Status: RESOLVED FIXED    
Severity: Major CC: connell, darin, dino, dustin.kerstein, ews-watchlist, jer.noble, kbr, kkinnunen, kondapallykalyan, kpiddington, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: Safari 15   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=236759
https://bugs.webkit.org/show_bug.cgi?id=218637
https://bugs.webkit.org/show_bug.cgi?id=232235
Bug Depends on: 220896    
Bug Blocks:    
Attachments:
Description Flags
Patch
none
STP 141 on MacBook Pro with M1 Pro none

Simon Taylor
Reported 2022-02-16 02:23:26 PST
Another major video texImage2D performance regression has appeared with iOS 15.4 beta. Using my usual test cases from Bug 216250, featuring a 720p video: https://tango-bravo.net/WebGLVideoPerformance/texImage2d_video.html (uses gl.RGB format and internal format) https://tango-bravo.net/WebGLVideoPerformance/texImage2d_video_rgba.html (uses gl.RGBA format and internal format) On an iPhone 12 Pro with iOS 15.4 beta 3, the first one results in ~60ms per frame for texImage2D and the second one is ~40ms. I've confirmed with systrace that the CPU is busy for all that time in the WebKit content process, and running on a performance core. This is around a 20x performance regression (previously typical timings were in the 2-3ms range when CPU is running with high clocks). --- In case it's helpful: I know in the past some of these regressions have been caused by the compressed internal IOSurface formats - I wonder if it might be worth explicitly requesting a non-compressed format for AVPlayerItemVideoOutput / AVCaptureVideoDataOutput? Something like this (ideally I suppose you'd pick the best match for the source video in terms of full / video range): NSDictionary *pixBuffAttributes = [NSDictionary dictionaryWithObjectsAndKeys:[NSNumber numberWithInt:kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange], (id)kCVPixelBufferPixelFormatTypeKey, nil]; AVPlayerItemVideoOutput* output = [[AVPlayerItemVideoOutput alloc] initWithPixelBufferAttributes:pixBuffAttributes]; I struggled to find much about the compressed IOSurface support, but this WWDC talk is the best reference I've seen: https://developer.apple.com/videos/play/wwdc2021/10047/?time=1843
Attachments
Patch (2.45 KB, patch)
2022-02-17 00:03 PST, Kimmo Kinnunen 		no flags
DetailsFormatted DiffDiff
STP 141 on MacBook Pro with M1 Pro (2.04 MB, video/quicktime)
2022-03-15 04:03 PDT, Simon Taylor 		no flags
Details
View All Add attachment proposed patch, testcase, etc.
Radar WebKit Bug Importer
Comment 1 2022-02-16 07:22:20 PST
<rdar://problem/89024095>
Simon Taylor
Comment 2 2022-02-16 09:37:46 PST
This isn't reproducible on an iPod touch 7th gen with 15.4b3, which points towards this being another one specific to the compressed IOSurface formats.
Jer Noble
Comment 3 2022-02-16 19:58:48 PST
Thanks for the report. It looks like this is due to a change in ANGLE, introduced in r286603. The compressed pixel buffers are failing validation inside IOSurfaceSurfaceMtl::ValidateAttributes().
Kenneth Russell
Comment 4 2022-02-16 23:59:21 PST
To clarify, that was the ANGLE roll performed in Bug 220896 in early December 2021, which resolved 1+ year of divergent work. Can this bug be reproduced on macOS, perhaps on the M1 GPU? Which incoming compressed pixel format is failing validation? Was an entry dropped in ANGLE's format tables during the upstreaming process?
Kimmo Kinnunen
Comment 5 2022-02-17 00:03:37 PST
Created attachment 452332 [details] Patch
EWS Watchlist
Comment 6 2022-02-17 00:05:24 PST
Note that there are important steps to take when updating ANGLE. See https://trac.webkit.org/wiki/UpdatingANGLE
EWS
Comment 7 2022-02-17 05:14:24 PST
Committed r290011 (247397@main): <https://commits.webkit.org/247397@main> All reviewed patches have been landed. Closing bug and clearing flags on attachment 452332 [details].
Darin Adler
Comment 8 2022-02-17 06:31:17 PST
Regression test to prevent his kind of mistake from being reintroduced in future?
Darin Adler
Comment 9 2022-02-17 06:33:08 PST
Oh, I see: coming in bug 236759
Jer Noble
Comment 10 2022-02-17 11:14:54 PST
(In reply to Kenneth Russell from comment #4) > To clarify, that was the ANGLE roll performed in Bug 220896 in early > December 2021, which resolved 1+ year of divergent work. > > Can this bug be reproduced on macOS, perhaps on the M1 GPU? > > Which incoming compressed pixel format is failing validation? Was an entry > dropped in ANGLE's format tables during the upstreaming process? The format in question was kCVPixelFormatType_Lossless_420YpCbCr8BiPlanarVideoRange, documented (kind of) here: <https://developer.apple.com/documentation/corevideo/3746862-anonymous/kcvpixelformattype_lossless_420ypcbcr8biplanarvideorange>. The compression means that each "element" in a given plane is a block of multiple pixels. While each pixel within that element conforms correctly to the validation (two-components, each component occupies 1 byte, etc.), the element as a whole does not, and so the return value of IOSurfaceGetBytesPerElementOfPlane() is much, much larger than ANGLE expects.
Dustin Kerstein
Comment 11 2022-02-18 07:39:31 PST
Are there any known workarounds for this issue in case this makes it out in the official iOS 15.4? From the original post it seems to affect both RGB and RGBA textures, but if there are any ways to avoid this perf regression, it'd be good to post here for folks who may encounter it.
Simon Taylor
Comment 12 2022-02-18 08:54:04 PST
(In reply to Kenneth Russell from comment #4) > Can this bug be reproduced on macOS, perhaps on the M1 GPU? It does reproduce on my M1 Pro MBP in the latest Safari Tech Preview 140. Weirdly there it is reporting 60 FPS still and upload timings around 13ms but the video is noticeably jankier that those timings suggest. (In reply to Dustin Kerstein from comment #11) > Are there any known workarounds for this issue in case this makes it out in > the official iOS 15.4? From the original post it seems to affect both RGB > and RGBA textures, but if there are any ways to avoid this perf regression, > it'd be good to post here for folks who may encounter it. This is basically the same bug that was with us in iOS 14 up until 14.6 and I don't know of any workarounds from that time. It's a small patch so I'm guessing the chances are good it will be fixed by 15.4 final, but Apple of course will not comment on such a thing as they enjoy giving us all sleepless nights waiting for the release to drop to a few hundred million users :)
Simon Taylor
Comment 13 2022-03-15 04:00:05 PDT
I've just checked with the 15.4 RC build on iPhone 12 Pro, and the performance regression has been resolved there, so I can sleep a little more easily waiting for the final 15.4 to drop now. On the Mac side, the latest STP 141 (from March 3rd) still suffers from this issue on M1 systems.
Simon Taylor
Comment 14 2022-03-15 04:03:48 PDT
Created attachment 454688 [details] STP 141 on MacBook Pro with M1 Pro Attached a video of the RGBA test page in STP 141 on my M1 Pro MBP. You can see it claims to be hitting 60 FPS consistently, but the upload time is high. The actual playback does look much jankier than usual, so there is something else slightly odd happening - possibly just the decode progress being somewhat starved of CPU time by the software fallback upload path?
Note You need to log in before you can comment on or make changes to this bug.