Bug 217118

Summary: REGRESSION(r267402): [ macOS iOS Release ] imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker.html is a flaky failure
Product: WebKit Reporter: Hector Lopez <hector_i_lopez>
Component: New BugsAssignee: Alex Christensen <achristensen>
Status: RESOLVED FIXED    
Severity: Normal CC: achristensen, clopez, ews-watchlist, sam, thorton, webkit-bot-watchers-bugzilla, webkit-bug-importer, youennf
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
Patch none

Description Hector Lopez 2020-09-29 19:40:02 PDT 
imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker.html

Test is a flaky failure according to history on macOS Release and iOS Release. First occurrence of failure at https://trac.webkit.org/changeset/267402/webkit

History:

https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fuser-timing%2Fmeasure-l3.any.worker.html&platform=ios&platform=mac&limit=50000

Diff:

--- /Volumes/Data/slave/catalina-release-tests-wk1/build/layout-test-results/imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker-expected.txt
+++ /Volumes/Data/slave/catalina-release-tests-wk1/build/layout-test-results/imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker-actual.txt
@@ -1,5 +1,5 @@
 
 PASS When the end mark is given and the start is unprovided, the end time of the measure entry should be the end mark's time, the start time should be 0.
-PASS When the start mark is given and the end is unprovided, the start time of the measure entry should be the start mark's time, the end should be now.
+FAIL When the start mark is given and the end is unprovided, the start time of the measure entry should be the start mark's time, the end should be now. assert_greater_than_equal: expected a number greater than or equal to 13.000000000000002 but got 13
 PASS When start and end mark are both given, the start time and end time of the measure entry should be the the marks' time, repectively

Was able to reproduce this with r267769:

run-webkit-tests imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker.html  --iterations 1000 --exit-after-n-failures 3 -1 -f

running 1 test

Running 6 DumpRenderTrees in parallel.

[35/1000] imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker.html failed unexpectedly (text diff)
[46/1000] imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker.html failed unexpectedly (text diff)
[47/1000] imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker.html failed unexpectedly (text diff)
Exiting early after 3 failures. 42 tests run.


39 tests ran as expected, 3 didn't (958 didn't run):


Regressions: Unexpected text-only failures (1)
  imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker.html [ Failure ]

Diff:
--- /Volumes/Data/Builds/build-267769/layout-test-results/imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker-expected.txt
+++ /Volumes/Data/Builds/build-267769/layout-test-results/imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker-actual.txt
@@ -1,5 +1,5 @@
 
 PASS When the end mark is given and the start is unprovided, the end time of the measure entry should be the end mark's time, the start time should be 0.
-PASS When the start mark is given and the end is unprovided, the start time of the measure entry should be the start mark's time, the end should be now.
+FAIL When the start mark is given and the end is unprovided, the start time of the measure entry should be the start mark's time, the end should be now. assert_greater_than_equal: expected a number greater than or equal to 26.000000000000004 but got 26
 PASS When start and end mark are both given, the start time and end time of the measure entry should be the the marks' time, repectively
Comment 1 Radar WebKit Bug Importer 2020-09-29 19:40:23 PDT 
<rdar://problem/69774340>
Comment 2 Hector Lopez 2020-09-29 19:45:59 PDT 
Test expectation while investigated:

https://trac.webkit.org/changeset/267778/webkit
Comment 3 Alex Christensen 2021-01-08 11:00:40 PST 
Created attachment 417279 [details]
Patch
Comment 4 EWS Watchlist 2021-01-08 11:01:40 PST 
This patch modifies the imported WPT tests. Please ensure that any changes on the tests (not coming from a WPT import) are exported to WPT. Please see https://trac.webkit.org/wiki/WPTExportProcess
Comment 5 Tim Horton 2021-01-08 11:56:35 PST 
Comment on attachment 417279 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=417279&action=review

> LayoutTests/imported/w3c/web-platform-tests/user-timing/measure-l3.any.js:25
> +  assert_greater_than_equal(Number(endTime(measureEntry).toFixed(2)), endMin);
> +  assert_greater_than_equal(endMax, Number(endTime(measureEntry).toFixed(2)));

Would it be better to do this by subtraction and picking a maximum delta instead?
Comment 6 Alex Christensen 2021-01-08 11:57:52 PST 
Comment on attachment 417279 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=417279&action=review

>> LayoutTests/imported/w3c/web-platform-tests/user-timing/measure-l3.any.js:25
>> +  assert_greater_than_equal(endMax, Number(endTime(measureEntry).toFixed(2)));
> 
> Would it be better to do this by subtraction and picking a maximum delta instead?

Probably, but this is consistent with what is already done elsewhere in this test.
Comment 7 EWS 2021-01-08 13:50:00 PST 
Committed r271320: <https://trac.webkit.org/changeset/271320>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 417279 [details].