| Summary: | imported/w3c/web-platform-tests/webaudio/the-audio-api/the-biquadfilternode-interface/no-dezippering.html | ||||||
|---|---|---|---|---|---|---|---|
| Product: | WebKit | Reporter: | Jon Lee <jonlee> | ||||
| Component: | Web Audio | Assignee: | Chris Dumez <cdumez> | ||||
| Status: | RESOLVED FIXED | ||||||
| Severity: | Normal | CC: | cdumez, clopez, eric.carlson, ews-watchlist, glenn, jer.noble, philipj, ryanhaddad, sergio, webkit-bot-watchers-bugzilla, webkit-bug-importer, youennf | ||||
| Priority: | P2 | Keywords: | InRadar | ||||
| Version: | WebKit Nightly Build | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Attachments: |
|
||||||
|
Description
Jon Lee
2022-04-04 21:52:43 PDT
--- /Volumes/Data/worker/ios-simulator-15-release-gpuprocess-arm64-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webaudio/the-audio-api/the-biquadfilternode-interface/no-dezippering-expected.txt
+++ /Volumes/Data/worker/ios-simulator-15-release-gpuprocess-arm64-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/webaudio/the-audio-api/the-biquadfilternode-interface/no-dezippering-actual.txt
@@ -36,9 +36,20 @@
PASS Output from gain setter matches setValueAtTime output is true.
PASS < [Test 3] All assertions passed. (total 5 assertions)
PASS > [Test 4] No dezippering of frequency vs JS filter
-PASS Output from lowpass filter equals [expected array] with an element-wise tolerance of {"absoluteThreshold":5.9607e-7,"relativeThreshold":0}.
-PASS Output matches JS filter results is true.
-PASS < [Test 4] All assertions passed. (total 2 assertions)
+FAIL X Output from lowpass filter does not equal [expected array] with an element-wise tolerance of {"absoluteThreshold":5.9607e-7,"relativeThreshold":0}.
+ Index Actual Expected AbsError RelError Test threshold
+ [82] -5.640129e-1 -5.640122e-1 6.556511e-7 1.162477e-6 5.960700e-7
+ [83] -4.615946e-1 -4.615939e-1 6.854534e-7 1.484971e-6 5.960700e-7
+ [84] -3.461582e-1 -3.461576e-1 6.854534e-7 1.980178e-6 5.960700e-7
+ [85] -2.209939e-1 -2.209932e-1 6.705523e-7 3.034267e-6 5.960700e-7
+ [86] -8.966466e-2 -8.966400e-2 6.556511e-7 7.312311e-6 5.960700e-7
+ ...and 1 more errors.
+ Max AbsError of 6.854534e-7 at index of 83.
+ Max RelError of 1.419336e-5 at index of 87.
+ [87] 4.409384e-2 4.409447e-2 6.258488e-7 1.419336e-5 5.960700e-7
+ assert_true: expected true got false
+FAIL X Output matches JS filter results is not true. Got false. assert_true: expected true got false
+FAIL < [Test 4] 2 out of 2 assertions were failed. assert_true: expected true got false
PASS > [Test 5] Test with modulation
PASS Modulation: At time 0, frequency is equal to 350.
PASS Modulation: At time 0.015625, frequency is equal to 10.
@@ -46,5 +57,5 @@
PASS Modulation: Output from frequency setter equals [expected array] with an element-wise tolerance of {"absoluteThreshold":0,"relativeThreshold":0}.
PASS Modulation: Output from frequency setter matches setValueAtTime output is true.
PASS < [Test 5] All assertions passed. (total 5 assertions)
-PASS # AUDIT TASK RUNNER FINISHED: 6 tasks ran successfully.
+FAIL # AUDIT TASK RUNNER FINISHED: 1 out of 6 tasks were failed. assert_true: expected true got false
(In reply to Alexey Proskuryakov from comment #1) > --- > /Volumes/Data/worker/ios-simulator-15-release-gpuprocess-arm64-tests-wk2/ > build/layout-test-results/imported/w3c/web-platform-tests/webaudio/the-audio- > api/the-biquadfilternode-interface/no-dezippering-expected.txt > +++ > /Volumes/Data/worker/ios-simulator-15-release-gpuprocess-arm64-tests-wk2/ > build/layout-test-results/imported/w3c/web-platform-tests/webaudio/the-audio- > api/the-biquadfilternode-interface/no-dezippering-actual.txt > @@ -36,9 +36,20 @@ > PASS Output from gain setter matches setValueAtTime output is true. > PASS < [Test 3] All assertions passed. (total 5 assertions) > PASS > [Test 4] No dezippering of frequency vs JS filter > -PASS Output from lowpass filter equals [expected array] with an > element-wise tolerance of > {"absoluteThreshold":5.9607e-7,"relativeThreshold":0}. > -PASS Output matches JS filter results is true. > -PASS < [Test 4] All assertions passed. (total 2 assertions) > +FAIL X Output from lowpass filter does not equal [expected array] with an > element-wise tolerance of > {"absoluteThreshold":5.9607e-7,"relativeThreshold":0}. > + Index Actual Expected AbsError RelError Test threshold > + [82] -5.640129e-1 -5.640122e-1 6.556511e-7 1.162477e-6 5.960700e-7 > + [83] -4.615946e-1 -4.615939e-1 6.854534e-7 1.484971e-6 5.960700e-7 > + [84] -3.461582e-1 -3.461576e-1 6.854534e-7 1.980178e-6 5.960700e-7 > + [85] -2.209939e-1 -2.209932e-1 6.705523e-7 3.034267e-6 5.960700e-7 > + [86] -8.966466e-2 -8.966400e-2 6.556511e-7 7.312311e-6 5.960700e-7 > + ...and 1 more errors. > + Max AbsError of 6.854534e-7 at index of 83. > + Max RelError of 1.419336e-5 at index of 87. > + [87] 4.409384e-2 4.409447e-2 6.258488e-7 1.419336e-5 5.960700e-7 > + assert_true: expected true got false > +FAIL X Output matches JS filter results is not true. Got false. > assert_true: expected true got false > +FAIL < [Test 4] 2 out of 2 assertions were failed. assert_true: expected > true got false > PASS > [Test 5] Test with modulation > PASS Modulation: At time 0, frequency is equal to 350. > PASS Modulation: At time 0.015625, frequency is equal to 10. > @@ -46,5 +57,5 @@ > PASS Modulation: Output from frequency setter equals [expected array] > with an element-wise tolerance of > {"absoluteThreshold":0,"relativeThreshold":0}. > PASS Modulation: Output from frequency setter matches setValueAtTime > output is true. > PASS < [Test 5] All assertions passed. (total 5 assertions) > -PASS # AUDIT TASK RUNNER FINISHED: 6 tasks ran successfully. > +FAIL # AUDIT TASK RUNNER FINISHED: 1 out of 6 tasks were failed. > assert_true: expected true got false The actual and expected values are very close. This just looks like a precision issue because we're dealing with floating point values and vectorization (and different CPU architectures). I believe we'll just need to update the test to use a slightly error tolerance. In the mean time, a platform-specific baseline is the right thing to do IMO. Doesn't reproduce on my machine with either: Tools/Scripts/run-webkit-tests --release --no-build --ios-simulator --force imported/w3c/web-platform-tests/webaudio/the-audio-api/the-biquadfilternode-interface/no-dezippering.html or Tools/Scripts/run-webkit-tests --release --no-build --ios-simulator --force imported/w3c/web-platform-tests/webaudio/the-audio-api/the-biquadfilternode-interface/no-dezippering.html --use-gpu-process (In reply to Chris Dumez from comment #4) > Doesn't reproduce on my machine with either: > Tools/Scripts/run-webkit-tests --release --no-build --ios-simulator --force > imported/w3c/web-platform-tests/webaudio/the-audio-api/the-biquadfilternode- > interface/no-dezippering.html > or > Tools/Scripts/run-webkit-tests --release --no-build --ios-simulator --force > imported/w3c/web-platform-tests/webaudio/the-audio-api/the-biquadfilternode- > interface/no-dezippering.html --use-gpu-process I guess because my host is Intel. This probably need arm64 to repro. Created attachment 457555 [details]
Patch
This patch modifies the imported WPT tests. Please ensure that any changes on the tests (not coming from a WPT import) are exported to WPT. Please see https://trac.webkit.org/wiki/WPTExportProcess Committed r292832 (249605@main): <https://commits.webkit.org/249605@main> All reviewed patches have been landed. Closing bug and clearing flags on attachment 457555 [details]. The test has recovered on the bot and I have submitted the PR upstream: https://github.com/web-platform-tests/wpt/pull/33642. |