Bug 217934

Summary: Test landed broken: [mac-wk1] inspector/debugger/breakpoint-resolve-when-script-added.html is a flaky timeout
Product: WebKit Reporter: Ryan Haddad <ryanhaddad>
Component: Web InspectorAssignee: Devin Rousso <hi>
Status: NEW ---    
Severity: Normal CC: hi, inspector-bugzilla-changes, webkit-bot-watchers-bugzilla, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=217862

Description Ryan Haddad 2020-10-19 17:37:11 PDT
The test added with https://trac.webkit.org/changeset/268629, inspector/debugger/breakpoint-resolve-when-script-added.html is a flaky timeout on macOS WK1 bots.

https://results.webkit.org/?suite=layout-tests&test=inspector%2Fdebugger%2Fbreakpoint-resolve-when-script-added.html
Comment 1 Radar WebKit Bug Importer 2020-10-19 17:37:39 PDT
<rdar://problem/70464034>
Comment 2 Ryan Haddad 2020-10-19 17:42:09 PDT
Skipped test for mac-wk1 with https://trac.webkit.org/changeset/268703/webkit
Comment 3 Devin Rousso 2020-10-19 17:58:08 PDT
I am 100% sure this is an issue with the test, not any of the other code added in r268629.

r268629 is basically a partial revert of r266074 with an added test case to make sure that the regression doesn't happen again.

r266074 renamed `InspectorDebuggerAgent::m_javaScriptBreakpoints` to `InspectorDebuggerAgent::m_protocolBreakpointForProtocolBreakpointID`, but also incorrectly included an extra call to `m_protocolBreakpointForProtocolBreakpointID.clear()` inside `InspectorDebuggerAgent::clearDebuggerBreakpointState` (which is called each time the global object of the main frame changes, such as from a reload/navigation). r268629 removed that extra call.

One complication for this test is that it makes use of `InspectorTest.reloadPage`, which I vaguely remember having issues in WK1 in the past (some other tests that use `InspectorTest.reloadPage` are also skipped on WK1, such as  inspector/page/overrideUserAgent.html and inspector/dom-debugger/event-breakpoint-with-navigation.html).

I am also having trouble reproducing this locally, as `run-webkit-tests -1` appears to be using an older version of the test as I wrote it before I uploaded the patch, so it fails with a diff (instead of timeout) :|

I have tested the changes in r268629 using WK1 MiniBrowser and everything works just fine.
Comment 4 Alexey Proskuryakov 2020-10-21 17:21:31 PDT
As tests are a tool for confirming that behavior continues to be correct into the future, a manual test at any single point in time does not replace a working test.

> I am also having trouble reproducing this locally, as `run-webkit-tests -1` appears to be using an older version of the test as I wrote it before I uploaded the patch, so it fails with a diff (instead of timeout) :|

Not sure what you mean by this. Perhaps you have multiple checkouts, and are running tests against a different one?