Created attachment 456706 [details]
Patch

Comment on attachment 456706 [details]
Patch

I want to deploy this to our staging instance before landing.

(In reply to Jonathan Bedard from comment #2)
> Comment on attachment 456706 [details]
> Patch
> 
> I want to deploy this to our staging instance before landing.

This would be why we deploy things to the staging instance:

Looking at https://results.webkit.org/api/upload?suite=layout-tests&limit=1, looks like we don't always specify the "actual" result if a test passes. So our check needs to be a bit more sophisticated, I think. Probably the old check, plus an extra check that recognizes that a set of keys that can be nested?

(In reply to Jonathan Bedard from comment #3)
> (In reply to Jonathan Bedard from comment #2)
> > Comment on attachment 456706 [details]
> > Patch
> > 
> > I want to deploy this to our staging instance before landing.
> 
> This would be why we deploy things to the staging instance:
> 
> Looking at https://results.webkit.org/api/upload?suite=layout-tests&limit=1,
> looks like we don't always specify the "actual" result if a test passes. So
> our check needs to be a bit more sophisticated, I think. Probably the old
> check, plus an extra check that recognizes that a set of keys that can be
> nested?

Thanks for the extra testing! I've looked at the layout tests code and I think it's probably safer in the long run to keep the current termination check. I've modified https://bugs.webkit.org/show_bug.cgi?id=238806 to use two new fields in the existing test result instead of a nested dict.