Bug 215054

Summary: Prevent multiple invocations of generate-message-receiver.py
Product: WebKit Reporter: Keith Rollin <krollin>
Component: Tools / TestsAssignee: Keith Rollin <krollin>
Status: RESOLVED FIXED    
Severity: Normal CC: darin, dino, graouts, simon.fraser, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=215064
https://bugs.webkit.org/show_bug.cgi?id=211112
Attachments:
Description Flags
Patch none

Description Keith Rollin 2020-08-01 00:44:21 PDT 
WebKit/DerivedSources.make is written such that generate-message-receiver.py can be invoked N times, where N is the value passed to -j. This results in the N invocations of generate-message-receiver.py trying trying to generate the same set of files at the same time, possibly leading to their being corrupted. We have reports of generated files starting with a 4K block of NULs, and this contention may be causing it. Address this by using a "pattern rule" in the makefile, which is documented in GNU make's documentation as the approach to take when there is one build rule that creates multiple outputs, as is the case here.

See also Bug 166814 and <http://trac.webkit.org/changeset/210507>.
Comment 1 Keith Rollin 2020-08-01 00:44:26 PDT 
rdar://problem/63595589
Comment 2 Keith Rollin 2020-08-01 00:47:10 PDT 
Created attachment 405775 [details]
Patch
Comment 3 EWS 2020-08-01 10:54:49 PDT 
Committed r265178: <https://trac.webkit.org/changeset/265178>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 405775 [details].
Comment 4 Darin Adler 2020-08-01 13:40:25 PDT 
Comment on attachment 405775 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=405775&action=review

> Source/WebKit/DerivedSources.make:257
> +MESSAGES_FILES := $(addsufix Messages.h,$(notdir $(MESSAGE_RECEIVERS)))

The "addsuffix" here was missing an "f" so this didn’t quite work.

Also, I did some further research, and I think a better solution is a "grouped target" rather than a pattern rule.
Comment 5 Darin Adler 2020-08-01 14:08:10 PDT 
I put a fix for those things into bug 215064.
Comment 6 Keith Rollin 2020-08-02 18:09:24 PDT 
I commented in <rdar://problem/63595589> that "grouped targets" won't work.
Comment 7 Keith Rollin 2020-08-02 18:10:32 PDT 
That missing "f" was from the original code. I wonder how long it had been broken that way.
Comment 8 Keith Rollin 2020-08-02 18:12:32 PDT 
Issue was introduced May 6 with r261254. The first report of the generated file corruption was on May 25.
Comment 9 Darin Adler 2020-08-02 19:30:16 PDT 
That revision replaced the existing pattern rule with a non-pattern rule!

I think we found the root cause.