Bug 215054 - Prevent multiple invocations of generate-message-receiver.py
Summary: Prevent multiple invocations of generate-message-receiver.py
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Keith Rollin
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2020-08-01 00:44 PDT by Keith Rollin
Modified: 2020-08-03 09:30 PDT (History)
5 users (show)

See Also:


Attachments
Patch (3.66 KB, patch)
2020-08-01 00:47 PDT, Keith Rollin
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Keith Rollin 2020-08-01 00:44:21 PDT
WebKit/DerivedSources.make is written such that generate-message-receiver.py can be invoked N times, where N is the value passed to -j. This results in the N invocations of generate-message-receiver.py trying trying to generate the same set of files at the same time, possibly leading to their being corrupted. We have reports of generated files starting with a 4K block of NULs, and this contention may be causing it. Address this by using a "pattern rule" in the makefile, which is documented in GNU make's documentation as the approach to take when there is one build rule that creates multiple outputs, as is the case here.

See also Bug 166814 and <http://trac.webkit.org/changeset/210507>.
Comment 1 Keith Rollin 2020-08-01 00:44:26 PDT
rdar://problem/63595589
Comment 2 Keith Rollin 2020-08-01 00:47:10 PDT
Created attachment 405775 [details]
Patch
Comment 3 EWS 2020-08-01 10:54:49 PDT
Committed r265178: <https://trac.webkit.org/changeset/265178>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 405775 [details].
Comment 4 Darin Adler 2020-08-01 13:40:25 PDT
Comment on attachment 405775 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=405775&action=review

> Source/WebKit/DerivedSources.make:257
> +MESSAGES_FILES := $(addsufix Messages.h,$(notdir $(MESSAGE_RECEIVERS)))

The "addsuffix" here was missing an "f" so this didn’t quite work.

Also, I did some further research, and I think a better solution is a "grouped target" rather than a pattern rule.
Comment 5 Darin Adler 2020-08-01 14:08:10 PDT
I put a fix for those things into bug 215064.
Comment 6 Keith Rollin 2020-08-02 18:09:24 PDT
I commented in <rdar://problem/63595589> that "grouped targets" won't work.
Comment 7 Keith Rollin 2020-08-02 18:10:32 PDT
That missing "f" was from the original code. I wonder how long it had been broken that way.
Comment 8 Keith Rollin 2020-08-02 18:12:32 PDT
Issue was introduced May 6 with r261254. The first report of the generated file corruption was on May 25.
Comment 9 Darin Adler 2020-08-02 19:30:16 PDT
That revision replaced the existing pattern rule with a non-pattern rule!

I think we found the root cause.