Bug 216068

Summary: [GLIB] kill-old-processes: Remove stale Apache2 shm segments
Product: WebKit Reporter: Diego Pino <dpino>
Component: New BugsAssignee: Diego Pino <dpino>
Status: RESOLVED FIXED    
Severity: Normal CC: aakash_jain, commit-queue, ews-watchlist, lmoura, tsavell, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=216323
Bug Depends on: 216135    
Bug Blocks:    
Attachments:
Description Flags
Patch
none
Patch
none
Patch
none
Patch
none
Refactored patch.
none
Patch for landing none

Description Diego Pino 2020-09-01 23:09:34 PDT
kill-old-processes: Remove stale Apache shm segments
Comment 1 Diego Pino 2020-09-01 23:35:13 PDT
Created attachment 407742 [details]
Patch
Comment 2 Diego Pino 2020-09-01 23:47:18 PDT
This issue was happening in several EWS workers running in the UAT. For instance:

https://ews-build.webkit-uat.org/#/builders/34/builds/4926/steps/10/logs/stdio

Apache2 doesn't start because it believes there's already an Apache2 instance running, when actually what's happening is that there's an orphan shm segment which pid matches the pid of the new Apache2 instance. In that case layout-test-results/error-log shows an error like this:

[Sat Aug 29 20:02:16.013912 2020] [core:crit] [pid 26] (17)File exists: AH00001: unable to create or access scoreboard "/tmp/WebKit/httpd.scoreboard" (name-based shared memory failure)

And here's a post with the description of the problem and solution:
https://makandracards.com/fabio-silva/49967-apache-unable-to-create-scoreboard-shared-memory-failure
Comment 3 Diego Pino 2020-09-01 23:53:01 PDT
Created attachment 407744 [details]
Patch
Comment 4 Alexey Proskuryakov 2020-09-02 10:31:14 PDT
Comment on attachment 407744 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=407744&action=review

> Tools/BuildSlaveSupport/kill-old-processes:27
> +import subprocess as sp

Why rename the module? WebKit coding style is to avoid abbreviations.
Comment 5 Diego Pino 2020-09-02 14:18:04 PDT
Created attachment 407812 [details]
Patch
Comment 6 Diego Pino 2020-09-02 14:20:24 PDT
(In reply to Alexey Proskuryakov from comment #4)
> Comment on attachment 407744 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=407744&action=review
> 
> > Tools/BuildSlaveSupport/kill-old-processes:27
> > +import subprocess as sp
> 
> Why rename the module? WebKit coding style is to avoid abbreviations.

Sorry, I didn't know about this code convention. I pushed a new patch undoing the renaming.
Comment 7 EWS 2020-09-03 01:07:53 PDT
Committed r266507: <https://trac.webkit.org/changeset/266507>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 407812 [details].
Comment 8 Radar WebKit Bug Importer 2020-09-03 01:08:13 PDT
<rdar://problem/68262415>
Comment 9 Truitt Savell 2020-09-03 11:15:01 PDT
It looks like the changes in https://trac.webkit.org/changeset/266507/webkit 

broke our JSC EWS bot:
https://ews-build.webkit.org/#/builders/38/builds/1408

We will need to roll this out unless it can be fixed very quickly.


Traceback (most recent call last):
  File "Tools/BuildSlaveSupport/kill-old-processes", line 192, in <module>
    sys.exit(main(user))
  File "Tools/BuildSlaveSupport/kill-old-processes", line 184, in main
    removeOrphanShmSegmentsOf("apache2")
  File "Tools/BuildSlaveSupport/kill-old-processes", line 39, in removeOrphanShmSegmentsOf
    pid = futurePidOf(task)
  File "Tools/BuildSlaveSupport/kill-old-processes", line 56, in futurePidOf
    ps = subprocess.Popen(task, stderr=subprocess.STDOUT, stdout=devnull)
  File "/usr/lib/python2.7/subprocess.py", line 394, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1047, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory
Comment 10 WebKit Commit Bot 2020-09-03 11:40:46 PDT
Re-opened since this is blocked by bug 216135
Comment 11 Diego Pino 2020-09-03 20:51:38 PDT
Created attachment 407941 [details]
Patch
Comment 12 Lauro Moura 2020-09-08 14:00:48 PDT
Created attachment 408266 [details]
Refactored patch.
Comment 13 EWS 2020-09-08 20:55:35 PDT
ChangeLog entry in Tools/ChangeLog is not at the top of the file.
Comment 14 Lauro Moura 2020-09-08 21:30:52 PDT
Created attachment 408309 [details]
Patch for landing
Comment 15 EWS 2020-09-08 22:14:20 PDT
Committed r266775: <https://trac.webkit.org/changeset/266775>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 408309 [details].