Bug 242191 - [GLib] Modernize Tools/glib/apply-build-revision-to-files.py
Summary: [GLib] Modernize Tools/glib/apply-build-revision-to-files.py
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Local Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Philippe Normand
URL:
Keywords: InRadar
Depends on: 242176
Blocks:
  Show dependency treegraph
 
Reported: 2022-06-30 07:48 PDT by Adrian Perez
Modified: 2022-09-26 13:08 PDT (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Adrian Perez 2022-06-30 07:48:03 PDT
In the review for bug #242176 we talked about cleaning up further the script
by removing Subversion-specific code, now that we do not use Subversion anymore.
Comment 1 Adrian Perez 2022-06-30 07:55:11 PDT
Pull request: https://github.com/WebKit/WebKit/pull/1947
Comment 2 EWS 2022-06-30 14:15:12 PDT
Committed 252012@main (d22439347db4): <https://commits.webkit.org/252012@main>

Reviewed commits have been landed. Closing PR #1947 and removing active labels.
Comment 3 Radar WebKit Bug Importer 2022-06-30 14:16:13 PDT
<rdar://problem/96245384>
Comment 4 James Hilliard 2022-07-01 01:21:39 PDT
This commit causes wpewebkit builds to fail in buildroot on main:
<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)>
Failed to download setuptools, retrying
Traceback (most recent call last):
  File "/home/buildroot/buildroot/output/build/wpewebkit-custom/Tools/glib/apply-build-revision-to-files.py", line 25, in <module>
    import webkitpy  # noqa
  File "/home/buildroot/buildroot/output/build/wpewebkit-custom/Tools/Scripts/webkitpy/__init__.py", line 91, in <module>
    import webkitscmpy
  File "/home/buildroot/buildroot/output/build/wpewebkit-custom/Tools/Scripts/libraries/webkitscmpy/webkitscmpy/__init__.py", line 62, in <module>
    from webkitscmpy.commit import Commit
  File "/home/buildroot/buildroot/output/build/wpewebkit-custom/Tools/Scripts/libraries/webkitscmpy/webkitscmpy/commit.py", line 24, in <module>
    import six
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1002, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 945, in _find_spec
  File "/home/buildroot/buildroot/output/build/wpewebkit-custom/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 617, in find_spec
    loader = cls.find_module(fullname, path=path)
  File "/home/buildroot/buildroot/output/build/wpewebkit-custom/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 631, in find_module
    cls.install(name)
  File "/home/buildroot/buildroot/output/build/wpewebkit-custom/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 606, in install
    return all([to_install.install() for to_install in packages])
  File "/home/buildroot/buildroot/output/build/wpewebkit-custom/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 606, in <listcomp>
    return all([to_install.install() for to_install in packages])
  File "/home/buildroot/buildroot/output/build/wpewebkit-custom/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 273, in install
    AutoInstall.install('setuptools')
  File "/home/buildroot/buildroot/output/build/wpewebkit-custom/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 606, in install
    return all([to_install.install() for to_install in packages])
  File "/home/buildroot/buildroot/output/build/wpewebkit-custom/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 606, in <listcomp>
    return all([to_install.install() for to_install in packages])
  File "/home/buildroot/buildroot/output/build/wpewebkit-custom/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 285, in install
    raise ValueError('No archives for {}-{} found'.format(self.pypi_name, self.version))
ValueError: No archives for setuptools-56 found
make[6]: *** [Source/WebKit/CMakeFiles/WebKit-build-revision.dir/build.make:70: Source/WebKit/CMakeFiles/WebKit-build-revision] Error 1
make[5]: *** [CMakeFiles/Makefile2:1314: Source/WebKit/CMakeFiles/WebKit-build-revision.dir/all] Error 2
make[5]: *** Waiting for unfinished jobs....

Reverting this seems to fix the build:
https://github.com/WebKit/WebKit/commit/d22439347db4083c729de2a07ebbd6b17797fc29
Comment 5 Adrian Perez 2022-07-01 04:03:59 PDT
(In reply to James Hilliard from comment #4)
> This commit causes wpewebkit builds to fail in buildroot on main:
> <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed:
> unable to get local issuer certificate (_ssl.c:997)>
> Failed to download setuptools, retrying
> Traceback (most recent call last):
>   File
> "/home/buildroot/buildroot/output/build/wpewebkit-custom/Tools/glib/apply-
> build-revision-to-files.py", line 25, in <module>
>     import webkitpy  # noqa
>   File
> "/home/buildroot/buildroot/output/build/wpewebkit-custom/Tools/Scripts/
> webkitpy/__init__.py", line 91, in <module>
>     import webkitscmpy
>   File
> "/home/buildroot/buildroot/output/build/wpewebkit-custom/Tools/Scripts/
> libraries/webkitscmpy/webkitscmpy/__init__.py", line 62, in <module>
>     from webkitscmpy.commit import Commit
>   File
> "/home/buildroot/buildroot/output/build/wpewebkit-custom/Tools/Scripts/
> libraries/webkitscmpy/webkitscmpy/commit.py", line 24, in <module>
>     import six
>   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
>   File "<frozen importlib._bootstrap>", line 1002, in _find_and_load_unlocked
>   File "<frozen importlib._bootstrap>", line 945, in _find_spec
>   File
> "/home/buildroot/buildroot/output/build/wpewebkit-custom/Tools/Scripts/
> libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 617, in find_spec
>     loader = cls.find_module(fullname, path=path)
>   File
> "/home/buildroot/buildroot/output/build/wpewebkit-custom/Tools/Scripts/
> libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 631, in find_module
>     cls.install(name)
>   File
> "/home/buildroot/buildroot/output/build/wpewebkit-custom/Tools/Scripts/
> libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 606, in install
>     return all([to_install.install() for to_install in packages])
>   File
> "/home/buildroot/buildroot/output/build/wpewebkit-custom/Tools/Scripts/
> libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 606, in <listcomp>
>     return all([to_install.install() for to_install in packages])
>   File
> "/home/buildroot/buildroot/output/build/wpewebkit-custom/Tools/Scripts/
> libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 273, in install
>     AutoInstall.install('setuptools')
>   File
> "/home/buildroot/buildroot/output/build/wpewebkit-custom/Tools/Scripts/
> libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 606, in install
>     return all([to_install.install() for to_install in packages])
>   File
> "/home/buildroot/buildroot/output/build/wpewebkit-custom/Tools/Scripts/
> libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 606, in <listcomp>
>     return all([to_install.install() for to_install in packages])
>   File
> "/home/buildroot/buildroot/output/build/wpewebkit-custom/Tools/Scripts/
> libraries/webkitcorepy/webkitcorepy/autoinstall.py", line 285, in install
>     raise ValueError('No archives for {}-{} found'.format(self.pypi_name,
> self.version))
> ValueError: No archives for setuptools-56 found
> make[6]: ***
> [Source/WebKit/CMakeFiles/WebKit-build-revision.dir/build.make:70:
> Source/WebKit/CMakeFiles/WebKit-build-revision] Error 1
> make[5]: *** [CMakeFiles/Makefile2:1314:
> Source/WebKit/CMakeFiles/WebKit-build-revision.dir/all] Error 2
> make[5]: *** Waiting for unfinished jobs....
> 
> Reverting this seems to fix the build:
> https://github.com/WebKit/WebKit/commit/
> d22439347db4083c729de2a07ebbd6b17797fc29

What's failing is fetching the auto-installed Python modules that the
script uses; which probably has to do with Buildroot's host-python
package missing SSL support. It is possible to disable this behaviour
by setting “DISABLE_WEBKITCOREPY_AUTOINSTALLER=0” in the environment.

Probably it would be a good idea to disable autoinstall for non-developer
builds and have a fallback that spawns Git it webkitscmpy fails to import.

Phil, WDYT about that idea above? ^^
Comment 6 Philippe Normand 2022-09-25 10:21:12 PDT
I'm not sure what other unintended side-effects setting DISABLE_WEBKITCOREPY_AUTOINSTALLER=0 will have...

I'd rather revert to git log parsing, TBH.
Comment 7 Philippe Normand 2022-09-25 10:32:31 PDT
Pull request: https://github.com/WebKit/WebKit/pull/4686
Comment 8 EWS 2022-09-26 01:55:27 PDT
Committed 254851@main (792d6325aedb): <https://commits.webkit.org/254851@main>

Reviewed commits have been landed. Closing PR #4686 and removing active labels.