Bug 221068

Summary: [Flatpak SDK] Avoid "Invalid byte sequence in conversion input" errors and other encoding issues
Product: WebKit Reporter: Lauro Moura <lmoura>
Component: Tools / TestsAssignee: Lauro Moura <lmoura>
Status: RESOLVED FIXED    
Severity: Normal CC: aboya, aperez, bugs-noreply, pnormand, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=220781
Attachments:
Description Flags
Patch none

Description Lauro Moura 2021-01-27 16:38:56 PST 
After r271938, sometimes the flatpak invocation could raise the following error without more information:

Error: Invalid byte sequence in conversion input

This seems to happen due to LANG being deleted in flatpakutils.py before being set in webkit-bwrap.

Also, the webkit-flatpak script may fail with encoding issues when printing the messages "Running..." and "Command.... exited with non-zero status" with the non-ascii prompt.
Comment 1 Lauro Moura 2021-01-27 16:47:59 PST 
Created attachment 418600 [details]
Patch
Comment 2 Philippe Normand 2021-01-28 00:29:47 PST 
Comment on attachment 418600 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=418600&action=review

> Tools/ChangeLog:10
> +        (WebkitFlatpak.run_in_sandbox): Avoid iterating mutating dict and

What's wrong with that?
Comment 3 Alicia Boya García 2021-01-28 03:03:02 PST 
Comment on attachment 418600 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=418600&action=review

>> Tools/ChangeLog:10
>> +        (WebkitFlatpak.run_in_sandbox): Avoid iterating mutating dict and
> 
> What's wrong with that?

It's not supported in Python, it's a case of iterator invalidation:

e.g.

a = {"a":1, "b": 2, "c": 3}                                                                                                                                                                                                                           
for k in a.keys(): 
     print(k) 
     del a[k] 

RuntimeError: dictionary changed size during iteration
Comment 4 Lauro Moura 2021-01-28 03:13:39 PST 
(In reply to Alicia Boya García from comment #3)
> Comment on attachment 418600 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=418600&action=review
> 
> >> Tools/ChangeLog:10
> >> +        (WebkitFlatpak.run_in_sandbox): Avoid iterating mutating dict and
> > 
> > What's wrong with that?
> 
> It's not supported in Python, it's a case of iterator invalidation:
> 
> e.g.
> 
> a = {"a":1, "b": 2, "c": 3}                                                 
> 
> for k in a.keys(): 
>      print(k) 
>      del a[k] 
> 
> RuntimeError: dictionary changed size during iteration

This. Py2[1] had dict.keys()[1], returning a list, and dict.viewkeys()[2], which returned an iterator (a view) on them.

Py3 changed the behavior of keys() to return the view[3].

[1] https://python.readthedocs.io/en/v2.7.2/library/stdtypes.html#dict.keys
[2] https://python.readthedocs.io/en/v2.7.2/library/stdtypes.html#dict.viewkeys
[3] https://docs.python.org/3/library/stdtypes.html#dict.keys
Comment 5 EWS 2021-01-28 04:48:14 PST 
Committed r272011: <https://trac.webkit.org/changeset/272011>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 418600 [details].
Comment 6 Radar WebKit Bug Importer 2021-01-28 04:49:14 PST 
<rdar://problem/73704686>