python 3: Add python 3 support for html tests #22402

ziransun · 2020-03-23T19:44:36Z

[1] Fix some TypeErrors. Calling get API in reuqest.headers
() returns binary string values.
In python 2, bytes is alias to str type. In Python 3, str type is
different from bytes. This change uses utf-8 encoding for the request.headers
values using for Py2 and Py3 compability.
[2] Add python 3 support for accessing headers content in raw_headers
(xhr/resources/inspect-headers.py). Type of raw_headers is class
HTTPMessage. In Python 2, HTTPMessage derives from mimetools.Message while
in Python 3, it derives from email.message.Message. Interfaces from
mimetools.Message and email.message.Message are very different.
In Python 3, HTTPMessage doesn't have 'headers" attribute or equivalent.
[ref: https://bugs.python.org/issue4773]
[3] Fix some imports for python2 and python 3 compatibility

annevk

I'm not a big fan of all the encode/decode operations here. There might be a better way to accomplish casting from one type to another.

html/semantics/embedded-content/media-elements/track/track-element/cors/support/cors-tester.py

annevk · 2020-03-24T07:02:51Z

html/semantics/embedded-content/media-elements/track/track-element/cors/support/cors-tester.py

    cookie = request.cookies.first(id, None)
    cookie_value = cookie.value if cookie is not None else "no"
-    line = 'cors = ' + cors + ' | cookie = ' + cookie_value;
+    line = 'cors = ' + cors.decode("utf-8") + ' | cookie = ' + cookie_value


Is this casting a byte sequence to a string, basically? Are there really UTF-8 bytes in there?

html/semantics/embedded-content/the-img-element/404-response-with-actual-image-data.py

annevk · 2020-03-24T07:04:20Z

xhr/resources/inspect-headers.py

+    else:
+        header_list = raw_headers.headers
+    for line in header_list:
+        if not PY3:


This seems wrong.

Is it okay to be more specific? raw_headers derives from different classes in Python 2 and Python 3. Here is to handle them differently.

In particular, it seems like the code isn't executed at all in Python 3.

raw_headers derives from different classes in python 2 and python 3. In Python 3, HTTPMessage doesn't have 'headers" attribute or equivalent as we described in the commit message (see the reference as well). That's why this part code is not executed in Python 3.

We can't just drop testing coverage. We probably need a better solution to how we deal with header logic migration in that case.

Here a few new lines added for python 3 to perform similar functionality. It basically hasn't touched/removed any original python 2 code.

Where does the Python 3 code test for a missing CRLF?

[1] Fix some TypeErrors. Calling get API in reuqest.headers (<class wptserve.request.RequestHeaders>) returns binary string values. In python 2, bytes is alias to str type. In Python 3, str type is different from bytes. This change uses utf-8 encoding for the request.headers values using for Py2 and Py3 compability. [2] Add python 3 support for accessing headers content in raw_headers (xhr/resources/inspect-headers.py). Type of raw_headers is class HTTPMessage. In Python 2, HTTPMessage derives from mimetools.Message while in Python 3, it derives from email.message.Message. Interfaces from mimetools.Message and email.message.Message are very different. In Python 3, HTTPMessage doesn't have 'headers" attribute or equivalent. [ref: https://bugs.python.org/issue4773] [3] Fix some imports for python2 and python 3 compatibility

ziransun · 2020-03-24T20:12:34Z

Code has been updated as per review comments. Please review. Thanks!

annevk · 2020-03-25T06:57:29Z

html/semantics/forms/form-submission-0/resources/form-submission.py

-        result = request.body == 'foo=bara'
-    elif request.headers.get('Content-Type') == 'text/plain':
-        result = request.body == 'qux=baz\r\n'
+    if ensure_str(request.headers.get('Content-Type')) == 'application/x-www-form-urlencoded':


Wouldn't it be easier to use b"" here (and below) on the other side?

Shouldn't this kind information be string?

annevk · 2020-03-25T06:58:45Z

tools/wptserve/wptserve/request.py

 import tempfile

+from six import BytesIO, binary_type, ensure_str, text_type, iteritems, PY3
+from six.moves.http_cookies import BaseCookie


Why was this moved? @jgraham should probably look at this change.

Reordered these lines due to pylint complains.

annevk · 2020-03-25T06:59:35Z

xhr/resources/inspect-headers.py

+    else:
+        header_list = raw_headers.headers
+    for line in header_list:
+        if not PY3:


In particular, it seems like the code isn't executed at all in Python 3.

jgraham · 2020-04-01T10:01:43Z

I think per the decision not to work on converting handlers until we have a better testing story, we should close this one for now. Feel free to reopen if I got that wrong.

…orting guide This is a rework on web-platform-tests#22402 This change follows the porting guide discussed in RFC49: web-platform-tests/rfcs#49

… guide (#23363) This is a rework on #22402 This change follows the porting guide discussed in RFC49: web-platform-tests/rfcs#49 Co-authored-by: Robert Ma <[email protected]>

… Python file handlers p…, a=testonly Automatic update from web-platform-tests Python 3: Port some html tests following Python file handlers porting guide (#23363) This is a rework on web-platform-tests/wpt#22402 This change follows the porting guide discussed in RFC49: web-platform-tests/rfcs#49 Co-authored-by: Robert Ma <[email protected]> -- wpt-commits: e60c41610959f414e24fe2360151d94953436906 wpt-pr: 23363

wpt-pr-bot added cookies html infra wptserve xhr labels Mar 23, 2020

wpt-pr-bot assigned KyleJu Mar 23, 2020

wpt-pr-bot requested review from annevk, caitp, domenic, foolip, Hexcles, jdm, jgraham, KyleJu, LukeZielinski, Manishearth, mikewest, stephenmcgruer, tkent-google, wisniewskit and zcorpan March 23, 2020 19:44

annevk reviewed Mar 24, 2020

View reviewed changes

ziransun force-pushed the html branch from 6a5786b to dba610c Compare March 24, 2020 20:05

wpt-pr-bot requested a review from zqzhang March 24, 2020 20:05

annevk reviewed Mar 25, 2020

View reviewed changes

stephenmcgruer assigned annevk and unassigned KyleJu Mar 30, 2020

jgraham closed this Apr 1, 2020

Hexcles mentioned this pull request Apr 7, 2020

RFC 49: Python 3 migration for wptserve web-platform-tests/rfcs#49

Merged

ziransun mentioned this pull request May 1, 2020

Python 3: Port some html tests following Python file handlers p… #23363

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

python 3: Add python 3 support for html tests #22402

python 3: Add python 3 support for html tests #22402

ziransun commented Mar 23, 2020

annevk left a comment

annevk Mar 24, 2020

annevk Mar 24, 2020

ziransun Mar 24, 2020

annevk Mar 25, 2020

ziransun Mar 25, 2020

annevk Mar 25, 2020

ziransun Mar 25, 2020

annevk Mar 26, 2020

ziransun commented Mar 24, 2020

annevk Mar 25, 2020

ziransun Mar 25, 2020

annevk Mar 25, 2020

annevk Mar 25, 2020

ziransun Mar 25, 2020

annevk Mar 25, 2020

jgraham commented Apr 1, 2020

python 3: Add python 3 support for html tests #22402

python 3: Add python 3 support for html tests #22402

Conversation

ziransun commented Mar 23, 2020

annevk left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ziransun commented Mar 24, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jgraham commented Apr 1, 2020