Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: updating testing, rest-only generation, & minor bug-fixes #716

Merged
merged 40 commits into from
Dec 30, 2020

Conversation

yon-mg
Copy link
Contributor

@yon-mg yon-mg commented Dec 21, 2020

Updating unit tests on generated client rest transport. Most changes made to unit tests are with compute api in mind and are not yet comprehensive enough for showcase.

Fixed issues with rest-only transport generation. Again, changes made with compute api in mind especially since compute api doesn't have grpc endpoint.

Fixed bug with attribute keyword conflicts.

The goal with this PR is to be able to generate a minimum usable GCE client library. Since rest transport is still under work, it is not generated by default and must be set with the transport flag --python_gapic_opt="transport=grpc+rest".

yon-mg and others added 30 commits November 2, 2020 17:49
@google-cla google-cla bot added the cla: yes This human has signed the Contributor License Agreement. label Dec 21, 2020
@yon-mg yon-mg marked this pull request as draft December 21, 2020 16:16
@yon-mg yon-mg changed the title fix: updating testing, rest-only generation, & minor bug-fixes fix(WIP): updating testing, rest-only generation, & minor bug-fixes Dec 21, 2020
@@ -10,14 +10,19 @@ import math
import pytest
from proto.marshal.rules.dates import DurationRule, TimestampRule

from requests import Response
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be under "if" statements?

@pytest.mark.parametrize("client_class", [{{ service.client_name }}, {{ service.async_client_name }}])
@pytest.mark.parametrize("client_class", [
{{ service.client_name }},
{%- if 'grpc' in opts.transport %}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This construct is used everywhere in the template. Since it is so commonly used, please consider simplifying it by introducing some helper method, such that it look something like:

{%- if grpc %}
stuff
{%- endif %}

@codecov
Copy link

codecov bot commented Dec 23, 2020

Codecov Report

Merging #716 (7058f62) into master (f646767) will not change coverage.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff            @@
##            master      #716   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           26        27    +1     
  Lines         1608      1619   +11     
  Branches       328       328           
=========================================
+ Hits          1608      1619   +11     
Impacted Files Coverage Δ
gapic/utils/case.py 100.00% <ø> (ø)
gapic/generator/generator.py 100.00% <100.00%> (ø)
gapic/schema/wrappers.py 100.00% <100.00%> (ø)
gapic/utils/__init__.py 100.00% <100.00%> (ø)
gapic/utils/checks.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f646767...7058f62. Read the comment docs.

@yon-mg yon-mg marked this pull request as ready for review December 23, 2020 17:33
import re


def is_str(expr: str) -> bool:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this file is used only for testing, please put it where the testing code is. Also please make sure that we don't do anything like this (guessting field type by its value, and using regexps for it), because it is not reliable enough and is quite slow (the regexp part).

@vam-google
Copy link
Contributor

@software-dov PTAL. This is the last PR which blocks us from generating the first Pyghon Alpha client for GCE!

@@ -0,0 +1,42 @@
# Copyright 2018 Google LLC
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use field.type instead field.mock_value in your test templates. By doing that you may avoid guessing work here, as we discussed over GVC. Also, please update copyright to 2020 here and in all your new files.

Copy link
Contributor

@busunkim96 busunkim96 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me (I am not an expert on the generator so deferring to other folks for approvals). I'm excited to try this out! 🚀

gapic/templates/noxfile.py.j2 Outdated Show resolved Hide resolved
Comment on lines +189 to +190
query_params = ['{k}={v}'.format(k=k, v=v) for k, v in query_params.items() if v]
url += '?{}'.format('&'.join(query_params)).replace(' ', '+')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be easier to use urllib.parse.urlencode here (It will create the k=v structure and glue the pairs together with & characters).

https://docs.python.org/3/library/urllib.parse.html#urllib.parse.urlencode
https://docs.python.org/3/library/urllib.request.html#urllib-examples

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure how I missed this. Thanks!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To clarify, I'll leave this out for now so I don't accidentally break anything but I'll update in the subsequent PR

@yon-mg yon-mg changed the title fix(WIP): updating testing, rest-only generation, & minor bug-fixes fix: updating testing, rest-only generation, & minor bug-fixes Dec 28, 2020
@yon-mg yon-mg merged commit 56c31de into googleapis:master Dec 30, 2020
Copy link
Contributor

@vchudnov-g vchudnov-g left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did a quick once-over. Looks good! Thanks for doing this.

Two very minor comments.

{% endif %}

{% if "next_page_token" in method.output.fields.values()|map(attribute='name') and not method.paged_result_field %}
{# Cheeser assertion to force code coverage for bad paginated methods #}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does "cheeser" mean in this context?

assert {i.name for i in cgr.file} == {
"foo/some_service/transports/river.py",
"foo/some_service/transports/car.py",
"foo/some_service/transports/__init__.py",
"foo/some_service/transports/base.py",
# Only generate async client with grpc transport
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: This comment looks as thought it applies to the next line, but is a bit confusing. Suggest moving it below the next line and tweaking it to be something like: # no async_client.py, as that's only for grpc transport

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla: yes This human has signed the Contributor License Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Re-add HTTP transport and make it the fallback
5 participants