Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Envoy Retry Policy Support #1358

Merged
merged 13 commits into from
May 7, 2019
Merged

Conversation

LiVe
Copy link
Contributor

@LiVe LiVe commented Mar 22, 2019

Description

Adds support für envoys retry_policy to ambassador module and http mappings.

Related Issues

#1127

Testing

  • manual tests
  • deployed to development environment
  • started adding automated tests, i'd assume this needs more work

Todos

  • [ X ] Tests (check what needs to be done)
  • [ X ] Documentation

@kflynn
Copy link
Member

kflynn commented Mar 26, 2019

Hey, thanks for this! I'm afraid I've caused some trouble for you by dropping the old Envoy-V1 stuff and the old end-to-end tests -- can you take care of the merge conflicts (probably be deleting things) and we can talk further about tests? Are you on our Slack?

@richarddli
Copy link
Contributor

@containscafeine does this touch the work you're looking at right now?

@concaf
Copy link
Contributor

concaf commented Apr 4, 2019

@richarddli not directly, but this will work well in conjunction to circuit breakers 🎉

LiVe added 3 commits April 4, 2019 09:11
# Conflicts:
#	ambassador/templates/envoy.j2
#	ambassador/tests/000-default/gold.intermediate.json
@LiVe
Copy link
Contributor Author

LiVe commented Apr 4, 2019

@kflynn Hey, sorry for the late reply, busy days. i've merged the latest changes and removed the old / unnecessary files. i've also looked at the test descriptions and moved the old end to end test to a mapping test.

and yes, i'm on the datawire slack too if you want to continue the discussion over there :)

Copy link
Contributor

@concaf concaf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good for most part. Need some clarification on the test.
Great work, thanks for the contribution 🎉

@@ -236,6 +237,14 @@ def setup(self, ir: 'IR', aconf: Config) -> bool:
else:
return False

if amod and ('retry_policy' in amod):
self.retry_policy = IRRETRYPOLICY(ir=ir, aconf=aconf, location=self.location, **amod.retry_policy)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will be helpful if we can validate the retry policy here. Look at validate_load_balancer for reference 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i've added this to the IRRetryPolicy class and validated retry_on in setup. would that be a correct approach to this?

from .ir import IR


class IRRETRYPOLICY (IRResource):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: change this to IRRetryPolicy
I believe you must for gotten confused with IRCORS being all caps, but that's because of CORS being in caps. The rest of the classes are camel case ;)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah yes, i guess the ircors got me there you're right. i've changed this.

spec:
containers:
- name: retry
image: live/ambassador-retry-service:1.0.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mind committing the Dockerfile for this image with this PR as well? :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

of course. initially i've added the code and dockerfile to the now unused end-to-end tests (but thats now excluded via gitignore). where should i put those files?

""")

def queries(self):
yield Query(self.parent.url("%s/retry/" % self.name), expected=200)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain a bit more on how does this test the retry policy behavior? How do we know that it's retrying on 5xx?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'm not sure if this test fits any requirements or best practices at all, but i'd need a bit help / guidance on implementing a solid testing logic for this one (i'm lacking a bit of experience here).

when i've added the changes, all i wanted todo was to check if it retries failed requests. so i just wrote a simple service that flips between a 503 and 200 response (first one fails, second one succeeds, and so on). with the retry policy in place, a query should always succeed on this service, therefor expecting the 200. With multiple (parallel) tests the flipping responses, as they are now, would probably cause issues though (maybe running multiple queries would be enough?). but i guess that there is a better solution to this?

@concaf concaf mentioned this pull request May 2, 2019
@concaf
Copy link
Contributor

concaf commented May 2, 2019

@LiVe I've made some testing and docs changes at #1486, besides your commits. Do you want to take a look there? ;)

@flands
Copy link
Contributor

flands commented May 2, 2019

Can this be set once globally instead of per mapping?

@concaf
Copy link
Contributor

concaf commented May 3, 2019

@flands yes!

@kflynn kflynn merged commit 0569605 into emissary-ingress:master May 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants