Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proxy behavior is inconsistent #2981

Closed
voidfyoo opened this issue May 7, 2018 · 3 comments · Fixed by #2985
Closed

proxy behavior is inconsistent #2981

voidfyoo opened this issue May 7, 2018 · 3 comments · Fixed by #2985
Labels

Comments

@voidfyoo
Copy link

voidfyoo commented May 7, 2018

Long story short

request proxy behavior is inconsistent within one session:

import asyncio

import aiohttp


class ProxyCrawler:
    def __init__(self):
        loop = asyncio.get_event_loop()
        self.session = aiohttp.ClientSession(loop=loop)

    async def req(self):
        async with self.session.get('http://httpbin.org/ip') as resp:
            text = await resp.text()
            print('First request:')
            print(text)

        async with self.session.get('http://httpbin.org/ip', proxy='http://127.0.0.1:1087') as resp:
            text = await resp.text()
            print('Second request:')
            print(text)

        await self.session.close()


if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    c = ProxyCrawler()
    loop.run_until_complete(c.req())

Expected behaviour

The first request goes directly, and the second request goes through the specified proxy.

Actual behaviour

Both requests go directly without proxy.

( But if you set the first request go through a proxy, the second request will go through the specified proxy correctly. )

Steps to reproduce

Run the code above ( replace the proxy with yours ).

Your environment

aiohttp: 3.2.0
Mac OS X

@asvetlov
Copy link
Member

asvetlov commented May 7, 2018

Thanks for the report.
ClientRequest.connection_key should include proxy attribute.
I see the problem with custom SSL certificate also: we need to check SSL context as well.

asvetlov added a commit that referenced this issue May 8, 2018
asvetlov added a commit that referenced this issue May 10, 2018
…ttings (#2985)

* Fix #2981: Don't reuse a connection with the same URL but different proxy/TLS settings

* Add TLS to white list

* Fix broken TLS over proxy
asvetlov added a commit that referenced this issue May 10, 2018
…ttings (#2985)

* Fix #2981: Don't reuse a connection with the same URL but different proxy/TLS settings

* Add TLS to white list

* Fix broken TLS over proxy
@asvetlov
Copy link
Member

Fixed in aiohttp 3.2.1 release

kornicameister referenced this issue in kornicameister/korni-stats-collector Jun 7, 2018
This PR updates [aiohttp](https://pypi.org/project/aiohttp) from **3.2.1** to **3.3.1**.



<details>
  <summary>Changelog</summary>
  
  
   ### 3.3.0
   ```
   ==================

Features
--------

- Raise ``ConnectionResetError`` instead of ``CancelledError`` on trying to
  write to a closed stream. (`2499 &lt;https://github.com/aio-libs/aiohttp/pull/2499&gt;`_)
- Implement ``ClientTimeout`` class and support socket read timeout. (`2768 &lt;https://github.com/aio-libs/aiohttp/pull/2768&gt;`_)
- Enable logging when ``aiohttp.web`` is used as a program (`2956 &lt;https://github.com/aio-libs/aiohttp/pull/2956&gt;`_)
- Add canonical property to resources (`2968 &lt;https://github.com/aio-libs/aiohttp/pull/2968&gt;`_)
- Forbid reading response BODY after release (`2983 &lt;https://github.com/aio-libs/aiohttp/pull/2983&gt;`_)
- Implement base protocol class to avoid a dependency from internal
  ``asyncio.streams.FlowControlMixin`` (`2986 &lt;https://github.com/aio-libs/aiohttp/pull/2986&gt;`_)
- Cythonize ``helpers.reify``, 5% boost on macro benchmark (`2995 &lt;https://github.com/aio-libs/aiohttp/pull/2995&gt;`_)
- Optimize HTTP parser (`3015 &lt;https://github.com/aio-libs/aiohttp/pull/3015&gt;`_)
- Implement ``runner.addresses`` property. (`3036 &lt;https://github.com/aio-libs/aiohttp/pull/3036&gt;`_)
- Use ``bytearray`` instead of a list of ``bytes`` in websocket reader. It
  improves websocket message reading a little. (`3039 &lt;https://github.com/aio-libs/aiohttp/pull/3039&gt;`_)
- Remove heartbeat on closing connection on keepalive timeout. The used hack
  violates HTTP protocol. (`3041 &lt;https://github.com/aio-libs/aiohttp/pull/3041&gt;`_)
- Limit websocket message size on reading to 4 MB by default. (`3045 &lt;https://github.com/aio-libs/aiohttp/pull/3045&gt;`_)


Bugfixes
--------

- Don&#39;t reuse a connection with the same URL but different proxy/TLS settings
  (`2981 &lt;https://github.com/aio-libs/aiohttp/pull/2981&gt;`_)
- When parsing the Forwarded header, the optional port number is now preserved.
  (`3009 &lt;https://github.com/aio-libs/aiohttp/pull/3009&gt;`_)


Improved Documentation
----------------------

- Make Change Log more visible in docs (`3029 &lt;https://github.com/aio-libs/aiohttp/pull/3029&gt;`_)
- Make style and grammar improvements on the FAQ page. (`3030 &lt;https://github.com/aio-libs/aiohttp/pull/3030&gt;`_)
- Document that signal handlers should be async functions since aiohttp 3.0
  (`3032 &lt;https://github.com/aio-libs/aiohttp/pull/3032&gt;`_)


Deprecations and Removals
-------------------------

- Deprecate custom application&#39;s router. (`3021 &lt;https://github.com/aio-libs/aiohttp/pull/3021&gt;`_)


Misc
----

- 3008, 3011
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/aiohttp
  - Changelog: https://pyup.io/changelogs/aiohttp/
  - Repo: https://github.com/aio-libs/aiohttp
</details>
@lock
Copy link

lock bot commented Oct 28, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a [new issue] for related bugs.
If you feel like there's important points made in this discussion, please include those exceprts into that [new issue].
[new issue]: https://github.com/aio-libs/aiohttp/issues/new

@lock lock bot added the outdated label Oct 28, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Oct 28, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants