Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Recursion Error: maximum recursion depth exceeded while calling a Python object (First opening issue) #1072

Closed
quankhuc opened this issue Jul 7, 2023 · 4 comments
Closed

Recursion Error: maximum recursion depth exceeded while calling a Python object (First opening issue) #1072

quankhuc opened this issue Jul 7, 2023 · 4 comments
Assignees
@andrewsg
Labels
api: storage Issues related to the googleapis/python-storage API. type: question Request for information or clarification. Not an issue.

Comments

@quankhuc
Copy link

quankhuc commented Jul 7, 2023

I tried to create a simple get endpoint to test uploading a simple file to google cloud storage. When I tried to upload the file, I encounter the recursion error. I tried to use a tempfile, but it still gave me the same error. I have installed the latest google-cloud-storage package. I also tried to update all of my packages in my enviroment as well. However, the probelm still occurs. I would really appricate any guidances to get this to work.

Environment details

  • OS type and version: MacOSX on ARM M1
  • Python version: Python 3.11.3
  • pip version: pip 23.1.2
  • google-cloud-storage version: Version: 2.10.0

Steps to reproduce

  1. Create a quick start FastAPI project
  2. Upload an example file from file system to a google cloud storage bucket

Code from getting the file from the file system to upload

@live_trading_router.get("/test-sending-data-to-gcp")
async def test_sending_data_to_gcp():
    storage_client = storage.Client(credentials=service_account.Credentials.from_service_account_file("your-service-account.json"))
    bucket = storage_client.bucket("tradingbot-ml-models")
    blob = bucket.blob("test.txt")
    blob.upload_from_string("Hello World")
    return JSONResponse(status_code=200, content={"message": "Success"})

Code from creating a temp file to upload

@live_trading_router.get("/test-sending-data-to-gcp")
async def test_sending_data_to_gcp():
    # Create a temporary file with some content
    with tempfile.NamedTemporaryFile(mode="w") as temp_file:
        temp_file.write("Hello, World!")
        temp_file.flush()

        # Upload the temporary file to GCS
        storage_client = storage.Client.from_service_account_json("vertical-tuner-388917-60196f8a3475.json")
        bucket = storage_client.get_bucket("tradingbot-ml-models")
        blob = bucket.blob("test.txt")
        blob.upload_from_filename(temp_file.name)

    return JSONResponse(status_code=200, content={"message": "Success"})

Stack trace

INFO:     127.0.0.1:57907 - "GET /v1/live_trading/test-sending-data-to-gcp HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 435, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/fastapi/applications.py", line 282, in __call__
    await super().__call__(scope, receive, send)
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/starlette/middleware/cors.py", line 83, in __call__
    await self.app(scope, receive, send)
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/starlette/middleware/authentication.py", line 48, in __call__
    await self.app(scope, receive, send)
  File "/Users/quankhuc/Documents/GitHub/Trading-Bot-FastAPI/core/fastapi/middlewares/sqlalchemy.py", line 19, in __call__
    raise exception
  File "/Users/quankhuc/Documents/GitHub/Trading-Bot-FastAPI/core/fastapi/middlewares/sqlalchemy.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "/Users/quankhuc/Documents/GitHub/Trading-Bot-FastAPI/core/fastapi/middlewares/response_logger.py", line 37, in __call__
    await self.app(scope, receive, _logging_send)
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
    raise e
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/starlette/routing.py", line 66, in app
    response = await func(request)
               ^^^^^^^^^^^^^^^^^^^
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/fastapi/routing.py", line 241, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/fastapi/routing.py", line 167, in run_endpoint_function
    return await dependant.call(**values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/quankhuc/Documents/GitHub/Trading-Bot-FastAPI/api/v1/live_trading/live_trading.py", line 48, in test_sending_data_to_gcp
    blob.upload_from_string("Hello World")
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/google/cloud/storage/blob.py", line 2815, in upload_from_string
    self.upload_from_file(
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/google/cloud/storage/blob.py", line 2540, in upload_from_file
    created_json = self._do_upload(
                   ^^^^^^^^^^^^^^^^
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/google/cloud/storage/blob.py", line 2355, in _do_upload
    response = self._do_multipart_upload(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/google/cloud/storage/blob.py", line 1890, in _do_multipart_upload
    response = upload.transmit(
               ^^^^^^^^^^^^^^^^
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/google/resumable_media/requests/upload.py", line 153, in transmit
    return _request_helpers.wait_and_retry(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/google/resumable_media/requests/_request_helpers.py", line 155, in wait_and_retry
    response = func()
               ^^^^^^
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/google/resumable_media/requests/upload.py", line 145, in retriable_request
    result = transport.request(
             ^^^^^^^^^^^^^^^^^^
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/google/auth/transport/requests.py", line 545, in request
    self.credentials.before_request(auth_request, method, url, request_headers)
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/google/auth/credentials.py", line 151, in before_request
    self.refresh(request)
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/google/oauth2/service_account.py", line 434, in refresh
    access_token, expiry, _ = _client.jwt_grant(
                              ^^^^^^^^^^^^^^^^^^
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/google/oauth2/_client.py", line 312, in jwt_grant
    response_data = _token_endpoint_request(
                    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/google/oauth2/_client.py", line 272, in _token_endpoint_request
    response_status_ok, response_data, retryable_error = _token_endpoint_request_no_throw(
                                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/google/oauth2/_client.py", line 219, in _token_endpoint_request_no_throw
    request_succeeded, response_data, retryable_error = _perform_request()
                                                        ^^^^^^^^^^^^^^^^^^
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/google/oauth2/_client.py", line 195, in _perform_request
    response = request(
               ^^^^^^^^
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/google/auth/transport/requests.py", line 193, in __call__
    response = self.session.request(
               ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/urllib3/connectionpool.py", line 714, in urlopen
    httplib_response = self._make_request(
                       ^^^^^^^^^^^^^^^^^^^
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/urllib3/connectionpool.py", line 403, in _make_request
    self._validate_conn(conn)
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1053, in _validate_conn
    conn.connect()
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/urllib3/connection.py", line 400, in connect
    self.ssl_context = create_urllib3_context(
                       ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/site-packages/urllib3/util/ssl_.py", line 312, in create_urllib3_context
    context.options |= options
    ^^^^^^^^^^^^^^^
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/ssl.py", line 624, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/ssl.py", line 624, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/ssl.py", line 624, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
  [Previous line repeated 461 more times]
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/ssl.py", line 622, in options
    @options.setter
    
  File "/Users/quankhuc/anaconda3/envs/TradingBot/lib/python3.11/ssl.py", line 622, in options
    @options.setter
    
RecursionError: maximum recursion depth exceeded while calling a Python object
@product-auto-label product-auto-label bot added the api: storage Issues related to the googleapis/python-storage API. label Jul 7, 2023
@andrewsg andrewsg self-assigned this Jul 11, 2023
@andrewsg andrewsg added the type: question Request for information or clarification. Not an issue. label Jul 11, 2023
@andrewsg
Copy link
Contributor

Hi, thanks for reaching out. Looking at your traceback, it looks like you're calling storage; storage finds the client is unauthenticated and calls auth; auth calls requests; requests calls urllib3; and finally urllib3 calls the ssl module of the standard Python 3.11 library.

The ssl module then blows up by calling the options setter on SSLContext which fires super() recursively several hundred times.

I have to say I'm mystified with this behavior. The relevant code in SSLContext hasn't been changed for seven years (https://github.com/python/cpython/blame/3.11/Lib/ssl.py).

Unfortunately, as the Storage library doesn't seem to be doing anything here I don't have any real avenue to investigate this on my end. I would recommend looking at your Python, your urllib3, your requests and your Anaconda installs in that order. If you investigate and believe a Google client library is causing this issue (which is possible but doesn't seem very likely based on my initial reading of the traceback), please reach out to the googleapis/google-auth-library-python repository.

Best of luck with this truly bizarre issue. If you find out what happened, by all means please let me know!

@stdavis
Copy link

stdavis commented Jul 15, 2024

@quankhuc Did you ever solve this? I'm experiencing a similar issue.

@quankhuc
Copy link
Author

quankhuc commented Jul 16, 2024 •
edited
Loading

@quankhuc Did you ever solve this? I'm experiencing a similar issue.

Yes, but the solution might differ depending on your specific situation. The error message indicated that the request was formatted incorrectly and that Google had made too many attempts to process it, eventually stopping. In my case, converting the file to a CSV format resolved the problem. However, this might not be the solution for you. It's also possible that more than one functions provided by Google are being used incorrectly. I really wish the eco system was better at telling us what actually went wrong instead of an obscure message like this @stdavis

@stdavis
Copy link

stdavis commented Jul 16, 2024

I believe that this was my issue. Downgrading requests fixed my problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@andrewsg andrewsg

Labels
api: storage Issues related to the googleapis/python-storage API. type: question Request for information or clarification. Not an issue.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@andrewsg @stdavis @quankhuc