You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The SDK seems not to be able to upload large files (e.g. 5GB). I'm getting
Traceback (most recent call last):
File "/home/azureuser/.local/lib/python3.8/site-packages/cognite/client/_http_client.py", line 133, in _do_request
res = self.session.request(method=method, url=url, **kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 498, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', OSError("(104, 'ECONNRESET')"))
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "upload.py", line 17, in <module>
client.files.upload("data_25M_0.csv", external_id="file_25M_0", name="file with 1st 25M items", overwrite=True)
File "/home/azureuser/.local/lib/python3.8/site-packages/cognite/client/_api/files.py", line 592, in upload
return self._upload_file_from_path(file_metadata, path, overwrite)
File "/home/azureuser/.local/lib/python3.8/site-packages/cognite/client/_api/files.py", line 619, in _upload_file_from_path
file_metadata = self.upload_bytes(fh, overwrite=overwrite, **file.dump())
File "/home/azureuser/.local/lib/python3.8/site-packages/cognite/client/_api/files.py", line 698, in upload_bytes
self._http_client_with_retry.request(
File "/home/azureuser/.local/lib/python3.8/site-packages/cognite/client/_http_client.py", line 122, in request
raise e
File "/home/azureuser/.local/lib/python3.8/site-packages/cognite/client/_http_client.py", line 108, in request
res = self._do_request(method=method, url=url, **kwargs)
File "/home/azureuser/.local/lib/python3.8/site-packages/cognite/client/_http_client.py", line 151, in _do_request
raise CogniteConnectionError from e
cognite.client.exceptions.CogniteConnectionError
Describe the solution you'd like
The SDK should be able to deal with large files the same way it works with smaller files.
Describe alternatives you've considered
The solution right now is to get upload link from the API https://docs.cognite.com/api/v1/#tag/Files/operation/initFileUpload
Then use custom tool to upload the local file with the upload link. For example, using azcopy to upload to Azure environment.
The text was updated successfully, but these errors were encountered:
The SDK seems not to be able to upload large files (e.g. 5GB). I'm getting
Describe the solution you'd like
The SDK should be able to deal with large files the same way it works with smaller files.
Describe alternatives you've considered
The solution right now is to get upload link from the API https://docs.cognite.com/api/v1/#tag/Files/operation/initFileUpload
Then use custom tool to upload the local file with the upload link. For example, using azcopy to upload to Azure environment.
The text was updated successfully, but these errors were encountered: