Migrate to vehicles v2 API #462

rikroe · 2022-06-21T15:11:45Z

Breaking change

vehicle.doors_and_windows.convertible_top has been removed as we don't know how it is represented in the new API. It could be part of vehicle.doors_and_windows.lids if located similarly to the sunroof.

Proposed change

Using the demo vehicles feature in the MyBMW app I was able to piece together most of the changes for v2 of the vehicles API (currently used by MyBMW).

The hope is that this also solves the HTTP 429 issue we get from the old vehicles API.

There are some things missing/some tests failing due to missing data:

vehicle.doors_and_windows.convertible_top missing
everything related to charging time, i.e. when will the vehicle start/finish charging

Once #458 is merged and released, I'll mark some people who provided fingerprints before in this PR to get some additional data for these specific cases.

Type of change

Bugfix (non-breaking change which fixes an issue)
New feature (which adds functionality to this library)
Breaking change (fix/feature causing existing functionality to break)
Code quality improvements to existing code or addition of tests

Additional information

This PR fixes or closes issue: fixes #
This PR is related to issue:

Checklist

The code change is tested and works locally.
Tests have been added to verify that the new code works.

rikroe · 2022-06-22T13:16:27Z

Dear @k-janssens @a-n-r-o @teibaz @havrla @fabricioavil @mibo-wb-gh @jav4 @dicarsio @rich-eee!

As you all were kind enough to provide fingerprints in #327, we need your help again!
Please install the latest version (bimmer_connected==0.9.5) and get a fingerprint if your vehicle is in one the the following states:

Connected to a charger, but not charging (not in charging window)
Connected to a charger and charging

If your vehicle isn't charging, we probably don't need your fingerprint right now, we're at this point only curious about the charging state and charging start/end time.

@skrashevich could you also provide a fingerprint to get information about the convertible top?

There will be more files this time (both old and new endpoints). Please just upload all of the (but please make sure that they do not contain any information you don't like to share)!

Just add the files as answers to this pull request.

Many thanks!

havrla · 2022-06-22T13:33:34Z

Heloo
fingerptint exception:

[root@CL-ITNK-BMW ~]# scl enable rh-python38 'python3 /opt/rh/rh-python38/root/usr/local/lib/python3.8/site-packages/bimmer_connected/cli.py fingerprint [email protected] XXXXXXX rest_of_world '
DEBUG:asyncio:Using selector: EpollSelector
DEBUG:bimmer_connected.account:Getting vehicle list
DEBUG:bimmer_connected.api.authentication:Authenticating with MyBMW flow for North America & Rest of World.
DEBUG:httpx._client:HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-ucs/v1/presentation/oauth/config "HTTP/1.1 200 OK"
DEBUG:httpx._client:HTTP Request: POST https://customer.bmwgroup.com/gcdm/oauth/authenticate "HTTP/1.1 200 OK"
DEBUG:httpx._client:HTTP Request: POST https://customer.bmwgroup.com/gcdm/oauth/authenticate "HTTP/1.1 302 Found"
DEBUG:httpx._client:HTTP Request: POST https://customer.bmwgroup.com/gcdm/oauth/token "HTTP/1.1 200 OK"
DEBUG:httpx._client:HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v1/vehicles?apptimezone=120.0&appDateTime=1655904491740&tireGuardMode=ENABLED "HTTP/1.1 200 OK"
DEBUG:httpx._client:HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v1/vehicles?apptimezone=120.0&appDateTime=1655904491740&tireGuardMode=ENABLED "HTTP/1.1 429 Too Many Requests"
DEBUG:bimmer_connected.api.authentication:Sleeping 2 seconds due to 429 Too Many Requests
DEBUG:httpx._client:HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v1/vehicles?apptimezone=120.0&appDateTime=1655904491740&tireGuardMode=ENABLED "HTTP/1.1 429 Too Many Requests"
DEBUG:bimmer_connected.api.authentication:Sleeping 2 seconds due to 429 Too Many Requests
DEBUG:httpx._client:HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v1/vehicles?apptimezone=120.0&appDateTime=1655904491740&tireGuardMode=ENABLED "HTTP/1.1 429 Too Many Requests"
DEBUG:bimmer_connected.api.authentication:Sleeping 2 seconds due to 429 Too Many Requests
DEBUG:httpx._client:HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v1/vehicles?apptimezone=120.0&appDateTime=1655904491740&tireGuardMode=ENABLED "HTTP/1.1 429 Too Many Requests"
Traceback (most recent call last):
File "/opt/rh/rh-python38/root/usr/local/lib/python3.8/site-packages/bimmer_connected/cli.py", line 267, in
main()
File "/opt/rh/rh-python38/root/usr/local/lib/python3.8/site-packages/bimmer_connected/cli.py", line 263, in main
loop.run_until_complete(args.func(args))
File "/opt/rh/rh-python38/root/usr/lib64/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "/opt/rh/rh-python38/root/usr/local/lib/python3.8/site-packages/bimmer_connected/cli.py", line 135, in fingerprint
await account.get_vehicles()
File "/opt/rh/rh-python38/root/usr/local/lib/python3.8/site-packages/bimmer_connected/account.py", line 70, in get_vehicles
vehicles_responses: List[httpx.Response] = [
File "/opt/rh/rh-python38/root/usr/local/lib/python3.8/site-packages/bimmer_connected/account.py", line 71, in
await client.get(
File "/opt/rh/rh-python38/root/usr/local/lib/python3.8/site-packages/httpx/_client.py", line 1751, in get
return await self.request(
File "/opt/rh/rh-python38/root/usr/local/lib/python3.8/site-packages/httpx/_client.py", line 1527, in request
return await self.send(request, auth=auth, follow_redirects=follow_redirects)
File "/opt/rh/rh-python38/root/usr/local/lib/python3.8/site-packages/httpx/_client.py", line 1614, in send
response = await self._send_handling_auth(
File "/opt/rh/rh-python38/root/usr/local/lib/python3.8/site-packages/httpx/_client.py", line 1660, in _send_handling_auth
raise exc
File "/opt/rh/rh-python38/root/usr/local/lib/python3.8/site-packages/httpx/_client.py", line 1649, in _send_handling_auth
next_request = await auth_flow.asend(response)
File "/opt/rh/rh-python38/root/usr/local/lib/python3.8/site-packages/bimmer_connected/api/authentication.py", line 101, in async_auth_flow
response.raise_for_status()
File "/opt/rh/rh-python38/root/usr/local/lib/python3.8/site-packages/httpx/_models.py", line 736, in raise_for_status
raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Client error '429 Too Many Requests' for url 'https://cocoapi.bmwgroup.com/eadrax-vcs/v1/vehicles?apptimezone=120.0&appDateTime=1655904491740&tireGuardMode=ENABLED'
For more information check: https://httpstatuses.com/429
[root@CL-ITNK-BMW ~]#

status is ok

nilsvdb82 · 2022-06-22T13:49:33Z

Fingerprint X5 45e, connected to charger, not charging (fully charged a few minutes ago)

state_bmw_0.txt
vehicles_bmw_0.txt
vehicles_bmw_1.txt
WBATA000000C00000_bmw_0.txt

fabricioavil · 2022-06-22T13:53:00Z

Sure, glad to help! PTAL, let me know if you need anything else :)

Connected and NOT charging:

DEBUG:asyncio:Using selector: EpollSelector
DEBUG:bimmer_connected.account:Getting vehicle list
DEBUG:bimmer_connected.api.authentication:Authenticating with MyBMW flow for North America & Rest of World.
DEBUG:httpx._client:HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-ucs/v1/presentation/oauth/config "HTTP/1.1 200 OK"
DEBUG:httpx._client:HTTP Request: POST https://customer.bmwgroup.com/gcdm/oauth/authenticate "HTTP/1.1 200 OK"
DEBUG:httpx._client:HTTP Request: POST https://customer.bmwgroup.com/gcdm/oauth/authenticate "HTTP/1.1 302 Found"
DEBUG:httpx._client:HTTP Request: POST https://customer.bmwgroup.com/gcdm/oauth/token "HTTP/1.1 200 OK"
DEBUG:httpx._client:HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v1/vehicles?apptimezone=60.0&appDateTime=1655905274257&tireGuardMode=ENABLED "HTTP/1.1 200 OK"
DEBUG:httpx._client:HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v1/vehicles?apptimezone=60.0&appDateTime=1655905274257&tireGuardMode=ENABLED "HTTP/1.1 200 OK"
DEBUG:httpx._client:HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles "HTTP/1.1 200 OK"
DEBUG:httpx._client:HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles/VIN/state "HTTP/1.1 200 OK"
DEBUG:httpx._client:HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-crccs/v1/vehicles/VIN?fields=charging-profile&has_charging_settings_capabilities=true "HTTP/1.1 200 OK"
fingerprint of the vehicles written to /home/fabricioavil/vehicle_fingerprint/2022-06-22_14-41-14

2022-06-22_14-41-14 - connected and not charging.zip

Connected and charging:

DEBUG:asyncio:Using selector: EpollSelector
DEBUG:bimmer_connected.account:Getting vehicle list
DEBUG:bimmer_connected.api.authentication:Authenticating with MyBMW flow for North America & Rest of World.
DEBUG:httpx._client:HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-ucs/v1/presentation/oauth/config "HTTP/1.1 200 OK"
DEBUG:httpx._client:HTTP Request: POST https://customer.bmwgroup.com/gcdm/oauth/authenticate "HTTP/1.1 200 OK"
DEBUG:httpx._client:HTTP Request: POST https://customer.bmwgroup.com/gcdm/oauth/authenticate "HTTP/1.1 302 Found"
DEBUG:httpx._client:HTTP Request: POST https://customer.bmwgroup.com/gcdm/oauth/token "HTTP/1.1 200 OK"
DEBUG:httpx._client:HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v1/vehicles?apptimezone=60.0&appDateTime=1655905821705&tireGuardMode=ENABLED "HTTP/1.1 200 OK"
DEBUG:httpx._client:HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v1/vehicles?apptimezone=60.0&appDateTime=1655905821705&tireGuardMode=ENABLED "HTTP/1.1 200 OK"
DEBUG:httpx._client:HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles "HTTP/1.1 200 OK"
DEBUG:httpx._client:HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-vcs/v2/vehicles/VIN/state "HTTP/1.1 200 OK"
DEBUG:httpx._client:HTTP Request: GET https://cocoapi.bmwgroup.com/eadrax-crccs/v1/vehicles/VIN?fields=charging-profile&has_charging_settings_capabilities=true "HTTP/1.1 200 OK"
fingerprint of the vehicles written to /home/fabricioavil/vehicle_fingerprint/2022-06-22_14-50-21

2022-06-22_14-50-21 - connected and charging.zip

rikroe · 2022-06-22T14:20:15Z

Thanks all! Unfortunately, not the one we're needing: Basically, we need a fingerprint when the BMW app either displays "start charging at ..." or "finished charging at ..." (don't know how it is actually called, but you know what I mean).

@havrla argh this stupid 429 error... If possible, try the following: from cli.py, delete or comment out the whole line 136 (await account.get_vehicles()). The error shouldn't occur anymore in this case.
@havrla thanks for your mail, should be fine!

jav4 · 2022-06-22T15:27:17Z

Here you have now. It's currently charging...

state_bmw_0.txt
vehicles_bmw_0.txt
vehicles_bmw_1.txt
vehicles_mini_0.txt
WBA0P000X0FL00000_bmw_0.txt

codecov-commenter · 2022-06-22T15:58:28Z

Codecov Report

Merging #462 (283a7b7) into master (d751fea) will decrease coverage by 0.07%.
The diff coverage is 99.35%.

@@             Coverage Diff             @@
##            master     #462      +/-   ##
===========================================
- Coverage   100.00%   99.92%   -0.08%     
===========================================
  Files           16       16              
  Lines         1259     1256       -3     
===========================================
- Hits          1259     1255       -4     
- Misses           0        1       +1

Flag	Coverage Δ
3.10.x	`99.92% <99.35%> (-0.08%)`	⬇️
3.6	`99.83% <99.35%> (-0.17%)`	⬇️
3.7	`99.83% <99.35%> (-0.17%)`	⬇️
3.8	`99.83% <99.35%> (-0.17%)`	⬇️
3.9	`99.91% <99.35%> (-0.09%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
bimmer_connected/api/client.py	`100.00% <ø> (ø)`
bimmer_connected/vehicle/fuel_and_battery.py	`99.05% <98.24%> (-0.95%)`	⬇️
bimmer_connected/account.py	`100.00% <100.00%> (ø)`
bimmer_connected/const.py	`100.00% <100.00%> (ø)`
bimmer_connected/models.py	`100.00% <100.00%> (ø)`
bimmer_connected/vehicle/charging_profile.py	`100.00% <100.00%> (ø)`
bimmer_connected/vehicle/doors_windows.py	`100.00% <100.00%> (ø)`
bimmer_connected/vehicle/location.py	`100.00% <100.00%> (ø)`
bimmer_connected/vehicle/reports.py	`100.00% <100.00%> (ø)`
bimmer_connected/vehicle/vehicle.py	`100.00% <100.00%> (ø)`
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d751fea...283a7b7. Read the comment docs.

rikroe · 2022-06-22T16:24:27Z

Thanks guys, you're awesome!

I'll follow up on some other test related issue (but the main thing is done) and waiting for @skrashevich regarding the convertible top!

nilsvdb82 · 2022-06-22T17:00:44Z

Connected to charger, waiting for charging time window

state_bmw_0.txt
vehicles_bmw_0.txt
vehicles_bmw_1.txt
WBATA000000C00000_bmw_0.txt

rikroe · 2022-06-23T08:05:19Z

As this seems to mitigate the 429 issues, I'd rather have this merged & published soon.
The missing convertibleTop sensor can be added again afterwards.

github-actions · 2022-07-03T03:53:05Z

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

rikroe force-pushed the vehicles-v2 branch from efc6faf to 353aa4a Compare June 21, 2022 15:30

rikroe added 3 commits June 22, 2022 14:05

Migrate to vehicles v2 API

7fb24f1

Make pytest run at all on Py<3.9

31065f7

Don't fail on UNKOWN enum value

a48c991

rikroe force-pushed the vehicles-v2 branch from 1648cf0 to a48c991 Compare June 22, 2022 12:05

Adjustments for pytest

f300328

Updates for charging start/end time

45a9967

Fix zoneinfo test issue

283a7b7

Update tests for charging start/end time

c546a6f

rikroe mentioned this pull request Jun 22, 2022

BMW Connected Drive error 429 home-assistant/core#73370

Closed

rikroe changed the title ~~[WIP] Migrate to vehicles v2 API~~ Migrate to vehicles v2 API Jun 23, 2022

rikroe requested a review from gerard33 June 23, 2022 08:04

rikroe marked this pull request as ready for review June 23, 2022 08:04

rikroe added 2 commits June 23, 2022 10:40

Update fingerprint to only use new endpoints

64b0b0d

Update fingerprint to use vehicle brand

bd6181c

rikroe mentioned this pull request Jun 23, 2022

Fingerprint command fails while fetching charging session #459

Closed

4 tasks

gerard33 previously approved these changes Jun 23, 2022

View reviewed changes

Remove special ZoneInfo handling due to time_machine bugfix

a675d08

rikroe dismissed gerard33’s stale review via a675d08 June 24, 2022 18:37

Apply time_machine fix only for Py>=3.7

2057810

Fix time_machine requirements on Python 3.6

85f8ffe

rikroe requested a review from gerard33 June 24, 2022 18:50

gerard33 approved these changes Jun 24, 2022

View reviewed changes

rikroe merged commit a1e6c36 into bimmerconnected:master Jun 25, 2022

github-actions bot locked and limited conversation to collaborators Jul 3, 2022

rikroe deleted the vehicles-v2 branch July 9, 2022 09:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate to vehicles v2 API #462

Migrate to vehicles v2 API #462

rikroe commented Jun 21, 2022 •

edited

Loading

rikroe commented Jun 22, 2022

havrla commented Jun 22, 2022

nilsvdb82 commented Jun 22, 2022

fabricioavil commented Jun 22, 2022 •

edited by rikroe

Loading

rikroe commented Jun 22, 2022 •

edited

Loading

jav4 commented Jun 22, 2022

codecov-commenter commented Jun 22, 2022 •

edited

Loading

rikroe commented Jun 22, 2022

nilsvdb82 commented Jun 22, 2022

rikroe commented Jun 23, 2022

github-actions bot commented Jul 3, 2022

Migrate to vehicles v2 API #462

Migrate to vehicles v2 API #462

Conversation

rikroe commented Jun 21, 2022 • edited Loading

Breaking change

Proposed change

Type of change

Additional information

Checklist

rikroe commented Jun 22, 2022

havrla commented Jun 22, 2022

nilsvdb82 commented Jun 22, 2022

fabricioavil commented Jun 22, 2022 • edited by rikroe Loading

rikroe commented Jun 22, 2022 • edited Loading

jav4 commented Jun 22, 2022

codecov-commenter commented Jun 22, 2022 • edited Loading

Codecov Report

rikroe commented Jun 22, 2022

nilsvdb82 commented Jun 22, 2022

rikroe commented Jun 23, 2022

github-actions bot commented Jul 3, 2022

rikroe commented Jun 21, 2022 •

edited

Loading

fabricioavil commented Jun 22, 2022 •

edited by rikroe

Loading

rikroe commented Jun 22, 2022 •

edited

Loading

codecov-commenter commented Jun 22, 2022 •

edited

Loading