manually start/stop caused incorrect trips to be stored #693
Comments
|
Also I'm using a Samsung Galaxy S7 phone without a SIM card. Is a SIM card required to make it work? |
|
I have test phones without a SIM card and they work reasonably fine although they sometimes have a large gap at the start. The developer zone start and stop is not really designed for manual trip tracking like Strava, that's why it is in the developer zone instead of in the main app functionality. Once you have started the trip using the manual start, it will continue trying to detect a trip end as in the original FSM. You can attach the logs, or you can expand them yourself (https://github.com/e-mission/e-mission-phone/blob/42706f3f40423f91d02db443e011f45b8385505d/README.md#beta-testing-debugging) and search for you can then see why we apparently detected a trip end after 2km. |
|
Thanks for the tip. Here is the logger for 2 trips yesterday. It started at around 12:02 PM and finished at 12:24 PM, and then at 13:40 PM and finished at 14:04 PM: The trips that show up in the Diary started 12:04 PM and ended 12:09 PM, then started at 1:50 PM and ended at 2:01 PM. So they were the beginning of the first trip and the end of the second trip. The call to isTripEnded happened around 12:05 PM but the trip in the Diary didn't end at that time: The trip ended at 12:09 PM but this is what's in the log: Does this tell you anything to help understand what's going on? |
|
@sichen1234 as you can see from the logs, we only read 8 locations in that entire duration and there was a big gap between noon, 1pm and 2pm. That's why we created two trips, one around noon and the other around 13:40. This is due to background restrictions in sending locations to the app. This is a known issue for Samsung phones - e.g. https://dontkillmyapp.com/ Prior investigations of similar issues are:
|
|
That was helpful. Thank you. So I went to the Battery in my Settings, Advanced Settings, and turned off App Power Monitor. Now it is recording a lot more GPS locations: Still I was out between 3:07 PM, stopped in the middle, and came back around 3:30 PM. It only recorded a trip from 3:37 PM that ended at 4:01 PM well after I had stopped. Is the log showing how it's deciding when a trip starts and ends? |
Yup! In general, you can grep for the state transitions in the finite state machine. You can see the details of the FSM in my thesis. There are typically geofence exit logs before the And the "didn't move a lot in the last 10 minutes" checks before we detected that we stopped moving These are consistent with your ground truth, so the issue must be with the actual points sensed at that time. Looking at the first sensed point, it has accuracy 300m, which is pretty bad. Grepping further, we see that you consistently get bad points until The algorithm ignores bad quality points (to avoid irregular trajectories) so the trip appears to start around 15:40. I don't have a Galaxy S7, but when I tested with the Galaxy Edge S6, I had to turn off all power saving for the entire phone for decent tracking (#535 (comment)). See if there are additional power saving settings on your phone and if so, please disable them. Sorry this is so painful, but there's a reason why Samsung phones are top on the dontkillmyapp list. Also, the instructions on the dontkillmyapp site are not always perfect. On the S9, another user also had to turn off "Medium Power Saving Mode" #528 (comment) although it is not listed. So I think it is just a question of turning off optimizations until the tracking works properly. |
|
Note also that when we get the "bad quality" points, they are identical down to the 6th place after the decimal point. That's another characteristic of bad quality points - it is pretty clear that the OS is returning some cached value instead of actually sensing the location points... |
|
Thank you so much for your help with this. I set the phone to connect to wifi via a hotspot and use it along with GPS. I think the location readings are now much more accurate. I used the phone's google map navigation during the morning trip, so the locations for the morning trip were pretty acurate: On the way back, the google map was not running, so the locations were not as accurate. I took exactly the same path as in the first screenshot. So my questions are:
So why didn't it decide to end trip until 12:17:
Is the google map running causing it to read more locations?
Thanks for all your help. This is really interesting :) |
|
Google navigation definitely causes it to read more locations (see section 5.2.1). The android location API performs sensor batching under the hood. And since the google map navigation is in the foreground, not the background, the background restrictions don't apply to it. Note that, unlike traditional sensors, we don't control the hardware or the underlying OS. We get what we get from android/iOS and we need to do the best that we can with it. This says that we don't have enough points to decide whether we have stopped ( And indeed, after getting points every 30 secs consistently, we have a 10 min gap between 11:43 and 11:53. Around 12:17, there were enough points (10 + 4) and the distances in both were < 100m, so we detected a stop. Basically, the app requests location updates every 30 seconds so we can detect when the trip had ended. If the phone doesn't do this, the FSM will not work. Again, I would suggest turning off medium power accuracy (Settings -> Device Management -> Battery) if you want the tracking to work properly. And/or use a different phone, one running stock android, such as a Pixel.
WiFi location detection relies on mapping WiFi signal strengths to known locations. So I am not sure this will help since the phone will just see that it is connected to the same WiFi network; it won't know that the "network" is actually a hotspot which is also moving. I have test phones with no SIM cards and no WiFi hotspot and:
|
|
@sichen1234 making the plug for turning off power saving completely, please see my previous results with Samsung phones: The difference between the "mid" and "off" is essentially what you are seeing above. |
|
Thanks. I did turn off all power savings in the Samsung.
Do you know if the iPhone is better? I can try that as well.
Also is it the same reason why it didn’t think my second trip ended until
15:50 even though I was at the same location since 12:53?
On Sat, Dec 11, 2021 at 9:22 AM shankari ***@***.***> wrote:
@sichen1234 <https://github.com/sichen1234> making the plug for turning
off power saving completely, please see my previous results with Samsung
phones:
#535 (comment)
<#535 (comment)>
The difference between the "mid" and "off" is essentially what you are
seeing above.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#693 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AANAS4KAUPWOMGFOASGYC5DUQOCEJANCNFSM5JW6QNZA>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
--
-----
Si Chen
Open Source Strategies, Inc.
Free e-Book: Open Climate Investing
<https://climate-investing-book.opensourcestrategies.com/v/main/book>
|
@sichen1234 Could you send screenshots of your current power settings so I can confirm this?
iPhone and (as I said earlier) Pixel (=stock android) will not have unexpected behavior. Even on Samsung, we have generally been able to track trips properly after we really turn off all the power settings.
Yes. At around 12:30 we were getting points multiple times a minute. After that, we went to on update roughly every 10 minutes. and continued at that frequency.... The app requests locations every 30 seconds, and expects to receive locations roughly every 30 secs. If it receives locations every 10 minutes, the end detection algorithm won't necessarily work properly. |
|
This is where I turned off my battery settings -- is it correct? Does the log show why the app switched to requesting locations every 10 minutes? |
|
That checks the boxes on "off" power saving mode, and turned off App power monitor.
The app continues to request locations every 30 seconds. The phone OS (android) does not provide locations at the requested frequency because of background restrictions. |
|
Yes, I did turn off the "Optimize Battery Usage": I don't see separate screens for Device maintenace -> Battery -> Unmonitored apps and Device Management -> Power. My last screenshot is for Device Maintenance > Battery, and I've turned Power saving mode to OFF (not even MID from #535. The only thing that sounds like "Device maintenace -> Battery -> Unmonitored apps" is that option for App power monitor for me. My Samsung Galaxy s7 is on the factory default Android 8.0 |
|
yes, you definitely need "off" see the specific comment in #535 that I linked to.
Hm. that's weird. There are specific restrictions on android 8.0+ that send "location updates only a few times an hour" (https://developer.android.com/about/versions/oreo/background-location-limits). We need to start a foreground service along with a persistent notification to avoid this. But I'd already checked out the logs and we weren't starting the foreground service because we detected that this is Pre-Oreo.... |
|
Wait a minute, on line 175, it looks like we are checking for > OREO instead of >= OREO Gah!!! And I notice that you don't have any foreground notifications in any of your screenshots. @sichen1234 can you:
and see if that fixes the problem. A cheaper, non-code, non-upgrade check is to turn tracking off and on just before starting your trip. Basically, you need to have a notification on the left hand side of the notification bar which says "Ready to take a trip" or "Yay! You are on a trip!". Then the tracking will be fine. If this is indeed the problem, you've found a very subtle bug, thank you, and I will roll the fix into the current set of data collection plugin changes. I have tested on Oreo phones before (e.g. #567) but since they were on personal phones, I guess the phones had been upgraded to 8.1 instead of 8.0. |
|
Indeed I do have Android 8.0.0: How do I " to turn tracking off and on just before starting your trip."? What does this mean? "have a notification on the left hand side of the notification bar which says "Ready to take a trip" or "Yay! You are on a trip!". " Should I see these as notifications from e-mission in my notification panel? |
In the e-mission app, Profile -> Tracking toggle
Yup, on the left side of the notification panel (see the location icon next to the CanBikeCO icon in the screenshot below)
|
|
@sichen1234 were you able to get the notification to show up, and confirm that the location trajectory is accurate in that case? If so, we can rename this issue to the fix for android 8.0 ( |
|
I tried start and stop tracking, and it didn't work. I'm going to test it with the code change and let you know. |
|
@sichen1234 when you turned the tracking off and on, did the notification for the foreground service show up? |
|
Sorry I lost those logs when I installed the updated version. I'll try it again and get the logs. |
|
@sichen1234 sounds good. If you haven't bumped up the version number for the updated version, you need to turn tracking off and on to reinitialize the FSM. If you have bumped up the version number (which you would need to for publishing the app, we would do it automatically) |
|
@sichen1234 Also, please note whether the foreground service notification does show up this time or not. |
|
Here is an updated log: I think it is pushing in notifications regularly now. Yesterday there was a walking trip from about 10:40 to 10:55 which didn't get recorded. Today it recorded the driving trips in the morning. It's gathering a lot more location reads so maybe the trip start/stop records are more a function of the calibration of how it's determining the trips? |
|
@sichen1234 the start and stop tracking is not intended for use on every trip. It is intended for use when you don't want to be tracked for a period of time - e.g. when you are going on a multi-day backpacking trip and want to minimize battery drain. Once you have started tracking, and the foreground notification is displayed, the trip start/end detection should be automatic. In the case of the 10:40 walking trip, the trip started properly with a geofence exit But at 10:55, it looks like you manually turned tracking off and on. So we went never detected "stopped moving" and did not segment the trip correctly. Again, the start/stop tracking suggestion was a hack to get the foreground service notification to show up without code changes. With the code change, the notification should be on all the time in the top left part of your screen. And the trip start/stops should also be automatically detected.
The foreground service notification should stay on continuously, not pushed periodically. |
|
Looking at the foreground service checks, it looks like it has indeed been on since 11:13 yesterday. and there were never zero notifications |
|
Ah, and it looks like today morning you stopped doing the manual start/stop tracking, so we detected the trip start/end correctly. To recap, with the code fix, when the foreground service notification is active, you don't need to do anything special. Just let the app detect trips automatically. |
|
I tested it some more this weekend. It is working a lot better. So I think your fix for Android 8.0.0 is good and we can close this issue. |
|
@sichen1234 In #693 (comment) I said that the phone works well if it doesn't have a SIM card. That is true if it is in a WiFi-rich environment, since android uses fused location that combines GPS, WiFi and cellular. But if you go to a location without a lot of WiFi routers (like I did while hiking today, @lgharib), and don't have a SIM card that can access cellular networks, then the trip start is detected only when I am almost home. I should get a SIM card for my test phone and go hiking in the same location again 😄 |
|
Closing this issue for now. |
We were using `>` in this one location, which meant that if the version was exactly 8.0, then the foreground service would not be restarted when killed. Tested by @sichen1234 as part of e-mission/e-mission-docs#693 ``` ../plugins/e-mission-data-collection/src/android/location/TripDiaryStateMachineForegroundService.java|90| if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) { ../plugins/e-mission-data-collection/src/android/location/TripDiaryStateMachineForegroundService.java|106| if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) { ../plugins/e-mission-data-collection/src/android/location/TripDiaryStateMachineForegroundService.java|163| if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) { ../plugins/e-mission-data-collection/src/android/location/TripDiaryStateMachineForegroundService.java|175| if(Build.VERSION.SDK_INT >Build.VERSION_CODES.O) { ../plugins/e-mission-data-collection/src//android/verification/SensorControlChecks.java|85| if (android.os.Build.VERSION.SDK_INT >= android.os.Build.VERSION_CODES.O) { ``` This fixes e-mission/e-mission-docs#693
|
@sichen1234 can you confirm that I can then close this issue. |
|
Yes that was the change we made. Thanks again.
-----
Si Chen
Open Source Strategies, Inc.
Free e-Book: Open Climate Investing
<https://climate-investing-book.opensourcestrategies.com/v/main/book>
…On Mon, Dec 20, 2021 at 8:18 PM shankari ***@***.***> wrote:
@sichen1234 <https://github.com/sichen1234> can you confirm that
***@***.***
<e-mission/e-mission-data-collection@3f48d1f>
is the change that you made to get this to work?
I can then close this issue.
—
Reply to this email directly, view it on GitHub
<#693 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AANAS4PDAJKIOYKH7TIM7G3UR752PANCNFSM5JW6QNZA>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Yesterday I used the developer zone start and stop trip feature to record a trip to and back from a location 5.8 miles away.
When I tried to refresh the diary it was stuck on "Reading user cache" for a long time.
Then it came back with 2 trips that were only 2km and 0.7 km
I can upload the loggerDB or the server side mongo db if that would help figure out what happened?
The text was updated successfully, but these errors were encountered: