S3 Android Mobile SDK - TransferUtility - IllegalStateException reading from CursorWindow

[jhermida-agero]

To help us solve your problem better, please answer the following list of questions.

• What service are you using?
  S3 Android Mobile SDK

• In what version of SDK are you facing the problem?
  Started with 2.3.4, then updated to 2.4.1 and issue still occurs.

• Is the issue limited to Simulators / Actual Devices?
  Actual Devices

• Can your problem be resolved if you bump to a higher version of SDK?
  No.

• Is this problem related to specific Android/OS version?
  Crash reports show issues (see below) occur on KitKat through Nougat.

• Can you give us steps to reproduce with a minimal, complete, and verifiable example? Please include any specific network conditions that might be required to reproduce the problem.
  I have not been able to reproduce this issue. I'm only receiving crash reports from production builds. On average, there are 6-7 uploads per day, per user, for those who allow uploads on cellular data. Otherwise, data is uploaded only when connected to WiFi.

• Please include a stacktrace if applicable.
  There are two separate crashes being reported which result in the same exception, however the stacktraces are slightly different.

[Stacktrace 1 - Occurring on Marshmallow and Nougat]

Fatal Exception: java.lang.IllegalStateException: Couldn't read row 1054, col 0 from CursorWindow.  
Make sure the Cursor is initialized correctly before accessing data from it.
       at android.database.CursorWindow.nativeGetLong(CursorWindow.java)
       at android.database.CursorWindow.getLong(CursorWindow.java:524)
       at android.database.CursorWindow.getInt(CursorWindow.java:591)
       at android.database.AbstractWindowedCursor.getInt(AbstractWindowedCursor.java:69)
       at com.amazonaws.mobileconnectors.s3.transferutility.TransferService.ˊ(Unknown Source:2379)
       at com.amazonaws.mobileconnectors.s3.transferutility.TransferService$ˊ.handleMessage(Unknown Source:226)
       at android.os.Handler.dispatchMessage(Handler.java:102)
       at android.os.Looper.loop(Looper.java:158)
       at android.os.HandlerThread.run(HandlerThread.java:61)

[Stacktrace 2 - Occurring on KitKat and Lollipop]

Fatal Exception: java.lang.IllegalStateException: Couldn't read row 1044, col 0 from CursorWindow.  Make sure the Cursor is initialized correctly before accessing data from it.
       at android.database.CursorWindow.nativeGetLong(CursorWindow.java)
       at android.database.CursorWindow.getLong(CursorWindow.java:524)
       at android.database.AbstractWindowedCursor.getLong(AbstractWindowedCursor.java:75)
       at android.database.AbstractCursor.moveToPosition(AbstractCursor.java:220)
       at android.database.AbstractCursor.moveToNext(AbstractCursor.java:245)
       at com.amazonaws.mobileconnectors.s3.transferutility.TransferService.ˊ(Unknown Source:2378)
       at com.amazonaws.mobileconnectors.s3.transferutility.TransferService$ˊ.handleMessage(Unknown Source:226)
       at android.os.Handler.dispatchMessage(Handler.java:102)
       at android.os.Looper.loop(Looper.java:145)
       at android.os.HandlerThread.run(HandlerThread.java:61)

If you need help with understanding how to implement something in particular then we suggest that you first look into our developer guide. You can also simplify your process of creating an application by using Mobile Hub.

[rohandubal]

Hello,

Thanks for reporting this. We will investigate and revert.

Thanks,
Rohan

[jhermida-agero]

Hi @rohandubal. Thanks for the update. I did some additional investigation. Here's an update.

Stacktrace 1 (above) is actually occurring after a file is successfully uploaded. Here's the use case:

1. Upload is called on the file, its state is immediately changed to IN_PROGRESS
2. Within the next 5 seconds, the file upload completes with its state changed to COMPLETED
3. TransferUtility.deleteTransferRecord(transferObserver.getId()) is then called on the transfer by our app, which returns true
4. Within the next 10 seconds, the exception above is thrown

Depending on what the user is doing, a file of the same type can be uploaded every 1-2 minutes during a particular flow in our app, and the exception is being thrown almost every time after uploading the file. The file size on average is ~100KB. The app is typically running in the background when this is occurring. There are occasional scenarios where this file type is uploaded twice within a minute, but the exception is not being thrown. Please let me know if any additional information will help.

[jhermida-agero]

Hi @rohandubal. Any updates on this issue?

[ghost]

I've also seen this problem when a user tries to delete all completed UPLOAD transfers (using TransferUtility.getTransfersWithTypeAndState() and TransferUtility.deleteTransferRecord()) on a Samsung Galaxy S7 Edge, API 24, aws-android-sdk-s3:2.4.0.

I've been able to obtain a copy of the awss3transfertable.db and awss3transfertable.db-journal files from a user who experienced this problem, which I can give to AWS if that would help you diagnose the issue?

In my case, the stack trace was almost identical to what jhermida-agero posted:

java.lang.IllegalStateException: Couldn't read row 1167, col 0 from CursorWindow.  Make sure the Cursor is initialized correctly before accessing data from it.
    at android.database.CursorWindow.nativeGetLong(CursorWindow.java)
    at android.database.CursorWindow.getLong(CursorWindow.java:524)
    at android.database.CursorWindow.getInt(CursorWindow.java:591)
    at android.database.AbstractWindowedCursor.getInt(AbstractWindowedCursor.java:69)
    at com.amazonaws.mobileconnectors.s3.transferutility.TransferService.loadTransfersFromDB(TransferService.java:375)
    at com.amazonaws.mobileconnectors.s3.transferutility.TransferService.checkTransfers(TransferService.java:240)
    at com.amazonaws.mobileconnectors.s3.transferutility.TransferService$UpdateHandler.handleMessage(TransferService.java:222)
    at android.os.Handler.dispatchMessage(Handler.java:102)
    at android.os.Looper.loop(Looper.java:154)
    at android.os.HandlerThread.run(HandlerThread.java:61)

I tried opening the file in sqlite and running select * from awstransfer where _rowid_ = 1167;, which returns a row (I've obscured some of the contents for privacy):

1167||UPLOAD|COMPLETED|<bucket-name>|<file-name>||2195832|2195832|||0|</path/to/my-file>||0|0||||||||||||{}||||||

Please let me know if there's anything I can do to help.

[minbi]

This should be fixed with version 2.4.4, please re-open if the problem persists.

[jhermida-agero]

Hi,

We deployed the 2.4.4 AWS Android SDK update last month and although the crash[1] rate has decreased, the issue still exists and is affecting 1k+ users. We also found a new crash[2] which started after the 2.4.4 deployment and is also affecting 1k+ users.

[1]

Fatal Exception: java.lang.IllegalStateException: Couldn't read row 1054, col 0 from CursorWindow.  
Make sure the Cursor is initialized correctly before accessing data from it.
       at android.database.CursorWindow.nativeGetLong(CursorWindow.java)
       at android.database.CursorWindow.getLong(CursorWindow.java:524)
       at android.database.CursorWindow.getInt(CursorWindow.java:591)
       at android.database.AbstractWindowedCursor.getInt(AbstractWindowedCursor.java:69)
       at com.amazonaws.mobileconnectors.s3.transferutility.TransferService.ˊ(Unknown Source:2379)
       at com.amazonaws.mobileconnectors.s3.transferutility.TransferService$ˊ.handleMessage(Unknown Source:226)
       at android.os.Handler.dispatchMessage(Handler.java:102)
       at android.os.Looper.loop(Looper.java:158)
       at android.os.HandlerThread.run(HandlerThread.java:61)

[2]

Fatal Exception: java.lang.IllegalStateException: Cannot perform this operation because the connection pool has been closed.
       at android.database.sqlite.SQLiteConnectionPool.throwIfClosedLocked(SQLiteConnectionPool.java:976)
       at android.database.sqlite.SQLiteConnectionPool.waitForConnection(SQLiteConnectionPool.java:604)
       at android.database.sqlite.SQLiteConnectionPool.acquireConnection(SQLiteConnectionPool.java:353)
       at android.database.sqlite.SQLiteSession.acquireConnection(SQLiteSession.java:894)
       at android.database.sqlite.SQLiteSession.executeForCursorWindow(SQLiteSession.java:834)
       at android.database.sqlite.SQLiteQuery.fillWindow(SQLiteQuery.java:62)
       at android.database.sqlite.SQLiteCursor.fillWindow(SQLiteCursor.java:171)
       at android.database.sqlite.SQLiteCursor.getCount(SQLiteCursor.java:160)
       at android.database.AbstractCursor.moveToPosition(AbstractCursor.java:219)
       at android.database.AbstractCursor.moveToFirst(AbstractCursor.java:258)
       at com.amazonaws.mobileconnectors.s3.transferutility.ᐝ.ʽ(Unknown Source:804)
       at com.amazonaws.mobileconnectors.s3.transferutility.TransferService.ˎ(Unknown Source:300)
       at com.amazonaws.mobileconnectors.s3.transferutility.TransferService$if.handleMessage(Unknown Source:240)
       at android.os.Handler.dispatchMessage(Handler.java:102)
       at android.os.Looper.loop(Looper.java:168)
       at android.os.HandlerThread.run(HandlerThread.java:61)

[mutablealligator]

Thanks for reporting the issue.

We are aware of this issue. We fixed a part of the issue to reduce the frequency of crashes.
To get rid of the crashes, the SDK should paginate the cursor: Instead of performing one query on the Transfer Database which fetches all the records and stores in the cursor,
we should be fetching the records in batches. We will update once we have a fix on this.

[totalityorbust]

Any updates on this?

[totalityorbust]

What service are you using?
S3 Android Mobile SDK

In what version of SDK are you facing the problem?
I have been updating from 2.4.7 till 2.6.13 in hopes of resolving this.

Is the issue limited to Simulators / Actual Devices?
Actual Devices

Can your problem be resolved if you bump to a higher version of SDK?
No.

Is this problem related to specific Android/OS version?
Has been happening on various devices with Android 5, 6 and 7.

Can you give us steps to reproduce with a minimal, complete, and verifiable example? Please include any specific network conditions that might be required to reproduce the problem.

No predictable way to replicate the issue.

Please include a stacktrace if applicable.

java.lang.IllegalStateException: Cannot perform this operation because the connection pool has been closed.
at android.database.sqlite.SQLiteConnectionPool.throwIfClosedLocked(SQLiteConnectionPool.java:962)
at android.database.sqlite.SQLiteConnectionPool.waitForConnection(SQLiteConnectionPool.java:599)
at android.database.sqlite.SQLiteConnectionPool.acquireConnection(SQLiteConnectionPool.java:348)
at android.database.sqlite.SQLiteSession.acquireConnection(SQLiteSession.java:894)
at android.database.sqlite.SQLiteSession.executeForCursorWindow(SQLiteSession.java:834)
at android.database.sqlite.SQLiteQuery.fillWindow(SQLiteQuery.java:62)
at android.database.sqlite.SQLiteCursor.fillWindow(SQLiteCursor.java:144)
at android.database.sqlite.SQLiteCursor.getCount(SQLiteCursor.java:133)
at android.database.AbstractCursor.moveToPosition(AbstractCursor.java:197)
at android.database.AbstractCursor.moveToFirst(AbstractCursor.java:237)
at com.amazonaws.mobileconnectors.s3.transferutility.TransferDBUtil.getTransferById(TransferDBUtil.java:804)
at com.amazonaws.mobileconnectors.s3.transferutility.TransferService.execCommand(TransferService.java:300)
at com.amazonaws.mobileconnectors.s3.transferutility.TransferService$UpdateHandler.handleMessage(TransferService.java:240)
at android.os.Handler.dispatchMessage(Handler.java:102)
at android.os.Looper.loop(Looper.java:135)
at android.os.HandlerThread.run(HandlerThread.java:61)

I really would like an update this issue, if it is indeed being addressed.

[mutablealligator]

Hi @jhermida-agero, @totalityorbust We have fixed bugs and made some improvements to TransferUtility in 2.6.16. Can you please upgrade to the latest and see if this fixes the issue?

[totalityorbust]

We updated a week ago or so and there have been no issues so far. I'll update this thread if anything comes up. Thanks!

[mutablealligator]

Thank you @totalityorbust for the confirmation. Let us know if you face any issues during your development/testing.