Briefcase freezes when exporting forms that have a null version

[DavisRayM]

Software versions

Briefcase v1.17.1 & Briefcase v1.17.0

Problem description

Briefcase freezes while trying to export a form whose metadata.json has a null on the version. The logs show no error / warning and Briefcase just keeps infinitely loading.

Steps to reproduce the problem

Set version to null in the metadata.json of a form and try exporting the form

Expected behavior

Briefcase should raise/log an error and stop exporting

[ggalmazor]

Thanks for reporting this, @DavisRayM!

[ggalmazor]

Do you intend on working on a solution to this issue, @DavisRayM? Asking before jumping in :)

[DavisRayM]

Sadly, I'm a bit too swamped to work on this...

[ggalmazor]

No prob! I'll take it :)

[ggalmazor]

OK, so far, this is what I got:

• I pulled the Untitled Form that's currently available in the sandbox server, including 3 submissions.

  This form has no version. Therefore its related metadata.json file has a null version property

• I was able to export all three submissions

This rules out just having null version in the metadata.json file as the single cause for the reported error, which indicates that the issue has a more complex cause

[ggalmazor]

I was able to get an error if I pulled a form with a version and then, after closing Briefcase, I changed its version to null in the metadata.json file.

This is expected because we use a combination of the form's id and version as a key to identify forms and link them to their submissions. The error comes because there's a mismatch between the form's definition - which has version - and its metadata - which doesn't have version.

In any case, I'm not sure we should support this kind of scenarios where users tamper with files created by Briefcase.

Also, I'm not 100% sure this is what you originally did to get your error, @DavisRayM... Could you explain how to reproduce the error with more detail? Specifically, I'd be interested about:

• The initial state of the storage directory: did the form exist before pulling it?
• Where did you pull the form from. Could you maybe attach the form definition here and add a submission that produces the error?
• How did you pull the form? UI or CLI
• How did you export the form? UI or CLI

[DavisRayM]

Hey,

• On the first question, no the form did not exist before pulling it.
• Pulled the form from the Onadata aggregate server(https://odk.ona.io) - Form was built using the Form Builder app present on the platform. Would it be inconvenient if i provided user credentials and a form that causes the issue for you to utilize in testing ?
• Pulled using the UI
• Exported using the UI

Not quite sure if this would be of help but noticed that the metadata.json is a bit different from the one the Untitled Form brings back.

Metadata.json for the form raising issues

{
  "key" : {
    "name" : "lets_try",
    "id" : "aiw5KNVFDkB3TZWso5qpw4",
    "version" : null
  },
  "formDir" : "forms/lets_try",
  "hasBeenPulled" : true,
  "cursor" : {
    "type" : "empty",
    "value" : null
  }
}

[ggalmazor]

Would it be inconvenient if i provided user credentials and a form that causes the uses for you to utilize in testing ?

Yes! :) You can send them to [email protected] if you'd rather not publish them here ;)

[ggalmazor]

Not quite sure if this would be of help but noticed that the metadata.json is a bit different from the one the Untitled Form brings back.

If you're referring to the different cursor structure., it's fine. It's because Ona and Aggregate use different cursor approaches.

[DavisRayM]

Hey, sent the credentials.

[ggalmazor]

Thanks!

I've been able to narrow down the cause of the problem. It turns out that when Briefcase asks Ona for the remote form list, Ona it's reporting that the Testing Form has no version, and creates a metadata.json file with a null version accordingly.

The problem is that when the form is pulled, there's a mismatch between the form's definition, that has version v2E67Yj8FPMji2fK4r6bJ8, and the metadata file, that has no version, and this is what's causing issues during export.

[ggalmazor]

The bad news is that Briefcase doesn't expect to change identifying data such as the form's version when pulling them, which means that fixing this Briefcase-side will be a complex thing to do that will affect other pull sources too.

The easiest fix would involve Ona reporting the form's version correctly. I'll create an issue in their tracker to inform about this.

We can keep this issue open for now, just in case.

[DavisRayM]

The bad news is that Briefcase doesn't expect to change identifying data such as the form's version when pulling them, which means that fixing this Briefcase-side will be a complex thing to do.

The easiest fix would involve Ona reporting the form's version correctly. I'll create an issue in their tracker to inform about this.

We can keep this issue open for now, just in case.

Thank you, an issue to deal with Ona not returning a version has already been opened. Was just not quite sure if that was the only cause of this.

Feel free to comment or track the issue here

Thanks once again :)

[ggalmazor]

Thank you @DavisRayM!

[ggalmazor]

Closing this issue, since it looks like we're trying to address a solution in Ona first.