Set root node for created submissions to the XForms configured root node #1853

DavisRayM · 2020-07-27T06:10:12Z

Changes / Features implemented

Set the root node to data. By default the root node of an XForm is always set to data unless specifically set to something else
Make change backward compatible with forms that have the root node name set to something else other than the default(data)
Add management command that helps replace the XForm ID String as the root node for Instances

Steps taken to verify this change does what is intended

Updated tests
QA

Things to Note

Submissions made using RapidPro before the changes in this PR are in effect, will still have the different root node name issue. Suggestion: Those submissions should be updated manually on a case by case basis(Server wide submission XML edits seem a bit too unsafe)

ukanga · 2020-07-27T18:09:45Z

onadata/libs/serializers/data_serializer.py

@@ -46,7 +46,7 @@ def create_submission(request, username, data_dict, xform_id):
    """
    Returns validated data object instances
    """
-    xml_string = dict2xform(data_dict, xform_id)
+    xml_string = dict2xform(data_dict, xform_id, root='data')


What happens to older forms that still use the filename of the XLSForm that is in our historical data?

That's something I did not initially consider... I'll look into retrieving the root node name from the form. I believe pyxform provides this information on the survey object, I'll confirm and update the PR

Updated the PR. Now trying to retrieve the root node name from the XForm if a username is passed alongside the form_id, This should ensure all new submissions have the correct root node for the form.

By default XLSForms set the default node to 'data'

faith-mutua · 2020-08-04T11:28:08Z

@DavisRayM New RapidPro submissions can now be edited in Enketo whereas old submissions cannot. The attached error is returned when editing old RapidPro submissions.

DavisRayM · 2020-08-05T09:20:27Z

Old submissions from RapidPro will have to be updated via the management command

python manage.py replace_form_id_root_node -i <instance_ids>  # Prints out the changes that are going to be made on the XML
python manage.py replace_form_id_root_node -i <instance_ids> -c  # Makes changes to the Instances XML and saves them

@DavisRayM New RapidPro submissions can now be edited in Enketo whereas old submissions cannot. The attached error is returned when editing old RapidPro submissions.

ivermac

This looks good to me but might require @ukanga 's eyes as well

ivermac · 2020-08-05T09:29:44Z

onadata/apps/logger/management/commands/replace_form_id_root_node.py

+    initial_xml = inst.xml
+    form_id = re.escape(inst.xform.id_string)
+    if not root:
+        root = inst.xform.survey.name


Will the value of root always be data?

Not always, it'll be data as long as the user doesn't configure the form to have another root node name.

pld · 2020-08-05T12:06:17Z

Here is an overview of what got changed by this pull request:

Complexity increasing per file
==============================
- onadata/apps/logger/management/commands/replace_form_id_root_node.py  4

See the complete overview on Codacy

DavisRayM force-pushed the enketo_different_root_nodes_fix branch 2 times, most recently from 0b9e462 to 4bb6562 Compare July 27, 2020 11:48

DavisRayM changed the title ~~[WIP] Change root node for created submissions to data~~ Change root node for created submissions to data Jul 27, 2020

DavisRayM requested review from ivermac, ukanga and WinnyTroy July 27, 2020 13:55

DavisRayM force-pushed the enketo_different_root_nodes_fix branch from 4bb6562 to acf673d Compare July 27, 2020 13:56

ukanga reviewed Jul 27, 2020

View reviewed changes

DavisRayM changed the title ~~Change root node for created submissions to data~~ [WIP] Change root node for created submissions to data Jul 28, 2020

DavisRayM force-pushed the enketo_different_root_nodes_fix branch 10 times, most recently from 54ac36c to 5eaee15 Compare July 29, 2020 14:30

DavisRayM changed the title ~~[WIP] Change root node for created submissions to data~~ Change root node for created submissions to data Jul 29, 2020

DavisRayM force-pushed the enketo_different_root_nodes_fix branch from 5eaee15 to e87e511 Compare July 30, 2020 06:05

DavisRayM mentioned this pull request Jul 30, 2020

Add management command to increase an ODKTokens lifetime #1847

Merged

ukanga previously approved these changes Aug 4, 2020

View reviewed changes

Change root node for created submissions to data

19323ec

By default XLSForms set the default node to 'data'

DavisRayM dismissed ukanga’s stale review via a2b9698 August 4, 2020 08:13

DavisRayM force-pushed the enketo_different_root_nodes_fix branch from e87e511 to a2b9698 Compare August 4, 2020 08:13

DavisRayM changed the title ~~Change root node for created submissions to data~~ Set root node for created submissions to the XForms configured root node Aug 4, 2020

DavisRayM changed the title ~~Set root node for created submissions to the XForms configured root node~~ [WIP] Set root node for created submissions to the XForms configured root node Aug 4, 2020

DavisRayM changed the title ~~[WIP] Set root node for created submissions to the XForms configured root node~~ Set root node for created submissions to the XForms configured root node Aug 4, 2020

Retrieve root node name from XForm survey object

aff9675

DavisRayM force-pushed the enketo_different_root_nodes_fix branch 2 times, most recently from 34523fd to 8ff96fe Compare August 5, 2020 09:17

DavisRayM force-pushed the enketo_different_root_nodes_fix branch from 8ff96fe to 997d01f Compare August 5, 2020 09:37

ivermac previously approved these changes Aug 5, 2020

View reviewed changes

Add management command to replace form ID root node

46dc1a3

DavisRayM dismissed ivermac’s stale review via 46dc1a3 August 5, 2020 12:02

DavisRayM force-pushed the enketo_different_root_nodes_fix branch from 997d01f to 46dc1a3 Compare August 5, 2020 12:02

ukanga approved these changes Aug 5, 2020

View reviewed changes

DavisRayM merged commit d79754d into master Aug 6, 2020

DavisRayM deleted the enketo_different_root_nodes_fix branch August 6, 2020 06:55

DavisRayM mentioned this pull request Aug 10, 2020

Tag release v2.3.7 #1867

Merged

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Set root node for created submissions to the XForms configured root node #1853

Set root node for created submissions to the XForms configured root node #1853

DavisRayM commented Jul 27, 2020 •

edited

Loading

ukanga Jul 27, 2020

DavisRayM Jul 28, 2020

DavisRayM Jul 30, 2020 •

edited

Loading

faith-mutua commented Aug 4, 2020 •

edited

Loading

DavisRayM commented Aug 5, 2020 •

edited

Loading

ivermac left a comment

ivermac Aug 5, 2020

DavisRayM Aug 5, 2020 •

edited

Loading

pld commented Aug 5, 2020

Set root node for created submissions to the XForms configured root node #1853

Set root node for created submissions to the XForms configured root node #1853

Conversation

DavisRayM commented Jul 27, 2020 • edited Loading

Changes / Features implemented

Steps taken to verify this change does what is intended

Things to Note

ukanga Jul 27, 2020

Choose a reason for hiding this comment

DavisRayM Jul 28, 2020

Choose a reason for hiding this comment

DavisRayM Jul 30, 2020 • edited Loading

Choose a reason for hiding this comment

faith-mutua commented Aug 4, 2020 • edited Loading

DavisRayM commented Aug 5, 2020 • edited Loading

ivermac left a comment

Choose a reason for hiding this comment

ivermac Aug 5, 2020

Choose a reason for hiding this comment

DavisRayM Aug 5, 2020 • edited Loading

Choose a reason for hiding this comment

pld commented Aug 5, 2020

DavisRayM commented Jul 27, 2020 •

edited

Loading

DavisRayM Jul 30, 2020 •

edited

Loading

faith-mutua commented Aug 4, 2020 •

edited

Loading

DavisRayM commented Aug 5, 2020 •

edited

Loading

DavisRayM Aug 5, 2020 •

edited

Loading