Does oneOf in responseBodyTests support environment variables?

[jpjpjp]

I'm wondering if it's possible to create a content test to see if a field in a response body matches one of a few postman environment variables.

I've tried the following:

          tests:
            contentTests:
              - responseBodyTests:
                  - key: children
                    minLength: 2
                  - key: childrend[0].id
                    oneOf: ["{{{_testCreatedCategoryId}}}", "{{{_testCreatedCategoryId2}}}"]
                  - key: childrend[1].id
                    oneOf: 
                      - "{{_testCreatedCategoryId}}"
                      - "{{_testCreatedCategoryId2}}"

Both syntaxes are accepted by portman, but neither generates a valid test. Here is what the above generates:

// Response body should have "childrend[0].id"
pm.test("[PUT]::/categories/:id - Content check if 'childrend[0].id' exists", function() {
   pm.expect(_resChildrend0Id !== undefined).to.be.true;
});

// Response body should be one of the values "{{{_testCreatedCategoryId}}},{{{_testCreatedCategoryId2}}}" for "childrend[0].id"
if (_resChildrend0Id !== undefined) {
pm.test("[PUT]::/categories/:id - Content check if value for 'childrend[0].id' is matching one of: '{{{_testCreatedCategoryId}}},{{{_testCreatedCategoryId2}}}'", function() {
  pm.expect(jsonData.childrend[0].id).to.be.oneOf(["pm.collectionVariables.get("{_testCreatedCategoryId}")","pm.collectionVariables.get("{_testCreatedCategoryId2}")"]);
})};

// Set property value as variable
const _resChildrend1Id = jsonData?.childrend?.[1]?.id;

// Response body should have "childrend[1].id"
pm.test("[PUT]::/categories/:id - Content check if 'childrend[1].id' exists", function() {
   pm.expect(_resChildrend1Id !== undefined).to.be.true;
});

// Response body should be one of the values "{{_testCreatedCategoryId}},{{_testCreatedCategoryId2}}" for "childrend[1].id"
if (_resChildrend1Id !== undefined) {
pm.test("[PUT]::/categories/:id - Content check if value for 'childrend[1].id' is matching one of: '{{_testCreatedCategoryId}},{{_testCreatedCategoryId2}}'", function() {
  pm.expect(jsonData.childrend[1].id).to.be.oneOf(["pm.collectionVariables.get("_testCreatedCategoryId")","pm.collectionVariables.get("_testCreatedCategoryId2")"]);
})};

If this isn't supported, that is fine. Please don't treat this as a must-have feature request or bug report. I was just wondering if there was a different syntax that might work.

[thim81]

hi @jpjpjp,

Let me run some tests, how portman handles it. At first glance, I suspect an unneeded string conversion using backticks.

[thim81]

Thanks 🙏 for the explanation.
Based on this, the PR is ready and the fix will be included in the next release.

[jpjpjp]

Thanks! I ran the PR branch locally today against a config that included this:

            contentTests:
              - responseBodyTests:
                  - key: children
                    minLength: 2
                  - key: children[0].id
                    oneOf: ["{{{_testCreatedCategoryId}}}", "{{{_testCreatedCategoryId2}}}"]
                  - key: children[1].id
                    oneOf: 
                      - "{{_testCreatedCategoryId}}"
                      - "{{_testCreatedCategoryId2}}"

Here are the generated tests:

// Response body should be one of the values "{{{_testCreatedCategoryId}}},{{{_testCreatedCategoryId2}}}" for "children[0].id"
if (_reschildren0Id !== undefined) {
pm.test("[PUT]::/categories/:id - Content check if value for 'children[0].id' is matching one of: '{{{_testCreatedCategoryId}}},{{{_testCreatedCategoryId2}}}'", function() {
  pm.expect(jsonData.children[0].id).to.be.oneOf(["pm.collectionVariables.get("{_testCreatedCategoryId}")","pm.collectionVariables.get("{_testCreatedCategoryId2}")"]);
})};

// Response body should be one of the values "{{_testCreatedCategoryId}},{{_testCreatedCategoryId2}}" for "children[1].id"
if (_reschildren1Id !== undefined) {
pm.test("[PUT]::/categories/:id - Content check if value for 'children[1].id' is matching one of: '{{_testCreatedCategoryId}},{{_testCreatedCategoryId2}}'", function() {
  pm.expect(jsonData.children[1].id).to.be.oneOf(["pm.collectionVariables.get("_testCreatedCategoryId")","pm.collectionVariables.get("_testCreatedCategoryId2")"]);
})};

There are problems with both of these outputs, but in general the second syntax (which I believe is the one that should be supported anyway) is closer. I had thought that this would work if you wrapped the two pm.collectionVariables.get calls in the oneOf array using backticks and braces instead of double quotes, ie:

// Response body should be one of the values "{{_testCreatedCategoryId}},{{_testCreatedCategoryId2}}" for "children[1].id"
if (_reschildren1Id !== undefined) {
pm.test("[PUT]::/categories/:id - Content check if value for 'children[1].id' is matching one of: '{{_testCreatedCategoryId}},{{_testCreatedCategoryId2}}'", function() {
  pm.expect(jsonData.children[1].id).to.be.oneOf([`${pm.collectionVariables.get("_testCreatedCategoryId")}`,`${pm.collectionVariables.get("_testCreatedCategoryId2")}`]);
})};

But this didn't work either, as I got this result:

ontent check if value for 'children[1].id' is matching one of: '{{_testCreatedCategoryId}},{{_testCreatedCategoryId2}}' | AssertionError: expected 1105065 to be one of [ '1105065', '1105067' ]

The only way I was able to get this to work was by manually resolving the values before making the assertion:

// Response body should be one of the values "{{_testCreatedCategoryId}},{{_testCreatedCategoryId2}}" for "children[1].id"
if (_resChildren1Id !== undefined) {
pm.test("[PUT]::/categories/:id - Content check if value for 'children[1].id' is matching one of: '{{_testCreatedCategoryId}},{{_testCreatedCategoryId2}}'", function() {
  let id1 = pm.collectionVariables.get("_testCreatedCategoryId")
  let id2 = pm.collectionVariables.get("_testCreatedCategoryId")
  // pm.expect(jsonData.children[1].id).to.be.oneOf([`${pm.collectionVariables.get("_testCreatedCategoryId")}`,`${pm.collectionVariables.get("_testCreatedCategoryId2")}`]);
  pm.expect(jsonData.children[1].id).to.be.oneOf([id1, id2]);
})};

Which might be more trouble than it's worth? I'll reiterate that I think its completely reasonable if Portman doesn't support postman variables in the oneOf directive within responseBodyTests!

[thim81]

hi @jpjpjp

I'm not sure if you are using the correct version, since it the "" would be removed.

This is the config that I'm using:

version: 1
globals:
  stripResponseExamples: true
  collectionVariables:
    _testCreatedCategoryId: abc
    _testCreatedCategoryId2: 123
overwrites:
  - openApiOperationIds:
      - post-event
    overwriteRequestBody:
      - key: event.startDateTime
        overwrite: true
        value: '2002-12-01'
tests:
  contentTests:
    - openApiOperationId: post-event
      responseBodyTests:
        - key: children
          minLength: 2
#        - key: 'childrend[0].id'
#          oneOf:
#            - '{{{_testCreatedCategoryId}}}'
#            - '{{{_testCreatedCategoryId2}}}'
        - key: 'childrend[1].id'
          oneOf:
            - '{{_testCreatedCategoryId}}'
            - '{{_testCreatedCategoryId2}}'

I got this result (don't mind the commented out, that was to be able to run the request):

[thim81]

hi @jpjpjp

We just released Portman v1.30.2, this contains the improvements with handling collectionVariables for oneOf.
Can you try it out and see if this works like in my screenshot?

[jpjpjp]

Hi @thim81, I can confirm that this is working as you describe and producing the results I would expect.

Sorry for the confusion in my previous post. I must have pulled down the wrong PR branch or not properly linked my node_modules to my local copy.