Same endpoint, different query strings #54

tarciosaraiva · 2016-09-24T06:27:15Z

Hi there
recently a bug was opened on Pact JS relating to having two interactions with the same endpoint but with one of the interactions differing only by a query string on the request.

Here's the issue: pact-foundation/pact-js#12

I've tested locally and got the same results. Here's the test I created as part of the Pact JS suite: https://github.com/pact-foundation/pact-js/blob/test-query-string/test/dsl/integration.spec.js#L137

Edit:
I suspect the issue is on Pact Mock Server, hence the bug. Below is the output of my log file. The Pact file gets created with only one interaction:

I, [2016-09-24T16:03:37.783385 #9663]  INFO -- : Registered expected interaction GET /projects
D, [2016-09-24T16:03:37.783615 #9663] DEBUG -- : {
  "description": "a request for projects",
  "provider_state": "i have a list of projects",
  "request": {
    "method": "GET",
    "path": "/projects",
    "headers": {
      "Accept": "application/json"
    }
  },
  "response": {
    "status": 200,
    "headers": {
      "Content-Type": "application/json"
    },
    "body": [
      {
        "id": 1,
        "name": "Project 1",
        "due": "2016-02-11T09:46:56.023Z",
        "tasks": [
          {
            "id": 1,
            "name": "Do the laundry",
            "done": true
          },
          {
            "id": 2,
            "name": "Do the dishes",
            "done": false
          },
          {
            "id": 3,
            "name": "Do the backyard",
            "done": false
          },
          {
            "id": 4,
            "name": "Do nothing",
            "done": false
          }
        ]
      }
    ]
  }
}
I, [2016-09-24T16:03:37.785313 #9663]  INFO -- : Registered expected interaction GET /projects?from=today
D, [2016-09-24T16:03:37.785473 #9663] DEBUG -- : {
  "description": "a request for projects starting today",
  "provider_state": "i have a list of projects starting today",
  "request": {
    "method": "GET",
    "path": "/projects",
    "query": {
      "from": [
        "today"
      ]
    },
    "headers": {
      "Accept": "application/json"
    }
  },
  "response": {
    "status": 200,
    "headers": {
      "Content-Type": "application/json"
    },
    "body": [
      {
        "id": 1,
        "name": "Project 1",
        "due": "2016-02-11T09:46:56.023Z",
        "tasks": [
          {
            "id": 1,
            "name": "Do the laundry",
            "done": true
          },
          {
            "id": 2,
            "name": "Do the dishes",
            "done": false
          },
          {
            "id": 3,
            "name": "Do the backyard",
            "done": false
          },
          {
            "id": 4,
            "name": "Do nothing",
            "done": false
          }
        ]
      }
    ]
  }
}
I, [2016-09-24T16:03:37.794977 #9663]  INFO -- : Received request GET /projects
D, [2016-09-24T16:03:37.795086 #9663] DEBUG -- : {
  "path": "/projects",
  "query": "",
  "method": "get",
  "headers": {
    "Host": "localhost:1234",
    "Accept-Encoding": "gzip, deflate",
    "User-Agent": "node-superagent/2.3.0",
    "Accept": "application/json",
    "Connection": "close",
    "Version": "HTTP/1.1"
  }
}
I, [2016-09-24T16:03:37.795500 #9663]  INFO -- : Found matching response for GET /projects
D, [2016-09-24T16:03:37.795785 #9663] DEBUG -- : {
  "status": 200,
  "headers": {
    "Content-Type": "application/json"
  },
  "body": [
    {
      "id": 1,
      "name": "Project 1",
      "due": "2016-02-11T09:46:56.023Z",
      "tasks": [
        {
          "id": 1,
          "name": "Do the laundry",
          "done": true
        },
        {
          "id": 2,
          "name": "Do the dishes",
          "done": false
        },
        {
          "id": 3,
          "name": "Do the backyard",
          "done": false
        },
        {
          "id": 4,
          "name": "Do nothing",
          "done": false
        }
      ]
    }
  ]
}
I, [2016-09-24T16:03:37.801958 #9663]  INFO -- : Received request GET /projects?from=today
D, [2016-09-24T16:03:37.802037 #9663] DEBUG -- : {
  "path": "/projects",
  "query": "from=today",
  "method": "get",
  "headers": {
    "Host": "localhost:1234",
    "Accept-Encoding": "gzip, deflate",
    "User-Agent": "node-superagent/2.3.0",
    "Accept": "application/json",
    "Connection": "close",
    "Version": "HTTP/1.1"
  }
}
E, [2016-09-24T16:03:37.802210 #9663] ERROR -- : Multiple interactions found for GET /projects?from=today:
D, [2016-09-24T16:03:37.802344 #9663] DEBUG -- : {
  "description": "a request for projects",
  "provider_state": "i have a list of projects",
  "request": {
    "method": "GET",
    "path": "/projects",
    "headers": {
      "Accept": "application/json"
    }
  },
  "response": {
    "status": 200,
    "headers": {
      "Content-Type": "application/json"
    },
    "body": [
      {
        "id": 1,
        "name": "Project 1",
        "due": "2016-02-11T09:46:56.023Z",
        "tasks": [
          {
            "id": 1,
            "name": "Do the laundry",
            "done": true
          },
          {
            "id": 2,
            "name": "Do the dishes",
            "done": false
          },
          {
            "id": 3,
            "name": "Do the backyard",
            "done": false
          },
          {
            "id": 4,
            "name": "Do nothing",
            "done": false
          }
        ]
      }
    ]
  }
}
D, [2016-09-24T16:03:37.802467 #9663] DEBUG -- : {
  "description": "a request for projects starting today",
  "provider_state": "i have a list of projects starting today",
  "request": {
    "method": "GET",
    "path": "/projects",
    "query": {
      "from": [
        "today"
      ]
    },
    "headers": {
      "Accept": "application/json"
    }
  },
  "response": {
    "status": 200,
    "headers": {
      "Content-Type": "application/json"
    },
    "body": [
      {
        "id": 1,
        "name": "Project 1",
        "due": "2016-02-11T09:46:56.023Z",
        "tasks": [
          {
            "id": 1,
            "name": "Do the laundry",
            "done": true
          },
          {
            "id": 2,
            "name": "Do the dishes",
            "done": false
          },
          {
            "id": 3,
            "name": "Do the backyard",
            "done": false
          },
          {
            "id": 4,
            "name": "Do nothing",
            "done": false
          }
        ]
      }
    ]
  }
}
I, [2016-09-24T16:03:37.806536 #9663]  INFO -- : Writing pact with details {:consumer=>{:name=>"Consumer 1"}, :provider=>{:name=>"Provider 1"}}
I, [2016-09-24T16:03:37.806612 #9663]  INFO -- : Writing pact for Provider 1 to /home/tarcio/workspace/pact/pact-js/pacts/consumer_1-provider_1.json
I, [2016-09-24T16:03:37.808968 #9663]  INFO -- : Cleared interactions before example ""

The text was updated successfully, but these errors were encountered:

tarciosaraiva · 2016-09-25T12:34:43Z

OK I think I found the cause. In pact-support there are two matcher methods that check the difference excluding the query string:

Both of them are invoked by pact-mock_service in here and here.

There are 3 solutions to this in my mind right now:

new method in pact-support that does stricter checking (i.e. checks for qs) which would leave the decision making on which method to invoke to pact-mock_service
modify methods in pact-support to check difference using query string if query string is passed in (no code change in pact-mock_service)
just change the check to include query string and be done with it (least preferred option)

Please advise, happy to submit PR.

uglyog · 2016-10-04T00:50:55Z

Option 2 does feel like the better option.

bethesque · 2016-10-04T04:26:09Z

What does it do if you send GET /projects to it? Same thing? Yes, I think I agree that option 2 is correct.

It's this line that does it: https://github.com/bethesque/pact-support/blob/master/lib/pact/consumer_contract/request.rb#L48

def query_diff actual_query
  if specified?(:query)
    # If the query was expected, check the query. This should probably be if the query was expected OR a query was given.
    query_diff = query.difference(actual_query)
    query_diff.any? ? {query: query_diff} : {}
  else
    {}
  end
end

bethesque · 2016-10-04T04:33:02Z

The only problem is that this will be a breaking change - it could well impact existing projects. Major version release?

tarciosaraiva · 2016-10-04T04:44:30Z

@bethesque regarding your question above

What does it do if you send GET /projects to it? Same thing?

Not quite. The log above shows that a request for the endpoint without the query string finds a match right away.

With the query string it gets confused.

Blackbaud-JonathanBell · 2016-10-14T20:21:56Z

I'm having a similar issue with matching on Cookies. I have a request to the same endpoint but expect a different response based on whether you have a cookie provided (your session). However, when I do provide the cookie it complains about multiple matches. The interaction which doesn't specify the cookie is matching as well as the one that matches the cookie header exactly.

Any thoughts on this?

bethesque · 2016-10-14T20:28:51Z

My thoughts are that Tarcio was going to submit a PR ;-)

On 15 Oct 2016 7:21 AM, "Blackbaud-JonathanBell" [email protected]
wrote:

I'm having a similar issue with matching on Cookies. I have a request to
the same endpoint but expect a different response based on whether you have
a cookie provided (your session). However, when I do provide the cookie it
complains about multiple matches. The interaction which doesn't specify the
cookie is matching as well as the one that matches the cookie header
exactly.

Any thoughts on this?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#54 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAbPFAevkDYsL7YzKeXzbHhJw4XC1ZEGks5qz-RlgaJpZM4KFkdb
.

ptornhult · 2017-09-13T13:50:28Z

Just bumped into the same problem: "Multiple interactions found for POST". In my case only the request body is different but I think the problem is the same. Did anyone fix this or make a PR?

Right now I will try to create separate Pact files (changing consumer name) for each scenario, but this seems like a hack/workaround and I'd rather see this issue resolved.

@tarciosaraiva can make a PR to try and fix this issue?

bethesque · 2017-09-15T02:56:38Z

I'll have another look at it. Can you bump this if I haven't gotten back to you in a couple of days?

ptornhult · 2017-09-15T06:26:11Z

Awesome, thanks!

In case anyone else stumbles on this problem in the mean time. I've implemented a workaround in my Pact.js tests for now where I split all Pact specs (which use the same path) to use different consumer names (also logfiles and mock server port). This way multiple pact files are generated and we don't get the conflict.

bethesque · 2017-09-15T07:11:35Z

The mock service is designed so that you load just the interactions for a single test, run that test, then reset the interactions. Are you using it in a long lived way over multiple tests?

ptornhult · 2017-09-15T07:58:52Z

Since I couldn't load more than 1 interaction for the same endpoint in the mock service, I've split each interaction to use their own mock service

bethesque · 2017-09-15T21:42:56Z

You can load them all into the same mock service, just not at the same time. That's the way it's designed. You're not supposed to set up every interaction at the same time for the entire test suite. You just load the interactions for one test, run that, clear the interactions, then set up the interactions for the next test, run that, clear the interactions etc. Unless you need to test both of the conflicting interactions in the same test case, it shouldn't be a problem. Here's a gist showing how the mock service is designed to be used: https://gist.github.com/bethesque/9d81f21d6f77650811f4

ptornhult · 2017-09-18T07:32:13Z

My bad, I tried to merge my interactions into the same spec again and it's working :) I think I might have been confused about where/when to run provider.verify()

bethesque · 2017-10-02T04:59:35Z

Having a look at this again. If we don't specify a body, then no body match is performed either. What happens in the JVM impl Ron?

uglyog · 2017-10-02T05:54:39Z

In the JVM, if the expected interaction has no body, the actual body is ignored.

eatdrinksleepcode · 2017-12-07T03:52:30Z

I am now running into this (using the JS DSL; see pact-foundation/pact-js#12). Do I understand correctly that there is no way to specify one response for a GET without a query parameter, and another response for a GET with a query parameter?

cpul · 2017-12-07T15:20:37Z

I am also running into the original problem in this issue. Are there any plans to fix this?

bethesque · 2017-12-07T22:06:16Z

@eatdrinksleepcode you can specify both of those queries, just not at the same time. The mock service is not meant to have a single session for the entire test suite. You should set up one or two interactions for a single test, then do the test, and then clear the mock. Have a read of these usage instructions.

https://github.com/pact-foundation/pact-mock_service#mock-service-usage

If we can work out the "correct" logic for making the two interactions work in the same session, I'm happy to make a fix. The problem is, the consistent rule has been, if X is not specified, then accept any value of X. We can't change this without breaking backwards compatibility.

eatdrinksleepcode · 2018-01-18T06:08:39Z

@bethesque I guess my problem with that approach is that for this particular API, "if X is not specified, then accept any value of X" is not correct. For the test that is supposed to NOT send the query string, if my code DOES send the query string, I want that pact verification to fail.

Has there ever been any design work put into the general idea of negative specifications?

bethesque · 2018-01-18T22:38:56Z

This is a tricky one. Negative specifications have been deliberately not supported, because each consumer should specify what they care about, and ignore what they don't. It also kind of relates to this: https://github.com/pact-foundation/pact-ruby/wiki/FAQ#why-is-there-no-support-for-specifying-optional-attributes

I can see this is awkward in a case where you're defining behaviour based on the absence of a field however.

If you're not sure if the field is there or not for each interaction, it would seem to me that you're not fully in control of the provider state set up, and that may mean that pact is not the best option tool for the situation.

eatdrinksleepcode · 2018-01-19T05:14:32Z

Negative specifications have been deliberately not supported, because each consumer should specify what they care about, and ignore what they don't.

That's the second half of Postel's law: be liberal in what you accept. I am talking about the first half: be conservative in what you send. I want to exactly specify what the request (what I am sending) should be, and fail the test if the actual request does not exactly match the specification.

bethesque · 2018-01-21T22:30:35Z

If you specify a query string, then the string must match exactly, with no extra parameters. If you use a hash, then the order is ignored, but again, you can't have any extra parameters. If you want to specify that there are no query parameters, then try using an empty string or and empty hash. I'm not sure if that will work, but try it, and if it doesn't, we can modify the code.

mefellows · 2018-06-24T02:27:57Z

@eatdrinksleepcode I think that sort of test case is better served by traditional validated mocks that are local to your code base.

The reason is that these sorts of tests are very likely to create Pacts that are not enforceable by the Providing service, as they start to contain consumer-specific rules.

shavo007 · 2018-08-16T05:06:35Z

got the same issue with a post. issue was when to call verify. if it was in afterEach then it cleared the interactions before my second test case and failed stating "No matching interaction".

Changed the verify to run after all test cases pass

    after(async function() {
        provider.verify()
        this.timeout(10000) // it takes time to stop the mock server and gather the contracts
        await provider.finalize()

    })

now it works.

mefellows · 2018-08-16T06:28:49Z

Rather than a timeout, does await provider.verify() not work?

shavo007 · 2018-08-17T01:15:42Z

yea @mefellows ive changed the structure now and it works

    before(async function() {
        await provider.setup()
    })

    after(async function() {
        await provider.finalize()
    })

    afterEach(async function()  {
        await provider.verify()
        sandbox.restore()
    })

121371 · 2019-01-27T10:02:32Z

Really don't know is it right way or not but it works.

beforeEach(async () => {
await provider.setup();
});
afterEach(async () => {
await provider.finalize();
});

mefellows · 2019-01-29T21:05:49Z

@121371 what's your question? You can certainly use async/await (it's easier to just return the promise).

Is your question related to this issue?

tarciosaraiva mentioned this issue Sep 24, 2016

Two interactions that only differ by query string pact-foundation/pact-js#12

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Same endpoint, different query strings #54

Same endpoint, different query strings #54

tarciosaraiva commented Sep 24, 2016 •

edited

Loading

tarciosaraiva commented Sep 25, 2016 •

edited

Loading

uglyog commented Oct 4, 2016

bethesque commented Oct 4, 2016 •

edited

Loading

bethesque commented Oct 4, 2016

tarciosaraiva commented Oct 4, 2016

Blackbaud-JonathanBell commented Oct 14, 2016

bethesque commented Oct 14, 2016

ptornhult commented Sep 13, 2017

bethesque commented Sep 15, 2017

ptornhult commented Sep 15, 2017

bethesque commented Sep 15, 2017

ptornhult commented Sep 15, 2017

bethesque commented Sep 15, 2017

ptornhult commented Sep 18, 2017

bethesque commented Oct 2, 2017

uglyog commented Oct 2, 2017

eatdrinksleepcode commented Dec 7, 2017

cpul commented Dec 7, 2017

bethesque commented Dec 7, 2017

eatdrinksleepcode commented Jan 18, 2018

bethesque commented Jan 18, 2018

eatdrinksleepcode commented Jan 19, 2018

bethesque commented Jan 21, 2018

mefellows commented Jun 24, 2018

shavo007 commented Aug 16, 2018

mefellows commented Aug 16, 2018

shavo007 commented Aug 17, 2018

121371 commented Jan 27, 2019

mefellows commented Jan 29, 2019

Same endpoint, different query strings #54

Same endpoint, different query strings #54

Comments

tarciosaraiva commented Sep 24, 2016 • edited Loading

tarciosaraiva commented Sep 25, 2016 • edited Loading

uglyog commented Oct 4, 2016

bethesque commented Oct 4, 2016 • edited Loading

bethesque commented Oct 4, 2016

tarciosaraiva commented Oct 4, 2016

Blackbaud-JonathanBell commented Oct 14, 2016

bethesque commented Oct 14, 2016

ptornhult commented Sep 13, 2017

bethesque commented Sep 15, 2017

ptornhult commented Sep 15, 2017

bethesque commented Sep 15, 2017

ptornhult commented Sep 15, 2017

bethesque commented Sep 15, 2017

ptornhult commented Sep 18, 2017

bethesque commented Oct 2, 2017

uglyog commented Oct 2, 2017

eatdrinksleepcode commented Dec 7, 2017

cpul commented Dec 7, 2017

bethesque commented Dec 7, 2017

eatdrinksleepcode commented Jan 18, 2018

bethesque commented Jan 18, 2018

eatdrinksleepcode commented Jan 19, 2018

bethesque commented Jan 21, 2018

mefellows commented Jun 24, 2018

shavo007 commented Aug 16, 2018

mefellows commented Aug 16, 2018

shavo007 commented Aug 17, 2018

121371 commented Jan 27, 2019

mefellows commented Jan 29, 2019

tarciosaraiva commented Sep 24, 2016 •

edited

Loading

tarciosaraiva commented Sep 25, 2016 •

edited

Loading

bethesque commented Oct 4, 2016 •

edited

Loading