Request data doesn't clean up after calling newRequest()

[galimru]

Hey

I have an issue when trying to use the same connection after submitting the form.
The method newRequest() makes copy data params and it leads that this request preserve data from previous request.
There is no way to clean up data params. So when I use the same connection to "navigate link" it serializes data params as query parameters:

document.connection().newRequest()
                    .url(confirmationUrl)
                    .get();

And moreover it will keep send data from previous form if you use subsequence of form submit calls.

jsoup/src/main/java/org/jsoup/nodes/FormElement.java

Lines 72 to 75 in b0ac5e0

	Connection connection = owner != null? owner.connection().newRequest() : Jsoup.newSession();
	return connection.url(action)
	.data(formData())
	.method(method);

I believe we shouldn't do a copy of data params here:

jsoup/src/main/java/org/jsoup/helper/HttpConnection.java

Line 671 in b0ac5e0

data = new ArrayList<>(); data.addAll(copy.data()); // this is shallow, but holds immutable string keyval, and possibly an InputStream which can only be read once anyway, so using as a prototype would be unsupported

[jhy]

Thanks @galimru, I'll take a look. Can you confirm that if you start with Jsoup.newSession(), and then hit session.newRequest(), your flow works (cookies etc) and you don't get the repeated form data?

https://jsoup.org/apidocs/org/jsoup/Jsoup.html#newSession()

[roysjosh]

Hello,

I am seeing this as well on jsoup 1.15.2. I have to submit a series of forms to get through an OAuth2 login.

    protected Document submitForm(Document doc, String id, Map<String, String> values) throws IOException {
        // REDACTED CODE to look up form & set values

        logger.debug("submitting form " + id + " to " + form.attr("action"));
        form.formData().forEach((keyval) -> {
            logger.debug("- " + keyval.key() + " = " + keyval.value());
        });
        logger.debug("actual data");
        Connection foo = form.submit();
        foo.request().data().forEach((keyval) -> {
            logger.debug("- " + keyval.key() + " = " + keyval.value());
        });
        return foo.execute().parse();
        //return form.submit().execute().parse();
    }

submitting form userStoreForm to /SAAS/auth/login/username/v2
- isJavascriptEnabled =
- areCookiesEnabled =
- dest =
- useragent =
- workspaceId =
- jwt = REDACTED JWT
- isWindows10EnrollmentFlow = false
- isDeviceEnrollmentFlow = false
- userInput = REDACTED USERNAME

actual data
- is-local-admin = false
- jwt = REDACTED JWT
- relay-state = REDACTED UUID
- isJavascriptEnabled =
- areCookiesEnabled =
- dest =
- useragent =
- workspaceId =
- jwt = REDACTED JWT
- isWindows10EnrollmentFlow = false
- isDeviceEnrollmentFlow = false
- userInput = REDACTED USERNAME

Note the first 3 form data elements are left over from a previous form. I am calling Connection session = Jsoup.newSession() and that session object is passed to the first submit form function call. The return of that function is then used as the first parameter.

[roysjosh]

And adding doc.connection().request().data().clear(); immediately before calling form.submit(); fixes things.

[jhy]

Thanks, fixed!