CapacitorHttp POST application/x-www-form-urlencoded bug:

[nickredding]

Bug Report

Capacitor Version

5.2.2

Platform(s)

Android

Current Behavior

Http POST request with content type "application/x-www-form-urlencoded" does not properly send the data as an object, rather as an object with a single key consisting of the stringified data and value empty.

Expected Behavior

The data object should be sent as the object

Code Reproduction

This bug should be blindingly obvious from a code inspection of setRequestBody in the Java module CapacitorHttpUrlConnection.

I moved my app from Cordova to Capacitor in the hopes that the Capacitor platform would be better maintained and less buggy than Cordova. For the most part I have found this to be true and I don't regret making the move.

However, I am really surprised at the issues with the CapacitorHttp core plugin. It is obviously based on the community Http plugin which I have used successfully since my move to Capacitor. This issue with POST with content type "application/x-www-form-urlencoded" does not exist with the community plugin because it is handled properly in setRequestBody .

Below is the code from the community Http plugin. I have noted the code that was omitted from the core plugin which is why it fails to handle content type "application/x-www-form-urlencoded"

public void setRequestBody(PluginCall call, JSValue body) throws JSONException, IOException {
    String contentType = connection.getRequestProperty("Content-Type");
    String dataString = "";
    if (contentType == null || contentType.isEmpty()) return;
    if (contentType.contains("application/json")) {
        JSArray jsArray = null;
        if (body != null) {
            dataString = body.toString();
        } else {
            jsArray = call.getArray("data", null);
        }
        if (jsArray != null) {
            dataString = jsArray.toString();
        } else if (body == null) {
            dataString = call.getString("data");
        }
        this.writeRequestBody(dataString.toString());

the following code was inexplicably omitted from the core plugin

    } else if (contentType.contains("application/x-www-form-urlencoded")) {
        StringBuilder builder = new StringBuilder();
        JSObject obj = body.toJSObject();
        Iterator<String> keys = obj.keys();
        while (keys.hasNext()) {
            String key = keys.next();
            Object d = obj.get(key);
            builder.append(key).append("=").append(URLEncoder.encode(d.toString(), "UTF-8"));
            if (keys.hasNext()) {
                builder.append("&");
            }
        }
        this.writeRequestBody(builder.toString());

end of omitted code

    } else if (contentType.contains("multipart/form-data")) {
        FormUploader uploader = new FormUploader(connection);
        JSObject obj = body.toJSObject();
        Iterator<String> keys = obj.keys();
        while (keys.hasNext()) {
            String key = keys.next();

            String d = obj.get(key).toString();
            uploader.addFormField(key, d);
        }
        uploader.finish();
    } else {
        this.writeRequestBody(body.toString());
    }
}

I have verified that restoring the omitted code to the core plugin (using os.writeBytes instead of builder.append) makes the POST request work properly.

I have seen at least one other bug report on this POST issue but the responses to that don't touch on the obvious reason for this failure. Therefore, I have explained in detail here what is wrong and what needs to be done to fix it.

[ionitron-bot]

This issue needs more information before it can be addressed. In particular, the reporter needs to provide a minimal sample app that demonstrates the issue. If no sample app is provided within 15 days, the issue will be closed.
Please see the Contributing Guide for how to create a Sample App.
Thanks! Ionitron 💙

[nickredding]

I think this is an automated response because I went to the trouble of debugging this issue and providing both the problem analysis and the solution. I don't see why I should go to the effort of creating and posting a sample app with a single line of code to demonstrate this issue.

This issue does not arise on iOS because the routine getRequestData in module CapacitorUrlRequest correctly handles the case where Content-Type is ''application/x-www-form-urlencoded"

If you want to see this problem, open Chrome DevTools on any Android Capacitor app and type the following into the console:

window.Capacitor.Plugins.CapacitorHttp.post({url:'https://nickredding.ca/echo.php',method:'POST',data:{a:'x',b:'y'},headers:{'Content-Type':'application/x-www-form-urlencoded'}})

You will see the response

result CapacitorHttp.post (#67530483)
Object
data:
REQUEST_METHOD: POST

URL parameters (0)

POST data (1)
   {"a":"x","b":"y"} (string) => (string)
`

You can see from this output that the PHP module echo.php has reported a single POST data item consisting of the stringified object as a key with a null value. This is obviously wrong, and if you inspect the code you can easily see why. In the routine setRequestBody the content type doesn't match any of the options and control drops down to the default

this.writeRequestBody(body.toString());

If you don't want to bother to fix this then fine, I'll just continue to patch the setRequestBody code to work properly and Capacitor can continue to not properly handle the most basic and common HTTP POST request.

[jcesarmobile]

Sorry, but we require a sample app so we can reproduce the issue and once the issue gets verified, we can properly prioritize it internally, and then verify that your solution fixes the issue and properly apply it to the codebase.

While we are grateful that you went the extra mile and debugged the issue for us, we still need an actual app to verify the issue.
https://github.com/ionic-team/capacitor/blob/main/CONTRIBUTING.md#creating-a-code-reproduction

[nickredding]

OK here you go.

https://github.com/nickredding/bug6792

This sample app has three buttons:

"Post Data via jquery ajax" This button causes a jQuery ajax post with data {a:'x',b:'y'} which is handled correctly because jquery submits the post data to CapacitorHttp.request as the preprocessed data "a=x&b=y". You can see the echopost.php file reports $_POST as the associative array ('a'=>'x','b'=>'y') which is correct.

"Post Data via CapacitorHttp" This button causes a Cap[acitorHttp request with data {a:'x',b:'y'} which is handled INCORRECTLY because the data is sent as the string "{a:'x',b:'y'}"

"Post Data string via CapacitorHttp" This button does what jquery ajax does--send the preprocessed data as "a=x&b=y" and is handled correctly (note that this call is identical to that invoked by jquery ajax).

The semantics of POST requests is that the post data is submitted as an object (see the CapacitorHttp documentation) and handled correctly. Both jquery ajax and the original community HTTP plugin adhere to this. The CapacitorHttp plugin DOES NOT.

As I have pointed out, the code in the original community HTTP plugin that handles this was omitted from the CapacitorHttp plugin. The fix is simple.

At this point, there isn't much else I can do to convince you that this problem exists and has a simple solution.

[ionitron-bot]

This issue has been labeled as type: bug. This label is added to issues that that have been reproduced and are being tracked in our internal issue tracker.

[frederikbosch]

I hit this one today when I converted my Capacitor 3 app to version 5. My solution was to pass the data as form-urlencoded string:

let data  = new URLSearchParams(data).toString();

[philjones88]

I hit this one today when I converted my Capacitor 3 app to version 5. My solution was to pass the data as form-urlencoded string:

let data  = new URLSearchParams(data).toString();

I'm having to do this too but the problem is all the third party libraries I need to fork or edit to support this is a blocker.

Cna we look at adding the code back to handle URLSearchParams ?

[nickredding]

The fix is to modify setRequestBody in the module CapacitorHttpUrlConnection as follows

private void writeObjectRequestBody(JSObject object) throws IOException, JSONException {
    try (DataOutputStream os = new DataOutputStream(connection.getOutputStream())) {
        Iterator<String> keys = object.keys();
        while (keys.hasNext()) {
            String key = keys.next();
            Object d = object.get(key);
            os.writeBytes(key);
            os.writeBytes("=");
            os.writeBytes(URLEncoder.encode(d.toString(), "UTF-8"));

            if (keys.hasNext()) {
                os.writeBytes("&");
            }
        }
        os.flush();
    }
}

public void setRequestBody(PluginCall call, JSValue body, String bodyType) throws JSONException, IOException {
    String contentType = connection.getRequestProperty("Content-Type");
    String dataString = "";

    if (contentType == null || contentType.isEmpty()) return;
    if (contentType.contains("application/x-www-form-urlencoded")) {
        try {
            JSObject jsobject = body.toJSObject();
            this.writeObjectRequestBody(jsobject);
        } catch(Exception notanobject) {
            this.writeRequestBody(body.toString());
        }
    }
    else if (contentType.contains("application/json")) {
        JSArray jsArray = null;
        if (body != null) {
            dataString = body.toString();
        } else {
            jsArray = call.getArray("data", null);
        }
        if (jsArray != null) {
            dataString = jsArray.toString();
        } else if (body == null) {
            dataString = call.getString("data");
        }
        this.writeRequestBody(dataString != null ? dataString : "");
    } else if (bodyType != null && bodyType.equals("file")) {
        try (DataOutputStream os = new DataOutputStream(connection.getOutputStream())) {
            if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
                os.write(Base64.getDecoder().decode(body.toString()));
            }
            os.flush();
        }
    } else if (bodyType != null && bodyType.equals("formData")) {
        this.writeFormDataRequestBody(contentType, body.toJSArray());
    } else {
        this.writeRequestBody(body.toString());
    }
}

[philjones88]

Do you think the Capacitor team would accept a PR to add that in?

[ionitron-bot]

Thanks for the issue! This issue is being locked to prevent comments that are not relevant to the original issue. If this is still an issue with the latest version of Capacitor, please create a new issue and ensure the template is fully filled out.