JSON API: Fix for non-UTF-8 data

[mdawaffe]

json_encode() can only encode UTF-8 strings. Anything else will cause json_encode() to return false.

Core addresses this with wp_json_encode(), which converts all data to UTF-8 before calling json_encode().

WordPress.com addresses this with similar recursive pre-processing. (WordPress.com's solution here is different than Core's because it predates Core's use of json_encode() for anything interesting.)

This could have been fixed a long time ago :) I suspect it hasn't come up often because much of the data we return in JSON API requests comes from cached data, which is cleaned up by WordPress.com's pre-processing. At any rate, it needs fixing.

Changes proposed in this Pull Request:

Use wp_json_encode() instead of json_encode() so that oddly broken data doesn't completely break JSON API requests.

Is this a new feature or does it add/remove features to an existing part of Jetpack?

No. Bug fix.

Testing instructions:

1. On a test site, wipe your database and reinstall with defined( 'DB_CHARSET', 'latin1' ). For those using the docker container:
   1. yarn docker:uninstall
   2. Set DB_CHARSET to 'latin1' in wp-config.php.
   3. yarn docker:install
2. yarn docker:wp db cli
3. UPDATE wp_posts SET post_content = UNHEX( '48656C6C6FF7' ) WHERE ID = 1; The last byte (F7) is a byte that cannot exist in UTF-8 encoded data.
4. Connect Jetpack
5. curl --silent --dump-header /dev/stderr YOUR_SITE?rest_route=/wp/v2/posts/1
6. See that .content.rendered contains Hello?. The last byte of the post's contents has been converted to an ASCII ?. This is the expected output: json_encode() cannot encoded non-UTF-8 strings.
7. curl --silent --dump-header /dev/stderr 'https://public-api.wordpress.com/rest/v1.1/sites/YOUR_SITE/posts/1

Prior to this patch, see an error:

{"error":"jetpack_response_error","message":"A malformed response was received from the Jetpack site."}

After this patch, see the correct response: a JSON blob with .content containing Hello?.

Note that you will probably want to switch back to define( 'DB_CHARSET', 'utf8' ) and uninstall/reinstall again after testing.

Proposed changelog entry for your changes:

• Fix API responses which contain malformed (non-UTF-8) data.

[matticbot]

Caution: This PR has changes that must be merged to WordPress.com
Hello mdawaffe! These changes need to be synced to WordPress.com - If you 're an a11n, please commandeer, review, and approve D31505-code before merging this PR. Thank you!

[jetpackbot]

Thank you for the great PR description!

When this PR is ready for review, please apply the [Status] Needs Review label. If you are an a11n, please have someone from your team review the code if possible. The Jetpack team will also review this PR and merge it to be included in the next Jetpack release.

Scheduled Jetpack release: September 3, 2019.
Scheduled code freeze: August 27, 2019

Generated by 🚫 dangerJS against c21cc75