Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug fix: force to decode as utf-8 when header contains application/json to avoid text garbling. #1700

Conversation

cimadai
Copy link
Contributor

@cimadai cimadai commented Dec 18, 2018

PR checklist

  • Read the contribution guidelines.
  • Ran the shell script under ./bin/ to update Petstore sample so that CIs can verify the change. (For instance, only need to run ./bin/{LANG}-petstore.sh and ./bin/security/{LANG}-petstore.sh if updating the {LANG} (e.g. php, ruby, python, etc) code generator or {LANG} client's mustache templates). Windows batch files can be found in .\bin\windows\.
    • dart is not exists.
  • Filed the PR against the correct branch: master, 3.4.x, 4.0.x. Default: master.
  • Copied the technical committee to review the pull request if your PR is targeting a particular programming language.

Description of the PR

The original processing is using response.body to deserialize as json in api.mustache.
However, this is decoded by latin1 if the header contains only application/json instead of application/json; charset=utf-8.

Because of this behavior, if the response body is encoded UTF-8 but the headers doesn't contain charset, the body will garbling.

cf: dart-lang/http#175

Since playframework 2.6 returns Content-Type: application/json without charset=utf-8, I changed this parsing algolithm.

…o avoid text garbling.

The original processing is using `response.body` to deserialize as json.
However, this is decoded by latin1 if the header contains only "application/json" instead of "application/json; charset=utf-8".

Because of this behavior, if the response body is encoded UTF-8 but the headers doesn't contain charset, the body will garbling.

cf: dart-lang/http#175

Since playframework 2.6 returns "Content-Type: application/json" without "charset=utf-8", I changed this parsing algolithm.
@wing328 wing328 added this to the 4.0.0 milestone Jan 2, 2019
/// Because avoid to text garbling when header only contains "application/json" without "; charset=utf-8".
String _decodeBodyBytes(Response response) {
var contentType = response.headers['content-type'];
if (contentType != null && contentType.contains("application/json")) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cimadai later we may want to further improve it to detect other JSON mime type, eg.

in C# to detect the JSON mime type using a regular expression

@wing328
Copy link
Member

wing328 commented Jan 9, 2019

LGTM. Thanks for the fix 👍

@wing328 wing328 merged commit 1898493 into OpenAPITools:master Jan 9, 2019
@wing328 wing328 changed the title fix: force to decode as utf-8 when header contains application/json to avoid text garbling. Bug fix: force to decode as utf-8 when header contains application/json to avoid text garbling. Jan 9, 2019
jimschubert added a commit to jimschubert/openapi-generator that referenced this pull request Jan 9, 2019
* master: (52 commits)
  Support for "x-enum-descriptions" (OpenAPITools#1752)
  add new sample files
  Add a checklist to issue report (OpenAPITools#1851)
  Fix typo in template (OpenAPITools#1859)
  update samples
  add isModel to updatebooleanflagwithcodegenproperty (OpenAPITools#1844)
  [python-client] Add model default values (OpenAPITools#1776)
  fix: force to decode as utf-8 when header contains application/json to avoid text garbling. (OpenAPITools#1700)
  Better support for composed schema (allOf) (OpenAPITools#1842)
  Add additional properties to Java CodegenModel (OpenAPITools#1854)
  Minor Angular type improvements (OpenAPITools#1843)
  [DART] fix: set fields to null if json value is null. (OpenAPITools#1798)
  add options to maven plugin (OpenAPITools#1845)
  fix unqiue name in handling forward slash (OpenAPITools#1839)
  better handle of oauth (OpenAPITools#1838)
  [C] Support for authentication methods (OpenAPITools#1628)
  better error message when parameter ref not defined (OpenAPITools#1837)
  Add links to articles about openapi generator
  Add a link to @watiko article
  Delete langs command (OpenAPITools#1836)
  ...
A-Joshi pushed a commit to ihsmarkitoss/openapi-generator that referenced this pull request Feb 27, 2019
…o avoid text garbling. (OpenAPITools#1700)

* fix: force to decode as utf-8 when header contains application/json to avoid text garbling.

The original processing is using `response.body` to deserialize as json.
However, this is decoded by latin1 if the header contains only "application/json" instead of "application/json; charset=utf-8".

Because of this behavior, if the response body is encoded UTF-8 but the headers doesn't contain charset, the body will garbling.

cf: dart-lang/http#175

Since playframework 2.6 returns "Content-Type: application/json" without "charset=utf-8", I changed this parsing algolithm.

* fix: force to decode as utf-8 when header contains application/json to avoid text garbling on error.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants