[HOLD for payment 2023-06-23] [$1000] Invalid html code with large digit crashes android app and throws console error on other platforms

[kavimuru]

If you haven’t already, check out our contributing guidelines for onboarding and email contributors@expensify.com to request to join our Slack channel!

---

Action Performed:

1. Open the app in android
2. Open any report
3. Send invalid html code with large digit eg: �

Expected Result:

App shouldn't crash or provide console error on sending invalid html codes with large digit

Actual Result:

App crashes in android and provides console error on other devices when we send invalid html codes with large digit

Workaround:

Can the user still use Expensify without this being fixed? Have you informed them of the workaround?

Platforms:

Which of our officially supported platforms is this issue occurring on?

• Android / native
• Android / Chrome
• iOS / native
• iOS / Safari
• MacOS / Chrome / Safari
• MacOS / Desktop

Version Number: 1.3.20-0
Reproducible in staging?: y
Reproducible in production?: y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation

android.app.crash.invalid.html.code.long.digit.mp4

az_recorder_20230529_220029.1.mp4

Expensify/Expensify Issue URL:
Issue reported by: @dhanashree-sawant
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1685098437111809

View all open jobs on GitHub

Upwork Automation - Do Not Edit

• Upwork Job URL: https://www.upwork.com/jobs/~0137581aed483fd0c3
• Upwork Job ID: 1663778192565182464
• Last Price Increase: 2023-06-07

[melvin-bot]

Triggered auto assignment to @adelekennedy (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.

[melvin-bot]

Bug0 Triage Checklist (Main S/O)

• This "bug" occurs on a supported platform (ensure Platforms in OP are ✅)
• This bug is not a duplicate report (check E/App issues and #expensify-bugs)
  • If it is, comment with a link to the original report, close the issue and add any novel details to the original issue instead
• This bug is reproducible using the reproduction steps in the OP. S/O
  • If the reproduction steps are clear and you're unable to reproduce the bug, check with the reporter and QA first, then close the issue.
  • If the reproduction steps aren't clear and you determine the correct steps, please update the OP.
• This issue is filled out as thoroughly and clearly as possible
  • Pay special attention to the title, results, platforms where the bug occurs, and if the bug happens on staging/production.
• I have reviewed and subscribed to the linked Slack conversation to ensure Slack/Github stay in sync

[hungvu193]

Proposal

Please re-state the problem that we are trying to solve in this issue.

Invalid html code with large digit crashes android app and throws console error on other platforms

What is the root cause of that problem?

Problem came from this line:

/**
 * Html decode, shorten last message text to fixed length and trim spaces.
 * @param {String} lastMessageText
 * @returns {String}
 */
function formatReportLastMessageText(lastMessageText) {
    return Str.htmlDecode(String(lastMessageText)).replace(CONST.REGEX.AFTER_FIRST_LINE_BREAK, '').substring(0, CONST.REPORT.LAST_MESSAGE_TEXT_MAX_LENGTH).trim();
}

    htmlDecode(s) {
        // Use jQuery if it exists or else use html-entities
        if (typeof jQuery !== 'undefined') {
            return jQuery('<textarea/>').html(s).text();
        }
        return AllHtmlEntities.decode(s);
    },

We are using html-entities to decode, and there's issue with this library when decode the invalid html character.
mdevils/html-entities#48

What changes do you think we should make in order to solve the problem?

Coming from this conversation, we can use the fix from here, which was confirmed by the owner that fixed the problem, by adding the check for invalid html:
mdevils/html-entities@8678a50.
So we need to update the decode function inside html5-entities.js to:

                // if (!isNaN(code) || code >= -32768) {
                //     if (code <= 65535) {
                //         chr = String.fromCharCode(code);
                //     }
                //     else {
                //         chr = surrogate_pairs_1.fromCodePoint(code);
                //     }
                // }
                    chr = code >= 0x10ffff
                    ? entity
                    : code > 65535
                    ? fromCodePoint(code)
                    : fromCharCode(numericUnicodeMap[code] || code);

We also can consider to upgrade the html-entities version. Otherwise, we can do a patch-package file to apply the fix.

What alternative solutions did you explore? (Optional)

N.A

Result

Screen.Recording.2023-05-30.at.13.17.59.mov

[dhanashree-sawant]

Hi @adelekennedy, I am still able to reproduce on latest staging, let me know if you need any assistance or have any doubt, I will be available.

2023-05-31.09-05-39.mp4

[adelekennedy]

@dhanashree-sawant I'm not able to reproduce - but since you've been able to repeatedly I think it's safe to call this a bug

[melvin-bot]

Job added to Upwork: https://www.upwork.com/jobs/~0137581aed483fd0c3

[melvin-bot]

Current assignee @adelekennedy is eligible for the External assigner, not assigning anyone new.

[melvin-bot]

Triggered auto assignment to Contributor-plus team member for initial proposal review - @eVoloshchak (External)

[melvin-bot]

Triggered auto assignment to @iwiznia (External), see https://stackoverflow.com/c/expensify/questions/7972 for more details.

[rushatgabhane]

We also can consider to upgrade the html-entities version

Agree! Let's fix this issue upstream

[hungvu193]

More information about breaking change in case we wanna fix this upstream:
https://github.com/mdevils/html-entities/blob/v2.3.3/CHANGELOG.md

[benjaminbennet]

Proposal

Please re-state the problem that we are trying to solve in this issue.

Invalid html code with large digit crashes android app and throws console error on other platforms

What is the root cause of that problem?

We are using html-entities old version which had issue with invalid html.

What changes do you think we should make in order to solve the problem?

Update html-entities to latest version

What alternative solutions did you explore? (Optional)

N.A

[melvin-bot]

📣 @benjaminbennet! 📣
Hey, it seems we don’t have your contributor details yet! You'll only have to do this once, and this is how we'll hire you on Upwork.
Please follow these steps:

1. Get the email address used to login to your Expensify account. If you don't already have an Expensify account, create one here. If you have multiple accounts (e.g. one for testing), please use your main account email.
2. Get the link to your Upwork profile. It's necessary because we only pay via Upwork. You can access it by logging in, and then clicking on your name. It'll look like this. If you don't already have an account, sign up for one here.
3. Copy the format below and paste it in a comment on this issue. Replace the placeholder text with your actual details.
   
   Format:

Contributor details
Your Expensify account email: <REPLACE EMAIL HERE>
Upwork Profile Link: <REPLACE LINK HERE>

[melvin-bot]

⚠️ Invalid email. Please make sure to create an Expensify account with this email first here.

[melvin-bot]

⚠️ Invalid email. Please make sure to create an Expensify account with this email first here.

[melvin-bot]

✅ Contributor details stored successfully. Thank you for contributing to Expensify!

[iwiznia]

Not overdue, waiting for proposal reviews.

[iwiznia]

Yeah

[hungvu193]

cool!, PR is ready @iwiznia

[melvin-bot]

Based on my calculations, the pull request did not get merged within 3 working days of assignment. Please, check out my computations here:

• when @hungvu193 got assigned: 2023-06-08 17:49:36 Z
• when the PR got merged: 2023-06-14 18:04:36 UTC
• days elapsed: 5

On to the next one 🚀

[melvin-bot]

Reviewing label has been removed, please complete the "BugZero Checklist".

[melvin-bot]

The solution for this issue has been 🚀 deployed to production 🚀 in version 1.3.28-5 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:

• Update expensify-common version #20652

If no regressions arise, payment will be issued on 2023-06-23. 🎊

After the hold period is over and BZ checklist items are completed, please complete any of the applicable payments for this issue, and check them off once done.

• External issue reporter @dhanashree-sawant
• Contributor that fixed the issue @hungvu193
• Contributor+ that helped on the issue and/or PR @eVoloshchak

As a reminder, here are the bonuses/penalties that should be applied for any External issue:

• Merged PR within 3 business days of assignment - 50% bonus
• Merged PR more than 9 business days after assignment - 50% penalty

[melvin-bot]

BugZero Checklist: The PR fixing this issue has been merged! The following checklist (instructions) will need to be completed before the issue can be closed:

• [@eVoloshchak] The PR that introduced the bug has been identified. Link to the PR:
• [@eVoloshchak] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment:
• [@eVoloshchak] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion:
• [@eVoloshchak] Determine if we should create a regression test for this bug.
• [@eVoloshchak] If we decide to create a regression test for the bug, please propose the regression test steps to ensure the same bug will not reach production again.
• [@adelekennedy] Link the GH issue for creating/updating the regression test once above steps have been agreed upon:

[adelekennedy]

payment issued!

[hungvu193]

payment issued!

@adelekennedy Thank you. Also I think this issue is eligible for speed bonus right?

[eVoloshchak]

• The PR that introduced the bug has been identified. Link to the PR: Not applicable, this is a bug in an external library

• The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment: N/A

• A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion: N/A

• Determine if we should create a regression test for this bug. I don't think so, this is an edge case that can be observed only for specific symbols

[eVoloshchak]

Not overdue, this is awaiting for payment

[adelekennedy]

@hungvu193 not according to this comment is it wrong?

[hungvu193]

@hungvu193 not according to this comment is it wrong?

Yeah, that's wrong, because I created the PR in expensify-common, and it was merged within 2 days.
Also the PR I created in this repo was also merged within 2 days.
Expensify/expensify-common#545 (Created on 9 Jun and was merged on 10 Jun)

[adelekennedy]

@hungvu193 ty - I'll add the speed bonus!