Test does not complete when using ob_start and fpassthru #5546

nyamsprod · 2023-10-23T15:03:59Z

Q	A
PHPUnit version	10.4.x
PHP version	PHP8.3.0RC4 (alpine and github actions)
Installation Method	Composer

Summary

The test works:

with all versions of PHP8.1 until the recent PHP8.3.0RC4 in PHPUnit.
Outside of PHPUnit the code works as expected.
Test also work with PHPUnit 10.3.1 (I checked the results of the github action)

Current behaviour

The test never ends nor report any failure.

How to reproduce

public function testOutputDoesNotStripBOM(): void
{
      $raw_csv = "\xEF\xBB\xBFjohn,doe,john.doe@example.com\njane,doe,jane.doe@example.com\n";
      $stream = fopen('php://temp', 'r+');
      fwrite($stream, $raw_csv);
      ob_start();
      rewind($stream);
      echo "\xFE\xFF";
      fpassthru($stream);
      $result = ob_get_clean();
      fclose($stream);

      self::assertStringContainsString("\xFE\xFF", $result);
      self::assertStringContainsString("\xEF\xBB\xBF", $result);
      self::assertTrue(str_starts_with($result, "\xFE\xFF\xEF\xBB\xBF"));
}

Expected behavior

Works in PHP8.1+ with PHPUnit 10.4.0 the test ends and pass successfully

sebastianbergmann · 2023-10-23T16:47:34Z

I cannot reproduce this:

$ composer info                                                              
myclabs/deep-copy                  1.11.1  Create deep copies (clones) of your objects
nikic/php-parser                   v4.17.1 A PHP parser written in PHP
phar-io/manifest                   2.0.3   Component for reading phar.io manifest information from a PHP Archive (PHAR)
phar-io/version                    3.2.1   Library for handling version information and constraints
phpunit/php-code-coverage          10.1.7  Library that provides collection, processing, and rendering functionality for PHP code coverage information.
phpunit/php-file-iterator          4.1.0   FilterIterator implementation that filters files based on a list of suffixes.
phpunit/php-invoker                4.0.0   Invoke callables with a timeout
phpunit/php-text-template          3.0.1   Simple template engine.
phpunit/php-timer                  6.0.0   Utility class for timing
phpunit/phpunit                    10.4.1  The PHP Unit Testing framework.
sebastian/cli-parser               2.0.0   Library for parsing CLI options
sebastian/code-unit                2.0.0   Collection of value objects that represent the PHP code units
sebastian/code-unit-reverse-lookup 3.0.0   Looks up which function or method a line of code belongs to
sebastian/comparator               5.0.1   Provides the functionality to compare PHP values for equality
sebastian/complexity               3.1.0   Library for calculating the complexity of PHP code units
sebastian/diff                     5.0.3   Diff implementation
sebastian/environment              6.0.1   Provides functionality to handle HHVM/PHP environments
sebastian/exporter                 5.1.1   Provides the functionality to export PHP variables for visualization
sebastian/global-state             6.0.1   Snapshotting of global state
sebastian/lines-of-code            2.0.1   Library for counting the lines of code in PHP source code
sebastian/object-enumerator        5.0.0   Traverses array structures and object graphs to enumerate all referenced objects
sebastian/object-reflector         3.0.0   Allows reflection of object attributes, including inherited and non-public ones
sebastian/recursion-context        5.0.0   Provides functionality to recursively process PHP variables
sebastian/type                     4.0.0   Collection of value objects that represent the types of the PHP type system
sebastian/version                  4.0.1   Library that helps with managing the version number of Git-hosted PHP projects
theseer/tokenizer                  1.2.1   A small library for converting tokenized PHP source code into XML and potentially other formats

`tests/Issue5546Test.php`

<?php declare(strict_types=1);
use PHPUnit\Framework\TestCase;

final class Issue5546Test extends TestCase
{
    public function testOutputDoesNotStripBOM(): void
    {
        $raw_csv = "\xEF\xBB\xBFjohn,doe,john.doe@example.com\njane,doe,jane.doe@example.com\n";
        $stream = fopen('php://temp', 'r+');
        fwrite($stream, $raw_csv);
        ob_start();
        rewind($stream);
        echo "\xFE\xFF";
        fpassthru($stream);
        $result = ob_get_clean();
        fclose($stream);

        self::assertStringContainsString("\xFE\xFF", $result);
        self::assertStringContainsString("\xEF\xBB\xBF", $result);
        self::assertTrue(str_starts_with($result, "\xFE\xFF\xEF\xBB\xBF"));
    }
}

Test Runner Output

$ ./vendor/bin/phpunit tests
PHPUnit 10.4.1 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.2.11

.                                                                   1 / 1 (100%)

Time: 00:00.004, Memory: 6.00 MB

OK (1 test, 3 assertions)

Test Runner Event Log

$ ./vendor/bin/phpunit --no-output --log-events-text php://stdout tests 
PHPUnit Started (PHPUnit 10.4.1 using PHP 8.2.11 (cli) on Linux)
Test Runner Configured
Test Suite Loaded (1 test)
Event Facade Sealed
Test Runner Started
Test Suite Sorted
Test Runner Execution Started (1 test)
Test Suite Started (CLI Arguments, 1 test)
Test Suite Started (Issue5546Test, 1 test)
Test Preparation Started (Issue5546Test::testOutputDoesNotStripBOM)
Test Prepared (Issue5546Test::testOutputDoesNotStripBOM)
Assertion Succeeded (Constraint: contains "��" [Encoding detection failed](length: 2), Value: '��john,doe,john.doe@example.com\n
jane,doe,jane.doe@example.com\n
')
Assertion Succeeded (Constraint: contains "" [UTF-8](length: 3), Value: '��john,doe,john.doe@example.com\n
jane,doe,jane.doe@example.com\n
')
Assertion Succeeded (Constraint: is true, Value: true)
Test Passed (Issue5546Test::testOutputDoesNotStripBOM)
Test Finished (Issue5546Test::testOutputDoesNotStripBOM)
Test Suite Finished (Issue5546Test, 1 test)
Test Suite Finished (CLI Arguments, 1 test)
Test Runner Execution Finished
Test Runner Finished
PHPUnit Finished (Shell Exit Code: 0)

nyamsprod · 2023-10-23T17:08:24Z

yes it works on PHP 8,2.x and PHPUnit 10.4.1
but it odes not on PHP8.3.RC4 and PHPUnit 10.4.1

PHPUnit 10.4.1 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.0RC4
Configuration: /app/phpunit.xml

I have the test blocked as shown above

nyamsprod · 2023-10-23T17:10:59Z

using your settings I have the following

PHPUnit Started (PHPUnit 10.4.1 using PHP 8.3.0RC4 (cli) on Linux)
Test Runner Configured
Bootstrap Finished (/app/vendor/autoload.php)
Test Suite Loaded (1 test)
Test Runner Triggered Warning (No code coverage driver available)
Event Facade Sealed
Test Runner Started
Test Suite Sorted
Test Runner Execution Started (1 test)
Test Suite Started (League\Csv\BugTest, 1 test)
Test Preparation Started (League\Csv\BugTest::testOutputDoesNotStripBOM)
Test Prepared (League\Csv\BugTest::testOutputDoesNotStripBOM)
Assertion Succeeded (Constraint: contains "��" [Encoding detection failed](length: 2), Value: '��john,doe,john.doe@example.com\n
jane,doe,jane.doe@example.com\n
')

So the test passes but the result is never returned

Assertion Succeeded (Constraint: is true, Value: true)
Test Passed (Issue5546Test::testOutputDoesNotStripBOM)
Test Finished (Issue5546Test::testOutputDoesNotStripBOM)
Test Suite Finished (Issue5546Test, 1 test)
Test Suite Finished (CLI Arguments, 1 test)
Test Runner Execution Finished
Test Runner Finished
PHPUnit Finished (Shell Exit Code: 0)

☝🏾 The following never happened

sebastianbergmann · 2023-10-24T03:33:31Z

I read "This test works with all versions of PHP8.1 until the recent PHP8.3.0RC4 in PHPUnit" as "including the recent PHP8.3.0RC4". I am also confused by "Test also work with PHPUnit 10.3.1". You have two variables in your report: PHP version and PHPUnit version. I am not sure whether you mean "With PHPUnit 10.4.1 it does not work on PHP 8.3.0RC4 but with PHPUnit 10.3.1 the test does work on PHP 8.3.0RC4". Please be more specific, thanks.

nyamsprod · 2023-10-24T03:41:49Z

Sorry for the confusion. To recap

the test only fails with PHPUnit 10.4.x and PHP8.3.0RC4.
the test works with older version of PHPUnit on PHP8.3.0RC4.
the test works with PHPUnit 10.4.x on older version of PHP

I hope this clarify the report.

sebastianbergmann · 2023-10-24T04:23:46Z

The first commit (found using git bisect) for which the test does not work with PHP 8.3 is 969d64d.

At first glance, I do not see how this change could lead to the behavior we see.

sebastianbergmann · 2023-10-24T04:47:13Z

The changes made in 969d64d cause the following line of code to be executed:

phpunit/src/Framework/Constraint/String/StringContains.php

Line 128 in 62bd7af

$detectedEncoding = mb_detect_encoding($other, null, true);

With PHP 8.3, mb_detect_encoding() seems to "hang".

When I change the line to $detectedEncoding = false; (to not call mb_detect_encoding() here) then the test succeeds with PHP 8.3.

sebastianbergmann · 2023-10-24T04:51:02Z

@alexdowad I see that you worked on mb_detect_encoding() changes for PHP 8.3. Do you have an idea what might be going on here? Thanks!

sebastianbergmann · 2023-10-24T04:56:09Z

The mb_detect_encoding() call was introduced by @two-plus-two-equals-five in #5505.

nyamsprod · 2023-10-24T05:08:19Z

@sebastianbergmann I was able to reproduce the bug with the following code following your research

final class BugTest extends TestCase
{
    #[Test]
    public function it_shows_bug(): void
    {
        $result = "\xEF\xBB\xBF";

        self::assertStringContainsString("\xEF\xBB\xBF", $result);
    }
}

I do not know if it matters but the sequence is the BOM sequence for UTF-8. Can this be the source of the issue in regards to mb_detect_encoding 🤔

On 3val the following call fails too https://3v4l.org/H8l7c/rfc#vgit.master

var_dump(mb_detect_encoding("\xEF\xBB\xBF", null, true));

So indeed seems to be an issue with mb_detect_encoding

sebastianbergmann · 2023-10-24T05:12:31Z

At this point I think that this is an issue with mb_detect_encoding(), yes. But let's see what @alexdowad has to say.

sebastianbergmann · 2023-10-24T05:36:48Z

I have reported this bug for PHP and will now close this ticket.

alexdowad · 2023-10-24T06:19:35Z

Dear @sebastianbergmann, thanks very much for investigating this problem. Do you have a link for the bug ticket in php-src?

alexdowad · 2023-10-24T06:20:42Z

Ah, I found it.

alexdowad · 2023-10-27T19:47:43Z

Sorry for the delay. I have just opened a PR to fix this bug: php/php-src#12541

alexdowad · 2023-10-28T17:10:46Z

The fix has landed on the PHP-8.3 and master branches of php-src. It will be incorporated in the next PHP releases in the 8.3 and 8.4 series.

nyamsprod added type/bug Something is broken version/10 Something affects PHPUnit 10 labels Oct 23, 2023

sebastianbergmann added the status/waiting-for-feedback Waiting for feedback from original reporter label Oct 23, 2023

sebastianbergmann removed the status/waiting-for-feedback Waiting for feedback from original reporter label Oct 24, 2023

sebastianbergmann mentioned this issue Oct 24, 2023

mb_detect_encoding() does not terminate when called with BOM sequence for UTF-8 php/php-src#12505

Closed

sebastianbergmann closed this as completed Oct 24, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test does not complete when using ob_start and fpassthru #5546

Test does not complete when using ob_start and fpassthru #5546

nyamsprod commented Oct 23, 2023 •

edited

Loading

sebastianbergmann commented Oct 23, 2023

nyamsprod commented Oct 23, 2023

nyamsprod commented Oct 23, 2023 •

edited

Loading

sebastianbergmann commented Oct 24, 2023

nyamsprod commented Oct 24, 2023 •

edited

Loading

sebastianbergmann commented Oct 24, 2023

sebastianbergmann commented Oct 24, 2023

sebastianbergmann commented Oct 24, 2023

sebastianbergmann commented Oct 24, 2023

nyamsprod commented Oct 24, 2023 •

edited

Loading

sebastianbergmann commented Oct 24, 2023

sebastianbergmann commented Oct 24, 2023

alexdowad commented Oct 24, 2023

alexdowad commented Oct 24, 2023

alexdowad commented Oct 27, 2023

alexdowad commented Oct 28, 2023

Test does not complete when using ob_start and fpassthru #5546

Test does not complete when using ob_start and fpassthru #5546

Comments

nyamsprod commented Oct 23, 2023 • edited Loading

Summary

Current behaviour

How to reproduce

Expected behavior

sebastianbergmann commented Oct 23, 2023

tests/Issue5546Test.php

Test Runner Output

Test Runner Event Log

nyamsprod commented Oct 23, 2023

nyamsprod commented Oct 23, 2023 • edited Loading

sebastianbergmann commented Oct 24, 2023

nyamsprod commented Oct 24, 2023 • edited Loading

sebastianbergmann commented Oct 24, 2023

sebastianbergmann commented Oct 24, 2023

sebastianbergmann commented Oct 24, 2023

sebastianbergmann commented Oct 24, 2023

nyamsprod commented Oct 24, 2023 • edited Loading

sebastianbergmann commented Oct 24, 2023

sebastianbergmann commented Oct 24, 2023

alexdowad commented Oct 24, 2023

alexdowad commented Oct 24, 2023

alexdowad commented Oct 27, 2023

alexdowad commented Oct 28, 2023

nyamsprod commented Oct 23, 2023 •

edited

Loading

`tests/Issue5546Test.php`

nyamsprod commented Oct 23, 2023 •

edited

Loading

nyamsprod commented Oct 24, 2023 •

edited

Loading

nyamsprod commented Oct 24, 2023 •

edited

Loading