Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix CI #353

Merged
merged 3 commits into from
Oct 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 0 additions & 39 deletions .github/workflows/continuous-integration.yml
Original file line number Diff line number Diff line change
Expand Up @@ -131,45 +131,6 @@ jobs:
- name: "Run PHPUnit"
run: "php vendor/bin/simple-phpunit -v"

phpunit-guzzle5:
name: "PHPUnit (PHP ${{ matrix.php }} and Guzzle 5)"
runs-on: "ubuntu-20.04"

strategy:
matrix:
php:
- "7.4"

steps:
- name: "Checkout"
uses: "actions/checkout@v4"
with:
fetch-depth: 2

- name: "Install PHP"
uses: "shivammathur/setup-php@v2"
with:
php-version: "${{ matrix.php }}"
coverage: "none"
tools: composer:v2
extensions: tidy
env:
COMPOSER_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: "Install dependencies with Composer"
uses: "ramsey/composer-install@v2"

- name: "Setup adapter: Guzzle 5"
run: |
composer remove guzzlehttp/guzzle php-http/guzzle7-adapter --dev -n
composer require php-http/guzzle5-adapter --dev -n

- name: "Setup logs"
run: "mkdir -p build/logs"

- name: "Run PHPUnit"
run: "php vendor/bin/simple-phpunit -v"

phpunit-curl:
name: "PHPUnit (PHP ${{ matrix.php }} and cURL)"
runs-on: "ubuntu-20.04"
Expand Down
10 changes: 3 additions & 7 deletions .php-cs-fixer.php
Original file line number Diff line number Diff line change
Expand Up @@ -25,17 +25,13 @@
'no_useless_return' => true,
// Enabled by Symfony and changes properties without type hints but we cannot use those yet because they require PHP 8.
'no_null_property_initialization' => false,
// Enabled by @Symfony but is inconsistent.
'nullable_type_declaration_for_default_null_value' => [
'use_nullable_type_declaration' => true,
],
'ordered_class_elements' => true,
'ordered_imports' => true,
'php_unit_strict' => true,
'phpdoc_order' => true,
'phpdoc_to_param_type' => true,
'phpdoc_to_return_type' => true,
'phpdoc_to_property_type' => true,
'phpdoc_to_param_type' => ['union_types' => false],
'phpdoc_to_return_type' => ['union_types' => false],
'phpdoc_to_property_type' => ['union_types' => false],
// 'psr4' => true,
'strict_comparison' => true,
'strict_param' => true,
Expand Down
17 changes: 1 addition & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,10 +59,9 @@ Why `php-http/guzzle7-adapter`? Because Graby is decoupled from any HTTP client
Graby is tested & should work great with:

- Guzzle 7 (using `php-http/guzzle7-adapter`)
- Guzzle 5 (using `php-http/guzzle5-adapter`)
- cURL (using `php-http/curl-client`)

Note: if you want to use Guzzle 6, use Graby 2 (support has dropped in v3 because of dependencies conflicts, which does not happen with Guzzle 5 :shrug:)
Note: if you want to use Guzzle 5 or 6, use Graby 2 (support has dropped in v3 because of dependencies conflicts)

### Retrieve content from an url

Expand Down Expand Up @@ -193,20 +192,6 @@ $logs = $this->get('monolog.handler.graby')->getRecords();
If you need to define a timeout, you must create the `Http\Client\HttpClient` manually,
configure it and inject it to `Graby\Graby`.

- For Guzzle 5:

```php
use Graby\Graby;
use GuzzleHttp\Client as GuzzleClient;
use Http\Adapter\Guzzle5\Client as GuzzleAdapter;
$guzzle = new GuzzleClient([
'defaults' => [
'timeout' => 2,
]
]);
$graby = new Graby([], new GuzzleAdapter($guzzle));
```

- For Guzzle 7:

```php
Expand Down
2 changes: 1 addition & 1 deletion composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@
"php-http/guzzle7-adapter": "^1.0",
"php-http/mock-client": "^1.4",
"phpstan/extension-installer": "^1.0",
"phpstan/phpstan": "^1.2",
"phpstan/phpstan": "^1.12",
"phpstan/phpstan-deprecation-rules": "^1.0",
"phpstan/phpstan-phpunit": "^1.0",
"rector/rector": "^0.12.11",
Expand Down
8 changes: 4 additions & 4 deletions phpstan.neon
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@ parameters:
- vendor/bin/.phpunit/phpunit/vendor/autoload.php

ignoreErrors:
# because we check for some HTTP client to exist or not (Guzzle 5/6 & cURL)
# because we check for some HTTP client to exist or not (cURL)
-
message: '#Http\\Adapter\\Guzzle5\\Client\\|Http\\Adapter\\Guzzle6\\Client\\|Http\\Client\\Curl\\Client given#'
message: '#Http\\Client\\Curl\\Client given#'
path: %currentWorkingDirectory%/tests/Extractor/HttpClientTest.php
# phpstan does not seem to recognize the class override for JSLikeHTMLElement
-
Expand All @@ -30,7 +30,7 @@ parameters:
-
message: '#expects DOMElement, DOMNode given#'
path: %currentWorkingDirectory%/src/Graby.php
-
identifier: missingType.iterableValue

inferPrivatePropertyTypeFromConstructor: true
checkMissingIterableValueType: false
checkGenericClassInNonGenericObjectType: false
2 changes: 1 addition & 1 deletion src/Extractor/HttpClient.php
Original file line number Diff line number Diff line change
Expand Up @@ -183,7 +183,7 @@ public function fetch(UriInterface $url, bool $skipTypeVerification = false, arr
// (regex inspired from here: https://stackoverflow.com/a/55083809/954513)
preg_match_all('/<!--(?:\[| ?<!).+?-->/mis', $body, $matchesConditional);

if (isset($matchesConditional[0]) && (is_countable($matchesConditional[0]) ? \count($matchesConditional[0]) : 0) > 1) {
if (\count($matchesConditional[0]) > 1) {
foreach ($matchesConditional as $conditionalComment) {
$body = str_replace($conditionalComment, '', $body);
}
Expand Down
2 changes: 1 addition & 1 deletion src/Extractor/HttpClientConfig.php
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ public function __construct(array $config)
$resolver->setNormalizer('rewrite_url', function (Options $options, $value) {
foreach ($value as $url => $action) {
if (!\is_string($url)) {
throw new InvalidOptionsException(sprintf('The option "rewrite_url" with key "%s" is expected to be of type "string", but is of type "%s".', $url, get_debug_type($url)));
throw new InvalidOptionsException(\sprintf('The option "rewrite_url" with key "%s" is expected to be of type "string", but is of type "%s".', $url, get_debug_type($url)));
}

$this->validateArray($action, 'rewrite_url[' . $url . ']');
Expand Down
12 changes: 6 additions & 6 deletions src/Graby.php
Original file line number Diff line number Diff line change
Expand Up @@ -256,7 +256,7 @@ private function doFetchContent(UriInterface $url): Content
$effectiveUrl = $effectiveUrl->withPath(str_replace(' ', '%20', $effectiveUrl->getPath()));
$response = $response->withEffectiveUri($effectiveUrl);
if (!$this->isUrlAllowed((string) $effectiveUrl)) {
throw new \Exception(sprintf('Url "%s" is not allowed to be parsed.', $effectiveUrl));
throw new \Exception(\sprintf('Url "%s" is not allowed to be parsed.', $effectiveUrl));
}

// check if action defined for returned Content-Type, like image, pdf, audio or video
Expand Down Expand Up @@ -451,7 +451,7 @@ private function validateUrl($uri): UriInterface
$uriIdnSafe = idn_to_ascii($uri->getHost());

if (false === $uriIdnSafe) {
throw new \InvalidArgumentException(sprintf('Url "%s" is not valid IDN to ascii.', (string) $uri));
throw new \InvalidArgumentException(\sprintf('Url "%s" is not valid IDN to ascii.', (string) $uri));
}

$uri = $uri->withHost($uriIdnSafe);
Expand All @@ -470,17 +470,17 @@ private function validateUrl($uri): UriInterface
$url = (string) $uri;

if (false === filter_var($url, \FILTER_VALIDATE_URL)) {
throw new \InvalidArgumentException(sprintf('Url "%s" is not valid.', $url));
throw new \InvalidArgumentException(\sprintf('Url "%s" is not valid.', $url));
}

$url = filter_var($url, \FILTER_SANITIZE_URL);

if (false === $url) {
throw new \InvalidArgumentException(sprintf('Sanitizing url "%s" failed.', $url));
throw new \InvalidArgumentException(\sprintf('Sanitizing url "%s" failed.', $url));
}

if (false === $this->isUrlAllowed($url)) {
throw new \InvalidArgumentException(sprintf('Url "%s" is not allowed to be parsed.', $url));
throw new \InvalidArgumentException(\sprintf('Url "%s" is not allowed to be parsed.', $url));
}

return $this->uriFactory->createUri($url);
Expand Down Expand Up @@ -570,7 +570,7 @@ private function handleMimeAction(array $mimeInfo, EffectiveResponse $response):
);

if ('exclude' === $mimeInfo['action']) {
throw new \Exception(sprintf('This is url "%s" is blocked by mime action.', $effectiveUrl));
throw new \Exception(\sprintf('This is url "%s" is blocked by mime action.', $effectiveUrl));
}

$infos = $infos->withHtml('<a href="' . $effectiveUrl . '">Download ' . $mimeInfo['name'] . '</a>');
Expand Down
4 changes: 2 additions & 2 deletions src/HttpClient/Plugin/CookiePlugin.php
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ public function handleRequest(RequestInterface $request, callable $next, callabl
continue;
}

$cookies[] = sprintf('%s=%s', $cookie->getName(), $cookie->getValue());
$cookies[] = \sprintf('%s=%s', $cookie->getName(), $cookie->getValue());
}

if (!empty($cookies)) {
Expand Down Expand Up @@ -114,7 +114,7 @@ private function createCookie(RequestInterface $request, string $setCookieHeader
try {
$expires = CookieUtil::parseDate((string) $value);
} catch (UnexpectedValueException $e) {
throw new TransferException(sprintf('Cookie header `%s` expires value `%s` could not be converted to date', $name, $value), 0, $e);
throw new TransferException(\sprintf('Cookie header `%s` expires value `%s` could not be converted to date', $name, $value), 0, $e);
}

break;
Expand Down
8 changes: 4 additions & 4 deletions src/OptionsResolver/ArrayStringOptionsTrait.php
Original file line number Diff line number Diff line change
Expand Up @@ -16,19 +16,19 @@ public function validateArray(array $array, string $option, ?string $key = null)
if (null === $key) {
foreach ($array as $arrayKey => $arrayValue) {
if (!\is_string($arrayKey)) {
throw new InvalidOptionsException(sprintf('The option "%s" with key "%s" is expected to be of type "string", but is of type "%s".', $option, $arrayKey, get_debug_type($arrayKey)));
throw new InvalidOptionsException(\sprintf('The option "%s" with key "%s" is expected to be of type "string", but is of type "%s".', $option, $arrayKey, get_debug_type($arrayKey)));
}
if (!\is_string($arrayValue)) {
throw new InvalidOptionsException(sprintf('The option "%s" with value "%s" is expected to be of type "string", but is of type "%s".', $option, $arrayValue, get_debug_type($arrayValue)));
throw new InvalidOptionsException(\sprintf('The option "%s" with value "%s" is expected to be of type "string", but is of type "%s".', $option, $arrayValue, get_debug_type($arrayValue)));
}
}
} elseif (!empty($array[$key])) {
foreach ($array[$key] as $arrayKey => $arrayValue) {
if (!\is_string($arrayKey)) {
throw new InvalidOptionsException(sprintf('The option "%s" with key "%s" is expected to be of type "string", but is of type "%s".', $option, $arrayKey, get_debug_type($arrayKey)));
throw new InvalidOptionsException(\sprintf('The option "%s" with key "%s" is expected to be of type "string", but is of type "%s".', $option, $arrayKey, get_debug_type($arrayKey)));
}
if (!\is_string($arrayValue)) {
throw new InvalidOptionsException(sprintf('The option "%s" with value "%s" is expected to be of type "string", but is of type "%s".', $option, $arrayValue, get_debug_type($arrayValue)));
throw new InvalidOptionsException(\sprintf('The option "%s" with value "%s" is expected to be of type "string", but is of type "%s".', $option, $arrayValue, get_debug_type($arrayValue)));
}
}
}
Expand Down
6 changes: 6 additions & 0 deletions src/SiteConfig/ConfigBuilder.php
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,8 @@ public function addToCache(string $key, SiteConfig $config): void
$key = strtolower($key);
if (str_starts_with($key, 'www.')) {
$key = substr($key, 4);
// For PHPStan on PHP < 8.0: it cannot fail since the prefix checked above has four characters.
\assert(false !== $key);
}

if ($config->cache_key) {
Expand All @@ -90,6 +92,8 @@ public function getCachedVersion(string $key): ?SiteConfig
$key = strtolower($key);
if (str_starts_with($key, 'www.')) {
$key = substr($key, 4);
// For PHPStan on PHP < 8.0: it cannot fail since the prefix checked above has four characters.
\assert(false !== $key);
}

if (\array_key_exists($key, $this->cache)) {
Expand Down Expand Up @@ -127,6 +131,8 @@ public function buildForHost(string $host, bool $addToCache = true): SiteConfig
$host = strtolower($host);
if (str_starts_with($host, 'www.')) {
$host = substr($host, 4);
// For PHPStan on PHP < 8.0: it cannot fail since the prefix checked above has four characters.
\assert(false !== $host);
}

// is merged version already cached?
Expand Down
10 changes: 8 additions & 2 deletions tests/Extractor/ContentExtractorTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -469,12 +469,18 @@ public function testApplyStripDisplayNoneAndInstapaper(string $html, string $rem
public function dataForStripAttr(): array
{
return [
[['//*/@class'], '<html><body><div class="hello world"><i class="class">bar</i>class="foo"' . str_repeat('this is the best part of the show', 10) . ' <a class="hc" href="void">link</a></div></body></html>', [
[
['//*/@class'],
'<html><body><div class="hello world"><i class="class">bar</i>class="foo"' . str_repeat('this is the best part of the show', 10) . ' <a class="hc" href="void">link</a></div></body></html>',
[
'removedContent' => ['class="class"', 'class="hello world"', 'class="hc"'],
'keptContent' => ['class="foo"', '<a href="void"', '<em>bar'],
],
],
[['//img/@class', '//p/@class'], '<html><body><img class="bar-class" src="void" /><a class="hello" href="void">link</a> <p class="yes">' . str_repeat('this is the best part of the show', 10) . '</p></body></html>', [
[
['//img/@class', '//p/@class'],
'<html><body><img class="bar-class" src="void" /><a class="hello" href="void">link</a> <p class="yes">' . str_repeat('this is the best part of the show', 10) . '</p></body></html>',
[
'removedContent' => ['class="bar-class"', 'class="yes"'],
'keptContent' => ['class="hello"'],
],
Expand Down
8 changes: 0 additions & 8 deletions tests/Extractor/HttpClientTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -266,14 +266,6 @@ public function testTimeout(): void
$isGuzzle = true;
$guzzle = new \GuzzleHttp\Client(['timeout' => 2]);
$adapter = new \Http\Adapter\Guzzle7\Client($guzzle);
} elseif (class_exists('Http\Adapter\Guzzle6\Client')) {
$isGuzzle = true;
$guzzle = new \GuzzleHttp\Client(['timeout' => 2]);
$adapter = new \Http\Adapter\Guzzle6\Client($guzzle);
} elseif (class_exists('Http\Adapter\Guzzle5\Client')) {
$isGuzzle = true;
$guzzle = new \GuzzleHttp\Client(['defaults' => ['timeout' => 2]]);
$adapter = new \Http\Adapter\Guzzle5\Client($guzzle);
} elseif (class_exists('Http\Client\Curl\Client')) {
$isCurl = true;
$adapter = new \Http\Client\Curl\Client(
Expand Down
5 changes: 3 additions & 2 deletions tests/GrabyTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,8 @@ public function dataForFetchContent(): array

$test = (string) file_get_contents($file->getRealpath());

preg_match('/-----URL-----\s*(.*?)\s*-----URL_EFFECTIVE-----\s*(.*?)\s*-----HEADER-----\s*(.*?)\s*-----LANGUAGE-----\s*(.*?)\s*-----AUTHOR-----\s*(.*?)\s*-----TITLE-----\s*(.*?)\s*-----SUMMARY-----\s*(.*?)\s*-----RAW_CONTENT-----\s*(.*?)\s*(------RAW_CONTENT2-----\s*(.*?)\s*)?----PARSED_CONTENT-----\s*(.*)\s*/sx', $test, $match);
$parses = preg_match('/-----URL-----\s*(.*?)\s*-----URL_EFFECTIVE-----\s*(.*?)\s*-----HEADER-----\s*(.*?)\s*-----LANGUAGE-----\s*(.*?)\s*-----AUTHOR-----\s*(.*?)\s*-----TITLE-----\s*(.*?)\s*-----SUMMARY-----\s*(.*?)\s*-----RAW_CONTENT-----\s*(.*?)\s*(------RAW_CONTENT2-----\s*(.*?)\s*)?----PARSED_CONTENT-----\s*(.*)\s*/sx', $test, $match);
\assert(1 === $parses, \sprintf('File %s does not match the required pattern', $file->getRealpath()));

$tests[] = [
$match[1], // url
Expand Down Expand Up @@ -221,7 +222,7 @@ public function testMimeTypeActionExclude(): void

$graby = new Graby([
'content_type_exc' => [
'application/x-msdownload' => ['action' => 'exclude', 'name' => 'we do not want virus'],
'application/x-msdownload' => ['action' => 'exclude', 'name' => 'we do not want virus'],
],
], $httpMockClient);

Expand Down
Loading