Plugin doesn't work when using http proxy #80

Koc · 2018-09-14T12:28:24Z

Steps required to reproduce the problem

use proxy for accessing Internet
define env vars for using proxy https://gist.github.com/ozh/9751639
create project using composer, update some vendors - composer works ok
try run composer normalize

Expected Result

composer.json was normalized.

Actual Result

docker-compose exec app composer normalize -vvv
Reading ./composer.json
Loading config file ./composer.json
Checked CA file /etc/ssl/certs/ca-certificates.crt: valid
Executing command (/srv/app): git branch --no-color --no-abbrev -v
Reading /root/.composer/composer.json
Loading config file /root/.composer/composer.json
Reading /srv/app/vendor/composer/installed.json
Reading /root/.composer/vendor/composer/installed.json
Loading plugin PackageVersions\Installer
Loading plugin Symfony\Flex\Flex
Loading plugin Localheinz\Composer\Normalize\NormalizePlugin
Loading plugin Pyrech\ComposerChangelogs\ChangelogsPlugin
Running 1.7.2 (2018-08-16 16:57:12) with PHP 7.2.9 on Linux / 4.15.0-29-generic
Do not run Composer as root/super user! See https://getcomposer.org/root for details
Reading ./composer.json
Loading config file ./composer.json
Executing command (/srv/app): git branch --no-color --no-abbrev -v
Reading /root/.composer/composer.json
Loading config file /root/.composer/composer.json
Reading /srv/app/vendor/composer/installed.json
Reading /root/.composer/vendor/composer/installed.json
Loading plugin PackageVersions\Installer_composer_tmp0
Loading plugin Symfony\Flex\Flex_composer_tmp1
Loading plugin Localheinz\Composer\Normalize\NormalizePlugin_composer_tmp2
Loading plugin Pyrech\ComposerChangelogs\ChangelogsPlugin_composer_tmp3
Reading ./composer.lock
Reading ./composer.lock
Schema URI "https://getcomposer.org/schema.json" does not reference a document that could be read.

see for reference how composer handles HTTP_PROXY env var https://github.com/composer/composer/blob/d5a9d86ee4e934d06073521e630556d4f337629c/src/Composer/Util/StreamContextFactory.php#L43

The text was updated successfully, but these errors were encountered:

localheinz · 2018-09-14T12:52:05Z

Thank you for your report, @Koc!

I will take a look on the weekend.

localheinz · 2018-10-07T17:39:55Z

@Koc

Apologies for not handling this yet, I promise I will!

Koc · 2018-10-07T17:56:48Z

Thank you, I hope on this)

b1rdex · 2018-10-15T06:53:45Z

@Koc could you please run composer diagnose and curl -I https://getcomposer.org/schema.json when proxy is set? Do you set only http_proxy var or https_proxy too? Is https_proxy set up to force http proto?

localheinz · 2018-10-22T13:05:53Z

@Koc

I already have a simple idea how to fix it, but won't have time before the weekend, unfortunately!

Koc · 2018-10-22T14:23:20Z

composer diagnose
Do not run Composer as root/super user! See https://getcomposer.org/root for details
Checking composer.json: WARNING
require.symfony/asset : unbound version constraints (*) should be avoided
require.symfony/console : unbound version constraints (*) should be avoided
require.symfony/framework-bundle : unbound version constraints (*) should be avoided
require.symfony/messenger : unbound version constraints (*) should be avoided
require.symfony/translation : unbound version constraints (*) should be avoided
require.symfony/twig-bundle : unbound version constraints (*) should be avoided
require.symfony/validator : unbound version constraints (*) should be avoided
require.symfony/yaml : unbound version constraints (*) should be avoided
Checking platform settings: OK
Checking git settings: OK
Checking http connectivity to packagist: OK
Checking https connectivity to packagist: OK
Checking HTTP proxy: OK
Checking HTTP proxy support for request_fulluri: OK
Checking HTTPS proxy support for request_fulluri: OK
Checking github.com rate limit: OK
Checking disk free space: OK
Checking pubkeys: FAIL
Missing pubkey for tags verification
Missing pubkey for dev verification
Run composer self-update --update-keys to set them up
Checking composer version: OK
Composer version: 1.7.2
PHP version: 7.2.10
PHP binary path: /usr/local/bin/php

docker-compose exec app curl -I https://getcomposer.org/schema.json
HTTP/1.1 200 Connection established

HTTP/1.1 200 OK
Server: nginx
Content-Type: application/json
Connection: keep-alive
Vary: Accept-Encoding
Cache-Control: no-cache
Strict-Transport-Security: max-age=31104000
Date: Mon, 22 Oct 2018 14:22:36 GMT

both http_proxy and https_proxy env vars are properly configured.

Koc · 2018-11-01T09:12:02Z

@localheinz I'm afraid that thanges should be done inside external deps https://github.com/justinrainbow/json-schema/tree/master/src/JsonSchema/Uri/Retrievers

or you can download schema using Composer\Util\StreamContextFactory::getContext, put to tmp and use downloaded copy

localheinz · 2018-11-01T12:50:44Z

@Koc

Yes, I was thinking about a ChainUriRetriever, which composes the available retrievers.

localheinz · 2018-11-04T20:13:16Z

@Koc

Can you do me a favour and require the fix/proxy branch to verify it works for you?

$ composer require "localheinz/composer-normalize:dev-fix/proxy#92de02b as 0.10.0"

You might need to set the minimum stability to dev.

Koc · 2018-11-06T11:10:26Z

I've tryed (deleted and reinstalled global package).

composer normalize -vvv
Reading ./composer.json
Loading config file ./composer.json
Checked CA file /etc/ssl/certs/ca-certificates.crt: valid
Executing command (/srv/app): git branch --no-color --no-abbrev -v
Reading /root/.composer/composer.json
Loading config file /root/.composer/composer.json
Reading /srv/app/vendor/composer/installed.json
Reading /root/.composer/vendor/composer/installed.json
Loading plugin PackageVersions\Installer
Loading plugin Symfony\Flex\Flex
Loading plugin Localheinz\Composer\Normalize\NormalizePlugin
Running 1.7.2 (2018-08-16 16:57:12) with PHP 7.2.10 on Linux / 4.15.0-29-generic
Do not run Composer as root/super user! See https://getcomposer.org/root for details
Reading ./composer.json
Loading config file ./composer.json
Executing command (/srv/app): git branch --no-color --no-abbrev -v
Reading /root/.composer/composer.json
Loading config file /root/.composer/composer.json
Reading /srv/app/vendor/composer/installed.json
Reading /root/.composer/vendor/composer/installed.json
Loading plugin PackageVersions\Installer_composer_tmp0
Loading plugin Symfony\Flex\Flex_composer_tmp1
Loading plugin Localheinz\Composer\Normalize\NormalizePlugin_composer_tmp2
Reading ./composer.lock
Reading ./composer.lock
Original JSON is not valid according to schema "https://getcomposer.org/schema.json".

composer normalize --dry-run -vvv
Reading ./composer.json
Loading config file ./composer.json
Checked CA file /etc/ssl/certs/ca-certificates.crt: valid
Executing command (/srv/app): git branch --no-color --no-abbrev -v
Reading /root/.composer/composer.json
Loading config file /root/.composer/composer.json
Reading /srv/app/vendor/composer/installed.json
Reading /root/.composer/vendor/composer/installed.json
Loading plugin PackageVersions\Installer
Loading plugin Symfony\Flex\Flex
Loading plugin Localheinz\Composer\Normalize\NormalizePlugin
Running 1.7.2 (2018-08-16 16:57:12) with PHP 7.2.10 on Linux / 4.15.0-29-generic
Do not run Composer as root/super user! See https://getcomposer.org/root for details
Reading ./composer.json
Loading config file ./composer.json
Executing command (/srv/app): git branch --no-color --no-abbrev -v
Reading /root/.composer/composer.json
Loading config file /root/.composer/composer.json
Reading /srv/app/vendor/composer/installed.json
Reading /root/.composer/vendor/composer/installed.json
Loading plugin PackageVersions\Installer_composer_tmp0
Loading plugin Symfony\Flex\Flex_composer_tmp1
Loading plugin Localheinz\Composer\Normalize\NormalizePlugin_composer_tmp2
Reading ./composer.lock
Reading ./composer.lock
Schema URI "https://getcomposer.org/schema.json" does not reference a document with valid JSON syntax.

and my composer.json wasn't changed.

localheinz · 2018-11-07T06:55:27Z

@Koc

😢

I'll investigate further!

localheinz · 2018-12-24T14:56:37Z

@Koc

By any chance, can you provide a minmimum Docker setup that allows me to reproduce and fix this issue?

Koc · 2018-12-25T10:11:53Z

I had swiched the company to another so hasn't access to code anymore (and proxy too :) ).

It was standard alpine fpm image with some additional php extensions, libssl and localheinz/composer-normalize installed globally wia Dockerfile. http_proxy and https_proxy env vars pass from host to containers. Something like http_proxy=http://username:password@proxy.hostname:8080. Composer (including Flex parallel downloader and even simple-phpunit) works fine with this settings.

drupol · 2019-02-18T14:57:16Z

We are also having the same issue at European Commission.

drupol · 2019-02-18T17:31:37Z

I found a workaround by updating the package composer-json-normalize.

The idea is to download a local copy of composer and use the local schema file instead of using the remote one.

We still need to discuss about it.

drupol · 2019-02-19T08:55:42Z

Here's what I have in mind:

1.x...drupol:wip

As the branch name suggests it, it's a WIP but right now, it's fully working.

Basically, the plugins tries multiple methods to get the schema, here they are:

The plugin tries to download the remote schema.
The plugin check if the package composer/composer is available and get the schema from there
The plugin get the schema from the package localheinz/composer-normalize

I will try to polish it today, but if you can give some feedback already, it would be nice.

drupol · 2019-02-19T09:01:48Z

Have a look at this, from an European Commission computer: https://pastebin.com/5C6aDm5G

stale · 2019-04-20T18:51:23Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale · 2019-06-19T20:37:39Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale · 2019-08-18T22:56:02Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

localheinz · 2019-08-22T09:35:04Z

@b1rdex @drupol @Koc

Can you try

$ composer global require localheinz/composer-normalize:dev-feature/schema#1f965d4

or

$ composer require localheinz/composer-normalize:dev-feature/schema#1f965d4

and see if that works for you?

If it does, I'll cut a release soon!

localheinz self-assigned this Sep 14, 2018

localheinz added the bug label Sep 14, 2018

This was referenced Nov 4, 2018

Enhancement: Implement ChainUriRetriever ergebnis/json-normalizer#102

Merged

Enhancement: Use ChainUriRetriever ergebnis/json-normalizer#104

Merged

Fix: Update localheinz/json-normalizer #103

Closed

drupol mentioned this issue Feb 18, 2019

Issue #22: Add composer/composer as a dependency ergebnis/composer-json-normalizer#23

Closed

3 tasks

This was referenced Feb 19, 2019

Enhancement: Implements a mechanism to ensure the existence of composer.json's schema. #170

Closed

Enhancement: Implements a mechanism to ensure the existence of composer.json's schema. #171

Closed

Issue #171: Ensure composer.json file existence #172

Closed

stale bot added the stale label Apr 20, 2019

localheinz removed the stale label Apr 20, 2019

stale bot added the stale label Jun 19, 2019

localheinz removed the stale label Jun 19, 2019

stale bot added the stale label Aug 18, 2019

localheinz removed the stale label Aug 19, 2019

This was referenced Aug 22, 2019

Work when no internet available #187

Closed

Enhancement: Resolve local and fall back to remote schema #190

Merged

localheinz closed this as completed in #190 Aug 22, 2019

localheinz mentioned this issue Dec 15, 2019

Update composer-schema.json file path #255

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Plugin doesn't work when using http proxy #80

Plugin doesn't work when using http proxy #80

Koc commented Sep 14, 2018

localheinz commented Sep 14, 2018

localheinz commented Oct 7, 2018

Koc commented Oct 7, 2018 •

edited

Loading

b1rdex commented Oct 15, 2018 •

edited

Loading

localheinz commented Oct 22, 2018

Koc commented Oct 22, 2018

Koc commented Nov 1, 2018

localheinz commented Nov 1, 2018

localheinz commented Nov 4, 2018

Koc commented Nov 6, 2018

localheinz commented Nov 7, 2018

localheinz commented Dec 24, 2018

Koc commented Dec 25, 2018 •

edited

Loading

drupol commented Feb 18, 2019

drupol commented Feb 18, 2019

drupol commented Feb 19, 2019 •

edited

Loading

drupol commented Feb 19, 2019 •

edited

Loading

stale bot commented Apr 20, 2019

stale bot commented Jun 19, 2019

stale bot commented Aug 18, 2019

localheinz commented Aug 22, 2019

Plugin doesn't work when using http proxy #80

Plugin doesn't work when using http proxy #80

Comments

Koc commented Sep 14, 2018

Steps required to reproduce the problem

Expected Result

Actual Result

localheinz commented Sep 14, 2018

localheinz commented Oct 7, 2018

Koc commented Oct 7, 2018 • edited Loading

b1rdex commented Oct 15, 2018 • edited Loading

localheinz commented Oct 22, 2018

Koc commented Oct 22, 2018

Koc commented Nov 1, 2018

localheinz commented Nov 1, 2018

localheinz commented Nov 4, 2018

Koc commented Nov 6, 2018

localheinz commented Nov 7, 2018

localheinz commented Dec 24, 2018

Koc commented Dec 25, 2018 • edited Loading

drupol commented Feb 18, 2019

drupol commented Feb 18, 2019

drupol commented Feb 19, 2019 • edited Loading

drupol commented Feb 19, 2019 • edited Loading

stale bot commented Apr 20, 2019

stale bot commented Jun 19, 2019

stale bot commented Aug 18, 2019

localheinz commented Aug 22, 2019

Koc commented Oct 7, 2018 •

edited

Loading

b1rdex commented Oct 15, 2018 •

edited

Loading

Koc commented Dec 25, 2018 •

edited

Loading

drupol commented Feb 19, 2019 •

edited

Loading

drupol commented Feb 19, 2019 •

edited

Loading