Security: Private Access Token sent to wp.org

[hostz-frank]

Updating WP plugins and themes from private repos is a great thing that the Github Updater is capable to do.

Unfortunately the WordPress update checks send all headers (including even descriptions) of any plugins and themes, whether enabled or not, to wordpress.org.

I would consider this a security issue.
In general I feel that all plugins updated via Github should not be asking wp.org for updates because thats obviously quite useless.

[afragen]

Where is this plugin sending anything to WP.org?

[hostz-frank]

Every WP plugin sends its complete header and description to wp.org frequently - you don't need a special one to do this.

Look at the output of:

function let_me_see( $args, $url ) {
    if ( false === strpos( $url, 'api.wordpress.org/plugins/update-check' ) ) {
        return $args;
    }
    print '<pre>' . print_r( json_decode( $args['body']['plugins'] ), 1 ) . '</pre>';
    return $args;
}
add_filter( 'http_request_args', 'let_me_see', 5, 2 );

while going to wp-admin/plugins.php or wp-admin/update-core.php.

These are the args for the plugin update request and are sent all the time, at least twice a day by cron and when you're not logging in to your site on this day. But if you do, these requests are countless.

[afragen]

@hostz-frank it seems like this issue is fundamental to the way that WP functions. Is the problem the actual sending of all headers of every plugin or what WP.org might be doing with that data?

[hostz-frank]

Hi, these are my points:

1.) Github Updates don't need to be checked on wp.org and update requests should be meanigful.
2.) The wp.org-API expects their data format and not yours (Github headers are sent even with pure WP-plugins) - maybe that's why the plugin is not allowed to reside on wp.org??
3.) Secret data are generally meant to be kept secret. There are bad people even if not everyone is a bad guy. This has nothing to do with wp.org. Network traffic is not private as well.

Do you agree?

[afragen]

Let me try to address each of your points.

1. It seems that your complaint is in the way WordPress handles updates. If WordPress scans and sends all plugin headers there's no much I can do about how their core update mechanism functions. It always has been my goal and intent to utilize these core functions to ensure this plugin functions as effectively as possible.
2. Please read Why isn't this allowed in the wp.org plugin repo? #34 , your assumption here is wrong.
3. If WP.org is scraping all the headers of every installed plugin, I don't believe we will alter this behavior. If you're concerned that this data is going to WP.org over http and not https, I think that's a question for WP.org. I actually do not know the answer to this. I do know that actually plugin update downloads seem to be over https. All the data transfers from GitHub are over https.

[hostz-frank]

Hi Andy,
I'm sorry about my rudeness. I know it's great to have a plugin like this because I really need the functionality and I feel there is a strong need of many people to be able to update from Github. So your plugin saved my life at a certain point and I do thank you for this!

And of course: you're not responsible for my needs. It is a weakness of mine to see the potential of a project that is almost an ideal solution and then get impatient about that it is just on its way. It would be much better for sure to focus on how to get higher with what we can do on this.

So I would consider your plugin an even greater one if it could keep wp.org's eyes away from plugins and themes that are not meant for wp.org und with wich they cannot do very much. Since this step could enhance privacy also is for me a good reason to look even more in this direction.

Andy, I will have a look at the code and let you know about ideas which may occur and hopefully I can do more on this than just complaining.

Appreciating your work,
Frank

[afragen]

Frank, I didn't consider your questions or your responses rude. Not to worry.

If you can come up with a way too not send GitHub hosted plugins to WP.org I'm open to PRs.

Just an FYI. You can actually host your plugin on both GitHub and WP.org. If the branch is set to master then updates will come from WP.org. If the branch is develop or something other than master, then updates will come from GitHub.

I personally use this method for several of my WP.org plugins so that I have a simpler time pushing development versions to test sites.

[hostz-frank]

Hi Andy,
your FYI is an interesting point. To catch it right: why do you add the lines "Github Updater URI: ..." and "Github Branch: master" to headers on wp.org, if you could simply add them not and have the same effect this way?

[afragen]

I do my development in git/GitHub. I use a script to push updates from my local git repo to WP.org. It's much simpler for me to leave the headers in or if someone else is helping me test a beta they will get updates from GitHub. If they ever want to switch to the WP.org version all they have to do is change the branch from develop to master.

[afragen]

A header is just a comment. It's not any sort of classified information. The exception being for private repos, but those aren't on WP.org anyway.

I think the key is to determine what happens to the headers sent to WP.org and is there any way to stop them from being sent without disabling the plugin.

[hostz-frank]

Yes.
What happens if you set a Github URI into plugin headers but no branch? Will updates come from wp.org or from Github?

[afragen]

Having no branch is the same as having a branch set to master. If the plugin is hosted on WP.org it will only receive updates from WP.org. In this case.

[hostz-frank]

So this would fetch updates from wp.org, if the plugin would be there:

GitHub Plugin URI: https://github.com/afragen/github-updater
GitHub Branch:     master

And since it's not on wp.org it'll be fetched from Github? I think, this is essential behaviour and you should really mention this in the plugins README ...

Because I would certainly expect, that if I declare an update source inside a header that points to Github is has to come from there. Moreover, in fact you provide not a Github Updater but a mixture of an Github updater and a WordPress plugin and theme update replacement.

So your btw FYI is a bit like a shock for me at the moment :-)

[afragen]

Take a look at an actual plugin in the repo, http://wordpress.org/plugins/the-events-calendar-category-colors/

Also take a look at https://github.com/afragen/github-updater#hosting-plugin-in-wporg-repository

See these commits, 65e13dc 992f28f

It's not so much a mixed updater as I'm trying to "do the right thing". If the dev has their plugin up on WP.org then I'm going to respect that as the canonical release. Otherwise the update would never come from WP.org if the user has this plugin enabled.

I didn't mean to shock you.

[hostz-frank]

My conclusion is for now: I tend to change the behaviour of the Github Updater by forking it. Chances are, that changes will be fundamental. For me it's like this: if I develop a wp.org hosted plugin, it should - regarding to headers - follow the rules of a WP plugin, whatever CVS I use.

If I decide for a plugin to change its update repo to Github then I have to add the Github Updater headers and make it this way a Github hosted plugin. After this step the update source is on Github and nowhere else. If a named branch is master then it's the master of Github and not a wp.org-"master".

This approach would lead away from your actual Update Router plugin to a real Github Updater and it would make the plugin behave different.

1. The plugin checks all headers before anything will be sent anywhere. If "Github Plugin URI" is found then
2. perform the update from Github (what else?) AND
3. clear the plugin list which will be sent to wp.org from plugins that update via GH.
4. Now wp.org hosted plugins will get updated.
   This way wp.org would never see plugins, which are GH hosted plugins, and why should they? It's simply not their job and so they will - to come back to this issue - never see a secret token of a private Github project. - At the end this is what I expected to find with your plugin.

And I do thank you for getting me at this point and for your plugin as it is now.

[afragen]

If you simply don't include any of the update headers in the plugin that is on WP.org then that plugin will only ever poll and update from WP.org.

If you come up with a method to remove the GitHub plugins from being polled and sent in to WP.org and still maintain full update function I would really appreciate a PR.

Thanks Frank. If you have any difficulty understanding what I did with the code let me know.

[afragen]

@hostz-frank I checked with the WP.org maintainers and while it is true that headers are sent to WP.org if there are no matches -- the headers are ignored. Headers aren't stored. Download counts and numbers of plugins are stored but no raw data.

[Otto42]

Hi! Otto at WordPress here.

We do not store the raw header data, at all, in any form, period.

The reason core sends this data is because WP never really came up with a mechanism to uniquely identify plugins. There's nothing special about any given plugin, no specific "clearinghouse" as such. I argued for a GUID at one point, but it never really got any traction.

The plugin headers give the server something to compare and match on. It's not perfect, and it's easy to fool/break if you really want to try, but the bottom line is that wordpress.org does not store this data in any way. It can't, honestly. Hell, just storing the sheer number of requests made is a bit much. The "count" alone is overwhelming.

We do the best we can for plugins in our directory, but for non-matching items, the data is tossed and nothing whatsoever is stored. We're trying to serve our users by giving them updates. We're not interesting in spying on you, and realistically, the data in a plugin header isn't very interesting. The total count is useful, and we do store download counts and things like that, for all ZIP files that we serve from the site. At least, we try to. There have been issues with the stats in the past.

Still, we're interested in overall stats, and no raw data is stored by WordPress.org, because the fact is that when you're everywhere, you're more interested in the overall perspective, not the individual one. :)

If you have specific and detailed questions about this, I'm happy to try to answer them one-on-one. My email address is otto@wordpress.org. Feel free to email me :)