Moderation needed. #14621
Comments
ghost
added
the
|
|
I really really think that any new PackageIdentifer should have to be checked by someone on the winget team (or if they want to start a recognized contributor system I’d throw my hat in the ring). Fully automated adding of brand new packages will have tons of duplicates. |
|
What can I do to make my PackageIdentifier better? |
Oh! my bad, I checked the download url |
oh ok |
denelon
added
Issue-Feature
|
I'll talk with the team tomorrow when everyone is back from holiday. One of the options could be requiring a "second" approver on a "new" manifest in a "new" directory. The bot has a concept that might work for that scenario. I just don't want to put too much friction and time delay for people submitting manifests, nor too much pressure on "moderators". We've got a feature on the backlog to detect duplicates. It's more of a warning than a blocking action. We have some expected "valid" rename scenarios. I also put something similar on the wingetcreate backlog too. |
|
Would one of you like to take a stab at "How to submit a manifest" as an Issue? I'll pin it if it's got decent directions on finding the right installer type, testing, etc... |
If possible, people with XX amount of commits should skip the second approval since they most likely already learned how the repo/packages works.
People use wingetcreate which skips ALL of the pre checks, they don't even look at the PR comments. |
|
Maybe wingetcreate manifests should require manual review? I know we are trying not to waste the moderator's time, but since they are committing known bad metadata by default (for example, not updating the ProductCode attribute), the bot doesn't realize it and then someone who knows that the bug exists has to go back and fix all of the errors (or live with the metadata being wrong, which is a tragedy ;D). |
|
@denelon I am kinda appalled we are where we are right now, and Winget was announced as being GA/1.0 when core issues like this on the trust and safety side haven't been addressed. I am absolutely baffled how this repo doesn't link to anything about what release channels or versions should be selected in the package manager, or any sort of common behavior about how some choices should be made. There's nothing telling me why I should believe I can trust anything the automated pipeline has accepted into the repo, or that the downloads are in any way safe. Right now, using WinGet would be at best no safer than randomly grabbing installer files off popup ads and running them. I am not sure anything can fix this repo short of a "nuke it from orbit and start over" approach, but nobody security-side at Microsoft seems to have noticed this repo is here? I found a policy document on the Docs site, but it looks more like legal boilerplate, and lacks any cohesive answers on questions which are crucial to package managers. But in the most severe case, in light of these glaring and painful issues, it's concerning your primary focus is preventing delay in approvals. Shut the bots down and start the manual review process, and roll this back to beta until there's clear documentation about how packages should be added or not. |
|
@ocdtrekkie every PR up until #12424 (22 days ago) have been manually/semi manually approved, and every url is being run trough VirusTotal so there is some sort of protection in place. And i do very much agree that Microsoft have not thought very much about this since one idiot could ruin the whole thing with a fully automated system, every Package Manager have some kind of Manual Approval/Maintainer of that Program. |
|
What would happen if I sent a PR that caused the Chrome package to install Firefox instead? Would any checks prohibit it from being automatically merged? There's a very wide variety of things you can do without pushing an executable that will flag on VirusTotal. But there's a lot of issues here Microsoft doesn't even seem to be prepared for. Even if you knew for sure every manifest was legitimate and correct, use of WinGet would still lead the Windows platform to a worse security posture than it has now: Without any ownership by either Microsoft or official app developer channels, WinGet package manifests may or may not be updated in a timely manner, if at all, and without any practices or policy about architectures, release channels, deployment configurations, etc. users may be getting 32-bit versions on their 64-bit machine when 64-bit versions exist, or be stuck on very old versions, or get broken releases instead of stable ones. Third party package managers like PDQ generally are going to maintain a restricted list of packages they maintain, because they're taking responsibility for ensuring they're pushing the latest, working updates of a given package they support. That approach would be possible for Microsoft, but it would need to be much more restrictive, and manually reviewed. The other alternative is one Microsoft has tried before, it's the Windows Store: Only support software released to the repository by the developer of said package. Obviously it's a lot less useful if major brands don't participate, but then the responsibility of keeping the deployment working and up to date is on the developers themselves. Look at Chocolately's moderation document: https://docs.chocolatey.org/en-us/community-repository/moderation/ It covers everything. Release channels. Processor architectures. Licensing questions. Packages are at minimum tied to specific maintainers responsible for them, such that you can trust it hasn't been changed by someone else in the past hour since you looked at the repo. And of course, everything is robustly and manually reviewed by humans before anything is made available to deploy to anyone's computers. |
|
I think this repo is too free for anyone to edit/add things the way they like, which should not be the case at all. I would rather have people tell the application they want here, by creating a issue rather than do it themselves (because they do it badly). After V1 announcement it was kind of overwhelming and frustrating here. Soo many duplicate/bad manifests one after other. There is documentation on how to submit a manifest correctly but no one seems to read it. I think we should stop accepting new manifests ASAP for a while and spend time correcting bad manifests with very little information. Automation has eased process for moderator but with its negative consequences.
Optional But currently my main point is that Moderation should be taken seriously. Group of moderators should be created because this is not a one person job and so that winget's repository remains sane. Decision for this issue needs to be made immediately! @denelon What do you think about it? |
|
"So now, when we face a choice between adding features and resolving security issues, we need to choose security." - Bill Gates https://www.wired.com/2002/01/bill-gates-trustworthy-computing/ As it stands right now, WinGet adds a feature at an exceptional security cost. If Microsoft isn't interested in implementing a review team, I sincerely believe Microsoft should look at it's options to roll back and deprecate WinGet. |
|
The "automated merge" has been stopped. We're making changes to the wingetcreate tool to detect duplicates, and require "update" vs. "new" for existing packages. |
|
@KaranKad I created a discussion topic for Moderation. |
|
Yes! please think about creating a moderation team. I think its the best solution. For manifests to be perfect, only moderators should be allowed submission of new or updating existing manifests. Because people submitting new applications using wingetcreate, don't really do it that well either. |
|
@jedieaston since you volunteered, we've added the logic so that when you "approve" a PR by reviewing it, the "Moderator-Approved" label is added, and if the "Azure-Pipeline-Passed" and the "Validation-Completed" labels are present, the PR will be merged. Please test this when you get a chance. |
|
I've approved 3 PRs and they've merged successfully, thanks! |
|
Just to say, that as the developer of an app I was about to submit here, I found someone else had an open PR on the app only hours before, with an erroneous link that would have caused validation issues as soon as I push a new version of the app. Fortunately I was able to get in touch with this person and ask them to close their PR so I could submit one with the proper information. I have to say I was extremely surprised to find that someone conpletely unrelated to my app (albeit well intentioned) was able to fill out all the information, accept the CLA, etc., on my behalf without my knowing anything about it. Winget devs, how didn't you think this through? At the very least there needs to be a check that the person submitting a manifest has the right to accept agreements, etc. Contrast this with the rigorous checks of organizations on the Microsoft Store -- it took several months for us to get the right documentation to open an org account on the MS Store -- and you see the problem immediately. And what are you going to do about "inconvenient" but legal issues such as age ratings, etc.? IMHO winget should have started with packages already validated and approved in the MS Store, and then opened up more gradually from there with the right checks in place. A pain, yes, but this is Windows, not Linux. Linux has like 3% desktop market share, mostly used by responsible devs, compared to Windows 80%-ish, used by everyone from 8-year-olds to 80-year-olds. |
|
@Jaifroid |
Albeit a little unrelated but having submitted a few packages recently. The process of generating ProductCode itself is convoluted and not very well defined at least to my knowledge. |
|
@KaranKad I've created a project to track work for the next "milestone". I've marked this work item "In Progress". I didn't want to close it until the Issues listed above have been corrected. Let me know when you feel like it's safe to close this. I don't believe we have figured out our final solution for moderation, but we do have something in place currently to help. We will continue to refine how moderation should work for the community repository. We will continue by gathering feedback from the community, and creating new Issues as the program needs to adapt. Thank you for taking the time to report the issue and provide specifics. |
Sadly, it's useless... In my culture, there is a slang "too long to have look at". This will also fit for all the EULA, issue, template, etc. You cannot imagine everyone will first look through all the documents before they take action, especially at this public repo. There are many people, who know little about how to contribute and respect others' works, trying to write something. We should find a more efficent way, maybe peer review. |
The goal of such a document need not be that everyone exhaustively be expected to read it, but that it's there for both contributors and moderators and serves as the bible to use when determining the answer to controversial decisions. |
|
It would be nice if the moderation policy included a section on submitting non-stable versions of software. Currently 7zip alpha 21.02 is offered as the upgrade for 7zip 19.00. IMHO this should not be a default, as users are likely to prefer stable versions. What happens if there's another release from the 19.xx branch later? Will winget select it as the default or will it keep the 21.xx alpha branch? |
7-Zip have caused the issue them self by how they made the EXE and MSI, as you can CLEARLY see 7-Zip Alpha is not in the Stable branch, it is under the Alpha Branch. Mainly this is a issue on https://github.com/microsoft/winget-cli or 7-Zip dev which we are trying to work around. Feel free to try to fix it and make it compatible 🤷♂️ VLC is actually published under the public branch http://download.videolan.org/pub/videolan/vlc/ 2021-06-09 |
|
Thank you. I appreciate all the hard work you and other moderators have put into this project. I fully agree that as these issues result from developers' mistakes/inconsistencies, it would be unreasonable to expect the moderation team to do a lot of extra work on every submission. VLC still for some reason puts 3.0.14 as latest despite having put 3.0.15 in the releases section, so they might be behaving like Firefox in the 3.x era, which put release candidates as releases on ftp and replaced them if any issues were found during that stage - having to confirm such cases for every application would be a nightmare. |
|
@KaranKad is it safe to close this issue now? I just wanted to leave it open long enough for the issues identified way up top to be resolved before closing it. If it's still being worked on, I'm happy to keep it open 😊 |
|
I think it can be closed now. |
Describe issue
People are submitting bad or duplicate manifests without checking if the app already exists or not in this repository.
Some examples
Duplicate pr's - #14546 #14606 #14498 #14605 #14610
Bad pr's - #14393 #14362 #14389 #14551 #14523
Overwriting existing manifests with half of the information - #14544 #14382
Luckily not every pr is merged.
Solution
Create a group of active contributors who know what they are doing, with ability to close a pr so they can prevent bad or duplicate pr's from getting in.
The text was updated successfully, but these errors were encountered: