Idea: Use PowerShell script as a dynamic manifest

[KiruyaMomochi]

Description of the new feature/enhancement

TL;DR

Use PowerShell script as a dynamic manifest.
A program then parses the script and generate a manifest.
winget reads the manifest and install the package.

Example

Here is a minimal example script. With a helper script/program, it generates real 7zip manifest.
Almost everything is same as the manifest, except that Version will always be the latest, and $Url now dynamic changes as $Version change.

$License = 'MIT',  'Copyright (C) 1999-2020 Igor Pavlov. - GNU LGPL'
$LicenseUrl = 'https://github.com/microsoft/msix-packaging/blob/master/LICENSE', 'https://7-zip.org/license.txt'
$Tags = '7zip','compression','file compression' ,'utility', 'tool', 'zip'
# $Publisher, $Author, $Homepage, $Description, $InstallerType skipped
function Version {
    # do something and return the latest version of 7zip
}
# $Version = (Version) # this line can be omitted since parser knows to add it
$Installers = (@{
    Arch = 'x64'
    Url = "https://www.7-zip.org/a/7z$((Version).Replace('.',''))-x64.msi" # <- notice here!
    Sha256 = 'A7803233EEDB6A4B59B3024CCF9292A6FFFB94507DC998AA67C5B745D197A5DC'
})

More details

Manifest is a great idea, I like it! But I don't think they are 'Packages'...
actually what they do is telling winget how to get a package and install it silently.
A package should be msi/msix files, especially msix since it can handle dependence.

Definition of terms

• Script: a shell script that provides instructions on how to build a manifest file
• Manifest: a file describes the information about the Package
• Package: a msix/msi/exe/... file that can be used to install program offline.
• Parser: External script or program, or winget itself, parse the Script and generate the Manifest.
• winget: the program that reads the Manifest, download the Package and install it

Workflow example

How winget now works:
    Manifest -> Package -> Program
How Arch AUR Works:
    Script -[use Script as Manifest]-> Package -> Program
What the issue wants:
    Script -[parse]-> Manifest -> Package -> Program

How Script converted to Manifest

The Parser reads the script first, for each field if it is a function,
run it and use the return value as new field value.
Then use these field variables to generate the yaml Manifest file.

Further Step

• Now there is a offiicial Manifest repository (official repo) hosted by Microsoft.
  We can have another Script repository (user repo) hosted by community.
  User can submit Scripts to user repository, or submit Manifest to official repository.

• There maybe also some helpers that automatically submit manifest generate by script.

• There may can be a way that software developer can submit their package into official repo,
  so user can download both manifest and package from Microsoft.
  And such a package can be also available in Microsoft Store.

Some questions and problems

• Should the Script use a dynamic Version function or static Version variable?
  • If it's in a user repo, it's better to use the static one that user can update their package by consulting the user repo.
  • However, if we considering generate latest Manifest from the Script, the dynamic one may be a better choice.
• Should we consider using Script as Manifest?
• Should the Script be a .ps1 script that can run directly, or something like PKGBUILD?
• One more step: can we automatically generate MSIX package from a Manifest, for a portable/standalone program?
• Is Git suitable for a package repo?
  We should have another real manifest manage system, instead of using a git repo.
  As there are more and more pull requests coming, it can be really difficult to manage.
  It's also confusing that I fork and clone the whole repo, commit a file,
  only for submitting a new manifest.
• Is this too complex? Actually we can just write our own script to generate Manifest, and submit a PR when new version is out automatically. Maybe this is a better choice.

Related issue: #223

[KevinMarquette]

If using PowerShell to generate the manifests is a popular idea, it would not be much work to take a DSL style approach to it. Having syntax like this is very doable with PowerShell.

function Get-Version {
    # do something and return the latest version of 7zip
}

New-WinGetManifest -Path "./7zip$(Get-Version).yml" @{
    # $Publisher, $Author, $Homepage, $Description, $InstallerType skipped
    License = 'MIT',  'Copyright (C) 1999-2020 Igor Pavlov. - GNU LGPL'
    LicenseUrl = 'https://github.com/microsoft/msix-packaging/blob/master/LICENSE', 'https://7-zip.org/license.txt'
    Tags = '7zip','compression','file compression' ,'utility', 'tool', 'zip'
    Installers = (@{
        Arch = 'x64'
        Url = "https://www.7-zip.org/a/7z$((Get-Version).Replace('.',''))-x64.msi" # <- notice here!
        Sha256 = 'A7803233EEDB6A4B59B3024CCF9292A6FFFB94507DC998AA67C5B745D197A5DC'
        Version = Get-Version
    })
}

Its basically a function that takes a path and a hashtable. Because its PowerShell, you can call whatever code you need to in there. Either calling a function or just adding inline code. I'm ignoring the magic mapping for version from your example, but this would be trivial to document and more intuitive for a seasoned PowerSheller to look at an example and just go.

Get-Help New-WinGetManifest  -Example

[javydekoning]

Actually we can just write our own script to generate Manifest, and submit a PR when new version is out automatically. Maybe this is a better choice.

This is how I handle it in Chocolatey today. The CI pipeline (which runs on a schedule) runs a script that fetches the latest version (usually using invoke-webrequest). If the remote version is newer, the url, version and filehash are updated in the manifest and pushed.

[jantari]

I think this is a very bad idea.

1. It intoduces the horrible security risk of the manifest file itself executing code during its parsing (e.g. $Tags = $(Remove-Item ~ -Recurse -Force))
2. It makes it much harder if not impossible to validate the manifest file schema (which is currently trivial and easy)
3. It pollutes git diffs and general manifest file changes over time because it makes them more complicated. You're now editing functions, parameters, code vs. just a few YAML values
4. It introduces the problem of badly "coded" manifests not working for everyone because of unquoted paths, unicode issues, differences betweeen PowerShell versions, function/alias name conflicts etc. etc. - endless bugs and problems!
5. This is completely unneccessary because you can just template/create a YAML manifest file with a PowerShell function or a tool like cookiecutter which is way safer and can be automated and "dynamic" just the same
6. It would ruin the performance of winget - during a regular scan for updates, it would have to execute potentially a hundred PowerShell scripts (aka dynamic manifests) - one or more for each package (since scripts can source and run other scripts)
7. It makes it possible to "obfuscate" package manifests e.g. by obscuring or even encrypting the URL an installer is downloaded from which makes it easy to sneak in malware - even malware that only triggers on certain conditions. Imagine a package that's called VLC and downloads and installs VLC completely normally 99% of the time, except when the current computer is part of the US Military it downloads something else instead

EDIT: In addition, Microsoft has already publicly stated that they will not allow scripts during the installation of a package source - it stands to reason they especially would not allow scripts during the searching for / indexing of packages

My suggestion to OP and anyone else who wants to do this is to use cookiecutter which is a tool that reads a template file (in our case a winget manifest YAML) and inserts/edits values at certain points in the file - either fully automatic or by asking for each value one by one. It's cross platform and can be used for much more than just winget manifests as well.

[denelon]

Hey all, one of our tenets is not to run "arbitrary" scripts. We haven't been able to reason about a good way to treat a script as a manifest. It is possible to generate a local manifest using a script and then call winget install -m <path to directory containing manifest>.