Some extension resource url return 404 or 500 #432

jeanp413 · 2022-03-04T02:34:16Z

This package.json urls return 404

https://open-vsx.org/vscode/asset/viper-admin/viper/2.4.0/Microsoft.VisualStudio.Code.WebResources/extension/package.json
https://open-vsx.org/vscode/asset/pokey/parse-tree/0.16.0/Microsoft.VisualStudio.Code.WebResources/extension/package.json
https://open-vsx.org/vscode/asset/huacat/office-theme/1.1.3/Microsoft.VisualStudio.Code.WebResources/extension/package.json
https://open-vsx.org/vscode/asset/unthrottled/doki-theme/74.1.0/Microsoft.VisualStudio.Code.WebResources/extension/package.json
https://open-vsx.org/vscode/asset/huacat/pink-theme/0.4.0/Microsoft.VisualStudio.Code.WebResources/extension/package.json
https://open-vsx.org/vscode/asset/idleberg/hopscotch/0.8.2/Microsoft.VisualStudio.Code.WebResources/extension/package.json
https://open-vsx.org/vscode/asset/PKief/material-product-icons/1.1.2/Microsoft.VisualStudio.Code.WebResources/extension/package.json
https://open-vsx.org/vscode/asset/activecove/marquee/2.0.0/Microsoft.VisualStudio.Code.WebResources/extension/package.json
https://open-vsx.org/vscode/asset/crystal-lang-tools/crystal-lang/0.8.4/Microsoft.VisualStudio.Code.WebResources/extension/package.json
https://open-vsx.org/vscode/asset/artlaman/chalice-icon-theme/1.2.17/Microsoft.VisualStudio.Code.WebResources/extension/package.json

Also this theme json resources return error 500

https://open-vsx.org/vscode/asset/abelfubu/abelfubu-dark/1.3.4/Microsoft.VisualStudio.Code.WebResources/extension/themes/abelFubu%20Dark+-color-theme.json
https://open-vsx.org/vscode/asset/abelfubu/abelfubu-dark/1.3.4/Microsoft.VisualStudio.Code.WebResources/extension/themes/abelFubu%20Darker+-color-theme.json
https://open-vsx.org/vscode/asset/muchiros/attention-theme/0.0.4/Microsoft.VisualStudio.Code.WebResources/extension/themes/Attention%20Dark-color-theme.json

This is the query to get the above extensions, this query is used in vscode when previewing themes directly from marketplace

curl 'https://open-vsx.org/vscode/gallery/extensionquery' \
  -H 'content-type: application/json' \
  --data-raw '{"filters":[{"criteria":[{"filterType":5,"value":"themes"},{"filterType":8,"value":"Microsoft.VisualStudio.Code"},{"filterType":12,"value":"4096"}],"pageNumber":1,"pageSize":40,"sortBy":0,"sortOrder":0}],"assetTypes":[],"flags":950}' \
  --compressed

Also from this result I found that the publisher.displayname is null, this is required for the quickpick list description

{
  "extensionId": "02f7b704-1474-4f25-b451-a60afbf63dd2",
  "extensionName": "frontend-delight",
  "displayName": "Frontend Delight",
  "shortDescription": "Frontend Delight color scheme for VSCode",
  "publisher": {
    "displayName": null,
    "publisherId": "a5233a78-f3da-4233-91c7-fd4e857007f0",
    "publisherName": "RobinBoers"
  },
  "versions": [
...
  ],
  "tags": [
    "__web_extension",
    "color-theme",
    "theme"
  ],
  "releaseDate": null,
  "publishedDate": null,
  "lastUpdated": null,
  "categories": [
    "Themes"
  ],
  "flags": ""
}

You can also repro 404 errors when previewing product icon themes

cc @amvanbaren @filiptronicek

The text was updated successfully, but these errors were encountered:

jeanp413 · 2022-03-04T02:37:14Z

Seems 500 error is related to this issue #426

amvanbaren · 2022-03-04T10:16:24Z

Also from this result I found that the publisher.displayname is null

Yes, publisher.displayName is not set.

openvsx/server/src/main/java/org/eclipse/openvsx/adapter/VSCodeAdapter.java

Lines 363 to 365 in f475efa

    
           queryExt.publisher = new ExtensionQueryResult.Publisher(); 
        
           queryExt.publisher.publisherId = namespace.getPublicId(); 
        
           queryExt.publisher.publisherName = namespace.getName();

It looks like some extensions have a displayName that is different than publisherName (namespace name).
Winnie Lin: https://marketplace.visualstudio.com/items?itemName=onecentlin.laravel-blade

While for other extensions the displayName is the same as the publisherName.
codingyu: https://marketplace.visualstudio.com/items?itemName=codingyu.laravel-goto-view

I can add queryExt.publisher.displayName = namespace.getName(); as a quick fix.

amvanbaren · 2022-03-04T10:22:22Z

Seems 500 error is related to this issue #426

Yes, I'll finish up issue #426.

The 404 error is new, I'll try to replicate the issue.

amvanbaren · 2022-03-09T11:25:18Z

I replicated the 404 error for https://open-vsx.org/vscode/asset/viper-admin/viper/2.4.0/Microsoft.VisualStudio.Code.WebResources/extension/package.json

The viper extension is not a web extension and therefore webresources aren't extracted from the vsix package.

openvsx/server/src/main/java/org/eclipse/openvsx/ExtensionProcessor.java

Lines 301 to 302 in f475efa

    
           if (isWebExtensionKind(extension)) 
        
               resources.addAll(getWebResources(extension));

@jeanp413 Did the Open VSX API return this url in a JSON response or was the url constructed by another app (VS Code, Gitpod)?

A fail-safe way to get the package.json: https://open-vsx.org/vscode/asset/viper-admin/viper/2.4.0/Microsoft.VisualStudio.Code.Manifest

jeanp413 · 2022-03-09T12:12:31Z

@amvanbaren the extension is returned as part of this query but it doesn't have web resources in the response, I searched for in MS marketplace and it also doesn't have web resources, so it seems vscode asumes all the extensions returned by this query are web extensions and openvsx is filtering incorrectly when doing a search

curl 'https://open-vsx.org/vscode/gallery/extensionquery' \
  -H 'content-type: application/json' \
  --data-raw '{"filters":[{"criteria":[{"filterType":5,"value":"themes"},{"filterType":8,"value":"Microsoft.VisualStudio.Code"},{"filterType":12,"value":"4096"}],"pageNumber":1,"pageSize":40,"sortBy":0,"sortOrder":0}],"assetTypes":[],"flags":950}' \
  --compressed

I see the viper extension response from openvsx contains this properties and the response from MS marketplace doesn't, if these properties are used when filtering, then this is causing the issue

{
  "key": "Microsoft.VisualStudio.Services.Branding.Color",
  "value": "",
},
{
  "key": "Microsoft.VisualStudio.Services.Branding.Theme",
  "value": "",
},

I'm also seeing some missing resources like "assetType": "Microsoft.VisualStudio.Services.VsixManifest", and assetType: "Microsoft.VisualStudio.Services.Icons.Small" and properties like "key": "Microsoft.VisualStudio.Services.Links.Getstarted", "key": "Microsoft.VisualStudio.Services.Links.Support", etc

MS response for viper extension

{
	"results": [
		{
			"extensions": [
				{
					"publisher": {
						"publisherId": "40c87fab-912c-4304-b2ee-b6c71e280a3c",
						"publisherName": "viper-admin",
						"displayName": "Chair of Programming Methodology - ETH Zurich",
						"flags": "verified",
						"domain": null,
						"isDomainVerified": false
					},
					"extensionId": "20da0370-13e1-4393-8d10-4e90c9c9e6e2",
					"extensionName": "viper",
					"displayName": "Viper",
					"flags": "validated, public",
					"lastUpdated": "2022-02-28T12:00:56.653Z",
					"publishedDate": "2018-10-14T20:51:27.093Z",
					"releaseDate": "2018-10-14T20:51:27.093Z",
					"shortDescription": "This extension provides interactive IDE features for verifying programs in Viper (Verification Infrastructure for Permission-based Reasoning).",
					"versions": [
						{
							"version": "2.4.0",
							"flags": "validated",
							"lastUpdated": "2022-02-28T12:00:56.653Z",
							"files": [
								{
									"assetType": "Microsoft.VisualStudio.Code.Manifest",
									"source": "https://viper-admin.gallerycdn.vsassets.io/extensions/viper-admin/viper/2.4.0/1646049517014/Microsoft.VisualStudio.Code.Manifest"
								},
								{
									"assetType": "Microsoft.VisualStudio.Services.Content.Details",
									"source": "https://viper-admin.gallerycdn.vsassets.io/extensions/viper-admin/viper/2.4.0/1646049517014/Microsoft.VisualStudio.Services.Content.Details"
								},
								{
									"assetType": "Microsoft.VisualStudio.Services.Content.License",
									"source": "https://viper-admin.gallerycdn.vsassets.io/extensions/viper-admin/viper/2.4.0/1646049517014/Microsoft.VisualStudio.Services.Content.License"
								},
								{
									"assetType": "Microsoft.VisualStudio.Services.Icons.Default",
									"source": "https://viper-admin.gallerycdn.vsassets.io/extensions/viper-admin/viper/2.4.0/1646049517014/Microsoft.VisualStudio.Services.Icons.Default"
								},
								{
									"assetType": "Microsoft.VisualStudio.Services.Icons.Small",
									"source": "https://viper-admin.gallerycdn.vsassets.io/extensions/viper-admin/viper/2.4.0/1646049517014/Microsoft.VisualStudio.Services.Icons.Small"
								},
								{
									"assetType": "Microsoft.VisualStudio.Services.VsixManifest",
									"source": "https://viper-admin.gallerycdn.vsassets.io/extensions/viper-admin/viper/2.4.0/1646049517014/Microsoft.VisualStudio.Services.VsixManifest"
								},
								{
									"assetType": "Microsoft.VisualStudio.Services.VSIXPackage",
									"source": "https://viper-admin.gallerycdn.vsassets.io/extensions/viper-admin/viper/2.4.0/1646049517014/Microsoft.VisualStudio.Services.VSIXPackage"
								}
							],
							"properties": [
								{
									"key": "Microsoft.VisualStudio.Services.Links.Getstarted",
									"value": "https://github.com/viperproject/viper-ide.git"
								},
								{
									"key": "Microsoft.VisualStudio.Services.Links.Support",
									"value": "https://github.com/viperproject/viper-ide/issues"
								},
								{
									"key": "Microsoft.VisualStudio.Services.Links.Learn",
									"value": "https://github.com/viperproject/viper-ide#readme"
								},
								{
									"key": "Microsoft.VisualStudio.Services.Links.Source",
									"value": "https://github.com/viperproject/viper-ide.git"
								},
								{
									"key": "Microsoft.VisualStudio.Services.Links.GitHub",
									"value": "https://github.com/viperproject/viper-ide.git"
								},
								{
									"key": "Microsoft.VisualStudio.Code.Engine",
									"value": "^1.45.0"
								},
								{
									"key": "Microsoft.VisualStudio.Services.GitHubFlavoredMarkdown",
									"value": "true"
								},
								{
									"key": "Microsoft.VisualStudio.Code.ExtensionDependencies",
									"value": ""
								},
								{
									"key": "Microsoft.VisualStudio.Code.ExtensionPack",
									"value": ""
								},
								{
									"key": "Microsoft.VisualStudio.Code.LocalizedLanguages",
									"value": ""
								},
								{
									"key": "Microsoft.VisualStudio.Code.ExtensionKind",
									"value": "workspace"
								}
							],
							"assetUri": "https://viper-admin.gallerycdn.vsassets.io/extensions/viper-admin/viper/2.4.0/1646049517014",
							"fallbackAssetUri": "https://viper-admin.gallery.vsassets.io/_apis/public/gallery/publisher/viper-admin/extension/viper/2.4.0/assetbyname"
						}
					],
					"categories": [
						"Programming Languages",
						"Snippets",
						"Themes",
						"Education"
					],
					"tags": [
						"__ext_log",
						"__ext_sil",
						"__ext_vpr",
						"automatic deduction",
						"color-theme",
						"formal methods",
						"keybindings",
						"log",
						"separation logic",
						"snippet",
						"specification",
						"theme",
						"verification",
						"viper"
					],
					"statistics": [
						{
							"statisticName": "install",
							"value": 2131.0
						},
						{
							"statisticName": "averagerating",
							"value": 5.0
						},
						{
							"statisticName": "ratingcount",
							"value": 1.0
						},
						{
							"statisticName": "trendingdaily",
							"value": 0.0
						},
						{
							"statisticName": "trendingmonthly",
							"value": 4.178403755868545
						},
						{
							"statisticName": "trendingweekly",
							"value": 0.61032863849765262
						},
						{
							"statisticName": "updateCount",
							"value": 2188.0
						},
						{
							"statisticName": "weightedRating",
							"value": 4.4574588448266113
						},
						{
							"statisticName": "downloadCount",
							"value": 55.0
						}
					],
					"deploymentType": 0
				}
			],
			"pagingToken": null,
			"resultMetadata": [
				{
					"metadataType": "ResultCount",
					"metadataItems": [
						{
							"name": "TotalCount",
							"count": 1
						}
					]
				}
			]
		}
	]
}

amvanbaren · 2022-03-14T11:27:09Z

The 404 errors are caused by the resourceUrlTemplate configuration in openvscode-server product.json.
It uses the Open VSX webresource endpoint: https://open-vsx.org/vscode/asset/{publisher}/{name}/{version}/Microsoft.VisualStudio.Code.WebResources/{path}. Not all extensions are web extensions, which cause the 404 errors.
https://github.com/gitpod-io/openvscode-server/blob/8cfe967898b16877f9b0745dd83438459d743c44/product.json#L740-L746

The resourceUrlTemplate is loaded by extensionResourceLoader. It's then used by workbenchThemeService by calling extensionResourceLoader.getExtensionGalleryResourceURL()

MS VS Code uses https://{publisher}.vscode-unpkg.net/{publisher}/{name}/{version}/{path} as resourceUrlTemplate. The endpoint lets you browse through the extension package. If {path} points to a file then the file contents are returned, else a JSON array of paths is returned (similar to the ls command on Linux). To try it out: https://viper-admin.vscode-unpkg.net/viper-admin/viper/2.4.0/

jeanp413 · 2022-03-14T20:41:28Z

I see, thanks for the investigation @amvanbaren 🙏. So this should still be fixed in openvsx right? as trying to access https://open-vsx.org/vscode/asset/viper-admin/viper/2.4.0/ shows an error:

amvanbaren · 2022-03-16T09:21:22Z

So this should still be fixed in openvsx right?

Yes, I've added a /vscode/unpkg/{namespace}/{extension}/{version}/{path} endpoint to VSCodeAdapter.
I've changed ExtensionProcessor to always get all resources from a vsix package.

The main challenge is to write a database migration that re-processes all extension versions to get all resources from its vsix package. Flyway offers Java migrations besides SQL scripts, so it is possible.

amvanbaren self-assigned this Mar 4, 2022

amvanbaren mentioned this issue Mar 10, 2022

URI encode FileResource urls #433

Merged

This was referenced Mar 23, 2022

Some extension resource url return 404 or 500 #435

Merged

Use Open VSX /vscode/unpkg endpoint as resourceUrlTemplate in product.json gitpod-io/openvscode-server#330

Closed

amvanbaren closed this as completed in #435 Apr 4, 2022

amvanbaren mentioned this issue Sep 9, 2024

Are we managing extension files efficiently #980

Open

kineticsquid mentioned this issue Sep 10, 2024

Comms for support of deprecated extensions EclipseFdn/open-vsx.org#2750

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Some extension resource url return 404 or 500 #432

Some extension resource url return 404 or 500 #432

jeanp413 commented Mar 4, 2022 •

edited

Loading

jeanp413 commented Mar 4, 2022

amvanbaren commented Mar 4, 2022

amvanbaren commented Mar 4, 2022

amvanbaren commented Mar 9, 2022

jeanp413 commented Mar 9, 2022

amvanbaren commented Mar 14, 2022

jeanp413 commented Mar 14, 2022

amvanbaren commented Mar 16, 2022

Some extension resource url return 404 or 500 #432

Some extension resource url return 404 or 500 #432

Comments

jeanp413 commented Mar 4, 2022 • edited Loading

jeanp413 commented Mar 4, 2022

amvanbaren commented Mar 4, 2022

amvanbaren commented Mar 4, 2022

amvanbaren commented Mar 9, 2022

jeanp413 commented Mar 9, 2022

amvanbaren commented Mar 14, 2022

jeanp413 commented Mar 14, 2022

amvanbaren commented Mar 16, 2022

jeanp413 commented Mar 4, 2022 •

edited

Loading