Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't use dotnet add package with a private repository (basic authentication) #5163

Closed
a-h opened this issue May 4, 2017 · 21 comments
Closed
Assignees
Labels
Platform:Xplat Product:NuGet.exe NuGet.exe Resolution:Duplicate This issue appears to be a Duplicate of another issue Type:Bug
Milestone

Comments

@a-h
Copy link

a-h commented May 4, 2017

Details about Problem

  • dotnet --version
    • 1.0.1
  • dotnet
    • 1.1
  • NuGet
    • 3.5.0.1996
  • MacOS Sierra
    • 10.12.4

Steps to reproduce

  1. Create a private NuGet repository (in my case using AppVeyor)
  2. Add it as a source:
nuget sources add -Name xxx -Source https://ci.appveyor.com/nuget/xxx -Username xxx@xxxx.com -Password xxx
  1. Prove that it's possible to list packages:
nuget list -s xxx
Using credentials from config. UserName: xxx@xxx.com
my.nuget.package 1.0.1

4 Attempt to add the private NuGet package to a dotnet project using the CLI:

dotnet add package -s https://ci.appveyor.com/nuget/xxx my.nuget.package

Expected

For the command to add the NuGet package to the project.

Actual

Got a message saying that it couldn't authenticate.

Microsoft (R) Build Engine version 15.1.548.43366
Copyright (C) Microsoft Corporation. All rights reserved.

  Writing /var/folders/v0/gv8rbbt9157g5599sh8qljpr0000gn/T/tmpz9doUg.tmp
info : Adding PackageReference for package 'my.nuget.package' into project '/Users/xxx/Documents/my.nuget.package/src/xxx/xxx.csproj'.
log  : Restoring packages for /Users/xxx/Documents/xxx/src/xxxLibrary/xxx.csproj...
info :   GET https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package'
info :   Unauthorized https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package' 4042ms
log  : Retrying 'FindPackagesByIdAsyncCore' for source 'https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package''.
log  : Response status code does not indicate success: 401 (Unauthorized).
info :   GET https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package'
info :   Unauthorized https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='xxx' 997ms
log  : Retrying 'FindPackagesByIdAsyncCore' for source 'https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='xxx'.
log  : Response status code does not indicate success: 401 (Unauthorized).
info :   GET https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package'
info :   Unauthorized https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='xxx' 1007ms
error: Failed to retrieve information about 'my.nuget.package' from remote source 'https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package''.
error:   Response status code does not indicate success: 401 (Unauthorized).
@a-h
Copy link
Author

a-h commented May 4, 2017

Really long winded workaround:

  • mkdir download

  • cd download

  • nuget install my.nuget.package

  • Wait for all the downloads to complete

  • Note that it's been added to the cache:

    • ls /Users/xxx/.nuget/packages/my.nuget.package/
  • Run dotnet restore from the solution directory, and it will work

@mishra14
Copy link
Contributor

mishra14 commented May 5, 2017

@a-h
I tried this scenario and I was able to install a package from a private feed using dotnet add -


F:\validation\test> dotnetlocal add package -s https://mishra14.pkgs.visualstudio.com/_packaging/temp/nuget/v3/index.json testapp
Microsoft (R) Build Engine version 15.2.93.5465
Copyright (C) Microsoft Corporation. All rights reserved.

  Writing C:\Users\anmishr\AppData\Local\Temp\tmp3568.tmp
info : Adding PackageReference for package 'testapp' into project 'F:\validation\test\test.csproj'.
log  : Restoring packages for F:\validation\test\test.csproj...
info :   GET https://mishra14.pkgs.visualstudio.com/_packaging/8e401993-92e2-4c09-8755-7c6003be8320/nuget/v3/flat2/testapp/index.json
info :   OK https://mishra14.pkgs.visualstudio.com/_packaging/8e401993-92e2-4c09-8755-7c6003be8320/nuget/v3/flat2/testapp/index.json 554ms
info :   GET https://mishra14.pkgs.visualstudio.com/_packaging/8e401993-92e2-4c09-8755-7c6003be8320/nuget/v3/flat2/testapp/1.0.0/testapp.1.0.0.nupkg
info :   OK https://mishra14.pkgs.visualstudio.com/_packaging/8e401993-92e2-4c09-8755-7c6003be8320/nuget/v3/flat2/testapp/1.0.0/testapp.1.0.0.nupkg 1213ms
log  : Installing TestApp 1.0.0.
info : Package 'testapp' is compatible with all the specified frameworks in project 'F:\validation\test\test.csproj'.
info : PackageReference for package 'testapp' version '1.0.0' added to file 'F:\validation\test\test.csproj'.


F:\validation\test> dotnetlocal --info
.NET Command Line Tools (2.0.0-preview2-005910)

Product Information:
 Version:            2.0.0-preview2-005910
 Commit SHA-1 hash:  f3a4b22457

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.15063
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   E:\cli\artifacts\win10-x64\stage2\sdk\2.0.0-preview2-005910

Microsoft .NET Core Shared Framework Host

  Version  : 2.0.0-preview2-002093-00
  Build    : 1a195046fd92bd73d993d804bea2f4506c2b56f1

The feed - https://mishra14.pkgs.visualstudio.com/_packaging/8e401993-92e2-4c09-8755-7c6003be8320/nuget/v3/flat2/testapp/index.json is a private feed.

Please make sure that the config file has the right credentials. In the mean time I will try with a V2 feed as well.

@mishra14 mishra14 self-assigned this May 5, 2017
@mishra14 mishra14 modified the milestones: 4.3 - 4/17 thru 5/6, 4.3 May 5, 2017
@mishra14
Copy link
Contributor

mishra14 commented May 5, 2017

I was able to add package from the V2 feed as well -

F:\validation\test> dotnetlocal add package -s https://mishra14.pkgs.visualstudio.com/_packaging/temp/nuget/v2 testapp
Microsoft (R) Build Engine version 15.2.93.5465
Copyright (C) Microsoft Corporation. All rights reserved.

  Writing C:\Users\anmishr\AppData\Local\Temp\tmp6E20.tmp
info : Adding PackageReference for package 'testapp' into project 'F:\validation\test\test.csproj'.
log  : Restoring packages for F:\validation\test\test.csproj...
info :   GET https://mishra14.pkgs.visualstudio.com/_packaging/temp/nuget/v2/FindPackagesById()?id='testapp'
info :   OK https://mishra14.pkgs.visualstudio.com/_packaging/temp/nuget/v2/FindPackagesById()?id='testapp' 1649ms
info :   GET https://mishra14.pkgs.visualstudio.com/_packaging/temp/nuget/v2?id=testapp&version=1.0.0
info :   OK https://mishra14.pkgs.visualstudio.com/_packaging/temp/nuget/v2?id=testapp&version=1.0.0 1084ms
log  : Installing TestApp 1.0.0.
info : Package 'testapp' is compatible with all the specified frameworks in project 'F:\validation\test\test.csproj'.
info : PackageReference for package 'testapp' version '1.0.0' updated in file 'F:\validation\test\test.csproj'.

@a-h
Copy link
Author

a-h commented May 5, 2017

OK, here's a script which reproduces the same problem inside a Linux container:

I created a file called test.sh.

mkdir TestProject
cd TestProject
echo "Creating new library..."
dotnet new library
echo "Installing Mono to run nuget.exe"
apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
echo "deb http://download.mono-project.com/repo/debian wheezy main" | tee /etc/apt/sources.list.d/mono-xamarin.list
echo "deb http://download.mono-project.com/repo/debian wheezy-apache24-compat main" | tee -a /etc/apt/sources.list.d/mono-xamarin.list
echo "deb http://download.mono-project.com/repo/debian wheezy-libjpeg62-compat main" | tee -a /etc/apt/sources.list.d/mono-xamarin.list
apt-get update
apt-get install -y mono-complete
echo "Actually installing nuget"
wget https://dist.nuget.org/win-x86-commandline/latest/nuget.exe
echo "Adding nuget source..."
mono nuget.exe sources add -Name xxx -Source https://ci.appveyor.com/nuget/xxx -Username xxx@example.com -Password -xxx
echo "Listing sources..."
mono nuget.exe list -s xxx
echo "Adding package..."
dotnet add package -s https://ci.appveyor.com/nuget/xxx my.nuget.package

Then I executed the file inside a container:

docker run -v $(pwd)/test.sh:/test.sh --rm microsoft/dotnet:latest /bin/bash test.sh

I got exactly the same results on Linux as I did on my Mac.

Package Source with Name: xxx added successfully.
Listing sources...
Using credentials from config. UserName: xxx@xxx.com
Duration 1.0.1
my.nuget.package 1.0.15
Adding package...
Microsoft (R) Build Engine version 15.1.548.43366
Copyright (C) Microsoft Corporation. All rights reserved.

  Writing /tmp/tmpr9t9l7.tmp
info : Adding PackageReference for package 'my.nuget.package' into project '/TestProject/TestProject.csproj'.
log  : Restoring packages for /TestProject/TestProject.csproj...
info :   GET https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package'
info :   Unauthorized https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package' 915ms
log  : Retrying 'FindPackagesByIdAsyncCore' for source 'https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package''.
log  : Response status code does not indicate success: 401 (Unauthorized).
info :   GET https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package'
info :   Unauthorized https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package' 569ms
log  : Retrying 'FindPackagesByIdAsyncCore' for source 'https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package''.
log  : Response status code does not indicate success: 401 (Unauthorized).
info :   GET https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package'
info :   Unauthorized https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package' 494ms
error: Failed to retrieve information about 'my.nuget.package' from remote source 'https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package''.
error:   Response status code does not indicate success: 401 (Unauthorized).

Not sure what the state of your Windows machine is. Maybe the problem is only on Linux and OSX.

@mishra14
Copy link
Contributor

mishra14 commented May 5, 2017

@a-h - Thanks for the script. I will see if I can run stuff on a mac and get a repro.

One thought I had was -just try dotnet add package my.nuget.package ?

If the command is picking up the right config file then it should resolve the feed. That might give us more information.

@mishra14
Copy link
Contributor

mishra14 commented May 5, 2017

@a-h : I did some more digging and found an older issue: #2887

The problem is that dotnet does not support encrypted passwords on non-windows platform yet. mono nuget.exe works because mono seems to support the encryption apis that we use.

So if you do -
mono nuget.exe sources add -Name xxx -Source https://ci.appveyor.com/nuget/xxx -Username xxx@example.com -Password -xxx -StorePasswordInClearText

This is going to put your password into the config file in clear text, so its not the best solution but should unblock you. Let me if that works.

@a-h
Copy link
Author

a-h commented May 5, 2017

I updated the script to add the -StorePasswordInClearText flag, ran it, and got exactly the same error.

I then tried -StorePasswordInClearText AND replaced dotnet add package -s https://ci.appveyor.com/nuget/xxx my.nuget.package with dotnet add package my.nuget.package. In this case, it simply couldn't find my private package in the public NuGet repo:

Microsoft (R) Build Engine version 15.1.548.43366
Copyright (C) Microsoft Corporation. All rights reserved.

  Writing /tmp/tmpZnd5LG.tmp
info : Adding PackageReference for package 'xxx' into project '/TestProject/TestProject.csproj'.
log  : Restoring packages for /TestProject/TestProject.csproj...
info :   GET https://api.nuget.org/v3-flatcontainer/xxx/index.json
info :   NotFound https://api.nuget.org/v3-flatcontainer/xxx/index.json 139ms
error: Unable to resolve 'xxx' for '.NETStandard,Version=v1.4'.
error: Package 'xxx' is incompatible with 'all' frameworks in project '/TestProject/TestProject.csproj'.

So I'm inclined to think that there's an underlying problem.

@mishra14
Copy link
Contributor

mishra14 commented May 5, 2017

@a-h :

The problem is that mono and dotnet resolve user and machine wide config files differently. When you add source using mono, it goes to the user wide config file that mono uses but dotnet does not know about.

Tested workaround on linux -

What this does is that it adds the source to the local NuGet.config file. This is then picked up by dotnet.

ddcloud@ddanmishr005:~/dev/temp2$ mono nuget.exe sources Add -Name "temp" -Source "https://mishra14.pkgs.visualstudio.com/_packaging/temp/nuget/v2" -username xxx -password xxx -StorePasswordInClearText -configfile ./NuGet.config
Package Source with Name: temp added successfully.
ddcloud@ddanmishr005:~/dev/temp2$ ../temp/dotnet add package testapp
Microsoft (R) Build Engine version 15.2.93.5465
Copyright (C) Microsoft Corporation. All rights reserved.

  Writing /tmp/tmpUvdT4X.tmp
info : Adding PackageReference for package 'testapp' into project '/home/ddcloud/dev/temp2/temp2.csproj'.
log  : Restoring packages for /home/ddcloud/dev/temp2/temp2.csproj...
info :   GET https://dotnetmyget.blob.core.windows.net/artifacts/dotnet-core/nuget/v3/flatcontainer/testapp/index.json
info :   CACHE https://mishra14.pkgs.visualstudio.com/_packaging/temp/nuget/v2/FindPackagesById()?id='testapp'
info :   CACHE https://mishra14.pkgs.visualstudio.com/_packaging/temp/nuget/v2?id=testapp&version=1.0.0
info :   GET https://api.nuget.org/v3-flatcontainer/testapp/index.json
info :   NotFound https://dotnetmyget.blob.core.windows.net/artifacts/dotnet-core/nuget/v3/flatcontainer/testapp/index.json 197ms
info :   NotFound https://api.nuget.org/v3-flatcontainer/testapp/index.json 274ms
info : Package 'testapp' is compatible with all the specified frameworks in project '/home/ddcloud/dev/temp2/temp2.csproj'.
info : PackageReference for package 'testapp' version '1.0.0' updated in file '/home/ddcloud/dev/temp2/temp2.csproj'.

@a-h
Copy link
Author

a-h commented May 6, 2017

Thanks for the workaround! :) I'll just let my team know that until the dotnet command adds support for adding / restoring packages from private repositories and adding additional package sources, they need to:

  • Install Mono
  • Install NuGet
  • Add private package sources into a local NuGet.config file, making sure to not encrypt their credentials

Has the NuGet team thought of updating the <PackageReference> element of the .csproj format (https://docs.microsoft.com/en-us/dotnet/articles/core/tools/csproj) to include the URL of the package source? Then, when dotnet restore is executed, it would know the correct source to search instead of searching all sources for the package (it looks like a waste of HTTP requests to me).

@mishra14
Copy link
Contributor

mishra14 commented May 8, 2017

@a-h : I have opened a bug to track this request - #5184

@mishra14
Copy link
Contributor

mishra14 commented May 8, 2017

duplicate of #4413. Closing.

@mishra14 mishra14 closed this as completed May 8, 2017
@a-h
Copy link
Author

a-h commented May 9, 2017

Just for anyone experiencing the same problem, there's of course, no way to upgrade / update packages either.

To do this, delete them from the cache, then add it again:

rm -rf /Users/xxx/.nuget/packages/xxx/
dotnet add ./src/xxx/ package xxx
dotnet restore

@a-h
Copy link
Author

a-h commented May 9, 2017

What a massive waste of time this was.

It's actually really simple to get dotnet restore to work. Simply create a NuGet.config file in the solution directory and it will automatically be used.

I wrote a Python file to do it based on environment variables I pass into the build server:

from xml.sax.saxutils import quoteattr
import os

f = open('NuGet.config', 'w')
f.write('<?xml version="1.0" encoding="utf-8"?>\n')
f.write('<configuration>\n')
f.write('  <packageSources>\n')
f.write('    <add key="xxx" value="https://ci.appveyor.com/nuget/xxx" />\n')
f.write('  </packageSources>\n')
f.write('  <packageSourceCredentials>\n')
f.write('    <xxx>\n')
f.write('      <add key="Username" value=' + quoteattr(os.environ['APPVEYOR_NUGET_USERNAME']) + ' />\n')
f.write('      <add key="ClearTextPassword" value=' + quoteattr(os.environ['APPVEYOR_NUGET_PASSWORD']) + ' />\n')
f.write('    </xxx>\n')
f.write('  </packageSourceCredentials>\n')
f.write('</configuration>\n')
f.close()

@rrelyea rrelyea added Resolution:Duplicate This issue appears to be a Duplicate of another issue Type:Bug Product:NuGet.exe NuGet.exe Platform:Xplat labels Aug 15, 2017
@arnotes
Copy link

arnotes commented May 26, 2020

doesnt work when using global Nuget.config

dotnet v: 3.1.300
OS: Windows 10

@Luqswe
Copy link

Luqswe commented Jun 2, 2020

same here, please reopen (@mishra14 ).

.NET Core SDK :
Version: 3.1.202
Commit: 6ea70c8dca

Runtime Environment:
OS Name: Windows
OS Version: 10.0.17763
OS Platform: Windows
RID: win10-x64
Base Path: C:\Program Files\dotnet\sdk\3.1.202\

dotnet add package output:
error: Response status code does not indicate success: 401 (Unauthorized).

my NuGet.Config (also tried with encrypted password):

@nbarnwell
Copy link

I'm not sure you can call this "closed" if the solution involves having credentials in a file in a repository... I've installed the Azure DevOps Artifacts Credentials Provider and that's not working, either. Bit of a shame to only consider nuget.org and ignoring auth issues.

@dhurimkelmendi
Copy link

I solved this by using Visual Studio for Mac to add the source. Solution/Project -> Manage Nuget repositories -> Add source.

@nbarnwell
Copy link

@mishra14 Can you state categorically that the suggested solution (and reason for closing the case) is that we should all have nuget.config files in our solutions with private nuget feed URLs and cleartext passwords, please? I assume I speak for anyone who knows what they're doing with computers when I say it should support the Azure DevOps Artifacts Credentials Provider on Windows, as regular nuget.exe does.

@zivkan
Copy link
Member

zivkan commented Mar 23, 2023

@nbarnwell NuGet accumulates settings from multiple locations, so the credentials can be saved in the user-profile nuget.config, so there's no risk of accidentally committing credentials (encrypted or otherwise) into the repo nuget.config.

However, if you're using Azure DevOps, then we absolutely recommend using the credential provider, and no credentials in nuget.config files will be needed. If it's not working for you, that's a separate issue than this one.

@nbarnwell
Copy link

We all know that plaintext credentials in text files (whether they might get accidentally committed to source control or not) are undesirable. It's a feature that shouldn't exist. My point is that dotnet [package] commands don't use the credential provider (while vanilla nuget.exe does). Which is basically what this ticket is about - "[Can't use dotnet add package with a private repository". So it's not a separate issue. In fact, the nuget.config solution simply isn't the solution to this issue.

@zivkan
Copy link
Member

zivkan commented Mar 24, 2023

There's an issue about supporting encrypted passwords on Linux and Mac: #1851. We prioritize work based on a number of factors, but the number of upvotes (👍 reactions to the original comment) is a one consideration.

dotnet add package and dotnet list package absolutely should use the credential provider, provided it's installed. It's the reason why dotnet add package and dotnet list package have --interactive arguments. The team that implemented dotnet workload commands also reached out to us to find out how to make the cred provider work (dotnet workload install also has --interactive), and I'm confident that we talked to the team that owns dotnet tool install (which also has --interactive) to ensure the cred provider works with that as well.

If it's not working for you there's some kind of issue going on, but I disagree it's on-topic for this issue. I'm not aware of any cred provider for app veyor, and the discussion earlier in the thread was specifically about saving the credentials in a nuget.config, since dotnet nuget [add|update] source wasn't implemented in 2017.

Again, I expect dotnet add package to automatically detect any installed credential providers and use them. Without any additional information, the most likely reason is that you have Visual Studio installed, which somehow nuget.exe can find the .NET Framework credential provider for, but I guess you don't have the .NET (Core) credential provider installed. https://github.com/microsoft/artifacts-credprovider has instructions how to install it. If that doesn't work for you, then I encourage you to create a new issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Platform:Xplat Product:NuGet.exe NuGet.exe Resolution:Duplicate This issue appears to be a Duplicate of another issue Type:Bug
Projects
None yet
Development

No branches or pull requests

8 participants