Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Invoke-AzAksRunCommand with help throw a Permission denied error. #17454

Closed
vitalii-lebedev opened this issue Mar 12, 2022 · 7 comments · Fixed by #18580
Closed

Invoke-AzAksRunCommand with help throw a Permission denied error. #17454

vitalii-lebedev opened this issue Mar 12, 2022 · 7 comments · Fixed by #18580
Assignees
@wyunchi-ms
Labels
AKS bug This issue requires a change to an existing behavior in the product in order to be resolved. customer-reported needs-team-attention This issue needs attention from Azure service team or SDK team

Comments

@vitalii-lebedev
Copy link

vitalii-lebedev commented Mar 12, 2022 •
edited by isra-fel
Loading

Description

I'm using Powershell on Linux in the Ubuntu subsystem.
I have a helm chart in the folder. This folder is attached with the CommandContextAttachment parameter.

Invoke-AzAksRunCommand -ResourceGroupName $DestinationResourceGroupName -Name $ClusterName -Command "helm install new-tenant new-tenant" -Force -CommandContextAttachment "yamls"

The response:

Id                : dad300222d03414881c779f934c904fd
ProvisioningState : Succeeded
ExitCode          : 1
StartedAt         : 03/12/2022 10:35:37
FinishedAt        : 03/12/2022 10:35:37
Logs              : Error: open /command-files/new-tenant/.helmignore: permission denied

Reason            :

Execution of
Invoke-AzAksRunCommand -ResourceGroupName $DestinationResourceGroupName -Name $ClusterName -Command "ls new-tenant -la" -Force -CommandContextAttachment "yamls"

Shows

Id                : 19004189917e4a489eef28803e6e8392
ProvisioningState : Succeeded
ExitCode          : 0
StartedAt         : 03/12/2022 10:37:32
FinishedAt        : 03/12/2022 10:37:32
Logs              : total 24
                    drwxr-xr-x 3 nonroot aks 4096 Mar 12 10:37 .
                    drwxr-xr-x 3 nonroot aks 4096 Mar 12 10:37 ..
                    ---------- 1 nonroot aks  349 Mar 12  2022 .helmignore
                    ---------- 1 nonroot aks 1146 Mar 12  2022 Chart.yaml
                    drwxr-xr-x 2 nonroot aks 4096 Mar 12 10:37 templates
                    ---------- 1 nonroot aks 1877 Mar 12  2022 values.yaml

Reason            :

Command:
Invoke-AzAksRunCommand -ResourceGroupName $DestinationResourceGroupName -Name $ClusterName -Command "whoami" -Force -CommandContextAttachment "yamls"

Returns:

Id                : e2c50e6ad1994fc7a010f529762fa48a
ProvisioningState : Succeeded
ExitCode          : 0
StartedAt         : 03/12/2022 10:39:15
FinishedAt        : 03/12/2022 10:39:15
Logs              : nonroot

Reason            :

Issue script & Debug output

PS /home/vlebedev/repos/trg-docs> Invoke-AzAksRunCommand -ResourceGroupName $DestinationResourceGroupName -Name $ClusterName -Command "helm install new-tenant new-tenant" -Force -CommandContextAttachment "yamls"
DEBUG: 11:43:45 - RunAzureRmAksCommand begin processing with ParameterSet 'GroupNameParameterSet'.
DEBUG: 11:43:45 - using account id 'vl@sitewish.ru'...
DEBUG: [Common.Authentication]: Authenticating using Account: 'vl@sitewish.ru', environment: 'AzureCloud', tenant: 'db3eca71-68bb-43e0-8ed6-3a53f6dbc0ed'
DEBUG: 11:43:45 - [SilentAuthenticator] Calling SharedTokenCacheCredential.GetTokenAsync - TenantId:'db3eca71-68bb-43e0-8ed6-3a53f6dbc0ed', Scopes:'https://management.core.windows.net//.default', AuthorityHost:'https://login.microsoftonline.com/', UserId:'vl@sitewish.ru'
DEBUG: SharedTokenCacheCredential.GetToken invoked. Scopes: [ https://management.core.windows.net//.default ] ParentRequestId:
DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:45 - ] Azure region was not configured or could not be discovered. Not using a regional authority.
DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:45] Found 1 cache accounts and 0 broker accounts
DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:45] Returning 1 accounts
DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:45 - ] MSAL MSAL.NetCore with assembly version '4.30.1.0'. CorrelationId(6cd4c55e-c949-4a95-a927-cb29197e580a)
DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:45 - ] === AcquireTokenSilent Parameters ===
DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:45 - ] LoginHint provided: False
DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:45 - ] Account provided: True
DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:45 - ] ForceRefresh: False
DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:45 - ]
=== Request Data ===
Authority Provided? - True
Scopes - https://management.core.windows.net//.default
Extra Query Params Keys (space separated) -
ApiId - AcquireTokenSilent
IsConfidentialClient - False
SendX5C - False
LoginHint ? False
IsBrokerConfigured - False
HomeAccountId - False
CorrelationId - 6cd4c55e-c949-4a95-a927-cb29197e580a

DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:45 - ] === Token Acquisition (SilentRequest) started:

        Authority Host: login.microsoftonline.com
DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:45 - ] Azure region was not configured or could not be discovered. Not using a regional authority.
DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:45 - ] Access token is not expired. Returning the found cache entry. [Current time (03/12/2022 10:43:45) - Expiration Time (03/12/2022 11:04:21 +00:00) - Extended Expiration Time (03/12/2022 11:04:21 +00:00)]
DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:45 - ] Returning access token found in cache. RefreshOn exists ? False
DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:45 - ] Fetched access token from host login.microsoftonline.com.
DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:45 - ] === Token Acquisition finished successfully. An access token was returned with Expiration Time: 03/12/2022 11:04:21 +00:00 and Scopes https://management.core.windows.net//user_impersonation https://management.core.windows.net//.default
DEBUG: SharedTokenCacheCredential.GetToken succeeded. Scopes: [ https://management.core.windows.net//.default ] ParentRequestId:  ExpiresOn: 2022-03-12T11:04:21.0000000+00:00
DEBUG: [Common.Authentication]: Received token with LoginType 'User', Tenant: 'db3eca71-68bb-43e0-8ed6-3a53f6dbc0ed', UserId: 'vl@sitewish.ru'
DEBUG: ============================ HTTP REQUEST ============================

HTTP Method:
GET

Absolute Uri:
https://management.azure.com/subscriptions/fd0d8686-c438-4b8f-9427-5bf2682fecf8/resourceGroups/rg-main-cluster/providers/Microsoft.ContainerService/managedClusters/trg-main-cluster?api-version=2021-05-01

Headers:
x-ms-client-request-id        : a03e2319-d882-47c9-86ad-b093af337957
Accept-Language               : en-US

Body:



DEBUG: ============================ HTTP RESPONSE ============================

Status Code:
OK

Headers:
Cache-Control                 : no-cache
Pragma                        : no-cache
x-ms-routing-request-id       : NORWAYEAST:20220312T104346Z:4f764d11-7442-4fed-94e5-4ce3ec8d8dc2
x-ms-ratelimit-remaining-subscription-reads: 11982
x-ms-correlation-request-id   : 4f764d11-7442-4fed-94e5-4ce3ec8d8dc2
x-ms-request-id               : 11d6d71d-942b-4c5d-b8bc-38733be4066a
Strict-Transport-Security     : max-age=31536000; includeSubDomains
X-Content-Type-Options        : nosniff
Server                        : nginx
Date                          : Sat, 12 Mar 2022 10:43:45 GMT

Body:
{
  "id": "/subscriptions/fd0d8686-c438-4b8f-9427-5bf2682fecf8/resourcegroups/rg-main-cluster/providers/Microsoft.ContainerService/managedClusters/trg-main-cluster",
  "location": "eastus",
  "name": "trg-main-cluster",
  "tags": {
    "Application identifier": "contracts",
    "Business unit": "trg"
  },
  "type": "Microsoft.ContainerService/ManagedClusters",
  "properties": {
    "provisioningState": "Succeeded",
    "powerState": {
      "code": "Running"
    },
    "kubernetesVersion": "1.21.9",
    "dnsPrefix": "5mtk3oknhr5rg",
    "fqdn": "5mtk3oknhr5rg-58fbceab.hcp.eastus.azmk8s.io",
    "azurePortalFQDN": "5mtk3oknhr5rg-58fbceab.portal.hcp.eastus.azmk8s.io",
    "agentPoolProfiles": [
      {
        "name": "npsystem",
        "count": 2,
        "vmSize": "Standard_DS2_v2",
        "osDiskSizeGB": 80,
        "osDiskType": "Ephemeral",
        "kubeletDiskType": "OS",
        "vnetSubnetID": "/subscriptions/fd0d8686-c438-4b8f-9427-5bf2682fecf8/resourceGroups/rg-network-spokes/providers/Microsoft.Network/virtualNetworks/vnet-spoke-trg-main-cluster-00/subnets/snet-clusternodes",
        "maxPods": 30,
        "type": "VirtualMachineScaleSets",
        "availabilityZones": [
          "1",
          "2",
          "3"
        ],
        "maxCount": 2,
        "minCount": 1,
        "enableAutoScaling": true,
        "provisioningState": "Succeeded",
        "powerState": {
          "code": "Running"
        },
        "orchestratorVersion": "1.21.9",
        "enableNodePublicIP": false,
        "nodeTaints": [
          "CriticalAddonsOnly=true:NoSchedule"
        ],
        "mode": "System",
        "osType": "Linux",
        "osSKU": "Ubuntu",
        "nodeImageVersion": "AKSUbuntu-1804gen2containerd-2022.03.02",
        "upgradeSettings": {
          "maxSurge": "33%"
        },
        "enableFIPS": false
      },
      {
        "name": "vl",
        "count": 1,
        "vmSize": "Standard_DS2_v2",
        "osDiskSizeGB": 120,
        "osDiskType": "Managed",
        "vnetSubnetID": "/subscriptions/fd0d8686-c438-4b8f-9427-5bf2682fecf8/resourceGroups/rg-network-spokes/providers/Microsoft.Network/virtualNetworks/vnet-spoke-trg-main-cluster-00/subnets/snet-clusternodes",
        "maxPods": 30,
        "type": "VirtualMachineScaleSets",
        "availabilityZones": [
          "1",
          "2",
          "3"
        ],
        "maxCount": 2,
        "minCount": 1,
        "enableAutoScaling": true,
        "provisioningState": "Succeeded",
        "powerState": {
          "code": "Running"
        },
        "orchestratorVersion": "1.21.9",
        "enableNodePublicIP": false,
        "nodeLabels": {
          "tenant": "vl"
        },
        "mode": "System",
        "osType": "Linux",
        "osSKU": "Ubuntu",
        "nodeImageVersion": "AKSUbuntu-1804gen2containerd-2022.03.02",
        "enableFIPS": false
      }
    ],
    "windowsProfile": {
      "adminUsername": "azureuser",
      "enableCSIProxy": true
    },
    "servicePrincipalProfile": {
      "clientId": "msi"
    },
    "addonProfiles": {
      "aciConnectorLinux": {
        "enabled": false,
        "config": null
      },
      "azureKeyvaultSecretsProvider": {
        "enabled": true,
        "config": {
          "enableSecretRotation": "true",
          "rotationPollInterval": "30s"
        },
        "identity": {
          "resourceId": "/subscriptions/fd0d8686-c438-4b8f-9427-5bf2682fecf8/resourcegroups/trg-main-cluster-nodepools/providers/Microsoft.ManagedIdentity/userAssignedIdentities/azurekeyvaultsecretsprovider-trg-main-cluster",
          "clientId": "8b8e8201-9e24-4ceb-98e4-0aaf600db9c6",
          "objectId": "360eeba6-d71a-4d8d-9f4c-2d527d27490c"
        }
      },
      "azurepolicy": {
        "enabled": true,
        "config": null,
        "identity": {
          "resourceId": "/subscriptions/fd0d8686-c438-4b8f-9427-5bf2682fecf8/resourcegroups/trg-main-cluster-nodepools/providers/Microsoft.ManagedIdentity/userAssignedIdentities/azurepolicy-trg-main-cluster",
          "clientId": "58c9438f-257b-4400-bd63-b4a853c59cba",
          "objectId": "0679619a-0f0e-4ee4-8d4e-c5e16d70d460"
        }
      },
      "httpApplicationRouting": {
        "enabled": false,
        "config": null
      }
    },
    "nodeResourceGroup": "trg-main-cluster-nodepools",
    "enableRBAC": true,
    "enablePodSecurityPolicy": false,
    "networkProfile": {
      "networkPlugin": "azure",
      "networkPolicy": "azure",
      "loadBalancerSku": "Standard",
      "serviceCidr": "172.16.0.0/16",
      "dnsServiceIP": "172.16.0.10",
      "dockerBridgeCidr": "172.18.0.1/16",
      "outboundType": "userDefinedRouting"
    },
    "aadProfile": {
      "managed": true,
      "adminGroupObjectIDs": [
        "50a513ed-78f2-427b-9f2c-a4aeb1e6fbfb"
      ],
      "enableAzureRBAC": true,
      "tenantID": "db3eca71-68bb-43e0-8ed6-3a53f6dbc0ed"
    },
    "maxAgentPools": 100,
    "apiServerAccessProfile": {
      "enablePrivateCluster": false
    },
    "identityProfile": {
      "kubeletidentity": {
        "resourceId": "/subscriptions/fd0d8686-c438-4b8f-9427-5bf2682fecf8/resourcegroups/trg-main-cluster-nodepools/providers/Microsoft.ManagedIdentity/userAssignedIdentities/trg-main-cluster-agentpool",
        "clientId": "16c1df5a-e515-47d9-8b50-ca0f425047e2",
        "objectId": "c7c2b9cd-44ed-4368-a3aa-93e3bb6052d4"
      }
    },
    "autoScalerProfile": {
      "balance-similar-node-groups": "false",
      "expander": "random",
      "max-empty-bulk-delete": "10",
      "max-graceful-termination-sec": "600",
      "max-node-provision-time": "15m",
      "max-total-unready-percentage": "45",
      "new-pod-scale-up-delay": "0s",
      "ok-total-unready-count": "3",
      "scale-down-delay-after-add": "10m",
      "scale-down-delay-after-delete": "20s",
      "scale-down-delay-after-failure": "3m",
      "scale-down-unneeded-time": "10m",
      "scale-down-unready-time": "20m",
      "scale-down-utilization-threshold": "0.5",
      "scan-interval": "10s",
      "skip-nodes-with-local-storage": "true",
      "skip-nodes-with-system-pods": "true"
    },
    "podIdentityProfile": {
      "enabled": true
    },
    "disableLocalAccounts": true
  },
  "identity": {
    "type": "UserAssigned",
    "userAssignedIdentities": {
      "/subscriptions/fd0d8686-c438-4b8f-9427-5bf2682fecf8/resourceGroups/rg-main-cluster/providers/Microsoft.ManagedIdentity/userAssignedIdentities/mi-trg-main-cluster-controlplane": {
        "clientId": "86347c18-a493-4dab-a180-24f4048be343",
        "principalId": "9c491543-28f5-48a2-ab73-b64f3b4dd1be"
      }
    }
  },
  "sku": {
    "name": "Basic",
    "tier": "Paid"
  }
}


DEBUG: Will zip all the files under /home/vlebedev/repos/trg-docs/yamls.
DEBUG: 11:43:46 - [SilentAuthenticator] Calling SharedTokenCacheCredential.GetTokenAsync - TenantId:'db3eca71-68bb-43e0-8ed6-3a53f6dbc0ed', Scopes:'6dae42f8-4368-4678-94ff-3960e28e3630/.default', AuthorityHost:'https://login.microsoftonline.com/', UserId:'vl@sitewish.ru'
DEBUG: SharedTokenCacheCredential.GetToken invoked. Scopes: [ 6dae42f8-4368-4678-94ff-3960e28e3630/.default ] ParentRequestId:
DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:46 - ] Azure region was not configured or could not be discovered. Not using a regional authority.
DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:46] Found 1 cache accounts and 0 broker accounts
DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:46] Returning 1 accounts
DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:46 - ] MSAL MSAL.NetCore with assembly version '4.30.1.0'. CorrelationId(e0b143a7-ba94-4f6c-bfab-0f2a358ffafe)
DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:46 - ] === AcquireTokenSilent Parameters ===
DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:46 - ] LoginHint provided: False
DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:46 - ] Account provided: True
DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:46 - ] ForceRefresh: False
DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:46 - ]
=== Request Data ===
Authority Provided? - True
Scopes - 6dae42f8-4368-4678-94ff-3960e28e3630/.default
Extra Query Params Keys (space separated) -
ApiId - AcquireTokenSilent
IsConfidentialClient - False
SendX5C - False
LoginHint ? False
IsBrokerConfigured - False
HomeAccountId - False
CorrelationId - e0b143a7-ba94-4f6c-bfab-0f2a358ffafe

DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:46 - ] === Token Acquisition (SilentRequest) started:

        Authority Host: login.microsoftonline.com
DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:46 - ] Azure region was not configured or could not be discovered. Not using a regional authority.
DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:46 - ] Access token is not expired. Returning the found cache entry. [Current time (03/12/2022 10:43:46) - Expiration Time (03/12/2022 11:12:15 +00:00) - Extended Expiration Time (03/12/2022 11:12:15 +00:00)]
DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:46 - ] Returning access token found in cache. RefreshOn exists ? False
DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:46 - ] Fetched access token from host login.microsoftonline.com.
DEBUG: (False) MSAL 4.30.1.0 MSAL.NetCore Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 [03/12/2022 10:43:46 - ] === Token Acquisition finished successfully. An access token was returned with Expiration Time: 03/12/2022 11:12:15 +00:00 and Scopes 6dae42f8-4368-4678-94ff-3960e28e3630/user_impersonation 6dae42f8-4368-4678-94ff-3960e28e3630/.default
DEBUG: SharedTokenCacheCredential.GetToken succeeded. Scopes: [ 6dae42f8-4368-4678-94ff-3960e28e3630/.default ] ParentRequestId:  ExpiresOn: 2022-03-12T11:12:15.0000000+00:00
DEBUG: ============================ HTTP REQUEST ============================

HTTP Method:
POST

Absolute Uri:
https://management.azure.com/subscriptions/fd0d8686-c438-4b8f-9427-5bf2682fecf8/resourceGroups/rg-main-cluster/providers/Microsoft.ContainerService/managedClusters/trg-main-cluster/runCommand?api-version=2021-05-01

Headers:
x-ms-client-request-id        : a03e2319-d882-47c9-86ad-b093af337957
Accept-Language               : en-US

Body:
{
  "command": "helm install new-tenant new-tenant",
  "context": "UEsDBBQAAAAIAHddbFQW7DIn8QAAAF0BAAAWAAAAbmV3LXRlbmFudC8uaGVsbWlnbm9yZUxPQU7DMBC8+xWLeoEK2Y9oOXACKahX5MRbe6ljW/YmAQ68nU2qCi6j3dmZ8XgHr5YZa2rAGcinXBGWgAn6iaKj5KHY4WI9Nq128BaoQZtKyZVlCBgj+Jh7GC0PQdSPUDFaphnFx+Efb5OTgIRerjnBfal4pk90sJDo7h40vKT4BTltzrUSFKwQKaFW+ti9dyzdJOKQx1ECTocOHNWmtCc2G17rK91/V7PhjQjerHBb25zMX1Av/5sKnCliU3vdliLY24sgj+ucJUftf8RxspXy1OD5+CTvlpo/cGClyaE1V7lQSs9tyA6N+gUAAP//AwBQSwMEFAAAAAgAd11sVDrBZw2QAwAAVQcAABYAAABuZXctdGVuYW50L3ZhbHVlcy55YW1sjFVNb9tGEL3zVwziQy6W4uQU8CY4aGHAdoTYblEUPayWQ3Hb5S67H5LZov+9b5aU5DgKEECAyN03X2/eDC/oE7cq20Q7ZTNHan0gx/tFYqdcWlYX9NiZSPgp+m11d7sAoFcpcUOtsSyAT6ytCgwPwaiNhZPkacM0qBgBMw6vo8+BEveDVYnjsqoCD9Zode2zSzW9ryrTqy3XFRFufDTJh7EmtzXuGWdDtnbtYYCzm/bep3XgyC7h6oI+7zgE00jcjqn4oaS2tO98ZGrm+sx0rTsVEqlh+IVDNN4t4QLgmt68mXNYI9YD68Ap1vT7H5VTPR9CFFgLwDeHVeSwM5pXWpeSSmYPA2vTGmS27xjRA0iccaQmIMXOZ9sIXwgJchpYTk81pZC5OFo555NKyLeQq5pG/qSeV+6AVidsTf/+V+wfgZSUybfnrMRZjrws2JuW4AAQ0OSaOZdCH7K5RAXFEd637DhIxrA1blscH7g59hou5X3iaPDN6lV2cga2czBpvPYu8XM6Zt3Gn4PPQ00frq6uhOHzMK0GtTHWJBA98U7UBD8cnhe0ur0tzyil+ezs+MX79BPUG8eIPF/wHLJbxXvvBPD6+AmkQalzKoU/iZDGAdVd2wxX4WYtYvUBxh8BAyvQaUkK44TRaGpqlY3iFTMT4/2BmnNtk8h/5Q0HxzIzxr+b/S2L7Wk6vgUmGxdKF99ShPjHMKSSiRAiL/U0Cwt+VmgVL63XypZ7lKBSN4MnAzmo6d3xZII8ltpvxLzHNJbsZ81rocZOE1TasICiZKimkr8KvQBw7tUpzYtD6HNZYn1EbBTNJ4n/ypBhVtaOaLP2PRJqipCh7VhyGo+74GhdFA6AZbWDZmXTKdl0Gl3QxucpLd15GRZZjaJxTEpYTmsRvfRYcDIjEd6QqtPTf0hRZEPeofU7E7wThrAITOoIWk12ltYhlUuKWXcS/s44I+1cyihicVLjaa/cV5W8MMtuqjZNA+it9XsIpXi3xglENX9CnnLfSwDHMIwqjJel/sC9L9VjBeUA/jZBFW5aKJrenqh+u5yd9ubUIz3kMhX9/N7Dm6zu9x8+3pm5xL/xZflRi0rl5COajBrODk5v3Jfp4xHlw4ED9fziAOMpGz1sOV2vn54SFsM/RZhrRhEQKb4yMpwSe4LdlfjfR1bON/zAljU+StPSSt7K6puGFQqvVNuia2ks1/8DAAD//wMAUEsDBBQAAAAIAHddbFRwkbIXDAIAAHoEAAAVAAAAbmV3LXRlbmFudC9DaGFydC55YW1sjFQ7k9MwEO79K3YuRWAGlISC4jo6GOi4oVekdSxOL/Rw8L9n14oTXxhuKCWtvteuJKP5gSmb4B9h/NB56fARPJ7fF/TSl05jVsnEMhd8gs9oHahBpgJ9SPC1HjF5LJi7bkPH7URJD0cENGXABLTYyhitUZJRtkD3JGytOSaZpm27I7oNA9zK2nYGmZCqVbAW1bwfeijoopVECmWQV7oo1bM8oQbjS4CxmaKlTGowIyncAO1TocZow4S6cX5rOha+mMJoNELN2FcLtRhriiEqUt1XP2vIs3cyd/GrcUQbIiYBTwNOW5JsvLJVMzvzSuZEr9GriQ3Iv32SNON/kkfCDRlXxNLrFfNsjqkT4yXjT4QfTURrPIp7NzqADyywp1NCmlbZMS43CMkMcohc+SKeMkUahpVYbvLTYEhFXtm/RC3a0WUFvjqaDchDqFYzLkWS0KEvlApKNUAxDmEKFZx8nrH8aenSDZxVGs5n0f3uki05n8tI3lUBXb4Mcxsc/B0pUeIjyJ5GKJzhOzoaa6OWQoZ5M5QS8+Nul9ERlAjptHvbjcuz2IuD2N9bv7PJU9nEXBt7RMa+pvl6OoT+f/ks6ayYxEvT3MaVcUL+t/U2sIsMGgXbJvDm794VeaiZbxLul8LLhCo4x9M450wPhxoGZ3r88KsG6pjoCOD6yTwcxOGj2D90fwAAAP//AwBQSwMEFAAAAAgAd11sVK2PGtOEAAAArAAAACIAAABuZXctdGVuYW50L3RlbXBsYXRlcy9uYW1lc3BhY2UueW1sZI0xDsIwEAR7v+I+gBGtJb5AhegPe6VYcc6W70wT5e84tHSj1WiWW36ha64S6HNza5YU6MEbtHGE22Cc2Dg4kjkG2nfyTwiLnRIdhyMWqcY2Ezo1Io0L0ijonktb2K/jjS4wqM/1KjXhoiiIVnsg+7Xu/1ltiPNuwmwPPfELAAD//wMAUEsBAhQDFAAAAAgAd11sVBbsMifxAAAAXQEAABYAAAAAAAAAAAAAAAAAAAAAAG5ldy10ZW5hbnQvLmhlbG1pZ25vcmVQSwECFAMUAAAACAB3XWxUOsFnDZADAABVBwAAFgAAAAAAAAAAAAAAAAAlAQAAbmV3LXRlbmFudC92YWx1ZXMueWFtbFBLAQIUAxQAAAAIAHddbFRwkbIXDAIAAHoEAAAVAAAAAAAAAAAAAAAAAOkEAABuZXctdGVuYW50L0NoYXJ0LnlhbWxQSwECFAMUAAAACAB3XWxUrY8a04QAAACsAAAAIgAAAAAAAAAAAAAAAAAoBwAAbmV3LXRlbmFudC90ZW1wbGF0ZXMvbmFtZXNwYWNlLnltbFBLBQYAAAAABAAEABsBAADsBwAAAAA=",
  "clusterToken": "redacted"
}


DEBUG: ============================ HTTP RESPONSE ============================

Status Code:
Accepted

Headers:
Cache-Control                 : no-cache
Pragma                        : no-cache
Location                      : https://management.azure.com/subscriptions/fd0d8686-c438-4b8f-9427-5bf2682fecf8/resourceGroups/rg-main-cluster/providers/Microsoft.ContainerService/managedclusters/trg-main-cluster/commandResults/e538157ba57e426a8cb4e957de5a7b13?api-version=2021-05-01
x-ms-ratelimit-remaining-subscription-writes: 1194
x-ms-correlation-request-id   : 6e27023b-3c25-4285-9f90-28df7074179b
x-ms-request-id               : e538157b-a57e-426a-8cb4-e957de5a7b13
Strict-Transport-Security     : max-age=31536000; includeSubDomains
Server                        : nginx
x-ms-routing-request-id       : NORWAYEAST:20220312T104352Z:6e27023b-3c25-4285-9f90-28df7074179b
X-Content-Type-Options        : nosniff
Date                          : Sat, 12 Mar 2022 10:43:51 GMT

Body:



DEBUG: ============================ HTTP REQUEST ============================

HTTP Method:
GET

Absolute Uri:
https://management.azure.com/subscriptions/fd0d8686-c438-4b8f-9427-5bf2682fecf8/resourceGroups/rg-main-cluster/providers/Microsoft.ContainerService/managedclusters/trg-main-cluster/commandResults/e538157ba57e426a8cb4e957de5a7b13?api-version=2021-05-01

Headers:
x-ms-client-request-id        : a03e2319-d882-47c9-86ad-b093af337957

Body:



DEBUG: ============================ HTTP RESPONSE ============================

Status Code:
OK

Headers:
Cache-Control                 : no-cache
Pragma                        : no-cache
x-ms-ratelimit-remaining-subscription-reads: 11981
x-ms-correlation-request-id   : 2c0490fd-224d-4e54-8bf4-f66d0ca279e0
x-ms-request-id               : 4880287d-d8bf-4435-ab3d-6569267537c2
Strict-Transport-Security     : max-age=31536000; includeSubDomains
X-Content-Type-Options        : nosniff
Server                        : nginx
x-ms-routing-request-id       : NORWAYEAST:20220312T104422Z:2c0490fd-224d-4e54-8bf4-f66d0ca279e0
Date                          : Sat, 12 Mar 2022 10:44:22 GMT

Body:
{
  "id": "e538157ba57e426a8cb4e957de5a7b13",
  "properties": {
    "provisioningState": "Succeeded",
    "exitCode": 1,
    "startedAt": "2022-03-12T10:43:58Z",
    "finishedAt": "2022-03-12T10:43:58Z",
    "logs": "Error: open /command-files/new-tenant/.helmignore: permission denied\n"
  }
}


DEBUG: ============================ HTTP REQUEST ============================

HTTP Method:
GET

Absolute Uri:
https://management.azure.com/subscriptions/fd0d8686-c438-4b8f-9427-5bf2682fecf8/resourceGroups/rg-main-cluster/providers/Microsoft.ContainerService/managedclusters/trg-main-cluster/commandResults/e538157ba57e426a8cb4e957de5a7b13?api-version=2021-05-01

Headers:
x-ms-client-request-id        : a03e2319-d882-47c9-86ad-b093af337957

Body:



DEBUG: ============================ HTTP RESPONSE ============================

Status Code:
OK

Headers:
Cache-Control                 : no-cache
Pragma                        : no-cache
x-ms-ratelimit-remaining-subscription-reads: 11980
x-ms-correlation-request-id   : a58cffc9-496d-467e-bd9e-bbec58bcc7c6
x-ms-request-id               : 25d9a9e9-3b97-4a0d-b6c6-c3f24cb33a6d
Strict-Transport-Security     : max-age=31536000; includeSubDomains
X-Content-Type-Options        : nosniff
Server                        : nginx
x-ms-routing-request-id       : NORWAYEAST:20220312T104423Z:a58cffc9-496d-467e-bd9e-bbec58bcc7c6
Date                          : Sat, 12 Mar 2022 10:44:23 GMT

Body:
{
  "id": "e538157ba57e426a8cb4e957de5a7b13",
  "properties": {
    "provisioningState": "Succeeded",
    "exitCode": 1,
    "startedAt": "2022-03-12T10:43:58Z",
    "finishedAt": "2022-03-12T10:43:58Z",
    "logs": "Error: open /command-files/new-tenant/.helmignore: permission denied\n"
  }
}



Id                : e538157ba57e426a8cb4e957de5a7b13
ProvisioningState : Succeeded
ExitCode          : 1
StartedAt         : 03/12/2022 10:43:58
FinishedAt        : 03/12/2022 10:43:58
Logs              : Error: open /command-files/new-tenant/.helmignore: permission denied

Reason            :

DEBUG: AzureQoSEvent: Module: Az.Aks:3.1.1; CommandName: Invoke-AzAksRunCommand; PSVersion: 7.2.1; IsSuccess: True; Duration: 00:00:37.9216352
DEBUG: Finish sending metric.
DEBUG: 11:44:23 - RunAzureRmAksCommand end processing.

Environment data

PS /home/vlebedev/repos/trg-docs> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      7.2.1
PSEdition                      Core
GitCommitId                    7.2.1
OS                             Linux 5.10.60.1-microsoft-standard-WSL2 #1 SMP …
Platform                       Unix
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Module versions

PS /home/vlebedev/repos/trg-docs> Get-Module Az*

ModuleType Version    PreRelease Name                                ExportedCommands
---------- -------    ---------- ----                                ----------------
Script     2.7.2                 Az.Accounts                         {Add-AzEnvironment, Clear-AzContext, Clear-AzDefault, Connect-AzAccount…}
Script     3.1.1                 Az.Aks                              {Disable-AzAksAddOn, Enable-AzAksAddOn, Get-AzAksCluster, Get-AzAksNodePool…}

Error output

No response
@vitalii-lebedev vitalii-lebedev added bug This issue requires a change to an existing behavior in the product in order to be resolved. needs-triage This is a new issue that needs to be triaged to the appropriate team. labels Mar 12, 2022
@ghost ghost added customer-reported and removed needs-triage This is a new issue that needs to be triaged to the appropriate team. labels Mar 12, 2022
@dingmeng-xue
Copy link
Member

Thanks for reporting. We will look into it.

@wyunchi-ms
Copy link
Contributor

Hi @vitalii-lebedev , as your log shows:

Id                : 19004189917e4a489eef28803e6e8392
ProvisioningState : Succeeded
ExitCode          : 0
StartedAt         : 03/12/2022 10:37:32
FinishedAt        : 03/12/2022 10:37:32
Logs              : total 24
                    drwxr-xr-x 3 nonroot aks 4096 Mar 12 10:37 .
                    drwxr-xr-x 3 nonroot aks 4096 Mar 12 10:37 ..
                    ---------- 1 nonroot aks  349 Mar 12  2022 .helmignore
                    ---------- 1 nonroot aks 1146 Mar 12  2022 Chart.yaml
                    drwxr-xr-x 2 nonroot aks 4096 Mar 12 10:37 templates
                    ---------- 1 nonroot aks 1877 Mar 12  2022 values.yaml

Reason            :

nonroot doesn't have permission to access .helmignore. I think you need to grant the permission for the file first.

@wyunchi-ms wyunchi-ms added the needs-author-feedback More information is needed from author to address the issue. label Apr 6, 2022
@ghost ghost added the no-recent-activity There has been no recent activity on this issue. label Apr 13, 2022
@ghost
Copy link

ghost commented Apr 13, 2022

Hi, we're sending this friendly reminder because we haven't heard back from you in a while. We need more information about this issue to help address it. Please be sure to give us your input within the next 7 days. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

@vitalii-lebedev
Copy link
Author

Hi @wyunchi-ms. Sorry, but I don't understand how your advice can help me to solve the issue. Locally I have all the required permissions. The output you see is from the remote AKS.

  1. So, again, locally I have all the permissions.
  2. Execution of the Invoke-AzAksRunCommand command with -CommandContextAttachment parameters copied the folder defined in this parameter to the server.
  3. But after copying from the local machine to the remote host the permissions are lost.

Again, I don't copy this folder manually. Everything is done by Invoke-AzAksRunCommand and -CommandContextAttachment parameter.

@ghost ghost added needs-team-attention This issue needs attention from Azure service team or SDK team and removed needs-author-feedback More information is needed from author to address the issue. no-recent-activity There has been no recent activity on this issue. labels Apr 16, 2022
@Zaldos
Copy link

Zaldos commented May 10, 2022

I am also having this issue. Locally everyone has permissions to read (this is ran from Microsoft hosted agent running in an Azure DevOps pipeline):
image

However on using Invoke-AzAksRunCommand the files lose read permissions (ran Invoke-AzAksRunCommand with the command ls -la and also attempted a helm install in the same line)
image

It does seem to be an issue with transferring files to wherever this runs from.

@vitalii-lebedev
Copy link
Author

Hey @Zaldos! I found a workaround. It looks ugly but at least it works.

Invoke-AzAksRunCommand -ResourceGroupName resource-group -Name cluster-name -Command "chmod -R 777 namespace && $cmd" -Force -CommandContextAttachment "yamls"

@wyunchi-ms wyunchi-ms linked a pull request Jun 16, 2022 that will close this issue
[Aks]Add parameter CommandContextAttachmentZip to cmdlet Invoke-AzAksRunCommand #18580
Merged
8 tasks
@wyunchi-ms
Copy link
Contributor

Hi @vitalii-lebedev & @Zaldos, sorry for causing trouble for you. As I investigated that the zip API we used in DotNet Standard doesn't support well in Linux. And we don't have a plan to migrate it to DotNet Core in recently.
I will add a new parameter CommandContextAttachmentZip to supply a workaround for Linux users. You can zip your folder locally and pass the path to this parameter.
This feature will be published around next release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@wyunchi-ms wyunchi-ms

Labels
AKS bug This issue requires a change to an existing behavior in the product in order to be resolved. customer-reported needs-team-attention This issue needs attention from Azure service team or SDK team
Projects
None yet
Milestone
No milestone
4 participants
@Zaldos @vitalii-lebedev @wyunchi-ms @dingmeng-xue