Proposal - simplifying resource referencing (part 1)

[anthony-c-martin]

Proposal - simplifying resource referencing (part 1)

Part 1 / Part 2

Problem statement

Passing around / obtaining references to resources in a type-safe manner is overly complex. Rather than inventing non-type-safe mechanisms to refer to resources or resource properties, we should provide a first-class syntax for doing so, with full type-safety and editor support.

New 'resource' function

The az namespace will expose an additional function named resource, which can be used to obtain a reference to a resource within the current scope.

The resource() function takes the first parameter a string (following the resource type format), and a variable number of arguments based on the number of qualified types in the type string. The semantics and validation for the type string parameter behave exactly as those for the type string in resource declarations.

Objects of type scope will also expose a resource() function, which can be used similarly to obtain a reference to a resource at a different scope.

Examples

Global function

var server = resource('Microsoft.Sql/servers@2020-02-02-preview', serverName)
// 'server' can now be used to access resource properties.
var serverProp = server.properties.someProp

Scope function

var otherRg = resourceGroup('otherRgName')
var otherRgServer = otherRg.resource('Microsoft.Sql/servers@2020-02-02-preview', serverName)
// 'otherRgServer' is a reference to a resource in another scope.
var otherRgServerProp = otherRgServer.properties.someProp

Use to set parent property

// here we can inline the function to provide a reference to the parent resource
resource mySubnet 'Microsoft.Network/virtualNetworks/subnets@2020-08-01' = {
  parent: resource('Microsoft.Network/virtualNetworks@2020-08-01', vnetName)
  name: 'mySubnet'
  ...
}

Use to set scope property

// here we can inline the function to set a scope for the extension resource
resource diagnosticSettings 'Microsoft.Insights/diagnosticSettings@2020-01-01' = {
  scope: resource('Microsoft.Compute/virtualMachines@2020-01-01', vmName)
  name: 'diags'
  ...
}

Notes

1. This provides the same functionality as the existing keyword, but as a more convenient one-liner.

The 'child' function on a resource

All resources will expose an additional function named child, which can be used to obtain a reference to a child resource of a given type.

The child() function takes a string for the first parameter (following the nested resource type format), and a second argument for the child resource name.

Examples

Obtaining child resource references

resource myVnet 'Microsoft.Network/virtualNetworks@2020-06-01' = {
  name: 'myVnet'
  ... 
}

// inheriting the same API version
var subnetRef1 = myVnet.child('subnets', subnetName)

// obtaining a reference with a different API version
var subnetRef2 = myVnet.child('subnets@2020-08-01', subnetName)

EDIT 5/28/21: Added info on child() function, removed proposal for extension resources

[majastrz]

Exposing it as a method has a really nice effect of reducing the number of parameters that you have to pass to the resource() function and improves composability.

Even though the name resource() is very generic, I would prefer for us to have one function instead of parent() and child(). The experience will be more seamless because the user doesn't have to think about choosing the right function - it's always resource().

[miqm]

I lean towards parent and child. with resource it's not possible to get parent reference.

as for extension resources in option 1, they start with provider namespaces like Microsoft. can we use that as something which will separate extension from child?

[shenglol]

@miqm that's a good point. I'm inclined to using different functions as it's unambiguous. Also I'd prefer the function names to be more explicit:

// child resource
var myChild = storageAcc.childResource('blobServices', 'default')
// extension resource
var myExt = storageAcc.extensionResource('Microsoft.Insights/diagnosticSettings@2020-01-01', 'diags')
// parent resource
var storageAcc = blobServices.parentResource()

[majastrz]

Oh my bad. I read things too quickly and mixed up parent() and child() with the child() and extension() discussion we had previously ☹.

We have two types of "hierarchies" here (resource nesting and resource extending) and two directions of travel (up and down). In the worst case, we end up with 4 functions.

For resource nesting, parent() and child() absolutely make sense. The language is clear, unambiguous, and symmetric. One of these also matches the parent property name, which is great because they are related semantically.

Where it gets confusing for me is what do we call traversing the extension "hierarchy". What is the name of a resource being extended relative to an extension? extendee()? Definitely not fond of that ☹. The other direction seems fine with extension(). What also doesn't make naming easy here that the property that specifies the extendee resource on an extension resource is called scope.

The other wrinkle is that resource extending also typically does not use parent/child terminology in any of our docs and other content. Could we fix that/teach users about it? I think the answer is yes here, but would it be going too far?

Going up or down the hierarchy, it's technically feasible to distinguish between nesting and extending by inspecting the resource types (aka counting the / chars, etc.) because they are never expressions and are always available.

But does it make sense to combine nesting and extending into a pair of functions if we've already split parent and scope? Or do we stick with it and introduce separate pairs of functions for traversal of each type of hierarchy? I have a really strong preference for consistency in that regard. (Either have 2 pairs of functions or combine scope and parent.)

So I think the big choice we need to make is between these options (ignore names for now):

1. parent()/child() for everything
2. parent()/child()/extension()/extendee()

After that, things should fall into place IMO.

With choice 1 above, there's also a tiny risk that we may have missed some rare case by combining the two hierarchy traversals. I can't think of anything, but we would be abstracting concepts that are technically separate in the runtime.

@shenglol's point about more explicit function names (*resource()) applies to all the choices and is something we should consider.

[miqm]

idea: what about hving a method for traversing downwards (child, extension) and property (parent, scope) for going upwards? scope in case of classic resources would return resource group. scope of resource group returns subscription.

[anthony-c-martin]

It's also worth thinking about what this means for the nested resource scenario. I know that @marcre was keen for us to try to unify deploying children & deploying extensions - e.g.:

Child:

resource vnet 'Microsoft.Network/virtualNetworks@2020-08-01' = {
  name: vnetName

  resource subnet 'subnets' = {
    name: subnetName

  }
}

Extension:

resource vm 'Microsoft.Compute/virtualMachines@2020-12-01' = {
  name: vmName

  resource diags 'microsoft.insights/diagnosticSettings@2015-07-01' = {
    name: diagsName

  }
}

If we were to enable this scenario, then conceptually (in Bicep at least), it might also feel confusing to people that with nested resources, there's a single hierarchy, but outside of nested resources, there are 2 hierarchies. e.g. using the above two examples:

// these two references are equivalent
var subnet = vnet.child('subnets', subnetName)
var subnet2 = vnet::subnet

// these two references are equivalent, but why do I have to remember to use a different function?
var diags = vm.extension('microsoft.insights/diagnosticSettings@2015-07-01', diagsName)
var diags2 = vm::diags

To me, I feel like this leads to two options:

1. Support extensions in nested resources, unify parent/child & scope/extension into a single hierarchy/set of functions.
2. Do not support extensions in nested resources, have two separate hierarchies & sets of functions for traversing.

I'm curious to hear whether this changes anything, and if so, which option (or other suggestion!) would be preferred.

[johndowns]

I prefer option 2. Although option 1 feels simpler, in fact it's obscuring the underlying concepts (e.g. the differences between child and extension resources) to a degree that I think it could be more confusing than helpful overall.

[anthony-c-martin]

I've revised this spec to remove references to extension resources. There's some benefit to providing similar functionality for them, but I feel like child references are going to be more useful - and it feels we can safely punt this decision.

[dciborow]

The docs page points here for the experimental feature resourceTypedParamsAndOutputs

Was a little tricky to find the details here.

This change implements functionality for declaring strongly type
parameters and outputs using resource types.

Example:

param storage resource 'Microsoft.Storage/storageAccounts@2020-01-01'

This declares a parameter that can be interacted with as-if it were an
'existing' resource type declaration of the provided type.

In addition you can do the same with outputs:

output out resource 'Microsoft.Storage/storageAccounts@2020-01-01' = foo

or using type inference (outputs):

output out resource = foo

These features together allow you to pass resources across module
boundaries, and as command line parameters (using the resource ID).

[ghost]

Hi anthony-c-martin, this issue has been marked as stale because it was labeled as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment. Thanks for contributing to bicep! 😄 🦾

[onionhammer]

This bot should close as not planned, not completed... Or just not close things

[StephenWeatherford]

This bot should close as not planned, not completed... Or just not close things

Agreed. #10874

[jachin84]

Is this still happening?

[alex-frankel]

Don't have a clear ETA as there is a dependency on a lower-level improvement to how we process expressions, but this is very much still on our committed list!

[guimatheus92]

Don't have a clear ETA as there is a dependency on a lower-level improvement to how we process expressions, but this is very much still on our committed list!

Also intered on this, noticed this issue is very popular now. Specially regarding RoleAssignments as a module. Hope this get the proper attention a an ETA soon. Thanks for all support.

[alex-frankel]

This unfortunately won't get done in the current semester, but it's working its way up to the top of the list. Added it to our next semester which runs from March '25 - September '25.