Add support for Azure Data Factory Linked Service Data Lake

[eXXL]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

I would like to provision a data factory and configure a data lake Gen2 account as linked service.
I can provision both services but cannot configure the data lake as a linked service.

See this guide for details:
https://github.com/MicrosoftDocs/azure-docs/blob/master/articles/data-factory/load-azure-data-lake-storage-gen2.md

New or Affected Resource(s)

azurerm_data_factory (maybe affected)
azurerm_data_factory_linked_service_datalake (new)

Potential Terraform Configuration

# Copy-paste your Terraform configurations here - for large Terraform configs,
# please use a service like Dropbox and share a link to the ZIP file. For
# security, you can also encrypt the files using our GPG public key.

References

• #0000

[r0bnet]

Looks useful.
I would create an own more generic resource for that to achieve a better modularity though:

• ARM Template: https://docs.microsoft.com/de-de/azure/templates/microsoft.datafactory/2018-06-01/factories/linkedservices
• Go SDK type: https://github.com/Azure/azure-sdk-for-go/blob/master/services/datafactory/mgmt/2018-06-01/datafactory/models.go

[mbfrahry]

Hey @r0bnet. I really wanted to make the data factory resources as generic as possible but I couldn't find a clean way to do it. There are 50+ sub-resources for Linked Services (as well as Datasets) with their own unique parameters and managing 50+ different ways to create a Linked Service in a clean/user/dev friendly way seemed too complex. I might be missing something here though and am curious on your thoughts on how to make a generic resource.

[r0bnet]

Hi @mbfrahry. You're right and i was way too optimistic tbh. Same is true for runtimeIntegrations as i recognized yesterday that there are different params for each service (e.g. SSIS). So there probably is no other way to create one resource for each linkedService unfortunately.

[mbfrahry]

Glad we're on the same page here! Did you want to tackle this resource? If not, I can take a crack at it tomorrow.

[r0bnet]

No, you may ;) Currently working on the analysis services server so feel free :)

[jungopro]

No, you may ;) Currently working on the analysis services server so feel free :)
hi @r0bnet
analysis services resource is coming soon? any eta for it?

[r0bnet]

@jungopro Tbh not really yet. What currently already works in my branch: Creating and deleting resource with basic required properties such as name, location, resource_group_name and sku.
Importing is only working partially and i don't know why yet but will investigate.

Will try to follow up tomorrow but can't promise anything.

#1734

Issue is already there if you didn't noticed.

[mbfrahry]

Hey @eXXL. This resource has been merged into master and will make it into the next release. Thanks again for opening this issue!

[eXXL]

Thanks, that was fast :-) I really look forward to it! Keep up the great work.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!