Support for creation of Azure Data Explorer table and mapping creation

[sonic1981]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

Currently the azurerm_kusto_eventhub_data_connection accepts the parameters table_name and mapping_rule_name but this comes with the warning that:

Mapping rule/Table must exist before resource is created.

This for me breaks the general terraform flow of creating infrastructure. If I can't create these using terraform it forces me to create the data connection, add the table and mapping manually, then update the terraform to reflect this.

There is Kusto support to add these things, so it should be possible to accept a string containing a kusto query to create these things at the same time as the resource:

e.g.

.create table liam (activeports: dynamic)

.create table test ingestion json mapping 'TestMapping'

New or Affected Resource(s)

• azurerm_kusto_eventhub_data_connection
• azurerm_kusto_cluster

Potential Terraform Configuration

resource "azurerm_kusto_eventhub_data_connection" "eventhub_connection" {
  name                = "my-kusto-eventhub-data-connection"
  resource_group_name = azurerm_resource_group.rg.name
  location            = azurerm_resource_group.rg.location
  cluster_name        = azurerm_kusto_cluster.cluster.name
  database_name       = azurerm_kusto_database.database.name

  eventhub_id    = azurerm_eventhub.evenhub.id
  consumer_group = azurerm_eventhub_consumer_group.consumer_group.name

  table_creation = ".create table liam (activeports: dynamic)"
  mapping_creation = ".create table test ingestion json mapping 'TestMapping'"

  table_name        = "my-table"         #(Optional)
  mapping_rule_name = "my-table-mapping" #(Optional)
  data_format       = "JSON"             #(Optional)
}

or possible need a new resource type(s):

• azurerm_kusto_database_table
• azurerm_kusto_database_mapping

[sonic1981]

Here are the Go SDK docs to perform these actions:

• Create Table
• Create mapping

[favoretti]

Hi there and thank you for reporting this. Apparently I overlooked docs update in this PR:
#10913

ADX allows empty mapping rule creation now, so that you can assign table/mapping to it afterwards. Provider currently doesn't interact with ADX directly as the links you've sent aren't ARM resources, they are ADX API resources. I'll leave this up to provider maintainers to decide whether it's a wanted functionality, but in the mean time shoot in a PR to update the docs.

[favoretti]

Re: docs, nevermind, I'm sorry I should have done more research before commenting. Docs are entirely correct, those must exist before being specified. Since tables and mapping can't be created with ARM, I don't think this falls under the scope of azurerm provider.

[sonic1981]

Fair enough, I'll close this can't fix for now

[favoretti]

@sonic1981 FWIW I've whipped up a temp band-aid for us (we have the same use-case) this morning: https://github.com/favoretti/terraform-provider-adx

Still very fresh and unpublished, writing tests, etc, etc, but might help your case in a couple of days :)

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!