mount ADLS as a user and honor their ACLs

[gravesee]

Is there some combination of config options that allows blobfuse2 to mount ADLS with HNS as a user and honor their ACLs? We would like to mount our datalake into linux DSVMs in Azure but want to restrict where the user can read/write based on our Azure AD groups and ACLs assigned to the datalake folders.

[vibhansa-msft]

By user if you mean native linux user then no there is no such config. This is mainly because in unix uid/gid are integers while in Azure backend these are GUIDs so there is no direct correlation. However, if you have set ACLs for one of your identities then you can auth using MSI and then provide enterprise application's object ID as "object id" in blobfuse config and it will honor the ACLs set. This will also need you to provide an extra flag to "respect-acls". Once this id done you will see the same ACLs as that of identity given in your account/container/blobs. Object-id to be provided can be found from the portal or you set ACL for any blob for your identity and an GUID will be displayed against your identity in ACL section we need that GUID as object-id.

[gravesee]

Thank you for the response and sorry for the confusion. For user I am referring to a User Principal in Azure AD. We have a system where colleagues can start a Data Science VM in Azure and connect via SSH. Each User Principal can access our ADLS datalake with security controlled by ACLs only (no RBAC). I have tried using a User Principal ID for the objid in the config.yml along with honour-acls: true yet authentication fails. Based on your response, it seems that only managed identities and not User Principals will work with this approach.

[vibhansa-msft]

Can you share one sample config for me to review. Also, give me a snapshot from one of the ACLs set in the blob container for a directory or file so that I can match and validate config.

[gravesee]

Here is a file:

And here is a sample config:

logging:
  type: base
  level: log_debug
  file-path: /home/gravesee/logs.txt

file_cache:
  path: /mnt/cache

components:
  - libfuse
  - file_cache
  - attr_cache
  - azstorage

libfuse:
  attribute-expiration-sec: 120
  entry-expiration-sec: 120
  negative-entry-expiration-sec: 240

azstorage:
  type: adls
  account-name: datalakestoragenfsawwsj9
  container: datalakegen2filesystem
  endpoint: https://datalakestoragenfsawwsj9.dfs.core.windows.net
  mode: msi
  tenantid: <redacted>
  objid: d0d7693d-f0b6-4c49-a774-cfc301b4795a
  honour-acls: true

And here is the error in the log:

ri Nov  3 17:18:32 UTC 2023 : blobfuse2[248240] : LOG_TRACE [datalake.go (167)]: Datalake::SetupPipeline : Setting up
Fri Nov  3 17:18:32 UTC 2023 : blobfuse2[248240] : LOG_TRACE [datalake.go (128)]: Datalake::getCredential : Getting credential
Fri Nov  3 17:18:32 UTC 2023 : blobfuse2[248240] : LOG_DEBUG [azauth.go (80)]: azAuth::getAzAuth : Account: datalakestoragenfsawwsj9, AccountType: ADLS, Protocol: https, Endpoint: https://datalakestoragenfsawwsj9.dfs.core.windows.net/
Fri Nov  3 17:18:32 UTC 2023 : blobfuse2[248240] : LOG_DEBUG [azauthmsi.go (254)]: azAuthBfsMSI::getCredential : Going for conventional fetchToken
Fri Nov  3 17:18:32 UTC 2023 : blobfuse2[248240] : LOG_ERR [azauthmsi.go (260)]: azAuthBfsMSI::getCredential : Failed to get credential [failed to get token from msi, status code: 400]
Fri Nov  3 17:18:32 UTC 2023 : blobfuse2[248240] : LOG_ERR [datalake.go (138)]: Datalake::getCredential : Failed to get credential
Fri Nov  3 17:18:32 UTC 2023 : blobfuse2[248240] : LOG_ERR [datalake.go (173)]: Datalake::SetupPipeline : Failed to get credential
Fri Nov  3 17:18:32 UTC 2023 : blobfuse2[248240] : LOG_ERR [azstorage.go (141)]: AzStorage::configureAndTest : Failed to create container URL [failed to get credential]
Fri Nov  3 17:18:32 UTC 2023 : blobfuse2[248240] : LOG_ERR [azstorage.go (101)]: AzStorage::Configure : Failed to validate storage account [failed to get credential]
Fri Nov  3 17:18:32 UTC 2023 : blobfuse2[248240] : LOG_ERR [pipeline.go (69)]: Pipeline: error creating pipeline component azstorage [failed to get credential]
Fri Nov  3 17:18:32 UTC 2023 : blobfuse2[248240] : LOG_ERR [mount.go (409)]: mount : failed to initialize new pipeline [failed to get credential]

[vibhansa-msft]

Object id looks correct as per your config, but mount is failing. You might have to check the identity has the required permissions or not. Also, this identity should have at least 'X' (execute) permission throughout the path otherwise it will not be able to iterate the path. So on container and all paths till the file shall have permissions assigned.

[vibhansa-msft]

In azstorage section of your config file add honour-acl: true and objid: <object id for MSI> to make this thing work.