"failed to build archive" with ImDisk ram disk

[superblaubeere27]

Problem

Cargo fails to compile on ImDisk ram disk

Steps

1. Install ImDisk on Windows
2. Create a ramdisk and a cargo package on it
3. Try to compile the package

Possible Solution(s)

No response

Notes

The resulting log of cargo will look like:

PS R:\kss> cargo run
warning: could not canonicalize path: 'R:\kss'
warning: could not canonicalize path: 'R:\'
   Compiling autocfg v1.0.1
   Compiling cfg-if v1.0.0
   Compiling winapi v0.3.9
   Compiling bitflags v1.3.2
   Compiling proc-macro2 v1.0.32
   Compiling lazy_static v1.4.0
   Compiling crossbeam-utils v0.8.5
   Compiling syn v1.0.81
   Compiling crossbeam-epoch v0.9.5
error: failed to build archive: function not supported

error: could not compile `bitflags` due to previous error
warning: build failed, waiting for other jobs to finish...
error: build failed

Version

cargo 1.56.0 (4ed5d137b 2021-10-04)

[ehuss]

I have transferred this to rust-lang/rust, as this error is being generated by rustc when generating the archive.

[ChrisDenton]

ImDisk ram disks cause a lot of problems because they don't implement essential kernel interfaces. Usually it's canonicalize that's the problem but the error would be different in that case. Hm... the error message comes from here which suggests the error originates in LLVM?

[ChrisDenton]

On further investigation this does seem to be an LLVM issue. The "function not supported" error seems to be what LLVM maps ERROR_INVALID_FUNCTION to. Which is indeed what I'd expect from calling GetFinalPathNameByHandleW on a driver that doesn't implement the interface.

[the8472]

To ask the obvious: since people keep stumbling over this every now and then, has anyone reported the issue to ImDisk?

[ChrisDenton]

It seems like a fundamental issue that they're aware of https://www.ltr-data.se/opencode.html/#ImDisk

It runs on very old versions of Windows NT as well as modern Windows versions. However, because of this compatibility design and because it emulates disk volumes rather than complete disks, it is not always compatible with all applications and drivers. For instance, you cannot manage things like mount points, drive letters and similar for ImDisk drives using mountvol command line tool or in Disk Management in Windows. As another example, you cannot create or access shadow copies on ImDisk drives. So, applications that use similar Windows features as Disk Management dialog to enumerate disks and disk volumes to find disk properties like sector sizes and similar, might possibly not work as expected with ImDisk drives.
...
If you need full disk emulation, including integration with Disk Management, Volume Shadow Copy Services, mountvol and diskpart command line tools as well as better compatibility with applications, we provide another open source virtual disk project, called Arsenal Image Mounter.

Other than the aforementioned Arsenal Image Mounter there's also free (as in price) software such as Radeon Ramdisk or Ultra RAMDisk lite. However, ImDisk has the name recognition so I suspect we'll continue seeing these issues crop up.

[MolotovCherry]

It seems like a fundamental issue that they're aware of https://www.ltr-data.se/opencode.html/#ImDisk

It runs on very old versions of Windows NT as well as modern Windows versions. However, because of this compatibility design and because it emulates disk volumes rather than complete disks, it is not always compatible with all applications and drivers. For instance, you cannot manage things like mount points, drive letters and similar for ImDisk drives using mountvol command line tool or in Disk Management in Windows. As another example, you cannot create or access shadow copies on ImDisk drives. So, applications that use similar Windows features as Disk Management dialog to enumerate disks and disk volumes to find disk properties like sector sizes and similar, might possibly not work as expected with ImDisk drives.
...
If you need full disk emulation, including integration with Disk Management, Volume Shadow Copy Services, mountvol and diskpart command line tools as well as better compatibility with applications, we provide another open source virtual disk project, called Arsenal Image Mounter.

Other than the aforementioned Arsenal Image Mounter there's also free (as in price) software such as Radeon Ramdisk or Ultra RAMDisk lite. However, ImDisk has the name recognition so I suspect we'll continue seeing these issues crop up.

Thanks to the above comment, I was able to find a solution to these problems.

Edit: I created a repo with all the steps to do what I said below. Recommend to check the repo for the most recent updates to the script

Download and install the driver for arsenal image mounter, then go to this page and download aim_ll.zip, then you can create a volume (as a startup script task scheduler) like so

%~dp0\aim_ll -a -t vm -s 3G -p "/fs:ntfs /v:RamDisk /q /y" -o rw,fix,hd -m R:

See the cmdline details for more info, but with -s you can set the size, format as ntfs, make it a hd, etc

This does work with cargo, cause I just tested it!

As a bonus, I'll also include a custom powershell proxy script I made which redirects all cargo builds to the temp folder automatically (what you do is, set your temp folder env var to your ramdrive). It changes CARGO_BUILD_TARGET_DIR every time you cd to a different folder, with a unique id too, so it will always refer to only the one you're building in.

Also has some powershell commands:

• Clean-RustDir: Clean out the tmp rust project target dir (e.g. R:\Temp\rust\MyProject-1234\*)
• Clean-Rust: Clean out the entire tmp rust folder (e.g. R:\Temp\rust\*)
• Rust-TargetDir: Opens explorer to the rust target dir (e.g. R:\Temp\rust\MyProject-1234\*)

Powershell script

function _Get-FNVHash {
    param(
        [string]$InputString
    )

    # Initial prime and offset chosen for 32-bit output
    # See https://en.wikipedia.org/wiki/Fowler–Noll–Vo_hash_function
    [uint32]$FNVPrime = 16777619
    [uint32]$offset = 2166136261

    # Convert string to byte array, may want to change based on input collation
    $bytes = [System.Text.Encoding]::UTF8.GetBytes($InputString)

    # Copy offset as initial hash value
    [uint32]$hash = $offset

    foreach($octet in $bytes)
    {
        # Apply XOR, multiply by prime and mod with max output size
        $hash = $hash -bxor $octet
        $hash = $hash * $FNVPrime % [System.Math]::Pow(2,32)
    }
    return $hash
}

$rustBaseDir = "$env:TMP\rust"

function _Get-RustTmpDir {
    return "$rustBaseDir\$(Split-Path (Get-Location) -Leaf)-$(_Get-FNVHash (Get-Location))"
}


function Set-Location {
    [CmdletBinding(DefaultParameterSetName='Path', HelpUri='https://go.microsoft.com/fwlink/?LinkID=2097049')]
    param(
        [Parameter(ParameterSetName='Path', Position=0, ValueFromPipeline=$true, ValueFromPipelineByPropertyName=$true)]
        [string]
        ${Path},

        [Parameter(ParameterSetName='LiteralPath', Mandatory=$true, ValueFromPipelineByPropertyName=$true)]
        [Alias('PSPath','LP')]
        [string]
        ${LiteralPath},

        [switch]
        ${PassThru},

        [Parameter(ParameterSetName='Stack', ValueFromPipelineByPropertyName=$true)]
        [string]
        ${StackName})


    dynamicparam
    {
        try {
            $targetCmd = $ExecutionContext.InvokeCommand.GetCommand('Microsoft.PowerShell.Management\Set-Location', [System.Management.Automation.CommandTypes]::Cmdlet, $PSBoundParameters)
            $dynamicParams = @($targetCmd.Parameters.GetEnumerator() | Microsoft.PowerShell.Core\Where-Object { $_.Value.IsDynamic })
            if ($dynamicParams.Length -gt 0)
            {
                $paramDictionary = [Management.Automation.RuntimeDefinedParameterDictionary]::new()
                foreach ($param in $dynamicParams)
                {
                    $param = $param.Value

                    if(-not $MyInvocation.MyCommand.Parameters.ContainsKey($param.Name))
                    {
                        $dynParam = [Management.Automation.RuntimeDefinedParameter]::new($param.Name, $param.ParameterType, $param.Attributes)
                        $paramDictionary.Add($param.Name, $dynParam)
                    }
                }

                return $paramDictionary
            }
        } catch {
            throw
        }
    }

    begin
    {
        try {
            $outBuffer = $null
            if ($PSBoundParameters.TryGetValue('OutBuffer', [ref]$outBuffer))
            {
                $PSBoundParameters['OutBuffer'] = 1
            }

            $wrappedCmd = $ExecutionContext.InvokeCommand.GetCommand('Microsoft.PowerShell.Management\Set-Location', [System.Management.Automation.CommandTypes]::Cmdlet)
            $scriptCmd = {& $wrappedCmd @PSBoundParameters }

            $steppablePipeline = $scriptCmd.GetSteppablePipeline($myInvocation.CommandOrigin)
            $steppablePipeline.Begin($PSCmdlet)
        } catch {
            throw
        }
    }

    process
    {
        try {
            $steppablePipeline.Process($_)
        } catch {
            throw
        }
    }

    end
    {
        try {
            $steppablePipeline.End()
            $env:CARGO_BUILD_TARGET_DIR = _Get-RustTmpDir
        } catch {
            throw
        }
    }
    <#

    .ForwardHelpTargetName Microsoft.PowerShell.Management\Set-Location
    .ForwardHelpCategory Cmdlet

    #>
}

function Rust-TargetDir {
    $location = $env:CARGO_BUILD_TARGET_DIR

    if (Test-Path -Path $location) {
        explorer $location
    } else {
        Write-Host "No Rust target folder exists for this directory"
    }
}

function _Clean-Rust {
    param(
        [string]$location
    )

    if (Test-Path -Path $location) {
        Remove-Item $location -Recurse -Force
        Write-Host "Cleaned out temp rust folder: $location"
    } else {
        Write-Host "Temp Rust folder already clean"
    }
}

function Clean-RustDir {
    _Clean-Rust "$env:CARGO_BUILD_TARGET_DIR\*"
}

function Clean-Rust {
    _Clean-Rust "$rustBaseDir\*"
}

# Set the initial value
$env:CARGO_BUILD_TARGET_DIR = _Get-RustTmpDir

To install it, create ~\Documents\PowerShell\Microsoft.PowerShell_profile.ps1 with contents
Import-Module 'C:\Full\Path\To\Documents\PowerShell\Set-Location.psm1' -DisableNameChecking

[Enselic]

Triage: Closing as duplicate of #48249 since the root cause is the same.