os: Remove/RemoveAll should remove read-only folders on Windows

[Nnnes]

What version of Go are you using (go version)?

go version go1.10.2 windows/amd64

Does this issue reproduce with the latest release?

Untested, but I don't see any changes that would have fixed it

What operating system and processor architecture are you using (go env)?

windows/amd64

What did you do?

package main

import (
	"os"
)

func main() {
	err := os.RemoveAll("./folder_with_icon")
	if err != nil {
		panic(err)
	}
}

folder_with_icon is an empty folder that has had a custom icon added via Windows Explorer's interface (Properties => Customize => Change Icon...).

What did you expect to see?

folder_with_icon should be deleted.

What did you see instead?

panic: remove ./folder_with_icon: Access is denied.

goroutine 1 [running]:
main.main()
        C:/[path]/test.go:10 +0x6a

Notes

This is similar to #9606.

Setting a custom folder icon on Windows sets the read-only attribute on the folder and creates a desktop.ini file with the system attribute inside the folder. Despite being read-only, the folder can be deleted both through File Explorer and by rd /s folder_with_icon by its owner, even if the owner is not an administrator account. The above Go program will not delete the folder, even if it is run with administrator privileges.

os.RemoveAll() removes the desktop.ini but does not delete the folder.

[ianlancetaylor]

By analogy with #9606 it seems that we should do this.

[iamoryanmoshe]

I'm working on it

[gopherbot]

Change https://golang.org/cl/125261 mentions this issue: os: remove read-only directories

[gopherbot]

Change https://golang.org/cl/203599 mentions this issue: os: Remove/RemoveAll removes read-only folders on Windows

[Nnnes]

This issue is only half fixed. As stated in the title, os.Remove() had the same problem as os.RemoveAll() for read-only directories. Using Go 1.14:

package main

import (
	"fmt"
	"io/ioutil"
	"os"
)

func main() {
	path := "./folder_with_icon"
	fmt.Println(count_files(path))
	err := os.Remove(path + "/desktop.ini")
	if err != nil {
		panic(err)
	}
	fmt.Println(count_files(path)) // works fine up to here
	err = os.Remove(path)
	if err != nil {
		panic(err)
	}
}

func count_files(path string) int {
	files, _ := ioutil.ReadDir(path)
	return len(files)
}

As before, folder_with_icon is a folder with a custom icon.

C:\[path]>26295.exe
1
0
panic: remove ./folder_with_icon: Access is denied.

goroutine 1 [running]:
main.main()
        C:/[path]/26295.go:19 +0x1d5

[bcmills]

The analogy to #9606 does not seem to hold: while Unix systems do allow read-only files to be removed from read-write directories, they do not generally allow files to be removed from within a read-only directory without first making the directory writable.

That subtlety is what led us to add the -modcacherw flag to cmd/go in 1.14 (see #31481).

[iwdgo]

The difference of behavior between Remove and RemoveAll is stated in the documentation

RemoveAll removes path and any children it contains. It removes everything it can

os.Remove will only attempt removal and return the error.
This behavior seems desirable even if support to read-only is discussed #35042

[bcmills]

@iwdgo, my point is that the proposed change here makes RemoveAll on Windows remove more than it does on Unix, but the justification given is consistency with Unix. That doesn't seem right.

If folks want to argue that the behavior on Unix should be expanded to be consistent with the documentation, that's fine by me, but then this issue would not be Windows-specific.

[iwdgo]

@bcmills The original issue was on Windows and the difference with Unix seems to start in Go1.12 as #29983 details.

The comment in _at code and what follows does not seem to consider an update of permissions in RemoveAll.

This test expects read-only directories to remain. It is skipped on Windows for the same reason. If this would change, an error would be returned when permission is insufficient although nil seems currently returned. I agree that RemoveAll on Unix could be doing more.

[bcmills]

@iwdgo, #29983 describes a situation in which files within a writable directory were not removed when that directory was itself within a read-only directory. That situation does not seem very closely related to the one described here.

[iwdgo]

@bcmills IMHO, RemoveAll on Unix should attempt a permission update but it was not in the scope of the issue as originally filed. Such a change for Unix would seem along the line of past comments.

[bcmills]

RemoveAll on Unix should attempt a permission update but it was not in the scope of the issue as originally filed. Such a change for Unix would seem along the line of past comments.

That seems fine to me, but that should be a separate issue — and implementing the behavior described in this issue should probably wait for a decision on the broader (cross-platform) RemoveAll behavior.

[odeke-em]

Moving to Go1.17.

[gopherbot]

Change https://go.dev/cl/448315 mentions this issue: os: remove read-only directories in Remove on Windows

[Nnnes]

For the sake of completeness, here's the official Microsoft documentation on read-only attributes on Windows folders:

FILE_ATTRIBUTE_READONLY 1 (0x00000001)
A file that is read-only. Applications can read the file, but cannot write to it or delete it. This attribute is not honored on directories. For more information, see You cannot view or change the Read-only or the System attributes of folders in Windows Server 2003, in Windows XP, in Windows Vista or in Windows 7.

(source: https://learn.microsoft.com/en-us/windows/win32/fileio/file-attribute-constants#file_attribute_readonly)

Excerpt from the linked article:

Note Unlike the Read-only attribute for a file, the Read-only attribute for a folder is typically ignored by Windows, Windows components and accessories, and other programs. For example, you can delete, rename, and change a folder with the Read-only attribute by using Windows Explorer.
The Read-only and System attributes is only used by Windows Explorer to determine whether the folder is a special folder...

[bcmills]

(CC @golang/windows)

[qmuntal]

Looking at what other frameworks do, and there are a few that discard the readonly attribute for either files and folders:

• microsoft/stl: https://github.com/microsoft/STL/blob/c53ac59abae552e50654943f7f80f23376a8039a/stl/src/filesystem.cpp#L578
• boostorg/filesystem: https://github.com/boostorg/filesystem/blob/6f174ba14369143d28ae72f4c697660c65cecae6/src/operations.cpp#L1895C13-L1895C145
• node/libuv: implement posix delete on windows, if supported libuv/libuv#4318
• zig: https://github.com/ziglang/zig/blob/dd1fc1cb8c2258256566fd0035deb6fbf700fc69/lib/std/os/windows.zig#L916