Considerations for Go 1.19 update

[0x1a8510f2]

Reading through the Go 1.19rc1 release notes, I've noticed some things which may have an impact on Dendrite and may be worth considering when Go 1.19+ becomes the minimum version supported:

• The file descriptor limit increase code added in Attempt to raise the file descriptor limit at startup #2527 can be removed as the Go runtime will automatically increase the file descriptor limit in binaries importing the os package:

On Unix operating systems, Go programs that import package os now automatically increase the open file limit (RLIMIT_NOFILE) to the maximum allowed value; that is, they change the soft limit to match the hard limit. This corrects artificially low limits set on some systems for compatibility with very old C programs using the select system call. Go programs are not helped by that limit, and instead even simple programs like gofmt often ran out of file descriptors on such systems when processing many files in parallel. One impact of this change is that Go programs that in turn execute very old C programs in child processes may run those programs with too high a limit. This can be corrected by setting the hard limit before invoking the Go program.

• Files with a unix build constraint, such as the one with the code mentioned above, can be simplified:

The build constraint unix is now recognized in //go:build lines. The constraint is satisfied if the target operating system, also known as GOOS, is a Unix or Unix-like system. For the 1.19 release it is satisfied if GOOS is one of aix, android, darwin, dragonfly, freebsd, hurd, illumos, ios, linux, netbsd, openbsd, or solaris. In future releases the unix constraint may match additional newly supported operating systems.

• As per arewep2pyet, Dendrite must

Ensure memory usage of embedded instances is bounded e.g cache sizes.

This means that the new soft memory limit may come in handy:

The runtime now includes support for a soft memory limit. This memory limit includes the Go heap and all other memory managed by the runtime, and excludes external memory sources such as mappings of the binary itself, memory managed in other languages, and memory held by the operating system on behalf of the Go program. This limit may be managed via runtime/debug.SetMemoryLimit or the equivalent GOMEMLIMIT environment variable. The limit works in conjunction with runtime/debug.SetGCPercent / GOGC, and will be respected even if GOGC=off, allowing Go programs to always make maximal use of their memory limit, improving resource efficiency in some cases. See the GC guide for a detailed guide explaining the soft memory limit in more detail, as well as a variety of common use-cases and scenarios. Please note that small memory limits, on the order of tens of megabytes or less, are less likely to be respected due to external latency factors, such as OS scheduling. See issue 52433 for more details. Larger memory limits, on the order of hundreds of megabytes or more, are stable and production-ready.