Allow running a single unmodified regular (non-PIE) Linux executable #190
Comments
|
@nyh |
|
It is possible (and very easy) to know if a dynamically-linked executable
|
|
Sound like we need a script which given a binary code of a program, |
|
On Thu, Feb 13, 2014 at 8:38 PM, Tzach Livyatan [email protected]:
IMHO, the way to do it is to just let the program run and abort on fork().
|
|
Good idea. I'll do it right after I figure out a way to square a circle :-)
|
Doesn't really affect the overall point, but you can also have false negatives, if someone forks directly via syscall without using the |
|
which would be the case for all golang executables (they use the libc only for the resolver which is configurable, and AFAIK for getpwuid_r) (sorry for the noise, i've no idea why i got notified by github for this thread, but anyway if i already got the message i thought i could also write back about it). |
|
@copumpkin you can still find that out, by scanning the assembly instructions. This heuristic should work fine for "normal" programs without JIT or self modified code. |
|
@elazarl determining actual executable code to disassemble in a compiled binary is pretty hard to approximate, and generally impossible to get right :) either way, this discussion is entering the realm of impracticality |
|
objdump -d ELF|grep syscall is not always correct, but would work for many On Mon, Aug 17, 2015 at 9:40 PM, Daniel Peebles [email protected]
|
|
@elazarl I don't think we're disagreeing 😄 |
wkozaczuk
changed the title
|
It looks like most of the discussion is about fork. I understand that fork is not supported on OSv and we have a stub that warns and returns -1 if called. Why is it important to detect if non-PIE uses fork? What am I missing. I think the most important issue is that non-PIE object would collide with OSv kernel in memory. If that is true, wouldn't a solution be to map kernel so somewhere very high? More specifically we would still load it at 0x200000 in physical memory but then instead of linearly mapping first 1GB 1:1 we would map the kernel part (OSV_KERNEL_BASE) to be the last 10 MB of virtual memory. Would that work in most cases, assuming most executables would typically be in way lower? Or I am missing some even more fundamental problem. |
|
@wkozaczuk you're not missing anything - this issue was hijacked ;-) It was supposed to be about being able to run a regular (not PIE) executable, but then people started to wonder how we'll ever know if this executable can be run because maybe it uses fork() or other things we never implemented. But that's unrelated to the original issue... |
|
I spent some time researching 64-bit part of Linux kernel boot process and here is what I found:
So the question is: could we do something similar where our loader.elf would still be loaded at 0x200000 in physical RAM but start32 (and new vmlinux_entry64) load slightly different version of ident_pt_l4 paging tables where the kernel code portion (elf_start-elf_end) would map to the very high part of virtual memory (higher than ffffffff81000000). I think the higher half of virtual memory in Linux is reserved by kernel so I think no Linux executable would ever collide with OSv kernel (if that is the main problem). Also OSV_KERNEL_BASE would need to match new base (possibly ffffffff80200000 = ffffffff80000000 + 200000 or higher) to make gcc build loader.elf with higher addresses. Is this as simple as this? Or would there be some bigger ripple effects of changing OSV_KERNEL_BASE to much higher address (large mcmodel=large ?, as described here) |
|
I have experimented a bit to see if my theory about moving kernel away is right (and what I have found is not necessarily a proof but just a good indication) and I have managed to run simple non-PIE executable: #include <stdio.h>
int main(int argc, char* argv[]) {
printf("Hello!\n");
}built like so: with following changes to OSv: and yielded this output: Obviously my patch assumes the ELF executable loads somewhere below 32MB (updated KERNEL_BASE) and I have a sense it has some other holes. I even tried more complex example - python3x where I replaced python executable by pointing to python3.6 one on my host (Ubuntu) that happened to be ELF executable (unlike 2.6 which is a pie). This time the app actually hangs: because probably there is a hole in my patch that sets _base of usr/lib/libexpat.so.1 to set to 0x0 (does not seem to be right). What is interesting per my patch the kernel ELF itself _base is set to 0 as well which I thought was wrong so I tried another patch which I thought was more correct (see elf.cc changes only): and the same hello EXEC that worked with previous patch hang this time: I wonder if there is some bug with setting a _base for kernel ELF. I remember we had an issue related to this and had to create libvdso.so. |
|
With a slightly better patch: I got python3 as non-pie executable running: |
|
Java 11 worked as well: with the same problem of hanging in the and as with a pie as I reported on emailing list. |
|
@nyh So it seems there is no fundamental problem in supporting non-PIE executables except to make sure they do not collide with kernel. What if we make the changes to dynamic loader to accommodate 0-based offset as my research shows. And then during build process we verify there is no collision between executable segment addresses in memory and OSv kernel and calculate value of KERNEL_BASE to avoid collision and print to user what he needs to change KERNEL_BASE to in Makefile. We probably should do the same check for collision in runtime. Also I think in most cases the default offset of the executable is typically between 0x400000 and 0x500000 so users would not waste much memory. The bonus would be to finally fix arch_memory_setup to add memore below kernel elf_start to avoid any such waste, In future if we manage to somehow make kernel be mapped very high in virtual memory this collision would never happen. |
|
With the latest 2 patches I have recently sent:
OSv fundamentally supports running single non-PIE executable as long as it does not collide with kernel. The 1st issue enhances OSv dynamic loader and the 2nd one makes OSv utilize physical memory below the kernel (typically 2M). The 2nd one seems to be unrelated however until we address #1043, this allows us to move kernel_base to the address high enough for at least some executables (tiny java or node.JS) to not collide with kernel and not lose any memory. We could even enhance the makefile/build script to calculate correct kernel_base for given executable and relink the kernel accordingly. Obviously moving kernel higher in virtual memory is the most desired solution to this collision problem. |
This patch provides necessary changes to OSv dynamic linker to allow running non-PIEs (= Position Dependant Executables) as long as they do not collide with kernel. Please note this patch does not fully address issue cloudius-systems#190 though it provides necessary groundwork to fully address it in future. To truly support running unmodified arbitrary non-PIEs (that typically load at 0x400000 = 4th MB) we need to modify OSv to load kernel way higher that 0x200000. There are 2 ways to run non-PIEs with OSv with this patch: 1) Link it by forcing text segment address so it does not collide with kernel using -Wl,-Ttext-segment,0x?????? option. 2) Change kernel_base in OSv makefile to a higher address that makes it not collide with the executable (for example 0xa00000); most non-PIEs by default load at 0x400000 Signed-off-by: Waldemar Kozaczuk <[email protected]>
This patch provides all necessary changes to move OSv kernel by 1 GiB higher in virtual memory space to start at 0x40200000. Most changes involve adding or subtracting 0x40000000 (OSV_KERNEL_VM_SHIFT) in all relevant places. Please note that the kernel is still loaded at 2MiB in physical memory. The motivation for this patch is to make as much space as possible (or just enough) in virtual memory to allow running unmodified Linux non-PIE executables (issue #190). Even though due to the advancement of ASLR more and more applications are PIEs (Position Independent Executables) which are pretty well supported by OSv, there are still many non-PIEs (Position Dependent Executables) that are out. The most prominent one is actualy JVM whose most distributions come with tiny (~20K) bootstrap java non-PIE executable. There are many other examples where small non-PIE executable loads other shared libraries. As issue #1043 explains there are at least 3 possible solutions and this patch implements the 3rd last one described there. Please note that in future with little effort we could provide slightly beter scheme for OSV_KERNEL_VM_SHIFT that would allow us to place the kernel even higher at the end of the 2GiB limit (small memory model) and thus support virtually any non-PIE built using small memory model. Due to its impact this patch has been tested on following hypervisors: - QEMU without KVM - QEMU with KVM - Firecracker - VirtualBox 6 - VMware Player - XEN on EC2 - XEN locally in HVM mode Fixes #1043 Signed-off-by: Waldemar Kozaczuk <[email protected]> Message-Id: <[email protected]>
OSv currently can only run position-independent shared objects.
This means we usually can't run a random Linux application without recompiling it (with -fPIC -shared), unless we're lucky and the application is already mostly a .so (like is the case in the JVM).
I want to allow running a single position-dependent executable, while moving the rest of OSv to where this executable allows us.
This will probably come with some strings attached: Obviously, only one such executable can be run at a time. Also, we may need to know at boot time about this executable. But I think this should be doable, and will make it really easy to create OSv images out of existing Linux software (e.g., Fedora RPMs) without any recompilation of anything.
The text was updated successfully, but these errors were encountered: