Be more selective on symbols exported from the kernel #97
Comments
|
I was looking at the visibility suport at G++ and it looks like it can accomplish that. http://gcc.gnu.org/wiki/Visibility I can try to work on this, but I would need to know which symbols we need to keep, which aren't important. Do you have any tip on how to have this list? Do you think this is the right way to go? |
|
On Fri, Jan 17, 2014 at 4:21 PM, efpiva [email protected] wrote:
It's not clear for from me from that description if the "compilation unit" If the former, this scheme won't work: we need symbols defined in one file If the later, then this may work beautifully. Can you verify that ? Also, don't worry too much about which symbols. New symbols appear everyday We can fine tune this later.
|
|
I think it's on the result ELF. I'm not sure if what I did on my small test I did a small Hello World splitted on two .cc files, the second with 2 without the hidden G++ flag.$ g++ -fPIC -c helloWorld.cc with the hidden G++ flag.$ g++ -fvisibility=hidden -fvisibility-inlines-hidden -fPIC -c On Fri, Jan 17, 2014 at 10:34 AM, Glauber Costa [email protected]:
Eduardo Fernandes Piva |
|
On Fri, Jan 17, 2014 at 2:34 PM, Glauber Costa [email protected]:
As far as I know, it is possible to tell the linker to treat all symbols,
I agree - we need to first try to see how this actually works on some
While I agree with this direction in the long term (I'm the one who created I think it would be interesting to try to think of a mechanism which can
We'll have an additional annoyance of deciding which non-Linux OSv APIs So I think that solving this issue will give us a lot of headaches, but in |
|
On Fri, Jan 17, 2014 at 5:40 PM, nyh [email protected] wrote:
As we have added those symbols, they were added in a fairly localized So I don't think it would be that bad. But of course, having something that |
|
Coming back to this issue, actually -fvisibility=hidden do what we need. The kernel compile successfully, so all symbols can see each other and anything can be linked with anything. The symbols are messed up just at the final linkage stage. So, during build, I got this error (as expected): And, the final ELF stripped size after this change is 10432656 bytes, before it was 11006096, so it has reduced by half Mb. Another benefit that the GCC wiki says is that, when symbols are supposed to be hidden, some optimization can happen on compile time, so we may have a better performance also. Off course this I can check after all symbols that should be market as visible are correct. One thing that I was thinking to do is to create a macro called OSV_API, that applies the visibility hing that GCC needs, and apply that macro to all needed API. One thing that would be nice is to enforce during build time that all OSV_API had some sort of documentation, like Doxygen, so we could extract (and publish) in a easy way all APIs that OSv supports (specially for those that are exclusive for OSv), so people don't need to actually read OSv source code to find them out. |
|
Here is what I propose to identify symbols that need to stay exported/public from kernel (somewhat inspired by trying to solve #1040) and use this code in core/elf.cc as a basis:
The step 3 might be pretty tedious but maybe we can ease the pain by compiling groups of source files (all directories as is) like musl with NO |
|
Based on this take from the GCC Visibility article: `To aid you converting old code to use the new system, GCC now supports also a #pragma GCC visibility command: extern void foo(int); it seems we should be able to put |
|
I think we don't have to play (just yet) with the visibility of all the OSV symbols, and could perhaps just surround the #include of boost headers with a pragma push visibility hidden. I can't check this now, I'm standing in line in Zurich airport. So good luck :-) |
|
Some critical code snippets intended to make it easier to understand my proposal:
// Our kernel already supplies the features of a bunch of traditional
// shared libraries:
static const auto supplied_modules = {
"libresolv.so.2",
"libc.so.6",
"libm.so.6",
#ifdef __x86_64__
"ld-linux-x86-64.so.2",
"libc.musl-x86_64.so.1",
// As noted in issue #1040 Boost version 1.69.0 and above is
// compiled with hidden visibility, so even if the kernel uses
// this library, it will not be visible for the application and
// it will need to load its own version of this library.
#if BOOST_VERSION < 106900
"libboost_system.so.1.55.0",
#endif
#endif /* __x86_64__ */
#ifdef __aarch64__
"ld-linux-aarch64.so.1",
#if BOOST_VERSION < 106900
"libboost_system-mt.so.1.55.0",
#endif
#endif /* __aarch64__ */
"libpthread.so.0",
"libdl.so.2",
"librt.so.1",
"libstdc++.so.6",
"libaio.so.1",
"libxenstore.so.3.0",
"libcrypt.so.1",
};
$(out)/kernel.elf: $(stage1_targets) arch/$(arch)/loader.ld $(out)/empty_bootfs.o $(loader_options_dep)
$(call quiet, $(LD) -o $@ --defsym=OSV_KERNEL_BASE=$(kernel_base) \
--defsym=OSV_KERNEL_VM_BASE=$(kernel_vm_base) --defsym=OSV_KERNEL_VM_SHIFT=$(kernel_vm_shift) \
-Bdynamic --export-dynamic --eh-frame-hdr --enable-new-dtags -L$(out)/arch/$(arch) \
$(^:%.ld=-T %.ld) \
--whole-archive \
$(libstdc++.a) $(libgcc_eh.a) \
$(boost-libs) \
--no-whole-archive $(libgcc.a), \
LINK kernel.elf)The initial attempt (or at least one of variants of it, see above) to hide most symbols envisioned compiling all sources with the flags The solution I am proposing below relies mostly on the version script mechanism (the linker flag So here is a fairly comprehensive recipe of how we can hide most symbols and expose only those needed.
One disadvantage of hiding most symbols is that that the backtrace printed to the console when OSV or app on it crashed would be less friendly possible missing most symbols names like in this example: vs We could provide a script (tool) that could easily recreate more readable backtrace: In most cases, the backtrace would still have at least the libc symbol name because that would still be present in |
This patch adds new build configuration option - conf_hide_symbols - that allows to build OSv kernel with all non-glibc symbols hidden when enabled (set to 1). By default the conf_hide_symbols is set to disabled so the kernel is still built with all symbols exported. In order to build kernel with most symbols hidden, one can use following command: ``` ./scripts/build image=native-example fs=rofs conf_hide_symbols=1 -j$(nproc) ``` The main idea behind the changes to the makefile below, is to compile all source files except the ones under musl/ and libc/ directories with the special compiler flags - '-fvisibility=hidden' and '-fvisibility-inlines-hidden' (C++ only) if conf_hide_symbols is enabled. This makes the symbols in all those relevant files as hidden except the ones annotated with OSV_***_API macros to expose them as public. On other hand, the musl sources come with its own symbol visibility mechanism where the symbols to be hidden are annotated with the 'hidden' macro and everything else is public. Therefore we do not need to compile the musl files with the visibility flags. Same goes for the files under libc/ that originate from musl. Lastly, the C++ files under libc/ that have been written from scratch to provide parts of glibc API (like libc/pthread.cc) are compiled with the compatibility flags. They are part of the libc_to_hide set. Also depending on conf_hide_symbols, the makefile uses different linker flags to link the standard C++ and others fully or not. Relatedly, when conf_hide_symbols is enabled, the OSv dynamic linker (core/elf.cc) does not advertise libstdc++.so.6 anymore. The symbol hiding mechanism enabled with conf_hide_symbols is powerful enough to hide most non-glibc symbols and leaves under 1,700 symbols exported including some vtable and typeinfo left C++ ones which is ~10% of the original number. The remaining C++ symbols will be removed from symbols table once we enable version script when linking in future patches. With conf_hide_symbols on, the resulting kernel-stripped.elf is ~ 5.1 MB in size, down from 6.7 MB, mainly due to libstdc++.a not linked fully. Once we enable linker garbage collection, the size should go down even more. Please note that the kernel with hidden symbols does not support building ZFS images as some of the symbols libzfs.so, zfs.so and zpool.so depend on are no longer visible. To fix this we will probably need to change how this apps are linked so they do not depend on those symbols exported by kernel. In addition around 35 unit tests cannot run on the kernel with most hidden symbols as they directly use OSv internal symbols. Finally most OSv apps and modules like httpserver.so rely on OSv specific API symbols and they will not work either. To address this, we will need to expose some of the OSv C++ API as C. It is not clear if this patch fully addresses the issue #97. We could however close it and open smaller ones to address remaining gaps. Refs #97 Signed-off-by: Waldemar Kozaczuk <[email protected]>
|
I believe we can close this issue as it has been addressed by this commit and the related ones that allow us to build the kernel with most symbols hidden according to the plan outlined in the previous comment. I have also added a new issue to address outstanding functionality. |
Right now, we export all symbols from the kernel, and they can all be used by shared objects.
This has a number of negative implications.
First is of course that applications can accidentally use kernel objects which we never intended to expose.
The second is that all these symbols (and C++ symbols have L-O-N-G names...) appear in the .dynstr and .dynsym sections of the kernel (loader-stripped.elf), which amounts to almost 1MB.
We should consider only exporting some of the symbols, not all of them.
The text was updated successfully, but these errors were encountered: