-
-
Notifications
You must be signed in to change notification settings - Fork 602
Google Summer of Code 2015
For this year, we will be accepting applications for both OSv itself, and Seastar, our newly released framework for writing asynchronous and non-blocking applications. If you want to know more about Seastar, take a look at seastar's website. Note that for development simplicity, much of seastar's functionality is available on Linux. If you select a Seastar project, chances are you can code most of your project without using OSv at all. You will, however, be required to demonstrate your final deliverable running on OSv.
Feel free to discuss any aspect of your idea with us, as early as you like.
For seastar-specific questions, you may be better served at [email protected]
To be accepted, an application will have to be detailed, and convince us that:
- You are qualified enough to accomplish the project
- Your implementation plan is feasible. Saying, for instance "I intend to implement a 9p driver for OSv", without saying how you plan to do it, and which steps are involved in doing so (even on a high level), is guaranteed to get your application rejected.
We are providing you below with some example ideas, but they are by no means extensive. You are welcome and encouraged to come up with your own ideas. In that case, however, we expect you to tell us, in addition to the items above:
- Why is my idea relevant for OSv/Seastar ?
Implement a 9p driver that would allow the host filesystem to be exposed within an OSv guest, allowing one to boot without an in-guest filesystem. That would allow us to simplify our deployments for local users, and speed up the boot process even further.
Details of a similar application for Linux can be found here
Skills Required: 9p protocol, C++
Difficulty Level: Medium
OSv is implementing the POSIX API layer on a need-to-do basis. The first task of porting a new runtime is identifying which calls are needed for go that are not implemented by OSv, and then of course, implementing them.
Golang presents some challenges on its own. First of all, the default golang compiler will only produce static binaries, that bypasses a lot of the glibc wrapper calls and go to OS syscalls directly. Key examples are the sbrk and clone syscalls. Those are syscalls we would rather not support if possible.
Changing go itself is a possible way to go forward.
Applicants interested in working on this project are required to have a very detailed plan, with a well-defined strategy on how to make that happen. This is harder than it sounds, so feel free to discuss it with us beforehand, but applications that does not say much more than "I will port Go" without touching the "how" will be promply refused.
Required Skills: Go internals, C, C++.
Difficulty: Very Hard.
Currently, OSv’s Xen support is restricted to HVM guests, which means hardware virtualization extensions are needed. However, that is not Xen's main mode of operation. The student tackling this project will need to implement all the relevant hypercalls that will enable OSv to boot on top of Xen, and also make sure his implementation is efficient
Skills Required: Xen PV / PVH protocol, C++, general paravirtualization techniques
Difficulty: Hard.
Due to initial technical constraints, OSv does not have the ability to run an ELF binary. We have our own ELF loader and from there, we execute a shared object (.so). While this works, this means native compiled applications that wants to be ran on OSv need to be recompiled as a shared library. It is technically possible to lift that limitation, by: Loading a full ELF file and reading its list of mappings If the kernel mapping overlaps the binary (it probably does), remap the kernel somewhere else, where the application left a hole unfilled Audit all the places in which the kernel symbols are used directly: those places will access the old addresses, and will need to be fixed.
Difficulty: Easy to Medium
Required skills: POSIX, C++
Non Volatile Memory Express (NVMe) is a new storage technology for accessing high performance solid-state drives (SSDs). NVMe features include excellent virtualization support; it is also already supported by QEMU.
As such, NVMe is a great candidate for providing high throughput I/O to OSv instances. The student will implement an OSv driver for NVMe and test it on QEMU/KVM.
Required skills: device drivers, C++.
Difficulty: Medium
(for extreme rapid boots on VMs that do not need much storage -- Don)
novm is a new legacy-free, type 2 hypervisor for Linux, using KVM. novm exposes a flexible filesystem interface instead of virtual block devices, to make it easier to manage independent software and data bundles and combine them into a single virtual machine instance.
https://github.com/google/novm
Required Skills: Low-level virtualization, x86, C++
Difficulty: Hard.
Hyper-V is a VMM provided from Microsoft. It’s requires non-standard para-virtual drivers to run GuestOS, called “Virtual Service Client”(VSC). NetVSC is for networking, StorVSC is for block device. VSCs are communicate through VMBus, which is similar to Xen’s XenBus. Hyper-V Architecture details are here: https://msdn.microsoft.com/en-us/library/cc768520%28v=bts.10%29.aspx
Note that Open Source Hyper-V drivers already exists for other Operating Systems, and can be used as a basis. In particular, the BSD drivers are license compatible and could be used as a starting point.
Required Skills: Device Drivers, the Hyper-V protocol, C++
Difficulty: Easy.
OSv is implementing the POSIX API layer on a need-to-do basis. The first task of porting a new runtime is identifying which calls are needed for go that are not implemented by OSv, and then of course, implementing them.
Currently, The OSv kernel is always statically linked with all features even if some of them are actually never used (ex: vmware drivers never used on kvm).
We would like a kernel build configuration system to customize build for each VM image.
The target of customization is not just select drivers, we need to enable/disable network stack and ZFS, select VM image format, enable/disable REST server and CLI, specify application name, specify command line(optional).
Bonus points, would include runtime disabling of code. For example, an image that is built for all hypervisors can dispose of the memory it uses for Xen support once it detects it booted on VMware.
We can refer to Linux kernel’s building system (we don’t have to implement menuconfig but it’s good to refer its config file), but cannot port their code since it should GPL licensed.
Required Skills: C++, some script language (if build-time configuration), ELF format (if runtime configuration)
Difficulty: Easy
Currently we only supported single queue virtio-net, implement multiqueue support on it and optimize network stack for multiqueue devices.
Required Skills: C++, virtio protocol
Difficulty: Easy to Medium
Implement SR-IOV Virtual Function driver on OSv. ixgbevf is good choice since it’s most common.
Required Skills: C++, SR-IOV, device drivers.
Difficulty: Easy to Medium
Seastar is a novel framework, based on future and promises, asynchronous programming, and which heavily uses C++14. If you submit an application for Seastar, you will have to demonstrate that you are familiar with future and promises and assynchronous programming. You don't have to demonstrate expert levels, so if you are interested in the general topics but doesn't quite grasp the fundamentals, don't worry: feel free to read about it, go through the code, and drop questions at the mailing list. But we expect all of that to happen before the selection deadline.
Currently the Seastar networking stack supports IPv4 only. The project will extend it to support IPv6 packets and IPv6 configuration protocols.
Required skills: ipv4, ipv6, general networking, C++14.
Difficulty: Medium
Seastar relies on a heavily distributed, shared nothing architecture. However, since there is only one NIC in a regular system, that creates a point of contention. If multi-queue NICs are involved, this is not an issue. But we can't always rely on the presence of multi-queue.
For the general case, one could add support for multiple NICs in the same seastar application. The tricky part of that, is that since the client application is not aware of the existance of all the NICs, routing and forwarding techniques have to applied so that this happens transparently.
Required skills: networking, device drivers, C++14
Difficulty: Medium
Non Volatile Memory Express (NVMe) is a new storage technology for accessing high performance solid-state drives (SSDs). NVMe features include strong support for kernel bypass technologies and excellent multi-core support.
Seastar is a user-space server application framework that achieves high performance via a share-nothing SMP model and using kernel bypass for I/O.
As such, the two technologies are well suited for each other. The student will develop an NVMe driver for seastar, which can be done via QEMU emulation. Also, measure its performance on hardware that we will provide (remotely).
Required skills: NVMe, device drivers, C++14.
Difficulty: Medium
We would like to have applications that demonstrate the potential for performance and scalability of the seastar framework.
We currently have a version of memcached and a simple httpd that serves requests from memory. Both perform better than their standard counterparts.
Possible ideas include: extending httpd to be a fully-featured http server, ... , or coming up with an application on your own. A streaming server, a proxy, or anything that is expected to handle an abnormally high number of transactions.
Please be as descriptive as possible. Your proposal should include:
Difficulty Level: application-dependent
Skills required C++14 + other application-dependent.