SPINE - A Safe Programmable and Integrated Network Environment

Marc E. Fiuczynski Brian N. Bershad
{mef,bershad}@cs.washington.edu
Department of Computer Science and Engineering
University of Washington, Seattle, WA 98195

The goal of our research is to provide a safe environment for applications to directly compute on the network interface (NI). Several network interfaces, for example the Myrinet, as well as devices for the new I2O architecture, provide the infrastructure necessary to compute on the NI itself. We intend to apply the SPIN technology (use of a safe language, extensible interfaces, and resource management) to the NI, which allows applications to customize their interaction with the network. The motivation is to migrate application-specific functionality directly to the NI, thereby reducing I/O related data and control transfers to the host system.

Applications we believe will benefit from this architecture range from distributed memory management systems (e.g., the Global Memory Systems, Network RAM, DSM), cluster based storage management (e.g., Swarm, Petal), peer-to-peer DMA transfers (e.g., network to disk or framebuffer), to support for packet filtering (e.g., Lazy Receive Processing, Scout Paths).

Our strategy is to provide an extensible runtime environment, called SPINE, for programmable network interface cards. SPINE extends the fundamental ideas in SPIN - type safe code downloaded into a trusted execution environment - to the network interface. Specifically, SPINE has four properties that are key to the construction of application-specific solutions:

1.
Performance. The system allows a SPINE extension to perform better than the general solution provided by the operating system or firmware vendor. SPINE extensions run in the same address space as the firmware, which places it close to the network device and doesn't require the host CPU to be interrupted.
2.
Safety. The use of a SPINE extension does not compromise the safety of other applications, the firmware, or the host operating system. Extensions are isolated from the system and one another through the use of a type safe language and inexpensive access control mechanisms.
3.
Openness. An application, regardless of its privilege level, may define a SPINE extension.
4.
Runtime adaptation. Applications may add extensions to the NI at any point during the system's execution without requiring super-user privileges or a system reboot. This property allows extensions to be safely loaded and unloaded into a running system so that they can come and go with their corresponding applications.

We do not expect the processor architecture of programmable network interfaces to provide support for multiple address spaces. A safe language enables extensions and SPINE to run in the context of a single address space. Our design provides extensible interfaces to the underlying hardware and allows extensions to provide their own extensible services. The SPINE runtime will be a small Modula-3 runtime that includes support for threads, synchronization, and garbage collection. The system will provide a set of core interfaces to low-level services (such as network, DMA, and registers).

SPINE will not depend explicitly on a host running the SPIN operating system. The SPINE runtime can be split across the network adapter and the host. The host resident runtime will provide safe access to the host operating system (e.g. SPIN as a loadable device driver to Linux or Windows NT), which most likely consists of support for VM operations and network data delivery.

SPINE will export safe external, internal, I/O, and ``user-level visible'' interfaces. External interfaces are used by kernel resident agents (such as SPIN extensions, Linux modules, or dynamically linked drivers). Internal interfaces are used by extensions that are loaded onto the NI, which consists of the standard Modula-3 interface, ``plain old kernel services'', and safe access to the underlying hardware (e.g., DMA engines). For the I/O interface we will implement an I2O compliant interface which eases portability to other systems and enables an apples-to-apples performance comparison. We will also develop an I2O interface accessible to SPINE extensions, thereby enabling them to setup peer-to-peer communication with other I2O devices (e.g., disks). Finally, SPINE will only export a bootstrap ``user-level visible'' interface to enable the loading of extensions code to the NI. Using this mechanism, SPINE extensions can define their own ``user-level visible'' interface (e.g., Generic Active Messages, Hamlyn, U-NET, MPI).

A prototype implementation runs on the Myrinet interface and has a base image size of less than 100Kbyte. We are currently implementing the I2O message layer for the Myrinet interface. More information about SPINE can be found at the following URL: http://www.cs.washington.edu/homes/mef/spine