Data-centric serverless cloud architecture
View/ Open
Date
08/06/2022Author
Ustiugov, Dmitrii
Metadata
Abstract
Serverless has become a new dominant cloud architecture thanks to its high scalability
and flexible, pay-as-you-go billing model. In serverless, developers compose their
cloud services as a set of functions while providers take responsibility for scaling each
function’s resources according to traffic changes. Hence, the provider needs to timely
spawn, or tear down, function instances (i.e., HTTP servers with user-provider handles),
which cannot hold state across function invocations.
Performance of a modern serverless cloud is bound by data movement. Serverless
architecture separates compute resources and data management to allow function instances
to run on any node in a cloud datacenter. This flexibility comes at the cost of
the necessity to move function initialization state across the entire datacenter when
spawning new instances on demand. Furthermore, to facilitate scaling, cloud providers
restrict the serverless programming model to stateless functions (which cannot hold or
share state across different functions), which lack efficient support for cross-function
communication.
This thesis consists of four following research contributions that pave the way for
a data-centric serverless cloud architecture. First, we introduce STeLLAR, an opensource
serverless benchmarking framework, which enables an accurate performance
characterization of serverless deployments. Using STeLLAR, we study three leading
serverless clouds and identify that all of them follow the same conceptual architecture
that comprises three essential subsystems, namely the worker fleet, the scheduler, and
the storage. Our analysis quantifies the aspect of the data movement problem that is
related to moving state from the storage to workers when spawning function instances
(“cold-start” delays). Also, we study two state-of-the-art production methods of crossfunction
communication that involve either the storage or the scheduler subsystems, if
the data is transmitted as part of invocation HTTP requests (i.e., inline).
Second, we introduce vHive, an open-source ecosystem for serverless benchmarking
and experimentation, with the goal of enabling researchers to study and innovate across
the entire serverless stack. In contrast to the incomplete academic prototypes and
proprietary infrastructure of the leading commercial clouds, vHive is representative of
the leading clouds and comprises only fully open-source production-grade components,
such as Kubernetes orchestrator and AWS Firecracker hypervisor technologies. To
demonstrate vHive’s utility, we analyze the cold-start delays, revealing that the high
cold-start latency of function instances is attributable to frequent page faults as the
function’s state is brought from disk into guest memory one page at a time. Our analysis
further reveals that serverless functions operate over stable working sets - even across
function invocations.
Third, to reduce the cold-start delays of serverless functions, we introduce a novel
snapshotting mechanism that records and prefetches their memory working sets. This
mechanism, called REAP, is implemented in userspace and consists of two phases.
During the first invocation of a function, all accessed memory pages are recorded and
their contents are stored compactly as a part of the function snapshot. Starting from the
second cold invocation, the contents of the recorded pages are retrieved from storage
and installed in the guest memory before the new function instance starts to process the
invocation, allowing to avoid the majority of page faults, hence significantly accelerating
the function’s cold starts.
Finally, to accelerate the cross-function data communication, we propose Expedited
Data Transfers (XDT), an API-preserving high-performance data communication
method for serverless. In production clouds, function transmit intermediate data to other
functions either inline or through a third-party storage service. The former approach is
restricted to small transfer sizes, the latter supports arbitrary transfers but suffers from
performance and cost overheads. XDT enables direct function-to-function transfers
in a way that is fully compatible with the existing autoscaling infrastructure. With
XDT, a trusted component of the sender function buffers the payload in its memory
and sends a secure reference to the receiver, which is picked by the load balancer and
autoscaler based on the current load. Using the reference, the receiver instance pulls the
transmitted data directly from sender’s memory, obviating the need for intermediary
storage.