Composable and streamable Play apps

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

at

Composable and streamable Play apps

About me

Part of the Service Infrastructure team, which is bringing
the Play Framework to LinkedIn engineers.

At LinkedIn, we’ve been running Play apps
in production for over a year

More than 60 apps on Play now, including:

Channels

Recruiter Mobile

Jobs

Polls in Groups

Play Labs (e.g. InDemand)

Many internal tools (e.g. REST search)

Plus many backends services that don’t
have sexy screenshots

We love Play… but we’ve had a
couple tiny problems:

We love Play… but we’ve had a
couple tiny problems:
1. Unmanageable complexity

We love Play… but we’ve had a
couple tiny problems:
1. Unmanageable complexity
2. Terrible performance

It’s probably not what you think.

This talk is about a couple techniques to
deal with complexity and performance
problems.

These techniques are experimental.

But fun.

Enjoy!

You can find the code from this presentation at:
https://github.com/brikis98/ping-play

Outline
1. Complexity
2. Composition
3. HTTP
4. Performance
5. Enumerators
6. Streaming

Outline
1. Complexity
2. Composition
3. HTTP
4. Performance
5. Enumerators
6. Streaming

Web pages can get complex

For example, consider the LinkedIn home page

Almost every part of the page is dynamic, highly
customized for the user, and interactive.

Trying to cram this all into one controller and
one template is completely unmaintainable

“Managing complexity is the most important
technical topic in software development.”
Steve McConnell, Code Complete

Abstraction and Composition

“Abstraction is the ability to engage with a
concept while safely ignoring some of its details.”
Steve McConnell, Code Com...

Abstraction allows you to take a simpler view of a
complex concept

Composition: assemble complex behavior by
combining simpler behavior

Abstraction: structure your app so you
can focus on one part while safely
ignoring the rest

Abstraction: structure your app so you
can focus on one part while safely
ignoring the rest

Composition: structure your a...

Outline
1. Complexity
2. Composition
3. HTTP
4. Performance
5. Enumerators
6. Streaming

We need:
1. Data: WVYP count, search count
2. Template: to render the HTML
3. Controller: to tie it all together

Backend
Server
Frontend
Server
Internet

Load
Balancer

Backend
Server

Frontend
Server

Data
Store

Backend
Server

Front...

For this talk, we’re going to simulate service
calls by calling a mock endpoint

object Mock extends Controller {
def mock(serviceName: String) = Action.async {
serviceName match {
case "wvyp" => respond...

GET

/mock/:serviceName

controllers.Mock.mock(serviceName: String)

conf/routes

The routes entry for the mock endpoint

object ServiceClient {
def makeServiceCall(serviceName: String): Future[String] = {
WS.url(s"http://localhost:9000/mock/$s...

@(wvypCount: Int, searchCount: Int)
<html>
<head>
<link rel="stylesheet" href="/assets/stylesheets/wvyp.css"/>
</head>
<bo...

object WVYP extends Controller {
def index = Action.async {
val wvypCountFuture = ServiceClient.makeServiceCall("wvyp")
va...

object WVYP extends Controller {
def index = Action.async {
val wvypCountFuture = ServiceClient.makeServiceCall("wvyp")
va...

GET

/mock/:serviceName

controllers.Mock.mock(serviceName: String)

GET

/wvyp

controllers.WVYP.index

conf/routes

Add ...

The result

We now have one small module

Now, imagine we also have another standalone module
called “Who’s Viewed Your Updates” (WVYU)

GET

/mock/:serviceName

controllers.Mock.mock(serviceName: String)

GET

/wvyp

controllers.WVYP.index

GET

/wvyu

contr...

How can we combine the two modules?

trait Action[A] extends EssentialAction {
def apply(request: Request[A]): Future[SimpleResult]
}

play/api/mvc/Action.scal...

Request[A] => Future[SimpleResult]

The actions in each standalone module are just functions
from Request to Result. Funct...

@(wvypCount: Int, searchCount: Int)

<html>
<head>
<link rel="stylesheet" href="/assets/stylesheets/wvyp.css"/>
</head>
<b...

def index(embed: Boolean) = Action.async {
val wvypCountFuture = ServiceClient.makeServiceCall("wvyp")
val searchCountFutu...

GET

/mock/:serviceName

controllers.Mock.mock(serviceName: String)

GET

/wvyp

controllers.WVYP.index(embed: Boolean ?= ...

object Pagelet {

def readBody(result: SimpleResult)(implicit codec: Codec): Future[Html] = {
result.body.run(Iteratee.con...

object Aggregator extends Controller {
def index = Action.async { request =>
val wvypFuture = Wvyp.index(embed = true)(req...

object Aggregator extends Controller {
def index = Action.async { request =>
val wvypFuture = Wvyp.index(embed = true)(req...

object Aggregator extends Controller {
def index = Action.async { request =>
val wvypFuture = Wvyp.index(embed = true)(req...

GET
@(wvypBody:/mock/:serviceName controllers.Mock.mock(serviceName: String)
Html, wvyuBody: Html)
GET

/wvyp

controllers...

The result. We’ve composed two Play modules!

Wins
1.

Abstraction: we can focus on just one small part of the page while safely
ignoring all the rest.

2.

Composition...

Caveats
1.

Standalone modules may be inefficient in terms of duplicated service calls.
However, de-duping is straightforw...

Outline
1. Complexity
2. Composition
3. HTTP
4. Performance
5. Enumerators
6. Streaming

The composition code glosses over a few details

Some questions:
1. How do we set cookies?
2. How do we handle errors?
3. How do we aggregate static content?

It turns out HTTP is an elegant way to answer
these questions.

object WVYP extends Controller {
def index(embed: Boolean) = Action {
// [snip]
Ok(views.html.wvyp.wvyp(wvypCount.toInt, s...

object Aggregator extends Controller {
def index = Action.async { request =>
val wvypFuture = Wvyp.index(embed = true)(req...

object Pagelet {

def mergeCookies(results: SimpleResult*): Seq[Cookie] = {
results.flatMap { result =>
result.header.head...

def index = Action.async { request =>
val wvypFuture = Wvyp.index(embed = true)(request)
val wvyuFuture = Wvyu.index(embed...

We can see the aggregate endpoint is now setting cookies

HTTP headers are a good way to handle error
cases too.

A quick review of HTTP status codes

For example, if a module has no content, return a 204

Invalid module request? Return 404.

Plus other important status codes

def index = Action.async { request =>
val wvypFuture = Wvyp.index(embed = true)(request)
for {
wvyp <- wvypFuture
wvypBody...

CSS and JS dependencies can be set as a
header and aggregated too.

object StaticContent {
val cssHeaderName = "X-CSS"
val jsHeaderName = "X-JS"

def asHeaders(css: Seq[String], js: Seq[Stri...

def index(embed: Boolean) = Action {
// [...] all other code is the same as before [...]

val css = Vector("/assets/styles...

object StaticContent {
def mergeCssHeaders(results: SimpleResult*): Seq[String] = mergeHeaderValues(cssHeaderName, results...

def index = Action.async { request =>
val wvypFuture = Wvyp.index(embed = true)(request)
val wvyuFuture = Wvyu.index(embed...

Wins
1.

Modules are truly standalone

2.

Can dynamically compose modules using Play’s router

3.

Can compose modules fr...

Caveats
1.

Managing static content in a controller, instead of a view, feels clunky.

Outline
1. Complexity
2. Composition
3. HTTP
4. Performance
5. Enumerators
6. Streaming

So far, we’ve been using a mock endpoint to
simulate remote service calls

What happens if one of the remote calls is slow?

object Mock extends Controller {
def mock(serviceName: String) = Action.async {
serviceName match {
case "wvyp" => respond...

Time to first byte is 2 seconds! Everything has to wait for
the one slow service, including static content.

Is there a way to start sending the response
before all the data is available?

Facebook solves this problem with BigPipe
https://www.facebook.com/note.php?note_id=389414033919

Can we build BigPipe with Play?

(Well, yea, or I wouldn’t have made this talk)

We can use Enumerators!

Outline
1. Complexity
2. Composition
3. HTTP
4. Performance
5. Enumerators
6. Streaming

Enumerators are part of the Play Iteratees library.

Quick Review
1.

An Enumerator is a Producer. It pumps out chunks of data.

2.

An Iteratee is a Consumer. It reactively c...

Producer, consumer, and adapter

These are all composable abstractions for
working with streams of data

Let’s look at some examples

object EnumeratorExample extends Controller {

def index = Action {
Ok.chunked(/* We need an Enumerator here */)
}
}

app/...

object EnumeratorExample extends Controller {

def index = Action {
Ok.chunked(Enumerator("Created", " using", " Enumerato...

Basic “Hello World” example using Enumerator.apply.

object EnumeratorExample extends Controller {

def index = Action {
Ok.chunked(Enumerator.repeatM(Promise.timeout("Hellon"...

The word “Hello” is pumped out every 500ms. Note: when
testing streaming, use “curl -N” so curl doesn’t buffer.

def index = Action {
val helloEnumerator = Enumerator("hello ")
val goodbyeEnumerator = Enumerator("goodbyenn")

val hello...

With andThen, we see all the data from the first enumerator
and then all the data from the second one

def index = Action {
val helloEnumerator = Enumerator.repeatM(Promise.timeout("Hellon", 500))
val goodbyeEnumerator = Enum...

With interleave, data can come in any order. Above, we see
“Hello” every 500ms and “Goodbye” every 1000ms.

Let’s use Enumerators to stream the results
of our remote service calls

object WVYPEnumerator extends Controller {
def index = Action {
val wvypCountFuture = ServiceClient.makeServiceCall("wvyp"...

object WVYPEnumerator extends Controller {
def index = Action {
val wvypCountFuture = ServiceClient.makeServiceCall("wvyp"...

object WVYPEnumerator extends Controller {
def index = Action {
val wvypCountFuture = ServiceClient.makeServiceCall("wvyp"...

GET
@(wvypBody:/mock/:serviceName controllers.Mock.mock(serviceName: String)
Html, wvyuBody: Html)
GET

/wvyp

controllers...

Almost immediately, we see “56”, from the wvyp service.

2 seconds later, we see “10”, from the search service.

We’re streaming!

However, we’re only streaming plain text.
How do we stream a template?

Outline
1. Complexity
2. Composition
3. HTTP
4. Performance
5. Enumerators
6. Streaming

play.Keys.templatesTypes ++= Map("stream" -> "ui.HtmlStreamFormat")
play.Keys.templatesImport ++= Vector("ui.HtmlStream", ...

The new template type must define an
“Appendable” and a “Format” for it

trait Appendable[T] {
def +=(other: T): T
}

play/api/templates/Templates.scala

The Appendable trait is simple: it’s anyt...

class Html(buffer: StringBuilder) extends Appendable[Html](buffer) {
def +=(other: Html) = {
buffer.append(other.buffer)
t...

case class HtmlStream(enumerator: Enumerator[Html]) extends Appendable[HtmlStream] {
def +=(other: HtmlStream): HtmlStream...

$object HtmlStream { def interleave(streams: HtmlStream*): HtmlStream = { HtmlStream(Enumerator.interleave(streams.map(_.e...$

object HtmlStream {

def apply(text: String): HtmlStream = {
apply(Html(text))
}

def apply(html: Html): HtmlStream = {
Ht...

trait Format[T <: Appendable[T]] {
type Appendable = T

def raw(text: String): T

def escape(text: String): T
}

play/api/...

object HtmlStreamFormat extends Format[HtmlStream] {

def raw(text: String): HtmlStream = {
HtmlStream(text)
}

def escape...

object HtmlStreamImplicits {

// Implicit conversion so HtmlStream can be passed directly to Ok.feed and Ok.chunked
implic...

@(body:ui.HtmlStream)
<html>
<head>
<link rel="stylesheet" href="/assets/stylesheets/wvyp.css"/>
</head>
<body>
<div class...

object WVYPStream extends Controller {
def index = Action {
val wvypCountFuture = ServiceClient.makeServiceCall("wvyp")
va...

object WVYPStream extends Controller {
def index = Action {
val wvypCountFuture = ServiceClient.makeServiceCall("wvyp")
va...

object WVYPStream extends Controller {
def index = Action {
val wvypCountFuture = ServiceClient.makeServiceCall("wvyp")
va...

object WVYPStream extends Controller {
def index = Action {
val wvypCountFuture = ServiceClient.makeServiceCall("wvyp")
va...

GET
@(wvypBody:/mock/:serviceName controllers.Mock.mock(serviceName: String)
Html, wvyuBody: Html)
GET

/wvyp

controllers...

The page loads almost immediately and we see the WVYP
count right away.

2 seconds later, the search count appears.

Also, the CSS starts downloading immediately!

This is a huge win if the stuff at the top of the
page loads quickly. But what if it’s slow?

object Mock extends Controller {
def mock(serviceName: String) = Action.async {
serviceName match {
case "wvyp" => respond...

Again, the page loads almost immediately, but this time,
neither count is visible.

2 seconds later, both counts appear.

Fortunately, the CSS still loads right at the beginning.

So it’s still a big win, even if the top of the
page is slow, but not by as much.

Is there a way to stream the data out of
order but still render it in order?

JavaScript!

@(contents: Html, id: String)

<script type="text/html-stream" id="@id-contents">
@contents
</script>

<script type="text/...

object Pagelet {
def render(html: Html, id: String): Html = {
views.html.ui.pagelet(html, id)
}

def renderStream(html: Ht...

@(body: ui.HtmlStream)
<html>
<head>
<link rel="stylesheet" href="/assets/stylesheets/wvyp.css"/>
</head>
<body>
<div clas...

def index = Action {
val wvypCountFuture = ServiceClient.makeServiceCall("wvyp")
val searchCountFuture = ServiceClient.mak...

def index = Action {
val wvypCountFuture = ServiceClient.makeServiceCall("wvyp")
val searchCountFuture = ServiceClient.mak...

def index = Action {
val wvypCountFuture = ServiceClient.makeServiceCall("wvyp")
val searchCountFuture = ServiceClient.mak...

The page loads almost immediately, but now we see the
search count first

2 second later, the WVYP count appears.

We now have the basics for out-of-order,
BigPipe style rendering!

Wins
1.

Much faster time to first byte

2.

Static content can start loading much earlier

3.

User can see and start int...

Caveats
1.

Once you’ve streamed out the HTTP headers, you can’t change them. This
means cookies and error handling may ha...

You can find the code from this presentation at:
https://github.com/brikis98/ping-play

Thank you!

Top clipped slide