Building high-performance APIs for the video game industry with Goliath, Grape and EventMachine

Building high-performance APIs for
the video game industry with Goliath,
Grape, and EventMachine
Matt E. Patterson
Digimonkey Studios
Monday, June 10, 13

Matt Patterson
Software Consultant
(Ruby, Rails, Agile, etc.)
Web software since 1998 (ColdFusion and
PHP)
Ruby and Rails since 2006 (Rails 1.0)
Monday, June 10, 13

Game Web Services
Must be fast.
Must be reliable.
Must be scalable...
...and able to handle sudden bursts
(launches, new DLC, etc.)
Monday, June 10, 13

Goliath
open-source, non-blocking, asychronous Ruby web server
framework from postrank-labs
EventMachine reactor.
HTTP parser, Rack, Ruby 1.9+, Ruby Fibers
each request executes in its own Fiber
Monday, June 10, 13

Ubiquitous Goliath
“Hello, world” example
require 'goliath'
class Hello < Goliath::API
def response(env)
[ 200, {}, "Hello, world!" ]
end
end
Monday, June 10, 13

Ubiquitous Goliath
require 'goliath'
def response(env)
[ 200, {}, "Hello, world!" ]
end
end
$ ruby hello.rb -sv -p 8000
Monday, June 10, 13

Ubiquitous Goliath
require 'goliath'
def response(env)
[ 200, {}, "Hello, world!" ]
end
end
[45584:INFO] 2013-05-28 11:52:52 :: Starting server on
0.0.0.0:8000 in development mode. Watch out for stones.
Monday, June 10, 13

Ubiquitous Goliath
require 'goliath'
def response(env)
[ 200, {}, "Hello, world!" ]
end
end
$ curl http://lvh.me:8000/
Monday, June 10, 13

Ubiquitous Goliath
require 'goliath'
def response(env)
[ 200, {}, "Hello, world!" ]
end
end
$ curl http://lvh.me:8000/
Hello, world!
Monday, June 10, 13

What I needed...
a full-featured API
versioned endpoints
multiple resources
standard RESTful CRUD
JSON requests / responses
secured requests
logging
localization / translation
file attachments with S3
storage
asynchronous MySQL
queries
tests!
Monday, June 10, 13

What I used...
Grape
REST-like API micro-
framework for Ruby
Carrierwave, Fog, MiniMagick
file uploads and S3 support
Rspec + FactoryGirl
testing
Rabl + MultiJson
JSON requests / responses
em-synchrony + mysql2
async MySQL
globalize3
localization / translation
standalone_migrations
Rails-style migrations
capistrano
deployment stuff
Monday, June 10, 13

Simple Goliath + Grape
./app.rb
require 'rubygems'
require 'bundler/setup'
require 'goliath'
require 'em-synchrony/activerecord'
require 'grape'
Dir["./app/models/*.rb"].each { |f| require f }
require './app/api'
class Application < Goliath::API
def response(env)
::API.call(env)
end
end
Monday, June 10, 13

./app/api.rb
Dir["./app/apis/v1/*.rb"].each { |f| require f }
class API < Grape::API
mount APIv1::Unlocks
resource 'servicehealth' do
desc "Returns a basic status report."
get "/" do
MultiJson.dump({
status: 'OK',
environment: Goliath::env })
end
end
end
Monday, June 10, 13

./app/apis/v1/unlocks.rb
class APIv1
class Unlocks < Grape::API
version 'v1', using: :path, format: :json
resource :unlocks do
# GET /unlocks/1.json
desc "Returns a single Unlock record by ID"
get "/:id" do
unlock = Unlock.find(params[:id])
custom_render "api_v1/unlocks/show", unlock, 200
end
end
end
end
Monday, June 10, 13

./app/apis/v1/unlocks.rb
class APIv1
class Unlocks < Grape::API
version 'v1', using: :path, format: :json
resource :unlocks do
# GET /unlocks/1.json
desc "Returns a single Unlock record by ID"
get "/:id" do
unlock = Unlock.find(params[:id])
custom_render "api_v1/unlocks/show", unlock, 200
end
end
end
end
Wait, what?
Monday, June 10, 13

back to our ./app/api.rb
helpers do
def custom_render(rabl_template, object, status, args={})
args[:format] ||= 'json'
args[:success] ||= true
render_options = { format: args[:format] }
render_options[:locals] = args[:locals] if args[:locals]
data = Rabl::Renderer.new(rabl_template, object, render_options).render
%({ "success": #{args[:success]}, "data": #{data} })
end
end
...
Monday, June 10, 13

./app/views/api_v1/unlocks/show.json.rabl
attributes :id, :name, :code, :description, :created_at
node(:unique_tags) { |unlock| unlock.tags.uniq }
child :images => :images do
attributes :id, :caption, :mime_type, :url
end
Monday, June 10, 13

end
renders =>
Monday, June 10, 13

end
renders =>
{ "id":1,"name":"Fancy
Thing","code":"001FT","description":"Enim doloribus id
minima.","unique_tags":["foo","bar","woot"],"images":
[{"image":{"id":"1","caption":"dolor sit
amet","mime_type":"image/jpg","url":"http://s3.amazon.com/
whatever.jpg"}}],"created_at":1364916167.000000000 }
Monday, June 10, 13

end
renders =>
{ "success": true, "data": {"id":1,"name":"Fancy
whatever.jpg"}}],"created_at":1364916167.000000000} }
Monday, June 10, 13

./app/models/unlock.rb
Well, that would have worked, if we had an Unlock model...
Monday, June 10, 13

require 'rocket_tag'
class Unlock < ActiveRecord::Base
has_many :images, as: :image_attachable, dependent: :destroy
attr_taggable :tags
validates :name, presence: true
validates :code, presence: true
end
Monday, June 10, 13

require 'rocket_tag'
class Unlock < ActiveRecord::Base
has_many :images, as: :image_attachable, dependent: :destroy
attr_taggable :tags
validates :name, presence: true
validates :code, presence: true
end
Ordinary ActiveRecord like you’re accustomed to...
Monday, June 10, 13

Try it out!
Monday, June 10, 13

Try it out!
$ ruby app.rb -sv -p 8000
Monday, June 10, 13

Try it out!
Monday, June 10, 13

Try it out!
$ curl http://lvh.me:8000/v1/unlocks/1
Monday, June 10, 13

Try it out!
$ curl http://lvh.me:8000/v1/unlocks/1
{ "success": true, "data": {"id":1,"name":"Fancy
whatever.jpg"}}],"created_at":1364916167.000000000}}} }
Monday, June 10, 13

“Could the API
give us users
too? That’d be
great...”
Monday, June 10, 13

./app/api.rb
get "/" do
MultiJson.dump({
status: 'OK',
end
end
end
Remember me?
Monday, June 10, 13

./app/api.rb
Remember me?
mount APIv1::Users
get "/" do
MultiJson.dump({
status: 'OK',
end
end
end
Monday, June 10, 13

Need communication?
Build a Client Gem!
Monday, June 10, 13

Need communication?
Build a Client Gem!
require 'virtus'
require 'rest_client'
require 'multi_json'
module UnlocksClient
class Unlock
include Virtus
attribute :id
attribute :name
attribute :code
attribute :description
attribute :tags
attr_accessor :media_rewards
def self.find(id, params={})
client = RestClient::Resource.new("#{BASE_URL)}/unlocks/#{id}")
response = client.get({params: params})
data = MultiJson.load(response)["data"]
return nil if !response || data.empty?
new(params)
end
end
end
Monday, June 10, 13

Asynch MySQL Gotcha
Used to Rails?
If you’re not paying attention, you might do this...
Monday, June 10, 13

Asynch MySQL Gotcha
Used to Rails?
development:
host: localhost
adapter: mysql2
database: unlocks_dev
pool: 20
timeout: 5000
reconnect: true
username: dbuser
password: 123whatever
Monday, June 10, 13

Asynch MySQL Gotcha
Used to Rails?
development:
host: localhost
adapter: em_mysql2
pool: 20
timeout: 5000
reconnect: true
username: dbuser
Monday, June 10, 13

Asynch MySQL Gotcha
Used to Rails?
development:
host: localhost
adapter: em_mysql2
pool: 20
timeout: 5000
reconnect: true
username: dbuser
YMMV
Monday, June 10, 13

Goliath Tips
curl is your friend on the command line.
Console mode!
ruby app.rb -svC
using Pry to debug stuff: Just add binding.pry
Monday, June 10, 13

Links
Goliath / Grape / EM Stuff
https://github.com/
postrank-labs/goliath
https://github.com/
igrigorik/em-synchrony
https://github.com/
intridea/grape
Matt Stuff
code.digimonkey.com
mepatterson.net
github.com/mepatterson
twitter.com/mepatterson
Monday, June 10, 13

Building high-performance APIs for the video game industry with Goliath, Grape and EventMachine

Recommended

Recommended

More Related Content

Recently uploaded

Recently uploaded (20)

Building high-performance APIs for the video game industry with Goliath, Grape and EventMachine