Massively Scalable, Massively Fault Tolerant
•Download as KEY, PDF•
1 like•566 views
This talk covers the basics of Erlang/OTP, a framework for highly scalable, highly fault tolerant applications.
Recommended
More Related Content
Recently uploaded
Recently uploaded (20)
Featured
Featured (20)
Massively Scalable, Massively Fault Tolerant
- 1. MASSIVELY SCALEABLE MASSIVELY FAULT-TOLERANT Trotter Cashion Philly Lambda, 2011
- 2. Erlang BUT FIRST... AND OTP IN ACTION Buy this Book! http://bit.ly/otp-book Martin Logan Eric Merritt Richard Carlsson FOREWORD BY ULF WIGER MANNING
- 3. Erlang NO SERIOUSLY... IN ACTION AND OTP http://bit.ly/otp-book Martin Logan Eric Merritt Richard Carlsson FOREWORD BY ULF WIGER MANNING
- 4. Erlang AVAILABLE IN PDF... AND OTP IN ACTION http://bit.ly/otp-book Martin Logan Eric Merritt Richard Carlsson FOREWORD BY ULF WIGER MANNING
- 5. AND NOW FOR ME...
- 6. CO-FOUNDER
- 7. @CASHION http://www.charlesheflin.com/wp-content/uploads/2010/06/fail-whale.jpg
- 9. CHLOE • Realtime Web Server • Proxies WebSocket (and fallback) connections to your app • http://github.com/mashion/chloe • Written in Erlang/OTP (plus a lot of JavaScript)
- 10. BAI PHILLY! Leaving in July... http://www.flickr.com/photos/dennis/156048151/
- 11. HAI CALIFORNIA! See you in August... http://www.flickr.com/photos/salim/402618628/
- 12. AN ERLANG PRIMER Yoinked from the Erlang Movie. Go Watch It!!!!
- 13. THE BASICS • Functional • Single Assignment Variables • Pattern Matching is Big Here • Everything’s a Process • One Module Per File
- 15. ANONYMOUS FUNCTIONS make_hello() -> fun () -> io:fwrite("Ohai!n") end. Hello = make_hello(). Hello().
- 16. SINGLE ASSIGNMENT VARS 1> Hello = 3. 3 2> Hello = 2. ** exception error: no match of right hand side value 2 3> f(). ok 4> Hello = 2. 2
- 17. OH NOES! WHAT WENT WRONG? destructuring_bind() -> [{Name, Title}, _] = [{"Trotter", "Speaker"}, {"Aaron", "Organizer"}], io:fwrite("~pn", [Name]), %% => Trotter io:fwrite("~pn", [Title]). %% => Speaker
- 18. PATTERN MATCHING Erlang Ruby match_me(1) -> def match_me(val) loneliest; case val match_me(2) -> when 1 next_to_loneliest; :loneliest match_me(friend) -> when 2 all_ok; :next_to_loneliest match_me(_) -> when :friend what_is_this. :all_ok else :what_is_this end end
- 19. MAKING A PROCESS process.erl concurrent_stuffs() -> spawn(fun () -> timer:sleep(1000), io:fwrite("So slown") end), spawn(fun () -> io:fwrite("Super fastn") end). erlang shell 5> process:concurrent_stuffs(). Super fast <0.39.0> So slow
- 20. RECEIVE LOOPS food_loop() -> receive pizza -> io:fwrite("let's eatn"), food_loop(); death -> io:fwrite("all done heren"); _ -> io:fwrite("Nothing to don"), food_loop() end.
- 21. SENDING MESSAGES make_process() -> spawn(?MODULE, food_loop, []). pizza_death() -> Proc = make_process(), Proc ! pizza, Proc ! hi, Proc ! death, %% Goes into the void Proc ! huh.
- 22. HOW THAT WORKS process 1 pizza_death
- 23. HOW THAT WORKS process 1 spawn/3 pizza_death
- 24. HOW THAT WORKS process 1 process 2 spawn/3 pizza_death food_loop
- 25. HOW THAT WORKS process 1 process 2 spawn/3 pizza_death food_loop food_loop’s Mailbox
- 26. HOW THAT WORKS process 1 process 2 spawn/3 pizza_death food_loop food_loop’s Mailbox
- 27. HOW THAT WORKS process 1 process 2 spawn/3 pizza_death food_loop food_loop’s Mailbox pizza
- 28. HOW THAT WORKS process 1 process 2 spawn/3 pizza_death food_loop food_loop’s Mailbox pizza hi
- 29. HOW THAT WORKS process 1 process 2 spawn/3 pizza_death food_loop food_loop’s Mailbox pizza hi death
- 30. HOW THAT WORKS process 1 process 2 spawn/3 pizza_death food_loop food_loop’s Mailbox pizza hi death huh
- 31. HOW THAT WORKS process 1 process 2 spawn/3 pizza_death food_loop food_loop’s Mailbox pizza hi death huh
- 32. HOW THAT WORKS process 1 process 2 spawn/3 pizza_death food_loop food_loop’s Mailbox pizza hi death huh
- 33. HOW THAT WORKS process 1 process 2 spawn/3 pizza_death food_loop food_loop’s Mailbox pizza “let’s eatn” hi death huh
- 34. HOW THAT WORKS process 1 process 2 spawn/3 pizza_death food_loop food_loop’s Mailbox pizza “let’s eatn” hi death huh
- 35. HOW THAT WORKS process 1 process 2 spawn/3 pizza_death food_loop food_loop’s Mailbox pizza hi death huh
- 36. HOW THAT WORKS process 1 process 2 spawn/3 pizza_death food_loop food_loop’s Mailbox pizza hi “Nothing to don” death huh
- 37. HOW THAT WORKS process 1 process 2 spawn/3 pizza_death food_loop food_loop’s Mailbox pizza hi “Nothing to don” death huh
- 38. HOW THAT WORKS process 1 process 2 spawn/3 pizza_death food_loop food_loop’s Mailbox pizza hi death huh
- 39. HOW THAT WORKS process 1 process 2 spawn/3 pizza_death food_loop food_loop’s Mailbox pizza hi death “all done heren” huh
- 40. HOW THAT WORKS process 1 process 2 spawn/3 pizza_death food_loop food_loop’s Mailbox pizza hi death “all done heren” huh
- 41. HOW THAT WORKS process 1 process 2 spawn/3 pizza_death food_loop food_loop’s Mailbox pizza hi death huh
- 42. HOW THAT WORKS process 1 process 2 spawn/3 pizza_death food_loop food_loop’s Mailbox pizza hi death huh
- 43. HOW THAT WORKS process 1 spawn/3 pizza_death food_loop’s Mailbox pizza hi death huh
- 44. HOW THAT WORKS process 1 spawn/3 pizza_death pizza hi death huh
- 45. HOW THAT WORKS process 1 spawn/3 pizza_death pizza hi death
- 46. BUT DOES IT SCALE? Hells yea! http://www.flickr.com/photos/24736216@N07/4808606178/
- 47. TWO WAYS TO SCALE Name Implementation Horizontally Buy a lot of machines Vertically Buy a really big machine
- 48. ERLANG SCALES WELL ON BOTH WITH NEARLY ZERO CODE CHANGE
- 49. BOOM!
- 50. VERTICAL SCALING • Your processes will take advantage of all cores available • Yay! No code changes required
- 51. HORIZONTAL SCALING • New nodes added to the cluster can be auto-detected • Code must know that a service could live on another node • You’ll need resource discovery
- 52. CONNECTING NODES $ erl -name b 1> AHost = 'a@awesome-town.local'. 'a@awesome-town.local' 2> io:fwrite("Current nodes: ~p~n", [nodes()]). Current nodes: [] 3> io:fwrite("Ping? ~p~n", [net_adm:ping(AHost)]). Ping? pong 4> io:fwrite("Current nodes: ~p~n", [nodes()]). Current nodes: ['a@awesome-town.local' 5> q().
- 53. SERVER REFERENCES Type Definition <Pid> globally unique process identifier RegisteredName locally registered process process locally registered on {RegisteredName, Node} another node {global, GlobalName} globally registered process
- 54. LISTEN FOR INCOMING MESSAGE listen() -> register(listener, self()), listen_loop(). listen_loop() -> receive {From, pizza} -> From ! "Nom noms", listen_loop(); {From, _} -> From ! "Boo!", listen_loop(); _ -> io:fwrite("Listen is dying!") end.
- 55. SENDING MESSAGE TO NODES $ erl -name b 1> AHost = 'a@awesome-town.local'. 'a@awesome-town.local' 2> {listener, AHost} ! {self(), pizza}. ok 3> receive PizzaResp -> PizzaResp end.
- 56. RESOURCE DISCOVERY Method Purpose add_target_resource(Type) specify a resource type you want to use add_local_resource(Type, Pid) specify a resource type you have fetch_resources(Type) fetch all resources of specified type trade_resources() trade resource information See Chapter 8 of Erlang And OTP In Action
- 57. HORIZONTAL CAVEATS • Passing large amounts of data between nodes is inefficient • Too many nodes will impact performance (Keep it < 100) • There are ways around both these problems
- 58. OTP BASICS Cause managing all the processes by hand is dumb Thanks, Homer
- 59. OTPWTFBBQ? • Open Telecom Platform • Abstracts common systems behavior (servers, supervisors, ...) • Easier than pure Erlang! • Gives Erlang its reputation for fault tolerance
- 60. BASIC CONSTRUCTS Construct Purpose Encapsulates a large set of code Application meant to run as one unit. Supervisor Starts and restarts workers. Worker Actually does stuff
- 61. HTTP://GITHUB.COM/MASHION/CHLOE Chloe (Application) chloe_sup (Supervisor) websocket_sup session_sup session_manager Other Stuffs (Supervisor) (Supervisor) (Worker)
- 62. WORKERS • Use the `gen_server` behaviour (for the most part) • Contain your application logic • You define a module with callbacks • OTP runs a server calls back to module when messages arrive
- 63. HOW WORKERS WORK client process (your code) gen_server functions gen_server code (part of OTP) gen_server callbacks worker code (your code) worker process
- 64. SENDING MESSAGES TO WORKERS say_hello_sync(Name) -> gen_server:call(?SERVER, {say_hello, Name}). say_hello_async() -> gen_server:cast(?SERVER, say_hello). gen_server_examples.erl
- 65. EXAMPLE CALLBACKS handle_call({say_hello, Name}, _From, State) -> Response = string:concat("Hello, ", Name), {reply, {ok, Response}, State}. handle_cast(say_hello, State) -> io:fwrite("Hello world!n"), {noreply, State}. gen_server_examples.erl
- 66. WORKER CALLBACKS Callback Purpose init(Args) Initialize the server process handle_call(Request, From, State) Handle a synchronous call handle_cast(Request, State) Handle an asynchronous cast handle_info(Info, State) For timeouts and random messages terminate(Reason, State) Clean up the process when it dies code_change(OldVsn, State, Extra) Called when upgrading code
- 67. WORKER FUNCTIONS Callback Purpose start_link(Module, Args, Options) Start a supervised worker start(Module, Args, Options) Start an unsupervised worker call(ServerRef, Request, Timeout) Synchronously call a worker cast(ServerRef, Request) Asynchronously cast to a worker man gen_server
- 68. SUPERVISORS • Use the `supervisor` behaviour • Contain little to no application logic • Know how to start and restart failed workers • Keep your application from ever fully dying
- 69. CHILD SPECS {Id, StartFunc, Restart, Shutdown, Type, Modules} Unique name used to identify child process.
- 70. CHILD SPECS {session_manager, StartFunc, Restart, Shutdown, Type, Modules} Unique name used to identify child process.
- 71. CHILD SPECS {session_manager, StartFunc, Restart, Shutdown, Type, Modules} Tuple {M, F, A} used to start the child.
- 72. CHILD SPECS {session_manager, {session_manager, start_link, []}, Restart, Shutdown, Type, Modules} Tuple {M, F, A} used to start the child.
- 73. CHILD SPECS {session_manager, {session_manager, start_link, []}, Restart, Shutdown, Type, Modules} Determines when a terminated process should be restarted.
- 74. CHILD SPECS {session_manager, {session_manager, start_link, []}, permanent, Shutdown, Type, Modules} Determines when a terminated process should be restarted.
- 75. CHILD SPECS {session_manager, {session_manager, start_link, []}, permanent, Shutdown, Type, Modules} Determines how to kill child processes.
- 76. CHILD SPECS {session_manager, {session_manager, start_link, []}, permanent, 5000, Type, Modules} Determines how to kill child processes.
- 77. CHILD SPECS {session_manager, {session_manager, start_link, []}, permanent, 5000, Type, Modules} Is this child a worker or a supervisor?
- 78. CHILD SPECS {session_manager, {session_manager, start_link, []}, permanent, 5000, worker, Modules} Is this child a worker or a supervisor?
- 79. CHILD SPECS {session_manager, {session_manager, start_link, []}, permanent, 5000, worker, Modules} Used by code reloading. Make it the same as the other module name.
- 80. CHILD SPECS {session_manager, {session_manager, start_link, []}, permanent, 5000, worker, [session_manager]} Used by code reloading. Make it the same as the other module name.
- 81. RESTART STRATEGIES Strategy When a child dies.... one_for_one restart it one_for_all restart all children restart it and all children after it rest_for_one in child list restart it (also allows you to simple_one_for_one dynamically add children)
- 82. ONE_FOR_ONE Supervisor Child 1 Child 2 Child 3
- 83. ONE_FOR_ONE Supervisor Child 2 Child 3
- 84. ONE_FOR_ONE Supervisor Child 1 Child 2 Child 3
- 85. ONE_FOR_ALL Supervisor Child 1 Child 2 Child 3
- 86. ONE_FOR_ALL Supervisor Child 2 Child 3
- 87. ONE_FOR_ALL Supervisor Child 3
- 88. ONE_FOR_ALL Supervisor
- 89. ONE_FOR_ALL Supervisor Child 1
- 90. ONE_FOR_ALL Supervisor Child 1 Child 2
- 91. ONE_FOR_ALL Supervisor Child 1 Child 2 Child 3
- 92. REST_FOR_ONE Supervisor Child 1 Child 2 Child 3
- 93. REST_FOR_ONE Supervisor Child 1 Child 3
- 94. REST_FOR_ONE Supervisor Child 1
- 95. REST_FOR_ONE Supervisor Child 1 Child 2
- 96. REST_FOR_ONE Supervisor Child 1 Child 2 Child 3
- 97. SIMPLE_ONE_FOR_ONE Supervisor
- 98. SIMPLE_ONE_FOR_ONE Supervisor Child 1
- 99. SIMPLE_ONE_FOR_ONE Supervisor Child 1 Child 2
- 100. SIMPLE_ONE_FOR_ONE Supervisor Child 1 Child 2 Child 3
- 101. SIMPLE_ONE_FOR_ONE Supervisor Child 2 Child 3
- 102. SIMPLE_ONE_FOR_ONE Supervisor Child 1 Child 2 Child 3
- 103. STARTING A SUPERVISOR init([]) -> Worker = {gen_server_examples, {gen_server_examples, start_link, []}, permanent, 5000, worker, [gen_server_examples]}, RestartStrategy = {one_for_one, 5, 30}, {ok, {RestartStrategy, [Worker]}}.
- 104. SUPERVISOR CALLBACK callback purpose return data structure telling what init(Args) to supervise
- 105. SUPERVISOR FUNCTIONS Callback Purpose start_link(Module, Args) Start a supervised supervisor start_child(SupRef, ChildSpec) Start a child under the supervisor man supervisor
- 106. FAULT TOLERANCE • When you hit an error, let the worker die • Supervisor will restart it • Either try what killed it again or move on • Life is grand
- 108. THANK YOU! • Questions? • http://bit.ly/erlang-otp-gist • http://bit.ly/otp-book • http://mashion.net
Editor's Notes
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n