Decomposing Applications for Scalability and Deployability (April 2012)
•
32 likes•18,537 views
Today, there are several trends that are forcing application architectures to evolve. Users expect a rich, interactive and dynamic user experience on a wide variety of clients including mobile devices. Applications must be highly scalable, highly available and run on cloud environments. Organizations often want to frequently roll out updates, even multiple times a day. Consequently, it’s no longer adequate to develop simple, monolithic web applications that serve up HTML to desktop browsers. In this talk we describe the limitations of a monolithic architecture. You will learn how to use the scale cube to decompose your application into a set of narrowly focused, independently deployable back-end services and an HTML 5 client. We will also discuss the role of technologies such as NodeJS and AMQP brokers. You will learn how a modern PaaS such as Cloud Foundry simplifies the development and deployment of this style of application.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Decomposing Applications for Scalability and Deployability (April 2012)
Similar to Decomposing Applications for Scalability and Deployability (April 2012) (20)
More from Chris Richardson
More from Chris Richardson (20)
Recently uploaded
Recently uploaded (20)
Decomposing Applications for Scalability and Deployability (April 2012)
- 1. Decomposing applications for deployability and scalability Chris Richardson Author of POJOs in Action Founder of the original CloudFoundry.com @crichardson crichardson@vmware.com 1
- 2. Presentation goal Decomposing applications to improve deployability and scalability and How Cloud Foundry helps 2
- 3. About Chris 3
- 4. (About Chris) 4
- 5. About Chris() 5
- 6. About Chris 6
- 7. About Chris http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/ 7
- 8. vmc push About-Chris Developer Advocate for CloudFoundry.com Signup at http://cloudfoundry.com Promo code: CFOpenTour2012 8
- 9. Agenda § The (sometimes evil) monolith § Decomposing applications into services § How do services communicate? § Presentation layer design § Decomposing a monolithic application § How Cloud Foundry helps 9
- 10. Let’s imagine you are building an e- commerce application 10
- 11. Traditional web application architecture WAR StoreFrontUI Accounting Service MySQL Browser Apache Database InventoryService Shipping Service Simple to Tomcat develop test deploy scale 11
- 12. But there are problems 12
- 13. Users expect a rich, dynamic and interactive experience on mobile devices and desktop h oug d en oo ’t g HTTP Request e isn r ctu Java Web Browser e hit HTML/Javascript Application c I ar ty le U s Old Real-time web ≅ NodeJS 13
- 14. One dimensional scalability § Scales to handle transaction volumes § Higher transaction rate required run more instances § Some clouds will do this automatically, others easily § No code changes required BUT § Does not scale with increasing data volumes § Caching is less effective § Requires more memory § Requires more I/O 14
- 15. Components that don’t cluster § Most application components are easily clustered: • Most services are stateless • Server-side session state: sticky session routing/state server BUT there are exceptions § Singletons that maintain global application state: • e.g. CEP engine analyzing event stream over HTTP § Commercial software with a per-server license fee § If one application component can’t be clustered you can’t cluster the application! 15
- 16. Obstacle to frequent deployments § Need to redeploy everything to change one component § Interrupts long running background (e.g. Quartz) jobs § Increases risk of failure Fear of change § Updates will happen less often § e.g. Makes A/B testing UI really difficult 16
- 17. Large application Slow IDE Slow start times for web container 17
- 18. Scaling development WAR StoreFrontUI Accounting Scalable != InventoryService development Shipping § Forces teams to synchronize development efforts § Teams need to coordinate updates 18
- 19. Long-term commitment to a single technology stack § Let’s suppose you go with the JVM • Some polyglot support • Many (but not all) JVM languages interoperate easily • e.g. Scala, Groovy modules within a Java application • But you can’t use non-JVM languages § Application depends on a particular: • Framework - e.g. Spring or Java EE • Container - e.g. Tomcat § Switching technology stack touches entire application • Painful • Rarely done 19
- 20. Agenda § The (sometimes evil) monolith § Decomposing applications into services § How do services communicate? § Presentation layer design § Decomposing a monolithic application § How Cloud Foundry helps 20
- 21. 3 dimensions to scaling Y axis - functional decomposition Scale by s ng splitting ila g n hi g s oni rt different things iti rt im pa ta in da ittl s- sp i ax by Z ale X axis - horizontal duplication Sc Scale by cloning 21
- 22. X axis scaling - application level Load WAR Balancer Tomcat 22
- 23. X axis scaling - database level Writes Consistent reads MySQL Master Inconsistent reads MySQL MySQL MySQL Slave 1 Slave 2 Slave N Slaves are clones 23
- 24. Benefits and drawbacks of X axis splits § Benefits • Easy and fast to implement • Handles increasing transaction volumes • Simple to manage § Drawbacks • Doesn’t scale with increasing data volumes • Doesn’t address application/development complexity • Couples entire application to a particular technology stack 24
- 25. Z axis scaling - data partitioning § Each server runs the identical code base § Different servers responsible for different data § Route request based on • Attribute of the request • Identity/attribute of the requestor/customer § For example: • Sharding - partition data by primary key • Free customer vs. paying customer Server 1 Request: ... Some attribute ... “Router” Server 2 ... 25
- 26. Z axis scaling - database sharding Partition column RESTAURANT table ID Name … 1 Ajanta 2 Eggshop … X axis scaling too! Database server 1 Database server 2 ID Name … ID Name … 1 Ajanta 2 Eggshop … … 26
- 27. Z axis scaling - application example Content Query Update Router Query Aggregator Search Service Search Service Search Service Search Service Cache Cache Database server 1 Database server 2 Partition 1 Partition 2 27
- 28. Benefits and drawbacks of Z axis splits § Benefits • Each server only deals with a subset of the data • Improved caching • Reduced memory usage and reduced I/O • Improved transaction scalability • Improved fault isolation - failure only makes part of the data inaccessible § Drawbacks • Doesn’t scale with increasing development and application complexity • Increased management/deployment complexity 28
- 29. Y axis scaling - functional partitioning § Splits monolithic application into a set of services § Each service implements related set of functionality § Partitioning schemes: • Partition functionality by noun or by verb • Single Responsibility Principle • e.g. Unix utilities - do one focussed thing well 29
- 30. Y axis scaling - application level billing web application Accounting Service Store front web application inventory web application Store Front InventoryService shipping web application ShippingService Apply X axis cloning and/or Z axis partitioning to each service 30
- 31. Y axis scaling - databases Application MySQL MySQL MySQL MySQL Order Customer table table .... .... Apply X axis cloning and/or Z axis partitioning to each database 31
- 32. Real world examples http://techblog.netflix.com/ Between 100-150 services are accessed to build a page. http://highscalability.com/amazon-architecture http://www.addsimplicity.com/downloads/ eBaySDForum2006-11-29.pdf http://queue.acm.org/detail.cfm?id=1394128 32
- 33. Benefits and drawbacks of Y axis splits § Benefits • Scales development: focussed two pizza devops teams • Deploy services independently • Scale services independently • Improves fault isolation • Eliminates long-term commitment to a single technology stack • Enforces well defined interfaces between components § Drawbacks • Implementation complexity • Deployment complexity 33
- 34. Two levels of architecture § System level • Defines the inter-service glue: interfaces and communication mechanisms • Slow changing Versus § Service level • Defines the internal architecture of each service • Far fewer constraints on technology • Each service could use a different technology stack • Rapid evolving 34
- 35. If services are small... § Regularly rewrite using a better technology stack § Pick the best developers rather than best <pick a language> developers § Adapt system to changing requirements and better technology without a total rewrite Fred George “Developer Anarchy” 35
- 36. Moreover: you are not the same you ... § Cell lifetimes: Can we build software • hours - some white blood cells systems with these characteristics? • days - stomach lining cells • years - bone cells Too much technical debt • lifetime - brain cells component death? § 50 to 70 billion of your cells die each day § Yet you (the system) remains intact http://dreamsongs.com/Files/ DesignBeyondHumanAbilitiesSimp.pdf http://dreamsongs.com/Files/WhitherSoftware.pdf 36
- 37. Agenda § The (sometimes evil) monolith § Decomposing applications into services § How do services communicate? § Presentation layer design § Decomposing a monolithic application § How Cloud Foundry helps 37
- 38. Inter-service communication options § Multiple collaborating services need a communication mechanism § Many choices: • Synchronous asynchronous • Transports: HTTP, AMQP, ... • Formats: JSON, XML, Protocol Buffers, Thrift, ... • Even via the database § Distributed application error handling strategies 38
- 39. Synchronous communication wgrus-billing.war Accounting Service REST wgrus-store.war wgrus-shipping.war SOAP StoreFrontUI Shipping MySQL Service Thrift ... wgrus-inventory.war InventoryService Lots of choices 39
- 40. Service discovery options § Clients need to know the coordinates of their dependencies § Option #1: Caller is configured with dependencies’s URL • e.g. Environment variables, system properties § Option #2: Services announce their location • Relies on ‘broadcast’ protocol or Gossip protocol to announce location • e.g. RabbitMQ message broker § Option #3: configuration server • Maintains all the configuration information • e.g. Zookeeper 40
- 41. Benefits and drawbacks § Benefits • Simple to implement and use § Drawbacks • Poor fault isolation: caller is blocked if server is down/slow • Caller needs to know server’s coordinates (URL, ...) • Less scalable than asynchronous protocols 41
- 42. Asynchronous message-based communication wgrus-billing.war Accounting Service wgrus-store.war RabbitMQ wgrus-inventory.war StoreFrontUI (Message MySQL Broker) Widget InventoryService wgrus-inventory.war InventoryService 42
- 43. Benefits and drawbacks § Benefits • Decouples caller from server • Caller unaware of server’s coordinates (URL) • Message broker buffers message when server is down/slow § Drawbacks • Additional complexity of message broker • RPC using messaging is more complex 43
- 44. Shared databases? Order Management Customer Management Database Orders Customers 44
- 45. Benefits and drawbacks § Benefits • Simple • Single (consistent) source of data § Drawbacks • Defeats the purpose of having multiple services • Less scalable • More coupling 45
- 46. Better: separate databases Customer Update Event Order Management Customer Management Database Database Orders Customers Customers (copy) 46
- 47. Maintaining consistency without 2PC begin transaction update order tables save intent to update customer tables commit transaction for each saved intent begin transaction delete intent queue message to update customer Customer management commit transaction dequeue message Order management begin transaction if (update has not been applied) { record update as applied update customer tables BASE: An Acid Alternative by Dan Pritchett } http://queue.acm.org/detail.cfm?id=1394128 commit transaction acknowledge message 47
- 48. Writing code that calls services 48
- 49. Composable futures § Problem: • Service A needs to call services B and C and then D • Makes sense to call B and C parallel • Yet most concurrency APIs are low-level, error-prone etc § Solution: • Use Akka composable futures = really nice abstraction val futureB = callB() Two calls execute in parallel val futureC = callC() val futureD = for { b <- futureB.mapTo[SomeType] c <- futureC.mapTo[SomeType] d <- callD(b, c) And then invokes D } yield d val result = Await.result(futureD, 1 second). asInstanceOf[SomeOtherType] Get the result of D http://doc.akka.io/docs/akka/2.0.1/scala/futures.html http://en.wikipedia.org/wiki/Futures_and_promises 49
- 50. Spring Integration § Builds on Spring framework § High-level of abstraction for building message based applications § Implements EAI patterns § Provides plumbing for exchanging messages between application components § Promotes loosely coupled components § Integrates with external messaging infrastructure: JMS, AMQP, HTTP, Email, File transfer 50
- 51. Spring Integration concepts § Message channel • Virtual pipe connecting producer and consumer § Message endpoints • The filter of a pipes-and-filter architecture • Read from and/or write to channel § Endpoint types: • Transformer • Filter • Router • Splitter • Aggregator • ServiceActivator • Inbound channel adapter - read from external source, writes to channel • Outbound channel adapter - read from channel write to external destination 51
- 52. Spring Integration insulates components from the underlying communication mechanism 52
- 53. Example of reconfigurability - local @Service public class OrderServiceImpl { @Service public class ShippingServiceImpl { @Autowired private ShippingService shippingService; public void shipOrder(String orderId) { System.out.println("shipped order: " + orderId); public void placeOrder() { } String orderId = generateOrderId(); … } shippingService.shipOrder(orderId); } } Order Shipping Service service Messaging Gateway Channel Service Activator 53
- 54. Example of reconfigurability - distributed Code unchanged in new deployment Order Shipping Service service Messaging Gateway RabbitMQ Channel AMQP AMQP Channel Service Activator 54
- 55. Handling failure § Errors happen (especially in distributed systems) § Use timeouts and retries • Never wait forever • Some errors are transient so retry § Use per-dependency bounded thread pool with bounded queue • Limits number of outstanding requests • Fails fast if service is slow or down § Use circuit breaker • High rate of errors stop calling temporarily • Avoids calling service that has issues § On failure • Returned cached or default data • Invoke custom error handler http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html 55
- 56. Agenda § The (sometimes evil) monolith § Decomposing applications into services § How do services communicate? § Presentation layer design § Decomposing a monolithic application § How Cloud Foundry helps 56
- 57. Modular application Choice of presentation layer technology + Redeploy UI frequently/independently 57
- 58. NodeJS is the fashionable technology Many JavaScript client frameworks have a NodeJS counterpart e.g. socket.io 58
- 59. NodeJS isn’t the only game in town JVM-based http://vertx.io/ 59
- 60. A modern web application Service 1 RESTful WS Node JS Browser HTML 5 Server Application Application Service 2 Events Socket.io Socket.io client server ... 60
- 61. Alternatively: multiple front-end servers Node JS Service 1 Browser HTML 5 Node JS Service 2 Application Node JS ... Needs single-sign-on 61
- 62. NodeJS - using RESTful WS and AMQP REST Service Requests Node JS Events socket.io AMQP AMQP RabbitMQ Service 62
- 63. Updating the UI is easy § Update the UI independently of rest of system § Easily run A/B tests § Enables fast iteration of the UI http://theleanstartup.com/principles 63
- 64. But coordination with backend changes required § Let’s imagine that you are deploying an advanced search feature: • Enhancements to search service • Enhancements to UI § Before • Deploy new war § Now: • Some coordination required • Deploy updated backend service • Deploy updated NodeJS and browser code • Enable feature using feature switch 64
- 65. Agenda § The (sometimes evil) monolith § Decomposing applications into services § How do services communicate? § Presentation layer design § Decomposing a monolithic application § How Cloud Foundry helps 65
- 66. The challenge § In the beginning: • It would be easy to use Y axis/functional decomposition • But you don’t need it and it would slow you down § Later on: • You do need it • But your application logic is tangled and decomposition is difficult 66
- 67. Lower cost way to start with Y axis decomposition § Enforce boundaries between services, e.g. DTOs § But run in a single JVM with a single database § Use Spring Integration • Course-grained message-based communication • Via in-memory channels § Downside: • Implementing the glue layers are extra work 67
- 68. Special case: Components that don’t cluster Application Monitoring Rest of domain classes A C B D Esper Tangled dependencies prevent Y axis scaling Doesn’t cluster No X axis scaling/cloning 68
- 69. Don’t let this happen! Use well-defined interfaces Ensure loose coupling 69
- 70. Traditional application architecture Monolithic application Service A Service B Service C Shared domain model A C Difficult to untangle B X Z Y 70
- 71. Extracting a service = untangling dependencies Application Service Service C Service A Service B Trouble! Domain model 1 Domain model 2 X A C Y B Z 71
- 72. Untangling dependencies § Domain object is part of an association class A { • Read only - replicate private B other; • Update only - notification interface } • Update then read - replicate § Domain object is a request parameter class A { • Read-only - replace with DTO void m(B other) { ... • Update only - notification interface } • Update then read - tricky } § Note - Need acyclic dependencies • If used object isn’t part of a service = assumption is that it belongs in calling service 72
- 73. Bounded context is a useful idea § Different parts of a system different domain models § Different services have a different view of some domain objects, e.g. • User Management = complex view of user • Rest of application = PK + ACL + Name § Different services can have a different domain model § Services exchange messages to synchronize data 73
- 74. Untangling dependencies 1 Service A Service B Service C Domain model 1 Domain model 2 X A C doSomething(Y) doSomethingElse() Y B getSomething() Z 74
- 75. Untangling dependencies 2 Service A Service B Service C Domain model 1 Domain model 2 doSomething(YDTO) C X A Facade A doSomethingElse() Y B Facade B Z 75
- 76. Untangling dependencies 3 Service A Service B Service C Domain model 1 Domain model 2 C doSomething(YDTO) X Facade A A Message Broker Listener Y Listener B doSomethingElse() Z Facade B 76
- 77. Agenda § The (sometimes evil) monolith § Decomposing applications into services § How do services communicate? § Presentation layer design § Decomposing a monolithic application § How Cloud Foundry helps 77
- 78. Traditional tools: monolithic applications 78
- 79. Developing service-oriented apps is harder § Many more moving parts to manage • Infrastructure services: SQL, NoSQL, RabbitMQ • Application services § Who is going to setup the environments: • the developer sandbox? • ... • QA environments? 79
- 80. Cloud Foundry makes it easier § Easy to deploy applications § Easy to provision services § Manifest describes the structure of a set of services § vmc env-add for binding 'app services' together § Micro Cloud Foundry is your personal cloud/sandbox § Caldecott exposes services for use by integration tests 80
- 81. Services available on Cloud Foundry 81
- 82. Creating a service instance $ vmc create-service mysql --name mysql1 Creating Service: OK $ vmc services ...... =========== Provisioned Services ============ +-------------+---------+ | Name | Service | +-------------+---------+ | mysql1 | mysql | +-------------+---------+
- 83. Multi-application manifest - part 1 --- applications: inventory/target: Path to application name: inventory url: cer-inventory.chrisr.cloudfoundry.me framework: name: spring info: mem: 512M description: Java SpringSource Spring Application exec: mem: 512M instances: 1 services: si-rabbit: Required services type: :rabbitmq si-mongo: type: :mongodb si-redis: type: :redis 83
- 84. Multi-application manifest - part 2 store/target: Path to application name: store url: cer-store.chrisr.cloudfoundry.me framework: name: spring info: mem: 512M description: Java SpringSource Spring Application exec: mem: 512M instances: 1 services: Required services si-mongo: type: :mongodb si-rabbit: type: :rabbitmq 84
- 85. One command to create services and deploy application $ vmc push Would you like to deploy from the current directory? [Yn]: Pushing application 'inventory'... Creating Application: OK Creating Service [si-rabbit]: OK Binding Service [si-rabbit]: OK Creating Service [si-mongo]: OK Binding Service [si-mongo]: OK Creating Service [si-redis]: OK Binding Service [si-redis]: OK Uploading Application: Checking for available resources: OK Processing resources: OK Packing application: OK Uploading (12K): OK vmc push: Push Status: OK •Reads the manifest file •Creates the required services Staging Application 'inventory': OK Starting Application 'inventory': OK Pushing application 'store'... Creating Application: OK •Deploys all the applications Binding Service [si-mongo]: OK Binding Service [si-rabbit]: OK Uploading Application: Checking for available resources: OK Processing resources: OK Packing application: OK Uploading (5K): OK Push Status: OK Staging Application 'store': OK 85 Starting Application 'store': ...
- 86. Application configuration via environment variables $ vmc env-add cf1 PAYMENT_SVC=http://... Adding Environment Variable [PAYMENT_SVC=http://...]: OK Stopping Application: OK Staging Application: OK String value = System.getenv("PAYMENT_SVC") @Value("#{systemEnvironment['PAYMENT_SVC']}") private String envVariable;
- 87. Micro Cloud Foundry: new developer sandbox App Instances Services Open source Platform as a Service project 10.04 A PaaS packaged as a VMware Virtual Machine Use as a developer sandbox • Use the services from Junit integration tests • Deploy your application for functional testing • Remote debugging from STS 87
- 88. Using Caldecott… $ vmc tunnel 1: mysql-135e0 2: mysql1 Which service to tunnel to?: 2 Password: ******** Stopping Application: OK Redeploying tunnel application 'caldecott'. Uploading Application: Checking for available resources: OK Packing application: OK Uploading (1K): OK Push Status: OK Binding Service [mysql1]: OK Staging Application: OK Starting Application: OK Getting tunnel connection info: OK Service connection info: username : uMe6Apgw00AhS password : pKcD76PcZR7GZ name : d7cb8afb52f084f3d9bdc269e7d99ab50 Starting tunnel to mysql1 on port 10000. 1: none 2: mysql Which client would you like to start?: 2
- 89. …Using Caldecott Launching 'mysql --protocol=TCP --host=localhost --port=10000 -- user=uMe6Apgw00AhS --password=pKcD76PcZR7GZ d7cb8afb52f084f3d9bdc269e7d99ab50' Welcome to the MySQL monitor. Commands end with ; or g. Your MySQL connection id is 10944342 Server version: 5.1.54-rel12.5 Percona Server with XtraDB (GPL), Release 12.5, Revision 188 Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or 'h' for help. Type 'c' to clear the current input statement. mysql>
- 90. Running JUnit test with Caldecott Configure your test code to use port + connection info 90
- 91. Summary § Monolithic applications are simple to develop and deploy BUT applying the scale cube § Decomposes your application into services § Enables scaling for transactions and data volumes § Tackles application complexity § Enables scaling for development § Enables frequent, independent deployments § Make it easy to leverage other technologies AND § Cloud Foundry simplifies the development and deployment of “service-oriented” applications 91
- 92. @crichardson crichardson@vmware.com Questions? www.cloudfoundry.com promo code CFOpenTour2012