Recommended
Recommended
More Related Content
More from mfrancis
More from mfrancis (20)
Recently uploaded
Recently uploaded (20)
Complexity, Components & Clouds (Paremus)
- 1. Complexity, Components & Clouds Dr R Nicholson FITE club - April 2010 www.paremus.com FITE Club (London) www.paremus.com April 2010 Copyright © 2010 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 2. FITE Club? “Fight Club is a metaphor for the need to push through the walls we put around ourselves and just go for it” “...probe the frustrations of the people that live in the system...” FITE Club (London) www.paremus.com April 2010 Copyright © 2010 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 3. What do we mean by Complexity? FITE Club (London) www.paremus.com April 2010 Copyright © 2010 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 4. Complexity Information !"#"!$"%"& A suitable Representation FITE Club (London) www.paremus.com April 2010 Copyright © 2010 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 5. Complexity Information !"#"!$"%"& A suitable Representation FITE Club (London) www.paremus.com April 2010 Copyright © 2010 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 6. Complexity (m) Volume = V (x,x’) (z,z’) (y,y’) pressure = P Temperature = T Information !"1(m,x,y,z,x',y',z') Information !"#$#%& A suitable Abstraction FITE Club (London) www.paremus.com April 2010 Copyright © 2009 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 7. Complexity (m) Volume = V (x,x’) (z,z’) (y,y’) pressure = P Temperature = T Information !"1(m,x,y,z,x',y',z') Information !"#$#%& A suitable Abstraction FITE Club (London) www.paremus.com April 2010 Copyright © 2009 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 8. Accidental Complexity • Redundant Information • Same System may be fully described in a more succinct manner • Ability to easily re-factor enables Accidental Complexity to be driven out over time FITE Club (London) www.paremus.com April 2010 Copyright © 2010 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 9. Necessary Complexity When all accidental complexity is stripped away - we are left with necessary complexity... •The actual ‘structure’ of the System •Adaptive Systems are by necessity more complex than non- adaptive Systems Strategies for dealing with necessary complexity? •Abstraction •Perhaps a change Perspective / Representation FITE Club (London) www.paremus.com April 2010 Copyright © 2010 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 10. Abstraction & Virtualisation Virtualisation - Emulates the characteristics of an entity Abstraction - Masks the complexity of the entity (physical or virtual), providing a simplified description Abstraction and Virtualisation are orthogonal & complementary concerns • As system complexity increases - stop attempting a microscopic description use emergent macroscopic characteristics • Boundaries - Where/when does it makes sense to change point of view? • Several such boundaries may exist! • Manage via emergent macroscopic properties and operational complexity should dramatically reduce • Conversely virtualisation ‘strategies’ never decrease operational complexity Source - Richard Nicholson - Paremus https://adaptevolve.paremus.com/?p=18 FITE Club (London) www.paremus.com April 2010 Copyright © 2010 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 11. Components: Why? FITE Club (London) www.paremus.com April 2010 Copyright © 2010 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 12. Components: The need to Abstract! as change isolate change occurs modules and their dependencies Source - Kirk Knoernschild (Burton Group) http://techdistrict.kirkk.com/2010/02/26/osgi-devcon-slides/ FITE Club (London) www.paremus.com April 2010 Copyright © 2009 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 13. Components: Something is Missing? ? Reuse Release Equivalence: Unit of reuse is the unit of release! Source - Kirk Knoernschild (Burton Group) http://techdistrict.kirkk.com/2010/02/26/osgi-devcon-slides/ FITE Club (London) www.paremus.com April 2010 Copyright © 2010 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 14. Components: Something is Missing? Reuse Release Equivalence: Unit of reuse is the unit of release! Source - Kirk Knoernschild (Burton Group) http://techdistrict.kirkk.com/2010/02/26/osgi-devcon-slides/ FITE Club (London) www.paremus.com April 2010 Copyright © 2010 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 15. Components: Benefits - reus e - re du ce co m - ease plexity mainte - incr nance ease exten sibility Reduces Complexity HOW? • OSGi Modularity - enables rapid ongoing code refactoring - essential for driving out Accidental Complexity • OSGi Modularity - provides the required Source - Kirk Knoernschild (Burton Group) http://techdistrict.kirkk.com/2010/02/26/osgi-devcon-slides/ abstraction, masking Necessary Complexity FITE Club (London) www.paremus.com April 2010 Copyright © 2010 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 16. Components: Benefits - reus - re du e Increases architectural agility! ce co m - ease plexity mainte - incr nance ease exten sibility Reduces Complexity HOW? • OSGi Modularity - enables rapid ongoing code refactoring - essential for driving out Accidental Complexity • OSGi Modularity - provides the required Source - Kirk Knoernschild (Burton Group) http://techdistrict.kirkk.com/2010/02/26/osgi-devcon-slides/ abstraction, masking Necessary Complexity FITE Club (London) www.paremus.com April 2010 Copyright © 2010 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 17. Components: Modularity is not a new story “First they ignore you, then they ridicule you, then they fight you, then you win.” -- Mahatma Gandhi Source - Kirk Knoernschild (Burton Group) http://techdistrict.kirkk.com/2010/02/26/osgi-devcon-slides/ FITE Club (London) www.paremus.com April 2010 Copyright © 2010 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 18. Components: Modularity is not a new story “First they ignore you, then they ridicule you, then they fight you, then you win.” -- Mahatma Gandhi OSGi is a disruptive technology that will transform how enterprise Java applications are designed, developed, and managed! Source - Kirk Knoernschild (Burton Group) http://techdistrict.kirkk.com/2010/02/26/osgi-devcon-slides/ FITE Club (London) www.paremus.com April 2010 Copyright © 2010 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 19. Components: Business Driver 90% of application lifetime cost concerns maintaining and evolving the Production Systems Modularity is a MUST FITE Club (London) www.paremus.com April 2010 Copyright © 2010 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 20. Clouds FITE Club (London) www.paremus.com April 2010 Copyright © 2010 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 21. Cloud: The bandwagon is rolling! What are the critical characteristics required of any credible Cloud solution? FITE Club (London) www.paremus.com April 2010 Copyright © 2010 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 22. Cloud 1. An Elastic Resource Landscape • All resource is transient • Resource landscape will expand / contract over time • Resource characteristics will change over time FITE Club (London) www.paremus.com April 2010 Copyright © 2009 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 23. Cloud 2. Simplicity through Abstraction • Manage Populations - not Instances • Manage Business Systems - not Components FITE Club (London) www.paremus.com April 2010 Copyright © 2009 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 24. Cloud 3. Robust & Recovery Oriented A system accident is an "unanticipated interaction of multiple failures" in a complex system. This complexity can either be technological or organizational, and often has elements of both. (Normal Accidents - Perrow 1984) Complex Systems tend to suffer correlated & cascading failures! FITE Club (London) www.paremus.com April 2010 Copyright © 2009 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 25. Cloud 3. Robust & Recovery Oriented A system accident is an "unanticipated interaction of • multiple failures" in a complex system. This complexity can Agile Systems - Are Robust Systems either be technological or organizational, and often has • Robust Systems - Are Agile Systems elements of both. • Self-Forming / Auto-Repairing (Normal Accidents - Perrow 1984) • Optimised for MTTR not MTBF • No single point oftend to suffer correlated failure (SPoF) • No single point of Complex Systems reference (SFoR) & cascading failures! FITE Club (London) www.paremus.com April 2010 Copyright © 2009 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 26. Cloud 4. Must understand Composite Systems Deployment via VM images NOT GOOD ENOUGH Remember - Reuse Release Equivalence: Unit of Release is the unit of Reuse! FITE Club (London) www.paremus.com April 2010 Copyright © 2009 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 27. Cloud 5. Is Multi-Tenant Isolation / Visibility - control at all levels of structural hierarchy / abstraction: • Groups of Systems • Groups of Composites within a System FITE Club (London) www.paremus.com April 2010 Copyright © 2009 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 28. Cloud 6. Is Architecturally Invisible The Cloud runtime while providing options should not constrain or enforce: •Communication protocol •Middleware Services •Data Services (CAP) •Caching Services (CAP) •Or type of development framework FITE Club (London) www.paremus.com April 2010 Copyright © 2009 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 29. Cloud 7. Support Adaptive / Reactive Applications Dynamically assemble, and if necessary re-assemble, in response to changes in operational / business environment: •Rapid patch / fix and roll back capabilities •In future? - Dynamic component re-wiring to account for co-locality - Policy based roll-back based on System behaviour - Self-Tuning / SLA aware Service assembly behaviours Deployment via VM images IS DEFINITELY NOT GOOD ENOUGH FITE Club (London) www.paremus.com April 2010 Copyright © 2009 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 30. The Paremus Service Fabric FITE Club (London) www.paremus.com April 2010 Copyright © 2010 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 31. The Paremus Service Fabric SaaS (1..m) “Systems” may run upon a single Service Fabric IaaS (1..n) “private Resource Clouds” may contribute to a Service Fabric FITE Club (London) www.paremus.com April 2010 Copyright © 2009 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 32. Atlas Resource Management ‘Target State’ driven resource acquisition Service Fabric ! Production "#"Cloud ! Blue$ "&"Cloud ! Red$"os%name!Darwin$$$ • Ongoing discovery • To join a Service Fabric - Atlas agent must have appropriate X509 certificate • Only resource with correct characteristics are assimilated into Service Fabric • Resource may leave at any point in time Create a “Red” & “Blue” Service Fabric FITE Club (London) www.paremus.com April 2010 Copyright © 2009 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 33. Service Fabric - Infrastructure Services Registry Provisioner Service Advertisements System Managers Principles: • No ‘special’ nodes • Node ‘Roles’ may/do change over time C A Techniques: B • Dynamic Group formation & subsequent membership • Dynamic leadership election • Eventual Consistency across group members Repository Management & OSGi bundles System Descriptions Monitoring Nimble Policies WAR EAR General artifacts FITE Club (London) www.paremus.com April 2010 Copyright © 2009 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 34. Service Fabric - A Model-Driven Runtime A model of the desired business service is submitted. The Service Fabric dynamically deploys all required business and infrastructure components. System Running System Description Abstraction Corresponding ‘Form’ runtime Entity FITE Club (London) www.paremus.com April 2010 Copyright © 2009 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 35. Service Fabric - Model-Driven Management The Service Fabric continually compares the runtime structure with its corresponding model Target State Runtime State =1 =3 =1 Deploy FITE Club (London) www.paremus.com April 2010 Copyright © 2009 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 36. Service Fabric - Model-Driven Management The Service Fabric continually compares the runtime structure with its corresponding model Provision Delta Target State Runtime State Planned Deltas =1 e.g. Configuration M!del changes =3 Runtime Target State Entity "Structure#SLA$ Unplanned Deltas =1 e.g. Resource failures Bundle Bundle Bundle Bundle Bundle Bundle Bundle Bundle Bundle Bundle Bundle Bundle Bundle Bundle Bundle Bundle Bundle Monitor Bundle FITE Club (London) www.paremus.com April 2010 Copyright © 2009 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 37. Service Fabric - Model-Driven Management The Service Fabric continually compares the runtime structure with its corresponding model Provision Delta Target State Runtime State Planned Deltas =1 e.g. Configuration M!del changes =3 Runtime Target State Entity "Structure#SLA$ Unplanned Deltas =1 e.g. Resource failures Necessary Complexity Bundle Bundle Bundle Bundle Bundle Bundle Bundle Bundle masked by ‘Nimble’ Bundle Bundle Bundle Bundle Bundle Bundle Bundle Bundle Bundle Monitor Bundle FITE Club (London) www.paremus.com April 2010 Copyright © 2009 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 38. Service Fabric - Model-Driven Management To change a complex runtime System simply change its model! Target State Runtime State =1 =5 =1 Re-Configure FITE Club (London) www.paremus.com April 2010 Copyright © 2009 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 39. Service Fabric - Model-Driven Management To change a complex runtime System simply change its model! Provision Delta Target State Runtime State Planned Deltas =1 e.g. Configuration changes M!del =5 Target State Runtime Unplanned Deltas Entity "Structure#SLA$ =1 e.g. Resource failures Monitor FITE Club (London) www.paremus.com April 2010 Copyright © 2009 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 40. Anatomy of a Business System System Target Scaling Behaviour SCA Structure Resource Contract State (Replication Handlers) = (os.name=linux) & (CPU.speed > 3 Ghz) = = fl(x) + = fm(y) = fn(z) + = !(os.name=Windows) = (cost_center=engineering) Composites & Scale behaviour Resources Bindings required FITE Club (London) www.paremus.com April 2010 Copyright © 2009 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 41. Dynamic Scaling in response to Load Store input stream a-e CEP Processing u-z Store input stream a-b CEP Processing y-z FITE Club (London) www.paremus.com April 2010 Copyright © 2009 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 42. Scoping & Visibility System " $Group A% System ! $Group A% System # Level Description A service at composite visibility is only bound to Composite references within the same composite A service at system visibility is visible to any System reference in a composite in the same system A service at group visibility is visible to any reference Group within composites in systems in the same group A service at fabric visibility is visible to all references Fabric within the fabric A service at fabric visibility is visible to all references Public within the fabric FITE Club (London) www.paremus.com April 2010 Copyright © 2009 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 43. Development The Service Fabric does not mandate any particular IoC framework. A System may consists of any mix of: • Java POJO’s • OSGi Declarative Services • OSGi Blueprint • iPojo • Spring and Spring DM • Google Guice / Peaberry • Scala (or any other language that targets the JVM) FITE Club (London) www.paremus.com April 2010 Copyright © 2009 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 44. System Architecture - Communication Service Fabric does not enforce any particular architectural ‘form’ • Messaging middleware / protocols are the concerns of each System; via choice of bindings and infrastructure components • SEDA, ESB, Broker, P2P patterns are all possible • Low latency, batch grid, order matching & transactional message based Systems may be concurrently supported on the same Fabric FITE Club (London) www.paremus.com April 2010 Copyright © 2009 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 45. System Architecture - Data / Caching Service Fabric does not enforce an particular data model. A true No-SQL Platform as a Service (PaaS): • Unstructured data processing - Hadoop • Key / Value - Voldemort • Column - Cassandra • Graph Database - Neo • Relational - Derby, MySQL FITE Club (London) www.paremus.com April 2010 Copyright © 2009 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 46. A self-configuring PaaS Service Fabric - ‘I need to deploy a Servlet containers for this WAR based service!’ EAR EAR war Avoid middleware lock-in. Avoid bending you business systems to the dictates of monolithic middleware. The Paremus Service Fabric is a PaaS runtime that dynamically assembles and configures itself around the requirements of each business service it hosts FITE Club (London) www.paremus.com April 2010 Copyright © 2009 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 47. What to know more? • Talk to Paremus. • on OSGi? Join UK OSGi Forum - http://uk.osgiusers.org/Main/HomePage • on OSGi? Buy the Book - http://www.manning.com/hall/ FITE Club (London) www.paremus.com April 2010 Copyright © 2009 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.
- 48. Thank You www.paremus.com/servicefabric www.paremus.com/nimble http://felix.apache.org/site/apache-felix-sigil.html FITE Club (London) www.paremus.com April 2010 Copyright © 2009 Paremus Ltd. May not be reproduced by any means without express permission. All rights reserved.