APIs Inside Enterprise - SOA Displacement?

Editor's Notes

1. Creative Commons Attribution-Share Alike 3.0 United States License
2. In the beginning there were apps. Actually, before the beginning there were digital computers, and then mass storage to store system state And pretty soon people wanted to set out formally codified rules that modified the state in formal ways. Data and logic. We used to call these “two tiered systems”. Nobody says that any more. What was the first system you worked on Greg? ….I did a bunch of numerical programming in the engineering program I was part of. And also I was a hobbyist so I wrote code just for fun. But the first job I got paid for was a two-tiered app. It was in my college years, a dBase II project. dBase was a sort of integrated environment with the data and logic all melded together. It was a order and customer management system for an investment brokerage company. I paid a lot of bills working on that system. A ton of people did. The world ran on these kinds of systems.
3. Here’s a look at the computer from my first commercial job. 8-inch floppies! Double-sided! 64k RAM !This thing made a noise that sounded like a delivery truck. It was so loud.
4. Applications were useful, so they proliferated. And it was good. But then the problem became interconnecting these things. The system that handled loan origination needed to integrate with the system that was used for business planning, or portfolio management. And that’s a problem we’ve never really 100% solved. That problem continues to this day, just on different scales. About loan origination - I just talked with a company that has a twelve year old system that does loan origination. They’ve run their business on this system for more than a decade. Custom built. They also built a portfolio management system – once the borrower signs for the loan, it becomes part of the portfolio. It needs to be managed, checked, verified. But: The two systems don’t share a common customer model. Also - There’s no feedback mechanism in the portfolio management back to origination. If there are patterns, if for example loans made for mini-golf places are more risky than loans made for car washes, there’s no way to feed that information into the risk rating during the origination process. This is typical. Happens everywhere. Even in “modern” enterprises.
5. There were a number of different approaches taken for interconnecting different systems. Numerous. But in 2002 to 2005, we all settled on SOA. What’s your view on SOA, Greg? Is it real? Useful? ….(basically, yes. It got overhyped and over-productized. But it works as a model)…. SOA is an approach for modeling large systems as services. In contrast to objects. Anyone remember CORBA? DCOM? Those were distributed object models. Very popular. Objects were everywhere. OOPSLA was born. With the distributed objects model, cooperating applications were remote objects, and programmers could interconnect by invoking methods on remote objects. SOA’s different. Rather than invoking methods on objects, SOA said these remote things (applications) were really services, and we would send them messages. This was a profound switch. SOA’s Only Architecture
6. At that point, we modeled each standalone system as a service. And each service accepts one or more different sorts of messages. The list of messages a service could receive could be called the “contract” – and it was rigidly specified in the form of WSDL. We, as an industry, spent a lot of time focusing on the messages, describing them formally, describing how to map in-memory representations to on-the-wire, textual representations of messages. Trying to be very thorough. XML Schema. XML Data Binding. JAXB. XML namespaces. The effort we as an industry put into figuring out how to handle these messages. An astounding amount of effort. It amazes me. The XML Infoset. An attribute on an XML element can be namespace qualified. Remember that? A namespace with a trailing slash is different than a namespace with no trailing slash. Namespaces look like URLs but aren’t really. -----“Services” can also be clients of other services. Extending that model we have a connected web of services. They all interoperate by sending and receiving messages according to the defined contracts. It was all about re-use of systems, with tight control. Primarily system-to-system integration. And while all of this was happening, the web was maturing.
7. The term “Bus” is an abstraction, an analogy. It refers to the hardware bus – the bus that moves data between CPUs and memory. But it can be more than an abstraction. It can be concrete. Companies deploy ESB products, to facilitate the control and management of the SOA interconnections. Logging, controlling the access.
8. The price of enterprise network has plunged steadily and continues to decline. It used to be expensive to acquire and provision a 10mb ethernet switch. Now 1gbethernet is yesterday, and 10gbe is standard. Network is plentiful.Compute has followed a similar trajectory. It continues to drop in cost. Storage likewise. There was a time when people actually had to “reserve time” on a shared computer. Now, compute power is cheap and getting cheaper. And at the same time, the iphoneHere’s the thing: SOA was designed with the old constraints in mind. The formality of SOA messages was necessary, because compute was still scarce, and network was relatively scarce or expensive. Proper system architecture had to consider the cost and availability of these resources, which meant we needed asynchronous queues in front of systems, so they would not be overloaded. This implied that the messages on those queues needed a great deal of formality, so they could be stored with all their context, indefinitely. This way, when re-constituted from the queue, everything was still there. Time of origination. Identity of the sender. Identities of other intermediaries. Response path.Many people shied away from using HTTP as a transport inside corp networks for system-to-system connection, because of the overhead of an http request, as compared to TCP. Have you ever heard of :”Chargeback”? There was a time when shared resources on a corporate network were metered, and if your system used those resources, your department was charged for that usage. Like a toll road. But we don’t have those constraints now. Now most enterprises have so much network capacity, it makes no sense within a corporation to charge for it. It’s like air. It also makes no sense to reject HTTP for its “overhead” in most casts. And there is less imperative to introduce asynchrony. Compute power is so inexpensive that most requests can be fulfilled RIGHT NOW, and systems can connect to each other directly. It’s more efficient this way.
9. One more huge change: Mobile computing.
10. APIs address a set of requirements that are different than those addressed by SOA.If you think of iterations in IT architecture, Some might say SOA was a necessary step for us to take. At this point though, we’ve got requirements being driven by the ubiquity of mobile computers, and the opportunity to be accessible in new ways to people who carry those computers with them at all times of the day and night. SOA wasn’t designed for that. There are a bunch of hard feature/functions that come from that.Rate limitingOauthAuditingAnalyticsQuota managementVersioningDeveloper enablement and onboarding
11. Design Metaphors are just ways of thinking about modeling a problem. For a while we used Objects as a metaphor. EJBs –distributed objectsSOA – services Now, APIs are the metaphor we use. It’s THAT important. The metaphor gets us to think differently about framing problems. It’s not just lip service.Thinking differently guides the solutions we build. What’s different? The Focus on Programmability. It’s in the NAME, fer pete’ssake. The loosecontracts. They’redescribed in human documentation form, or they’re not describedat all. JSON for universalaccess. A simple data format. YAGNI. Loosecontractsmeansthingscan change often, and they do! Change leads us to iterations, whichbegetsagility. Change of the interface issimply not something SOA wasaimed at. In fact the mantra for SOA waswas: change the implementation, and keep the interface exactly the same.
12. Andreesen QuoteExamples: Google for newspaper ads, classified ads, magazines.Napster for Tower RecordsEmail for USPSCoursera for Education
13. Not quite the same thing.But APIs and Cloud are driving the opportunity in software. Examples:Expensify– a cool app for working people who submit expense reports. You can snap a photo of a receipt, then upload it to an expense report, all in a few seconds. So easy. Pay tolls through your smartphone – Today they have these magic toll bridges that take a photo of your license plate then send a dead-tree bill to your residence. That’s cool software! But wouldn’t it be great to be able to pay tolls automatically via a smartphone, billed right to your wireless account? A phone knows where it is, and could connect directly with the bridge, in realtime.
14. So APIs are eating software. The opportunity is there to API-enable virtually every enterprise. Should people turn off SOA? Should we un-SOA and un-ESB enterprises in order to put APIs in? No. Obviously not. The right approach is to use APIs as a Façade.(The old patterns are the best.)To connect to systems in your SOA, you can stand up an API in front of it. An API as a proxy. If your services are not readily consumable with a simple transformation, then you can “mash up” services, or compose them, to use the old word. This to achieve an element of function that is the right size for consumption on mobile devices.
15. DNA isn’t 100% deterministic. (We don’t think so, anyway.) You cannot say for certain that a person will be of a certain height or weight, based on DNA alone. But DNA can give a big head start! Yao Ming: If your parents are each 7 feet tall, you’re more likely to be tall, and thus to be successful as a professional basketball player. If you can get API mojo into internal systems, you stand to gain agility.It’s not just about “doing APIs”. It’s about shifting the organization to “think in APIs”. To think about programmability, and iterative evolution, and and gaining insight from traffic, and so on .