Netflix in the Cloud
•
23 likes•7,444 views
[Full slides now also available at http://www.slideshare.net/adrianco/netflix-on-cloud-combined-slides-for-dev-and-ops] Short summary of why Netflix is running on the Amazon cloud, what is running there, what we have learned and where this is taking us. This is the introduction section to a series of public presentations that will go into much more detail. The Silicon Valley Cloud Computing Meetup was on Oct 14th, QCon San Francisco November 3rd.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to Netflix in the Cloud
Similar to Netflix in the Cloud (20)
More from Adrian Cockcroft
More from Adrian Cockcroft (6)
Recently uploaded
Recently uploaded (20)
Netflix in the Cloud
- 1. Ne#lix in the Cloud Oct 14, 2010 Adrian Cockcro: @adrianco #ne#lixcloud h=p://www.linkedin.com/in/adriancockcro:
- 2. Beta Slides "I didn't have -me to write a short le4er, so I wrote a long one instead.” Mark Twain Teaser intro slides now at slideshare.net/adrianco Oct 14th Beta #devops subset -‐ Cloud CompuKng Meetup Nov 3rd GA – QConSF and full slides on slideshare.net
- 3. With more than 15 million subscribers in the United States and Canada, Ne#lix, Inc. is the world’s leading Internet subscripKon service for enjoying movies and TV shows. .
- 4. Synopsis • Why Give This Talk? Why Use AWS? • The Goals – Faster, Scalable, Available and ProducKve • Datacenter AnK-‐pa=erns – the things we wanted to change and why • Cloud Architecture Features • Cloud Bring-‐up Strategy • Developer TransiKons and Tools • Roadmap and Next Steps
- 5. Why Give This Talk? • Ne#lix is Pathfinding – Cloud ecosystem is evolving very fast – Share with and learn from the cloud community • We want to use clouds, not build them – Cloud technology should be a commodity – Public cloud and open source for agility and scale • We are looking for talent to help… – How do we connect with the very best engineers? – h=p://www.quora.com/10X-‐Engineers
- 6. Why Use AWS? • We stopped building our own datacenters – Capacity growth rate is acceleraKng, unpredictable – Product launch spikes -‐ iPhone, Wii, PS3, XBox – Datacenter is large inflexible capital commitment • Leverage AWS Scale – the biggest public cloud – AWS investment in tooling and automaKon – AWS zones for high availability, scalability – AWS skills are common on resumes… • Leverage AWS Feature Set – most advanced – EC2, S3, SDB, SQS, EBS, EMR, ELB, ASG, RDB..
- 7. “The cloud lets its users focus on delivering differen-a-ng business value instead of was-ng valuable resources on the undifferen)ated heavy li0ing that makes up most of IT infrastructure.” -‐ Werner Vogels August 25, 2009, ‘All Things Digital’
- 8. Ne#lix Deployed on AWS • “batch” Movie Encoding farm (2009) – Thousands of EC2 instances, Petabytes of S3 – Movie files staged out to CDNs for delivery • Hadoop -‐ ElasKc Map-‐Reduce (2009) – Large scale log processing and analyKcs • Streaming Service Back-‐end (early 2010) – Highly available and scalable “play bu=on” • Web site, a page at a Kme (through 2010) – Ramp from 25% of views to 80% in the coming weeks • API for TV devices and iPhone etc. (2010) – Personalized movie choosing algorithm back-‐end
- 9. Learnings… • Datacenter oriented tools don’t work – Ephemeral instances, high rate of change • Cloud Tools Don’t Scale for Enterprise – Built our own tools, drove vendors hard • “fork-‐li:ed” apps don’t work well – Fragile, too many datacenter oriented assumpKons baked in • It’s faster in the end to re-‐code than Knker – Re-‐architected and re-‐wrote much of our code base – Fine grain web services – Leveraging open source in Java – SystemaKcally instrumented – “NoSQL” SimpleDB backend. In the datacenter, robust code is best prac-ce. In the cloud, it’s essen-al.
- 10. Next Few Years… • “System of Record” moves to Cloud – Cut the datacenter to cloud replicaKon link • InternaKonal Expansion – Global Clouds – Rapid deployments to new markets • GPU Clouds opKmized for video encoding • Cloud StandardizaKon – Cloud features and APIs should be a commodity not a differenKator – DifferenKate on scale and quality of service – CompeKKon drives cost down – Higher resilience – Higher scalability We would prefer to be an insignificant customer in a giant cloud
- 11. Takeaway NeIlix is path-‐finding the use of public AWS cloud to replace in-‐house IT for non-‐trivial applica-ons with hundreds of developers and thousands of systems. h=p://www.linkedin.com/in/adriancockcro: @adrianco #ne#lixcloud