Devops Drop 014
Thursday, September 8, 2011 at 6:41AM 
Follow John Willis on Twitter: @botchagalupe
Follow Damon Edwards on Twitter: @damonedwards
Notes:
-
The Last Lean Startup Bundle: 48 hours to claim $3,000,000 in prizes
http://www.startuplessonslearned.com/2011/09/last-lean-startup-bundle-48-hours-to.html
One book 750 worth of stuff. the 5 and 10 books kinda not so good...
HP Launches Private Beta Of Cloud Compute, Storage Services
started a twitter discussion. I pointed out that I am pretty sure (certain) that they did not use opsware as a the deployment tool of choice...
meta point is that second gen CM tools are dead.. Tivoli, Bladelogic, Opsware... #puppet and #chef rule the cloud... no turning back...
Fault Tolerance and Protection
http://www.kitchensoap.com/2011/09/08/fault-tolerance-and-protection/comment-page-1/#comment-8339
GN&C FAULT PROTECTION FUNDAMENTALS
Thinking outside the box...
-----------------------
Autometrics: Self-service metrics collection
http://engineering.linkedin.com/52/autometrics-self-service-metrics-collection
lnked in system
uses zookeeper for coordination.
Kafka
- 500k+ metrics collected in a production data center every minute or about 8800 per second.
- The average number of metrics per service is about 400, although some services have thousands
- 1 minute resolution is maintained for 30 days, 5 minute for 90, 2 years of 1 hour resolution.
Amazon deploys every 11.6 seconds
Jon Jenkins
Amazon.com
June 16, 2011
http://www.youtube.com/watch?v=dxk8b9rSKOo
http://news.ycombinator.com/item?id=2971521
some gems in here.
nov 10 2010 all amazn web servers went ec2...
Makes a great argument for utility computing (i.. cloud)
Apllo deployment system.
one moth stats 11.6 depolys per second, peak 1079 in one hr,
10k avg sim deploys
30k peak.
But even their tooling reflects decoupling. Every tool follows the self-service model ("YOU do what you WANT to do with YOUR stuff"). Their deployment system (named Apollo, mentioned in the slides) and their build system, and their many other tooling, all reflect this model.
Cons. What happens is that you might be reinventing the wheel at Amazon. Often. Code reuse is very low across teams. So there's no shared cost of ownership at Amazon, more often than not. It's the complete opposite at Google w.r.t. code reuse. There are many very high-quality libraries at Google that are designed to be shared. Guava (the Java library) is a great example.
Another con. You may not know what you're doing. But as a team you will still build a rickety solution that gets you to a working solution. This is the result of giving a team complete ownership: they'll build what they know with what they have. Amazon is slowly correcting some of these problems by having teams own specific Hard Problems. A good example is storage systems.
And a lack of consistency is a common issue across Amazon. Code quality and conventions fluctuate wildly across teams.
Overall, Amazon has figured out how to decouple things very well.
-----------------------
Data scientist: The hot new gig in tech
http://tech.fortune.cnn.com/2011/09/06/data-scientist-the-hot-new-gig-in-tech/
A recent report from the McKinsey Global Institute says that by 2018 the U.S. could face a shortage of up to 190,000 workers with analytical skills.
