Devops Drop 026

Direct download

Follow John Willis on Twitter: @botchagalupe
Follow Damon Edwards on Twitter: @damonedwards 



Untethered, CFEngine hitches a lift on Android




The latest chapter of this story is the touching of two branches of this freedom: the internet information ecology and mobile communications. We are talking, of course, about smartphones and their heavier brethren `pads'. What about configuration management of these devices?


-encryption of memory,

-pin code or password protection,

-software updates, 

-application version control, are pressing.


Open Source android does not allow the notion of a privileged monitor on which most management frameworks are built.


 CFEngine has an andriod prototype.. 


Android 2.2 introduces support for enterprise applications byhttp://developer.android.com/guide/topics/admin/device-admin.html






The next five years of cloud and client computing, Mark Shuttleworth, Ubuntu - YouTube



Mark Shuttleworth....  Get past the initial introductions in French.. then initial talk about tablets and smart devices... About 1/2 of the way in the data center...


Lingua Franca-  http for api's, Iaas for cloud, openstack as the apache of cloud.


Leadership is key factor from spinning out of control.. 


Virt platofrms... 

virt ... kvm .... standard virt platform... 

lxc .. containers ... (trade isolation fro performance)  trusted workloads choice lxc... 


Ubuntu ... Openstack work with LXC...


Cloudfoundry.... savy move... generic harness for PaaS... bridging layer...   

Openstack.. Cloudfoundry ... 


Efficiency... ARM... ARM in the datacenter.  Ubuntu 11.10 first widely release Ubuntu that will treat ARM as a first class

power density, performance density.. 


Human scalability...

  Ops became much more of an eng/dev opposed to maint.. server rations up 10 to 20x


1 to 10k or higher in the future... 


How to scale in non flat orgs (legacy IT) .. Chef and Puppet... Spray it... 


Devops... Defining the language that we speak.. turnover... 


New service deployment... Config doesn't address.  Which 10k machines need that change..what order


Key driver .. scaling CM.. "Service Orchestration"  blurs the line between IaaS and PaaS.. Juju.. Ubuntu means Humans. Juju means super natural humans..


pkg metaphor...  for services... like a package knows where all the pieces for a software pkgs need to go juju knows where all the 


In the next five years everything will change except the beauty of paris... 





Multi-Platform Messaging with RabbitMQ


Rob Harrop 


5 different Languages and two different protocols...  


Java2days 2009 - Java Concurrency in 5 Languages ( Scala, Java, Erlang, Clojure, Groovy).


Rob Harrop demoes how to use RabbitMQ from a variety of languages (Java, Python, Ruby and Erlang) and different environments using AMQP and STOMP to achieve for multi-platform communication.


4 Languages... 

Java, Python, Ruby, Javascript, Erlang, and Telnet....


RabbitMQ is naitivly multiprotocal . stomp, amqp. .... 


Great overview of RabbitMQ... 


Most important thing in RabbotMQ is it is not JMS.   Forget what you know about JMS and re-discover...



Java Consumer to Pyhton publisher... w/AMQP

Java Consumer to Erlang publisher... w/AMQP



STOMP Adapter (ActiveMQ) 

Ruby Consumner Telnet publisher 


Then gets xrazy... 

Multiple languages ruby, telnet, NodeJS... 




NodeJS and SocketIO to link RabbitMQ and Node together... 




Recommend book for Javascript 



JavaScript: The Good Parts


Devops Drop 025

Direct download

Follow John Willis on Twitter: @botchagalupe
Follow Damon Edwards on Twitter: @damonedwards 



Got open source cloud storage? Red Hat buys Gluster



-Red Hat’s $136m acquisition of open source storage vendor Gluster marks Red Hat’s biggest buy since JBoss

-GWOS and Zenoss rumors... 

-Given its recent quarterly earnings report and topping the $1 billion annual revenue mark, Red Hat seems again to be bucking the bad economy.




1 Million User Party: Mon Oct 17 in SF


Openbar till 11pm... Wish I could be there.... 





2012 Fukuoka Ruby Award Competition & FOG


Wesley Beary







Grand Prize (1 Million Yen)

Outstanding Performance Awards (100,000 Yen)




Ubuntu and the Cloud...



Ubuntu Orchestra


JUJU using LXC on a Laptop .. can run it in virtulization.. openstack or EC2.... 


In the front of the stage, we have a portable rack of servers (a 40-core Intel Emerald Ridge, a 24-core HP Proliant, a 16-core Dell Precision, with a System76 local Ubuntu mirror, and Cisco networking hardware).  We've used Ubuntu Orchestra to remotely install the systems, and we've deployed OpenStack to the rack.  Once OpenStack is running, Clint has a series of Hadoop jobs that he spins up and runs against dozens of instances on the local Nova compute node.  And for the real whiz-bang, Clint uses gource for dynamic visualization of the Hadoop cluster, the various nodes, and their relationships.  It is absolutely stunning to behold!


To get started download the image from:



Ubuntu 11.10 Oct 13...




Open Stack Summit Summary...


Matt Ray... Bell of the ball.... I worked my ass off to hire him at Opscode... 

Ironic story... 


User adoption is really taking off, and we’ll be hearing from MercardoLibre, CERN, Sony Computer Entertainment America, Fidelity, Disney, and NeCTAR Summit...


Rackspace’s intention to form an OpenStack Foundation in 2012



-organizations backing OpenStack is now well over 110, including industry giants such as Cisco, HP, and Dell.

-Clouscaling, Piston and Nebula

-Techstars recently announced an incubator program for Cloud start ups

-User adoption is really taking off, and we’ll be hearing from MercardoLibre, CERN, Sony Computer Entertainment America, Fidelity, Disney, and NeCTAR

-HP using Openstack huge contributor... 

-They also plan on posting their own Chef cookbooks and apparently they have their own crowbar like implementation. 

-Dell w/Crowbar was giving demos... 


Complete list for blueprints...




Git support for gt/jenkins....





Windows Registry-based Idempotence Check with Chef



registry_helper.rb is used to check for an already installed product... 




Puppet vs. Chef, Fight!





If you aren't yet using Puppet or Chef for managing your *nix infrastructure, you should seriously consider it.


The Criteria


-Community Strength


-Corporate Adoption

-Technical Merits

-Hands-on Experience


Community Size


Number of people on IRC on Sunday Oct 9th, 2011 10 AM GMT

#puppet   541 users

#chef       233 users


Mailing Lists

puppet-users  3363 members, 5961 topics

chef: 600 members, # of topics unknown


Quality of Cookbook Sites





Github Repositories:

puppet:  1622 repositories

chef:      1299 repositories



 Pro Puppet, by Puppet Labs engineer James Turnbull, is an awesome introduction to Puppet. I found it immensely helpful. There is a 50-page O'Reilly publication on Chef, Test-Driven Infrastructure with Chef, but it cannot match the breadth of Pro Puppet...




Luke... I spoke to Luke .. he told me he actually put a lot of efford into becoming a CEO... 

Scott Johnston new VP or Product and Marketing

Director, Product Management, New Business Ventures atCisco Systems

Director, Customer Operations at Loudcloud

Director, Product Management at Geocast Network Systems

Director, Product Management, Web/Application Servers atNetscape

 Partner at Alloy Ventures




Opscode now has Mitch Hill: Chief Executive Officer

8 guys to almost 1bill exit...


Opscode has battery and Puppet kliener perkins


Devops Drop 024

Direct download

Follow John Willis on Twitter: @botchagalupe
Follow Damon Edwards on Twitter: @damonedwards 



New - Amazon S3 Server Side Encryption for Data at Rest



Amazon S3 Server Side Encryption handles all encryption, decryption, and key management in a totally transparent fashion. When you PUT an object and request encryption (in an HTTP header supplied as part of the PUT), we generate a unique key, encrypt your data with the key, and then encrypt the key with a master key. For added protection, keys are stored in hosts that are separate and distinct from those used to store your data. 



chef.vim is plugin which make it easy for



-jump between attributes and recipes

-jump between LWRP's providers and resources

-open recipes by extract filename from include_recipe

-open templates and files

-jump to attributes

-jump to definition

-jump to LWRP







Stats! { 'collaborators' => 16, 'downloads' => 245,745, 'forks' => 260, 

'open_issues' => 13, 'watchers' => 1521 }


-MVP! Patrick Debois 







How Ruby is beating Python in the battle for the Soul of System Administration




- In many ways it is the love child of perl and smalltalk.

  • Ruby's greatest strength is its amazing flexibility.
  • Python intentionally has minimal magic. It's greatest strengths are the best practices it enforces across its community.




-For CM, you missed bcfg2 which is written in Python.













Using Chef to Customize Multi-Node Cloud Foundry Deployments 




Chef-Solo Install





LEAN STARTUP, by Eric Ries






Operations is a Strategic Weapon (PuppetConf)






OpenStack Diablo is a Quantum Leap for Open Source Cloud 




-Web-based dashboard for managing and provisioning OpenStack-based clouds.







HBase @ Facebook 



-25 TB per month of msgs and Chat data... 

-Previously stored in Mysql

-Looking for a system with high wrote throughput with good reads... 

-Ironic because they actually invented Cassandra..

-Need to scale gracefully... 





Riak 1.0 Is Officially Released!









This package is to make deployment and testing of mod_security easier with chef. Right now it centers entirely around the OWASP Core Rule Sets of mod_security rules. In future, it will allow you to manage/deploy custom rule/rulesets of your own.




DevOps is DOA



Engine Yard co-founder Tom Mornini weighs in on whether #PaaS is the #DevOps killer. Hmmm. zd.net/mRqy5I


Also CA.. 





Gartner Says Worldwide Platform as a Service Revenue is on Pace to Reach $707.4 Million in 2011



Worldwide platform as a service (PaaS) revenue is on pace to reach $707.4 million in 2011, up from $512.4 million in 2010, according to Gartner, Inc. The market will experience consistent growth with worldwide PaaS revenue totaling $1.8 billion in 2015.


DevOps Cafe Episode 20

Live onstage with the masters of the Puppet.

Luke Kanies (Puppet Labs)
James Turnbull (Puppet Labs)
Teyo Tyree (Puppet Labs)
Scott Johnston (Puppet Labs) 

L to R: John Willis, Teyo Tyree, Luke Kanies, Scott Johnston, James Turnbull, Damon Edwards


Direct download

Follow John Willis on Twitter: @botchagalupe
Follow Damon Edwards on Twitter: @damonedwards 


  • This episode was recorded live on stage at PuppetConf 2011 in Portland, OR!!


Please leave comments or questions below and we'll read them on the show!


Devops Drop 023

Direct download

Follow John Willis on Twitter: @botchagalupe
Follow Damon Edwards on Twitter: @damonedwards 



Hack Day in China...



Minimalist pipeline.. CI loop, CD model... 

Customer was Centos based.... 

SVN,Jenkins, Rundeck, JBoss and Mysql... (5 Servers).  

Due to cultural differences....

Make a change to the code and see the CI loop, delivery loop take place.  

The do a Roll back....


Talk about Rundeck and the mechanics - Jenkins Rundeck Plugin.  Drives the rundeck job automatically.  Also Rundeck pulls the latest, last and history of builds.  


Normally we would use Chef or Puppet on the back end of Rundeck... 


Point about manual vs automated process




A Contentious Question by Chris Hoff (@beaker)



Contentios question by @beaker.. no,  Actually if you guys don’t follow @beaker you should.  He is our #devops mirror for networking and security (imho).  He tried to keep us honest...


“Given the recent influx of virtual networking solutions, many of which are OpenFlow-based, what possible in-roads and value can they hope to offer in heavily virtualized enterprise environments wherein the virtual networking is owned and controlled by VMware?”


Specifically, if the only third-party VMware virtual switch to date is Cisco’s and access to this platform is limited (if at all available) to startup players, how on Earth do BigSwitch, Nicira, vCider, etc. plan to insert themselves into an already contentious environment effectively doing mindshare and relevance battle with the likes of mainline infrastructure networking giants and VMware?


If you’re answer is “OpenFlow and OpenStack will enable this access,...

Not meaning to piss anyone off, but many of these startups’ business plans are shrouded in the mystical vail of “wait and see.”





Pro Puppet [Kindle Edition]







Ask about BigDrops?


 Learning and Hadoop




September 2011 – HUG– Atlanta, GA

Machine Learning With Hadoop

Josh Patterson | Sr Solution Architect


Open source work at 







After the refining process, one barrel of crude oil yielded more than 40% gasoline and only 3% kerosene, creating large quantities of waste gasoline for disposal.”

--- Excerpt from the book “The American Gas Station”

Hadoop Today: The Oil Industry Circa 1900


Digital universe grew by 62% last year to 800K petabytes and will grow to 1.2 “zettabytes” this year


Speed @ Scale is the new Killer App

Results in that previously took 1 day to process can gain new value when created in 10 minutes.


ML Focused on in Mahout


An algorithm that looks at a user’s past actions and suggests





What is time series data?


What is ISAX Lumberyard?




Packages For Hadoop



UDFs in Pig

used at LinkedIn in many of off-line workflows for data derived products

"People You May Know”






Hadoop Not Good At in Data Mining


We’ve talked about this before with twitter's storm...

Not everything fits great in MapReduce

Mahout as evidence of this






Common Challenges in DevOps Change Management



Matt Ray


Starts out with meat and potato .. everything in source control


Talks about spiceweasel .. yaml or jason..



Think roles not nodes... nodes can be ephemeral.. 

Start from scratch don’t reuse nodes...

Don’t hard code IP addresses .. m&p

Golden mages are an anti-pattern.  <The don’t have to be...




Three Drunk in SysAdmins....



Continuous Delivery of Server Configurations


Puppet, Git, Mcollective,Jeninks,and Capistrano


Three part series... 






Announcing the MongoDB Monitoring Service (MMS)


MongoDB Monitoring Service (MMS) to the public for free. MMS is a SaaS based tool that monitors your MongoDB cluster and makes it easy for you to see what’s going on in a production deployment.





Devops Chicago and Devops Camp




Interview w/Martin J. Logan


Oct 22 - 23 in Chicago


Camp Devops 




Devopsdays Goteborg 2011

The conference will be a two day event on Friday 14 and Saturday 15 of October 2011