Devops Drop 023

Direct download

Follow John Willis on Twitter: @botchagalupe
Follow Damon Edwards on Twitter: @damonedwards 

Notes:

Hack Day in China...

http://kb.dtosolutions.com/wiki/Duke%27s_Bank_J2EE_Jenkins/Rundeck_example

Minimalist pipeline.. CI loop, CD model... 

Customer was Centos based.... 

SVN,Jenkins, Rundeck, JBoss and Mysql... (5 Servers).  

Due to cultural differences....

Make a change to the code and see the CI loop, delivery loop take place.  

The do a Roll back....

Talk about Rundeck and the mechanics - Jenkins Rundeck Plugin.  Drives the rundeck job automatically.  Also Rundeck pulls the latest, last and history of builds.  

Normally we would use Chef or Puppet on the back end of Rundeck... 

Point about manual vs automated process

--------------------------------------------------------------------------

A Contentious Question by Chris Hoff (@beaker)

http://www.rationalsurvivability.com/blog/?p=3304

Contentios question by @beaker.. no,  Actually if you guys don’t follow @beaker you should.  He is our #devops mirror for networking and security (imho).  He tried to keep us honest...

“Given the recent influx of virtual networking solutions, many of which are OpenFlow-based, what possible in-roads and value can they hope to offer in heavily virtualized enterprise environments wherein the virtual networking is owned and controlled by VMware?”

Specifically, if the only third-party VMware virtual switch to date is Cisco’s and access to this platform is limited (if at all available) to startup players, how on Earth do BigSwitch, Nicira, vCider, etc. plan to insert themselves into an already contentious environment effectively doing mindshare and relevance battle with the likes of mainline infrastructure networking giants and VMware?

If you’re answer is “OpenFlow and OpenStack will enable this access,...

Not meaning to piss anyone off, but many of these startups’ business plans are shrouded in the mystical vail of “wait and see.”

--------------------------------------------------------------------------

Pro Puppet [Kindle Edition]

http://www.amazon.com/gp/product/B005PZ28R8/ref=as_li_ss_tl?ie=UTF8&tag=hardeninglinu-20&linkCode=as2&camp=217145&creative=399373&creativeASIN=B005PZ28R8

--------------------------------------------------------------------------

Ask about BigDrops?

 Learning and Hadoop

http://www.slideshare.net/jpatanooga/machine-learning-and-hadoop

September 2011 – HUG– Atlanta, GA

Machine Learning With Hadoop

Josh Patterson | Sr Solution Architect

Open source work at 

http://openpdc.codeplex.com 

https://github.com/jpatanooga 

Lumberyard“

https://github.com/jpatanooga/Lumberyard 

After the refining process, one barrel of crude oil yielded more than 40% gasoline and only 3% kerosene, creating large quantities of waste gasoline for disposal.”

--- Excerpt from the book “The American Gas Station”

Hadoop Today: The Oil Industry Circa 1900

Digital universe grew by 62% last year to 800K petabytes and will grow to 1.2 “zettabytes” this year

Speed @ Scale is the new Killer App

Results in that previously took 1 day to process can gain new value when created in 10 minutes.

ML Focused on in Mahout

An algorithm that looks at a user’s past actions and suggests

Products

Services

People

What is time series data?

What is ISAX Lumberyard?

Bioinformatics

Packages For Hadoop

DataFu

http://sna-projects.com/datafu/ 

UDFs in Pig

used at LinkedIn in many of off-line workflows for data derived products

"People You May Know”

"Skills”

Techniques

PageRank

MRv2

Hadoop Not Good At in Data Mining

We’ve talked about this before with twitter's storm...

Not everything fits great in MapReduce

Mahout as evidence of this

--------------------------------------------------------------------------

Common Challenges in DevOps Change Management

http://www.slideshare.net/mattray/common-challenges

Matt Ray

Starts out with meat and potato .. everything in source control

Talks about spiceweasel .. yaml or jason..

http://wiki.opscode.com/display/chef/Spiceweasel

Think roles not nodes... nodes can be ephemeral.. 

Start from scratch don’t reuse nodes...

Don’t hard code IP addresses .. m&p

Golden mages are an anti-pattern.  <The don’t have to be...

--------------------------------------------------------------------------

Three Drunk in SysAdmins....

http://www.threedrunkensysadsonthe.net/2011/05/a-new-hope/

Continuous Delivery of Server Configurations

Toolbox

Puppet, Git, Mcollective,Jeninks,and Capistrano

Three part series... 

--------------------------------------------------------------------------

10Gen

Announcing the MongoDB Monitoring Service (MMS)

MongoDB Monitoring Service (MMS) to the public for free. MMS is a SaaS based tool that monitors your MongoDB cluster and makes it easy for you to see what’s going on in a production deployment.

--------------------------------------------------------------------------

Devops Chicago and Devops Camp

http://dev2ops.org/blog/2011/9/26/devops-chicago-and-devops-camp.html

Interview w/Martin J. Logan

Oct 22 - 23 in Chicago

Camp Devops 

--------------------------------------------------------------------------

Devopsdays Goteborg 2011

The conference will be a two day event on Friday 14 and Saturday 15 of October 2011

Devops Drop 022

Direct download

Follow John Willis on Twitter: @botchagalupe
Follow Damon Edwards on Twitter: @damonedwards 

Notes:

Gartner Cites Application Release Automation Tools as Key to DevOps

http://blog.noliosoft.com/gartner-cites-application-release-automation-tools-as-key-to-devops/

So many things wrong with this...   (ARA) Nolio 

 As an emerging movement, DevOps may have improved communication and collaboration between development and IT operations teams but it still hasn’t absolutely mastered its ultimate goal of unifying their work. 

 In his report, Ronni J. Colville argues that DevOps can be greatly enhanced by the use of application release automation (ARA) tools and specifically cites Nolio as one such solution.

Can tools help “Cuture”  this question was asked at Puppetconf w/Luke... 

Visibility also requires the establishment of a model of the application and its configuration for each environment. ARA tools can provide a mechanism to create application models for each environment, with externalized configuration settings that typically vary by environment.

---------------------------------------------------------------------

Selenium and Nagios

http://devops-abyss.blogspot.com/2010/06/selenium-and-nagios.html

I've implemented a Nagios check for Selenium test cases. With this check it is possible to put your recorded test cases from your Selenium IDE into Nagios to use them for monitoring.

Test-->Selenium IDE-->Export-->check_selenium (nagios plugin)-->Selenium Remote Control

---------------------------------------------------------------------

DevOps in Milliseconds

http://techblog.appnexus.com/2011/devops-in-milliseconds/

AppNexus engineers have it good. 

They don’t lie awake at night wondering if we can handle the next increase of impressions. 

They don’t worry that our systems are down and we don’t know it. 

They don’t develop in a bubble, toss their code over the wall to a mysterious group of people, and wash their hands clean.

Monitoring: Nagios - 1200 services

Metrics: Graphite - 1 million datapoints every minute.

Nagios plugin that queries Graphite and alerts if values of certain metrics go above or below specified thresholds.

Deployment: Puppet and Maestro

Puppet backed by a MySQL database and fronted by an in-house application we call Maestro.

At AppNexus there is no wall between engineers and operations, and automation is crucial to scaling our infrasctructure. 

Engineers control their own destiny, and we give them the tools to dive deep into production problems and give them tools to dive deep into production problems, make fixes, and improve their products as quickly as they can code.

---------------------------------------------------------------------

CI vs Zombies 

http://www.build-doctor.com/2011/09/26/ci-vs-zombies/

Runaway builds.

--A runaway build occurs when not all processes created by the build exit cleanly. 

--Zombies – may hang the build, or simply stay around in the background waiting to wreak havoc. 

--They interfere with test isolation. If processes can hang around from an earlier build (or earlier test within the same build) they may affect unrelated tests.

--difficult-to-diagnose failures.

-- eventually leading to exhaustion.

--Manual intervention is required to kill them and clean up. 

---------------------------------------------------------------------

Openstack Compute API v1.1 support

https://github.com/geemus/fog/pull/525

Implement fog support for the Openstack Compute API v1.1. Includes support for legacy v1.0 style auth and v2.0 keystone auth.

---------------------------------------------------------------------

Mean time to pretty chart- DevOps meets data porn

http://gigaom.com/cloud/mean-time-to-pretty-chart-devops-meets-data-porn/

Alex Benik is a principal at Battery Ventures. Battery Ventures is an investor in DataDog and Tracelytics. 

The current mantra in Web operations is to track, record and monitor everything. Data is valuable and storage is cheap.

Favorite Velocity  John Rauser at Amazon and Kellan Elliot-McCrea from Etsy.

Mean Time to Pretty Chart (MTPC). For full buzzword compliance, let’s say that WebOps + BigData + Information/Graphic Design = MTPC.

MTPC attempts to quantify the amount of time required to determine the root cause of an operational issue and depict it in an eye-catching way. The MTPC metric is challenging because it encompasses a number of challenges spanning large volumes of data acquisition, storage, correlation and design/representation.

http://tracelytics.com/

http://www.datadoghq.com/

http://www.nodeable.com/

A highly incomplete list of relevant commercial and open source tools would include Ganglia, Nagios, Cati, Graphite, Munin, Splunk, New Relic, Tracelytics (see disclosure), DataDog (see disclosure), and AppDynamics.

Enter the Data Scientist. While correlation doesn’t imply causation, with large enough sample sizes the old adage “where there is smoke there is usually fire” often applies. When you can visualize that smoke in a pretty chart, it’s easier to pinpoint the fire.

---------------------------------------------------------------------

Jesse Robbins interview on DevOps Cafe #19 (w/ full transcript!)

http://www.opscode.com/blog/2011/09/22/jesse-robbins-interview-on-devops-cafe-19-w-full-transcript/

Devops Drop 021

Direct download

Follow John Willis on Twitter: @botchagalupe
Follow Damon Edwards on Twitter: @damonedwards 

Notes:

Puppet Conf Recap ...

Great show.. first class.. venue, food, content...

“Operations as a Strategic Weapon”

Damon and I did our combined talks right after Luke’s Keynote.  I thought we rocked.  They will be posting the videos

http://www.slideshare.net/botchagalupe/operations-as-a-strategic-weapon

http://www.slideshare.net/botchagalupe/operations-as-a-strategic-weapon-part-2

Devops Cafe Roundtable with Luke, Teyo, James and Scott  ..

Basically the management team at Puppet Labs

Scott story about joining Puppetlabs... His Loudcloud experience. 

Damon killed.  We talked about Service Orchestration, PaaS, culture patterns.. great stuff... We will post the audio on Devops Cafe site and the Video should be up in a week or two...

---------------------------------------------------

Puppet Enterprise 2.0

http://puppetlabs.com/puppet/puppet-enterprise/?htb1

A lot of new integration with Mcollective and the GUI...

New GUI, right out-of-the-box PE 2.0 automatically discovers all resources – packages, hosts, groups, and users.  Uses Mcollective to discover. 

Visually Clone Resources To Scale Quickly, Efficiently, and Reliably(From the GUI)

With PE 2.0’s new  compliance capability, you now can visually monitor for any unauthorized changes against your desired-state baseline. Can run compliance reports once a day and watch for changing trends...  Give auditors GUI control to see what they need to see...

PE 2.0’s new provisioning capability allows you to quickly and easily create new instances of VMware and Amazon EC2.  KInd of like “Knife” with the added bare metal sauce... 

---------------------------------------------------

“Operating at Scale”

Pedro Canahuati

SRE Manager... 

Dealing with issues at “SCALE” and I mean scale....

Switched from XEN to LXC to to overhead at scale...

Been using cFengine for years... About to change to Chef or Puppet.. Looking at both. 

All the #devops thing are going on at FB  CD, Agile in operations, collect and store everything.  Like Google, they had to build a lot of their own stuff.  

They build there own TSDB kind of like Opentsdb.  They have built there own monitoring framework, looking framework (they use Scribe).

ODS tool the abstracts and visualizes all events (very cool) 

I was able to talk to Pedro at the speakers dinner and the following day.  I am a junkie and groupie for guys like this and stuff like .. we talked about CEP and monitoring.  Also about Chef and Puppet.  

---------------------------------------------------

Beyond the Node: Arkestration with Noah

John Vincent

---------------------------------------------------

Puppet and Juju, scaling the cloud

Marc Cluet & Adam Gandelman

These boys showed up to a gin fight with a knife... 

Slideware of how you can use puppet and Juju together.  I am not a mean guy unless you propose something that you can’t explain in a presentation...

Split brain... Needs to be a hackday .. talked to Dan Bodie about this... Interesting...

---------------------------------------------------

Mårten Mickos

Eucalyptus

CEO Eucalyptus

Great presentation... Talked about what the cloud has done to operations.  Also acknowledges cloud needs devops.... 

My Zing question ... great answer....

We also had some one on one podcasts with the Redhat guys about Openshift and how it works.  

Ended up with an interview with Jay Lyman of 451 group... Post on DTO....

Oh yeah  on the way to have drinks with Gene KIm I got to get my picture talked with Merle Haggard.  

Devops Drop 020

Direct download

Follow John Willis on Twitter: @botchagalupe
Follow Damon Edwards on Twitter: @damonedwards 

Notes:

Goteborg 2011 - program

http://devopsdays.org/events/2011-goteborg/program/

Friday 14 October and Saturday

Yours truly doing the keynote...

----------------------------------------------------------------------

Announcing Xeround Cloud Database API

http://xeround.com/blog/2011/09/xeround-cloud-database-api

Xeround is an elastic, always-on database-as-a-service
for your MySQL applications.

AWS, Rackspce and Heroku

Benchmarks against an RDS Large at  $0.44 vs the $0.08 standard instance Xeround

----------------------------------------------------------------------

How GitHub Uses GitHub to Build GitHub

http://zachholman.com/talk/how-github-uses-github-to-build-github

Everyone can push, everyone can deploy 

Master is always deployable

Deploy 10 to 40 times a day

Pull requests are our code review

Master -> Branch -> Pull request -> Master

Pull requests are RAD no meeting, email is your interface, non techs get involved

Culture...   Hack days... make things fun... 

Hubot, our valiant Campfire bot, has continued to grow in complexity. A tiny list of his (current) capabilities:

-unlock the door to our office

-print out a list of the people currently in the office based on their wifi presence

-find an apartment in the area to rent

-deploy GitHub

-say an arbitrary string over the office speakers

-play an audio sample of deadmau5 to everyone through hacked Propane HTML5

-give you a quote from any movie or TV show

-tell you the build status of any git branch

-track and map packages

-SMS any GitHubber from Campfire

-embed a seven day weather forecast

----------------------------------------------------------------------

PuppetConf as a Service (PCaaS): Sign up for the Free Live Stream

http://puppetlabs.com/blog/puppetconf-as-a-service-pcaas-sign-up-for-the-free-live-stream/?utm_campaign=blog&utm_medium=socnet

Mårten Mickos

SRE’s from Facebook and Google

John Vincent @lusis Noah dude

Luke of course

Adrian Cole jClouds

Chad Metcalf Cloudera

Jinesh Varia AWS

Mark Hinkle @mrhinkle

----------------------------------------------------------------------

Puppet Change Management for DevOps

What is Puppet?

At Atlassian, we use Puppet extensively with our internal systems, our Hosted products, and our build engineering infrastructure. Here's how we do it in build engineering.

Jira, Bamboo,  Greenhopper Rapid Board

Bamboo with puppet...

----------------------------------------------------------------------

IBM Infrastructure as a Service (IaaS) -

http://www-935.ibm.com/services/us/igs/cloud-development/?S_CMP=swnews_twitter#tab:fallpromo/#leadspace:default

From September 12 – November 11, you can provision select virtual machines at the Toronto, Ehningen, Tokyo and Singapore IBM SmartCloud data centers—subject to availability—at no charge. You can access:

Virtual machines to run Linux® (Red Hat or Novell SUSE) or Microsoft® Windows® Server 2003/2008

1 block (256 gigabytes) of persistent storage

----------------------------------------------------------------------

DataStax gets $11M, fuses NoSQL and Hadoop

http://gigaom.com/cloud/datastax-gets-11m-fuses-nosql-and-hadoop/

Brisk, Hadoop based on Cassandra

Neo raises $10.6M for Neo4j as graph DBs take off

http://gigaom.com/cloud/neo-raises-10-6m-for-neo4j-as-graph-dbs-take-off/

----------------------------------------------------------------------

Building Scalable Systems: an Asynchronous Approach

http://www.infoq.com/presentations/Building-Scalable-Systems-Asynchronous-Approach

Node.js and rabittMQ

DevOps Cafe Episode 19

Love it or hate it, the long form interview is back! 

Guest: Jesse Robbins (Opscode) 

http://www.technologyreview.com/tr35/profile.aspx?TRID=1108

Direct download

Follow John Willis on Twitter: @botchagalupe
Follow Damon Edwards on Twitter: @damonedwards 
Follow Jesse Robbins on Twitter: @jesserobbins 

Notes:

• Learn more about Jesse's company, Opscode (makers of Chef)
• John and Damon will be speaking at and doing a live episode of this podcast from PuppetConf in Portland (9/22)

Please leave comments or questions below and we'll read them on the show!