Follow John Willis on Twitter: @botchagalupe
Follow Damon Edwards on Twitter: @damonedwards
Notes:
Data science teams need people with the skills and curiosity to ask the big questions.
http://radar.oreilly.com/2011/09/building-data-science-teams.html
People You May Know (PYMK) LInkedin, Facebook
Netflix and Zynga
Google, Amazon,
A recent report from the McKinsey Global Institute says that by 2018 the U.S. could face a shortage of up to 190,000 workers with analytical skills.
http://tech.fortune.cnn.com/2011/09/06/data-scientist-the-hot-new-gig-in-tech/
http://www.dataspora.com/2011/09/data-scientists-or-data-composers/
-------------------------------------------------------------------------
New CycleCloud HPC Cluster Is a Triple Threat: 30000 cores, $1279/Hour, & Grill monitoring GUI for Chef
We have now launched a cluster 3 times the size of Tanuki, or 30,000 cores, which cost $1279/hour to operate for a Top 5 Pharma. It performed genuine scientific work -- in this case molecular modeling -- and a ton of it. The complexity of this environment did not necessarily scale linearly with the cores.
c1.xlarge instances 3,809
cores 30,472
RAM 26.7-TB
AWS Regions 3 ( us-east, us-west, eu-west )
Compute Years of Work 10.9 years
Spot Instances at an average cost of 0.286 USD / instance / hour (0.036 USD / core / hour). Compare that to the 0.68 USD / instance / hour for the same On Demand instance. That’s 57% savings!
-------------------------------------------------------------------------
What Exactly is Complex Event Processing Today?
http://blog.cloudeventprocessing.com/2011/09/18/what-exactly-is-complex-event-processing-today/
Colin Clark...
-------------------------------------------------------------------------
https://github.com/nathanmarz/storm/wiki/Rationale
Storm is a distributed realtime computation system. Similar to how Hadoop provides a set of general primitives for doing batch processing, Storm provides a set of general primitives for doing realtime computation. Storm is simple, can be used with any programming language, and is a lot of fun to use!
The lack of a "Hadoop of realtime" has become the biggest hole in the data processing ecosystem.
-------------------------------------------------------------------------
Building a Devops team
http://agilesysadmin.net/building-a-devops-team
Brian Henerey, from Sony Computer Entertainment Europe.
First interview - remote technical test
Instructions...
Ec2 instance .. install Wordpress with a broken Mysql install
Tomcat log scraping...
Using screen to watch them...
Round 2 - Face to face interview
Whiteboard test
Pair programming
-------------------------------------------------------------------------
How to Think Like a Computer Scientist
http://greenteapress.com/thinkpython/thinkCSpy/
Learning with Python
-------------------------------------------------------------------------
Node.js and MongoDB on Ubuntu
http://cloud.ubuntu.com/2011/09/node-js-and-mongodb-on-ubuntu/
haproxy to catch inbound web traffic and route it to our node.js app cluster
mongodb for app storage
With a sample application....
-------------------------------------------------------------------------
First steps with Cloud Foundry on Amazon EC2
http://www.cloudsoftcorp.com/blogs/first-steps-with-cloud-foundry-on-amazon-ec2
Setting up an IP address and domain name
Making it start the right modules at boot