Devops Drop 018
Friday, September 16, 2011 at 7:28AM 
Follow John Willis on Twitter: @botchagalupe
Follow Damon Edwards on Twitter: @damonedwards
Notes:
NoSQL Benchmark
http://blog.cubrid.org/dev-platform/nosql-benchmarking/
Yahoo Cloud Servicing Benchmark
Basic operations are Insert, Update, Read, and Scan. There are basic workload sets that combine the basic operations, but new additional workloads can also be created.
This article contains tests conducted on the following products and versions.
Cassandra-0.7.4
Although Cassandra’s latest version is 0.8.0, we have decided to use the previous version known to be stable. Because when testing with the 0.8 version, the gossip protocol between nodes malfunctioned and the node up/down information was incorrect.
HBase-0.90.2 (Hadoop-0.20-append)
The HBase-0.90.2 (Hadoop-0.20-append) was selected because, if not the Hadoop-append version, there may be problems on decreased durability in HDFS.
MongoDB-1.8.1
Insert, Read Only and Read and Update
Insert - Cassandra kills
Read and update Cassandra beats HBase by a little
Read Hbase wins of course but only by a little against Cassandra
Mongo get blow out...
Which leads me into .. why I would love to make this event...
------------------------------------------------------
Using Cassandra, Brisk, and Mahout to Manage Time Series, and Predict Future Events
http://predictingfuture.eventbrite.com/
Datastax ... Brisk a cassandra based Hadoop...
------------------------------------------------------
What is glu?
http://linkedin.github.com/glu/docs/latest/html/index.html
glu is a free/open source deployment and monitoring automation platform.
a glu agent is running on each of those nodes
ZooKeeper is used to maintain the live state as reported by the glu agents (blue arrows)
the glu orchestration engine is the heart of the system
Glu Script is a Groovy Class with named closures for the actions... (can be groovy or java)
install, configure, start, stop, unconfigure and uninstall
The doc is pretty cool .. however, when I started getting into the state machine stuff I had to stop...
Orchestration .. Zookeeper to build live state, compare live and desired state.
generate delta
------------------------------------------------------
NODE.JS AND THE JAVASCRIPT AGE
http://metamarketsgroup.com/blog/node-js-and-the-javascript-age/
Three months ago, we decided to tear down the framework we were using for our dashboard, Python’s Django, and rebuild it entirely in server-side JavaScript, using node.js. (If there is ever a time in a start-ups life to remodel parts of your infrastructure, it’s early on, when your range of motion is highest.)
This decision was driven by a realization: the LAMP stack is dead.
1991-1999: The HTML Age.
2000-2009: The LAMP Age.
2010-??: The JavaScript Age.
------------------------------------------------------
From $0-100million with no sales people. The Atlassian 10 commandments for startups.
Jira, Confluence
3 ppl to 300 ppl...
Start with two founders.. 50/50
Bootstrapping .. first round is 60M
-Sell itself, affordable, global, open
-Use your own product.... Passionately use your own product...
-Measure everything... Capture everything.... even if you can’t analyze
-Test everything... 5 users free .. raised money for charity
-ABM... ... always sponsor the beer at conference.. like Dyninc...
-Send stuff in the mail.. t-shirts...
-Make everything into a campaign.. Turned hiring into a marketing campaign - .. send only 4 resumes otherwise you are black listed...
-Don’t be afraid to let your first product will fail..
------------------------------------------------------
Devops Dude of the Week....
Jordon Sissel
FPM and Logstash and now...
eventmachine-tail
https://github.com/jordansissel/eventmachine-tail
Jordon Sissel..
This project contains two EventMachine extensions.
First, it adds an event-driven file-following similar to the unix ‘tail -f’
command. For example, you could use it to follow /var/log/messages the same way
tail -f would.
Second, it adds event-driven file patterns allowing you to watch a given file
pattern for new or removed files. For example, you could watch /var/log/*.log
for new/deleted files.
For logstash, the log agents were
event-driven using EventMachine. The log agents mainly get their data from
logfiles. To that end, we needed a way to treat log files as a stream.
There’s a ruby gem ‘file-tail’ that implements tailing, but not in an
event-driven way. This makes it hard to use in EventMachine programs like
logstash.
Thus, eventmachine-tail was born.
Further, the usage patterns for logstash required the ability to watch a
directory (or a file pattern) for new log files.
rtail -x "*.gz" "/var/log/**/*"
-
Reader Comments