Managing a Network with Cricket

Jeff Allen
WebTV Networks, Inc.

--
The Problem
Monitoring a network is hard:
? Short-term issues make us act reactively
? Need data that we often don't have to make good long-term decisions
? Lots of types of devices, operating at all levels of the protocol stack

--
Common Questions:
Short term:
? Is the link to Europe up?

--
Long term:
? Do we need more bandwidth to Europe?
Better Questions:
? What is the current state of the link? 
? What has it been recently? 
? Is it what we expect it to be?
? What long-term trends can we discern? 

Answering questions like these requires a good data collection and graphing system.

--
The System:			Cricket!
Cricket is a tool for storing and viewing time-series data.

? Very flexible
? Extremely Legible Graphs
? Space and Time efficient
? Platform Independent

--
How it works
? Cricket's collector runs from cron every 5 minutes and polls devices.
? Data is stored in the RRD files.
? Cricket's grapher CGI script is used interactively to browse the data.
Both the collector and the grapher rely on a hierarchical configuration system called a Config Tree.

--
Collection
Cricket gathers data from:
? SNMP
? Shell Scripts
? Files
? URL's
? Perl Procedures

--
Storing Data
Data lives in Round Robin Database (RRD) files.
? Automatically discards old data, maintaining constant size database.
? Automatically rolls-up data into summaries on various time scales.
? Uses binary format on disk for speed.

--
Graphing the Data
? Graphing is actually done by RRD - thus the very close resemblance to MRTG graphs.
? The graphs are useful because they have:
? Enforced data density
? Enough info to tell the whole story
? The Right Scale

--
The Config Tree
Hierarchical structure for config files:
? Uses inheritance to avoid repeated configuration info.
? Easy to add new targets.
? Easy to parallelize collector (and administrators).
? Could be used by other apps in the future

--
Future directions
? The capacity to draw 4500 graphs hardly qualifies as a proactive monitoring tool.
? Humans must check the graphs now.
? Wouldn't be nice if Cricket could check the graphs itself? How would a computer know if a graph "looks right"?
--
Future directions (cont.)
? Database export via text files
? Makes data live on, even though RRD deletes old data.
? Can use data mining techniques on the data
? Can check historical data against SLAs.
? Cricket could send traps to an Alert Manager
? For more about this, see my talk at LISA '99

--
Contact info:
Where: http://www.munitions.com/~jra/cricket
	You can download it for free from here (under the GPL), and find out about the mailing lists related to Cricket.

Who: Jeff Allen <jra@corp.webtv.net>