Two interesting comments recently that remind us to be careful what we are measuring and what we are reporting. Anything beyond the basic measurement of length, mass and value are always biased -- and someone could probably argue that these are biased too. Acknowledging the bias up front helps us make better decisions over the long run.
Corante: Social Software: Buzzmaker
Unfortunately, as you can see, the data is nearly worthless. It suffers from several classic mistakes -- the database was too small in the beginning to be worth using, collapsed again in August, and is being added to cumulatively, so all searches also collapse at the end. In addition, the word count is not indexed to overall posts, so even if occurrences of a word are falling in relative terms, it can still be presented as rising if its absolute numbers are going up.
Management by Baseball: Part Two -- The Seductions (& Giant Sucking Sounds) of Metrics
Each ball park has a set of factors (including distance to fences, altitude, relative humidity, prevailing wind direction and strength) that affect performance of players. The Colorado Rockies' park amplifies run production (more for left-handed hitter than right-handed), and because baseball statistics are like double-entry accounting, this both increases apparent hitter proficiency and diminishes apparent pitcher proficiency. For a few years, before other teams' front-offices caught on, the Rockies were able to trade away hitters with fab-looking numbers that pretty quickly faded when they were no longer playing half their games in Denver.
There are many, many other examples where we need to be careful:
What you measure is what you get.
"If you cannot measure it, you cannot improve it." Lord Kelvin.
"To measure your staff's productivity is not the same as to improve it." Bob Lewis, Inforworld, 1996.
Hawthorne effect: When you measure something, you influence it. This is one of the core aspects of Tim Gallway's The Inner Game series of books.