Graphing What Cannot Be

I've always liked John Tukey's work...

At the other extreme, they must, at almost the same time, be honest in assessing the uncertainties of their final results. In the latter they cannot be satisfied with allowance for only the likely size of "sampling errors", a task with which routine manipulations can often help them; they must, most particularly and responsibly, make explicit allowance for the likely size of "nonsampling errors", for the extent to which the data given to them was neither what it purported to be nor what it ought to have been. No other profession must support itself over so wide a span from security to insecurity. (Tukey, The Future of Data Analysis, 1965; 23-24)

Instead of studying and graphing what we know, Tukey suggested graphing what we know cannot be, and improving graphs, �by shifting emphasis from "what might be," inevitably truly fuzzy, to "what we know cannot be," which only has fuzzy edges� (Tukey, 1986, 73). He presents an evolution from the most certain estimate of what we know to the most certain estimate of what we don�t know: from best estimate, to the likely interval, to range of the impossible:


The difficulty we face (in economics) is that total error bounds are not known; nonsampling errors have not been investigated enough for me to come to any general empirical conclusions. For macroeconomic data charts like those above are mere guesswork.

We simply do not know enough about nonsampling error, nor are we likely to during my entire professional career. What stance am I to take towards data? How am I to use it seriously, without prejudice or bias? That�s it, I�m not really concerned with the bias in the data as much as I am with my own personal bias in understanding and using it. I want to know, if the unemployment rate is 5.4%, what kinds of stories using that number are real, and which are imaginary?


Powered by Movable Type 5.02

About this Entry

This page contains a single entry by Kevin published on January 13, 2005 5:24 PM.

Accuracy vs. Precision in Economics was the previous entry in this blog.

Arctic Refuge is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.