Wednesday, April 23, 2014

5 things you should ask about any graph or figure

1) Do the data come from questions asked of people, a records database, or observation

Note: some "findings" reported in the media are judgements of committees

2) What is the actual statistic(s) reported?  And how are they displayed (e.g., over time, grouped). In other words, what are the x and why axis?

3) Was there an experimental manipulation? Or is this just a presentation of "observational" or "correlational" data? If the latter what other things could correlate with the trend or difference that the author is trying to assert?

4) Does the finding match with other findings you know. If it doesn't, that doesn't mean it's wrong.  How solid and thorough was your previous knowledge anyway?

5) Is the change or difference "statistically significant" or outside of what would be expected by chance (through some accepted metric of variability quantification)? Most popular media reports will not address this. Every scientific report should.

No comments:

Post a Comment