Data Pluralism

Data pluralism does not mean 'anything goes' (just as pluralism does not mean 'anything goes'). It means we have to check our conclusions by asking if the result depends on the way we have interpreted the data. So if we have two definitions of GDP then a conclusion is only robust if it yields the same qualitative result for both definitions. Otherwise it is 'data-interpretation-dependent' and is not fully reliable. So this is more stringent, not less stringent, than data-monotheoreticism

Data pluralism also does not mean we rely only on data. It means that, to the extent that a researcher (or a professor, or a student) produces conclusions that depend on data, then the results are not robust unless she has checked them against other data that claims to measure the same thing, but is quantitatively different. This often occurs: for example, PPP measures of GDP or 'real GDP' measured by traditional means. These give different conclusions about international inequality.

Also, "data is theory-laden" is a gross over-statement; most philosophers of science now reject it. Theory has some influence on observation, but many scientific theories are based primarily on observation (including experiment). How the blood circulation works (and the rest of physiology). That dolphins are mammals not fish, and bats are not birds. The germ theory of disease in all its various forms. Tectonic plate theory. I could go on …

And know who created it, how and why

I think we need to walk a fine line here. I agree that data at face value can often lead to misinterpretation of the data if we do not understand what we are really measuring (such as the GDP versus happiness debate). However, if we focus too much on "data relying on theory" then we have opened the door to "believing whatever data we like, based on the theories we like, and dismissing the data we don't like." If taken too far, we will lose our ability to perform objective, empirical analysis.

Data can be very powerful and is clearly open to abuse and misuse even if it is not the full picture.

I know that the University of Copenhagen have specific modules on how to extract, treat and apply data from the central bank, national statistics office, etc. This could be supplemented with more qualitative sources of data as well. This seems powerful to two ways, first allowing the student to explore data and dissect data (as pointed out by Francis and Alan), and secondly provide robust skills to the students.

Data is theory-laden

Thanks John and Michael. Two points. (1) some reasonably precise economic measurements can be made, for example total sales. This is a money magnitude; one simply adds up the receipts. But when this finds its way to the national accounts it is reconstructed (according to a theory) so that, eg, the NA figure for depreciation is very different from what the firm records. (2) therefore not sure there is a 'correct' order - the issue is 'what will persuade a practitioner that a theory is wrong?'

Yes but, most important stuff going on right now scapes, in my opinion, that which is measurable by data. We need (as real economist who really care about what's going on) to go beyond the measurable. Truth, and this is the most radical thing i have to say, lies beyond facts: truth is an emotional phenomenon and thus this is another reason why feelings need to be permitted, included, emphasized and encouraged

I find that there is a theoretical way of assembling the facts on which the data can then be compared. But it is likely to be so general as to hide some of the detail obtainable from the measured details in this data, and consequently the theory is not of much value here. Let us first concentrate on the "why" of the theory and later attach what facts and data are pertinent, rather than the reverse.

Data pluralism is crucial. I use it in the sense of multiple *types* of evidence, including economic history, surveys, experiments, case studies, etc. Statistical analyses usually only show *that* a causal relationship exists (and quantifies it), but not "how" it works - the mechanism. See https://evidence-based-economics.org/ (section A) for more.

Thanks Mike Joffe for two interesting points. But the second point seems more a point against ..? Could instantiate the philosophers of science who reject the idea that observation is theory-laden? To me, your examples refute your own point. To decide if a bat is a bird, we have first to define a bird. So if I define a bird as a 'flying creature' then yes, a bat is a bird. And if I define a mammal as a creature that does not lay eggs, then a platypus is not a mammal.

It is an iterative process between theory and evidence. For more on this, see my paper at https://bit.ly/2ENFccK. For a fairly recent review of the philosophy of science discourse, see the article by James Bogen in the online Stanford Encylopedia: https://stanford.io/2E1TWCw - though this article would be better if it also considered natural sciences such as biology and geology, not just the physical sciences.

Thanks Alan. In natural sciences, definitions play little part - rather, it is about classifying what is observed to exist. A good classification, e.g. "mammal", makes sense of all the evidence, whereas its predecessor "quadruped" obscured the mammalian nature of bats and dolphins. Your examples show why this gives better results, and why observations are not wholly theory laden. If you insist on a definition, you are rigid. If your classification can respond to evidence, you make progress.

Note, however, that actually, there are very few data in economics in the sense of actual measurements such as a car's velocity. Population is counted, ok, but the labor force is not. Who is in it is a matter of definition. So economic "data" are not carved in stone: they are manipulated to serve a purpose. I believe that the true unemployment rate in the U.S. is twice the official rate. So all the analysis based on the official rate are not good. The CPI is manipulated using hedonic pricing.

David, I am puzzled why you start with the theory and its "why", then look for the pertinent data and facts. From the viewpoint of natural science methodology, this is in the wrong order.

That's a really important point John, thank you. My take is that the economy is a multi-causal system, so the causes considered should include the reasons why unemployment rates understate the true figure. Similarly with CPI. I wonder how generally true it is. My current empirical work is on industry profit rates, and on historical prices. I need to think whether they are affected. Of course in a time series or panel analysis, the manipulation would need to vary over time to distort the results.