Monday, 20 May 2013

On consensus and dissent in science - consensus signals credibility

Since Skeptical Science published the Pac Man of The Consensus Project, the benign word consensus has stirred a surprising amount of controversy. I had already started drafting this post before, as I had noticed that consensus is an abomination to the true climate ostrich. Consensus in this case means that almost all scientists agree that the global temperature is increasing and that human action is the main cause. That the climate ostriches do not like this fact, I can imagine, but acting as if consensus in itself is in bad thing in itself sounds weird to me. Who would be against the consensus that all men have to die?

Also the Greek hydrology professor Demetris Koutsoyiannis echoes this idea and seems to think that consensus is a bad thing (my emphasis):
I also fully agree with your statement. "This [disagreement] is what drives science forward." The latter is an important agreement, given a recent opposite trend, i.e. towards consensus building, which unfortunately has affected climate science (and not only).
So, what is the role of consensus in science? Is it good or bad is it helpful or destructive, should we care at all?

Credibility

In a recent post on the value of peer review for science and the press, I have argued that one should not dramatize the importance of peer review, but that it is a helpful filter to determine which ideas are likely worth studying. A paper which has passed peer review, has some a-priory credibility.

In my view, consensus is very similar, consensus lends an idea credibility. It does not say that an idea is true; if formulating carefully a scientist will never state that something is true, not even about the basics of statistical mechanics or evolution, which are nearly truisms and have been confirmed via many different lines of research.

You could thus say that peer review of an article, is a first step on the credibility ladder. The longer an idea holds, the more people have invested more time in studied it from many different angles, the more credibility an idea gains. A consensus among the people that actually studied the problem is stronger than a consensus among people that did not. In this respect, I value the IPCC reports, where groups of experts described our current understanding, a better information source on the consensus as the counting of scientific articles performed in The Consensus Project. Most of the scientists acknowledging the climate consensus, will not have studied the problem in much detail.

Credibility, makes the probability that a theory is wrong smaller. Consequently, it also makes the honour of finding a problem larger. There is nothing more beautiful for a scientist as to destroy a consensus, no better way to show that you are a good scientist. In principle, the bigger the consensus, the better. However, the bigger the consensus, the more difficult is will also likely be to find a problem with it. Thus you need some confidence if you go after a big one. It is likely smart to build your scientific career on a mixture of problems with varying degrees of difficulty, only trying to destroy century old theories is a high-risk strategy. Next to making a portfolio, a productive strategy is to first build up evidence for a smaller question, before you go for the big kill.

Just as an expert can ignore peer-review and simply judge ideas from the non-reviewed literature himself, there is no reason whatsoever to respect the consensus opinion in the field you are working in. If you have a good idea and strong arguments why the consensus is wrong, go for it! If a new idea requires the consensus of one or even neighbouring fields to be wrong as well, the situation becomes complicated. That would require a lot of study to become expert in those other fields and quite likely in the end you will not find the hole you need in their consensus. If you have nothing else to work on or the consequences would be enormous, by all means try it, but I would not prioritize pursuing such an idea.

Dikran Marsupial summarises this section very well:
... the existence of a consensus is not scientific evidence of anthropogenic climate change (and nobody is claiming it is), the consensus is a result of the scientific evidence of anthropogenic climate change.

Standing on the shoulders of giants

It is a bon mot that we can only see so far because we are standing on the shoulders of giants. This foundation, an indispensable network of interlinked ideas, is the consensus of the preceding research. Not all of it is right, it does not have to, it should help in finding new fruitful questions that in the end help us understanding the world a little better.

If you want to add a new piece to this network, you may have to destroy a piece of the old network because it does not fit. The more "consensus" you would have to destroy, the less likely it is that your new idea will turn out to be an improvement. By the way, often you can also do something completely new, improve methodologies, find interesting consequences of known theories, etc. Thus often there need be no conflict whatsoever.

I have always wondered why the climate "sceptics" often claim that CO2 is of minor importance as a greenhouse gas. That would mean that either quantum mechanics or the radiative transfer (RT) equation should be wrong. The RT equation is so basic, that it if were wrong, also much of astronomy, remote sensing by satellites and many laboratory experiments would be have been based on the wrong equations. That is quite a consensus the destroy. Not impossible, but highly unlikely, especially by someone with almost no expertise in any of the fields.

It may take a bit more thinking to realise their importance, but my advice to the ostriches would thus be to look at climatic feedbacks, they are unique to climate science and thus have much less consensus behind them and are consequently much more likely to be wrong. Strong negative feedbacks have the potential to make the global temperature increase due to CO2 very small, which is all a climate ostrich needs. My personal favourite feedback would be the cloud feedback, Roger Pielke Sr. would probably mention land-surface feedbacks.

Encouraging dissent


Obey, photo by Peat Bakke, CC2.0 license.


Dissent in itself is neither good nor bad. Ideally a scientific paper should be so strong that even if someone initially held the opposite opinion, he can only agree that the article is right. In this respect, you should strive for consensus. In praxis, you will often need multiple papers to build up a case and answer criticisms you did not think of in the first paper. Thus in praxis there may be a phase with dissent, but I do not see it as something to strive for as a scientist.

Still, humans have a tendency to seek consensus and to copy opinions of high-status individuals. To counter this tendency, the scientific community should explicitly encourage dissent, naturally without relaxing our quality standards. Dissent, just for the fun of it, with unfounded arguments and rookie errors only hinders science.

Science and society are two very different realms. In a democracy it is much more important to encourage dissent. In a democracy there is not just one truth, different people have different interests. Even if the interests of everyone would be the same, the kind of questions that need to be negotiated are too complex to be proven optimal. Science done right, on the other hand, splits up problems in sufficiently small questions that have a clear answer. (This is also why governments by experts are bad, experts are not able to deal with this ambiguity, think in terms of one right answer and not experienced in building coalitions with groups of different interests.)

In science dissent is a sign that there is work to do. In a democracy it is the natural state.

Climate ostriches

Interesting is that that consensus is not opposite of dissent. As HotWopper noticed the climate ostriches do not agree about much:

WUWT deniers can't agree about what is happening to the climate (it's the sun, it's natural, it's not warming, it's cooling, we're heading for an ice age, it's thunderstorms, there is no greenhouse effect, it's cosmic rays, it's warming, it's not warming, it's ENSO, it's an ice age) - but when it comes to conspiracies they all agree that climate science is a hoax! A conspiracy! A scam! A SECRET scam!

While every ostrich seems to have its own alternative hypothesis, that is there is no consensus, there is also no dissent. Like a pack of wolfs the regulars at WUWT attack people who dare to claim that the global temperature is increasing and that humans are the main cause. However, if one of pseudoskeptics makes an obvious mistake, e.g. a clear misquotation, no one will criticise this, there is no sign of dissent.

If the climate ostriches would like to develop a better alternative hypothesis, which I doubt after reading WUWT for over a year, it is time the ostriches allow for dissent and start discussing the strengths and weaknesses of their ideas and not just the stupidity of scientists and scientific ideas.

Further reading

17 comments:

Daneel Olivaw said...

Also, the fact that there's a consensus about one question does not mean that there's no dissent in other areas. Knowing scientists (and studying to become one) I'm certain that, while the climatology community has reach an agreement about what causes global warming, they are still arguing about what's on the edges of our understanding.

Victor Venema said...

Good point. I had planned to state something like that, but the post was getting too long. I guess the discussion about the details is what helps the climate ostriches to argue that there is no consensus. And who knows, one day a discussion about a detail may escalate into a challenge to the entire theory. Unlikely, but possible, that is what makes science so fascinating.

Lars Karlsson said...

Without consensus, science would never be able to progress. We would be stuck arguing the most basic issues forever.

Consensus about one question means that we can go on to tackle other questions.

Wotts Up With That Blog said...

The issue of consensus is very interesting and it might be another example of where common usage of a term isn't quite the same as what is meant when used by, for example, scientists. I saw an interesting twitter debate today where people were arguing about whether or not evolution was a fact or a theory. Those who were clearly scientists argued that it is still a theory. Those who were more involved with the public suggested that this makes it seem as though it is still uncertain and hence we should really regard it as a fact. It seemed that people perceive the term theory in different ways.

Similarly, I think the term consensus is viewed differently by different people. Scientists recognise that it means that there is general agreement about the strength of the evidence but doesn't mean that everything is settled (as Daneel suggested above). Others might think that a consensus now means that everything's agreed and that scientists will no longer be probing this area in any detail. It might require educating people so that they recognise what these terms means when used by scientists. There might, however, be some who don't really want to learn this lesson.

Victor Venema said...

That could be a problem. What surprised me was that the Wikipedia page on Consensus is actually a page on Consensus decision-making. Thus people may also have the idea that there is social pressure to conform to find a consensus decision.

Does anyone know a better term?

It will be hard to stop scientists from trying to poke holes in a theory, no matter how established it is. :-) Probably needs the the threat of violence.

I keep on dreaming of destroying Quantum Mechanics one day. I do not like the observer in the theory and hope that we can somehow replace the observer with a complex system, which would be more objective. Which may make it a job for a climatologist familiar with complex systems.

Anonymous said...

Dictionaries are always helpful in these situations.

The dictionary definition of consensus is crystal clear.

It means majority opinion.

Thus, scientific consensus means the opinion of a majority of scientists.

As Dikran says, it is not evidence.

Why not?

As a matter of inviolate axiom running through the last 300 or so years of science, the age of the Modern Scientific Method so called, opinion is not evidence.

Unfortunately this debunks the title of this piece, "consensus signals credibility."

Only evidence signals credibility, and consensus is not evidence (it's just opinion).

So everything after the title (i.e. the entire piece) is falsely premised and should be ignored.

This article is not only valueless but (to any reader who actually believes it) misinformative.

Brad Keyes said...

Your whole premise, that "consensus signals credibility," is false by definition and your argument is therefore valueless.

Dikran is correct:

Consensus is not evidence.

This follows from, and is nothing more than a special case of, the axiom known to all scientists (with the possible exception of Mr Venema) which dictates that opinion is not evidence and can never be used as a form of evidence.

But in science, only one thing can signal credibility: evidence.

Evidence signals credibility. Only evidence signals credibility. Credibility is not signalled by anything other than evidence.

(If you don't agree with me, Victor, that's OK—but it means you're not a scientist.)

Consensus is not evidence, so it is absurd to think that it "signals credibility."

This article is wrong.

Take it down or it will keep disinforming people.

cRR Kampen said...

Victor, I may need some correction, for I have a saying: 'Credibility is a phrase to slap onto any source whose credibility is a given'. Whereby I hold that credibility as a concept has no place in science at all.

To find my way out of this, I'm trying 'plausibility'. I'd love some light from you on this idea/concept.

/cRR

Victor Venema said...

I am not a native speaker, so maybe I get this wrong.

Credibility sounds to me like having a good reputation. That you trust something before you have checked it. It is a trust that is earned in the past and not yet based upon checking this specific piece of information.

Before you call something plausible, you have to check it, not?

Consensus is important information for people that do not have the time, interest, skills or whatever to check the information. The others can make up their own mind by checking the information and can ignore the consensus. At least in principle, because you cannot check everything and have to place trust somewhere. The larger the consensus, the easier it would be for someone (else) to check the facts, the more easy it is to place trust.

Victor Venema said...

Is it necessary to respond to Anonymous/Keyes? I have no idea why he feels the need to attack this post and me. Okay, maybe I have, but it is not related to something being wrong with this post.

The text already states: "Dikran Marsupial summarises this section very well: ... the existence of a consensus is not scientific evidence of anthropogenic climate change (and nobody is claiming it is), the consensus is a result of the scientific evidence of anthropogenic climate change."

cRR Kampen said...

The etymological thing with 'credibility' is to do with belief.
'Credibility' translates to 'geloofwaardigheid' in Dutch and this would translate back to 'beliefworthiness'. This is not the kind of concept we would like to associate with science. The problem is, I feel, acute because consensus as such has often arisen over theses that are unfalsifiable to say the least or are de facto nonsense - here, consensus and belief system are the same thing and gone is any relation to evidence!

A considerable fraction of the public appears unable to get their head around this. So they respond to research showing climate scientific consensus to be huge, over 97%, as if to any election result of a dictator, or as if to some fundamentalistic church with totalitarian aspirations.

"Before you call something plausible, you have to check it, not?"

Yes, also, not all can be checked: plausibility implies incomplete knowledge. Wrt AGW much of the public is in this situation: they can, in principle, check some aspects but not others. I do believe the public can check enough for themselves to gain at least a sense of plausibility on the AGW hypothesis (or, rather, reality).

The reason for this exercise is to try and get rid of the word 'consensus' wrt scientific endeavour wholesale.

Brad Keyes said...

cRR,

Well put. "Credibility," like the words "recognised," "respectable" and "reputation," belongs to the lexicon of social proof, a pre-scientific system of knowledge/opinion.

Social proof was rendered obsolete by the adoption of the scientific method hundreds of years ago, and was abandoned and forgotten by people seeking to understand nature.

Good riddance to bad religion, I say.

Social proof concepts were reintroduced into polite conversation by climate "science" (a discipline related to, but not to be confused with, science). Thus it is not unusual for climate "scientists" like Dr Will Steffen, of the now defunct Australian Govenment Climate Commission, to emit sequences of words like:

"So if you go through that list of 31 thousand scientists I couldn't recognise any that I recognise as publishing in the range of literature that covers climate science[sic]. So the issue there is that list really doesn't carry any weight at all in the credible [climate] scientific community. They don't publish, they don't go to the conferences we do, they're not (interjection)—would you let me finish—so I think the issue there is that the credibility of the scientists involved, you earn that, you keep that by publishing in the peer-review literature."

To a scientist, of course, the above sounds like word salad, and a kind of parodic perversion of the last 300 years of epistemology. But that's because they're scientists. Only climate scientists are qualified to interpret Steffen's claims.

I have no idea why Venema feels the need to attack the natural sciences and me. Okay, maybe I have, but it is not related to something being wrong with science.

Brad Keyes said...

Victor, this is a fantasy:

"The larger the consensus... the more easy it is to place trust."

You just conflated how many scientists think that X with how confident scientists are that X!

That's the oldest, corniest rhetorical fallacy in the book.

Just because "98% of doctors are convinced that X" doesn't mean "doctors are 98% convinced that X."

You cannot tell how confident any individual scientist is from a vote of scientists.

Use your brain, Victor.

Brad Keyes said...

Lars,

This misconception about how science works (which Victor hasn't corrected, for some reason) is laughably wrong:

"Consensus about one question means that we can go on to tackle other questions."

The only scientists who can't seem to move on until everyone agrees with them are climate scientists. All normal scientific communities always work on multiple questions at once and NOBODY cares if they've CONVINCED everyone or not.

Do you literally imagine scientists must get together and vote to approve an answer before anyone is allowed to apply for a grant to work on a different question?

Because that's a delusion. You're thinking of the IPCC, not science.

Brad Keyes said...

Let's consider the myth,

"Scientific consensus comes from the evidence."

If opinion throughout the scientific community is evidence-based, then why isn't it unanimous?

In other words, if you're naive enough to believe "scientists's opinions are based on evidence," then how do you account for the fact that different scientists have different opinions?

Hello? Aren't they looking at the same evidence? How many different evidences are there?

Use your brain, people.

Victor Venema said...

Brad Keyes, I have withheld one of your comments for its tone. The published three are on the margin and will not be accepted in future. There is a reason why you are blocked on twitter. Especially, please be friendly to my guests. There is no reason to write "laughably wrong", rather than simply "wrong".

cRR Kampen, Consensus or credibility are no replacements for evidence, but they are still important in science. Except for the religious tone geloofwaardigheid is what I am thinking of. If it were a person, you could write, without invoking religion, that consensus gives an idea a "good reputation".

When you are reading up on your literature, would you rather read pseudo-sceptics blogs (WUWT, Brad Keyes' Climate Nurenburg) or the scientific literature? I would chose the scientific literature, it had more credibility, over time the scientific literature has build up a reputation for accuracy. Some scientific articles are wrong or even fraudulent and some of the blog post may be right once in a while, but on average you are better served by reading the scientific literature.

The pseudo-sceptics blogs have managed to build up negative reputation. What is written there is more likely wrong than right. It is consequently not a high priority to reading that stuff? If they had something of a consensus, I would be more willing to read up on that idea as the current random selection that only has in common that they are against mainstream science. Currently, the pseudo-sceptics cannot even convince each other, why would I waste my time on "ideas" with so little credibility?

When you want to hire someone for a scientific project, would you hire someone like Anthony Watts or Brad Keyes or would you rather hire someone who produced solid scientific articles in the past. I would go for the scientist. Whether the work they do is plausible, I can only judge at the end. In the beginning all I have is reputation, is credibility.

And also in the production of scientific knowledge consensus and credibility is important for setting priorities. If a new idea of mine would conflict with what we know about sea level rise, I would not worry to much. In that field the physical models and the semi-empirical models do not agree with each other. This signals that there is still something not well understood and opportunities to make relatively large changes. I would get into that literature and try to understand if we can understand and resolve the conflict.

If that new idea of mine would require radiative transfer to be wrong, or there to have been no warming in the last century, or for gravity only half as strong, I would first look for errors in my own ideas or whether the conflict is really there. If in the end I find no errors in my thinking and there is much other evidence for the idea, I may have a look at the theory of gravity, but that would not be first thing I would do.

cRR Kampen: This [credibility] is not the kind of concept we would like to associate with science.

I would be allergic to anyone using it as evidence, but as a description of what is, I have no problems with it. Scientific knowledge is more credible as non-scientific or even pseudo-scientific "knowledge".

cRR Kampen: I do believe the public can check enough for themselves to gain at least a sense of plausibility on the AGW hypothesis (or, rather, reality).

It would be great if they do and I would encourage everyone to do so. Best starting with a book on climate. While we both love talking about climate, I guess we have to acknowledge that this is not the case for everyone. Just like I am no expert for almost any topic and in those cases have to place trust based on credibility. That's life.

Victor Venema said...

Brad Keyes said: "Victor, this is a fantasy:"

And this was also not what I wrote.

Brad Keyes: "Lars, This misconception about how science works (which Victor hasn't corrected, for some reason) is laughably wrong: "Consensus about one question means that we can go on to tackle other questions.""

I did not correct it because if you read it in good faith, it is not wrong. Naturally, science is multitasking and working on multiple problems at the same time. If only because science is a very disperse and bottom up organisation and everyone has its own view what the priorities should be and has its own skills and based upon that selects what to do next. This is so natural that I see no problem with Lars' shorthand.

Brad Keyes: Do you literally imagine scientists must get together and vote to approve an answer before anyone is allowed to apply for a grant to work on a different question?

No, only your side of the "debate" sometimes suggests that that is how science works.

Brad Keyes: You're thinking of the IPCC, not science.

The IPCC only reviews the scientific literature. Their reports are no different from review articles, which are written in all sciences. The reports are just more comprehensive and thus thicker. The IPCC reports do not only describe what the consensus is, but also where there is disagreement and need for further study. For scientists the latter is actually the most useful aspect of the reviews.

Brad Keyes: If opinion throughout the scientific community is evidence-based, then why isn't it unanimous?

Easy, I do not make the assumptions that all scientists are perfectly rational. And if the topic is important to me, I also judge whether the arguments of the minority make sense. In particular, if someone like John Christy is able to do this, I no longer expect to be able to convince him with scientific arguments, even if he is officially still a scientist.

Dissenters are no problem for science and, if they do not do the things Christy did, their opposition can be fruitful. Einstein never accepted Quantum Mechanics. Because he played by the rules of science and he did know the scientific literature, he contributed a lot with his opposition.

This fringe is only a problem for the political public debate in combination with a media that is willing to inflate the importance of this fringe in a way they only do for few topics.