Response to Dan – the “best available evidence” man – Kahan

Preface and background

There’s a chap by the name of Dan Kahan who contributes to a blog called The Cultural Cognition Project at Yale Law School

The homepage comes complete with (subliminal?!) advertising such as this image:

Note: name of this image is "global_warming" .. it's their name, not mine!

Note: name of this image is “global_warming” .. it’s their name, not mine!

above a blurb with the very impressive title of “The 2nd National Risk and Culture Study”

I could be wrong (it has been known to happen!) but this suggests to me that the “uniform priors” of the populace of this particular project do not predispose its proponents to participating in anything that remotely resembles “scientific” investigation.

IOW, the authors appear to be engaged in mindlessly reinventing their version of the IPCC party-line. Back in 2010, the “word” was consensus, consensus, consensus.

But “consensus” as authority has worn a little thin in the intervening years and the new, improved mantra of choice – amongst those who claim expertise in the neo-natal “science” of “science communication” – appears to be a distinctly nebulous phrasing and unfounded confidence in what these (self-appointed) “experts” have dubbed “best available evidence” (BAE). As “expert” Kahan himself has acknowledged – unlike all other evidence which a lawyer might examine and consider – all climate change related prognostications appear to be exempt from cross-examination by Kahan and his fellow barristers.

Dr. Judith Curry has featured Kahan’s two recent essays – along with some questions posed by yours truly! She began her post as follows:

Is “best available evidence” a new, improved “reframing” of the so-called “consensus” (that is not really holding up too well, these days)? Is it simply a way of sweeping aside the validity of any acknowledgement/discussion of the uncertainties? Or is it something completely different?! – Hilary Ostrov

Curry’s cogent analysis and commentary are well worth a read. It could be confirmation bias on my part, but I particularly concur with her concluding observation:

So why does ‘climate communication’ seem to be failing? Here is my take. Dan Kahan seems to think that rational people are able to identify the best available scientific evidence, i.e. they know it when they see it (sort of like pornography). I say that identifying best available scientific evidence is difficult even for scientists if the uncertainties are large. What rational people are able to to do is to identify BS (see BS detectors). Overconfidence, failure to present evidence that does not support your thesis, dismissal of skeptics and skeptical arguments, appeal to consensus, advocacy, etc. all can act to trigger someone’s BS detector. Its about trust; without trust, expertise does not equal credibility.

But back to Kahan. For the record, when he posted his second post, ostensibly in reply to my questions, he emailed me a copy of his post. Because he made no comment or explanation in this E-mail, I thought it rather curious. But I get lots of odd emails … don’t we all, eh?! So I guessed that this was his way of “communicating” that he expected a response from me!

I lurked for a while, fully intending to reply at some point. Unlike some I could name, I’ve never been a knee-jerk thoughtless commenter! I prefer to take my time, make sure that I’m not repeating something that others might have said etc. etc. But, evidently, I made the mistake of responding to another comment from Kahan:

Dan writes:

The only “peer review” that I myself view as particularly reliable is the one that occurs after a study has been published (and “published” as in made public in any particular way). That sort literally never ends;

I think few would disagree, Dan. But how does one square this with the noble tradition of the “gold standard” IPCC “assessment” process?

This “inclusive, transparent” process often results in the inclusion of supporting citations derived from material that is so new that this post-publication “review” has not even begun (as I had observed here and more recently here) Well, except via the blogs, which the IPCC’s new, improved rules have specifically excluded from their list of “acceptable sources of information for IPCC reports”.

In light of the above, would you say that the IPCC’s reports are based on “best available evidence” (whatever your definition and/or criteria might – or might not – be)?

And I was rewarded with not one, but two responses – the first included the following:

But in response to your earlier, quite reasonable question, I wrote an entire blog post on why the question just asked me isn’t relevant to the question I asked you & others to help me try to sort through.

But he seems to have had second thoughts, which he expressed in a second similar (but different) comment … And it was the latter to which I began my reply.

Response to Kahan

@Dan wrote:

I don’t know as much as I should about all the IPCC processes. Moreover, I have observed outstanding climate scientists — includuing ones who have participated in the IPCC process — complain about one or another apsect of it, or one or another conclusion that the assessments have reached.

[… (sorry, Dan, but Mike Hulme said this first and in a more polished, albeit equally unsustainable, manner! -hro)]

But I wrote an entire blog for you to try to explain why what you just asked me isn’t relevant to the question I asked you to help me try to sort through.

You wrote an “entire blog” just for me in order to explain why my questions were not relevant?! It’s funny, but when I first read your introduction to this post, I thought I detected a slight undertone/undertow (?!) of sarcasm in your observation to the effect that (while you did not name me) my “question” (which by my count was more than one, but let’s not get picky!) “was interesting, difficult, and important enough that [you] concluded it deserved its own post.”

And I do apologize both for my delay in responding – and for evidently missing your summary of your “answer” which (as you have now reiterated) is to the effect that my “question(s)” was/weren’t “relevant” to those you had posed in your first post – and, no doubt, “irrelevant” to this one as well!

Initially I had chalked up my slightly less than favourable first impression to the probability that one (or both) of us was having a “bad communication day”. Hey, it happens to the best of us :-) So I decided to take a step back, while giving you the benefit of the doubt – and taking some time to explore your blog in order to ascertain ‘where Dan Kahan is coming from’, as we used to say in the good old days!

As for your admittedly limited knowledge of the IPCC, I certainly shan’t ask for the names of those in your roster of “outstanding climate scientists”! Nor do I expect you to necessarily accept my “evidence based” opinions of this less than laudable process. But it wouldn’t be a bad idea for you to take a look at the report of the InterAcademy Council; the big picture is far more damning than these “outstanding climate scientists” appear to have led you to mistakenly believe.

At this point, I can only hope that you are at least sufficiently informed that you have some awareness of the extent to which the “tenets” (cf Anderegg et al) of the IPCC have gained not only a mere toe-hold but also chapter and verse recycling amongst all those conference-goers. These are people whose influential (and probably far better known) organizations and institutions have issued “statements” (on behalf of their members – but perhaps unbeknownst to many of them) that rely on the proclamations of the IPCC’s “Summary for Policy Makers”, without any evidence that they have conducted any independent verification or exercises in due diligence.

And speaking of this process (even if you continue to maintain that it is not “relevant” to the question(s) you posed in your first post), I wonder if you have ever come across the work of Jason Scott Johnson. He’s an academic and he speaks your language – well, at least your “former” language, that of a lawyer.

I think you’ll find his credentials and publications quite impressive. Certainly far more impressive than anything one might have encountered from the likes of Stephan Lewandowsky, Adam Corner, John Cook, Dana Nuccitelli (or even the late great Saint Stephen of Stanford) – all of whom appear to have aspirations of maven-hood in what you (and possibly others) have dubbed the “science of science communication”.

But there’s one 2010 publication of Johnson’s that you might find quite informative and enlightening: Global Warming Advocacy Science: A Cross Examination (62 page pdf). I would quote from the abstract, but I suspect you will deem it to be “non-responsive” to your questions/argument(s).

Before I do provide what I hope you will consider a somewhat more “responsive” reply, permit me to let you know where I’m coming from! For over twenty years I was involved (in many capacities) in what came to be known as the “human service field”; in particular, the field of services to people with developmental handicaps and their families. In my day (and even today, according to my former peers and colleagues) it was a field that was rife with gurus of the month and ever-changing jargon and mantras. Some gurus were more informative (and/or entertaining) than others; but the jargon became an increasingly obstructive barrier to … wait for it … communication!

Do I have any “scientific” polls/surveys or “peer reviewed” studies to support the above? No, because the grad/post grad academic path is not one that I chose to follow. Oh, don’t get me wrong, Dan; to coin a phrase, some of my best friends are academics:-)

Does this make it an invalid finding unworthy of further consideration in your books? Perhaps. Anyway, let me give you some examples of what I found to be almost insurmountable “barriers” in your articulation of the merits of the “science of science communication”.

First, I gather that you don’t attach any “relevance” to the fact that your first essay included six instances of the phrase “best available evidence” – which in your second essay you changed to “best available scientific evidence”, as though this somehow obviated the need for you to provide some explanation as to why you chose to use it no less than six times in the first!

I fully concede that it may have no “relevance” to the argument you were trying to make, but to this layperson’s ears it’s nothing less than marketing a mantric (if not meaningless) message.

The next stumbling blocks I encountered were contained in your:

But I certainly don’t have a set of criteria for identifying the “best available scientific evidence.” Rather I have an ability, one that is generally reliable but far from perfect, for recognizing it.

I think that is all anyone has—all anyone possibly could have that could be of use to him or her in trying to be guided by what science knows.

Intermission

At this point (looking at my word-count) I decided that it would be a rather ungracious (and, who knows, perhaps unwelcome) intrusion to continue discussing what Kahan might well consider to be “irrelevant” to his burning question(s).

So I invited Kahan (if he and/or other readers were so inclined) over here where I continue my response as follows …

With all due respect, Dan, I don’t find “best available scientific evidence” to be any less nebulous, confident or all-encompassing than “best available evidence”. Incidentally, when you were a practicing lawyer, did you not have any criteria that you applied when you were sifting through and determining which evidence would best support your case? And (because I don’t speak “cultural cognition”), just a reality check: in my books “identifying” and “recognizing” in this particular context are virtually synonymous. Is this equally true in yours – or is there some nuanced difference I’ve missed because I don’t speak “cultural cognition”?

And whether it is or it isn’t, I have to say that your less than perfect “ability … for recognizing [best available whatever]” reminds me of Phil Jones’ dedication to keeping the peer review process humming along by using “intuition” (as opposed to <<gasp>> requesting and examining the underlying data and methodology of a paper).

But moving right along to your second paragraph above … “guided by what science knows.” Excuse me?! Science does not and cannot “know” anything. Are you trying to “reframe” the previously ubiquitous (and equally ludicrous, IMHO) “science says …” or “science tells us …”? In the real world, scientists (some of whom may be “outstanding”, others, not so much) may “say” or “tell us”, but not “science”.

So now we come to the point where, I’m sorry to say, you completely lost me!

Nullius in verba–the Royal Society motto that translates to “take no one’s word for it”– can’t literally meant what it says: […]

What the motto is best understood as meaning is don’t take the word of anyone except those whose claim to knowledge is based on science’s way of knowing–by disciplined observation and inference– as opposed to some other, nonempirical way grounded in the authority of a particular person’s or institution’s privileged insight.

Oh, my … there you go again with a variant of your “science knows”! But that aside, notwithstanding your claim to the contrary, I’m inclined to think that the Royal Society’s motto means (or at least meant!) exactly what they said it meant when they adopted it 350 years ago.

As I had observed when reviewing Andrew Montford’s chronology of the descent of this once respected institution, this motto, embedded in the RS heraldry appears only on the “history” page of the RS site where they note that the Royal Society’s motto Nullius in verba:

“roughly translates as ‘take nobody’s word for it’. It is an expression of the determination of Fellows to withstand the domination of authority and to verify all statements by an appeal to facts determined by experiment.” [emphasis added -hro]

Please note, Dan, “determined by experiment” – not by “inference” (disciplined or otherwise) and, in the case of “climate science” more often than not such “inferences” are drawn from simulation exercises undertaken via computerized “models” – the outputs of which are about as far away from the results of an “experiment” as one could possibly imagine!

Speaking of simulations and models vs experiments, I don’t know if you are familiar with a site called AccessIPCC. One of the tags used is “MoS – Model or Simulation Reference appears to be a model or simulation, not observation or experiment.” Approx. 13% of the citations and references found in the 44 chapters of AR4 were tagged as “MoS” – of which the lion’s share, so to speak, is found in the chapters of Working Group 1. (Full disclosure: I was involved in the development of this site) But I digress …

The RS motto may not work for you, Dan, but it certainly works for me! Unfortunately, after 300 years it seemed to stop working for the RS whose journal, as Montford observed, once wisely carried a notice that:

‘It is neither necessary nor desirable for the Society to give an official ruling on scientific issues, for these are settled far more conclusively in the laboratory than in the committee room’.

They’ve “re-branded” their image, and all that’s left of their motto is a mere fragment found in the name of their “Science Policy Centre” blog: In Verba. And the RS has become, in effect, an advocacy organization with a mandate that is so far removed from its original charter as to be almost unrecognizable. This “Science Policy Centre” proudly claims to provide:

independent, timely and authoritative scientific advice to UK, European and international decision makers.

We champion the contribution that science and innovation can make to economic prosperity, quality of life and environmental sustainability […]

How and why did this once respected Society – whose Fellows were once united in their determination to “withstand the domination of authority” – morph into one which provides “authoritative scientific advice to … decision makers”?

To my mind, this is a far more interesting – and “relevant” – question pertaining to “science communication” than why don’t skeptics use “conferences” and your “science of science communication”.

In your first post you had said:

But I don’t see skeptics grappling in the earnest—even obsessive, anxious—way that climate-change policy advocates are with the task of how to promote better public understanding.

I can’t speak for other skeptics, but this particular concern of yours reminded me of the very wise words of a woman (mistakenly feared by many in the field!) during “conferences” when the topic of day was ‘how do we promote better public understanding and acceptance of group homes in our communities?’ In effect we were “grappling” (albeit not in an earnest – even obsessive, anxious way) with how can we do better PR? Her response was, “The best PR is a well-run program.” And guess what, Dan … she was soooooo right, as was proven in subsequent years, many times over.

Perhaps the same principle might work for those who are “grappling” with the quite possibly unnecessary task of “promoting better public understanding” of climate science (of which the alleged primary finding has very recently been rebranded by Obama as “carbon pollution”)

Added to comment at Kahan’s

It is interesting, though (albeit, perhaps not to you) that in July 2009, when Pachauri was articulating his “vision” for AR5, he had declared:

[T]he IPCC AR5 is being taken in hand at a time when awareness on climate change issues has reached a level unanticipated in the past. Much of this change can be attributed to the findings of the AR4 which have been disseminated actively through a conscious effort by the IPCC, its partners and most importantly the media. [emphasis added -hro]

Perhaps back in those halcyon days the public must have thought that the IPCC was “running a good program”. Certainly there was no indication of any “grappling in the earnest—even obsessive, anxious—way” of today’s climate-change policy advocates. Are there any “cultural cognition” explanations for this – or any studies that can be found in the “science of science communication”?

Come to think of it, was this “science of science communication” anything more than a gleam in the eye of those who put so much stock in the jargonized realm of “cultural cognition”?

Which reminds me … if I were to attempt to elevate “science communication” to an academic (or pseudo-academic) stream I would have thought that “art” would have been a more apt school of thought than “science”. I mean how many other “communication” conundrums have landed in the “science” bucket? And how did they get there, eh?! In particular, how did “science communication” make the leap from the (2010) “challenge” frying pan into the (2013 if not sooner) “scientific” fire?

Perhaps this question is worthy of a proposal to the NAS so that you can expand the lofty annals of certitude within the universe of “cultural cognition”.

P.S. Considering your unwavering faith in the value of your findings (which does not seem to have budged an inch since you [long self-promotional messianic video alert] articulated them at Cambridge in 2010, has the thought occurred to you that most people (with the notable exception, it seems, of “scientists” – and/or those who claim to be “scientists”) have a distinct aversion to being placed in convenient little research boxes by those who have not demonstrated anything beyond scant superficial acquaintance with the issues about which they claim to have “scientific” understanding (with or without appeal to the meaningless authority of “best available [whatever] evidence”)?

P.P.S. One final thought/question, Dan … If you possess sufficient vision to see beyond the realm of your little boxes … Have you considered that there may be some merit to the words of (long-time former IPCC-nik and consensus cookbook coordinator extraordinaire) Joseph Alcamo, UNEP Chief Scientist, whose address to the IPCC plenary – at Bali in October 2009 – included the following (perhaps not so enchanted/enchanting):

[…] as policymakers and the public begin to grasp the multi-billion dollar price tag for mitigating and adapting to climate change, we should expect a sharper questioning of the science behind climate policy.

I’ve asked this question of others, but I’m still grappling (albeit not in the “earnest—even obsessive, anxious—way that climate-change policy advocates are with the task of how to promote better public understanding”) with the question of what is it that such “best available evidence” [whatever this might – or might not – be] promoters are having such difficulty understanding about “multi-billion dollar price tag“?

If you and/or your fellow “cultural cognitionists (cognoscenti?!)” have an answer to this question, I’d love to hear it :-)

-30-

Well, that’s the view from here … sorry this was such a long post, but thank you for reading to the bitter end :-)

12 thoughts on “Response to Dan – the “best available evidence” man – Kahan

  1. So, words no longer “literally mean what they say”???!!! You could not make this stuff up. Reading to the bitter end? How could I not!

  2. The best available evidence shows that Kahan lacks fundamental grasp of the CAGW issue. On the basis of that we can surmise that his attempts at communicating “the science” are completely flawed. This can be inferred also by his rhetorical somersaults around basic definitions and simple Latin.

    We can thereby declare Kahan as John Cook’s talkative cousin -with same levels of intelligence

  3. That Kahan has an opinion about the scientific evidence of AGW, and considers that mitigation should be attempted is just that: an opinion. As he stated he is not interested in discussing his opinion of that. He wants to discuss other items. He seldom tries to communicate the science. He is interested in the science of communication. Remember, he is of the opinion that is basically settled well enough, yet knows science continues marching on with or without the IPCC.

    I disagree with him in that I think what he sees as pathological is not and should be expected. It should be expected for the reason that the science is what it is. The range literally covers from we should encourage use of fossil fuels for decades, to that we should start draconian measures to stop CO2 emissions in the immediate future. In such case, cultural reasoning is not only expected but is correct. One can be scientific with either one as a position.

  4. Anyone who wants to find out what Kahan is about should watch a bit of the video that Hilary linked to (it is a big ask to watch the whole thing!) The last 12 minutes are particularly instructive. He simply assumes, across several controversial issues, that disagreement is merely a function of defective communication. He says that breaking down people’s “culturally defined beliefs” is central to science communication.

    It will be a cold day in Hell before I accede to giving up my culturally defined beliefs to him and his buddies.

    it’s just more of the Lewandowsky meme – people who don’t agree with him are broken and need fixing, or at a minimum, diagnosis.This kind of thinking is not only the refuge of the mediocre, it is about as anti-science as it is possible to get.

  5. In Nature magazine, January 2010, Dan Kahan writes:

    “It would not be a gross simplification to say that science needs better marketing. Unlike commercial advertising, however, the goal of these techniques is not to induce public acceptance of any particular conclusion, but rather to create an environment for the public’s openminded, unbiased consideration of the best available scientific information.

    As straightforward as these recommendations might seem, however, science communicators routinely flout them. The prevailing approach is still simply to flood the public with as much sound data as possible on the assumption that the truth is bound, eventually, to drown out its competitors. If, however, the truth carries implications that threaten people’s cultural values, then holding their heads underwater is likely to harden their resistance and increase their willingness to support alternative arguments, no matter how lacking in evidence.”

    But who decides whether or not “the truth” carries threatening implications? You just did, Dan.

  6. Dan Kahan’s all right. His analyses of the climate debate can often be jaw-droppingly naive, but he has never stopped me pointing this out on his blog. When he says he wants to create the conditions for people to make up their own mind, he really means it, as far as I can tell. Until he starts censoring and lying, any comparison with John Cook would, I think, be unfair (sorry omnologos). Kahan is the only “climate psychologist” who goes out of his way to *talk* to skeptics and I think it would be a wasted opportunity for us if we made an enemy when we could be making a friend. Let’s not tar him with the same brush as Lewandowsky. I always enjoy your sardonic takedowns of lying propagandists, Hilary, but please don’t assume Kahan’s arguments are coded versions of the lies we’ve heard a million times before.

    I realise nobody asked my opinion, so I hope you’ll forgive the interjection.

    • Welcome Brad … and thanks for your comments.

      “Jaw-droppingly naïve about the climate debate”, seems like quite an apt description to me! But the problem I have is that he doesn’t seem to have any interest in learning anything from the skeptics with whom (at whom?!) he talks! He is so focussed on his (very) narrow research interest – that there’s no getting past it. And his “communication skills” – by his own admission – are appalling!

      I certainly don’t tar him with the same brush as Lewandowsky, Cook, Nuccitelli and/or Corner – who are clearly propagandists who’ve sacrificed any credibility (academic or otherwise) they might once have had, to their over-riding dedication to “the cause”. And I don’t think I did in my post, either. My guess would be that Kahan knows as little of the alarmists’ “arguments” as he does of the skeptics’. But he goes along (with the alarmists) to get along, so to speak!

      My intention in this post was not a “takedown” of Kahan, btw; but rather it was an (admittedly blunt – and probably futile!) attempt to provide some sparks which might persuade him to broaden his horizons. At the very least, I thought he might have been interested in the link I had provided to Jason Scott Johnson’s paper. Who knows, perhaps he did follow it and found it of interest. But if he did, he didn’t bother to thank me for it!

      Anyway … your opinion is always welcome here, Brad … particularly when I’m doing one of my takedowns … I like the positive feedback ;-)

    • Thanks, Hilary. I should add that, while I’ve found Kahan to be genuinely respectful of criticism, he’s only human and in order to circumvent the ego and pride issues that vitiate online rhetorical duels you might try emailing him. It takes patience—our POV is so alien to him, and his cult[ure] cuts him off from so much information you and I take for granted, that we can only really expect his paradigm to shift at a glacial rate.

      He’s self-effacing about his communication skills. But I think he communicates about as well as can be expected in the contemporary context of mutual incomprehension and suspicion, don’t you?

    • Hi Brad,

      First of all my apologies for not responding sooner to your comment of Sept. 11 (not sure where the time goes!) … I had wanted to give some thought to your suggestion of initiating an E-mail correspondence with Kahan.

      And I did give it some thought; but after reviewing his comments on that thread (not to mention his non-responsive reply to my own contributions – along with his decision to ignore my question on his earlier post in which he had praised Cook’s paper), I had decided that it wasn’t worth my time.

      All he seems to be interested in is discussion of his own multi-squared/rectangled messianic “communication” paradigm! This was also evident in the video of his performances on the same “how can we communicate the science” panel/platform as Mann.

      From my perspective, Kahan’s “lack of hospitality” to Willis was by far the least of his transgressions on that thread! For whatever reason, it seemed to me that Willis was making a mountain out of a semi-pseudonymous molehill.

      To my mind, Kahans failure to engage/respond to NIV’s observations was far more egregious and deserving of highlighting (and response by Kahan).

      But I quite agree …the bandwidth wasted on such petty “climate unpleasantness” is both “sad and dumb”. :-)

    • Hi Hilary—thanks for replying. I totally understand and respect your decision :-). (I avoided watching the video where Kahan shared the stage with Mann, in light of the nauseatingness of the latter.)

      Keep up the good work your blog is doing!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s