WHO REALLY READS THIS STUFF ?!?!
My apologies for being a week late with this month’s meandering thoughts on the IC, but it seems my trip to attend the DoDIIS Conference in mid March took more out of me than excursions like this in the past use to. Then there are is all that “day job” stuff at Oracle’s National Security Group that keeps diverting me.
Certainly there has been no lack of interesting topics to think about or comment on since last month such as the release of the DNI Information Sharing Strategy; a joint DNI McConnell/Congresswoman Eshoo OpEd in the Wall Street Journal; assertions by Democratic Congressional leaders that DNI McConnell has lapsed into becoming a “lobbyist” for the President, a Rand Study on Intelligence Analysis, and finally DoDIIS itself. So much material and so little time so I am going to limit myself here to the Rand Study on “Assessing the Trade Craft of Intelligence Analysis because it has received little coverage and in many ways touches on all the other topics in some way or another.
The full Rand study is available at http://www.rand.org/pubs/technical_reports/200 . Not surprisingly this report agrees with a growing perception that intelligence analysis is in trouble because it has failed to “get it right” and/or is often ignored by decision makers. For me this Rand Study says little that is new about the state of intelligence analysis and its recommendations have been made in some form or an other before, but still I thought it was compelling to read with the following points standing out to me:
- Analysis is more than reporting or answering the questions of the day
- There are two kinds of intelligence problems: puzzles and mysteries and the way you analyze them has to be different because of their inherently different natures
- Most analytical tools are not of much use to intelligence analysts because they are not relevant and/or require more training time than is available
- Baseline education and training are key to consistency of analysis across the IC and effectively incorporating analytical tools as well as encouraging information sharing and collaboration
- Quality of analysis is not determined by the number of sources used
- An IC organized around INTs vice issues or functions does not align well with an integrated approach to intelligence analysis
I was surprised that the study does not take note of the DNI’s Analytical Transformation Initiative, but believe this says more about when the Rand Study was actually written than it does about whether this initiative has merit or not.
With some reflection, it occurred to me that the Rand authors overlooked a structural issue with IC analysis that immediately devalues it both to those who produce it as well as to those who consume it ---- intelligence analysis is seen as a “free good” to its consumers. Of course as the primary output of a $43.6 billion plus a year enterprise, intelligence analysis is anything but a free a “free good.” Rather it is something users have pre-paid for but don’t realize it! The Rand Study recommendations aside, all of which are reasonable, and the DNI’s Analytical Transformation Initiative that is tools based, the quickest most focused way for the IC to improve its analysis is for its consumers to demand better quality for the intelligence analysis they have financed with dollars that theoretically could have gone to their own organization’s budgets.
Sans consumer demands with budgetary consequences for not improving the quality of intelligence analysis, the IC could do much to improve the quality of its products and services by looking at them in terms of the “users’ experiences” instead of from a “producers’ view of intelligence.” This requires regular candid exchanges between intelligence consumers and producers about what is good, bad, and indifferent. It also requires IC analytics for what types of intelligence are most useful to whom and why. I am not a big fan of “most used” or “best seller” metrics for assessing the value of intelligence products, which means “segmenting” intelligence consumers both for targeting products and understanding who valued them or not is important. This gets at the need for the IC to have a good sense of who its users are, what they want, and how to measure user experiences with IC analysis.
I know information sharing and collaboration have important roles to play in improving the quality of IC analysis, but they can also encourage reporting and reuse at the expense of in depth analysis. To avoid this the IC needs an analytic that shows when intelligence products are being primarily used by other intelligence professionals instead of external consumers. Internal IC use to produce better analysis is a good thing, but if the audience for various analytical products is primarily intelligence professionals, then their value should be suspect.
That’s what I think; what do you think?
