Accounting for the uncountable

Everything that can be counted does not necessarily count; everything that counts cannot necessarily be counted. —Albert Einstein

I know now more than ever why he made this point.  Back in 2009 I blogged on this topic and wanted to revisit it briefly before I start in a direction that will ultimately place me back in the industry where this research can be applied.

In my independent study of Gregory Bateson and Alfred Korzybski I truly understood for myself that the name is not the things named or as some would say the map is not the territory.  I call your attention to this manner of thinking because we have a problem with metrics in that the count is not the things counted.  Many metrics for risk and compliance describe beautiful mathematical formulas but only see a limited success because the classification of the things being counted is narrowly understood beyond a few individuals.  This blog posting makes the assertion that our problem with effective metrics is not one of numbers but one of semantics; not of the counts but of the things counted.  Since April of this year, I’ve been working on a few computational systems that side step the requirement put forth by number systems and I’m excited to test them out in the real world.

The things being counted must be named, defined, and ultimately understood by a community of practice.  The very act of naming is an act of mapping or classification; it comes with a certain level of precision and consequences. A useful classification standard for one community may be useless for another. To the degree that this mapping or classification is common with others in your community of practice, you achieve a mutual semantic coherence (some call this objectivity but I reject that term).  The durability of a set of metrics is challenged when multiple communities of practices are asked to engage in a common objective for the business.  Such is the case when one proposes a standard terminology and metrics that apply across a large enterprise consisting of multiple communities of practice and diverse personas.  To be useful one must know what these metrics mean and to be able to draw inferences from experience.  Needless to say, the process of stabilizing semantics across communities are extremely expensive and for those performing this in purely information spaces like information security, rapid change makes this practically impossible.

A measurement system must be judged on the notion of “usefulness to a community of practice” and this scoping must be made explicit.  The utility is a function of the audience’s ability to draw inference from the counts and things counted.  Let me share with you an example I experienced with a Canadian co-worker back in 2009.  I said “Dude, it was in the 90’s in San Francisco today”.  A blank face appeared as I saw him think and convert this implicit 90 degrees Fahrenheit to Celsius ((F – 32) x 5/9) because he could not draw an inference from Fahrenheit.  Inferences like it being weather for shorts, no jacket required, that it is odd for San Francisco to have a high of 32 Celsius, that homes in San Francisco don’t have AC because it is never that hot and so on and so on.

When you look at the notion of temperature, you can see that the different communities have chosen different standards because of the way they have come to know those units and it is more about the semantics than the mathematics.  This becomes exponentially more difficult when the syntax is the same but the semantics vary.  Take terms like ‘asset’ or ‘platform’ and you can fill a page with what it means in certain context with certain communities even within the same enterprise.  Each community of practice has come to know the term ‘asset’ in very different ways; this person has encoded work and meaning in ways that are different than others.  While mathematics remains important, we must turn our focus to formal ways to share semantics. Only then can we share both the numbers (the count) within their intended context (the things counted); semantics that can only be seen through a keen ethnographic eye that respects heterogeneous sense-making and the diverse viewpoints of an enterprise.

So while I am not going to spill the beans just yet, I will say that more important than numbers and counts, are the means to compute membership to classes.  You are probably saying to yourself, well, does that not require numbers?  I mean I need to score higher than 70 to pass this test, I need to score less than 40 to pass this audit, etc.  We got so hung up on number systems to help us compute membership to a set that everyone forgot to explore the other techniques.  Welcome to the wonderful world of semantic reasoning and in the coming months, I will have many stories to tell.  Thanks to the great works of Einstein, Bateson, and Korzybski, accounting for the uncountable will finally make sense.