Web Analytics - Puzzle or Mystery - the article

Recently I came across an old article from a popular Web Analytics blog, Occam's Razor that is as applicable today as when it was published in 2007.

In 2007, Malcolm Gladwell published an article in The New Yorker expanding on the concept of 'mysteries' and 'puzzles' as originated by Gregory Treverton, national-security expert. From Treverton's definitions:

• Puzzles - have a correct, factual answer which is unknown because we do not have enough information.
  
• Mysteries - do not have a single, correct answer. We have a lot of information and require knowledge and judgment to make an assessment of the answer.
  

I find this a fascinating way to look at the unknown. If you think your question is a puzzle, then look for the information to solve the puzzle. If you think your question is a mystery, then look for people with the knowledge and judgment to make sense of the information you have.

The poster from the Web Analytics blog argues that web analytics is a mystery and I would largely agree. In my experience, there are few puzzles in web analytics, mostly around data collection and data definitions, although many data definitions are mysteries. Interpreting and acting on the resulting data are most certainly mysteries. Yet, I've come across many people, especially those new to web analytics, or even outside it, who think it will be a puzzle - "If we could only report on 'X', we'd know what to do". I think that explaining web analytics as a mystery instead of a puzzle would go a long way towards helping people understand web analytics.