Two fun videos that portray why you should pay attention to the data, not to your gut feel, using sports metaphors – with links to online analytics of the sports stats yourself, on the ever-growing Experience SAP site.

Baseball: “know it all – without being one”

Play with the stats yourself at the Baseball section of the Experience SAP site.

Rugby: “keep your mouth shut if you don’t have the data to back it up”

Play with the stats yourself at the Rugby section of the Experience SAP site.

Hockey doesn’t rate a video (yet?) but here’s a link to the statistics section.

The New York Times reports that a respected psychology journal is due to publish a paper purporting to show “strong evidence” for extra-sensory perception:

“A software program randomly posted a picture behind one curtain or the other — but only after the participant made a choice. Still, the participants beat chance, by 53 percent to 50 percent, at least when the photos being posted were erotic ones. They did not do better than chance on negative or neutral photos.”

Crucially, no “topflight statisticians” were part of the peer review. When I was at university, struggling to use a sophisticated statistics package on a mainframe as part of my econometrics degree, I dreamed of having a program that would just cruise through all the possible combinations of variables, and tell me which ones were correlated. That ability now exists, but the danger is that few people realize how much higher the bar must be set for a result to be deemed significant in such circumstances.

Given a large enough set of random numbers, you will always be able to find a “significant” relationship – especially if that’s exactly what you’re looking and hoping for.

To me, the experiment above sounds like it may have this problem – for example, if there were lots of different categories of photos, and the “significant” relationship was cherry-picked from the available results. And even if the level of significance has indeed been increased to take account of this, the result could still be random (if there’s a choice between changing everything we know about science and it being a fluke result, I’m going with the latter).

In science, thankfully, it’s easy for somebody else to repeat the experiment and validate the correlation, ideally before a respected journal makes a fool of itself (although I suspect they’re simply making a calculated bid for more publicity, and it’s working very successfully).

In business, it’s much harder to know if your “results” are valid. The same problem exists – people are looking for a certain type of result, and keep running the numbers until they find something that looks like a relationship: “Look! Customer satisfaction is correlated with their age!” . But it’s much harder to “rerun the experiment”, and businesses don’t always have/take the time to check their results.

Despite having worked in BI for over twenty years (or maybe because of it), I’m deeply distrustful of most corporate analytics. I believe business analytics is essential, but that it’s also essential to assume that any relationship you find is a working hypothesis, to be validated through further analysis (e.g. correlation is not causation), and expert discussion (as with peer-reviewed science papers, the best way to deal with potential analysis problems is greater transparency — social BI technologies like Streamwork are becoming increasingly important).

[Update: there's a great New Yorker Article that talks about the issues of finding, and replicating, significant results in studies -- much of accepted science may be false?]

There are only three matches left in this year’s World Cup, and the competition is wide open after a series of upsets have knocked out some of the big international players, including the winners of the last three world cup tournaments: Italy, France, and Brazil.

To enhance your enjoyment of the game – or simply to see a real-life example of what you can do with business intelligence technology, check out the suite of tools offered as part of the SAP BusinessObjects 2010 Football Experience:

The site lets you:

• Easily review historical data from matches going back nearly 50 years to gain perspective on the game.
• Check out details on the teams and players as they compete for the title.
• Analyze match information, such as shot location, to search for patterns of play and success.
• Gain insight, spot trends, and use the information you uncover to make better decisions.
• Receive live commentary during the matches so you can stay informed.

In order to get answers to questions such as:

• What positions typically score the most points?
• What is a likely final score?
• Does being the host country influence success?
• Which teams consistently make it to the final rounds?
• Which players have previous tournament experience?
• Is there a particular area of the goal that a goalkeeper is less successful in defending?
• Is one player or team receiving more yellow cards than the others?

Trivia contest

Once you have the hang of the different tools below, why not take part in the Trivia Contest? Each week day a game is on, there will be a question tweeted about from the @businessobjects Twitter account, and the first correct response is the winner!.

The first thing to do is register with the site (you can still access some of the functionality without doing this, but you won’t get the full flavor).

You can then go to the main screen and choose which of the seven main areas you’d like to explore:

• Go Explore! World cup statistics with SAP BusinessObjects Explorer
• Predictor: Predictions using SAP BusinessObjects Xcelsius
• Goal History: Reporting using SAP Crystal Reports
• Social Network Analyzer: The relationship between teams, matches, players…
• Team Information: An Xcelsius dashboard of team information
• Goal History by team: Reporting using SAP Crystal Reports
• Jive community space: Join in the conversation!

Go Explore!

Click the Explorer button to open up a choice of information spaces: either the full statistics, or a view of other people’s predictions (see next section) (note, if you only see one choice, check again that you’re registered / cookies are enabled)

Click on the first one and start investigating – e.g. one of the semi-finals is Germany vs. Spain – here’s the result of the previous time the two teams met:

There’s also a set of shortcuts that you can use to get an idea of what kinds of questions are possible:

• What unlucky player has hit the goal post on 4 separate occasions in a single World Cup tournament? ANSWER
• Jürgen Klinsmann is an former player who is now working as an ESPN commentator. When did he play, and how many goals did he score? ANSWER
• What Italian goal keeper has had a remarkable 69 saves in World Cup play? ANSWER
• I heard that Real Madrid is a good club. How are the players from that team in this year’s World Cup?
  Has anyone scored? ANSWER

Predictor

Using Xcelsius, you can predict the scores of the different matches, and then see how good your prediction record is over time:

And then you can access a leaderboard report that gives you the top predictors, overall, or by round, as a pdf document:

(and remember, you can do exploration of the same data by opening the other information space in the explorer section above).

Goal History

The goal history section opens up a Crystal Report with detailed data about exactly where and how goals were scored during matches:

And you can drill down on information by different teams – here’s Germany’s scoring pattern in the last world cup, for example – something that Spain might like to look into before their next match…

World Cup Social Network Analyzer

The SAP Social Network Analyzer aggregates existing enterprise data to display and discover organizational relationships. It automatically generates useful social networks that can be used:

• to find and connect people,
• to take actions based on individual/organization/company information,
• to send an email, meeting request or call a person,
• to build the right team,
• to better manage and control processes,
• to understand the relationships between suppliers and buyers,
• to analyze people’s information and organization using BI tools,
• to integrate social network information inside any application…

All the data from the world cup has been fed into the online prototype, to give you an idea of how the players and their teams relate. For example, here’s the network of matches played by the USA over the years:

Here are some same views:

• Player with current national team, coach, number of goals in world cup, size, height, position or jersey number
• Teams with their respective players and coaches
• Players who scored during a world cup still in 2010.
• Who scored the most
• Previous world cup matches between two teams

Team Information

This section gives you access to an interactive Xcelsius dashboard that shows information about the different teams, e.g. by location:

And by previous team experiences in the world cup:

And the players involved:

Goals per Team

Another example of a Crystal Report, this time using the Crystal Reports Viewer plugin.

Jive Community Space

Join in the discussion! Will the Dutch be able to set the pace in the next match?

Other Dashboards

Other folks have also taken the plunge and used the SAP BusinessObjects tools to create useful dashboards… here’s a nice one from the MyXcelsius.com web site, using the great Google Maps 2.0 plugin for Xcelsius:

SAP Football Ad:

If you’re into football, you’ll enjoy this SAP Ad from 2008, courtesy of www.danielfreestyle.com (filmed entirely around San Francisco, which does a pretty good job of looking like a trip around the globe!)