In my previous post, I talked about how the American Recovery and Reinvestment act is perhaps the largest, most complex business intelligence application ever. And since the US has now appointed its first ever “Chief Performance Officer”, Nancy Killefer, who will work with economic officials to increase efficiencies and eliminate waste in government spending, it seemed like a great time to talk about other great examples of US government transparency and business intelligence.

Open Government

The Recovery Act is only one tiny part of government transparency. Dan Everett Jr. of SAP has written a great blog post about the subject and points out that in May the National Academy of Public Administration started soliciting input from the public on how to make the government more transparent at http://opengov.ideascale.com/ (sadly, there’s no moderator on the site, and the nutters have taken over: check out the dialog around making data more accessible).

Another Obama administration initiative, the http://data.gov web site is starting to providing lots of other US government data to anybody who wants to analyze it:

“The purpose of Data.gov is to increase public access to high value, machine readable datasets generated by the Executive Branch of the Federal Government. Although the initial launch of Data.gov provides a limited portion of the rich variety of Federal datasets presently available, we invite you to actively participate in shaping the future of Data.gov by suggesting additional datasets and site enhancements to provide seamless access and use of your Federal data. Visit today with us, but come back often. With your help, Data.gov will continue to grow and change in the weeks, months, and years ahead.”

There’s a content for applications that can best make use of this data at http://sunlightlabs.com/contests/appsforamerica2/

There doesn’t seem to be much uptake so far – maybe because the first submitted application seems a little trivial – it lets you play a memory game with photos of FBI Fugitives culled from the FBI’s feeds of fugitives (but it doesn’t even tell you who they are!)

Federal and Regional BI Examples

In an article on how ARRA Will be a Boon for Business Intelligence, Dr. Ramon Barquin reports how education provisions are also driving business intelligence, and gives some examples of mandated metrics as communicated by the Department of Education last month:

• Number and percent of teachers and principals rated at each performance level in each local education association’s (LEA) teacher evaluation system
• Number and percent of LEA teacher and principal evaluation systems that require evidence of student achievement outcomes
• Most recent math and reading National Assessment of Educational Progress (NAEP) scores
• Percent of ELLs and students with disabilities tested in math and English language arts (ELA)
• Number and percent of students who graduate and complete one year of college
• Number of schools in restructuring status that demonstrated substantial gains in achievement, closed or consolidated – last 3 years
• Number of schools in the bottom 5% of those schools that demonstrated substantial gains in student achievement, closed or consolidated – last 3 years
• Number and percent of schools in restructuring status that have made progress in math and ELA in last year
• Charter school caps, number operating, number closed

The Federal Election Commission provides information about campaign financing

The State of Washington Transportation Improvement Board, an independent state agency that makes and manages street construction and maintenance grants throughout Washington State, uses Xcelsius dashboards to manage the agency’s $200 million of revenues generated from state gas taxes. In a nice example of transparency, they provide the public with the same view as the agency’s Executive Director.

Some of the benefits the agency has realized include

• Reduced delayed projects by 70%, saving millions in public funds
• Reduced accidents by 19% and injuries by 30%
• Reduced average payment cycle from 5 months to 17 days

New York City has a huge range of different statistics available, e.g. my neighborhood statistics:

NYCStat is New York City’s one-stop-shop for all essential data, reports, and statistics related to City services. Available at www.nyc.gov, NYCStat provides access to a wide array of performance-related information including citywide and agency-specific information, 311-related data, and interactive mapping features for selected performance data and quality-of-life indicators.

The State of Washington has a public “Government Management Accountability & Performance” site that provides easy access to dashboards of key performance indicators in areas such as economic vitality, government efficiency, health care, public safety, transportation, etc.

The State of Maryland has “StateStat” that provides dozens of key performance indicators from different state agencies (agriculture, environment, general services, state police, transportation, etc.)

The City of Henderson in Nevada uses SAP’s Strategy Management software to track the strategic planning process, and then provides that information to residents:

Other countries are getting in on the act, too. For example, the British Prime Minister has announced that Sir Tim Berners-Lee, the inventor of the web and an eager proponent of better access to government data, will be helping the government make more of its data available online

Think-Tanks and Watchdogs

The Pew Center on the States web site has followed state government performance for more than a decade, studying good and bad practices and analyzing what works. Their conclusion is that results-based budgeting systems works: states cut wasteful spending on programs that are not showing results, and instead direct the resources to more effective :

”The unfortunate truth is that most states today do not have the tools in place to make well-informed programmatic and budget decisions. There is no way  to know how much money states lose to mismanaged or underperforming programs. But those states that have begun to make policy decisions based on data
measuring the performance of government, a process called “performance-driven budgeting,” have saved impressive amounts of money—some in very short periods of time. The choices they have made have not been easier, but they have been smarter.”

The excellent Grading the States web site puts the Pew data into dashboard format, making it easy to compare states’ performance.

Here are some interactive models from the site, using Xcelsius dashboards — give them a try!

Govtrack.us keeps track of representatives voting records

Maplight.org “illuminates the connection between money and politics” by bringing together the campaign contributions of legislators and their voting records. Check out their intro video for a great overview.

Third-Party Sites that Promote Government Transparency

The Swivel.com web site features data that furthers public debate and increases transparency, and lets people upload and share data from other sources (although the last uploads seem to have been a year ago?)

IBM’s ManyEyes site does something very similar

The New York Times has a homicide analytics map:

Optimizing BI from Governments

The list above obviously only scratches the surface – there are many, many other US public sector bodies that are publishing information online.

The scattering of different projects across the different agencies and bodies is similar to the silos of BI projects inside large organizations, with many of the same problems.

There are three in particular that will need to be addressed:

Not enough goals. As Jos van Dongen pointed out to me on Twitter, the most glaring thing missing from most of these performance dashboards is concrete goals: numbers themselves are meaningless unless you have something to compare them to.

Not enough marketing. There’s not enough emphasis on “what you can do with this data”. It’s the classic business intelligence “build it and they will come issue”. Right now, the sites above all emphasize what data they have, but few of them have good examples of what you can actually do usefully with the data, or great examples of what others have done. Without this, government BI will face the same problem as many private projects: low usage and no grasp of any return on investment, leading to disinvestment over time.

Not enough standards. As you can tell from the bewildering variety of formats and statistics above, there’s clearly a lack of standards on what should be tracked, and how data can be compared between different states and agencies, and the choice of indicators changes over time.

This is exactly the same problems faced by organizations (right down to the choice of KPIs – a change of State Governor is much like a change of top executive – in both cases, they are likely to feel like they have to “change the agenda” and use different KPIs than their predecessor).

In companies, a BI competency center is the recommended approach, a body that explicitly has the authority to overview the different BI projects that are going on within the organization, and impose standards (tools, metadata, definitions, reporting formats…).

How will this be handled for the US Government? The fact that there are sites that now have the job of gathering information across the country, like recovery.gov and data.gov will help, and external organizations like the Pew Trust will help encourage areas. But these pressures alone will not be sufficient.

Ultimately, help will probably come primarily from organizations like the United Nations. As part of their work, they help organizations compare performance in various areas, such as education. They publish league tables of countries that encourage participation, and they have the mechanisms required to negotiate standards that everybody can agree on.

These standards are then increasingly used within countries as well — although, just like in organizations, some nations prefer their own, “better” or “more adapted” statistics (cynics might call this “leaving wriggle room to explain away bad numbers”).

SAP And Enterprise Support

SAP has agreed a set of key performance indicators with its user group SUGEN, in order to measure the effectiveness of the controversial SAP Enterprise Support services introduced last year. And there’s real skin in the game: SAP has agreed to postpone scheduled price increases until the targeted improvements measured by the SUGEN KPI Index are met.

The KPI Index, defined after discussions with customers, is calculated on KPIs in four categories: business continuity, business process improvement, protection of investment, and total cost of operations.

“We are convinced that SAP Enterprise Support delivers unparalleled value to all customers and, as a demonstration of our commitment, we will provide tangible reductions in their operational costs on a defined schedule,” said Léo Apotheker, co-CEO, SAP AG

“SAP’s goal to provide a rating of application support via key performance indicators is innovative and challenging,” said Peter Wesche, research director, Gartner. “The extensive benchmarking phase and mapping of key performance indicators to business value will allow customers to fully understand the benefits of a comprehensive support program.”

Gross National Happiness

The New York Times reports today that the tiny nation of Bhutan has put together a sophisticated set of indicators to try to measure “gross national happiness”:

“…the Bhutanese produced an intricate model of well-being that features the four pillars, the nine domains and the 72 indicators of happiness.

Specifically, the government has determined that the four pillars of a happy society involve the economy, culture, the environment and good governance. It breaks these into nine domains: psychological well-being, ecology, health, education, culture, living standards, time use, community vitality and good governance, each with its own weighted and unweighted G.N.H. index.”

And it’s not as daft as it sounds – as the New York times reported several years ago, average happiness in many countries has hit a plateau even as median incomes have risen. Governments have always taken account of many different considerations when determining policy. To a certain extent, “gross national happiness” is just another way of looking at what voters want when they are looking to elect their democratic officials.

Measuring Government Ministers

Malaysia’s prime minister has announced that his government’s cabinet ministers and civil servants will be measured on KPIs, as part of his “1Malaysia” plan. Many other countries around the world, of course, have already implemented various types of performance-based budgeting, based on achieving a variety of social KPIs.

Stopping Yourself From Being Too Self-Righteous

The New York Times also has an article today about “self-righteousness”. Studies show that we’re hopeless at estimating our own willingness to do what’s morally right, but pretty good at estimating what other people will do. It’s not technically about KPIs, but if we regularly measured the delta between our expectations and what actually happens, we might get better at estimating our own real-life behavior!

USA Spending.Gov

USA Spending.Gov gives you access to the same information in the form of a public database. It’s the result of the 2006 Transparency Act which requires that a certain amount of information about each Federal contract must be included on a single, searchable website, accessible by the public, for free. It includes data from:

• The Federal Procurement Data System, which contains information about federal contracts
• The Federal Assistance Award Data System, which contains information about federal financial assistance such as grants, loans, insurance, and direct subsidies like Social Security.

Online information deserves online analysis

Using the labs version of Polestar OnDemand at http://polestar.ondemand.com, it’s easy to analyze this information. For example, I downloaded information about government contracts for the four BI megavendors – SAP, IBM, Oracle, and Microsoft, and combined it into a simple spreadsheet. (I later realized that I could have used the API interface to get all the information in a single go.)

Polestar lets me export the results as a handy image, ready for any presentation. It looks like there’s some upside opportunity for SAP in US government contracts (presumably, an analysis of European contracts would show the proportions reversed).

I challenged some other people to take a look at the data. For example, Adam Binnie of the DecisionVelocity Blog came up with the following analysis of Hurricane Katrina contracts. It’s easy to see at a glance how much of the business was awarded without any competitive tendering.

What can YOU come up with?

This is your opportunity to come up with interesting data about US government spending and share it with the rest of us. Are you up to the challenge?

If you’d like to start with the examples above, download the data into a spreadsheet, and then go to http://polestar.ondemand.com:

• USA spending data files
• Katrina contracts excel spreadsheet 
• BI megavendors government contracts

I’ve been competing with Cognos for over 15 years, and I consider them a very ethical organization. The big problem seems to be simply that state was very keen to implement performance management, and so missed a few steps in the process.

State senator Salvatore F. DiMasi is apparently a fan of performance management, and the Massachusetts Department of Education was required by the federal No Child Left Behind Act to generate reports about student, staff, and program performance.

This led local districts to start purchasing their own software, so the state decided on a larger, centralized data warehouse with data about students, teachers, and finances across the state.

The RFP process doesn’t seem to have gone smoothly, since Cognos was selected despite coming fifth on a scoring system that considered cost and other factors. I believe many (most?) RFPs are a waste of time and money, but they’re essential for transparency in the public sector, and if you’re going to do them, you have to at least pay lip service to the results.

Interestingly, the state was “sharply criticized” for doing something that I consider is almost a success factor for BI: taking a leap of faith that better access to data will bring improvements, without knowing exactly how the software will be used:

“the inspector general is urging that the administration first determine whether it needs to make such an expensive purchase. Before rebidding the contract, he said, the state should talk to employees “to see how or if they would use such a type of software.”

(For more on why I believe this, see point three of this post, or review this presentation).

Overall, there was apparently no great dissatisfaction with the Cognos software, so the biggest result is to delay 20,000 people from improving the Massachusetts education system through better performance management.