{"id":12263,"date":"2012-07-24T18:11:16","date_gmt":"2012-07-24T17:11:16","guid":{"rendered":"http:\/\/timoelliott.com\/blog\/?p=4119"},"modified":"2012-07-24T18:11:16","modified_gmt":"2012-07-24T17:11:16","slug":"test-learn-adapt-using-analytics-to-improve-public-policy","status":"publish","type":"post","link":"https:\/\/timoelliott.com\/blog\/2012\/07\/test-learn-adapt-using-analytics-to-improve-public-policy.html","title":{"rendered":"Test, Learn, Adapt: Using Analytics to Improve Public Policy"},"content":{"rendered":"

\"test_learn_adapt\"<\/p>\n

Ben Goldacre<\/a> of Bad Science<\/a> fame has co-written a UK Cabinet Office paper<\/a> on how to improve government policy through better analysis, in the form of randomized controlled trials (RCTs).<\/p>\n

A qualified doctor and professional debunker of bad medical analysis, Goldacre notes that RCTs are so widely used in areas such as medicine and web advertising that \u201cfailing to do them would be regarded as bizarre, or even reckless.\u201d<\/p>\n

However, using RCTs to test different approaches to solving public policy problems in social welfare, education, crime, and justice remains rare (see chart below).<\/p>\n

\"image\"<\/p>\n

The paper is a very clear introduction to how RCTs work and why they are so important. The various myths that prevent their wider usage (cost, effort, fairness, etc.) are thoroughly debunked.<\/p>\n

\"\"<\/a><\/p>\n

The document includes UK examples of RCT successes: trials showed that text messages were effective at prompting people to pay fines on time, but giving the unemployed administratively-cheaper online alternatives to \u201csigning on\u201d every two weeks increased the time it took them to find work.<\/p>\n

One example clearly communicates the value of the RCT approach. The US \u201cScared Straight<\/a>\u201d program attempted to deter juvenile delinquents from criminal behavior by exposing them to the \u201cfrightening realities of leading a life of crime.\u201d<\/p>\n

Scared Straight claimed success rates of up to 94%, and was copied in other countries \u2013 but the program had not included a control group. A later meta-analysis of randomized trials concluded that the program in fact<\/a> led to higher<\/em> rates of offending: \u201cdoing nothing would have been better than exposing juveniles to the program.\u201d<\/p>\n

The costs associated with the program were over 30 times higher than the benefits \u2013 meaning taxpayers spent a significant amount of money to actively increase crime!<\/strong><\/p>\n

The white paper is a great read for anybody interested in persuading others in their organization of the importance of using analytic techniques such as RCTs to improve outcomes — you can download it from the Cabinet Office website<\/a>.<\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

Randomized controlled trials are rarely used to improve public policy in areas such as social welfare, education, crime, and justice — and that’s a problem<\/p>\n","protected":false},"author":2,"featured_media":4117,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[2],"tags":[96,100,143,153,160,204,669,853,865,868,869,1086],"class_list":["post-12263","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-best-practice","tag-analysis","tag-analytics","tag-bad-science","tag-ben-goldacre","tag-bi","tag-business-intelligence","tag-justice","tag-public-policy","tag-randomized-controlled-trials","tag-rct","tag-rcts","tag-trials"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/timoelliott.com\/blog\/wp-content\/uploads\/2012\/07\/test_learn_adapt.jpg?fit=690%2C310&ssl=1","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p3X9RF-3bN","_links":{"self":[{"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/posts\/12263","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/comments?post=12263"}],"version-history":[{"count":0,"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/posts\/12263\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/media\/4117"}],"wp:attachment":[{"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/media?parent=12263"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/categories?post=12263"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/timoelliott.com\/blog\/wp-json\/wp\/v2\/tags?post=12263"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}