![](\"https:\/\/i0.wp.com\/timoelliott.com\/blog\/wp-content\/uploads\/2023\/12\/img_4848-scaled.jpg?resize=2491%2C2560&ssl=1\")
<\/p>\n<\/p>\n
At dinner the other day, my daughter was explaining a complex probability question in a recent mathematics test. My wife asked if she had the test paper, and suggested she put into ChatGPT to get help.<\/p>\n
I\u2019m sure everybody can relate with that idea. After all, the entire history of computing has told us that computers are good at maths and logic. So when we hear that artificial intelligence has had a considerable breakthrough we all automatically assume that it\u2019s in the same direction.<\/p>\n
But that\u2019s what\u2019s so confounding about the latest large language models. They\u2019re generally terrible<\/em> at mathematics and struggle with logic problems that children can easily manage.<\/p>\n Instead, what they\u2019re really good at is something we think of as much more human: creativity. They are deeply untrustworthy when it comes to facts and logic, but great at brainstorming and new ideas.<\/p>\n This wave of computing and AI is different<\/em>. Throw out your mental model of software that does what you tell it to do. Instead, you have to talk to these models as if you\u2019re trying to communicate to an alien species. They respond better to coaxing and threats than they do to orders and commands. Studies have shown that they give better answers if you are polite<\/a>, promise to tip<\/a>, or if you say that your life depends on it. It may even get sluggish just because it’s December<\/a>!<\/p>\n The good news is that the next wave will apparently be back in line with our initial expectations. Researchers are working on models that are dramatically better at step-by-step reasoning, and can work their way through complex mathematics problems like my daughter’s probability test (e.g. the rumors swirling around OpenAI\u2019s Q*<\/a> or the student math example from Google\u2019s Gemini Ultra<\/a>).<\/p>\n That really will open up the opportunity space for how useful these technologies are in the real world, and get us a lot closer to the holy grail of artificial general intelligence (AGI).<\/p>\n