A Chinese farmer had spent months learning to trust AI with his farm, until one recommendation wiped out nearly 25 acres of his sesame crop. The 67-year-old farmer, identified as Wu, had reportedly been using an AI application for agricultural advice, including questions about fertilizers and crop protection, and had grown more confident in its recommendations after earlier advice appeared to work. That confidence eventually led him to follow an AI-generated pesticide treatment for weeds and pests across roughly 150 mu of sesame. The problem is not that AI will always be wrong; it is that a system can repeatedly give useful answers, build a user’s confidence, and then produce a bad recommendation with the same level of confidence. That makes human verification especially important when the cost of an error is high. Details of the incident Several outlets reporting on the incident confirmed that the AI generated multi-chemical treatments. But the exact number of chemicals used varies by publication. Per Tom’s Hardware, the chatbot’s advice included mixing flupyrimethalin and flusulfasulfaether with thiamethoxazine and methyl …