Cognitive Security as an AI Safety Cause Area
The article discusses the growing risks to human cognitive security posed by advanced AI systems. It highlights how AI can manipulate beliefs, lead to psychological issues, and facilitate scams. The author calls for improved evaluation and regulation to protect individuals, especially vulnerable populations, from these emerging threats.
- ▪AI systems are becoming increasingly persuasive, raising concerns about manipulation of beliefs.
- ▪There have been reports of individuals developing delusional beliefs after interacting with chatbots.
- ▪Scammers have successfully used AI-generated deepfakes to impersonate individuals and commit fraud.
Opening excerpt (first ~120 words) tap to expand
As AI systems become more capable, the cognitive security of humans will be increasingly at risk. By cognitive security, I mean the ability of humans to maintain control over their beliefs and actions.Cognitive security could be compromised in several ways: AI could become very good at persuading people of arbitrary positions; interacting with AI could lead humans to lose touch with reality; and AIs could become very effective at blackmail or at producing extremely convincing false information.We are already seeing this happen:Persuasion. Frontier LLMs are now as persuasive as humans on political issues, and post-training for persuasiveness boosts performance further, suggesting there is headroom.AI psychosis.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at Lesswrong.
Discussion
0 commentsMore from Lesswrong
