← Back to AICS Next: Lecture 7 →

Lecture 6 discusses the rise of AI systems with socioaffective design features. We consider the growth of anthropomorphic and relatability-seeking AI, including companion applications. We discuss the impact of excessive validation (sycophancy), how design features of these products may prolong engagement and maximise self-disclosure, and consider evidence that they have a negative effect on wellbeing. We ask what ‘AI psychosis’ may be and how it is provoked. We consider the impact of AI relationships on people, and discuss the outlook for the future, including the possibility that AI systems are considered moral patients in need of “welfare”.

What you need to understand:

  1. The intentional Stance
  2. Steering vectors
  3. Companions applications
  4. AI Sycophancy
  5. AI psychosis
  6. “wanting” vs. “liking”
  7. AI Model welfare

Sample essay questions

What psychological impact do AI models with socioaffective design features have on their users?

AI systems are increasingly being judged to be “friend” rather than “tool”. What implications does this have for society?

Reading List

Nass 1994

Dennett 1971

Weizenbaum 1966

Kirk 2025

Akbulut 2025

Kirk 2026

Rauh 2026

Pataranutaporn 2021

Zhang2026b

Zhang 2026

De Freitas 2025

De Freitas 2024

Rathje 2026

Cheng 2026

Ibrahim 2026

Nakagomi 2026

De Freitas 2026

Dohnány 2026

Morrin 2026

Long 2024

Sofroniew & Suleyman 2026

Akbulut 2026