Algorithmic Self-Awareness

The phenomenon

Jack Dorsey, surfacing an idea from Yuval Noah Harari in conversation with Lex Fridman, describes self-awareness as being progressively offloaded to external algorithms, producing a population that knows less about itself than the systems built to serve it. In his paraphrase: "Google has a stronger sense of their preferences than they do. The self-awareness is being offloaded to other systems, particularly these algorithms."¹

For anyone who grew up alongside Google, Facebook, and Twitter, the algorithms behind those systems have been observing choices for years, and by now likely know what content a person engages with versus skips, what products they are likely to buy, what emotional states trigger what behaviors, and what health patterns correlate with their choices. That self-awareness is often more accurate than the person's own introspective account of their preferences. The concern is that outsourcing self-awareness to an algorithm forfeits the ability to know oneself independent of it, and because the algorithm's objective is usually engagement or sales rather than the user's own goals, the self it reflects back is shaped by those incentives rather than the person's.

Why it matters at scale

Dorsey's examples are mundane and constant: should I stand, should I walk today, what doctor should I choose, who should I date. Decisions that used to require self-knowledge, asking what one's own body or judgment actually needs, are increasingly delegated to recommendation systems. Any single delegation is small and often genuinely useful. The aggregate effect is a population whose muscle for self-directed choice has atrophied.

The double-edged nature

Dorsey does not frame this as purely negative. "The best of what recommender systems can do is to help guide you on a journey of learning, of learning period. It can be a great thing. But do you know you're doing that? Are you aware that you have that invitation and it's being acted upon?"¹ The recommender system itself is not the problem; unawareness of its operation is. The same recommendation that helps a self-aware user discover something genuinely valuable can trap an unaware user in a preference loop they never consciously chose. The load-bearing distinction is between an invited algorithm, one a person deliberately lets guide their reading or learning, and an uninvited one that shapes a person without their awareness.

The countermeasure: meditation and deliberate reflection

The prescription Dorsey attributes to Harari is meditation, turning attention inward and asking why a thought arose, where it came from, and why one reacted a certain way. This is offered not as a spiritual practice but as a technical countermeasure: meditation rebuilds the internal self-awareness infrastructure that algorithmic offloading erodes. Dorsey's own version includes treating mortality as a clarifier, thinking about death multiple times a day to ask whether he is spending his time on what actually matters, deliberate discomfort through fasting, speech club, and physical challenges that force him to encounter his own limits without algorithmic mediation, and single-pointed focus practices like eating one meal a day and sustained meditation, building the capacity to sit with an experience rather than immediately outsourcing its interpretation.

Open questions

Whether meditation is a sufficient countermeasure, or whether the sophistication of recommendation systems eventually outpaces any individual's capacity for self-knowledge, is unresolved; Harari frames it optimistically, but the trajectory is uncertain. As agents move from recommending to acting autonomously on a person's behalf, the offloading shifts from preference to action, and self-awareness of one's preferences may become less relevant than self-awareness of the values a person actually wants those agents to optimize for.