“We owe ourselves to death” (Derrida 2010). This paper takes mourning as its theme, tracing its changing roles in the archives of psychoanalysis and into contemporary debates on the place of the human, on subjectivity and on selfhood. Since Freud, at least, mourning has been understood as a particular form of memory work, a laying to rest of the other through recollection and forgetting. However writers and analysts since Freud have problematised this process, shifting the work of mourning from an incidental event in the life of the subject to one which is generative of the subject. The paper tracks these changes, tracing in the unworking of Freud’s vision of memory work the emergence of a very different figure of the subject; the subject taking shape through its inability to recall the other, and the other through their irreducibility to my recollection. That is to say the subject defined not in its positivity, though any particular trait or attribute or combination thereof, but rather in their constant deferral and withdrawal. The subject, that is you, facing away, in your distance, in your negativity. A negativity, a deferral or withdrawal, traced in the small and, conceptually, ever shrinking difference between the pronouns ‘you’ and ‘it’. What do I recall when I recall you? When I recall us?