Women show “fear of fatness” and men a “drive for muscularity.” Moreover, women perceive themselves as larger and men more muscular than they actually are. We tested potential memory biases congruent with these sex-specific body image concerns. Free recall performance for weight-related and muscle-related positive and negative words was assessed in 40 healthy undergraduate students (20 men). Men revealed a recall advantage for positive muscle words, while women showed a general advantage for positive and negative weight-related words. Thus, men revealed a memory bias congruent with their personal preference (more muscular), while women showed a general memory bias for weight information independent of their personal preference of being thinner. The absence of a positive memory bias in women might explain the higher incidence of body dissatisfaction and eating disorders in this population.