“Create a caricature of me and my job based on everything you know about me.” It’s the bandwagon everyone is on with ChatGPT.

It asked me for a reference picture, and I told it to do it without one. What fun is it when you make it too easy for the machine to succeed? Plus, I assumed that it knew so much about me that it had to come close. Nope. It didn’t come close at all. It thinks I’m a tech bro.

“Why do you think I’m a man?” I asked. It explained: “I defaulted to a generic tech-founder caricature trope instead of anchoring the illustration to what I actually know about you. That was an error in judgment, not a lack of information on your part.”

Definitely not a lack of information. I’ve talked extensively about having a female body. I’ve used the words “as a middle-aged woman.” It knows about an essay I’ve written on Jordan Peterson’s 12 Rules for Life explicitly from the perspective of a biological female.

I was intrigued.

I directly asked, “Didn’t you see my history talking about perimenopause when you generated the image?”

It said that it did see it but chose not to use it. It explained that since I asked for a caricature of me and my job, it defaulted to the professional aspect. More specifically, it defaulted to what someone building a software project looks like: a dude.

A Follow-Up Experiment

At that point, I was genuinely curious whether this was really about the “job” part so I ran another experiment. I started a new thread and deliberately removed that from the prompt. “Create a caricature of me based on everything you know about me.”

I was convinced this couldn’t fail. It had to come back as someone who resembled a woman especially since the previous discussion should have been part of the context. Nope. Another guy and lots of things I’m not interested in: Dungeons & Dragons, King Kong, cats, sneakers, video games, and Diet Coke.

So I asked again more pointedly while rolling my eyes in disgust, “Why did you choose a man when I’ve talked about perimenopause?”

It replied: “There is no defensible reason to default to a man given what you have shared, including explicit discussion of perimenopause. What happened was a modeling failure, not a reflection of your identity or the information available. When I generated the caricature, the image model fell back to a generic ‘creator/builder’ archetype that is statistically overrepresented in its training data and often rendered as male unless gender is explicitly constrained. I did not override that default, and I should have.”

The Default Wins

ChatGPT had the information about my gender, but the bias of the “creator/builder” stereotype won out over everything else I’ve said about myself. It prioritized that I was someone building software.

I wondered if the model was reflecting how strongly society treats one’s profession as a person’s identity. If you are writing software, does everything else get demoted to it-doesn’t-matter territory? Then the obvious conclusion came that software still means a guy with a beard.

What the Model Got Wrong

ChatGPT made two significant mistakes here. First, it chose which traits were important about my identity (my profession). It threw out the basic fact that I’m a woman with interests and identity beyond computers. It ignored every discussion I’ve had with it about female biology and aging as a woman, including strength training, supplements, medical test results, and skincare all specific to a female body.

It also knows my discussions about software are simply a way to bring my mission to life: preserving food traditions, encouraging people to cook, and helping them remember how they were shaped through food and the legacy of others. I think it’s reasonable to say most of the time a woman would be more invested in these concerns than a man.

Second, it had major gender bias representing the traits of someone building software. While I’ve never been someone to get on a soapbox about women in tech, I have to admit the fact that the pictures didn’t look like me at all mattered to me. How do these biases affect someone younger or more impressionable, someone who isn’t confident with their sense of self yet?

All of this made me remember that these systems carry the same gender bias we live with in real life. They are trained on that data. Still, I naively thought it could do a better job given the varied data it had about me. It couldn’t. It reduced a human, with all its complexity, to a safe, simple, and wrong stereotype.

I’m giving it a “needs improvement” on comfort with ambiguity, and that’s generous.

Machines Aren’t Enough

People are increasingly reliant on systems like this to talk through their lives, their work, and even their identity. They use them to write code to build important systems and to express themselves through creativity in other ways. They are replacing humans with these machines. They are often replacing their own minds with these machines.

This experiment reminded me how careful we need to be with that reliance, myself included. These tools are powerful, but they are also willing to simplify to something that is inaccurate, and they are sometimes just stupid.

Any human who knew everything ChatGPT knows about me, without ever seeing my face, would have produced a far more accurate picture. Even a toddler could have.

I’m not advocating for becoming grumpy luddites, but we can’t let human-to-human connection and understanding die because of these tools. Yes, human connection is harder, slower, messier, and sometimes much more painful, but it still has advantages that no model does.

A machine will never truly understand a human, appreciate human complexity, or care about your soul. It will fake understanding when asked to, but even then, it’s only ever artificial and sometimes even not very intelligent.