The summer of 2007 started like any other.

My ex-husband and I planned the trip for weeks: our two daughters, ages eight and ten, would board a plane and fly from Los Angeles to Florida. They would spend the entire summer with him, and a couple of weeks before school started, they’d return and our lives in California would go back to normal.

It was hard to watch the girls get on the plane alone, but I knew these trips would help them grow up to be independent women in a way that no other experience could.

That same summer, I landed a job with a documentary production company. I was fresh out of graduate school and this job held a lot of promise for putting me on a career track that was exactly in line with my goals. After much negotiation, my ex and I agreed that the girls would extend their visit with him until Christmas, giving me a good six months to focus on work without having to worry about daycare costs, after school pickups, or other realities of single parenting.

It all sounded like such a good idea at the time. On paper it made sense, but my lived experience was different.

I expected it to be hard and knew I would be sad, but I didn’t count on the stigma I would face then, and even now. I struggled with the internal guilt and feelings of selfishness but quickly rationalized them away. I knew (because both my ex-husband and I were veterans) that if we were still in the military and got called away, no one would consider either of us selfish for choosing the jobs we chose; they would say it was too bad and move on.

If we were still married and he decided to take a job somewhere internationally because it was a good career move and it had a higher salary, people would likely praise him for making the sacrifice all in the name of caring for his family and looking out for the best interest of the children. Again, I was pretty sure no one would call him selfish for making these imaginary choices.

I told myself that my situation was no different and that as long as my ex—their father after all—was willing and able, I should take advantage of the opportunity to get ahead so that I could ultimately provide the best environment for my daughters while showing them what it was like to be a professional working woman.

Despite how often I told myself these things, I could feel the stigma of not having my daughters living with me. When I became a single parent, no one offered me any special kudos, nor did I expect them—our situation was a product of divorce and I had to accept my new role with strength and dignity while being a leader for my family. But when my girls went to live with their dad, I was actually a little shocked at the number of people who praised him. “Wow!” they’d say. “He sounds like an exceptional man to agree to let the girls live with him!” At least one person called him a hero.

What? I couldn’t understand this. He was their father and I’d always believed that the minimum expectation from him should be that he watch out for our children. Furthermore, not that this was a tit for tat thing, but how come people didn’t have that same reaction when women agreed to have full custody of the children? What about women who took care of the entire household while their partners worked on their careers, married or not?

I realized at that point that rarely, if ever, did I hear someone refer to a noncustodial father as selfish—not selfish for breaking up the marriage, but selfish for not having full custody of the children.

On the one hand, I can’t say much of this was a surprise, having studied gender in both college and graduate school. But on the other, there was a part of me that secretly hoped that my university had given me a lot of outdated material to work from and that in the real world, things were different. But they weren’t and it made me sad.

In an article for The Establishment, Christina Vanvuren writes:

“According to the 2013 United States Census, 13.4 million parents are deemed ‘custodial’ and, of those, 1 in every 6 (17.5%) are fathers. That means about 2.5 million households are run by single fathers…This, according to the Pew Research Center, is up from 300,000 in 1960.”

The number of single fathers versus single mothers is still not equal, but clearly, this number is rising. Vanvuren’s article echoes the stigma of shame I felt in the early years. For me, the external shame nicely complemented the internal guilt.

By the time Christmas rolled around, I was very excited to have my girls back, even though I hadn’t really made any headway at work. I had a hard time focusing and found myself trying to hide my emotions with food and alcohol. I felt sluggish and depressed most of the time.

One day, my ex called to ask if the girls could stay through the end of the current school year. He put them on the phone and they told me about all their new friends, how much they loved their teachers, and about the cool activities coming up in the spring that they didn’t want to miss. “Please Mommy,” they pleaded. As much as it killed me, I said, “Okay.”

And there it was. If I’d known then that they would never come back, I would not have agreed so easily.

My daughters are now grown, both in college. I never lost touch with them and, in fact, when my job at the production company didn’t work out, I took jobs overseas with the idea that this would give them a great opportunity to travel—and it did. Before they were in high school, they’d visited more countries than the average American and that made me smile. I was able to reach my goal of broadening their horizons even if it came about in a circuitous way.

I never questioned my love for them, even though others did. Even now, I occasionally receive messages from people who originally believed me to be selfish, saying I gave up my girls to go travel. I’ve been told that I abandoned my children. I’ve been told that I didn’t love them. It seems that every step along the way, others were questioning my motives and adding to the guilt I already felt on my own.

After a while, I stopped telling people that I had children simply to avoid the harassment that came when they figured out I was the noncustodial parent. Yet, despite my belief that this was the best move for them, even all these years later, I occasionally wake up with tears in my eyes questioning if I did right thing.

Now that all these years have passed and I’m able to talk to my daughters more openly about it, I can see that I made the best choice—even though there were some very difficult times for everyone involved. This isn’t the best choice for everyone, but it was the best choice for my girls. Talking to them openly about these years has helped to chip away at the guilt, but I don’t know if it’ll ever completely fade.

There are a lot of reasons why mothers don’t have custody of their children. As a society, we should not immediately assume she’s in trouble with the law or that Social Services has whisked them away from her incapable hands. While it’s great when a father steps up, we need to stop acting like these fathers are more special than the hundreds of thousands of mothers who do the same job every day.

If we want women to be treated equally, we need to become aware of the deeply held prejudices that we all have and stop ourselves when we feel a judgmental thought approaching. If we can collectively begin to retrain how we think about women, men, and family, we will all be on a much better track.

So the next time you meet a mother without custody of her children, assume she loves them and know that there are reasons for this arrangement. And ask her about her children, because even if they don’t live with her full time, she is still their mother.