I admit fashion can seem like a superficial topic. It almost feels silly to talk about it in any serious way. But the truth is, what you wear does affect how you feel inside and how people see you. In fact, scientists have found that the clothes you wear may actually affect your psychological processes.

But this isn’t about the science of fashion. This is about the fact that you can blow a boatload of money on clothes precisely for this reason—because what we wear affects us so strongly. Nice clothes make us feel good for a variety of solid reasons, and they’re one place we can easily overspend.

The problem is, when you’re in wealth-building mode (as opposed to work-optional mode), this quest to feel amazing in something new can sabotage your best efforts. I know because I’ve been there. I used to be an over spender. I’d go into a store and see something cute and on sale (or not) and think, Hey, this is a great deal/ I can do something with this/I can’t pass this up, rather than thinking about whether it was good or bad for my finances.

Then I realized something. Half the time, my purchases just took up space in my closet. I’d buy what I thought was a cool sweater, wear it once, and then prefer other sweaters I had instead. (As it is, research shows we only use 20% of what we buy. That means 80% ends up in the donation bin or a garage sale or worse—cluttering our space). I could think of better uses for that money, like financial freedom.

So in an attempt to de-clutter my closets and my life and grow my net worth, I reined in my spending. I got pretty good at it, and yet still, I found myself unable to sock away as much as I’d planned to. What was happening? I couldn’t figure it out. After all, I’d become so selective and (dare I say) frugal.

To get to the bottom of it, I did a little experiment. Granted, it’s not scientific, but it spoke volumes. It went like this: I decided I’d spend about $500 on the winter clothes I needed. When I thought I had reached my $500 limit, I would total it and see how I did. For example, did those purchases enhance my life? And did I stick to my budget?

A few weeks later, my shopping was done. I made a list of what I bought and totaled it. First let me say, I thought I had purchased about 3 or 4 things and was under budget. In reality, and to my shock, I’d bought 11. In my mind, I’d spent just below $500; in reality, I’d spent $1100. That number turned me a slight shade of green, not because of the money, but because I honestly couldn’t recall what I ever bought. I had integrated the clothes into my wardrobe and the novelty had already worn off. What’s worse: The items I bought—in hindsight—I could have done without, except for one or two. Most didn’t increase my happiness level in any way. I felt like I wasted hard-earned cash that could have been in a savings account, or invested and making money.

In talking to some friends, I found it wasn’t just me. Many people feel like there’s some “black hole” that sucks up your money even when you feel like you’re living simply and spending less.

But there’s a scientific reason for this. Our subconscious brain plays tricks on us. The field of behavioral science has ample studies showing we’re not the logically-thinking creatures we think we are. We don’t think of facts as they are. We think in emotions. In our minds, what we’re emotionally tied to takes precedence over what we’re ambivalent about. Which means, the idea of feeling great in that new Ponte-knit dress can almost negate the fact that you parted with some Benjamins. Hence the dollar gets lost in the momentary joy of a new purchase. (When we spend, we actually feel a high that compares to a drug high). It’s like a spending blindspot; while we’re enjoying the thrill of the purchase, we don’t realize we’re spending as much as we are. And over the course of time, it adds up. That, for me, was the aha insight. That we spend more than we think we are, and it can compound ridiculously if left unchecked.