Business Owner?—Unlock your next level. Explore our Business Planning services

Hidden Portfolio Risks That Can Hurt Returns | Smallwood

Video Posted: Jul 23, 2025

Today, I have a concept that I want to talk about. There’s something in financial planning and the way we think about money that bothers me. And it has to do with time in the market and the volatility of a market.

I’m recording a screen here from JP Morgan that talks about a stock portfolio. I’m just going to focus on the green line, which is the stock portfolio.

The one-year volatility on this: the market could be up 52%, down 37%. If I’m in the market for 5 years, I can be up 29%, down 2%—somewhere in there. If I’m in the market for 10 years rolling, it’s 20% to 1%. If I’m in it for 20 years, they’re saying floor of about 6%, upside of about 18%.

When you start thinking about it, let’s assume I take $100,000 and I invest it, and it grows to $1,000,000 as an example. I don’t have a time frame. I’m not looking at it, but I’m in the S&P 500. Has my one-year volatility changed, or is it still going to be the same one-year volatility? Can I be up 52% or can I be down 37%?

And the answer to the question is yes, if I’m in the same bucket. But when I started with $100K, I don’t think I would be able to handle it if I was down 37%, right? So I’ve got to look at that. Now let’s look at what happens when I grew it to $1M, and now it’s down 37% or more in one-year. And if you remember ‘08–’09, ‘08 was down 37% and ‘09 was down 25% into March, which, collectively, you were down 52–53%. If I’m close to retirement and I have a million dollars that falls 40–50%, that is detrimental. So yes, my $100K is still worth $400K or $500K.

So the time of the money in the market is fine, but the one-year volatility has not changed. It never goes away.  As a matter of fact, it becomes more likely the longer that you’re in that you’re going to have one of those 50% drawdown periods. Like, this is one-year numbers versus two-year numbers.

And then the question is: do I diversify? According to this from 1950 to 2023, they’re saying bonds up 33, down 13 in one year if I put all money in bonds. And the index that they’re using for the bond market is Bloomberg Aggregate Bond. But the bonds only created $279K versus $869K per $100K. And then you look at it from a 60/40 stock/bond ratio standpoint, and the 60/40 created $592K, but I still have a one-year volatility of up 34%, down 20%.

The problem is the psychology of the investor. If I’m down 50% so $1M goes to $500K. To get back to even, I’ve got to make 100%, right? It’s not “I’ve got to make 50%.” I’ve got to make 100% to get back to break even. So now assume I’m withdrawing money from that portfolio and I’m selling shares. When the portfolio is worth a $1M, I’m withdrawing two shares to get my income. Now get the same income when it’s down 50%, I’ve got to withdraw four shares. And now, when the market recovers, I have two fewer shares than I should have had, meaning the portfolio doesn’t recover.

There was a book years ago by a gentleman named Ed Easterling. My father loved the book, and it was about risk. And his position is that risk doesn’t go away and said, “risk is not like fertilizer/ I do not spread it onto my portfolio and magically it grows”. I could put too much fertilizer and kill the portfolio. Too much risk. So, what I want you to take away from this conversation is that risk does not go away with time in the market. The one-year volatilities are always there.

The question is do I have other buffers and other things in place that, when it does happen, I don’t have to panic? Because we want you to get to the $800K versus the $600K versus the $300K, but how is it that you’re doing it? Are you taking money and repositioning it? Are you creating a defensive structure?

And when you simulate returns and they say the Monte Carlo simulation is going to get you 6% over the time frame, your expected outcome is you’ll have somewhere between 1 million and 3 million, and you’ll have money at 85, it’s creating a smooth ride. Nobody gets a smooth ride. You get volatility-adjusted returns. You don’t get the average of a thousand different markets. You get whatever the market is in that year, the following year, the following quarter, the following month/ You get those returns. And you don’t know what those returns are, because past performance is no guarantee of future success.

The market will not play out the same way. The predictions that say, “based on this event, the market should do this” don’t happen. Or when people say “we didn’t see this Black Swan event coming. We didn’t understand the credit crisis. We didn’t understand the dot-com bubble.” Sure, we did, afterwards.

But the best philosophy that you’re going to have is what are your goals in the plan? What’s the role of your money? How do you have other things buffered around it? What’s your total macro plan? And people that have a plan and understand that if the market dropped 50%, it’s not a reason to go and run away. It’s a chance to say, “I’ve got cash. Let me deploy it!” If the market’s down 30-something percent, that’s opportunity. It’s bad for your old money, but it’s also an opportunity.

But the issue is the emotional roller coaster. You’ve seen the charts. When the markets are running up, everybody wants to climb in. It’s the most dangerous point. The odds of it coming down on the back end of a great run-up are far greater than after a long sustained downturn when everybody’s despondent and think it’s different this time and corporations are having problems.

This is a plan. You need to create a plan. You need to understand that that’s going to happen. How do I reduce the volatility? Is it with bonds? Is it with put options? Is it with annuities? Is it with life insurance? Is it with cash in the bank? Is it with real estate? What is your diversification strategy? It’s unique and different for everybody.

But if you are sitting on a massive amount of money in the stock market and don’t have defense and volatility when the markets drop 50%, what’s my planned reaction? How am I going to react to that? Do I have contingency? Do I have backup? Do I have the potential?

There are so many things that I want to bring into this, but I want it to be short and sweet today. And I want it to be as colorful with what I was saying for you to not be fooled into thinking that risk goes away the longer that I’m in the market. Yes, your 100 went to 600 or 800, but it’s the one-year risk that people can’t handle. I’ve been doing this for 35 years. I know people can’t handle down 35–40%. The human being. But if my collective portfolio with all these other backup and redundancies is down 8%, I can handle that. And to get back to even, I only have to make about 13%. That’s a lot easier.

Now, if you’re listening to this podcast and you’re still here and understand this, you want more information. We have a book. It’s called It’s Your Wealth—Keep It on Amazon. It got number one bestselling in retirement planning and number two in wealth management.

It’s a good book because it talks about the psychology of how to build a full macro plan. We call it “beautiful balance.” You must have balance. If your plan is all risk on, it may or may not work.

Ultimately, my dad used to say, “Money isn’t everything, but without it, you can’t get a date.” In retirement, money is not everything, but it’s not good without it. It’s not as enjoyable as what you thought. If you work for 40 years, you want to make sure that you enjoy what you’re doing.

This was awesome. I want you to take action. I want you to build a plan. I want you to understand different types of risk. You have market risk. You have tax risk. You have credit risk. You have liquidity risk. You have real estate risk. You have rental risk. You have health risk. You have inflation risk. You have plan obsolescence risk. You have risk everywhere. Do you understand where your risk is? Do you really understand? We’d like to help you figure that out. If not, find somebody that can because it’s really important that you get it right.

Learn More About Financial Planning

This communication is strictly intended for individuals residing in the states of CA, CO, CT, DC, DE, FL, GA, IL, LA, MA, ME, NC, NH, NJ, NM, NY, OH, PA, RI, SC, TX, UT, VA. No offers may be made or accepted from any resident outside these states due to various regulations and registration requirements regarding investment products and services. Investments are not FDIC- or NCUA-insured, are not guaranteed by a bank/financial institution, and are subject to risks, including possible loss of the principal invested. Fixed insurance products and services are offered through Ash Brokerage or Smallwood Associates, Ltd. Fixed annuities are long-term insurance products. Before you purchase, be sure to talk to your financial professional about the annuity’s features, benefits, and fees, and whether the annuity is appropriate for you, based on your financial situation and objectives. All guarantees are based on the continued claims-paying ability of the issuing company. Investment Advisory Services are provided by Smallwood Wealth Investment Management, LLC, an SEC-registered investment advisor. Headquartered at 199 Broad Street, Red Bank NJ 07701-2056. Securities are offered through Purshe Kaplan Sterling Investments, Member FINRA/SIPC. Headquartered at 18 Corporate Woods Blvd., Albany, NY 12211. Purshe Kaplan Sterling Investments and Smallwood Wealth Investment Management, LLC are not affiliated companies. NOT FDIC INSURED. NOT BANK GUARANTEED. MAY LOSE VALUE, INCLUDING LOSS OF PRINCIPAL. NOT INSURED BY ANY STATE OR FEDERAL AGENCY.

Share This Post

More To Explore

Download Your Free Smallwood Wealth Business Owner Guides

Instant download — no spam, just pure value.