Key Takeaways The initial accumulation of $100,000 is the hardest and most important step in building wealth, Munger says. As Warren Buffett’s legendary right-hand man at Berkshire Hathaway, Charlie Munger spent decades building a $2.6 billion fortune. Munger’s advice on how to start building wealth was blunt and honest. Munger, who passed away in 2023, said the hardest part of getting rich is putting together that elusive first $100,000. Once you’ve achieved this, the power of compounding and steady investing makes wealth building much more achievable over time.