Disgraced Cavalier: How Light-Horse Harry Lee Lost His Fortune and Family

This is the tale of how debt, speculation, and imprudence turned Light Horse Harry Lee from a celebrated Revolutionary War hero praised for his cavalry raids, his tactical brilliance, and his closeness to George Washington into a disgraced scoundrel known throughout Virginia as a dishonorable debt dodger.

In 1782, Harry Lee returned to Stratford Hall to marry his wealthy cousin Matilda Lee, known as "the Dividine Matilda", and step into plantation life, but the quiet work of farming and managing property did not suit him. His military energy and ambition, which had served him well in war, became liabilities in peace.

Harry turned to speculation, debt, and land schemes, including failed ventures in Mississippi and the planned town of Matildaville near the Great Falls of the Potomac. He also tried to help veterans and pursued a political career as a Federalist, serving in the Continental Congress and the Virginia Convention. Despite these activities, his finances worsened, and his reputation suffered as he sold land, borrowed heavily, and failed to manage his estate.

Matilda's death and the trust she created limited Harry's control over Stratford, but did not stop his decline. He later married Ann Hill Carter, whose family protected her inheritance from his debts. His role as governor of Virginia and then commander during the Whiskey Rebellion briefly restored his military standing, but political opponents removed him from office soon after. Stratford fell into disrepair as creditors closed in and the family's resources shrank.

Harry's ruin deepened through fraudulent and reckless dealings, including disputes over land and repeated defaults on obligations. He alienated George Washington, and his financial desperation dragged his brothers into debt as well. In 1807, Robert E. Lee was born at the fortified, debt-ridden Stratford Hall. By 1809, Harry was arrested and sent to debtor's prison, where he wrote his memoirs in hopes of earning enough to recover.

After his release, Harry was forced out of Stratford and moved with Ann and their children to Alexandria. During the War of 1812, he was badly beaten by a mob in Baltimore while defending a Federalist newspaper office, leaving him disfigured and disabled. He then fled to the West Indies, where he spent his final years in exile, writing moral advice to his children while living in poverty.

In 1818, dying and in severe pain, he was carried to Cumberland Island, Georgia, where he spent his last weeks at Dungeness before dying and being buried there. His death left a lasting mark on Robert E. Lee, who built his own life around discipline, self-control, and avoidance of debt. In 1913, Harry's remains were finally brought back to Virginia and reburied beside his son at Lee Chapel.

CHAPTERS:

0:00 The Summit of Light Horse Harry Lee's Glory

3:05 Harry's Fall Begins: Land Boom Fever and Matilda's Will

18:34 The Courtship of Ann Lee: Courtship and Control

24:43 Debt and Decline: Betraying Friends, Including the Heroic Washington

42:38 Robert E Lee's Birth Amid Ruin and Debtor's Prison

50:00 The Baltimore Mob Attacks

53:30 Lee's West Indies Exile

1:01:49 How Harry Lee Shaped Robert E Lee

Sources Referenced in this Episode:

I am an Amazon Affiliate. If you would like to support the show at no added cost to yourself, you can do so by using the links below to order and read the sources I used to create this episode. Thanks!

Cole, Ryan, Light-Horse Harry Lee: The Rise and Fall of a Revolutionary Hero, https://amzn.to/4wiZOy3

Evans, Emory G., A Topping People: The Rise and Decline of Virginia's Old Political Elite, https://amzn.to/4z2x6nL

Freeman, Douglas Southall, R. E. Lee: A Biography, https://amzn.to/4gkaqrx

Hendrick, Burton J., The Lees of Virginia, https://amzn.to/4xshjNe

Lee, Cazenove Gardner, Jr., Lee Chronicle: Studies of the Early Generations of the Lees, https://amzn.to/4q7GiTI

McGaughy, Kent J., Richard Henry Lee of Virginia: A Portrait of an American, https://amzn.to/4xn1qaS

Middlekauff, Robert, The Glorious Cause: The American Revolution, 1763–1789, https://amzn.to/4cbhg05

Nagel, Paul C., The Lees of Virginia: Seven Generations of an American Family, https://amzn.to/4wUxkfi

Nichols, James L., General Fitzhugh Lee: A Biography, https://amzn.to/4qeURoz

Royster, Charles, Light-Horse Harry Lee and the Legacy of the American Revolution, https://amzn.to/3TKWspX

Sydnor, Charles S., Gentlemen Freeholders: Political Practices in Washington's Virginia, https://amzn.to/4ggLJfH

Unger, Harlow Giles, First Founding Father: Richard Henry Lee and the Call to Independence, https://amzn.to/4hWbm6D