The speaker explains that the difference between the rich and the broke is that broke people spend money on things they don't need and have nothing left to invest, while rich people invest first, then spend what remains. This ensures the broke remain broke and the rich get richer.