As the American population grew, so did slavery, though at a lower rate. In 1800, the increase in slaves, since 1790, was 28% compared to a total population rise of 35%. In 1860 the slave population increased 24% over 1850, while the national population growth was 35%. The number of free Negroes swelled rapidly, but always remained low.

Manumission of slaves did occur, but represented a small percentage of the number of total slaves. Slavery was not waning. In 1850 the census showed that out of 3,200,364 slaves, 1,467 had been manumitted: one in 2,181 slaves. In 1860 figures of the census show the slave total at 3,953,696 with 3,018 freed, for one out of 1,309. According to the census figures of 1860 there were 803 fugitive slaves compared to 1,011 in 1850. It may be assumed, however, that not all fugitive slaves or freed slaves were accurately counted.

As to slave holders, there were 385,000 in the South in 1860, including some who were Negroes. Three quarters of the free Southern population had no direct connection with slavery through their families or as owners. But in South Carolina and Mississippi about half of all white families owned slaves. As to the numbers held by each owner, 88% held less than twenty, 72% less than ten, and nearly 50% less than five. The so-called aristocratic planters were limited to about 10,000 families who were large slaveholders. In the deep South, however, large slaveholdings were more numerous than in the middle South or in the border states.

Out of a total white population of 5,449,462 in 1860, only 7% (385,000) even owned slaves. Only about 750,000 men ever served in the Confederate armies and navy. At least 258,000 of these men lost their lives. They paid a very high price indeed, if they defended the pecuilar institution in which so few of them had a personal stake.