You might find your ancestors in the 1860 census as much immigrants had swelled the colonial population by that time. Why not a member of your household struggled in the Civil War and you will see the individuals of the best age that would have struggled in the war