The above would seem to suggest that every piece of ferromagnetic material should have a strong magnetic field, since all the spins are aligned, yet iron and other ferromagnets are often found in an "unmagnetized" state.
The reason for this is that a bulk piece of ferromagnetic material is divided into tiny magnetic domains (also known as Weiss domains). Within each domain, the spins are aligned, but (if the bulk material is in its lowest energy configuration, i.e. unmagnetized), the spins of separate domains point in different directions and their magnetic fields cancel out, so the object has no net large scale magnetic field.
Ferromagnetic materials spontaneously divide into magnetic domains because the exchange interaction is a short-range force, so over long distances of many atoms the tendency of the magnetic dipoles to reduce their energy by orienting in opposite directions wins out. If all the dipoles in a piece of ferromagnetic material are aligned parallel, it creates a large magnetic field extending into the space around it. This contains a lot of magnetostatic energy. The material can reduce this energy by splitting into many domains pointing in different directions, so the magnetic field is confined to small local fields in the material, reducing the volume of the field. The domains are separated by thin domain walls a number of molecules thick, in which the direction of magnetization of the dipoles rotates smoothly from one domain's direction to the other.
Thus, a piece of iron in its lowest energy state ("unmagnetized") generally has little or no net magnetic field. However, if it is placed in a strong enough external magnetic field, the domain walls will move, reorienting the domains so more of the dipoles are aligned with the external field. The domains will remain aligned when the external field is removed, creating a magnetic field of their own extending into the space around the material, thus creating a "permanent" magnet. The domains do not go back to their original minimum energy configuration when the field is removed because the domain walls tend to become 'pinned' or 'snagged' on defects in the crystal lattice, preserving their parallel orientation. This is shown by the Barkhausen effect: as the magnetizing field is changed, the magnetization changes in thousands of tiny discontinuous jumps as the domain walls suddenly "snap" past defects.
This magnetization as a function of the external field is described by a hysteresis curve. Although this state of aligned domains found in a piece of magnetized ferromagnetic material is not a minimal-energy configuration, it is metastable, and can persist for long periods, as shown by samples of magnetite from the sea floor which have maintained their magnetization for millions of years.
Alloys used for the strongest permanent magnets are "hard" alloys made with many defects in their crystal structure where the domain walls "catch" and stabilize. The net magnetization can be destroyed by heating and then cooling (annealing) the material without an external field, however. The thermal motion allows the domain boundaries to move, releasing them from any defects, to return to their low-energy unaligned state.