If all horcruxes were destroyed, the anchor that kept the maker bound to the living world would be gone, and the maker would be mortal and killable again.

Destroying a horcrux required that the object containing the soul fragment be damaged to a point beyond any and all physical or magical repair, which in the case of living receptacles, meant that they must be killed.

However, horcruxes, by their very nature, appear to be extraordinarily durable, and seem completely impervious to most known physical destruction methods and magical spells. Only the most powerful and potently destructive magic and processes could truly destroy them.

Furthermore, as a safety measure to protect one's immortality and precious soul fragment, the creator would usually place powerful enchantments onto the artefact to help prevent damage.

The only things known to be capable of destroying a horcrux are Basilisk venom and Fiendfyre.

The killing curse can also be used to remove the foreign soul fragment from its vessel, though this was only seen used on a living horcrux, so it is unknown if the killing curse would also work on an inanimate object horcrux.