Foods that are moldy may also have invisible bacteria growing along with the mold. Yes, some molds cause allergic reactions and respiratory problems. And a few molds, in the right conditions, produce "mycotoxins," poisonous substances that can make you sick.