Can You feel it? If so, I'd probably scrub the mold with a plastic-bristled brush and soap, rinse and dry. Go easy, though, at first, because some Sandpaper is made with Aluminum Oxide.
May even wipe them with a Paper Towel before scrubbing...blow the Loosened Crud out.
Edit - New Molds do have "blemishes" in the cavities, so if You can't feel the blemishes, it's probably fine.
