Worm Oil / Softener / etc. can / are all considered plasticizers. Plasticizers are used to soften PVC (what soft plastic baits are made of - same stuff PVC pipe is made of - just a lot more plasticizer to make it soft). Volumes could be written about this, but to keep it short I'm just going to address the pigments and oil absorbing.
Pigments: I've never heard of a plasticizer causing a pigment to bleed. With that said, I do believe something that appears as bleeding could be caused if the original formula had an incompatible plasticizer in it (CCM Crystal Clear does not). However, this would manifest itself first by producing very oily baits (the plasticizer leaching out of the bait). More than likely what you have described sounds like you are using non-bleed colorant. There are pros and cons to both non-bleed and bleeding colorant. The NB doesn't bleed, but normally requires more pigment for the same color, and in some cases you may never get the same color. The bleeding colorant can produce better colors at times, but bleeds... So, you just have to weigh the pros and cons of what you are trying to achieve and which pigment will best meet your needs.
Oils being absorbed by baits: Yes, this will absolutely happen over time. Depending on the amount and compatibility of the oil (plasticizer) to the plasticizers in the original formula will dictate how fast it happens. You have a softened solid (vinyl) that is softened by an oil (plasticizer), the solid is going to continue to try and absorb extra oil if it is present (like a sponge). After quite a bit of testing we selected a Worm Oil based on it's performance, compatibility, lubricity, clarity and smell. I have reference samples that have been bagged for months that still have oil in them.
Are you using Crystal Clear Plastic or another brand with our worm oil? There is nothing wrong with doing this, but it may not deliver the same results we engineered for (i.e. faster absorption). Are you using our softener as worm oil? If so, the results will be very different. It will be absorbed into the baits in days. Our Softener and Worm Oil are not the same. They can be used the same, but the results will be different. Overall I don't know of any problems mixing and matching brands. In some cases there are even advantages. For example, there is a worm oil that I really like and is quite a bit cheaper than what we sell, however, it has phthalates in it and that's why we don't use it. Kind of like the pigments above, you just have to weigh the pros and cons of what you are trying to achieve and what will best meet your needs.
Just remember, you are assuming the role of chemist when you start formulating on your own. I think this is cool and would encourage it, just remember, there are a lot of variables. Documenting, test samples, and time are all critical to success (and safety). If you look at some of the best bait makers out there they are constantly trying new things. This is how we all grow and become better at our craft. If you aren't interested in this aspect of bait making, I would try and find a brand you like and work within their product line. This way when you have issues there is a single point to work through.
Hopefully this is somewhat helpful. There are other threads on the forum that go into more detail about different aspects of plastic, colorant, etc.
Jason