You definitely want to address the mold, and yes it can cause a musty smell.
Four things are necessary for growth of normally found molds in construction materials. Mold spores, which are everywhere. Temperatures in the range of 40-100 degrees F (that's why we refrigerate foods). A food source (organic materials, such as wood and paper facing on drywall). Relative humidity of 60% or greater or an active water source of 0.6. Certain species of molds will thrive in different combinations of temperature and humidity conditions. Some mold species will live on and consume other species as conditions change. Some less common molds will thrive in conditions radically different than those above (molds have been found in Antarctica for example). Mold colonies will produce spores while they are actively growing. When an organic building material is returned to it's relatively normal state, such as having been dried to normal levels of RH, mold growth ceases and the colony will become dormant. For short periods, the colony may remain intact and can become active once again if the material is again rewetted. If the colony remains dry it can basically fall apart into its base components. These components can be a source of allergic reactions to sensitive people, which is why mold should not only be killed, but cleaned up. Undisturbed, the spores left behind by the colony will generate new colonies if the supporting material reaches a temperature/humidity level to promote growth.
That is a very brief description of part of the life of a mold. There are thousands and thousands of species of molds. Some are very uncommon, while others are everywhere. All require an organic food source that we easily provide in our homes and buildings.
One of the best products you can get right now is Fiberlock Shockwave. You'll want to make sure you have access to all parts of the inside of the unit. Spray anywhere that you see visible mold, making sure that the affected areas remain wet with the product for 7-10 minutes. You may want to employ an HVAC guy to cut an access panel for you for future sprayings, as you will most likely not be able to completely and permanently get rid of every bit of mold, and the drip pan is an excellent breeding ground during the summer when the AC coils drip on it daily.
Allow the Shockwave to dry completely, then follow up with a 50/50 solution of bleach and water. Someone recommended that the bleach treatment be used first so that may be the better process to follow, but both are effective. Scrub off any visible mold and mildew with a 3M scrub pad or sponge and let dry completely.
You might also look into sealing any leaks in your returns that could draw basement air into your system. Usually HVAC caulk and foil tape are effective at the duct joints, but if you want a complete sealing process, look into Aeroseal.
Shockwave This is just a Quat cleaner (with a mild fragrance added, and possibly a small amount of a deodorant) and you can buy virtually the same thing for $4 a gallon at Sam's club. It's called Sanitizer, and it will have the word Quat listed in the ingredients.
Every janitorial supply store will have at least one Quat cleaner available.
For that matter, you can even use regular Lysol or some kinds of Mr. Clean, look at the labels, there are many of them that are Quats.
Some have water soluble deodorizers in them, which can help with odors, but in the case of mold, aren't really what you want to do, as they don't really solve any problems.
Diluted bleach really works at least (if not better) as well as anything else, but is more caustic, so if it is going to be used on a regular basis, the Quat cleaner is much less damaging.
All disinfectants rapidly lose there efficacy in the presence of organic contaminate. i.e. Clean the surface as much as possible prior to attempting to disinfect.
In my case, the entire problem was caused only by a clogged condensation drain for my HVAC unit. Once that got cleaned out and the coils were cleaned out, everything was fine. Of course, I also don't have any drainage issues around the foundation, nor are there any plumbing leaks--my crawlspace is the most dry I've ever seen (and I do not have poly laid down).
I am told that those condensation drains should be cleaned out twice yearly. Mine is particularly long (probably about 15 feet or so), and it doesn't have much of a fall, so it's susceptible to clogging.
"A Brief Guide to Mold, Moisture, and Your Home"