OMG what a disaster! I didn't know all of this! I am a Mac user and I am used to things just to work!!
I'm a Mac user too, even if one of my Macs mostly run Windows nowadays (just because of Flight Sim)...
So basically I don't have a choice but return the RAM?
If your system didn't had problems with less RAM, it's probably the best option and, with the money buy an SSD hard drive THAT will really help. More than 6GB it's probably overkill for FSX.
Also, there's the issue of DDR3 memory working best in multiples of 3 (triple-channel), which means 6GB it's usually faster then 8GB. This is valid for Macs too, no wonder the MacPro comes with 3 or 6GB standard.
I don't understand is that there are users that never have issues, in fact they always say that their FSX running smoothly like a butter.... etc. I don't understand how can they do that?
First, try this site here:http://www.venetubo.com/fsx.html
Which will analyse your FSX config and create an optimized version based on your system, it's free, and it's worth trying first.
If you are still having problems, maybe your system has more serious issues. In this case, you can try this service:http://fs-gs.com/
It's a pay service, but from what I've heard, the guy is a very knowledgeable professional, which will guide you to a complete system check and optimization for FSX.