Using MinGW on Windows Vista (or even XP) can bring many unexpected errors due to UAC and filesystem restrictions. Fortunately, there's a convenient way to install MinGW on portable devices. It's MinGW Portable which is a PortableApp running on the PortableApps platform. MinGW Portable is said to be working well. To be honest, I think installing MinGW Portable is a lot simpler and effective than going through a bunch of setup programs. Be warned that MinGW Portable is an experimental beta program.
Just download these 3 installers and run them in that order: