I hate to be the bearer of bad news, but what you're describing is a difficult problem to solve. There are a number of integration issues that you describe that will be non-trivial to coordinate. Your first step may very well be to take a step back and pick up a good .net book and gain some deeper knowledge of the framework. You'll need it to tackle the problem!
Not to discourage you, though. The problem is solvable. The first place you're going to need to look is at the DNN membership provider, and the concrete asp.net membership provider. You're very likely going to want to inherit your own hybrid provider from this, and extend it as necessary. Look at the AspNetMembershipProvider.vb class in the Library/Comonents/Providers/Membership directory of the SOURCE download (I'm reciting the path from memory, so forgive me if it differs slightly).
If you're looking for something quicker, I believe there is a module for sale that is supposed to handle login syncronization between two systems. You might want to search for that.
Finally, be very careful about planning on hosting an ASP application in an IFRAME. IE has some serious cross-scripting restrictions in place that heavily restricts cookies within IFRAMES (IIRC, you can't do any cookie stuff in an IFRAME). Search these forums for other people who have noticed the same thing and see if they've resolved it. Make sure this will work for your application before you spend a lot of time on it :)
Good luck!
B