Hello, I'm by no means expert on the forums module, but hopefully i can offer a little help.
1) the core forums and the forum project are the same thing. dotnetnuke.com is currently using the last release of the forums module (3.20.9). At the moment the next version (4.4) is going through the release process, and should be out shortly . You can read some details about it's enhancements at http://www.dotnetnuke.com/Projects/ModuleForums/tabid/820/Default.aspx and see it in action @ http://www.dnnforums.com/ .
As for the 3rd party modules, I've not used them, but the activeforums module (http://www.activemodules.com/) has a very good reputation. In addition, the main developer, Will, provides excellent support.
Regarding performance, both of these scale up to support many users - the amount depends on your webserver/database server capabilities e.g. dotnetnuke.com typically has a few hundred concurrent users (see the panel on the right on the homepage http://www.dotnetnuke.com/ for an idea of current users).
2) dotnetnuke uses role based permissions, and AFAIK all the forum modules leverage that to determine whether or not a user has access/can view various forums (go to the settings on the forum and you'll be able to set view or/and edit access for nominated groups - note: with 4.5.1 permissions can also be added at the user level). With regards the core module, the 4.4 release also has a large set of custom permissions so it's easy to set up who has access to view, post, have avatars etc.
Cathal