Thanks @ James Brown. Your guess is probably right. I am doing everything I possibly can to address this issue. I have now troubleshoot this issue to the following error:
HTTP Error 500.21 - Internal Server Error - Handler "ExtensionlessUrl-Integrated-4.0" has a bad module "ManagedPipelineHandler" in its module list
I have uninstalled and re-installed .Net and ASP.Net on this machine.
And regarding the port number, when you create a website with WebMatrix or the web installer, it creates its own bindings in C:\Users\%USERNAME%\Documents\IISExpress\config\applicationhost.config and for me this is the record that Web Installer inserted and I am not sure if there is any connection.
<site name="dnndev.me" id="4">
<application path="/" applicationPool="Clr4IntegratedAppPool">
<virtualDirectory path="/" physicalPath="C:\websites\dnndev.me" />
</application>
<bindings>
<binding protocol="http" bindingInformation="*:27265:localhost" />
</bindings>
</site>