I also just did a DNN 4.9.4 to 5.1.2 upgrade. I got all successes on my log. But when I login as Host I get this same error.

DotNetNuke.Services.Exceptions.ModuleLoadException: An entry with the same key already exists. ---> System.ArgumentException: An entry with the same key already exists. at System.ThrowHelper.ThrowArgumentException(ExceptionResource resource) at System.Collections.Generic.SortedList`2.Add(TKey key, TValue value) at DotNetNuke.Entities.Modules.DesktopModuleController.GetPortalDesktopModules(Int32 portalID) at DotNetNuke.UI.ControlPanels.IconBar.BindData() at DotNetNuke.UI.ControlPanels.IconBar.Page_Load(Object sender, EventArgs e) --- End of inner exception stack trace ---

I also had two versions of Google Analytics, but they had difference FriendlyNames (one had a space and the other didn't). Anyway, I totally deleted the module from the home page and went in and tried to uninstall, but it would recycle the page and both versions of the Analytics were there. So I went into the ModuleDefinitions table and deleted the rows there. Still cannot access any modules with their edit icon. I get

A critical error has occurred.
Input string was not in a correct format.

I checked and don't have any duplicates in my version table and there are no duplicates in the ModuleDef table.

Thanks for any help you can provide. Other than this the site seems to function well.

This may be a better representation of the error.  I just finished the upgrated to 5.1.2 and this is the event log when I click on module definitions:

ModuleId: 322
ModuleDefId: 64
FriendlyName: Module Definitions
ModuleControlSource: Admin/ModuleDefinitions/ModuleDefinitions.ascx
AssemblyVersion: 5.1.2
PortalID: 0
PortalName: North Alabama Football Officials Association
UserID: 1
UserName: host
ActiveTabID: 18
ActiveTabName: Module Definitions
RawURL: /tabid/18/portalid/0/Default.aspx
AbsoluteURL: /Default.aspx
AbsoluteURLReferrer: http://nafoa.net/tabid/16/portalid/0/Default.aspx
UserAgent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 GTB5 (.NET CLR 3.5.30729)
DefaultDataProvider: DotNetNuke.Data.SqlDataProvider, DotNetNuke.SqlDataProvider
ExceptionGUID: 53fe7e61-d7c8-485a-8a64-714e2dd42fa7
InnerException: e:\inetpub\zhost\mark\nafoa.net\www\Admin\ModuleDefinitions\ModuleDefinitions.ascx.vb(61): error BC30311: Value of type 'String' cannot be converted to 'System.Version'.
FileName:
FileLineNumber: 0
FileColumnNumber: 0
Method: System.Web.Compilation.BuildManager.CompileWebFile
StackTrace:
Message: DotNetNuke.Services.Exceptions.ModuleLoadException: e:\inetpub\zhost\mark\nafoa.net\www\Admin\ModuleDefinitions\ModuleDefinitions.ascx.vb(61): error BC30311: Value of type 'String' cannot be converted to 'System.Version'. ---> System.Web.HttpCompileException: e:\inetpub\zhost\mark\nafoa.net\www\Admin\ModuleDefinitions\ModuleDefinitions.ascx.vb(61): error BC30311: Value of type 'String' cannot be converted to 'System.Version'. at System.Web.Compilation.BuildManager.CompileWebFile(VirtualPath virtualPath) at System.Web.Compilation.BuildManager.GetVPathBuildResultInternal(VirtualPath virtualPath, Boolean noBuild, Boolean allowCrossApp, Boolean allowBuildInPrecompile) at System.Web.Compilation.BuildManager.GetVPathBuildResultWithNoAssert(HttpContext context, VirtualPath virtualPath, Boolean noBuild, Boolean allowCrossApp, Boolean allowBuildInPrecompile) at System.Web.UI.TemplateControl.LoadControl(VirtualPath virtualPath) at System.Web.UI.TemplateControl.LoadControl(String virtualPath) at DotNetNuke.UI.ControlUtilities.LoadControl[T](TemplateControl containerControl, String ControlSrc) at DotNetNuke.UI.Modules.ModuleHost.LoadModuleControl() --- End of inner exception stack trace ---
Source:
Server Name: C6004-104498

I did the same upgrade and found that not in ModuleDefinitions but in Desktop Modules there were two Google Analytics one which I presume was added in the upgrade because the path is Admin/GoogleAnalytics I renamed the FriendlyName to "DNN Google Analytics" and that fixed it for me.