Well the final answer was to go into my main web (this was not a dotnetnuke web) in my IIS configuration and add the ASPNET account as an administrator. It seems this message was popping up in the event logs:

Failed to initialize the AppDomain:/LM/W3SVC/1/ROOT

Exception: System.Configuration.ConfigurationErrorsException
Message: Exception of type 'System.Configuration.ConfigurationErrorsException' was thrown.
StackTrace:    at System.Web.Configuration.ErrorRuntimeConfig.ErrorConfigRecord.System.Configuration.Internal.IInternalConfigRecord.GetLkgSection(String configKey)
   at System.Web.Configuration.RuntimeConfigLKG.GetSectionObject(String sectionName)
   at System.Web.Configuration.RuntimeConfig.GetSection(String sectionName, Type type, ResultsIndex index)
   at System.Web.Configuration.RuntimeConfig.get_HostingEnvironment()
   at System.Web.Hosting.HostingEnvironment.StartMonitoringForIdleTimeout()
   at System.Web.Hosting.HostingEnvironment.Initialize(ApplicationManager appManager, IApplicationHost appHost, IConfigMapPathFactory configMapPathFactory, HostingEnvironmentParameters hostingParameters)
   at System.Web.Hosting.HostingEnvironment.Initialize(ApplicationManager appManager, IApplicationHost appHost, IConfigMapPathFactory configMapPathFactory, HostingEnvironmentParameters hostingParameters)
   at System.Web.Hosting.ApplicationManager.CreateAppDomainWithHostingEnvironment(String appId, IApplicationHost appHost, HostingEnvironmentParameters hostingParameters)
   at System.Web.Hosting.ApplicationManager.CreateAppDomainWithHostingEnvironmentAndReportErrors(String appId, IApplicationHost appHost, HostingEnvironmentParameters hostingParameters)

However the DotNetNuke site (on another web) was definately triggering something. Anyway the good thing is it forced me to upgrade to 4.3 and while I already use it for new sites I was "afraid of problems" to upgrade my existing sites.