Ordinarily, you'd be at the right spot, but we've recently launched a brand new community website... For the community, by the community.
Yay... Take Me to the Community!
Welcome to the DNN Community Forums, your preferred source of online community support for all things related to DNN.In order to participate you must be a registered DNNizen
Hello,
I notice that scheduler jobs are running a job that is supposed to run every 5 min, every second or less!!! (DNN 6.1.3)
I first noticed this in writing my own custom scheduler job, which I had to code around, but then I looked at the base DNN jobs and noticed the same thing is happening. Every job that is out there, it running every second or partial second. This is obviously putting a huge load on the server and severely impacting performance in multiple ways (server load, quick increases in DB size, etc.)
This is a low volume site and not sure if it is technically in a web farm or not. It is just a simple shared hosting account, which, not being a hardware person, not sure if these are typically run in a web farm or not. But in the below, it always states that the server is the same.
This is a multi-portal site and each website is set to use the DefaultAppPool (as set using the hosting providers client tools, not IIS directly)
Any Ideas?
Thanks in Advance,
Chad
This Job is configured to run EVERY 5 MIN with a 1 MIN Retry.
Sorry, I'm not clear on your first question...which particular schedule history are you looking for? The Retry is definitely 1 MIN and not 1 SEC.
Hi Ken,
Glad to hear this is in discussion with the team! I have not enabled the "Run on Server" option. It is not available as I don't/can't have the Host>Web Farm setting enabled as I have community edition, and I don't know for sure if it is running on a web farm...I don't think so as it is just a low level shared hosting space I am used.
Catchup Enabled is set to False.
I changed the timing to 10 min with a 2 min retry and we'll see what happens, but I doubt it will have an impact because of how I had my custom scheduler module set up (which gets me to your next question).
Yes, I have a custom scheduler module that is set to run every 15 min with no retry on failure (I disabled it in hopes of improvement as I thought any thing that pointed to running more in this case, was not desirable). It would run up to 12 instances of the job simultaneously. And each instance would pull data from another site and log 5 rows in a database table....the job was running constantly and filling up my database quite rapidly, as you can imagine. I ended up by using my own stored procedure to check to see if it was time to run again. After a couple of iterations, it seems to be mostly in check with my latest logic. (At first it was checking time in the schedule history table to determine if it was time to run again or not, but that didn't work so now I check the database table that the job is updating itself to see if it is time to run. This mostly works, but I still see instances where the job runs in 2 near simultaneous instances). You can see it below. The funny thing is, given the logic of my code, there is no way it should've run multiple times, but it did..made me wonder of there is some sort of buffer involved when updating the schedule History table, because if there is, that was what allowed the job to keep running I guess.
As far as the log.resources file, I have many errors in it....not sure if there is anything in particular you are looking for, but here is one that found that supports the concept of the Purge File Cache running multiple instances simultaneously. There are 2 threads (31 and 32) that are both going after the same file at the same exact time and locking each other out:
2012-03-08 01:05:48,777 [DNPWEB6-TX2][Thread:31][ERROR] DotNetNuke.Services.ModuleCache.FileProvider - The process cannot access the file 'd:\HostingSpaces\InciteClient\DNN\wwwroot\Portals\Doggie\Cache\Pages\980_ACABA51644A302FBB325E30E7931BAF8.data.resources' because it is being used by another process. System.IO.IOException: The process cannot access the file 'd:\HostingSpaces\InciteClient\DNN\wwwroot\Portals\Doggie\Cache\Pages\980_ACABA51644A302FBB325E30E7931BAF8.data.resources' because it is being used by another process. at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath) at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath) at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share) at System.IO.File.WriteAllBytes(String path, Byte[] bytes) at DotNetNuke.Services.ModuleCache.FileProvider.SetModule(Int32 tabModuleId, String cacheKey, TimeSpan duration, Byte[] output) 2012-03-08 01:05:48,777 [DNPWEB6-TX2][Thread:32][ERROR] DotNetNuke.Services.ModuleCache.FileProvider - The process cannot access the file 'd:\HostingSpaces\InciteClient\DNN\wwwroot\Portals\Doggie\Cache\Pages\980_ACABA51644A302FBB325E30E7931BAF8.data.resources' because it is being used by another process. System.IO.IOException: The process cannot access the file 'd:\HostingSpaces\InciteClient\DNN\wwwroot\Portals\Doggie\Cache\Pages\980_ACABA51644A302FBB325E30E7931BAF8.data.resources' because it is being used by another process. at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath) at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath) at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share) at System.IO.File.WriteAllBytes(String path, Byte[] bytes) at DotNetNuke.Services.ModuleCache.FileProvider.SetModule(Int32 tabModuleId, String cacheKey, TimeSpan duration, Byte[] output)
Oh, and here are some interesting errors I think:
System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached. at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection) at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory) at System.Data.SqlClient.SqlConnection.Open() at Microsoft.ApplicationBlocks.Data.SqlHelper.ExecuteReader(String connectionString, CommandType commandType, String commandText, SqlParameter[] commandParameters) at Microsoft.ApplicationBlocks.Data.SqlHelper.ExecuteReader(String connectionString, String spName, Object[] parameterValues) at DotNetNuke.Services.Scheduling.DNNScheduling.SqlDataProvider.GetNextScheduledTask(String Server) at DotNetNuke.Services.Scheduling.DNNScheduling.SchedulingController.GetNextScheduledTask(String Server) at DotNetNuke.Services.Scheduling.DNNScheduling.DNNScheduler.GetNextScheduledTask(String server) at DotNetNuke.Services.Scheduling.SchedulingProvider.set_ScheduleLastPolled(DateTime value) at DotNetNuke.Common.Initialize.RunSchedule(HttpRequest request)
Note I am having this issue on various scheduled jobs, not just Purge Users Online...but also Purge Log Buffer, and my custom module, and maybe others...I suspect all, but haven't checked all as of yet to confirm. Also having this on another DNN instance which is v6.0.0
And one last thing, not sure if this is related, but I think it is, I was having issues with the Purge Log Buffer job running up to 5 or 7 instances simultaneously as well job and have since disabled it. It seems like it was causing issues with this in whole as well as it was causing my PayPalIPN posts to not get processed. See here.
Sorry for the long reply, but you said you wanted as much detail as possible :). I am also not adverse to having you look at things directly if you need. Let me know.
Thanks, Chad
ALTER PROCEDURE [dbo].[Incite_MQ_GetSpotPriceSchedLastRunTime] -- Add the parameters for the stored procedure here @ScheduleTypeFullName nvarchar(200) AS BEGIN -- SET NOCOUNT ON added to prevent extra result sets from -- interfering with SELECT statements. SET NOCOUNT ON; -- Insert statements for procedure here DECLARE @PHLastInsert as DateTime SET @PHLastInsert = (Select MAX(InsertDateTime) from Incite_MQ_PriceHistory) SELECT TOP (1) CAST(CASE WHEN CURRENT_TIMESTAMP >= ISNULL(DATEADD(mi,15,@PHLastInsert), CAST('1/1/2000' as Date)) THEN 1 ELSE 0 END AS bit) AS RunNow, @PHLastInsert as PHLastRun, --Following uses logic it obtains from Scheduler table, but maybe its being buffered??? CAST(CASE WHEN CURRENT_TIMESTAMP >= ISNULL(SH.NextStart, CAST('1/1/2000' as Date)) THEN 1 ELSE 0 END AS bit) AS RunNowOLD, SH.StartDate as LastRunTime, CURRENT_TIMESTAMP as CurrentTime, ISNULL(SH.NextStart, CAST('1/1/2000' as Date)) as NextRunTime, SH.NextStart as DBNextRunTime,* FROM dbo.ScheduleHistory as SH INNER JOIN dbo.Schedule AS S ON SH.ScheduleID = S.ScheduleID WHERE (S.TypeFullName = 'Incite.Modules.MetalQuick.ScheduledTasks, Incite.MetalQuick') ORDER BY SH.StartDate DESC