DNN Version 7.2.1

I keep getting the error:

"Unable to create Lucene writer (lock file is in use). Please recycle AppPool in IIS to release lock."

(Full message at end)

This seems to be related to the Search Crawler starting. The message is logged six times in about a second (once for each Portal? - I have 6, that can't be coincidence, although the Event Message is not associated with a  particular portal.)

Once this happens the portal response time of all my portals becomes dire (>30seconds to get a page) until I manually recycle the AppPool. I suspect my sites are also not properly indexed.

All is then well (although the message continues to be logged) for several hours (the scheduler is set to trigger the Search:Crawler once per hour) but then for some unknown reason, the problem reoccurs and I start getting reports of non-responsiveness from the website.

The only reports of this issue I can find here talk about either web farms (I have a single Azure VM) or slow machines. I don't believe my Azure VM is slow and most of the time is trickling along at sub 10% CPU and memory usage of 70%..

I am going to try turning of the trigger for search and running it manually, followed by an AppPool recycle - but this isn't really a satisfactory solution.

Any help would be gratefully received as my sites are due to go live next week.

Thanks in Advance Nick

------------------------

DotNetNuke.Services.Exceptions.SearchException: Unable to create Lucene writer (lock file is in use). Please recycle AppPool in IIS to release lock. ---> System.IO.IOException: The process cannot access the file 'C:\inetpub\wwwroot\DNN_EGB\App_Data\Search\write.lock' because it is being used by another process.

   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)

   at System.IO.File.InternalDelete(String path, Boolean checkHost)

   at DotNetNuke.Services.Search.Internals.LuceneControllerImpl.get_Writer()

   --- End of inner exception stack trace ---

   at DotNetNuke.Services.Search.Internals.LuceneControllerImpl.get_Writer()

   at DotNetNuke.Services.Search.Internals.LuceneControllerImpl.Delete(Query query)

   at DotNetNuke.Services.Search.Internals.InternalSearchControllerImpl.DeleteSearchDocumentsByModule(Int32 portalId, Int32 moduleId, Int32 moduleDefId)

   at DotNetNuke.Services.Search.SearchDataStore.StoreSearchItems(SearchItemInfoCollection searchItems)

   at DotNetNuke.Services.Search.SearchEngine.IndexContent(DateTime startDate)

   at DotNetNuke.Services.Search.SearchEngineScheduler.DoWork()

Thank you for the quick reply Sebastian. I am using the DefaultAppPool and the user "DefaultAppPool" has Modify/Read & Execute/List Folder/Read/Write permissions on the DNN folder and sub-folders. I specifically checked ~/App_Data/Search and permissions were as expected. The job also runs if I run it manually, which tends to imply it is not file permissions per se (or AV) - unless when I do that there is a different security context, which I don't think is the case. My backup jobs run at 7am - the problems last night started around midnight with no overlap with backup. Hard to check AV, but there is no real reason to believe it is interfering, but I will see if I can exclude that file or folder from AV.

I don't know if it is related, but the "Purge Schedule History" job also appears to be failing and has not succeeded in 10days. The Message in the Schedule History is:

Purge Schedule History Schedule history purge failed.System.Data.SqlClient.SqlException (0x80131904): Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding. ---> System.ComponentModel.Win32Exception (0x80004005): The wait operation timed out at System.Data.SqlClient.SqlConnection. (SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction) at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose) at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady) at System.Data.SqlClient.SqlCommand.RunExecuteNonQueryTds(String methodName, Boolean async, Int32 timeout, Boolean asyncWrite) at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(TaskCompletionSource`1 completion, String methodName, Boolean sendToPipe, Int32 timeout, Boolean asyncWrite) at System.Data.SqlClient.SqlCommand.ExecuteNonQuery() at PetaPoco.Database.Execute(String sql, Object[] args) at DotNetNuke.Data.PetaPoco.PetaPocoHelper.ExecuteNonQuery(String connectionString, CommandType type, String sql, Object[] args) at DotNetNuke.Services.Scheduling.DNNScheduler.PurgeScheduleHistory() at DotNetNuke.Services.Scheduling.PurgeScheduleHistory.DoWork() ClientConnectionId:2d81a8db-8fa1-42bd-9385-46bd577d4690

I will do a manual purge.

I haven't had any errors in my event log since I disabled the Search Crawler job, but running it manually is not really a long term option.

I have confirmed that when I do run it manually, the index appears to be updating as I am then able to find new text entered on the system.

Where do I look next? Is 7.3 likely to help - I have been reluctant to upgrade this close to going live.

I have truncated the SchedulerHistory, which in itself was probably a side effect of the errors and the purge of the ScheduleHistory now works again.

The lock errors continue to occur, but only if the jog is run on a schedule - if I run it manually ("Run Now") then it works and no errors are logged.  I have tried on various intervals and I have also tried using a 3rd party scheduler (DNNSharp Sharp Scheduler).  This made no difference in that I still get the error, but now only once per cycle.

My instinct is that the problem started when I added additional sites to the installation - is there some unexpected interaction/issue when an installation is supporting multiple sites?  The "micro" sites run as IIS Applications in the same domain as the main site: The website is Endurancegb and has a non DNN .NET site running. endurancegb.org/dnn_EGB is the main DNN site as an IIS Application off EnduranceGB Then I have five other "micro" DNN sites all as IIS Applications off EnduranceGB: endurancegb.org/reddragaon endruancegb.org/boyton etc

I really need to get to the bottom of this. Is there anything I can do to try to trace the issue?