Registry bloat and SBS

Good morning.  I wanted to cover a strange issue I ran into yesterday.

error

When going to the Computers tab within the Network tab we would receive a popup with the error “The server cannot query power management configuration.”  I did some checking on this and found that if the Windows 7 and Vista GPO (Group Policy Object) was missing that it could cause this issue.  I checked and that GPO was present.  It also had the default settings.

So what could possibly be causing this then?  Anytime there is an error in the SBS (Small Business Server) console the console.log or console2.log should be reviewed.  This log can be found in the C:\Program Files\Windows Small Business Server\Logs directory.  When I checked that log I found this exception:

[45636] 170118.101919.9681: ClientSetup: Handled exception: ErrorCode:0
BaseException: Microsoft.WindowsServerSolutions.ClientSetup.PowerUtilityException: QuerySleepTimeoutOnAC —> Microsoft.WindowsServerSolutions.Common.GroupPolicy.GPOException: GPOperation.OpenDSGPO —> System.Runtime.InteropServices.COMException: Insufficient system resources exist to complete the requested service. (Exception from HRESULT: 0x800705AA)

Now I have the exception code and message.  Insufficient system resources exist to complete the requested service. (Exception from HRESULT: 0x800705AA).  I have seen these types of issues before in SBS, Essentials and Foundation, so I figured we were running out of heap or nonpaged pool memory.  A telltale sign of this is when the SILSVC (Software Infrastructure License service) fails due to a resource issue.  I checked the log for SILSVC and all checks were passing.  I then did some more searching and found that this error is linked to the SOFTWARE registry hive exceeding the size limit of 2GB.  I checked C:\Windows\System32\config and found that the SOFTWARE registry hive was 2,050,657 KB, which is just over 2GB.

With the issue now identified, I figured a fix should be pretty straightforward.  Unfortunately this was not the case.  The registry hive cannot be compacted while it is in use.  Also Microsoft does not have a tool to trim down the size of the registry.  To make matters worse, the latest backup of the registry hives was 3 years old.  So in this case we decided to try a 3rd party registry cleaning utility to trim down the size of the registry and this is where I left the issue.  With any luck my customer will be able to trim down the size of the registry hive.

I hope you found this informative.  If you have any comments or suggestions, please leave them in the comments below.

Advertisements

Exchange mail flow: Identifying the issue

This will be the first in a short series of articles I will be writing on mail flow troubleshooting.  I wanted to start with a general article on how to narrow the scope of the mail flow issue.  Mail flow issues typically fall into one of three categories.

  1. External mail is not coming in for all or some email.
  2. Internal mail is not going out to the Internet for all or some email.
  3. Internal mail flow is not working.

So before we start troubleshooting we need to determine which category to focus on.  Occasionally you may see an issue that encompasses more than one of the categories above.  If that is the case then it is best to start on internal mail flow.

When I start looking at a mail flow issue I like to start with a few questions.

  • Are you having any issues sending or receiving emails to other users in your organization?
    If the answer is yes, then this narrows down the scope of the issue tremendously.  If the answer is no, then we know we need to start on internal mail flow troubleshooting.
  • Are you having a problem with receiving emails from users outside your organization?
    If the answer is yes, then you can focus on troubleshooting mail coming in from the Internet.  If the answer is no, then the issue is likely mail going out to Internet is having issues.
  • Are you having problems with users outside your organization receiving the emails your users are sending?
    If the answer is yes, then you can focus on mail going out to other mail servers on the Internet.

So now that we have an understanding of how to narrow the scope of the issue, the next parts of this series will focus on troubleshooting each of the different categories.

How to disable WSUS in SBS

Since I am on the kick of disabling applications in SBS (Small Business Server), I figured I would post an article on disabling WSUS (Windows Server Update Services).  As with disabling SharePoint, it will save resources and if WSUS is broken then clients are not updating.  This process can be easily reversed if it is decided to re-enable WSUS later.  Lastly this process will not break the SBS console or any of the wizards.

Here is the process for disabling WSUS.

  1. Disable the following services in the services console.  (services.msc)
    • Update Services
    • Windows Internal Database (MICROSOFT##SSEE)*
  2. Disable the application pool and site in IIS (Internet Information Services) manager
    • Launch IIS manager.  (inetmgr)
    • Expand the server name.
    • Click on Application Pools.
    • Right-click WsusPool and choose Stop.
    • Click on Sites.
    • Right-click on WSUS Administration, highlight Manage Web Site, and click Stop.
    • Close the services console and the IIS manager.
  3. Unlink the group policies for WSUS.
    • Launch the Group Policy Management console.  (gpmc.msc)
    • Expand Forest
    • Expand Domains
    • Expand your domain name
    • Right-click on the following policies and select Delete.  You might get a popup indicating that you clicked on link.  Just click OK on that popup.
      • Update Services Client Computers Policy
      • Update Services Common Settings Policy
      • Update Services Server Computers Policy
    • Close the Group Policy Management console.

 

 

 

*For SBS 2008, do not disable this service unless you are also disabling SharePoint.

How to disable SharePoint (Companyweb) in SBS

This is a question I get from time to time.  How can I turn off Companyweb or SharePoint in SBS (Small Business Server)?

So why might you want to disable SharePoint (Companyweb)?

  1. SharePoint is not functioning properly and is clogging the application log with errors and warnings
  2. It is not being used, so why not free up the resources.  At least 500MB of RAM will be freed up.

Either way, here is my tested method for disabling SharePoint in less than 5 minutes.  This method allows SharePoint to be re-enabled again and does not cause problems with the wizards or the SBS console.  I prefer not to uninstall SharePoint as it takes a good amount of time to reinstall it and setup Companyweb.

  1. Disable the SharePoint services in the services console (services.msc)
    • SharePoint 2010 Administration
    • SharePoint 2010 Timer
    • SharePoint 2010 Tracing
    • SharePoint 2010 VSS Writer
    • SharePoint Foundation Search V4
    • SQL Server (SHAREPOINT)
  2. Stop the application pool and sites in IIS (Internet Information Services)
    • Launch IIS manager. (inetmgr)
    • Expand the server name and click Application Pools.
    • Stop the following application pools
      • SBS Sharepoint AppPool
      • SharePoint Central Administration v4
      • SharePoint Web Services Root
    • Click on Sites.
    • On each the following sites, right-click, highlight Manage Web Site, and choose Stop
      • Companyweb
      • SBS SharePoint
      • SharePoint Central Administration v4
      • SharePoint Web Services
    • Close the services console and IIS manager

Why is the SBS console crashing?

Since I have seen this issue three times this week, I figured it was time to write a post about how to troubleshoot SBS (Small Business Server)  console crashes.  This is an issue I usually see several times a month.  Most people want to repair the SBS console as the first step.  However I have never seen a SBS console crash that was caused by a problem with the console itself.  Console crashes are always caused by an application and/or service the console is trying to access.  The most common of these is WSUS (Windows Server Update Services).

So how do you troubleshoot a SBS console crash?
The answer is always in the console.log, or console2.log, file.  Here is the process I follow to find the cause for the crash.  I am using an example that I had yesterday, but the process is similar for all cases.

  1. Close the SBS console if it is open.
  2. Navigate to the C:\Program Files\Windows Small Business Server\Logs directory.
  3. Rename the console.log and console2.log if it exists.
  4. Launch the SBS console and get the console to crash as soon as possible.  When it crashes, do not relaunch it.
  5. Open the console.log with a text editor.
  6. Press Ctrl+End to navigate to the bottom of the log.

We are looking for the outer and inner exceptions.  The outer exception always comes first  and is immediately followed by the inner exception.  The inner exception will tell you the problem.  In the example below the problem is corruption in the WSUS database.  In this case the easiest fix is to reinstall WSUS.

 

Outer exception:

[15668] 160225.173854.0543: Exception:
—————————————
An exception of type ‘Type: System.Reflection.TargetInvocationException, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089’ has occurred.
Timestamp: 02/25/2016 17:38:54
Message: Exception has been thrown by the target of an invocation.
Stack:    at Microsoft.WindowsServerSolutions.Console.Pages.Computers.Computers.QueryComputerJob.OnCompleted(Object sender, RunWorkerCompletedEventArgs e)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncCtx)
at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
at System.Threading.ThreadPoolWorkQueue.Dispatch()
at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()

 

Inner exception:

—————————————
An exception of type ‘Type: System.Data.SqlClient.SqlException, System.Data, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089’ has occurred.
Timestamp: 02/25/2016 17:38:54
Message: SQL Server detected a logical consistency-based I/O error: incorrect pageid (expected 1:1793122; actual 7015:33685504). It occurred during a read of page (1:1793122) in database ID 5 at offset 0x0000036b8c4000 in file ‘C:\WSUS\SUSDB\\UpdateServicesDbFiles\SUSDB.mdf’.  Additional messages in the SQL Server error log or system event log may provide more detail. This is a severe error condition that threatens database integrity and must be corrected immediately. Complete a full database consistency check (DBCC CHECKDB). This error can be caused by many factors; for more information, see SQL Server Books Online.
Stack:    at Microsoft.UpdateServices.Internal.BaseApi.SoapExceptionProcessor.DeserializeAndThrow(SoapException soapException)
at Microsoft.UpdateServices.Internal.DatabaseAccess.AdminDataAccessProxy.ExecuteSPGetUpdateInstallationInfoForComputer(String computerId, String updateScopeXml, String preferredCulture, ExtendedPublicationState publicationState)
at Microsoft.UpdateServices.Internal.BaseApi.ComputerTarget.GetUpdateInstallationInfoPerUpdate(UpdateScope updatesToInclude)
at Microsoft.WindowsServerSolutions.SystemHealth.UpdateServices.DataProvider.GetPerComputerStatus(String FullDNSComputerName, DateTime starting)
at Microsoft.WindowsServerSolutions.Console.Pages.Computers.Computers.QueryCompUpdateJob.SetUpdateCompliance()
at System.ComponentModel.BackgroundWorker.WorkerThreadStart(Object argument)

Reinstall WSUS on SBS

UPDATE: WSUS 3.0 SP2 will not reinstall.  There is an expired certificate in the installer package for SUSDB that will cause the setup to fail.  Unless Microsoft decides to recompile the installer package this will not be fixed.

I figured I would create a post on WSUS (Windows Server Update Services) and SBS (Small Business Server).  More specifically I want to talk about the most common issue I see with my customers. The WSUS database grows to a very large size causing a number of symptoms.

  1. The WSUS console disconnects.
  2. The SBS console shows Update status as not available.
  3. The updates tab in the SBS console shows “An error occured while retrieving updates information” for all four update categories.
  4. Clients are not able to connect to WSUS and/or are not updating.

According to the research I have done Microsoft does not limit resources or the size of the WSUS database.  However in practice I have seen that when the SUSDB exceeds 10GB issues start to crop up.  I have found the easiest and fastest fix is to remove and reinstall WSUS.  Below are the steps for SBS 2011*:

  1. Remove WSUS.
    • Go to Control Panel then Programs and Features.
    • Highlight Windows Server Update Service 3.0 SP2 and choose Uninstall.
    • Check all three boxes to remove the database, logs and downloaded updates.
    • Complete the uninstall wizard.
    • If the uninstall fails, see this article.
  2. Reinstall WSUS.
    • Open an administrative command prompt
    • Change directories to C:\Program Files\Windows Small Business Server\Bin\CMPNENTS\WSUS_SP2
    • Run the following command**: WSUS30-KB972455-x64.exe /q DEFAULT_WEBSITE=0 CREATE_DATABASE=1 CONTENT_LOCAL=1 CONTENT_DIR=C:\WSUS WYUKON_DATA_DIR=C:\WSUS
    • The install will take anywhere from 2-10 minutes depending on hardware.  To verify the install is complete look for an event ID 1042 in the application event log.  If the install crashes, just rerun the above command again.
  3. Perform the SBS customizations.
    • Click Start, click Administrative Tools, and then click Microsoft Windows Server Update Services
    • In Update Services, expand the name of the server, and then click Options.
    • Click Products and Classifications, and then on the Products tab, verify that all check boxes are selected.
    • In the Products and Classifications dialog box, click the Classifications tab, and then check the following boxes:
      • Critical Updates
      • Definition Updates
      • Security Updates
      • Service Packs
      • Update Rollups
    • In Options, click Update Files and Languages, click the Update Languages tab, choose the option Download updates only in these languages, and then verify that English is selected.
    • In Options, click Synchronization Schedule, click Synchronize automatically, and then, in First synchronization, change the time to 10:00:00 P.M.  Click OK.
    • In Update Services, expand Computers, click All Computers, and then in Actions, click Add Computer Group.  Add the following computer groups:
      • Update Service Excluded Computers
      • Update Services Client Computers
      • Update Services Server Computers
  4. Update WSUS so that it will work with Windows 8 and higher clients.
    • Download the Microsoft update KB2734608.
    • Install the update.
    • Reboot the server.  The server must be rebooted before the first synchronization with Microsoft update servers.  If it is not, then WSUS and clients may require additional steps to repair.

* For the SBS 2008 the steps are the same, but WSUS 3.0 SP2 will need to be downloaded from the Microsoft site.  You can find it here.  Make sure and download the x64 version.

** Now is a great time to move the WSUS database and content repository to another drive.  It will be more difficult to move the WSUS DB later.  To move the location, change the C:\WSUS to another drive.  For example D:\WSUS.  Make sure and change both instances.