The bootfile is too small to support persistent snapshots

Good afternoon. It has been too long since I last posted. Today I found a solution to a problem I have seen several times and I wanted to share it.

I had a customer that was experiencing backup issues with a new load of Windows. When trying to backup the server in Windows Serer Backup the backup would always fail with the error “Windows Backup failed to create the shared protection point on the …”. An important point to note is the error would always occur during the VSS snapshot phase of the backup.

Below is the resulting Application event log with the key event highlighted.

At this point it is probably helpful to get a high level overview of how Windows Server Backup and VSS work. When Windows Server Backup starts a backup one of the first steps is to call VSS to take a snapshot. When the backup destination is local disk, the request is for both the backup destination and the backup source. This is so that Windows Server Backup can compare the blocks in both to perform an incremental backup. This means that a failure to snap the source or destination can cause the backup to fail.

I have seen this issue a handful of times and the consensus was the backup drive was causing the problem. While this can be the case, today I learned how to pinpoint which volume is actually causing this error with the event log. The key to determining this is the volume GUID ( Globally Unique Identifier) specified in the description of the event. This is the volume that cannot be snapped by VSS and is causing the backup to fail.

So how do you take the GUID and get the drive letter? This is the easiest part. Simply open an admin cmd window and run the command “mountvol”. At the end of the output all volumes with GUIDs and drive letters will be listed. In our case it was the D:\ drive that contained user data. We ran a test backup excluding the D:\ drive and it completed with no errors.

How do I fix the volume, so it will backup? Obviously we will not want to exclude a volume from the backup. There are two methods to repair this issue. First a chkdsk /f can be run to attempt and repair the volume. If that fails though, then you are likely looking at a bit of work to recreate the volume. Here is the process:

  1. Backup the data with robocopy or another file level backup utility. For robocopy an example command: robocopy <source> <destination> /MIR /XJ /W:5 /R:3 /LOG+:c:\robolog.txt
  2. Run diskpart and “clean” the disk. To do this run diskpart at an admin cmd, select the problem disk, then run the clean command.
  3. Recreate the volume
  4. Restore the data with robocopy or whatever file level backup utility used previously.

I hope you have found this post informative. If you have another way to solve this problem I would love to hear about it in the comments.

Advertisements

Windowsupdate.log is filled with GUIDs

Good afternoon. I found an interesting solution I wanted to share. I needed to collect the Windowsupdate.log file on a Windows 2016 server today. To do this I needed to run the PowerShell command Get-WindowsUpdateLog. The file is no longer continuously created as with previous Windows versions. This is all well and good, if the command worked 100% of the time. There have been some instances though where I ran that command and just get a file filled with GUIDs. See the example below.

1600/12/31 18:00:00.0000000 824 1056 Unknown( 10): GUID=638e22b1-a858-3f40-8a43-af2c2ff651a4 (No Format Information found).
1600/12/31 18:00:00.0000000 824 1056 Unknown( 11): GUID=bce7cceb-de62-3b09-7f4f-c69b1344a134 (No Format Information found).
1600/12/31 18:00:00.0000000 824 1056 Unknown( 11): GUID=638e22b1-a858-3f40-8a43-af2c2ff651a4 (No Format Information found).
1600/12/31 18:00:00.0000000 824 1056 Unknown( 50): GUID=6ffec797-f4d0-3bda-288a-dbf55dc91e0b (No Format Information found).
1600/12/31 18:00:00.0000000 824 1056 Unknown( 12): GUID=00497b4f-20f7-3ec8-96ab-8a593aa9824d (No Format Information found).

I have always wondered why this happened. I finally discovered the answer today. When I ran the PowerShell command I kept getting a popup about website security. I checked the box to not ask again and clicked OK. I then received a file full of nothing useful. I had a hunch that the command needed to grab information from the Internet to decode the GUIDs. Perhaps IE ESC (Internet Explorer Enhanced Security) was causing an issue with that process. I disabled IE ESC and re-ran Get-WindowsUpdateLog. Sure enough the file was created correctly.

So now you know. If you get a Windowsupdate.log file full of GUIDs there are two items to check. Verify the server has Internet connectivity and that IE ESC is turned off.

I hope you found this article informative. If you have anything to suggest or add to the content, please leave it in the comments below.

User profile corruption for Windows service accounts

Good morning. It has been a while since I posted, so I figured it was time for another article. I ran across an interesting issue this morning that I figured I would share. I had a customer that had recently experienced some file system corruption on the C: drive. Luckily chkdsk was able to correct the issue, but there was an issue that cropped up after running it. My customer was seeing an error in the Windows system log coming up frequently. The error was a 7005 with a source of Server Control Manager. The description was his concern though.
“The LoadUserProfile call failed with the following error:
The configuration registry database is corrupt.”

I did some research on this error and it is caused by a corrupt user profile. I figured it was probably a service user account as we had several services starting within seconds of each occurrence. Through a process of elimination I discovered that starting any service using the Network Service as the logon service caused the error.

So now I knew which account was causing the error, but how do you recreate the user profile for the Network Service user? I first checked the c:\users folder and the profile is not there. It is also not in the user profiles list in the system properties. I checked the registry as it has a list of all users with profile locations.
HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileList

Bingo!

The Network Service profile is located in C:\Windows\ServiceProfiles\NetworkService. I renamed the profile in the registry (S-1-5-20) to S-1-5-20.old and the NetworkService folder to NetworkService.old.

I then started a service that used the Network Service account, and success. The registry key was recreated, as was the folder, and we received no errors in the event log.

On a side note the above process will also work for the Local Service account. Just rename the appropriate registry key and folder.

I hope you found this article informative. If you have anything to add or would like to suggest an edit, please do so in the comments below.

Problems installing RDS CALs on a 2019 license server

Good morning.  I had an interesting issue this morning I figured I would share.  I had a customer that was having a tough time installing his RDS CALs.  At both activate.microsoft.com and in the install license wizard we were receiving the error “Invalid license code” and “The serial number is not valid.” respectively.  I assumed this was due to a bad license code, as I have seen that a couple of times in the past.  After emailing Microsoft, I was able to confirm the license code was good. 

The serial number is not valid.

After going back to my customer, I found out that the license server was running Windows 2019.  Normally this should not be a problem as you can install down-level CALs.  For instance, it is possible to install 2012 RDS CALs on a 2016 license server.  It appears that may no longer be the case with 2019 or this may be a bug.  Time will tell.  In the end we fixed the problem by installing licensing on the Windows 2016 server.  The CALs then installed without issue.

If this changes in the future I will update this blog post.  I hope you found this blog post useful.  If you have any corrections or anything to add, please do so in the comments below.

Windows Server Essentials Configuration Troubleshooter

Good morning,
It has been a little while since I last posted, so I figured it was time to share a little gem.  My team had a case that had us scratching our heads.  I had previously seen the error, but the solution that worked several times before did not work.  In case you are curious here is the error we were seeing:
web server settings error

(There is an error in your Web server settings)

we opted to engage Microsoft support.  The support engineer had a trick up his sleeve and was able to resolve the issue quickly.  After doing some preliminary checking, he ran the configuration troubleshooter.  The tool pointed at a path issue with the RemoteAppPool application pool in IIS.  After fixing the path the errors were resolved.

I tried out this tool and it works great at verifying settings are correct.  The tool can be used to test IIS settings, the certificate authority, Essentials services, and whether ports for Essentials are open.  I am definitely adding this to my tool belt as it will significantly speed up troubleshooting.  You can find the tool here.  Below is a screenshot.  I just ignored the error as it did not seem to impact functionality.

Windows ESS Config Troubleshooter

I hope you found this post useful.  What is your favorite troubleshooting tool for Essentials?  Put the answer in the comments section below.

Windows Server Essentials wizard failing at 16%

Good morning.  I wanted to document an issue I have seen several times.  The fix for this problem is pretty easy in PowerShell, but would take quite a bit of time using Server Manager.  The reason for the wizard is failing at 16% is due to the inability to connect to a domain controller in the domain.  This failure to connect is due to none of the roles being installed and therefore the server not being promoted to a domain controller.  This can all be discovered from the Essentials deployment logs in the C:\ProgramData\Microsoft\WindowsServer\Logs folder.

As I stated above the fix is pretty easy.  Run the three PowerShell commands below, changing domainname to the name you want for your domain and the P@ssW0rD! to a password of your choosing.  Keep in mind this password must meet complexity requirements with a length of at least 8 characters and 3 of 4 character types; capital letter, lowercase letter, number, special character.

NOTE: If you do not want the default computer name of WIN-<random string>, then you should change the computer name via the sysdm.cpl application or use netdom.

Install-WindowsFeature AD-Domain-Services,DNS,FileAndStorage-Services,File-Services,FS-FileServer,FS-BranchCache,FS-DFS-Namespace,Storage-Services,NPAS,RemoteAccess,DirectAccess-VPN,Remote-Desktop-Services,RDS-Gateway,Web-Server,Web-WebServer,Web-Common-Http,Web-Default-Doc,Web-Dir-Browsing,Web-Http-Errors,Web-Static-Content,Web-Http-Redirect,Web-Health,Web-Http-Logging,Web-Log-Libraries,Web-Request-Monitor,Web-Http-Tracing,Web-Performance,Web-Stat-Compression,Web-Security,Web-Filtering,Web-Basic-Auth,Web-Client-Auth,Web-IP-Security,Web-Windows-Auth,Web-App-Dev,Web-Net-Ext45,Web-ASP,Web-Asp-Net45,Web-ISAPI-Ext,Web-ISAPI-Filter,Web-Includes,Web-Mgmt-Tools,Web-Mgmt-Console,Web-Mgmt-Compat,Web-Metabase,Web-Scripting-Tools,ServerEssentialsRole,NET-Framework-45-Features,NET-Framework-45-Core,NET-Framework-45-ASPNET,NET-WCF-Services45,NET-WCF-HTTP-Activation45,NET-WCF-TCP-PortSharing45,BranchCache,GPMC,RSAT,RSAT-Role-Tools,RSAT-AD-Tools,RSAT-AD-PowerShell,RSAT-ADDS,RSAT-AD-AdminCenter,RSAT-ADDS-Tools,RSAT-ADCS,RSAT-ADCS-Mgmt,RSAT-DNS-Server,RSAT-NPAS,RSAT-RemoteAccess,RSAT-RemoteAccess-PowerShell,RPC-over-HTTP-Proxy,FS-SMB1,Windows-Defender-Features,Windows-Defender,Windows-Defender-Gui,Windows-Internal-Database,WAS,WAS-Process-Model,WAS-Config-APIs,Search-Service,Windows-Server-Backup,WoW64-Support

$Password = ConvertTo-SecureString “P@ssW0rD!” -AsPlainText -Force

Install-ADDSForest -DomainName “domainname.local” -SafeModeAdministratorPassword $Password -Force

After the above commands complete the server will automatically restart and the deployment wizard should complete without further errors.  If it is failed, then click Retry.  I have seen a few instances where a retry is necessary.

I hope you found this post helpful.  If you have anything to add, please do so in the comment section below.

Some settings in Windows 2016/10 giving an error

Good afternoon.  I ran into an interesting issue I figured I would share.  I had a customer that would receive the error below when clicking on some settings in Windows, for instance the change adapter options in the network section of settings.

user profile error

I did quite a bit of searching for a possible solution and I found quite a few forum posts and self-help guides, but none had the solution.  I broke out a tool I use from time to time to see I could figure out why Windows cannot access the device, path, or file; Process Monitor.  I ran a capture while duplicating the issue.  In pouring over the results I found a possible culprit in some registry paths in the HKCU registry hive.  It turns out that these setting options look at paths stored in the registry when clicked.  Unfortunately for my customer he was missing some paths and so Windows could not find the file.

To confirm this was the issue I created a copy of the administrator account and we logged on with that account.  Sure enough, no errors.

To fix the issue we simply deleted the user profile for the problem account.  This can be accessed by running sysdm.cpl, going to the Advanced tab and clicking user profiles Settings button.  After deleting the profile, we then logged the test account off and logged on with the administrator account and presto, no more errors.

I hope you found this article informative.  If you have anything to add or just want to comment, please do so below.