Content Troubleshooting
This deep dive covers how to troubleshoot content location issues on the client and server-side in Configuration Manager.
Video: Content Troubleshooting Deep Dive (Full Video)
Watch the Video Guide for Content Troubleshooting
Video: Content Troubleshooting Video Timeline
- Review Client Log Files CAS.log CIAgent.log CIDownloader.log CIStateStore.log ClientLocation.log CMBITSManager.log ContentTransferManager.log DataTransferService.log DCMAgent.log LocationServices.log StateMessage.log UpdatesDeployment.log : (4:50)
- Review MP_Location.log on the management point and the stored procedure MP_GetContentDPInfo: (9:20)
Scenario 1: Succesful Content Lookup Workflow
Pre-Conditions
- 1.Healthy ConfigMgr client.
- 2.The client is in a boundary.
- 3.There is a Distribution Point associated with the boundary group.
- 4.The content is distributed successfully to the DP.
- 5.The software / update / package is deployed to the device.
- 6.Enable verbose logging on the site system with the MP role and the client
We enabled Verbose logging on the cient and management point
Successful Content Lookup Flow
There are many moving parts when a content location request is sent to the management point and processes by a ConfigMgr client. Here are some key snippets of how this happens.
Log File
Log Text
CCIStateStore.log
CCIStateStore::GetCIState - Client is requesting CI ModelName ScopeId_CA3F999B-7A81-43BF-912C-56E267C1E565/RequiredApplication_4894a05f-49c0-4836-a22f-50526dba27c4, Revision 1 for User: S-1-5-21-1880864260-2612682489-34998949-6112
CIDownloader.log
CDownloadPayloadInfo::AddCI - CI with ModelName ScopeId_CA3F999B-7A81-43BF-912C-56E267C1E565/DeploymentType_677dac51-d2a2-4bbd-aeb0-4b845aa288cd, Version 1 is already available. Requested Model:VersionSpecific will be downloaded.
CAS.log
===== CacheManager: Content for Content_8ba1768d-abc6-4e42-919d-1e421fe610f0.1 was NOT found cache. =====
CAS.log
ContentLocationRequest : <ContentLocationRequest SchemaVersion="1.00" BGRVersion="1" ExcludeFileList=""><Package ID="UID:9c93cbee-aabc-4daf-996a-4580daa09c87" Version="1" DeploymentFlags="9223372036855300962"/><AssignedSite SiteCode="DM1"/><ClientLocationInfo LocationType="SMSUpdate" DistributeOnDemand="0" UseAzure="0" AllowWUMU="0" UseInternetDP="0" AllowHTTP="1" AllowSMB="1" AllowMulticast="1" AllowSuperPeer="1" DPTokenAuth="1"><ADSite Name="Default-First-Site-Name"/><Forest Name="CONTOSO.LOCAL"/><Domain Name="CONTOSO.LOCAL"/><IPAddresses><IPAddress SubnetAddress="192.168.10.0" Address="192.168.10.35"/></IPAddresses><Adapters><Adapter Name="Ethernet" IfType="6" PhysicalAddressExists="1" DnsSuffix="" Description="Microsoft Hyper-V Network Adapter"/></Adapters><BoundaryGroups BoundaryGroupListRetrieveTime="2021-04-28T15:00:21.637" IsOnVPN="0"><BoundaryGroup GroupID="16777218" GroupGUID="ad36e248-a857-45e6-ab82-c179bb680b18" GroupFlag="0"/></BoundaryGroups></ClientLocationInfo></ContentLocationRequest>
UpdatesHandler.log
Starting download on action (INSTALL) for Update (9c93cbee-aabc-4daf-996a-4580daa09c87)
UpdatesHandler.log
Content Text = <Content ContentId="9c93cbee-aabc-4daf-996a-4580daa09c87" Version="1"><FileContent Name="debcf2da-e7da-4c50-b718-fc95ffe40c9d_1.cab" Hash="78A05BCA9CCA039C62C1B2D494281E72ED864BD9" HashAlgorithm="SHA1" Size="79417630"/></Content>
ContentTransferManager
Submitted CTM job {C17823B5-A8A9-48F8-8375-64CFC55869CB} to download Content 9c93cbee-aabc-4daf-996a-4580daa09c87.1 under context System - ContentTransferManager
CTM job {C17823B5-A8A9-48F8-8375-64CFC55869CB} (corresponding DTS job {C54E37D3-BFDE-4629-AC82-32E6CE733750}) started download from 'http://DEMO1.CONTOSO.LOCAL/SMS_DP_SMSPKG$/9c93cbee-aabc-4daf-996a-4580daa09c87' for full content download.
DataTransferService.log
DTSJob {C54E37D3-BFDE-4629-AC82-32E6CE733750} created to download from 'http://DEMO1.CONTOSO.LOCAL:80/SMS_DP_SMSPKG$/9c93cbee-aabc-4daf-996a-4580daa09c87' to 'C:\Windows\ccmcache\1'.
UpdateURLWithTransportSettings(): NEW URL - http://demo1.contoso.local:80/SMS_DP_SMSPKG$/9c93cbee-aabc-4daf-996a-4580daa09c87/sccm?/debcf2da-e7da-4c50-b718-fc95ffe40c9d_1.cab
DataTransferService.log
Total content size from the downloaded manifest is 79417630
DataTransferService.log
Creating root directory...
DataTransferService.log
Directory 'C:\Windows\ccmcache\1' already exists.
DataTransferService.log
Starting BITS job '{97C9975B-820F-467A-9A0D-366C70F28EEC}' for DTS job '{C54E37D3-BFDE-4629-AC82-32E6CE733750}' under user 'S-1-5-18'.
DataTransferService.log
BITS compatible path: http://DEMO1.CONTOSO.LOCAL:80/SMS_DP_SMSPKG$/9c93cbee-aabc-4daf-996a-4580daa09c87/sccm?/debcf2da-e7da-4c50-b718-fc95ffe40c9d_1.cab
DataTransferService.log
BITSHelper, remote name = http://DEMO1.CONTOSO.LOCAL:80/SMS_DP_SMSPKG$/9c93cbee-aabc-4daf-996a-4580daa09c87/sccm?/debcf2da-e7da-4c50-b718-fc95ffe40c9d_1.cab, local name = C:\Windows\ccmcache\1\debcf2da-e7da-4c50-b718-fc95ffe40c9d_1.cab
DataTransferService.log
Job: {C54E37D3-BFDE-4629-AC82-32E6CE733750}, Total Files: 1, Transferred Files: 0, Total Bytes: 79417630, Transferred Bytes: 262144
CAS.log
CContentAccessService::NotifyDownloadProgressEx - downloaded size 52480, total size 77556
CAS.log
The hash we are verifying is SDMPackage:<Content ContentId="9c93cbee-aabc-4daf-996a-4580daa09c87" Version="1"><FileContent Name="debcf2da-e7da-4c50-b718-fc95ffe40c9d_1.cab" Hash="78A05BCA9CCA039C62C1B2D494281E72ED864BD9" HashAlgorithm="SHA1" Size="79417630"/></Content>
CAS.log
Hash verification succeeded for content 9c93cbee-aabc-4daf-996a-4580daa09c87.1 downloaded under context System
DataTransferService.log
Job: {C54E37D3-BFDE-4629-AC82-32E6CE733750}, Total Files: 1, Transferred Files: 1, Total Bytes: 79417630, Transferred Bytes: 79417630
Tip: You can open all content logs at once in CMTrace.exe by adding the following files: "CAS.log" "CIAgent.log" "CIDownloader.log" "CIStateStore.log" "ClientLocation.log" "CMBITSManager.log" "ContentTransferManager.log" "DataTransferService.log" "DCMAgent.log" "LocationServices.log" "StateMessage.log" "UpdatesDeployment.log"
Merge all content logs in CMTrace.exe
Snippets of Location Request
Example of the management point MP_Location.log calling the stored procedure MP_GetContentDPInfo
1
<ServerNameList>
2
<ServerName LocalityEx="256" Fallback="0">DEMO1.CONTOSO.LOCAL</ServerName>
3
</ServerNameList>,SMSUpdate,00000000,CONTOSO.LOCAL,CONTOSO.LOCAL,<ClientLocationInfo LocationType="SMSUpdate" DistributeOnDemand="0" UseAzure="0" AllowWUMU="0" UseInternetDP="0" AllowHTTP="1" AllowSMB="1" AllowMulticast="1" AllowSuperPeer="1" DPTokenAuth="1">
4
<ADSite Name="DEFAULT-FIRST-SITE-NAME"/>
5
<Forest Name="CONTOSO.LOCAL"/>
6
<Domain Name="CONTOSO.LOCAL"/>
7
<IPAddresses>
8
<IPAddress SubnetAddress="192.168.10.0" Address="192.168.10.35"/>
9
</IPAddresses>
10
<Adapters>
11
<Adapter Name="Ethernet" IfType="6" PhysicalAddressExists="1" DnsSuffix="" Description="Microsoft Hyper-V Network Adapter"/>
12
</Adapters>
13
<BoundaryGroups BoundaryGroupListRetrieveTime="2021-04-28T15:00:21.637" IsOnVPN="0">
14
<BoundaryGroup GroupID="16777218" GroupGUID="ad36e248-a857-45e6-ab82-c179bb680b18" GroupFlag="0"/>
15
</BoundaryGroups>
16
</ClientLocationInfo>,<BoundaryGroups BoundaryGroupListRetrieveTime="2021-04-28T15:00:21.637" IsOnVPN="0">
17
<BoundaryGroup GroupID="16777218" GroupGUID="ad36e248-a857-45e6-ab82-c179bb680b18" GroupFlag="0"/>
18
</BoundaryGroups>)
Copied!
Tip: The Fallback="0" or Fallback="1" controls if fallback is allowed. To learn more about fallback see this Microsoft Doc.
Example of a content location request from the client sent to the management point in CAS.log
1
<ContentLocationRequest SchemaVersion="1.00" BGRVersion="1" ExcludeFileList="">
2
<Package ID="UID:9c93cbee-aabc-4daf-996a-4580daa09c87" Version="1" DeploymentFlags="9223372036855300962"/>
3
<AssignedSite SiteCode="DM1"/>
4
<ClientLocationInfo LocationType="SMSUpdate" DistributeOnDemand="0" UseAzure="0" AllowWUMU="0" UseInternetDP="0" AllowHTTP="1" AllowSMB="1" AllowMulticast="1" AllowSuperPeer="1" DPTokenAuth="1">
5
<ADSite Name="Default-First-Site-Name"/>
6
<Forest Name="CONTOSO.LOCAL"/>
7
<Domain Name="CONTOSO.LOCAL"/>
8
<IPAddresses>
9
<IPAddress SubnetAddress="192.168.10.0" Address="192.168.10.35"/>
10
</IPAddresses>
11
<Adapters>
12
<Adapter Name="Ethernet" IfType="6" PhysicalAddressExists="1" DnsSuffix="" Description="Microsoft Hyper-V Network Adapter"/>
13
</Adapters>
14
<BoundaryGroups BoundaryGroupListRetrieveTime="2021-04-28T15:00:21.637" IsOnVPN="0">
15
<BoundaryGroup GroupID="16777218" GroupGUID="ad36e248-a857-45e6-ab82-c179bb680b18" GroupFlag="0"/>
16
</BoundaryGroups>
17
</ClientLocationInfo>
18
</ContentLocationRequest>
Copied!
Example of Content location reply from the management point to the client in the client's CAS.log
1
<ContentLocationReply SchemaVersion="1.00" BGRVersion="1">
2
<BoundaryGroups BoundaryGroupListRetrieveTime="2021-04-27T14:21:06.263" IsOnVPN="0">
3
<BoundaryGroup GroupID="16777218" GroupGUID="ad36e248-a857-45e6-ab82-c179bb680b18" GroupFlag="0"/>
4
</BoundaryGroups>
5
<ContentInfo PackageFlags="0" DeploymentFlags="9223372036855300962">
6
<ContentHashValues/>
7
</ContentInfo>
8
<Sites>
9
<Site>
10
<MPSite SiteCode="DM1" MasterSiteCode="DM1" SiteLocality="LOCAL" IISPreferedPort="80" IISSSLPreferedPort="443"/>
11
<LocationRecords>
12
<LocationRecord LocalityEx="BOUNDARYGROUP" DPFallbackIn="PT0M">
13
<URL Name="http://DEMO1.CONTOSO.LOCAL/SMS_DP_SMSPKG$/9c93cbee-aabc-4daf-996a-4580daa09c87" Signature="http://DEMO1.CONTOSO.LOCAL/SMS_DP_SMSSIG$/9c93cbee-aabc-4daf-996a-4580daa09c87.1.tar" Capability="0"/>
14
<ADSite Name="Default-First-Site-Name"/>
15
<IPSubnets>
16
<IPSubnet Address="192.168.10.0"/>
17
<IPSubnet Address=""/>
18
</IPSubnets>
19
<Metric Value=""/>
20
<Version>9040</Version>
21
<Capabilities SchemaVersion="1.0">
22
<Property Name="SSLState" Value="0"/>
23
</Capabilities>
24
<ServerRemoteName>DEMO1.CONTOSO.LOCAL</ServerRemoteName>
25
<DPType>SERVER</DPType>
26
<Windows Trust="1"/>
27
</LocationRecord>
28
</LocationRecords>
29
</Site>
30
</Sites>
31
<RelatedContentIDs/>
32
</ContentLocationReply>
Copied!
Here's a snippet of the download options for a software update download request in UpdatesDeploymentAgent.log
1
<DownloadOptions>
2
<Priority>Foreground</Priority>
3
<DPLocality>10</DPLocality>
4
<Timeout>
5
<Location>604800</Location>
6
<Download>86400</Download>
7
<PerDPInactivity>0</PerDPInactivity>
8
<TotalInactivity>0</TotalInactivity>
9
<UseBranchCache>true</UseBranchCache>
10
<PersistOnWriteFilterDevices>false</PersistOnWriteFilterDevices>
11
<OverrideServiceWindows>false</OverrideServiceWindows>
12
</Timeout>
13
</DownloadOptions>
Copied!
A point of interest is DPLocality flag determines how the client obtains distribution points, according to distribution point locality. Possible values are:
Value
UInt32
DP_DOWNLOAD_FROM_LOCAL
4
DP_DOWNLOAD_FROM_REMOTE
6
DP_NO_FALLBACK_UNPROTECTED
17
DP_ALLOW_WUMU
18
DP_ALLOW_METERED_NETWORK
19
SMS_CIAssignmentBaseClass Class - Configuration Manager
docsmsft
Scenario 2: Failed Content Lookup Due to Boundary Misconfiguration
Pre-Conditions
- 1.Healthy ConfigMgr client.
- 2.The client is NOT in a boundary.
- 3.There is a Distribution Point associated with the boundary group the client is NOT in.
- 4.The content is distributed successfully to the DP.
- 5.The software / update / package is deployed to the device.
- 6.Enable verbose logging on the site system with the MP role and the client
Client is not in boundary group
Notable Ways to Identity when a Client is Not in Boundary Group
If a client is not in a boundary group here are some snippets that can be helpful.
Log File
Log Description
LocationServices.log
Client is not in any boundary group and ConfigMgr is no longer managing WindowsDO GPO. Set WindowsDO GPO to default values. Mode = LAN. GroupID = empty
CAS.log
The number of discovered DPs(including Branch DP and Multicast) is 0
In the content location request in the CAS.log, you will also notice the BoundaryGroup element doesn't contain the fields GroupID, GroupGUID, or GroupFlag.
ContentLocationRequest comparison
Tip: Search the text "ContentLocationRequest" in the CAS.log to find location requests performed on the client-side logs.
In the CAS.log here is an example of a response from the management point when the client is in no boundary group, and fallback is disabled, and no content was found.
1
<ContentLocationReply SchemaVersion="1.00" BGRVersion="1" ContentFlags="86016" HashAlgorithm="32780" AlgorithmPreference="4" Hash="5230AF02CF06BDD36CDD10FD13C4DF795F465E46FC47E1B1A00E44E2589AA6AD" ExcludeFileListHash="" RelatedContentID="">
2
<BoundaryGroups BoundaryGroupListRetrieveTime="2021-04-28T15:36:56.910"/>
3
<ContentInfo/>
4
<Sites>
5
<Site>
6
<MPSite SiteCode="DM2" MasterSiteCode="DM2" SiteLocality="FALLBACK"/>
7
<LocationRecords/>
8
</Site>
9
</Sites>
10
<RelatedContentIDs/>
11
</ContentLocationReply>
Copied!
We can compare this to the response received in the CAS.log when the content was found and the client is in a boundary group.
1
<ContentLocationReply SchemaVersion="1.00" BGRVersion="1">
2
<BoundaryGroups BoundaryGroupListRetrieveTime="2021-04-28T15:00:21.637" IsOnVPN="0">
3
<BoundaryGroup GroupID="16777218" GroupGUID="ad36e248-a857-45e6-ab82-c179bb680b18" GroupFlag="0"/>
4
</BoundaryGroups>
5
<ContentInfo PackageFlags="0" DeploymentFlags="9223372036855300962">
6
<ContentHashValues/>
7
</ContentInfo>
8
<Sites>
9
<Site>
10
<MPSite SiteCode="DM1" MasterSiteCode="DM1" SiteLocality="LOCAL" IISPreferedPort="80" IISSSLPreferedPort="443"/>
11
<LocationRecords>
12
<LocationRecord LocalityEx="BOUNDARYGROUP" DPFallbackIn="PT0M">
13
<URL Name="http://DEMO1.CONTOSO.LOCAL/SMS_DP_SMSPKG$/9c93cbee-aabc-4daf-996a-4580daa09c87" Signature="http://DEMO1.CONTOSO.LOCAL/SMS_DP_SMSSIG$/9c93cbee-aabc-4daf-996a-4580daa09c87.1.tar" Capability="0"/>
14
<ADSite Name="Default-First-Site-Name"/>
15
<IPSubnets>
16
<IPSubnet Address="192.168.10.0"/>
17
<IPSubnet Address=""/>
18
</IPSubnets>
19
<Metric Value=""/>
20
<Version>9040</Version>
21
<Capabilities SchemaVersion="1.0">
22
<Property Name="SSLState" Value="0"/>
23
</Capabilities>
24
<ServerRemoteName>DEMO1.CONTOSO.LOCAL</ServerRemoteName>
25
<DPType>SERVER</DPType>
26
<Windows Trust="1"/>
27
</LocationRecord>
28
</LocationRecords>
29
</Site>
30
</Sites>
31
<RelatedContentIDs/>
32
</ContentLocationReply>
Copied!
If the content is not found, you will be able to note the following line in StateMessage.log
State message(State ID : 5003) with TopicType 1702 and TopicId ScopeId_CA3F999B-7A81-43BF-912C-56E267C1E565/DeploymentType_677dac51-d2a2-4bbd-aeb0-4b845aa288cd/1 has been recorded for S-1-5-21-1880864260-2612682489-34998949-6112
State messages - Configuration Manager
docsmsft
StateMessage ID 5003 with TopicType 1702
Scenario 3: Download Failure Due to Content Distribution Failure
This scenario covers some of the basics in troubleshooting content that has failed to distribute to a distribution point.
Log Files for Troubleshooting Content Distribution in ConfigMgr
Log File
Log Text
distmgr.log
Records details about package creation, compression, delta replication, and information updates. It can also include other activities from the distribution manager component. For example, installing a distribution point, connection attempts, and installing components. For more information on other functionality that uses this log, see Service connection point and OS deployment.
PkgXferMgr.log
Records the actions of the SMS_Executive component that is responsible for sending content from a primary site to a remote distribution point.
Understand components and threads - Configuration Manager
docsmsft
Example of Error if the Source Folder Doesn't Exist or the Site Server Doesn't Have Access
Log File
Log Text
distmgr.log
The source directory \\demo3\Sources\Applications\Igor Pavlov\7-Zip (x64) - MSI Install\aad16824-5471-4dd8-ae69-f71167d7051d\ doesn't exist or the SMS service cannot access it, Win32 last error = 2
distmgr.log
Failed to take snapshot of one or more contents in package DM300013
Scenario 4: Download Failure Due to Invalid Content
For scenario 4, we delete a file directly from the FileLib and modify a file directly in the FileLib. We will cover the content library in a future video. This builds off of scenario 3 by showing specific failure scenarios.
The fix for this type of scenario may be as simple as performing an 'update content' or 'redistribute' the content which is failing. Other times you may have to validate antivirus exclusions on your endpoints and servers. Using the log files to identify what type of error is occurring will be key to resolving the issue and determine where the issue lies.
Example of Error if the Hash of the Downloaded File Does Not Match the Expected Hash
Log File
Log Text
CAS.log
Failed to do hash verification with preference : 4. Try to verify at next hash algorithm
CAS.log
Download failed for content Content_ebb608a6-a6a7-4384-b168-63a0f446b7fb.1 under context S-1-5-21-1880864260-2612682489-34998949-6112, error 0x80091007
DCMAgent.log
appState: DownloadFailed
The above log files and lines would be helpful from the client when determining if there is a hash mismatch issue. It is good to note that the hash mismatch may not be due to a corrupt file in the Content Library, but it is still worth investigating. This can also commonly be due to Antivirus on the endpoint which lacks the prover exclusions for example.
Example of Error if a File is Missing from the Content Library When a Client Requests It
Log File
Log Text
DataTransferService.log
[CCMHTTP] ERROR: URL=http://DEMO4.CONTOSO.LOCAL:80/SMS_DP_SMSPKG$/Content_ac0256aa-503f-48f1-bfa0-972d685cf50d.1, Port=80, Options=224, Code=0, Text=CCM_E_BAD_HTTP_STATUS_CODE
DataTransferService.log
Successfully queued event on HTTP/HTTPS failure for server 'DEMO4.CONTOSO.LOCAL'.
DataTransferService.log
Error sending DAV request. HTTP code 404, status 'Not Found'
DataTransferService.log
GetDirectoryList_HTTP('http://DEMO4.CONTOSO.LOCAL:80/SMS_DP_SMSPKG$/Content_ac0256aa-503f-48f1-bfa0-972d685cf50d.1') failed with code 0x800705b4.
DataTransferService.log
Error retrieving manifest (0x800705b4).
IIS Logs on DP
2021-04-29 02:25:19 192.168.10.22 PROPFIND /SMS_DP_SMSPKG$/Content_ac0256aa-503f-48f1-bfa0-972d685cf50d.1 - 80 CONTOSO\DEMO4CLIENT$ 192.168.10.38 SMS+CCM+5.0 - 404 0 0 1704 2
Log Files for Troubleshooting
Client Logs
Log File
Description
CAS.log
The Content Access service. Maintains the local package cache on the client.
CIAgent.log
Records details about the process of remediation and compliance for compliance settings, software updates, and application management.
CIDownloader.log
Records details about configuration item definition downloads.
CIStateStore.log
Records changes in state for configuration items, such as compliance settings, software updates, and applications.
ClientLocation.log
Records tasks that are related to client site assignment.
CMBITSManager.log
Records information for Background Intelligent Transfer Service (BITS) jobs on the device.
ContentTransferManager.log
Schedules the Background Intelligent Transfer Service (BITS) or Server Message Block (SMB) to download or access packages.
DataTransferService.log
Records all BITS communication for policy or package access.
DCMAgent.log
Records high-level information about the evaluation, conflict reporting, and remediation of configuration items and applications.
LocationServices.log
Records the client activity for locating management points, software update points, and distribution points.
StateMessage.log
Records details about software update state messages that are created and sent to the management point.
UpdatesDeployment.log
Records details about deployments on the client, including software update activation, evaluation, and enforcement. Verbose logging shows additional information about the interaction with the client user interface.
Server Logs
Log File
Description
MP_Location.log
Records location request and reply activity from clients.
IIS Logs
Built-in IIS logging - useful for all IIS based ConfigMgr roles. Log location and name may vary.
Copy link
Contents
Video: Content Troubleshooting Deep Dive (Full Video)
Video: Content Troubleshooting Video Timeline
Scenario 1: Succesful Content Lookup Workflow
Pre-Conditions
Successful Content Lookup Flow
Snippets of Location Request
Scenario 2: Failed Content Lookup Due to Boundary Misconfiguration
Pre-Conditions
Notable Ways to Identity when a Client is Not in Boundary Group
Scenario 3: Download Failure Due to Content Distribution Failure
Log Files for Troubleshooting Content Distribution in ConfigMgr
Example of Error if the Source Folder Doesn't Exist or the Site Server Doesn't Have Access
Scenario 4: Download Failure Due to Invalid Content
Example of Error if the Hash of the Downloaded File Does Not Match the Expected Hash
Example of Error if a File is Missing from the Content Library When a Client Requests It
Log Files for Troubleshooting
Client Logs
Server Logs