SharePoint - Retention Policies

Updated on 3/7/2015

Site Properties

bool dlc_sitehasexpirationpolicy
bool dlc_sitehaspolicy
bool dlc_webhasexpirationpolicy
bool allowslistpolicy

Library Item Properties

DateTime _dlc_ExpireDate
DateTime _dlc_ExpireDateSaved
String _dlc_policyId (this will contain a location or content type id based on policy type)
String ItemRetentionFormula (Xml definition of the retention policy)
String _dlc_ItemStageId (id of the retention stage the item is in, delete this prop to reset, see script)
String _dlc_ItemScheduleId (id of retention schedule)

Note: these properties can be stored inside of Office documents and get promoted/demoted when a document containing these properties gets uploaded to a document library. View the properties using an Office client, File->Properties->Advanced Properties->Custom tab.

Library.RootFolder Properties

bool HasDirtyPolicy
bool UseListPolicy
bool SystemUsesPolicy
bool dlc_listHasExpirationPolicy

Timer Job "Information Management Policy"

Responsible for updating expiration dates. The Information management policy job will only update the expiration dates on a library that has the $library.RootFolder.Properties[“HasDirtyPolicy”] = $true. This only gets set to true after modifying a library’s retention policy. This job will have no effect on libraries that haven’t had a change to their retention policy. Note: HasDirtyPolicy only applies to location based (source=Libraries and Folders) retention policies.

To mark content type policies dirty, they need to have the following xml after the retention policy xml in the content type schemaxml (see below for full example):

<XmlDocument NamespaceURI="microsoft.office.server.policy.changes">
      <PolicyDirtyBag xmlns="microsoft.office.server.policy.changes">
        <Microsoft.Office.RecordsManagement.PolicyFeatures.Expiration op="Change" />
      </PolicyDirtyBag>
    </XmlDocument>

Timer Job “Expiration Policy”

Responsible for performing expiration actions on expired documents. When the expiration policy job finds a document that is expired it will recalculate the expiration date to verify that it is set correctly. If it finds that the newly calculated date is in the future, it will not perform the expiration action and it will update the expiration date of the document. So until the document gets processed by the expiration policy job, it may appear to have an incorrect expiration date.

Troubleshooting / Misc. Notes

• Policies for content types can be defined using OM code and can be viewed by inspecting the SchemaXML property for the CT of the list.
• Location based (List) policies are stored in the /Forms/RetentionPolicy.Xml file.
• Verify that the content types on the list do not have duplicated fields, this will cause the document parser to fail a schema validation and the expiration date might not get set correctly.
• Verify the properties listed above exist and have appropriate values.
• Quick test, download a problematic document from a library, rename it, then re-upload it to the library. Verify it gets uploaded successfully, check ULS for errors computing expiration.
• Ensure event receivers are defined in the SchemaXML

Example: Creating a Retention Policy Using OM Code

SPSite site = new SPSite("http://dmg-10/sites/test/");
SPWeb web = site.AllWebs["testsite"];
SPList list = web.Lists["DocumentLibrary2"];
SPContentType ct = list.ContentTypes["ctname"];
Policy policy = Policy.GetPolicy(ct);
if (policy == null)
{
Policy.CreatePolicy(ct, null);
policy = Policy.GetPolicy(ct);
} 
policy.Items.Add("Microsoft.Office.RecordsManagement.PolicyFeatures.Expiration", xmlDef);            
ct.Update();
list.Update();
web.Update();
web.Dispose();

XML Definition

This example shows a definition for both records and non-records.

<Schedules nextStageId='3' default='false'>
                <Schedule type='Default'>
                  <stages>
                    <data stageId='2'>
                      <formula id='Microsoft.Office.RecordsManagement.PolicyFeatures.Expiration.Formula.BuiltIn'>
                        <number>4</number>
                        <property>Created</property>
                        <propertyId>8c06beca-0777-48f7-91c7-6da68bc07b69</propertyId>
                        <period>years</period>
                      </formula>
                      <action type='action' id='Microsoft.Office.RecordsManagement.PolicyFeatures.Expiration.Action.MoveToRecycleBin' />
                    </data>
                  </stages>
                </Schedule>
                <Schedule type='Record'>
                  <stages>
                    <data stageId='1'>
                      <formula id='Microsoft.Office.RecordsManagement.PolicyFeatures.Expiration.Formula.BuiltIn'>
                        <number>8</number>
                        <property>Created</property>
                        <propertyId>8c06beca-0777-48f7-91c7-6da68bc07b69</propertyId>
                        <period>years</period>
                      </formula>
                      <action type='action' id='Microsoft.Office.RecordsManagement.PolicyFeatures.Expiration.Action.MoveToRecycleBin' />
                    </data>
                  </stages>
                </Schedule>
              </Schedules>

This is what it will look like in the UI

SchemaXml snippet for a content type that has a dirty policy

<XmlDocuments>
    <XmlDocument NamespaceURI="office.server.policy">
      <p:Policy xmlns:p="office.server.policy" id="" local="true">
        <p:Name>Document</p:Name>
        <p:Description>
        </p:Description>
        <p:Statement>
        </p:Statement>
        <p:PolicyItems>
          <p:PolicyItem featureId="Microsoft.Office.RecordsManagement.PolicyFeatures.Expiration" staticId="0x01010037A44EA1F10FD848AC40D618F901463C|-288256358" UniqueId="b370baf2-3345-4c75-9462-492f7eafed91">
            <p:Name>Retention</p:Name>
            <p:Description>Automatic scheduling of content for processing, and performing a retention action on content that has reached its due date.</p:Description>
            <p:CustomData>
              <Schedules nextStageId="4">
                <Schedule type="Default">
                  <stages>
                    <data stageId="1">
                      <formula id="Microsoft.Office.RecordsManagement.PolicyFeatures.Expiration.Formula.BuiltIn">
                        <number>1</number>
                        <property>Created</property>
                        <propertyId>8c06beca-0777-48f7-91c7-6da68bc07b69</propertyId>
                        <period>days</period>
                      </formula>
                      <action type="action" id="Microsoft.Office.RecordsManagement.PolicyFeatures.Expiration.Action.Record" />
                    </data>
                    <data stageId="3">
                      <formula id="Microsoft.Office.RecordsManagement.PolicyFeatures.Expiration.Formula.BuiltIn">
                        <number>25</number>
                        <property>Created</property>
                        <propertyId>8c06beca-0777-48f7-91c7-6da68bc07b69</propertyId>
                        <period>years</period>
                      </formula>
                      <action type="action" id="Microsoft.Office.RecordsManagement.PolicyFeatures.Expiration.Action.Delete" />
                    </data>
                  </stages>
                </Schedule>
                <Schedule type="Record">
                  <stages>
                    <data stageId="2">
                      <formula id="Microsoft.Office.RecordsManagement.PolicyFeatures.Expiration.Formula.BuiltIn">
                        <number>20</number>
                        <property>Created</property>
                        <propertyId>8c06beca-0777-48f7-91c7-6da68bc07b69</propertyId>
                        <period>years</period>
                      </formula>
                      <action type="action" id="Microsoft.Office.RecordsManagement.PolicyFeatures.Expiration.Action.Delete" />
                    </data>
                  </stages>
                </Schedule>
              </Schedules>
            </p:CustomData>
          </p:PolicyItem>
        </p:PolicyItems>
      </p:Policy>
    </XmlDocument>
    <XmlDocument NamespaceURI="microsoft.office.server.policy.changes">
      <PolicyDirtyBag xmlns="microsoft.office.server.policy.changes">
        <Microsoft.Office.RecordsManagement.PolicyFeatures.Expiration op="Change" />
      </PolicyDirtyBag>
    </XmlDocument>

PowerShell to reset retention stages and mark ct policies dirty

#Step 1: Run this Powershell Script, modify the web and list accordingly
$site = get-spsite http://dmg-10:88
foreach($web in $site.AllWebs)
{
	$title = $web.Title + " | " + $web.Url
	Write-Output "Processing web: $title" 
	foreach($list in $web.Lists)
	{
		$title =  $list.Title
		Write-Output "    Processing list: $title"		
				
		#undeclare each document as a record
		foreach($item in $list.Items)
		{
			if([Microsoft.Office.RecordsManagement.RecordsRepository.Records]::IsRecord($item))
			{
				$title = $item.Name
				Write-Output "        Undeclaring $title"
				[Microsoft.Office.RecordsManagement.RecordsRepository.Records]::UndeclareItemAsRecord($item)				
			}
		}
				 
		#remove any stage/schedule information from the items		
		foreach($item in $list.Items)
		{
			$modified = $false
			
			if($item.Properties.Contains("_dlc_ItemStageId"))  
			{
				$title = $item.Name
				Write-Output "        Removing _dlc_ItemStageId from: $title" 
				$item.Properties.Remove("_dlc_ItemStageId")
				$modified = $true
			}
			
			if($item.Properties.Contains("_dlc_ItemScheduleId"))  
			{
				$title = $item.Name
		    	Write-Output "        Removing _dlc_ItemScheduleId from: $title"
				$item.Properties.Remove("_dlc_ItemScheduleId") 
				$modified = $true
		    }
		
			if($modified -eq $true) 
			{
				$item.SystemUpdate();
			}
		}
		
		#refresh any policies on the list so that they get processed by the IMP timer job
		foreach($ct in $list.ContentTypes)
		{					
			$policy = [Microsoft.Office.RecordsManagement.InformationPolicy.Policy]::GetPolicy($ct)	
			if($policy -ne $null)
			{						
				foreach($item in $policy.Items)
				{
					$item.Update()
				}
				$policy.Update()
				$title = $ct.Name
				Write-Output "        Dirtying policy on ct: $title"
			}
		}		
	}
}
Write-Host "Script Complete"
Write-Output "Script Complete"
#Step 2: modify content type policy to mark it as dirty so the IMP timer job processes it
#Step 3: run "Information Management Policy" timer job
#Step 4: run "Expiration Policy" timer job