MarabouStork | Extract Data, Screen Scraping, Web Crawling, Competitor Price Analysis, Combine and Compare your competitors websites | Using Powershell to set up an SPTimerJob
post-template-default,single,single-post,postid-565,single-format-standard,ajax_fade,page_not_loaded,,side_area_uncovered_from_content,qode-theme-ver-16.0.1,qode-theme-bridge,wpb-js-composer js-comp-ver-5.4.7,vc_responsive

Using Powershell to set up an SPTimerJob

Using Powershell to set up an SPTimerJob

In a previous blog I talked you though a little console app which I wrote while on a train one morning that checked the validity of any url stored in xml (namely infopath) documents held in Sharepoint.

The next thing on the agenda was to set about integrating the code with our Sharepoint implementation. Now, don’t get me wrong, I have nothing against console apps, but it just didn’t seem like the right solution, after all it was going to need to be scheduled to run silently and frequently, and trying to MSDeploy to the windows task scheduler just seemed dirty. No, the right thing to do was going to be to implement it as a Sharepoint Timer Job.

This presented me with a couple of challenges:

  • Our Powershell deployment scripts would need to set up any dependencies that the functionality relied on (namely the sharepoint list that stored the references to the invalid documents).
  • The same powershell scripts would also need to create the SPTimerJob definition and set up the execution schedule.
  • The App.Config file would have to go, as there is no home for this in the SPTimerJob space, so an alternative would have to be found

I would also have to consider that the ideal solution would see that settings that controlled the execution schedule and the document libraries to query could be controlled separated per environment. This is because you might want to configure the job to run more frequently and with a smaller data set on your dev environment to reduce outbound traffic and make it easier to diagnose potential problems early.

But before we get into Powershell goodness lets talk briefly about some of the more significant changes that we are going to need to make to our console app.

SPTimerJob is our new entry point

In a console application the entry point is usually the static void Main(object[] args) method in program.cs. For SPTimerJobs however the entry point will be a class which derives SPTimerJob. In our case this class is contained within LinkCheckerTimerJob.cs. As well as implementing two standard constructors we also have to override the Execute method and provide code that will start our custom process (in this case the method ValidateDocumentUrls contained within helper.cs). You can see below that we are simply enumerating through the sites in the current farm and calling the ValidateDocumentUrls functionality for each of them (note also the signature of this method has changed a little from the console app to support additional parameters).

Later we will be instantiating this object and using it to create our new SPTimerJob definition using the Powershell deployment script.

using System;
using Microsoft.SharePoint;
using Microsoft.SharePoint.Administration;

namespace MarabouStork.Sharepoint.LinkChecker
    public class LinkCheckerTimerJob : SPJobDefinition
        public LinkCheckerTimerJob() : base() { }

        public LinkCheckerTimerJob(string jobName, SPWebApplication webapp) : base(jobName, webapp, null, SPJobLockType.Job) { }

        public override void Execute(Guid targetInstanceId)

            foreach (SPSite siteCollection in this.WebApplication.Sites)
                LinkChecker.ValidateDocumentUrls(siteCollection.Url, this.WebApplication, DateTime.MinValue);

Replacing app.config for SPTimerJobs

The next thing to do is replace app.config with something we can use with our SPTimerJob service. At this point I feel that I should thank Michal “LaM” Franc for his very informative blog on persistence of objects in Sharepoint.

In Sharepoint there is an SPPersistedObject class that you can derive which will ensure that whenever your object is instantiated it is automatically serialized and persisted to your sharepoint configuration database (assuming your not unhinged and have altered the default persistence provider).

So by creating a new class that implements SPPersistedObject we are able to create an instance of this class, populate it with the values we would normally store in the app.config file and the commit this to the Sharepoint object store for later retrieval by our SPTimerJob. Whats more we can use Powershell to create and store this object so we can choose which values to assign to it during deployment based on the environment we are deploying to.

using System.Runtime.InteropServices;
using Microsoft.SharePoint.Administration;

namespace MarabouStork.Sharepoint.LinkChecker
    public class LinkCheckerPersistedSettings: SPPersistedObject
        public string DocLibraries;

        public string FieldsToCheck;

        public bool UnpublishInvalidDocs;

        public LinkCheckerPersistedSettings() { }
        public LinkCheckerPersistedSettings(string name, SPPersistedObject parent) : base(name, parent) { }

Obviously you can store as many values here as you wish, the only thing you have to be very careful about is that the object remains serializable. The [Persisted] attributes above indicate which member values are persisted to your Sharepoint config database.

So how do we get at the settings once they are persisted? If you look at line 40 within Helper.cs (or at the code excerpt below) you will see that we use the current Farm to retrieve the LinkCheckerPersistedSettings object by passing the Name we gave to that object when we created it (this will become clear later in this blog), the Id of the webApplication and our LinkCheckerPersistedSettings type.

var farm = Microsoft.SharePoint.Administration.SPFarm.Local;
var settings = farm.GetObject("MarabouStork.Sharepoint.LinkChecker.LinkCheckerSettings", webApplication.Id, typeof(LinkCheckerPersistedSettings)) as LinkCheckerPersistedSettings;
If you look at the MSDN documentation you will see that there is a GetChild<T>(string name) method available directly on the SPWebApplication instance. Given this, it may seem that using the SPFarm.GetObject(string, Guid, Type) method is largely unnecessary. And I would agree, however the problem with GetChild<T>(string name) and GetChild<T>() is that they are overloaded generic methods and as of yet there is no easy way to execute these types of methods in PowerShell without using Reflection. This caused me quite a headache when it came to deleting the existing instance of the settings object before creating a new one within PowerShell (more on this later).

Heres the code in context:

public static void ValidateDocumentUrls(string siteUrl, SPWebApplication webApplication, DateTime modsSinceDate)
   baseSiteUrl = siteUrl;

   var farm = Microsoft.SharePoint.Administration.SPFarm.Local;
   var settings = farm.GetObject("MarabouStork.Sharepoint.LinkChecker.LinkCheckerSettings", webApplication.Id, typeof(LinkCheckerPersistedSettings)) as LinkCheckerPersistedSettings;

   if (settings != null)
       // Determine the list of document libraries to check
       string[] docLibs = settings.DocLibraries.Split(new char[] { ',', ';' });
       List docLibsToCheck = new List();
       foreach (string docLib in docLibs) docLibsToCheck.Add(string.Format("{0}/{1}", baseSiteUrl, docLib.Replace(" ", "%20")));

       // Determine the list of document fields to check
       List fieldsToCheck = new List(settings.FieldsToCheck.Split(new char[] { ',', ';' }));

       // Should we unpublish documents that contain invalid urls?
       shouldUnpublish = settings.UnpublishInvalidDocs;

There are quite a few minor alterations that have been made to the link checker code that I wont go into in depth because they are all self explanatory and can easily be identified by comparing the LinkCheckerConsoleHelper.cs and LinkCheckerHelper.cs files with a comparison tool such as Scooter Software’s “Beyond Compare”

One final point to draw you attention to is the fact that we have also signed our LinkChecker dll so that it can be placed into the GAC. It’s on my ever-increasing list of stuff to do to investigate whether SPTimeJobs can be de-GACified.

Powershell goodness – The InvalidUrlsInResources

The first thing we are going to do in our deployment script (LinkCheckerDeploy.ps1) is load the after the usual loading of Sharepoint cmdlets is load our custom dll into memory from the GAC.

#Load the custom timer job assembly
Write-Host Loading Assembly
[System.Reflection.Assembly]::Load("MarabouStork.Sharepoint.LinkChecker, Version=, Culture=neutral, PublicKeyToken=5f0a99c6679b2dcc")

#Restart the timer service and delete the existing timer job
Write-Host Restart Timer Service....
Restart-Service SPTimerV4
You will also notice that we restart the timer service. We have run into several problems in the past were we have altered and deployed new versions of existing dll’s to the GAC and have been dumbfounded as to why those changes have not taken effect. We often find in this situation that it is because the SPTimerV4 service and/or PowerGUI are using cached versions of the dll’s.

We then need to create the Sharepoint list which is used to store a reference to all documents containing invalid urls:

function Create-InvalidUrlsInResources()
	$listname = "InvalidUrlsInResources"

	$userfieldtype = [Microsoft.Sharepoint.SPFieldType]::User
	$textfieldtype = [Microsoft.Sharepoint.SPFieldType]::Text
	$urlfieldtype = [Microsoft.Sharepoint.SPFieldType]::URL

	# Drop and recreate the lists
	if($web.Lists[$listname] -ne $null)
		Write-Host Dropping list $listname
		while($web.Lists[$listname] -ne $null)
			Start-Sleep -Seconds 1

	Write-Host Creating List $listname
	$list = $web.Lists[$listname]
	$list.OnQuickLaunch = 1
	$list.Fields.Add("User", $textfieldtype, 1)
	$list.Fields.Add("Document", $urlfieldtype, 1)
	$list.Fields.Add("Message", $textfieldtype,  0)

	#Add the columns


	$spView = $web.GetViewFromUrl("/Lists/" + $listname + "/AllItems.aspx")
	$spView.ViewFields.Add( $list.Fields["User"])
	$spView.ViewFields.Add( $list.Fields["Document"])
	$spView.ViewFields.Add( $list.Fields["Message"])



This script performs 4 main actions 1) It obtains the field type enumerations that we will use when specifying the types of the fields in our list. 2) It deletes the list if it already exists 3) It creates a new list with the new columns and 3) It adds the new columns to the default view.

Powershell greatness – The LinkCheckerSettings (App.config replacement)

Once the list InvalidUrlsInDocuments we then need to create an instance of our settings class and persist it to the sharepoint config db. At line 110 you will see the same method being used as is used in Heler.cs to retrieve the named settings object. In this instance we are doing this because if a settings object already exists it needs to be deleted before a new one can be created.

function Persist-LinkCheckerSettings()
	Write-Host Persisting Timer Job Settings

	#Setup persisted parameters to configure link checker
	$site = $web.Site
	$webApplication = $site.WebApplication
	$configName = "MarabouStork.Sharepoint.LinkChecker.LinkCheckerSettings"

	$farm = [Microsoft.SharePoint.Administration.SPFarm]::Local
	$existingSettingsobj = $farm.GetObject($configName, $webApplication.Id, [MarabouStork.Sharepoint.LinkChecker.LinkCheckerPersistedSettings])
    if ($existingSettingsObj -ne $null) { $existingSettingsObj.Delete() }

	#Create a new persisted settings object
	[MarabouStork.Sharepoint.LinkChecker.LinkCheckerPersistedSettings] $settings = New-Object "MarabouStork.Sharepoint.LinkChecker.LinkCheckerPersistedSettings" ($configName, $webApplication)
	$settings.SiteUrl = $url
	$settings.DocLibraries = "Introduction Resources"
	$settings.FieldsToCheck = "my:URL;my:Description"
	$settings.UnpublishInvalidDocs = 1

At this point I am going to go off on a little tangent:

If your like me you don’t get a particularly warm fuzzy feeling from just accepting that your objects go into this black sharepoint hole and somehow get stored somewhere….probably, maybe??!? So at this point if you open up Microsoft SQL Server Management Studio ill show you where your object is.

Connect to your Sharepoint database server, Start a new query and execute the following:

use Sharepoint_config

select * from objects where name = 'LinkCheckerSettings'

You should now see the db entry representing your persisted object and the key values used to retrieve it in your code including the Name, the ParentId (which is the id of the WebApplication) and the fully qualified object type stored in the root node of the serialized object data. If you look at the serialized object data you should also recognise your settings.

Powershell gorgeousness – Creating the SPTimerJob

And finally….

function Create-LinkCheckerTimerJob()
	$web = Get-SPWeb($url)
	$jobName = "MarabouStork.Sharepoint.LinkChecker"

	#Delete the timer job if it already exists
	Get-SPTimerJob | where { $_.TypeName -eq "MarabouStork.Sharepoint.LinkChecker.LinkCheckerTimerJob" } | % { $_.Delete() }

	#Create an execution schedule
	[Microsoft.SharePoint.SPSchedule] $schedule = [Microsoft.SharePoint.SPSchedule]::FromString("Every 10 minutes between 0 and 59")

	#Instantiate the custom timer job object
	[MarabouStork.Sharepoint.LinkChecker.LinkCheckerTimerJob] $linkChecker = New-Object "MarabouStork.Sharepoint.LinkChecker.LinkCheckerTimerJob" ($jobName,(Get-SPWebApplication $url))

	$linkChecker.Schedule = $schedule
	$linkChecker.Title = $jobName


	Get-SPTimerJob -Identity $jobName | Start-SPTimerJob
No Comments

Post A Comment