Code Sample

Scripted Provisioning of Office 365 Unified Labels

I’ve recently been working on a project implementing O365 unified Labels, in a greenfield scenario where programmatic provisioning of the configuration was required.

Some of the Microsoft documentation covering how to configure Unified Labels via PowerShell are good, while others are very weak. Take for example the Set-Label cmdlet with the -examples switch:

Se-Label -Examples

OK, that isn’t very helpful 🙂

Additionally, the online docs (e.g. https://docs.microsoft.com/en-us/powershell/module/exchange/policy-and-compliance/set-label?view=exchange-ps) fail to reference some of the cmdlet parameters .

If we look at the parameters from the command “Get-Help Set-Label -Detailed”, we see:

Set-Label-Parameters

So the parameters that I wanted to set were LabelActions and Conditions. LabelActions configure headers, footers and watermarks, while Conditions define the O365 Sensitivity Types that are applied to a label.

The documentation for how to do this was non-existent, apart from some cryptic “Exchange” docs detailing how to define “MultiValuedProperty”, I was fairly stumped. I ended up up backwards engineering the configuration by setting it in the GUI, then capturing the Label in PowerShell. Once captured, look at the configuration of “Conditions” or LabelActions to see how those Properties are defined in the case of Unified Labelling.

The following script details how this all works together to create something useful. It provisions a new Label named “My Label”, with a green colour. Then it applies a header “HeaderText” and footer “FooterText” and then a watermark “WatermarkText”, all in Black and font size 10. Lastly it applies the O365 sensitivity types “ABA Routing Number” and “Argentina National Identity (DNI) Number” to the label, in Recommended mode.

Also covered for reference is the creation of a sub-label “My Sub Label”, beneath “My Label”.

# Define credentials
$AdminCredentials = Get-Credential "myadmin@oholics.net"
# Create the session
$Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://ps.compliance.protection.outlook.com/powershell-liveid/ -Credential $AdminCredentials -Authentication Basic -AllowRedirection
# Import the session
Import-PSSession $Session -DisableNameChecking
# Define the tenant
$MyTenant = "CN=Configuration,CN=<TenantID>.onmicrosoft.com,OU=Microsoft Exchange Hosted Organizations,DC=FFO,DC=extest,DC=microsoft,DC=com"
# Create the label
New-Label -DisplayName "My Label" -Name "My Label" -Comment "This is My Label" -Tooltip "My Label Tooltip" -AdvancedSettings @{color="#32CD32"}
# Set the properties of the label - header/ footer/ watermark and recommended conditions
Set-Label -Identity "CN=My Label,$MyTenant" -LabelActions '{"Type":"applycontentmarking","SubType":"header","Settings":[{"Key":"fontsize","Value":"10"},{"Key":"placement","Value":"Header"},{"Key":"text","Value":"HeaderText"},{"Key":"fontcolor","Value":"#000000"},{"Key":"margin","Value":"5"},{"Key":"alignment","Value":"Left"},{"Key":"disabled","Value":"false"}]}'
Set-Label -Identity "CN=My Label,$MyTenant" -LabelActions '{"Type":"applycontentmarking","SubType":"footer","Settings":[{"Key":"fontsize","Value":"10"},{"Key":"placement","Value":"Footer"},{"Key":"text","Value":"FooterText"},{"Key":"fontcolor","Value":"#000000"},{"Key":"margin","Value":"5"},{"Key":"alignment","Value":"Left"},{"Key":"disabled","Value":"false"}]}'
Set-Label -Identity "CN=My Label,$MyTenant" -LabelActions '{"Type":"applywatermarking","SubType":null,"Settings":[{"Key":"fontsize","Value":"10"},{"Key":"layout","Value":"Diagonal"},{"Key":"fontcolor","Value":"#000000"},{"Key":"disabled","Value":"false"},{"Key":"text","Value":"WatermarkText"}]}'
Set-Label -Identity "CN=My Label,$MyTenant" -Conditions '{"And":[{"Or":[{"Key":"CCSI","Value":"cb353f78-2b72-4c3c-8827-92ebe4f69fdf","Properties":null,"Settings":[{"Key":"minconfidence","Value":"75"},{"Key":"maxconfidence","Value":"100"},{"Key":"rulepackage","Value":"00000000-0000-0000-0000-000000000000"},{"Key":"mincount","Value":"1"},{"Key":"maxcount","Value":"2147483647"},{"Key":"policytip","Value":"AutoLabellingText"},{"Key":"name","Value":"ABA Routing Number"},{"Key":"groupname","Value":"Default"}]},{"Key":"CCSI","Value":"eefbb00e-8282-433c-8620-8f1da3bffdb2","Properties":null,"Settings":[{"Key":"minconfidence","Value":"75"},{"Key":"maxconfidence","Value":"100"},{"Key":"rulepackage","Value":"00000000-0000-0000-0000-000000000000"},{"Key":"mincount","Value":"1"},{"Key":"maxcount","Value":"2147483647"},{"Key":"policytip","Value":"AutoLabellingText"},{"Key":"name","Value":"Argentina National Identity (DNI) Number"},{"Key":"groupname","Value":"Default"},{"Key":"autoapplytype","Value":"Recommend"}]}]}]}'
# Check that the LabelActions and Conditions were set:
$Label=Get-Label -Identity "CN=My Label,$MyTenant"
$Label.LabelActions
$Label.Conditions
# If you want to create a sub label under "My Label":
# First get the top level label:
$MyLabel = Get-Label -Identity "CN=My Label,$MyTenant"
# Then create the new label referencing the top level label by name in the ParentId parameter:
New-Label -DisplayName "My Sub Label" -Name "My Sub Label" -ParentId $MyLabel.Name -Comment "This is My Sub Label" -Tooltip "My Sub Label Tooltip"
# When you are totally finished, disconnect the session
Remove-PSSession $Session

Once the labels are defined, we need to publish them with a policy. First create the policy, providing the label names and scope, then apply any required advanced settings to the policy.

Note the script below assumes that the last session was ended, we need to login again – else just continue the previous session.

# Define credentials
$AdminCredentials = Get-Credential "myadmin@oholics.net"
# Create the session
$Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://ps.compliance.protection.outlook.com/powershell-liveid/ -Credential $AdminCredentials -Authentication Basic -AllowRedirection
# Import the session
Import-PSSession $Session -DisableNameChecking
# Define the tenant
$MyTenant = "CN=Configuration,CN=<TenantID>.onmicrosoft.com,OU=Microsoft Exchange Hosted Organizations,DC=FFO,DC=extest,DC=microsoft,DC=com"
# Get the labels to add to the policy
$Label1=Get-Label -Identity "CN=My Label,$MyTenant"
$Label2=Get-Label -Identity "CN=My Sub Label,$MyTenant"
# Create the Label Policy
New-LabelPolicy -Name "My Label Policy" -Labels $Label1.name,$Label2.Name -ExchangeLocation "All"
# Set the advanced settings on the new policy
Set-LabelPolicy -Identity "My Label Policy" -AdvancedSettings @{EnableCustomPermissions="False"}
Set-LabelPolicy -Identity "My Label Policy" -AdvancedSettings @{AttachmentAction="Automatic"}
Set-LabelPolicy -Identity "My Label Policy" -AdvancedSettings @{HideBarByDefault="False"}
Set-LabelPolicy -Identity "My Label Policy" -AdvancedSettings @{RequireDowngradeJustification="True"}
# When you are totally finished, disconnect the session
Remove-PSSession $Session

Finally, the documentation states that label priority follows this rule: “A lower integer value indicates a higher priority, the value 0 is the highest priority“. However, in practice the opposite is true.

Say for example you have the following labels “Public”, “Internal” and “Secret”; for the advanced setting “RequireDowngradeJustification” to apply as expected, (following the documentation) you would set “Secret” = 0, “Internal” = 1 and “Public” = 2. This actually has the opposite effect, making a downgrade from Secret to Public not raise the justification dialog box, while Public to Secret is classed as a downgrade; also the order of labels in the toolbar is the wrong way around. So the proper order should be: “Public” = 0, “Internal” = 1 and “Secret” = 2.

Additionally, the priority can get quite messed up if you have any existing labels or if you deploy the labels in the wrong order. Continuing from my example, but also throwing in 2 sub labels per top level label….

First connect (or continue the existing session), then get the current priorities. If they don’t match the output shown in the script, then start fixing them! Start by interactively running the priority settings for the top level labels (only do those that are not correct), starting with the highest values and working down. Check the priorities after each change.

Once the top level labels are correct, start fixing the sub labels (assuming they are not right). Reset them individually, again setting the highest value first, check the priorities after each change. Rinse and repeat until the order is as desired, then go have a G & T 🙂

# Define credentials
$AdminCredentials = Get-Credential "myadmin@oholics.net"
# Create the session
$Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://ps.compliance.protection.outlook.com/powershell-liveid/ -Credential $AdminCredentials -Authentication Basic -AllowRedirection
# Import the session
Import-PSSession $Session -DisableNameChecking
# Define the tenant
$MyTenant = "CN=Configuration,CN=<TenantID>.onmicrosoft.com,OU=Microsoft Exchange Hosted Organizations,DC=FFO,DC=extest,DC=microsoft,DC=com"
# Get the priorities of all labels
$a=Get-Label
foreach ($i in $a)
{
Write-host $i.Name, $i.Priority
}
# Output should be:
#Public 0
#PublicSub1 1
#PublicSub2 2
#Internal 3
#InternalSub1 4
#InternalSub2 5
#Secret 6
#SecretSub1 7
#SecretSub2 8
# If the priority was NOT correct, then:
# Set the priority of the top level labels (do this first).
Set-Label -Identity "CN=Secret,$MyTenant" -Priority 6
Set-Label -Identity "CN=Internal,$MyTenant" -Priority 3
Set-Label -Identity "CN=Public,$MyTenant" -Priority 0
# If the sub level label order is not correct, then use the following to correct (note, you may only need to set individual entries).
Set-Label -Identity "CN=SecretSub1,$MyTenant" -Priority 7
Set-Label -Identity "CN=SecretSub2,$MyTenant" -Priority 8
Set-Label -Identity "CN=InternalSub1,$MyTenant" -Priority 4
Set-Label -Identity "CN=InternalSub2,$MyTenant" -Priority 5
Set-Label -Identity "CN=PublicSub1,$MyTenant" -Priority 1
Set-Label -Identity "CN=PublicSub2,$MyTenant" -Priority 2
# When you are totally finished, disconnect the session
Remove-PSSession $Session

Delegating Group Management – Using the Lithnet FIM PowerShell Module

Within my AD structure, group management is delegated within certain OU’s, I now need to replicate that functionality in the FIM portal.

The is no real way of identifying which groups should be managed by whom, except the OU within which the group currently resides.

So, to start off with I need to get the parent OU of the group into the portal:

Import the OU into the MV:

Case "adOU-Group-ADMA-Import"
mventry("adOU").Value = Replace(csentry.DN.ToString, csentry.RDN.ToString & ",", "")

Setup an export flow for adOU into the portal.

Then, by using the Lithnet PowerShell Module, we can create all the sets and MPR’s required, below is a sample for creating one delegated “collection”. In production, my XML file is much bigger – delegating group management to around ten different groups.

Note, that you first need to create references to all users who might be given the rights to manage groups. This includes the FimServiceAdmin and FimServiceAccount – referenced by their ObjectID, the others are referenced by their AccountName. All members referenced in this section, are added to the __Set:GroupValidationBypassSet. This set is defined in the non-administrators set – not in this set – this bypasses the group validation workflow:

AllNonAdministratorsSet

Create a set of groups to be managed – the filter being the OU that the groups belong to & MembershipLocked=False

Create a set of administrators for this delegation – adding the explicit members

Then create the two MPR’s to allow the members of the administrative set to manage those groups – the first MPR allows modification (Read, Add and Remove) of the ExplicitMember attribute, while the second allows creation and deletion.

<?xml version="1.0" encoding="utf-8" ?>
<Lithnet.ResourceManagement.ConfigSync>
<Variables>
<Variable name="#domain#" value="%userdomain%"/>
<Variable name="#PATH#" value ="C:\some-path\" />
</Variables>
<Operations>
<!-- Create a Bunch of References to Recipients -->
<ResourceOperation operation="None" resourceType="Person" id="user1">
<AnchorAttributes>
<AnchorAttribute>AccountName</AnchorAttribute>
</AnchorAttributes>
<AttributeOperations>
<AttributeOperation operation="none" name="AccountName">user1</AttributeOperation>
</AttributeOperations>
</ResourceOperation>
<!-- Create Reference to Recipient -->
<ResourceOperation operation="None" resourceType="Person" id="user2">
<AnchorAttributes>
<AnchorAttribute>AccountName</AnchorAttribute>
</AnchorAttributes>
<AttributeOperations>
<AttributeOperation operation="none" name="AccountName">user2</AttributeOperation>
</AttributeOperations>
</ResourceOperation>
<!-- Create Reference to Recipient -->
<ResourceOperation operation="None" resourceType="Person" id="user3">
<AnchorAttributes>
<AnchorAttribute>AccountName</AnchorAttribute>
</AnchorAttributes>
<AttributeOperations>
<AttributeOperation operation="none" name="AccountName">user3</AttributeOperation>
</AttributeOperations>
</ResourceOperation>
<!-- Create Reference to Recipient -->
<ResourceOperation operation="None" resourceType="Person" id="FIMServiceAccount">
<AnchorAttributes>
<AnchorAttribute>ObjectID</AnchorAttribute>
</AnchorAttributes>
<AttributeOperations>
<AttributeOperation operation="none" name="ObjectID">1009a2cb-e7f8-4db9-9f02-04b91b1d966d</AttributeOperation>
</AttributeOperations>
</ResourceOperation>
<!-- Create Reference to Recipient -->
<ResourceOperation operation="None" resourceType="Person" id="FIMServiceAdmin">
<AnchorAttributes>
<AnchorAttribute>ObjectID</AnchorAttribute>
</AnchorAttributes>
<AttributeOperations>
<AttributeOperation operation="none" name="ObjectID">7fb2b853-24f0-4498-9534-4e10589723c4</AttributeOperation>
</AttributeOperations>
</ResourceOperation>
<!-- MPR's Sets etc: -->
<!-- GroupValidation Bypass Set -->
<ResourceOperation operation="Add Update" resourceType="Set" id="GroupValidationBypassSet">
<AnchorAttributes>
<AnchorAttribute>DisplayName</AnchorAttribute>
</AnchorAttributes>
<AttributeOperations>
<AttributeOperation operation="replace" name="DisplayName">__Set:GroupValidationBypassSet</AttributeOperation>
<AttributeOperation operation="replace" name="Description">This set bypasses Group Management Workflow</AttributeOperation>
<AttributeOperation operation="add" name="ExplicitMember" type="xmlref">user2</AttributeOperation>
<AttributeOperation operation="add" name="ExplicitMember" type="xmlref">user1</AttributeOperation>
<AttributeOperation operation="add" name="ExplicitMember" type="xmlref">FIMServiceAccount</AttributeOperation>
<AttributeOperation operation="add" name="ExplicitMember" type="xmlref">FIMServiceAdmin</AttributeOperation>
<AttributeOperation operation="add" name="ExplicitMember" type="xmlref">user3</AttributeOperation>
</AttributeOperations>
</ResourceOperation>
<!-- Create ABC Manual Groups Set -->
<ResourceOperation operation="Add Update" resourceType="Set" id="ABCManualGroupsSet">
<AnchorAttributes>
<AnchorAttribute>DisplayName</AnchorAttribute>
</AnchorAttributes>
<AttributeOperations>
<AttributeOperation operation="replace" name="DisplayName">__Set:ABC Manual Groups</AttributeOperation>
<AttributeOperation operation="replace" name="Description">Use this set to allow ABC helpdesk staff to administer these groups.</AttributeOperation>
<AttributeOperation operation="replace" name="Filter" type="filter">/Group[(adOU = 'OU=Manual Groups,OU=ABC,DC=blah,DC=ac,DC=uk') and (MembershipLocked = False)]</AttributeOperation>
</AttributeOperations>
</ResourceOperation>
<!-- Create ABC Manual Group Administrators Set -->
<ResourceOperation operation="Add Update" resourceType="Set" id="ABCManualGroupsAdministratorsSet">
<AnchorAttributes>
<AnchorAttribute>DisplayName</AnchorAttribute>
</AnchorAttributes>
<AttributeOperations>
<AttributeOperation operation="replace" name="DisplayName">__Set:ABC Manual Groups Administrators</AttributeOperation>
<AttributeOperation operation="replace" name="Description">Use this set to allow ABC helpdesk staff to administer these groups.</AttributeOperation>
<AttributeOperation operation="add" name="ExplicitMember" type="xmlref">user1</AttributeOperation>
<AttributeOperation operation="add" name="ExplicitMember" type="xmlref">user2</AttributeOperation>
</AttributeOperations>
</ResourceOperation>
<!-- ABC Manual Group Modify MPR -->
<ResourceOperation operation="Add Update" resourceType="ManagementPolicyRule" id="ABCManualGroupModifyMPR">
<AnchorAttributes>
<AnchorAttribute>DisplayName</AnchorAttribute>
</AnchorAttributes>
<AttributeOperations>
<AttributeOperation operation="replace" name="DisplayName">__MPR:Group management: ABC Manual Group administrators can update ABC Manual group resources</AttributeOperation>
<AttributeOperation operation="replace" name="Description">Allows ##xmlref:ABCManualGroupsAdministratorsSet:DisplayName## members to modify membership of ABC Manual Groups</AttributeOperation>
<AttributeOperation operation="replace" name="Disabled">false</AttributeOperation>
<AttributeOperation operation="replace" name="GrantRight">true</AttributeOperation>
<AttributeOperation operation="replace" name="ManagementPolicyRuleType">Request</AttributeOperation>
<AttributeOperation operation="replace" name="PrincipalSet" type="xmlref">ABCManualGroupsAdministratorsSet</AttributeOperation>
<AttributeOperation operation="replace" name="ResourceCurrentSet" type="xmlref">ABCManualGroupsSet</AttributeOperation>
<AttributeOperation operation="replace" name="ResourceFinalSet" type="xmlref">ABCManualGroupsSet</AttributeOperation>
<AttributeOperation operation="add" name="ActionType">Read</AttributeOperation>
<AttributeOperation operation="add" name="ActionType">Add</AttributeOperation>
<AttributeOperation operation="add" name="ActionType">Remove</AttributeOperation>
<AttributeOperation operation="add" name="ActionParameter">ExplicitMember</AttributeOperation>
</AttributeOperations>
</ResourceOperation>
<!-- ABC Manual Group Create/Delete MPR -->
<ResourceOperation operation="Add Update" resourceType="ManagementPolicyRule" id="ABCManualGroupCreateDeleteMPR">
<AnchorAttributes>
<AnchorAttribute>DisplayName</AnchorAttribute>
</AnchorAttributes>
<AttributeOperations>
<AttributeOperation operation="replace" name="DisplayName">__MPR:Group management: ABC Manual Group administrators can create and delete ABC Manual group resources</AttributeOperation>
<AttributeOperation operation="replace" name="Description">Allows ##xmlref:ABCManualGroupsAdministratorsSet:DisplayName## members to Create and Delete ABC Manual Groups</AttributeOperation>
<AttributeOperation operation="replace" name="Disabled">false</AttributeOperation>
<AttributeOperation operation="replace" name="GrantRight">true</AttributeOperation>
<AttributeOperation operation="replace" name="ManagementPolicyRuleType">Request</AttributeOperation>
<AttributeOperation operation="replace" name="PrincipalSet" type="xmlref">ABCManualGroupsAdministratorsSet</AttributeOperation>
<AttributeOperation operation="replace" name="ResourceCurrentSet" type="xmlref">ABCManualGroupsSet</AttributeOperation>
<AttributeOperation operation="replace" name="ResourceFinalSet" type="xmlref">ABCManualGroupsSet</AttributeOperation>
<AttributeOperation operation="add" name="ActionType">Create</AttributeOperation>
<AttributeOperation operation="add" name="ActionType">Delete</AttributeOperation>
<AttributeOperation operation="add" name="ActionParameter">*</AttributeOperation>
</AttributeOperations>
</ResourceOperation>
</Operations>
</Lithnet.ResourceManagement.ConfigSync>

Use Import-RMConfig -File <PathToXML> -Preview -Verbose to validate your xml and see what it would do. Drop the “-Preview” to make the change

An Alternative To Using The Generic Array From File Function

While looking to improve on my method of getting exceptions or a long list of mail suffixes into an array, to be checked during code execution, I came across this: https://msdn.microsoft.com/en-us/library/windows/desktop/ms696048(v=vs.85).aspx

This seemed to me to be a really nice solution, just defining all exceptions and suffixes within one file, read it in on code execution, then check for existence or whatever in the code.

So, given the following xml file:

<rules-extension-properties>
<mail-suffixes>
<values>blah.ac.uk,moo.ac.uk,meow.ac.uk,woof.ac.uk,squawk.ac.uk,hiss.ac.uk,neigh.ac.uk</values>
</mail-suffixes>
<ignore-email-address-errors>
<values>smartypants@woof.ac.uk</values>
</ignore-email-address-errors>
<ignore-functional-id-owner>
<values>random.person@someother.ac.uk</values>
</ignore-functional-id-owner>
</rules-extension-properties>

Add the System.Xml Import and declare the variables, so they are global:

Imports Microsoft.MetadirectoryServices
Imports System
Imports System.directoryservices
Imports ActiveDs
Imports System.Globalization
Imports Microsoft.MetadirectoryServices.Logging
Imports System.IO
Imports System.Xml
Public Class MAExtensionObject_MYADMA
Implements IMASynchronization
'Date & Logginglevel variables for logging files:
Dim dtDateNowHour As Integer = Date.Now.Hour
Dim dtDateNowDay As Integer = Date.Now.Day
Dim dtDateNowMonth As Integer = Date.Now.Month
Dim dtDateNowYear As Integer = Date.Now.Year
Dim loggingLevel As Integer = 0
Dim ValidMailSuffixes As String
Dim IgnoreFunctionalIDOwner As String
Dim IgnoreEmailAddressErrors As String

Add the code to read the xml file into the Initialize Sub:

Public Sub Initialize() Implements IMASynchronization.Initialize
Try
Dim config As XmlDocument = New XmlDocument()
config.Load("C:\FIMControl\rules-config.xml")
'
Dim suffixesnode As XmlNode = config.SelectSingleNode("rules-extension-properties/mail-suffixes")
Dim suffixvaluenode As XmlNode = suffixesnode.SelectSingleNode("values")
Dim suffixesvalue As String = suffixvaluenode.InnerText
ValidMailSuffixes = suffixvaluenode.InnerText
'
Dim ieaenode As XmlNode = config.SelectSingleNode("rules-extension-properties/ignore-email-address-errors")
Dim ieeavaluenode As XmlNode = ieaenode.SelectSingleNode("values")
Dim ieaevalue As String = ieeavaluenode.InnerText
IgnoreEmailAddressErrors = ieeavaluenode.InnerText
'
Dim ifionode As XmlNode = config.SelectSingleNode("rules-extension-properties/ignore-functional-id-owner")
Dim ifiovaluenode As XmlNode = ifionode.SelectSingleNode("values")
Dim ifiovalue As String = ifionode.InnerText
IgnoreFunctionalIDOwner = ifiovaluenode.InnerText
Catch nre As NullReferenceException
'If a tag does not exist in the xml, then the stopped-extension-dll error will be thrown.
Throw nre
Catch e As Exception
Throw e
End Try
End Sub

Then, when you wish to look for those values within those variables – just like in the last post:

Case "emailAddressPresent-ADMA-Import"
'AD attributes required: mail and msExchHomeServerName
' Default setting = False
mventry("emailAddressPresent").Value = "False"
If csentry("mail").IsPresent And csentry("msExchHomeServerName").IsPresent Then
Dim suffix() As String = Split((csentry("mail").Value), "@")
If ValidMailSuffixes.Contains(suffix(1).ToLower) Then
mventry("emailAddressPresent").Value = "True"
Else
'If a suffix from the above is not found - raise an error, so that the suffix can be added to the text file or simply sorted out - where a mistake was made.
Throw New Exception("Invalid email suffix found: " & suffix(1))
End If
ElseIf csentry("mail").IsPresent And csentry("mailNickName").IsPresent Then
'This person is a mail enabled user, maybe with an island site email address or just something else, so we want them to be able to be added to distribution lists....
mventry("emailAddressPresent").Value = "True"
End If

An Update on my Generic Array From File post

In this post: https://blog.oholics.net/a-generic-array-from-file-function-to-cope-with-inevitable-exceptions/, I documented a method of generating an array of values from a text file.

While I was happy that this method worked, I was not entirely happy with the fact that I still had some hard coded values in the code. However, the way that the function operated meant that if I took my collection of mail suffixes (20+) and added them all to the text file, then the array would be built for each and every user that passed through the dll, not too efficient!

So, I was looking for something a little more elegant. I was happy for the array to simply be defined when the dll was loaded.

Here is my solution:

At the beginning of my AD MA, I declare my dates and logging levels etc, then generate those arrays using the function. These arrays are now static and are good for processing all users without being regenerated.

Public Class MAExtensionObject_MYADMA
Implements IMASynchronization
'Date & Logginglevel variables for logging files:
Dim dtDateNowHour As Integer = Date.Now.Hour
Dim dtDateNowDay As Integer = Date.Now.Day
Dim dtDateNowMonth As Integer = Date.Now.Month
Dim dtDateNowYear As Integer = Date.Now.Year
Dim loggingLevel As Integer = 0
'
Dim ValidMailSuffixes As ArrayList = generateArrayFromFile("C:\FIMControl\ValidMailSuffixes.txt") ' Extra suffixes can be added to the text file defined here
Dim IgnoreFunctionalIDOwner As ArrayList = generateArrayFromFile("C:\FIMControl\IgnoreFunctionalIDOwner.txt") ' Extra odd functionalID owners can be added to the text file defined here
Dim IgnoreEmailAddressErrors As ArrayList = generateArrayFromFile("C:\FIMControl\IgnoreEmailAddressErrors.txt") ' Extra odd email addresses can be added to the text file defined here
'
Public Function generateArrayFromFile(ByVal file As String) As ArrayList
Dim arrayFromFile As New ArrayList()
Try
Dim reader As New System.IO.StreamReader(file)
While Not (reader.Peek() = -1)
arrayFromFile.Add(reader.ReadLine())
End While
reader.Close()
reader.Dispose()
Catch ex As IOException
arrayFromFile.Add(ex.ToString())
End Try
Return arrayFromFile
End Function

When I wish to look into the array to validate a valid email suffix for example, I go from this (as in the last post):

Case "emailAddressPresent-ADMA-Import"
'AD attributes required: mail and msExchHomeServerName
' Default setting = False
mventry("emailAddressPresent").Value = "False"
If csentry("mail").IsPresent And csentry("msExchHomeServerName").IsPresent Then
Dim suffix() As String = Split((csentry("mail").Value), "@") 'mail.Split("@")
'Valid/allowed email suffixes are defined in the following array (amend as appropriate):
Dim validMailAddresses() As String = {"blah.ac.uk", "foo.ac.uk", "bar.ac.uk", "otherorg.ac.uk"}
If (Array.IndexOf(validMailAddresses, suffix(1).ToLower) <> -1) Then
mventry("emailAddressPresent").Value = "True"
ElseIf generateArrayFromFile("C:\FIMControl\AdditionalValidMailSuffixes.txt").Contains(suffix(1).ToLower) Then ' Extra suffixes can be added to the text file defined here
mventry("emailAddressPresent").Value = "True"
Else
'If a suffix from the above is not found - raise an error, so that the suffix can be added to the array/ text file or simply sorted out - where a mistake was made.
Throw New Exception("Invalid email suffix found: " & suffix(1))
End If
ElseIf csentry("mail").IsPresent And csentry("mailNickName").IsPresent Then
'This person is a mail enabled user, maybe with an island site email address or just something else so we want them to be able to be added to distribution lists....
mventry("emailAddressPresent").Value = "True"
End If

To this:

Case "emailAddressPresent-ADMA-Import"
'AD attributes required: mail and msExchHomeServerName
' Default setting = False
mventry("emailAddressPresent").Value = "False"
If csentry("mail").IsPresent And csentry("msExchHomeServerName").IsPresent Then
Dim suffix() As String = Split((csentry("mail").Value), "@")
If ValidMailSuffixes.Contains(suffix(1).ToLower) Then
mventry("emailAddressPresent").Value = "True"
Else
'If a suffix from the above is not found - raise an error, so that the suffix can be added to the text file or simply sorted out - where a mistake was made.
Throw New Exception("Invalid email suffix found: " & suffix(1))
End If
ElseIf csentry("mail").IsPresent And csentry("mailNickName").IsPresent Then
'This person is a mail enabled user, maybe with an island site email address or just something else, so we want them to be able to be added to distribution lists....
mventry("emailAddressPresent").Value = "True"
End If

Much cleaner – plus all suffixes can now just reside in a text file.

Note that updates to the text file will only be realised if the dll is reloaded and the array is regenerated. I believe that this is after 5 minutes of inactivity and seems to hold true from testing.

Process To Email The Manager Of A Service Account When Their End Date Is Approaching

A long term goal of mine, has been to get “account requestors” to take ownership of their Service Accounts.

Attempts have been made by my predecessors to record an owner of a service account, but it has simply been done as a string attribute of the AD object. Thus, when the person leaves and the account is deleted, the service account becomes orphaned, with an reference to a long forgotten ID.

So thinking of a way to carry this out….. I am already using the email address of the owner of an administrative account to make decisions about whether the administrative account should be enabled or disabled – based on the end date of the owner – discovered by looking up the email address in the MV.

I figured that I could do something similar for those Service Accounts. I’ll be creating service accounts via the portal, the owner of the account will be assigned to the manager attribute. So, how can I get the email address of the manager into the MV as a thing that I can lookup??? I can’t do an advanced flow rule on the FIMMA, and even if I could, Manager is a reference attribute, so I can’t do it anyway… I found an article about dereferencing another attribute, that get me going down this path….. The solution is simple. Create a new attribute and binding in the portal – “ManagerEmailAddress”, then setup a workflow as follows:

GetManagerEmailAddressWF

When the account falls into scope, the managers email address is set into that new attribute – in the sync engine create a direct flow to put that into the MV (I’m using “serialNumber” – for one reason or another, that I wont go into :)).

I have on the import from AD, some code to set an MV boolean flag – “functionalID” – if the DN of the person object contains the strings found in the Service Account OU’s, thenfunctionalID = True. This attribute is pushed into the portal and is used in set definitions.

So, I’m getting there. Now I need something to set another flag in the MV that will go to the portal. this one defines if the owner of the Service Account is approaching their end date (30 days prior):It is defined on the Import from AD and populates the MV attribute “functionalID-owner-expiring”

Case "functionalID-owner-expiring-ADMA-Import"
If csentry.DN.ToString.ToLower.Contains("service") Or csentry.DN.ToString.ToLower.Contains("somethingelse") Then
If mventry("serialNumber").IsPresent Then
Dim AdminEntry() As MVEntry = Utils.FindMVEntries("mail", mventry("serialNumber").Value)
If AdminEntry.Length <> 0 Then
'We got an entry, so work with it... If the employeeEndDate of the parent account is within 30 days, set the flag - used in the portal to email the manager of the account.
If AdminEntry(0).Item("employeeEndDate").IsPresent Then
Dim EndDate As Date = DateTime.ParseExact(AdminEntry(0).Item("employeeEndDate").Value.ToString, "yyyy-MM-ddTHH:mm:ss.000", provider).Date
Dim nowTime As Date = Date.Now.Date.AddDays(30)
If EndDate <= nowTime Then
'the parent account will be disabled within 30 days, so set the expiry flag in the MV to true:
mventry("functionalID-owner-expiring").BooleanValue = True
ElseIf EndDate > nowTime Then
'the parent account is still active so set the flag to false:
mventry("functionalID-owner-expiring").BooleanValue = False
End If
End If
ElseIf AdminEntry.Length = 0 Then
If Not generateArrayFromFile("C:\FIMControl\IgnoreFunctionalIDOwner.txt").Contains(mventry("serialNumber").Value.ToLower) Then
Throw New FailedSearchException("Functional ID Owner NOT Found!" & mventry("accountName").Value.ToLower)
End If
End If
End If

Of course after initial code definition, I found another of those inevitable exceptions, so added the generateArrayFromFile function, with a reference (in txt file) to the email address that should be ignored.

Create attribute and binding in the portal for FunctionalID-owner-expiring

Setup an Export in the FIMMA for the new attribute

Create a set: FunctionalID = True and FunctionalID-owner-expiring = True.

Create notification workflow and mail template: notification to [//Target/Manager], then the set transition MPR.

I think I have it, just need to do a little testing to see that it works as expected.

I’m still a long way from the stated goal, as I still need to find “owners” for all of those accounts that have been created in the past.

A Generic Array From File Function To Cope With Inevitable Exceptions

In the last few days, I have had a few more exceptions to cope with in my FIM Config.

  1. Another new mail suffix
  2. A user who is employed by one tenant, who has that tenants email address suffix; but who is on secondment to another tenant, who have a different mail suffix. The users attributes have been changed in the HR system, so that they gain access to the stuff in the other tenant, which is controlled by automatic groups, based on attribute data!

So, I’d been thinking for a while about having a method to add exceptions without having to add them to the code directly and thus forcing a rebuild followed by full syncs. I found a nice function to read a text file to an array, this is added to the top of the dll after the lines:

Public Class MAExtensionObject_YourMA
Implements IMASynchronization

Public Function generateArrayFromFile(ByVal file As String) As ArrayList
Dim arrayFromFile As New ArrayList()
Try
Dim reader As New System.IO.StreamReader(file)
While Not (reader.Peek() = -1)
arrayFromFile.Add(reader.ReadLine())
End While
reader.Close()
reader.Dispose()
Catch ex As IOException
arrayFromFile.Add(ex.ToString())
End Try
Return arrayFromFile
End Function

So, to put this use – take my previous port regarding generating validating email addresses: https://blog.oholics.net/defining-a-unique-email-address-and-validating-mail-suffix/, at line 97 I ask “Does the suffix match?” This chunk is now as follows:

'Does the suffix match?
If mventry.Item("mail").Value.ToLower.IndexOf(mailSuffix.ToLower) = -1 Then
'the suffix does not match, so raise an error..... Unless, the email address (in lower case) is in the text file referenced below...
If Not generateArrayFromFile("C:\FIMControl\IgnoreEmailAddressErrors.txt").Contains(mventry("mail").Value.ToLower) Then
Throw New Exception("Wrong email suffix for user with email address: " & mventry("mail").Value & " , suffix should be " & mailSuffix)
End If
End If

So, the referenced file simply has the email address of the user that I don’t want to be alerted about. If the email address does not match the expected value, look in the array generated from the text file; if it in not in there either raise an error to get this fixed or investigated.

Regarding the valid mail suffixes – I posted about this already: https://blog.oholics.net/emailaddresspresent-flag-setting-and-checking-email-suffix-validity/.

I have a hardcoded list of those that are already in use in the dll, if the suffix is not found in that array, it does a lookup of the array generated from the “suffixes” text file, if it is not in there it raises an error:

Case "emailAddressPresent-ADMA-Import"
'AD attributes required: mail and msExchHomeServerName
' Default setting = False
mventry("emailAddressPresent").Value = "False"
If csentry("mail").IsPresent And csentry("msExchHomeServerName").IsPresent Then
Dim suffix() As String = Split((csentry("mail").Value), "@") 'mail.Split("@")
'Valid/allowed email suffixes are defined in the following array (amend as appropriate):
Dim validMailAddresses() As String = {"blah.ac.uk", "foo.ac.uk", "bar.ac.uk", "otherorg.ac.uk"}
If (Array.IndexOf(validMailAddresses, suffix(1).ToLower) <> -1) Then
mventry("emailAddressPresent").Value = "True"
ElseIf generateArrayFromFile("C:\FIMControl\AdditionalValidMailSuffixes.txt").Contains(suffix(1).ToLower) Then ' Extra suffixes can be added to the text file defined here
mventry("emailAddressPresent").Value = "True"
Else
'If a suffix from the above is not found - raise an error, so that the suffix can be added to the array/ text file or simply sorted out - where a mistake was made.
Throw New Exception("Invalid email suffix found: " & suffix(1))
End If
ElseIf csentry("mail").IsPresent And csentry("mailNickName").IsPresent Then
'This person is a mail enabled user, maybe with an island site email address or just something else so we want them to be able to be added to distribution lists....
mventry("emailAddressPresent").Value = "True"
End If

Console App for enumerating userAccountControl integer values

When trying something new out with FIM Development, I often see how to do it in a console app beforehand. Then once I have the process/ method worked out, I translate it into FIM code. Usually this is a very clean process and is quicker than editing the FIM code directly, then doing sync’s on individual accounts.

When I was initially looking at exporting userAccountControl values to AD, I used Jorge’s code snippet: https://jorgequestforknowledge.wordpress.com/2010/07/29/managing-the-useraccountcontrol-attribute-in-ad-by-fim/ as the basis for my code. Initially, I had some difficulty understanding the differences between the “Or’s and And’s”, so used a console app to understand what integer values the different combinations made. The list of flags can be found here: https://msdn.microsoft.com/en-us/library/windows/desktop/aa772300(v=vs.85).aspx

My userAccountControl Export code became a bit of a monster, due to the number of rules needed to match the existing configuration.

The console app is super simple – fiddle with the different flags and operators to see the different results:

Imports ActiveDs
Module Module1
Sub Main()
Dim UACValue1 As Long
Dim UACValue2 As Long
Dim UACValue3 As Long
UACValue1 = ADS_USER_FLAG.ADS_UF_NORMAL_ACCOUNT Or ADS_USER_FLAG.ADS_UF_DONT_EXPIRE_PASSWD And (Not ADS_USER_FLAG.ADS_UF_ACCOUNTDISABLE)
UACValue2 = ADS_USER_FLAG.ADS_UF_NORMAL_ACCOUNT Or ADS_USER_FLAG.ADS_UF_PASSWD_CANT_CHANGE Or ADS_USER_FLAG.ADS_UF_TRUSTED_TO_AUTHENTICATE_FOR_DELEGATION
UACValue3 = ADS_USER_FLAG.ADS_UF_NORMAL_ACCOUNT Or ADS_USER_FLAG.ADS_UF_SMARTCARD_REQUIRED Or ADS_USER_FLAG.ADS_UF_TRUSTED_FOR_DELEGATION
Console.WriteLine(UACValue1 & " " & UACValue2 & " " & UACValue3)
Console.ReadLine()
End Sub
End Module

Note, that you need to add the reference to “Active DS Type Library”, else you will get squiggles under “ADS_USER_FLAG”:

AddDSRef

emailAddressPresent Flag Setting and Checking Email Suffix Validity

In my Organisation, not all users have a mailbox, while others are just mail enabled.

In order to define XPATH filters for those people who should be allowed into a distribution list, managed by the portal, I needed to set a flag.

This boolean flag defines if they could be mailed and therefore should belong to a distribution list.

A nice add on was the fact that it allowed me to check the mail suffix of the user as part of the import process. Only those suffixes defined in the array are allowed, only suffixes that the Exchange Organisation is authoritative for. This check exists, because sometimes people just add a new suffix, or make a typo – this bit of code highlights those events.

Case "emailAddressPresent-ADMA-Import"
'AD attributes required: mail and msExchHomeServerName
' Default setting = False
mventry("emailAddressPresent").Value = "False"
If csentry("mail").IsPresent And csentry("msExchHomeServerName").IsPresent Then
Dim suffix() As String = Split((csentry("mail").Value), "@")
'Valid/allowed email suffixes are defined in the following array (amend as appropriate):
Dim validMailAddresses() As String = {"blah.ac.uk", "foo.ac.uk", "bar.ac.uk", "otherorg.ac.uk"}
If (Array.IndexOf(validMailAddresses, suffix(1).ToLower) <> -1) Then
mventry("emailAddressPresent").Value = "True"
Else
'If a suffix from the array is not found - raise an error, so that the suffix can be added to the array or simply sorted out - where a mistake was made. Rare!
Throw New Exception("Invalid email suffix found: " & suffix(1))
End If
ElseIf csentry("mail").IsPresent And csentry("mailNickName").IsPresent Then
'This person is a mail enabled user, maybe with an island site email address or just something else, so we want them to be able to be added to distribution lists....
mventry("emailAddressPresent").Value = "True"
End If

Looking for (and finding) odd ID’s

As I have said previously, the HR data that feeds FIM is out of my direct control and has had some data quality issues.

As a result, I have ended up putting some consistency checking into my code. I’ll present a few from my MVExtension here, what I’m tending to be looking for is where the user has already been provisioned, but then the reference has been deleted in HR, but no-one has told me so that I can tidy up the account in AD and FIM.

The ID’s that showed up after putting in the bit looking for FIM only references, was caused by disconnecting a table that validates historical end dates. I was assured that I would not need it anymore, because the end dates would not be randomly set to a period in the past that mean that I would never receive that update… However, this did not pan out, so I re-attached the table, but did not reset the MV object deletion rule afterwards, so I ended up with ID fragments in the portal – referenced only by ObjectID.

Again I used the Lithnet PowerShell module to clear these up. There were around 40 to do, so I just got the ObjectID’s from the job xml, put them in a text file and ran this:

Import-Module LithnetRMA
$DNs=Get-Content "C:\FIMScripts\FIMMA_DNs.txt"
ForEach ($DN in $DNs)
{
Write-Host $DN
$a=Get-Resource -ObjectType Person -AttributeName ObjectID -AttributeValue "$DN" | Remove-Resource
}
If MyADADMAConnectors = 0 And HRMAConnectors > 0 Then
'This bit applies to ID's that are coming from HR - New users - a user should be provisioned here....
'what would follow is the code to created the various attributes needed for that new user...
End If
If MyADADMAConnectors = 0 And HRMAConnectors = 0 And FIMMAConnectors > 0 Then
'This point looks for ID's that exist only in the portal - there shouldn't be any right now,
'but in the future, I want to be able to create new service accounts via the portal.
'So, as there shouldn't be any of these now, lets just raise an error:
Throw New Exception("Something odd going on here - remnant in FIM only?")
End If
If MyADADMAConnectors = 1 Then
adDN = mventry("adDN").Value 'this is constructed as part of the HR input sync
DN = ADMA.CreateDN(adDN)
'There is already an AD connector, so...
csentry = ADMA.Connectors.ByIndex(0)
'If the expected DN has changed change it in AD too
If Not csentry.DN.ToString.ToLower.Equals(DN.ToString.ToLower) Then
csentry.DN = DN
End If
'Check for real people who have been previously provided by HR (they have an EndDate),
'who have become disconnected from the HR PersonAssignment table.
'i.e. they have been deleted but the message has not been passed on! So, raise an
'error to ensure that the user is manually deleted from AD and FIM.
'We only care about those who have an EndDate, as all service accounts and some particular
'Users do not have an EndDate - for example those who have historically been added, but are no longer referenced in HR.
If HRMAPAConnectors = 0 Then
If mventry("employeeEndDate").IsPresent Then
Throw New Exception("User in AD, but not in HR - possible duplicate deleted but not informed!")
End If
End If
If MyADADMAConnectors > 1 Then
'There should never be anything except 0 or 1 MyADADMAConnectors, so raise an error.
Throw New UnexpectedDataException("Multiple MyADADMAConnectors:" + MyADADMAConnectors.ToString)
End If
End If

 

Cleaning and validating input data

The HR data source, that I currently receive person data from, has historically had data quality issues. These are much better than they were in the past, but still cause a few issues.

When I attended FIM training at OCG, I raised the issue of data cleanliness and was told in simple terms – make sure the input data is clean! If only life was so simple…..

Back to reality, I have had to add code to my Advanced Flows to deal with, clean up and validate the input data.

A nice example follows – importing Surname from HR – dealing with:

  • Just plain bad data (null as a string/ value)
  • Validation (characters that should not be present – via regex replace)
  • Clean up (removing spaces from around hyphens – double barrelled names).- there is also a bit of trimming to remove and spaces before or after the string value
  • Surname missing!

Things like this remind me of why “Codeless Provisioning” was something I fought to get working (for too long), but ultimately had to abandon in favour of using code for almost everything. Doing so has been a real panacea for all of the rules and other funnies that I have had to accommodate.

Note: I made a little edit – I was not checking for the presence of AccountName before raising errors – should that attribute have been missing (highly unlikely, but not unknown to occur), that would have raised an error in itself. The edited code is a little more robust!

Case "sn-HRMA-Import"
'HR attributes required are:Surname, AccountName
'Ensure that this attribute has spaced Hypens corrected
Dim surnamelogFileName As String = dtDateNowDay & "-" & dtDateNowMonth & "-" & dtDateNowYear & "_HRMA_Surname.log"
Logging.SetupLogFile(surnamelogFileName, loggingLevel)
If csentry("Surname").IsPresent Then
If csentry("Surname").Value.ToLower = "null" Then
mventry("sn").Delete()
If csentry("AccountName").IsPresent Then
Throw New Exception("Error in Surname (null) for FedID: " & csentry("AccountName").Value)
Else
Throw New Exception("Error in Surname (null) for PID: " & csentry("PID").Value) 'PID is the anchor - if this is missing we have more serious problems
End If
Else
Dim tString As String = Regex.Replace(csentry("Surname").Value, "[^a-zA-ZÀÈÌÒÙàèìòùÁÉÍÓÚÝáéíóúýÂÊÎÔÛâêîôûÃÑÕãñõÄËÏÖÜŸäëïöüÿ_\-\'\ ]", "")
'tString = tString.Replace(" - ", "-")
If tString <> csentry("Surname").Value Then
If csentry("AccountName").IsPresent Then
Throw New Exception("Error in Last Name format for FedID: " & csentry("AccountName").Value & ", Firstname: " & csentry("Surname").Value)
Else
Throw New Exception("Error in Last Name format for PID: " & csentry("PID").Value & ", Firstname: " & csentry("Surname").Value) 'PID is the anchor - if this is missing we have more serious problems
End If
Else
mventry("sn").Value = Replace(csentry("Surname").Value.Trim, " - ", "-")
End If
End If
Else
If csentry("AccountName").IsPresent Then
Logging.Log("Surname not Present for: " & csentry("AccountName").Value, True, 0)
Else
Logging.Log("Surname not Present for: " & csentry("PID").Value, True, 0) 'PID is the anchor - if this is missing we have more serious problems
End If
End If