One of the things that I had to bear in mind for my FIM implementation was that the current legacy code provides email notifications of changes made. Due to the nature/ structure of the organisation, the different groups only want to know about changes to their own users.
So, when initially thinking about what people are currently informed about, and then the number of MPR’s, Workflows, Sets and Mail templates required to duplicate that functionality – I thought “no way!” It would be unmanageable. In the end and after discussion with the recipients of those mails, I got to a manageable number of things that those people really wanted to know about. These are:
A new user starting (a real new person), or a person who has moved horizontally within the organisation – they have moved INTO that department.
A user in their department becoming disabled
A user in their department becoming deleted
A user moving OUT of their department
So, I’m down to 4 things, but I have 9 different groups to inform.- So I have 36 Templates, 36 Workflows, 27 Sets and 36 MPR’s to define…..Even if I had a minion, I wouldn’t make them do that manually….
So, during Ryan’s presentation to the FIM User Group, I noted that he had introduced the ability to import these object by providing an XML file that defined them.
If all looks OK, run the command again without the Preview & Verbose switches.
My initial runs ran into errors due to the way that the employeeEndDate plus 180 days filter was being passed to the FIM service. Ryan has rapidly fixed and produced new versions to resolve these problems.
The great thing about this method, is that by using xmlref pointers, everything just joins up, resulting in a consistent set of objects that reference each other.
Note that the line:
Is used to provide a reference to the mail recipient, more recipients could be added, using the same structure, but a different id. Other pre-existing object references could also be added as appropriate.
So to summarise what is defined in the XML file:
4 Email templates – Add, Disable, Delete & Move – each with a corresponding and each slightly different html file
4 Workflows – Add, Disable, Delete & Move – each with a corresponding and each slightly different XOML file
3 Sets – User in OU=Blah, User in OU=Blah AND whose Status=Disabled, User in OU=Blah whose end date has passed, plus 180 days
4 MPR’s – Add, Disable, Delete & Move – all are transition in, except for the Move MPR, which is a transition out.
A few weeks ago, I created a new attribute in the portal called adOU. This attribute contained the Active Directory OU of a user and was defined in code from the HR database input – that’s where the OU definitions are defined…..
I planned to use the attribute to define Sets containing the users whose accounts resided in those OU’s. So, after creating and populating that attribute, I started looking at the Set’s Workflows, MPR’s and Email templates that I’d require to make use of that new attribute – these items were to be used to send email notifications.
When I got to the point of creating the Sets, I noted that I could not choose the adOU attribute as a criteria. I checked the relevant MPR’s, Admin filter etc., all looked OK. Then I realised that when I created the new attribute, I had configured it as an un-indexed string. Thus, this was the reason that I could not use it in my Set definition.
So, off I go trying to delete the binding and attribute, always fails – after doing the few normal bits that I remember that I need to do beforehand – I have been here a few times before!
So, its about time that I documented all of the steps, to save me and maybe others the pain in the future.
Delete the attribute mapping in the FIMMA:
2. Un-tick the attribute in the attribute chooser in the FIMMA
.
3. Remove any references to the attribute in any workflows, MPR’s, Sets, mail templates etc..Run an Export policy script like the following, then look for references (e.g. “[//Target/adOU]”) to the attribute in the resulting xml file:
4. Clear the attribute from all users. Previously, I have used an MPR and workflow to to this, but in this case, I found it unreliable. So turned to the LithnetRMA PowerShell module (https://lithnetrma.codeplex.com/):
5. Finally, delete the binding and then the attribute – they should go now – if not something has been missed.
6. Refresh the FIMMA schema
Now in my case, I then re-created the attribute – as an Indexed String, and then the Binding. Then refreshed the FIMMA Schema again, added the attribute to the FIMMA picker, recreated the FIMMA flow, reset the MPR’s (to allow the attribute to be managed) and Admin filter permissions. Then run the export to get the data back into the portal. The Set criteria now contains the option to use adOU.
While this script did the job, it took a rather long time to process, it was an overnight job. If it had failed overnight, I had to wait another day before doing the comparison against production auto groups.
In terms of performance, the Lithnet PowerShell module is much faster. My standard script took ~7 hours to run, while the Lithnet script only took 3 hours!
I can see that I’m going to get a lot more use out of the module in terms of making changes as well as getting information from the portal. Thanks Ryan!
So, as per the comments, Ryan’s suggestion really improves performance of the script! Here is the new script for reference:
I also need to detect where those addresses might need to be changed. The AD and Exchange infrastructure supports a number of different tenant organisations, each with their own needs. There is regular horizontal movement of users between these organisations.
So I have added to my code to define a unique email address – the additional content starts at Line 72 – statement “If mventry.Item(“mail”).IsPresent Then“. Note that attributes are set on the import from HR to define who is entitled to an MBX and those who should just be mail enabled. The current code just logs out those things for action, but I have also included making these events throw an exception to the Sync Engine.
It was interesting to see just how many people are not really entitled to a mailbox, but who have one anyway!
This resulting backup files from this script allows me to do a bare metal restore of a virtual domain controller within ~30 minutes. This assumes complete meltdown of your domain – catastrophic failure, schema issues, compromise, etc. – where you need to restore it from scratch – this is a last resort action!. Once restoration is complete move onto the rest of the recovery process. Of course this script will also do nicely for just backing up all critical drives and registry on any other server or client!
Another thing that I messed about with while “making FIM fit”…
I wanted to manage certain non-real people accounts. I did not need them to be present in the portal, as the code would manage the accounts alone. Therefore I needed to project them to the MV for the code to take effect.
To do this, I created a copy of the person object, trimmed a few unnecessary attributes – the new object was called “functionalID”
Now, after bringing those ID’s in as functionalID’s, I realised that some of the administrative accounts needed access to the portal to manage the real user accounts. So, I exported all functionalID’s to the portal, they were not visible, as the portal did not know what to do with them. After fiddling about, extending the portal schema and creating MPR’s etc. I was able to view and manage those ID’s. However, I wanted the administrative ID’s to be able to log into the portal to do their day to day work.
Now annoyingly I can’t find the link, but to summarise it stated that only person type objects could login to the portal! Thus, my copy of person type would never be able to login! Arrhhh!
So, a little rethink….. treat all functionalID’s as functional’s except those that were in the OU containing the Administrative ID’s that I wanted to have access to the portal – treat them as person objects – as shown below:
However, after all this I decided that it was far simpler to just treat all ID’s as person objects, and then flag those that were functional’s (with a boolean flag). That flag is then used to manage the accounts in code and can be used to exclude those accounts from view in the portal. Flag setting is done on the import from AD, using specific strings within the DN as the criteria:
While implementing FIM, I was looking for ways to filter out old records from the HR database. Note that this code is no longer used (the SQL view providing the data does it for me), but it provides a nice simple example of how to use the FilterForDisconnection function.
I used the following bit of code in the HR MA, to filter out all users whose end date had passed plus 190 days. Those that matched those criteria were made disconnectors.
The source of my “HR” data does not always return consistent data.
A classic example of this is employeeEndDate, where the time value returned by the HR DB is often 00:00:00, but could be anything else!
The result of this is that if the user disablement rule says – use date/time of now – compare it to the employeeEndDate; if the end date has passed, then disable the account. Thus if the time value is inconsistent, you will get inconsistent user disablement’s occurring
So, to fix this in code, I did this – on the import from HR:
Now all employeeEndDate times are consistent “T23:59:59.000”
As part of the FIM implementation within my organisation, I needed to replicate the management of automatic security and distribution groups.
Now this is ugly, not nice/ ideal etc, but to get it done – I need to avoid any changes to the Status Quo – my org does not like change! I can work on making things better later in time.
The current process has an interesting concept of what automatic means.
In that, where normally, an attribute e.g. representing the Finance dept. might be used to identify members of a group; the filter identifying members might use organisational attributes; there might also be users who do not fit within that structure who need to be added. Or there might be people who should be within a group because their organisational attributes match the filter, but for whatever reason, they do not want to be a member of the group.
So essentially, the current filter process might provide organisational attributes to populate the group, but there may also be additional explicit includes and excludes. All automatic groups also remove users who have been disabled.
Note that the organisation structure is not hierarchical
When I initially started looking at how to implement this in FIM, I thought that is was simply not possible. I looked at using http://www.wapshere.com/missmiis/who-needs-group-populator-when-you-have-multivalue-tables, but thought that this was too complicated for what I wanted to achieve – initial population would be troublesome, plus another interface would be required. I considered making the original automatic groups into manual groups then creating new (truly) automatic attribute based groups, and then nesting those within the manual group – but this would involve “change”, so this is now the long term aim, but will not be implemented immediately. Note that this method does not handle the explicit removes well either, they would be to be excluded within the filter.
So, the “no change” method is going to be used. Using the GUI to define the filters is just annoying and time consuming, so I took the current filters and translated them into XPath filters. In fact some of the more complicated filers cannot be rendered by the GUI, but pressing the show members button does return the correct number of users. Note that I’m using this: http://social.technet.microsoft.com/wiki/contents/articles/26674.fim-group-membership-export-gui-with-powershell.aspx to export FIM group membership to compare to production.
An example of a “complicated” filter and the resultant GUI:
/Person[(EmployeeStatus = ‘Enabled’) and (emailAddressPresent= ‘True’) and ((Site = ‘NORTH’) or (Site = ‘SOUTH’)) and (Department = ‘FIN’) and (not(AccountName = ‘User1’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User2’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User3’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User4’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User5’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User6’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User7’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User8’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User9’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User10’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User11’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User12’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User13’))]
The filter above is placed within the Filter attribute of the group. Accessible via Advanced view, Extended Attributes: <Filter xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” xmlns:xsd=”http://www.w3.org/2001/XMLSchema” Dialect=”http://schemas.microsoft.com/2006/11/XPathFilterDialect” xmlns=”http://schemas.xmlsoap.org/ws/2004/09/enumeration”>Filter goes here</Filter>>
If I break down this filter:
The main “Automatic” bit: (EmployeeStatus = ‘Enabled’) and (emailAddressPresent= ‘True’) and ((Site = ‘NORTH’) or (Site = ‘SOUTH’)) and (Department = ‘FIN’) = Enabled accounts, who have an email address (this is a distribution list), who are on Site NORTH or SOUTH and whose Department attribute is “FIN” Note that EmployeeStatus, emailAddressPresent and Site are custom attributes within the MV and Portal, that are populated via a classic import rules from HR/AD.
The explicit add and removes: (not(AccountName = ‘User1’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User2’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User3’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User4’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User5’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User6’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User7’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User8’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User9’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User10’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User11’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User12’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User13’)) = Remove/ disallow User1 (explicit remove), then add (explicit add) each of those other accounts, only if they are enabled (disabled accounts are removed from groups).
Nice eh? :/
A little note about (emailAddressPresent= ‘True’), this MV attribute is populated on the import from AD. It is defined by the presence of a mail address that matches those that are valid for the organisation and that the persons msExchHomeServerName is present. It is then exported to the Portal with the same name, for use in these filters:
(10-9-15) I had a few issues with the script referred to – namely the second to last line, which had “$importObject | Import-FIMConfig$undone.Count” , “$undone.Count” needs to (at least) be on a new line – it doesn’t seem to do anything anyway, so may just be removed….
I added a few more references to the filters, group names etc., as I discovered, during debugging, that my initial input file referred to the samAccountName and Filter, rather than the displayName plus Filter. Additionally, some groups had gone plus others renamed since I did the initial filter file – before the school summer holidays…, so the script highlights those.
I got some errors with certain groups:
Error suggested that the filter had not been applied, but it had. Further investigation shows that the order of the filter was incorrect – this and this and (that or that) and this…. should have been this and this and this and that or that.
Odd, did the entry via the portal – added OK..
Missing “]” from the end of the filter
Validating each group membership will take some time…. that’s the job I’ll start tomorrow…..
(11-9-15) OK, so been validating today….didn’t really fancy the idea of doing this group by group, so dug out some old scripts and put together some new ones to make my life easier and consistent….
First – one of my old VBS scripts to get the desired group members from AD:
Then another to get the members of those groups that I added the filters for (hacked from the LAZY GUYS OF FIM script referenced above):
Still waiting for the second script to output some of the larger groups…. once complete, I’ll use WinMerge to compare the resulting files. If I constructed all of the XPath filters correctly, they should all match!
(14-09-15) – So validating these groups is more troublesome than I initially thought..
To compare in WinMerge, the layout of the files needs to be the same, so the records within need to be sorted – PowerShell makes this easy:
2. I have significant disparity between production AD and FIM in terms of users present in each system and the rules defining who should be a member of a group. I have a users deletion workflow in FIM that runs continuously, while in production, a task is run manually at regular intervals to delete users who are in scope of deletion – these are out of step. Additionally, some members of groups in production are disabled, some of these are the same as those that are no longer present in FIM – the workflow has already deleted them. The simple solution is to remove all disabled users from all automatic groups in production – this is something that I have been wanting to do for a while anyway…
3. Some of my filters were incorrect – minor changes.
(16-09-15) – So I have now removed all disabled users from Automatic groups – the WinMerge results are much nicer now – only 46 files are different. Some results are unusual, where a user may be a member in Prod, but not FIM or vice versa – I now need to work through these to see what needs to be sorted.
On a side note….. a colleague at another university remarked that he wished to replicate the filtering method that he already had in place for group management – namely LDAP queries. Now although the FIM portal can’t natively do a direct LDAP query, you can always populate MV and Portal attributes with the data that you wish to query upon. For example within my org the DN of users is not a simple matter, so the DN is calculated by a classic import flow rule from HR. Thus, the DN is already stored in the MV. I could choose to export that data into a new Portal attribute and then make that part of the group filter.
This is one of the things that I really like about FIM – the ability to make it do what you want, with a little creative thinking!
While looking at how we were going to define an email address, there were a couple of options.
Many accounts already have a mailbox, so I just want to provide an email address and the other required attributes to those who fall into scope and did not have a mailbox already.
Note that the compulsory attributes required for a basic mailbox are: homeMDB, msExchHomeServerName & mailNickname. By adding the mail attribute, I am defining the primary SMTP address that the person will get. See here for more details: https://technet.microsoft.com/en-us/magazine/ff472471.aspx
However, there was some fear that if enabled it might have some undesirable effects on those existing mailboxes. So, I went back to code to do the work. The use of Utils.FindMVEntries is not necessarily expensive in this case as the number of people falling into scope at any one time should be low and the likelihood of the first choice not being unique is also relatively low.
Because my organisation is multi-tenant and multi site, there are a number of rules defined to get to the correct suffix and then a valid prefix/ suffix pair.
The MV attribute “provisionExchangeMailbox” is defined in the MV as the person is sync’ed from “HR” via an advanced flow rule. Here is the code:
