Sanitizing employeeEndDate

The source of my “HR” data does not always return consistent data.

A classic example of this is employeeEndDate, where the time value returned by the HR DB is often 00:00:00, but could be anything else!

The result of this is that if the user disablement rule says – use date/time of now – compare it to the employeeEndDate; if the end date has passed, then disable the account. Thus if the time value is inconsistent, you will get inconsistent user disablement’s occurring

So, to fix this in code, I did this – on the import from HR:

Now all employeeEndDate times are consistent “T23:59:59.000”

FIM 2010 Automatic Group Management via the portal

As part of the FIM implementation within my organisation, I needed to replicate the management of automatic security and distribution groups.

Now this is ugly, not nice/ ideal etc, but to get it done – I need to avoid any changes to the Status Quo – my org does not like change! I can work on making things better later in time.

The current process has an interesting concept of what automatic means.

In that, where normally, an attribute e.g. representing the Finance dept. might be used to identify members of a group; the filter identifying members might use organisational attributes; there might also be users who do not fit within that structure who need to be added. Or there might be people who should be within a group because their organisational attributes match the filter, but for whatever reason, they do not want to be a member of the group.

So essentially, the current filter process might provide organisational attributes to populate the group, but there may also be additional explicit includes and excludes. All automatic groups also remove users who have been disabled.

Note that the organisation structure is not hierarchical

When I initially started looking at how to implement this in FIM, I thought that is was simply not possible. I looked at using http://www.wapshere.com/missmiis/who-needs-group-populator-when-you-have-multivalue-tables, but thought that this was too complicated for what I wanted to achieve – initial population would be troublesome, plus another interface would be required. I considered making the original automatic groups into manual groups then creating new (truly) automatic attribute based groups, and then nesting those within the manual group – but this would involve “change”, so this is now the long term aim, but will not be implemented immediately. Note that this method does not handle the explicit removes well either, they would be to be excluded within the filter.

So, the “no change” method is going to be used. Using the GUI to define the filters is just annoying and time consuming, so I took the current filters and translated them into XPath filters. In fact some of the more complicated filers cannot be rendered by the GUI, but pressing the show members button does return the correct number of users. Note that I’m using this: http://social.technet.microsoft.com/wiki/contents/articles/26674.fim-group-membership-export-gui-with-powershell.aspx to export FIM group membership to compare to production.

An example of a “complicated” filter and the resultant GUI:

/Person[(EmployeeStatus = ‘Enabled’) and (emailAddressPresent= ‘True’) and ((Site = ‘NORTH’) or (Site = ‘SOUTH’)) and (Department = ‘FIN’) and (not(AccountName = ‘User1’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User2’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User3’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User4’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User5’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User6’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User7’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User8’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User9’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User10’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User11’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User12’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User13’))]

ComplicatedFilter

 

The filter above is placed within the Filter attribute of the group. Accessible via Advanced view, Extended Attributes: <Filter xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” xmlns:xsd=”http://www.w3.org/2001/XMLSchema” Dialect=”http://schemas.microsoft.com/2006/11/XPathFilterDialect” xmlns=”http://schemas.xmlsoap.org/ws/2004/09/enumeration”>Filter goes here</Filter>>

If I break down this filter:

The main “Automatic” bit: (EmployeeStatus = ‘Enabled’) and (emailAddressPresent= ‘True’) and ((Site = ‘NORTH’) or (Site = ‘SOUTH’)) and (Department = ‘FIN’) = Enabled accounts, who have an email address (this is a distribution list), who are on Site NORTH or SOUTH and whose Department attribute is “FIN”  Note that EmployeeStatus, emailAddressPresent and Site are custom attributes within the MV and Portal, that are populated via a classic import rules from HR/AD.

The explicit add and removes: (not(AccountName = ‘User1’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User2’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User3’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User4’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User5’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User6’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User7’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User8’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User9’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User10’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User11’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User12’)) or ((EmployeeStatus = ‘Enabled’) and (AccountName = ‘User13’)) = Remove/ disallow User1 (explicit remove), then add (explicit add) each of those other accounts, only if they are enabled (disabled accounts are removed from groups).

Nice eh? :/

A little note about (emailAddressPresent= ‘True’), this MV attribute is populated on the import from AD. It is defined by the presence of a mail address that matches those that are valid for the organisation and that the persons msExchHomeServerName is present. It is then exported to the Portal with the same name, for use in these filters:

 

Each of the filters are stored in a CSV file with the corresponding Display Name – I’ll use this PS script to import those filters into the respective group: https://social.technet.microsoft.com/Forums/en-US/63dcd915-c92d-4fa2-8183-45e2db601693/using-powershell-to-turn-static-groups-into-dynamic-groups?forum=ilm2

(10-9-15) I had a few issues with the script referred to – namely the second to last line, which had “$importObject | Import-FIMConfig$undone.Count” , “$undone.Count” needs to (at least) be on a new line – it doesn’t seem to do anything anyway, so may just be removed….

I added a few more references to the filters, group names etc., as I discovered, during debugging, that my initial input file referred to the samAccountName and Filter, rather than the displayName plus Filter. Additionally, some groups had gone plus others renamed since I did the initial filter file – before the school summer holidays…, so the script highlights those.

I got some errors with certain groups:

Csdrlo

  • Error suggested that the filter had not been applied, but it had. Further investigation shows that the order of the filter was incorrect – this and this and (that or that) and this…. should have been this and this and this and that or that.

DLES

  • Odd, did the entry via the portal – added OK..

Missing_SquareBracket

  • Missing “]” from the end of the filter

Validating each group membership will take some time…. that’s the job I’ll start tomorrow…..

(11-9-15) OK, so been validating today….didn’t really fancy the idea of doing this group by group, so dug out some old scripts and put together some new ones to make my life easier and consistent….

First – one of my old VBS scripts to get the desired group members from AD:

Then another to get the members of those groups that I added the filters for (hacked from the LAZY GUYS OF FIM script referenced above):

Still waiting for the second script to output some of the larger groups…. once complete, I’ll use WinMerge to compare the resulting files. If I constructed all of the XPath filters correctly, they should all match!

*** Note – new faster script provided here: https://blog.oholics.net/using-ryan-newingtons-lithnet-fim-powershell-module-to-improve-performance-of-group-membership-computedmember-reports/ ***

(14-09-15) – So validating these groups is more troublesome than I initially thought..

  1. To compare in WinMerge, the layout of the files needs to be the same, so the records within need to be sorted – PowerShell makes this easy:

2. I have significant disparity between production AD and FIM in terms of users present in each system and the rules defining who should be a member of a group. I have a users deletion workflow in FIM that runs continuously, while in production, a task is run manually at regular intervals to delete users who are in scope of deletion – these are out of step. Additionally, some members of groups in production are disabled, some of these are the same as those that are no longer present in FIM – the workflow has already deleted them. The simple solution is to remove all disabled users from all automatic groups in production – this is something that I have been wanting to do for a while anyway…

3. Some of my filters were incorrect – minor changes.

(16-09-15) – So I have now removed all disabled users from Automatic groups – the WinMerge results are much nicer now – only 46 files are different. Some results are unusual, where a user may be a member in Prod, but not FIM or vice versa – I now need to work through these to see what needs to be sorted.

On a side note….. a colleague at another university remarked that he wished to replicate the filtering method that he already had in place for group management – namely LDAP queries. Now although the FIM portal can’t natively do a direct LDAP query, you can always populate MV and Portal attributes with the data that you wish to query upon. For example within my org the DN of users is not a simple matter, so the DN is calculated by a classic import flow rule from HR. Thus, the DN is already stored in the MV. I could choose to export that data into a new Portal attribute and then make that part of the group filter.

This is one of the things that I really like about FIM – the ability to make it do what you want, with a little creative thinking!