Clean-Up Orphans and Refine Correlation Rules

Correlation rules may be very simple. A unique ID on an IAM solution should match the unique ID (or combinations of attributes) to form a one-to-one (1:1) relationship to the identity on a managed endpoint/application.

Most sites that had the opportunity have started using GUID/UUID values for the correlation ID on the IAM solutions and if the endpoint/application allows it, the same GUID/UUID on an open field, that likely is not the same as the login ID field.

Example below using a GUID/UUID format as the primary identifier with the IAM solution and the endpoint/application of an Active Directory domain.

We may also have many different correlation rules or primary/secondary correlation for every application/endpoint. Until the correlation is correct we have the likelihood of an incorrect correlation or default correlation.

If we wish to remove an incorrect correlation, this may be done manually to remove or re-attach the correct entries. However, this would not address future correlation processes if the rules are not updated.

Example of removing a correlation from the orphan ID “[default user]”

To assist with refinement of correlation rules, a feedback process/script may have value.

The below script demonstrates using OS ldapsearch/ldapdelete processes with the CA Identity Manager Provisioning Tier (TCP 20389/20390) a feedback process to cleanup the Orphans IDs under “[default user]”

The script will query all “inclusions” where an endpoint account has been incorrectly associated with the Global user “[default user]” and return a count of these records. The process will capture the dn values of these inclusions records, and then feed them to the Open LDAP ldapdelete process to have them removed. Since we are using the IMPS service (TCP 20389/20390) we are still allowing the solution to maintain referential integrity during the clean-up process.

After the deletion are complete, we will re-initialize a new E&C (explore & correlate) process using any new Correlation Rules that may have been added. It is this opportunity that an administrator may wish to adjust their own correlation rules; and then re-execute the script. If the correlation rules do not match, then the prior correlations will return to the “[default user]”.

#!/bin/bash
#####################################################################################################################
#
# Name: Clean Up [default user]
#
# Goal:  Script to clean up [default_user] correlations to allow for better orphan or rogue account identification
#  - Ensure that IMPS Service TCP 20389/20390 is used to maintain referential integrity of the inclusions entries
#    during delete operations.
#
# Ref:  CA IM r14.x solution & OS ldapsearch/ldapdelete
#
# A. Baugher, ANA, 04/2020
#
#####################################################################################################################
# set -xv
DATETZ=$(date -d "1970-01-01 00:00:00 `date +'%s'` seconds"  +'%Y-%m-%dT%H:%M:%S.%3NZ')
IMPSHOST=`hostname`
IMPSPORT=20390
IMPSUSERDN='eTGlobalUserName=etaadmin,eTGlobalUserContainerName=Global Users,eTNamespaceName=CommonObjects,dc=im,dc=eta'
# Use pwd file to avoid clear text passwords in script
# echo -n CLEAR_TEXT_PASSWORD > .imps.pwd
IMPSPWD=`cat .imps.pwd`
#####################################################################################################################
BASE_DN='eTSuperiorClass=eTGlobalUser,eTInclusionContainerName=Inclusions,eTNamespaceName=CommonObjects,dc=im,dc=eta'
SUP_DN_ENTRY='eTGlobalUserName=[default user],eTGlobalUserContainerName=Global Users,eTNamespaceName=CommonObjects,dc=im'
FILTER="(&(objectClass=eTInclusionObject)(eTSuperiorClassEntry=$SUP_DN_ENTRY))"
SEARCH=sub
ATTRIBUTES='dn eTInclusionID'
EXCLUDE="  -e ^$ "
#SIZE=" -z 10"
SIZE=" -z 0"
FILENAME=default_user_guid.txt
rm -rf $FILENAME
echo ""
echo "#################################################################################################"
echo "#### How many EA (endpoint accounts) are correlated to the Global User [default user] ###########"
echo "#################################################################################################"
LDAPTLS_REQCERT=never ldapsearch $SIZE -LLL -H ldaps://$IMPSHOST:$IMPSPORT -D "$IMPSUSERDN" -w $IMPSPWD -b $BASE_DN -s $SEARCH "$FILTER" $ATTRIBUTES | perl -p00e 's/\r?\n //g' | grep -v $EXCLUDE | awk -F": " '{print $2}'  | grep eTInclusionID | wc -l
echo "#################################################################################################"
LDAPTLS_REQCERT=never ldapsearch $SIZE -LLL -H ldaps://$IMPSHOST:$IMPSPORT -D "$IMPSUSERDN" -w $IMPSPWD -b $BASE_DN -s $SEARCH "$FILTER" $ATTRIBUTES | perl -p00e 's/\r?\n //g' | grep -v $EXCLUDE | awk -F": " '{print $2}'  | grep eTInclusionID > tmp_file
echo "LDAPTLS_REQCERT=never ldapsearch $SIZE -LLL -H ldaps://$IMPSHOST:$IMPSPORT -D '$IMPSUSERDN' -y ./.imps.pwd -b '$BASE_DN' -s $SEARCH '$FILTER' $ATTRIBUTES | perl -p00e 's/\r?\n //g' | grep -v $EXCLUDE | awk -F': ' '{print \$2}' | grep eTInclusionID "
uniq -i tmp_file > $FILENAME
echo "#################################################################################################"
echo "# of unique Endpoint Accounts that are Correlated to [default user] matching query filter : "`cat $FILENAME | wc -l`
rm -rf tmp_file
echo "#################################################################################################"



echo ""
echo "####################################################################################################################"
echo "#### Remove `cat $FILENAME | wc -l` EA (endpoint accounts) that are correlated to the Global User [default user] "
echo "####################################################################################################################"
LDAPTLS_REQCERT=never ldapdelete -v -c -H ldaps://$IMPSHOST:$IMPSPORT -D "$IMPSUSERDN" -w $IMPSPWD -f $FILENAME
echo ""
echo "#################################################################################################"
echo "#### How many EA (endpoint accounts) are correlated to the Global User [default user] ###########"
echo "#################################################################################################"
LDAPTLS_REQCERT=never ldapsearch $SIZE -LLL -H ldaps://$IMPSHOST:$IMPSPORT -D "$IMPSUSERDN" -w $IMPSPWD -b $BASE_DN -s $SEARCH "$FILTER" $ATTRIBUTES | perl -p00e 's/\r?\n //g' | grep -v $EXCLUDE | awk -F": " '{print $2}'  | grep eTInclusionID | wc -l
echo "#################################################################################################"


echo ""
echo "#################################################################################################"
echo "#### Re-explore & correlate to update Global User [default user] orphan bucket."
echo "#################################################################################################"
echo ""
IMPSADSBASEDN="eTADSDirectoryName=dc2016.exchange.lab,eTNamespaceName=ActiveDirectory,dc=im,dc=eta"
LDAPTLS_REQCERT=never ldapsearch $SIZE -LLL -H ldaps://$IMPSHOST:$IMPSPORT -D "$IMPSUSERDN" -w $IMPSPWD -b "$IMPSADSBASEDN" -s sub "(objectClass=*)" eTExploreUpdateEtrust
LDAPTLS_REQCERT=never ldapsearch $SIZE -LLL -H ldaps://$IMPSHOST:$IMPSPORT -D "$IMPSUSERDN" -w $IMPSPWD -b "$IMPSADSBASEDN" -s sub "(objectClass=*)" eTExploreCorrelateUsers

IMPSADSBASEDN="eTADSDirectoryName=dc2012.exchange2012.lab,eTNamespaceName=ActiveDirectory,dc=im,dc=eta"
LDAPTLS_REQCERT=never ldapsearch $SIZE -LLL -H ldaps://$IMPSHOST:$IMPSPORT -D "$IMPSUSERDN" -w $IMPSPWD -b "$IMPSADSBASEDN" -s sub "(objectClass=*)" eTExploreUpdateEtrust
LDAPTLS_REQCERT=never ldapsearch $SIZE -LLL -H ldaps://$IMPSHOST:$IMPSPORT -D "$IMPSUSERDN" -w $IMPSPWD -b "$IMPSADSBASEDN" -s sub "(objectClass=*)" eTExploreCorrelateUsers


echo ""
echo "#################################################################################################"
echo "#### How many EA (endpoint accounts) are correlated to the Global User [default user] ###########"
echo "#################################################################################################"
LDAPTLS_REQCERT=never ldapsearch $SIZE -LLL -H ldaps://$IMPSHOST:$IMPSPORT -D "$IMPSUSERDN" -w $IMPSPWD -b $BASE_DN -s $SEARCH "$FILTER" $ATTRIBUTES | perl -p00e 's/\r?\n //g' | grep -v $EXCLUDE | awk -F": " '{print $2}'  | grep eTInclusionID | wc -l
echo "#################################################################################################"
echo ""

Example of the output of the script (with 1000’s of lines remove for clarity). Includes E&C to two (2) ADS endpoints, where > 2000 identities will default correlation to the orphan Global User “[default user]”.

#################################################################################################
#### How many EA (endpoint accounts) are correlated to the Global User [default user] ###########
#################################################################################################
2184
#################################################################################################
LDAPTLS_REQCERT=never ldapsearch  -z 0 -LLL -H ldaps://vapp0001:20390 -D 'eTGlobalUserName=etaadmin,eTGlobalUserContainerName=Global Users,eTNamespaceName=CommonObjects,dc=im,dc=eta' -y ./.imps.pwd -b 'eTSuperiorClass=eTGlobalUser,eTInclusionContainerName=Inclusions,eTNamespaceName=CommonObjects,dc=im,dc=eta' -s sub '(&(objectClass=eTInclusionObject)(eTSuperiorClassEntry=eTGlobalUserName=[default user],eTGlobalUserContainerName=Global Users,eTNamespaceName=CommonObjects,dc=im))' dn eTInclusionID | perl -p00e 's/\r?\n //g' | grep -v   -e ^$  | awk -F': ' '{print $2}' | grep eTInclusionID
#################################################################################################
# of unique Endpoint Accounts that are Correlated to [default user] matching query filter : 2184
#################################################################################################

####################################################################################################################
#### Remove 2184 EA (endpoint accounts) that are correlated to the Global User [default user]
####################################################################################################################
ldap_initialize( ldaps://vapp0001:20390/??base )
deleting entry "eTInclusionID=df104a69-e746-49df-9a61-51e8c20038d0@67d6bf2c-1104-1039-96c4-ef7605d11763,eTSubordinateClass=eTADSAccount,eTSuperiorClass=eTGlobalUser,eTInclusionContainerName=Inclusions,eTNamespaceName=CommonObjects,dc=im,dc=eta"
Delete Result: Success (0)
Additional info: :ETA_S_0035<MGU>, Inclusion between Active Dir. Account 'firstname6 mi. lastname6' and Global User '[default user]' deleted successfully
deleting entry "eTInclusionID=df104a69-e746-49df-9a61-51e8c20038d0@65e02962-00bd-1039-830f-ae134a0f7638,eTSubordinateClass=eTADSAccount,eTSuperiorClass=eTGlobalUser,eTInclusionContainerName=Inclusions,eTNamespaceName=CommonObjects,dc=im,dc=eta"
Delete Result: Success (0)
Additional info: :ETA_S_0035<MGU>, Inclusion between Active Dir. Account 'firstname0002 lastname0002' and Global User '[default user]' deleted successfully

[Deleted > 5000 similar rows ]

deleting entry "eTInclusionID=df104a69-e746-49df-9a61-51e8c20038d0@ce05d098-1b32-1039-85ec-b0629a56714f,eTSubordinateClass=eTADSAccount,eTSuperiorClass=eTGlobalUser,eTInclusionContainerName=Inclusions,eTNamespaceName=CommonObjects,dc=im,dc=eta"
Delete Result: Success (0)
Additional info: :ETA_S_0035<MGU>, Inclusion between Active Dir. Account 'ffffff' and Global User '[default user]' deleted successfully
deleting entry "eTInclusionID=df104a69-e746-49df-9a61-51e8c20038d0@75a62f60-1b32-1039-85ea-b0629a56714f,eTSubordinateClass=eTADSAccount,eTSuperiorClass=eTGlobalUser,eTInclusionContainerName=Inclusions,eTNamespaceName=CommonObjects,dc=im,dc=eta"
Delete Result: Success (0)
Additional info: :ETA_S_0035<MGU>, Inclusion between Active Dir. Account 'eeeee' and Global User '[default user]' deleted successfully

#################################################################################################
#### How many EA (endpoint accounts) are correlated to the Global User [default user] ###########
#################################################################################################
0
#################################################################################################

#################################################################################################
#### Re-explore & correlate to update Global User [default user] orphan bucket.
#################################################################################################

Additional information: :ETA_S_0023<EDI>, Active Directory Endpoint 'dc2016.exchange.lab' exploration successful: (objects added: 0, deleted: 0, updated: 0, unchanged: 672, failures: 0)
Additional information: :ETA_S_0017<EDI>, Active Directory Endpoint 'dc2016.exchange.lab' correlation successful: (accounts correlated: 0, defaulted: 566, unchanged: 6, failures: 0)
Additional information: :ETA_S_0023<EDI>, Active Directory Endpoint 'dc2012.exchange2012.lab' exploration successful: (objects added: 0, deleted: 0, updated: 0, unchanged: 1871, failures: 0)
Additional information: :ETA_S_0017<EDI>, Active Directory Endpoint 'dc2012.exchange2012.lab' correlation successful: (accounts correlated: 0, defaulted: 1619, unchanged: 153, failures: 0)

#################################################################################################
#### How many EA (endpoint accounts) are correlated to the Global User [default user] ###########
#################################################################################################
2185
#################################################################################################

Modify the above script for your own application/endpoints and refine your correlation rules (or add additional ones as needed.)

If applications/endpoints identities are non-managed service IDs, a process that may be used to assist is shown below. Create a new Global User (similar format as [default user]), and then drag-n-drop the endpoint/application service ID accounts to the new Global User [endpoint A service ID].

The final goal is a “clean” orphan process, that will be able to alert us to any rogue accounts being created OOB (out-of-band) of the expected top-down IAM solution from an approved SOT (source-of-truth) solution, e.g. SAP HR/Workday or home-grown DB used with ETL processes. By removing the “noise” of incorrectly correlated accounts, we can now focus on identifying the true “orphans”.