Written by When your solution grows, it may be challenging to identify when and what team members added with regards to new business logic. We can parse through change control documentation, but that may be a long and frustrating process. One of the challenges is that objects created within the Symantec (Broadcom) Identity Suite solution may not have date stamps on the object within the database tables. Again, we could parse through logs and the Task Persistence and Archive Task Persistence databases.
Please stop this behavior.
Let’s introduce a new streamlined process to help your administrative and business IAG teams.
Below is a process to use existing tools & samples within the existing IAG solution.
Goals: Automate a daily backup and create an XML delta of new business objects that were created the prior days. The tools used will be the included Import Export utility with additional Linux commands. We focused this process on the Symantec (Broadcom) Identity Suite Virtual Appliance with the built-in userID of ‘config’.
To start we will copy the solution’s included Import/Export sample to the ‘config’ userID home folder, and the minimal libraries files required to execute this process. We will then modify the script to perform a delta compare every day that it is executed by the ‘config’ crontab.
# Step 1 - Create a copy of the Import / Export Utility under the config home folder
mkdir -p /home/config/backup/export
cp -r -p /opt/CA/IdentityManager/IAM_Suite/IdentityManager/tools/ImportExportUtility/ /home/config/backup/export/
# Step 2 - Copy the three (3) extra JAR files required by the Import / Export Utility
mkdir -p /home/config/backup/export/lib
cp -r -p /opt/CA/IdentityManager/IAM_Suite/IdentityManager/tools/lib/idmutils.jar /home/config/backup/export/lib/
cp -r -p /opt/CA/IdentityManager/IAM_Suite/IdentityManager/tools/lib/log4j.jar /home/config/backup/export/lib/
cp -r -p /opt/CA/IdentityManager/IAM_Suite/IdentityManager/tools/lib/bc-fips.jar /home/config/backup/export/lib/
# Step 3 - Backup the current shell script and properties file before our changes.
cd /home/config/backup/export/ImportExportUtility/
cp -r -p config.properties config.properties.org
cp -r -p ImportExportUtil.sh ImportExportUtil.sh.org
We now will update the config.properties with your own hostname and credentials.
Below is the contexts of a working file, with additional commentary to assist with the replacement of the PBES password encryption. Please note, that the mode=export, and that we have selected resourceType=RoleDefination. Over 99% of the business objects will reside within this single XLM file when it is exported. We set the localPath=. to be the current path to allow the automated scripts to rename files for use with an XML diff tool. You may wish to update the export path to a network path.
## provide IM server base url with port number
## Use netstat -apn | grep 8080 to confirm IP address
baseUrl=http://192.168.2.220:8080
## Login credential (in case Management console is protected), use JSAFE algorithm of PasswordTool to encrypt your plain text password
## Example: /opt/CA/IdentityManager/IAM_Suite/IdentityManager/tools/PasswordTool > ./pwdtools.sh -JSAFE -p Password01
userName=admin
password={PBES}:B8+4u/F3aiZ9sXus6HyDNA==
## provide mode import/export
mode=export
## provide resource type ALL/Directory/Environment/RoleDefinition
#resourceType=Directory
resourceType=RoleDefinition
#resourceType=Environment
#resourceType=All
## provide comma separated list of Directories to import/export, in case of import it should be xml file name
directories=ProvStore,UserStore,AuthenticationDirectory
## provide Environment name for Environment/Role Definition import/export, in case of environment import it should be zip file name
environment=identityEnv
## In case of Role Definition import please provide xml file name
roleDefFileName=env-RoleDefinitions
## provide local path to save/get the resources, in case of export directory structure will be created
localPath=.
## provide request time out in minutes
timeout=600
## restart Environment after import: yes/no, For restart to work environment name should be provided
restartEnv=no
We now will update the shell script of ImportExportUtil.sh.
We have renamed this shell script to ExportRoles.sh to clearly call out what we wish for this process to focus on, and what we will call it via a crontab entry. We have enhanced the JAVA_OPTS to speed up exports depending on the number of business objects, e.g. an IME with 40K provisioning roles may take over 60 minutes to export. We then have created a process that will generate an XML diff between the prior export and the latest export. We do date-time stamp the exports to allow past review of changes.
#!/bin/sh
#
# This batch file sets up the environment and runs the IM Import and Export Utility
#
if [ -z "$JAVA_HOME" ] ; then
echo "---------------------------------------------------------------------"
echo "ERROR: Cannot find JAVA_HOME"
echo "Please specify JAVA_HOME variable in this script file."
echo "---------------------------------------------------------------------"
exit
fi
export JAVA_HOME
MYCLASSPATH=.:./importExportUtility.jar:../lib/bc-fips.jar:../lib/idmutils.jar:../lib/log4j.jar
export MYCLASSPATH
##############
### ANA, 04/22
# Update JAVA_OPTS for speed as 40K Prov Roles may take 60-90 minutes to export
tz=`/bin/date --utc +%Y%m%d%H%M%S,3%N.0Z`
echo ""
echo "Starting at : $tz"
echo ""
#JAVA_OPTS="-Xms256m -Xmx512m $JAVA_OPTS"
JAVA_OPTS="-Xms256m -Xmx4g -Djava.net.preferIPv4Stack=true -Djava.awt.headless=true -Djava.security.egd=file:/dev/./urandom -Dcom.sun.jndi.ldap.object.disableEndpointIdentification=true $JAVA_OPTS"
##############
### ANA, 04/22
# Rename prior backup to be used for diff operation prior to exporting a new file
dos2unix identityEnv-RoleDefinitions.xml >/dev/null 2>&1
cp -r -p identityEnv-RoleDefinitions.xml identityEnv-RoleDefinitions_prior.xml >/dev/null 2>&1
export JAVA_OPTS
$JAVA_HOME/bin/java $JAVA_OPTS -cp $MYCLASSPATH com.ca.identitymanager.importexportutility.client.ImportExportClient
##############
### ANA, 04/22
# Setup config crontab to execute this task every day at 1:11 AM
# 11 1 * * * /home/config/scripts/create_pr_and_import_them/ExportRoles/ExportRoles.sh >/dev/null 2>&1
# Use https://crontab.guru/ to define the correct scheduler
# Rename vApp (all) files for future review and delta compares
dos2unix identityEnv-RoleDefinitions.xml >/dev/null 2>&1
cp -r -p identityEnv-RoleDefinitions.xml identityEnv-RoleDefinitions_$tz.xml >/dev/null 2>&1
##############
### ANA, 04/22
# Perform Diff operation between prior and new exports of IME Roles&Tasks.xml files
# Create an XML diff between two prior files.
#diff <(xmllint --c14n identityEnv-RoleDefinitions_prior.xml) <(xmllint --c14n identityEnv-RoleDefinitions.xml)
xmllint --c14n identityEnv-RoleDefinitions_prior.xml > identityEnv-RoleDefinitions_prior_xmllint.xml
xmllint --c14n identityEnv-RoleDefinitions.xml > identityEnv-RoleDefinitions_xmllint.xml
diff identityEnv-RoleDefinitions_prior_xmllint.xml identityEnv-RoleDefinitions_xmllint.xml > identityEnv-RoleDefinitions_DIFF_xmllint.xml
cp -r -p identityEnv-RoleDefinitions_DIFF_xmllint.xml identityEnv-RoleDefinitions_DIFF_xmllint_$tz.xml >/dev/null 2>&1
echo ""
echo "There are `wc -l identityEnv-RoleDefinitions_DIFF_xmllint.xml` rows different between the two files"
echo ""
echo "There are `grep -i '<ImsRole' identityEnv-RoleDefinitions_DIFF_xmllint.xml | wc -l ` Roles delta between the two files"
echo ""
echo "Please review if these deltas are correct: cat identityEnv-RoleDefinitions_DIFF_xmllint.xml | more "
echo ""
tz=`/bin/date --utc +%Y%m%d%H%M%S,3%N.0Z`
echo "Done at : $tz "
echo ""
Examples of this process executed. First, let’s generate about 5 provisioning roles to be loaded into the IME and then import them.
config@pwdha001 VAPP-14.3.0 (192.168.2.220):~/scripts/create_pr_and_import_them > ./create_5k_pr_xml_file_for_ime_mgmt_import.sh
real 0m0.028s
user 0m0.006s
sys 0m0.013s
Number of Provisioning Roles Created in XML File: 5
-rw-rw-r-- 1 config config 6.2K Apr 23 14:01 new_provisioning_roles.xmlNow we will import this XML file into the IME (see sample below for a Provisioning Role addition)
<?xml version="1.0" encoding="UTF-8"?>
<ims:ImsTemplate xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://imsenvironmentobjects/xsd imsconfig://schema/ImsEnv
ironmentObjects.xsd" xmlns:ims="http://imsenvironmentobjects/xsd" xmlns:imsrule="http://imsmemberrule/xsd" xmlns:imsscope="http://imsscoperule/xsd" x
mlns:imschange="http://imschangeaction/xsd">
<!-- ******************** Create 10K Provisioning Roles ******************** -->
<ImsRole name="prov-role_160823241129774" roletype="PROVISIONING" assignable="true" adminassignable="true" enabled="true" allowduplicatecusto
m="false" description="DESC_upto_128_Characters" custom01="CF01_upto_1000_Characters" custom02="CF02" custom03="CF03" custom04="CF04" custom05="CF05"
custom06="CF06" custom07="CF07" custom08="CF08" custom09="CF09" custom10="CF10">
<AdminPolicy assignable="true" adminassignable="true">
<imsrule:MemberRule><RoleMember><AdminRole name="User Manager"/></RoleMember></imsrule:MemberRule>
<imsscope:ScopeRule object="USER" purpose="*"><All/></imsscope:ScopeRule>
</AdminPolicy>
<AdminPolicy assignable="true" adminassignable="true">
<imsrule:MemberRule><All/></imsrule:MemberRule>
<imsscope:ScopeRule object="USER" purpose="*"><All/></imsscope:ScopeRule>
</AdminPolicy>
<OwnerPolicy>
<imsrule:MemberRule><RoleMember><AdminRole name="System Manager"/></RoleMember></imsrule:MemberRule>
</OwnerPolicy>
<Attribute name="comments">2022-04-23T20:01:47.000Z : COMMENTS_upto_128_Characters</Attribute>
<Attribute name="department">DEPT_upto_100_Characters</Attribute>
</ImsRole>
</ims:ImsTemplate>Output from an import with the ImportExportUtil shell script
-----------------------------------------------------------
-------------------Starting a new Import-------------------
-----------------------------------------------------------
Importing Role Definition to Environment 'identityEnv'...
############# Import Output #############
Warning: Updating the IdentityMinder environment "identityEnv"
Deploying role definitions...
Importing Roles...
*********
0 error(s), 0 warning(s)
Role Definition Imported Successfully!!!
Now we will run our new ExportRoles.sh script, where it will export the IME and do an XML delta compare operation between a prior export and the latest export.
config@pwdha001 VAPP-14.3.0 (192.168.2.220):~/backup/export/ImportExportUtility > time ./ExportRoles.sh
Starting at : 20220423190707,3582824826.0Z
-----------------------------------------------------------
-------------------Starting a new Export-------------------
-----------------------------------------------------------
Exporting Role Definition from Environment 'identityEnv'...
disposition attachment; filename=identityEnv-RoleDefinitions.xml;
Exported Filename: identityEnv-RoleDefinitions.xml
Role Definition exported Successfully!!!
There are 76 identityEnv-RoleDefinitions_DIFF_xmllint.xml rows different between the two files
There are 5 Roles delta between the two files
Please review if these deltas are correct: cat identityEnv-RoleDefinitions_DIFF_xmllint.xml | more
Done at : 20220423190713,3241475139.0Z
real 0m5.663s
user 0m2.509s
sys 0m0.394s
Here is a view of the delta file that is generated. We now KNOW what was added since the last export. We have a date range as well.
> <ImsRole adminassignable="true" allowduplicatecustom="false" assignable="true" custom01="CF01_upto_1000_Characters" custom02="CF02" custom03="CF03" custom04="CF04" custom05="CF0
5" custom06="CF06" custom07="CF07" custom08="CF08" custom09="CF09" custom10="CF10" description="DESC_upto_128_Characters" enabled="true" name="prov-role_160823241129774" roletype="PROVI
SIONING">
> <AdminPolicy adminassignable="true" assignable="true">
> <imsrule:MemberRule><RoleMember><AdminRole name="User Manager"></AdminRole></RoleMember></imsrule:MemberRule>
> <imsscope:ScopeRule object="USER" purpose="*"><All></All></imsscope:ScopeRule>
> </AdminPolicy>
> <AdminPolicy adminassignable="true" assignable="true">
> <imsrule:MemberRule><All></All></imsrule:MemberRule>
> <imsscope:ScopeRule object="USER" purpose="*"><All></All></imsscope:ScopeRule>
> </AdminPolicy>
> <OwnerPolicy>
> <imsrule:MemberRule><RoleMember><AdminRole name="System Manager"></AdminRole></RoleMember></imsrule:MemberRule>
> </OwnerPolicy>
> <Attribute name="comments">2022-04-23T20:01:47.000Z : COMMENTS_upto_128_Characters</Attribute>
> <Attribute name="department">DEPT_upto_100_Characters</Attribute>
> </ImsRole>
A view of the files generated in the export process. NOTE: The first time executed, there will be an error due to the missing 1st file. Make a change in the IME, and then run this process a 2nd time to see the delta.
For larger IME, the export may take 60-90 minutes. The below example shows for a full “ALL” export that may be configured in the config.properties file. This IME was updated with over 40K Provisioning Roles to help isolate a java memory leak.
Counting the roles (Admin, Access, Provisioning) within the IME (Oracle DB) table of IM_ROLE
When we tested loading 5K Provisioning Roles, we noted it would take about 30 minutes. If the input file was over 10 MB, then the Import process would fail with a generic java error.
