The Symantec (CA/Broadcom) Directory solution provides a mechanism for routing LDAPv3 traffic to other solutions. This routing mechanism allows Symantec Directory to act as a virtual directory service for other directories, e.g., MS Active Directory, SunOne, Novell eDirectory, etc.
The Symantec Identity Suite solution uses the LDAP protocol for its mid-tier and connector-tier components. The Provisioning Server is exposed on TCP 20389/20390, the JCS (Java Connector Server) is exposed on TCP 20410/20411, and the CCS (C++ Connector Server) is exposed on TCP 20402/20403.
We wished to isolate provisioning data challenges within the Symantec Identity Management solution that was not fully viewable using the existing debugging logs & features of the provisioning tier & connector tiers. Using Symantec Directory, we can leverage the routing mechanism to build a MITM (man-in-the-middle) methodology to track all LDAP traffic through the Symantec Identity Manager connector tier.
We focused on the final leg of provisioning and created a process to track the JCS -> CCS LDAP traffic. We wanted to understand what and how the data was being sent from the JCS to the CCS to isolate issues to the CCS service and MS Active Directory. Using the trace level of Symantec Directory, we can capture all LDAP traffic, including binds/queries/add/modify actions.
The below steps showcase how to use Symantec Directory as an approved MITM process for troubleshooting exercises. We found this process more valuable than deploying Wireshark on the JCS/CCS Server and decoding the encrypted traffic for LDAP.
Background:
Symantec Directory documentation on routing. Please note the concept / feature of “set transparent-routing = true;” to avoid schema challenges when routing to other directory/ldap solutions.
MITM Methodology for JCS->CCS Service:
The Symantec Identity Management connector tier may be deployed on MS Windows or Linux OS. If the CCS service is being used, then MS Windows OS is required for this MS Visual C++ component/service. As we are focused on the CCS service, we will introduce the Symantec Directory solution on the same MS Windows OS.
NOTE: We will keep the MITM process contained on a single host, and will not redirect the network traffic beyond the host.
Step 1: Deploy the latest Symantec Directory solution on MS Windows OS. This deployment is a blank slate for the next steps to follow.
Step 2: Copy the folders of schema, limits, and ssld from an existing Symantec Directory deployment of the Symantec Identity Manager solution. Using the existing schema files, references, and certificates will allow us to avoid any challenges during startup of the Router DSA due to the pre-defined provisioning/connector tier configurations. Please note when copying from a Linux OS version of Symantec Directory, we will need to update the path from Linux format to MS Windows format in the SSLD impd.dxc file for “cert-dir” and “ca-file” parameters.
# DXserver/config/ssld/impd.dxc
set ssl = {
cert-dir = "C:\Program Files\CA\Directory\dxserver\config\ssld\personalities"
ca-file = "C:\Program Files\CA\Directory\dxserver\config\ssld\impd_trusted.pem"
cipher = "HIGH:!SSLv2:!EXP:!aNULL:!eNULL"
#protocol = tlsv12
fips = false
};
Step 3: Create a new Router DSA DXI configuration file. This is the primary configuration file for Symantec Directory DSA. It will referenced the schema, knowledge, limits, and certificates for the DSA. Note the parameters for “transparent-routing” to avoid schema challenges with other solutions. Note the trace level used to trace the LDAP traffic in the Symantec Directory Router DSA trace log.
# DXserver/config/servers/admin_router_ccs_30402.dxi
# logging and tracing
close summary-log;
close trace-log;
source "../logging/default.dxc";
# schema
clear schema;
source "../schema/impd.dxg";
# access controls
clear access;
# source "../access/";
# ssld
source "../ssld/impd.dxc";
# knowledge
clear dsas;
source "../knowledge/admin_router_ccs_group.dxg";
# operational settings
source "../settings/default.dxc";
# service limits
source "../limits/impd.dxc";
# database - none - transparent router
set transparent-routing=TRUE;
# tunnel through eAdmin server error code and messages
set route-non-compliant-ldap-error-codes = true;
set trace=ldap,time,stats;
#set trace=dsa,time;
Step 4: Create the three (3) knowledge files. The “group” knowledge file will be used to redirect to the other two (2) knowledge files of the router DSA and the re-direct DSA to the CCS service.
# DXserver/config/knowledge/admin_router_ccs_group.dxg # The admin_router_ccs_30402.dxc PORT 30402 # will be used for the IAMCS (JCS) CCS port override configuration file # server_ccs.properties via proxyConnectionConfig.proxyServerPort=30402 source "admin_router_ccs_30402.dxc"; source "admin_ccs_server_01.dxc";
# DXserver/config/knowledge/admin_router_ccs_30402.dxc
# This file is sourced by admin_router_ccs_group.dxg.
set dsa admin_router_ccs_30402 =
{
prefix = <>
dsa-name = <dc etasa><cn admin_router_ccs_30402>
dsa-password = "secret"
address = ipv4 localhost port 30402
snmp-port = 22500
console-port = 22501
auth-levels = clear-password
dsp-idle-time = 100000
trust-flags = allow-check-password, trust-conveyed-originator
link-flags = ssl-encryption-remote
};
# DXserver/config/knowledge/admin_ccs_server_01.dxc
# This file is sourced by admin_router_ccs_group.dxg.
set dsa admin_ccs_server_01 =
{
prefix = <dc etasa>
dsa-name = <dc etasa><cn admin_ccs_server_01>
dsa-password = "secret"
address = ipv4 localhost port 20402
auth-levels = clear-password
dsp-idle-time = 100000
dsa-flags = load-share
trust-flags = allow-check-password, no-server-credentials, trust-conveyed-originator
link-flags = dsp-ldap
#link-flags = dsp-ldap, ssl-encryption
# Note: ssl will require update to /etc/hosts with: <IP_Address> eta_server
};
Step 5: Update the JCS configuration file that contains the TCP port that we will be redirecting to. In this example, we will declare TCP 30402 to be the new port.
#C:\Program Files (x86)\CA\Identity Manager\Connector Server\jcs\conf\override\server_ccs.properties
ccsWindowsController.ccsScriptPath=C:\\Program Files (x86)\\CA\\Identity Manager\\Connector Server\\ccs\\bin
proxyCCSManager.enabled=true
proxyCCSManager.startupWait=30
proxyConnectionConfig.proxyServerHostname=localhost
#proxyConnectionConfig.proxyServerPort=20402
proxyConnectionConfig.proxyServerPort=30402
proxyConnectionConfig.proxyServerUser=cn=root,dc=etasa
proxyConnectionConfig.proxyServerPassword={AES}pbj27RvWGakDKCr+DhRH4Q==
proxyConnectionConfig.proxyServerUseSsl=false
proxyCCSManager.controller.ref=ccsWindowsController
Overview of all files updated and their relationship to each other.
Validation
Start up the solution in the following order. Ensure that the new Symantec Directory Router DSA is starting with no issue. If there are any syntax issues, isolate them with the command: dxserver -d start DSA_NAME.
Start the Router DSA first, then restart the im_jcs (JCS) service. The im_ccs (CCS) service will be auto-started by the JCS service. Wait one (1) minute, then check that both TCP Ports 20402 (CCS) and 30402 (Router DSA) are both in the LISTEN state. If we do not see these both ports, please stop and restart these services.
May use MS Sysinternals ProcessExplorer to monitor both services and using the TCP/IP tab, to view which ports are being used.
Use a 3rd party LDAP client tool, such as Jxplorer to authenticate to both the CCS and the Router DSA ports, with the embedded service ID of “cn=root,dc=etasa”. We should see exactly the SAME data.
Use the IME or IMPS to perform a query to MS Active Directory (or any other endpoint that uses the CCS connector tier). We should now see the “cache” on the CCS service be populated with the endpoint information, and the base DN structure. We can now track all LDAP traffic through the Router DSA MITM process.
View of trace logs
We can monitor when the JCS first binds to the CCS service.
We can monitor when the IMPS via the JCS queries if the CCS is aware of the ADS endpoint.
Finally, we can view when the IMPS service decrypt its stored information on the Active Directory endpoint, and push this information to the CCS cache, to allow communication to MS Active Directory. Using Notepad++ we can tail the trace log.
Please note, this is a secure LDAP/S tunnel from the IMPS -> JCS -> CCS -> MS ADS.
We can now view how this data is pushed via this secure tunnel with the MITM process.
> [88] > [88] <-- #1 LDAP MESSAGE messageID 5 > [88] AddRequest > [88] entry: eTADSDirectoryName=ads2016,eTNamespaceName=ActiveDirectory,dc=im,dc=etasa > [88] attributes: > [88] type: eTADSobjectCategory > [88] value: CN=Domain-DNS,CN=Schema,CN=Configuration,DC=exchange,DC=lab > [88] type: eTADSdomainFunctionality > [88] value: 7 > [88] type: eTADSUseSSL > [88] value: 3 > [88] type: eTADSexchangeGroups > [88] value: CN=Mailbox Database 0840997559,CN=Databases,CN=Exchange Administrative Group (FYDIBOHF23SPDLT),CN=Administrative Groups,CN=ExchangeLab,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=exchange,DC=lab > [88] value: CN=im,CN=Databases,CN=Exchange Administrative Group (FYDIBOHF23SPDLT),CN=Administrative Groups,CN=ExchangeLab,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=exchange,DC=lab > [88] type: eTLogWindowsEventSeverity > [88] value: FE > [88] type: eTAccountResumable > [88] value: 1 > [88] type: eTADSnetBIOS > [88] value: EXCHANGE > [88] type: eTLogStdoutSeverity > [88] value: FE > [88] type: eTLog > [88] value: 0 > [88] type: eTLogUnicenterSeverity > [88] value: FE > [88] type: eTADSlockoutDuration > [88] value: -18000000000 > [88] type: objectClass > [88] value: eTADSDirectory > [88] type: eTLogETSeverity > [88] value: FE > [88] type: eTADSmsExchSystemObjectsObjectVersion > [88] value: 13240 > [88] type: eTADSsettings > [88] value: 3 > [88] type: eTADSconfig > [88] value: ExpirePwd=0 > [88] value: HomeDirInheritPermission=0 > [88] type: eTLogDestination > [88] value: F > [88] type: eTADSUserContainer > [88] value: CN=BuiltIn;CN=Users > [88] type: eTADSbackupDirs > [88] value: 000;DEFAULT;192.168.242.156;0 > [88] value: 001;DEFAULT;dc2016.exchange.lab;0 > [88] value: 002;site1;server1.domain.com;0 > [88] value: 003;site1;server2.domain.com;0 > [88] value: 004;site2;server3.domain.com;0 > [88] value: 005;site2;server4.domain.com;0 > [88] type: eTADSuseFailover > [88] value: 1 > [88] type: eTLogAuditSeverity > [88] value: FE > [88] type: eTADS-DefaultContext > [88] value: exchange.lab > [88] type: eTADSforestFunctionality > [88] value: 7 > [88] type: eTADSAuthDN > [88] value: Administrator > [88] type: eTADSlyncMaxConnection > [88] value: 5 > [88] type: eTADShomeMTA > [88] value: CN=Microsoft MTA,CN=EXCHANGE2016,CN=Servers,CN=Exchange Administrative Group (FYDIBOHF23SPDLT),CN=Administrative Groups,CN=ExchangeLab,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=exchange,DC=lab > [88] type: eTADSAuthPWD > [88] value: CAdemo123 > [88] type: eTADSexchangelegacyDN > [88] value: /o=ExchangeLab/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Configuration/cn=Servers/cn=EXCHANGE2016/cn=Microsoft Private MDB > [88] type: eTLogFileSeverity > [88] value: F > [88] type: eTADSprimaryServer > [88] value: dc2016.exchange.lab > [88] type: eTADScontainers > [88] value: CN=Builtin,DC=exchange,DC=lab > [88] value: CN=Computers,DC=exchange,DC=lab > [88] value: OU=Domain Controllers,DC=exchange,DC=lab > [88] value: OU=Explore,DC=exchange,DC=lab > [88] value: CN=ForeignSecurityPrincipals,DC=exchange,DC=lab > [88] value: CN=Keys,DC=exchange,DC=lab > [88] value: CN=Managed Service Accounts,DC=exchange,DC=lab > [88] value: OU=Microsoft Exchange Security Groups,DC=exchange,DC=lab > [88] value: OU=o365,DC=exchange,DC=lab > [88] value: OU=People,DC=exchange,DC=lab > [88] value: CN=Program Data,DC=exchange,DC=lab > [88] value: CN=Users,DC=exchange,DC=lab > [88] value: DC=ForestDnsZones,DC=exchange,DC=lab > [88] value: DC=DomainDnsZones,DC=exchange,DC=lab > [88] type: eTADSTimeBoundMembershipsEnabled > [88] value: 0 > [88] type: eTADSexchange > [88] value: 1 > [88] type: eTADSdomainControllerFunctionality > [88] value: 7 > [88] type: eTADSexchangeStores > [88] value: CN=EXCHANGE2016,CN=Servers,CN=Exchange Administrative Group (FYDIBOHF23SPDLT),CN=Administrative Groups,CN=ExchangeLab,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=exchange,DC=lab > [88] value: CN=Mailbox,CN=Transport Configuration,CN=EXCHANGE2016,CN=Servers,CN=Exchange Administrative Group (FYDIBOHF23SPDLT),CN=Administrative Groups,CN=ExchangeLab,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=exchange,DC=lab > [88] value: CN=Frontend,CN=Transport Configuration,CN=EXCHANGE2016,CN=Servers,CN=Exchange Administrative Group (FYDIBOHF23SPDLT),CN=Administrative Groups,CN=ExchangeLab,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=exchange,DC=lab > [88] type: eTADSKeepCamCaftFiles > [88] value: 0 > [88] type: eTADSmsExchSchemaVersion > [88] value: 15333 > [88] type: eTADSCamCaftTimeout > [88] value: 0000001800 > [88] type: eTADSMaxConnectionsInPool > [88] value: 0000000101 > [88] type: eTADSPortNum > [88] value: 389 > [88] type: eTADSDCDomain > [88] value: DC=exchange,DC=lab > [88] type: eTADSServerName > [88] value: 192.168.242.156 > [88] type: eTADSDirectoryName > [88] value: ads2016 > [88] type: eTAccountDeletable > [88] value: 1 > [88] controls: > [88] controlType: 2.16.840.1.113730.3.4.2 > [88] non-critical
We can now monitor all traffic and assist with troubleshooting any CCS/MS-ADS challenges.
This same MITM methodology/process may also be used for the IMPS (TCP 20389/2039) and the JCS (TCP 20410/20411) services. We have used this process to capture the IME (JIAM) LDAP traffic to the IMPS Service, to isolate multiple queries for Child Provisioning Roles. Which has been used by the product team to enhance the solution to lower startup durations of the IME in the latest releases.
Binds/queries/add/modification all work with this approach, but we do see an issue with OID for IMPS ADS endpoint “explore process” on ADS OU object. We are reviewing how to address this last challenge that states “critical extension is unavailable” for a LDAP control property of the OU object. The OIDs captured appear to be related to SunOne/Iplanet.
