Leveling Up: The Imperative of Upgrading Your Symantec Identity Suite Virtual Appliance to 14.5 (Centos Stream 9) for Robust Randomness, Enhanced Jitterentropy, and Bouncy Castle Entropy Insights

In the intricate world of cybersecurity and identity management, evolving threats and vulnerabilities demand our undivided attention. When considering upgrading your Symantec Identity Suite Virtual Appliance, understanding the nuanced technological landscape, including the perks of Jitterentropy and the challenges associated with Java’s Bouncy Castle entropy, can make a world of difference.

The Technological Need:

  1. Robust Randomness with Jitterentropy: Relying on the natural timing jitter of CPUs, Jitterentropy has emerged as a game-changing hardware random number generator (RNG). The latest renditions of the Symantec Identity Suite Virtual Appliance leverage this RNG, ensuring unparalleled randomness, making decoding by potential threats a herculean task.
  2. Operational Efficiency: Upgrades tuned with contemporary features promise optimized performance. Coupled with Jitterentropy, the RNG processes are turbocharged, promising minimal downtime and an elevated user experience.
  3. Challenges with Bouncy Castle Entropy in Java: Bouncy Castle, despite its vast utility in cryptographic operations in Java, has had its share of entropy-related issues. Some known problems include:
  • Predictability: Certain RNG implementations in Bouncy Castle have been found to be somewhat predictable, which could compromise security.
  • Seed Reuse: There have been instances where seeds were reused, which again poses security concerns.
  • Slow Entropy Accumulation: At times, the entropy collection is slower than expected, leading to potential operational delays. With security solutions the lack of entropy impacts scale and usability.

Business Justification for Rapid Response:

With the business landscape in perpetual flux, the right tech decisions can spell the difference between stagnation and growth:

  1. Enhanced Security: Incorporating Linux OS with Jitterentropy is synonymous with state-of-the-art security. Such forward-thinking measures drastically curtail potential security breaches.
  2. Cost Savings: Forward-looking upgrades, especially those that incorporate cutting-edge features like Jitterentropy, offer tangible long-term financial advantages. Fewer breaches, reduced system errors, and saved manual efforts contribute positively to the bottom line.
  3. Staying Competitive: In an era of rapid technological advancements, integrating elements like Jitterentropy ensures you’re leading from the front.
  4. Compliance and Regulatory Adherence: With cybersecurity standards constantly on the move, staying updated is non-negotiable. Evade potential legal issues and hefty fines by staying on top of these norms.
  5. Customer Trust: By showcasing a commitment to data safety through advanced systems (and by addressing known entropy issues like those in Bouncy Castle), businesses can strengthen customer trust and foster long-term loyalty.

Validating Jitterentropy Integration in the Linux Kernel: A Comprehensive Guide

As the world of Linux continues to evolve, one exciting development is the incorporation of jitterentropy into the kernel. This robust hardware random number generator (RNG) enhances the quality of randomness, making our systems even more secure. If you’re keen on understanding, implementing, or validating this feature in your Linux setup, this guide is tailored just for you.

What is Jitterentropy?

Jitterentropy is an RNG based on the natural timing jitter that occurs in CPUs. In the realm of cybersecurity, RNGs are of paramount importance; they generate the random numbers pivotal for cryptographic operations. The less predictable these numbers are, the tougher it becomes for malicious actors to crack them.

Why is Jitterentropy Essential?

For systems relying on cryptographic functions, such as encryption, the RNG’s caliber can’t be overstated. Jitterentropy guarantees first-rate randomness, upping your system’s security game. https://www.chronox.de/jent.html

How to Validate Jitterentropy Integration:

  1. Identify Your Kernel Version:
    Kick things off by determining your kernel version using the uname -r or uname -acommand.
   uname -r

This will provide insights into your system’s hostname, kernel version, build date, and architecture. You can deterermine if your Linux kernel is greater than 5.6, when entropy functionality was added directly to the kernel. https://github.com/torvalds/linux/commit/3f2dc2798b81531fd93a3b9b7c39da47ec689e55

  1. Is Jitterentropy Part of Your Kernel Configuration?:
    Deploy this simple grep command to figure out if jitterentropy is enabled in your kernel:
   grep -HRin jitter /boot/config*

An output showing CONFIG_CRYPTO_JITTERENTROPY=y confirms that jitterentropy is enabled. The “y” here indicates that the feature is in-built in the kernel.

  1. Time-Driven Testing for Jitterentropy:
    By simulating multiple pulls from the entropy source, you can gauge how efficient jitterentropy is:
   time for i in {1..1000}; do time dd if=/dev/random bs=1 count=16 2>/dev/null | base64; done

This command performs two functions:

  • It times each of the 1000 pulls from /dev/random, allowing you to measure the average time taken, basically emulating 1000 rapid password changes of 16 characters.
  • It provides an overall timing for 1000 pulls, letting you know the total duration for the entire operation. If your system remains responsive and completes the pulls swiftly, it’s a strong indication that your entropy source is in prime working condition. Which implies that any solution on the appliance has adequate entropy to service users and processes to scale.

Another command that add counters to see that 1000 iteration have passed. Note, if there is no entropy pump, this process will NOT succeed. The Linux OS entropy will be rapidly depleted and any solution on the host will be delayed. Ensure there is an entropy pump to keep the performance you need.

counter=1;MAX=1000;time while [ $counter -le $MAX ]; do echo "##########  $counter ##########" ; time dd if=/dev/random bs=16 count=1 2> /dev/null | base64; counter=$(( $counter + 1 )); done;

Wrapping Up:

The integration of Jitterentropy in the Linux kernel underscores the open-source community’s relentless dedication to fortifying security. By understanding, testing, and leveraging it, you ensure that your system is bolstered against potential threats, always staying a step ahead in the cybersecurity arena. Keep exploring, stay updated, and most importantly, remain secure!

Review upgrade your Symantec Identity Suite to improve your performance for users and scale to millions of transactions.

For non-appliances or older Linux OS (Kernel release < 5.6):

Review adding the haveged or jitterentropy packages to your Linux OS, to avoid delays to any business processes. See prior blog discussing entropy, of how adding an entropy pump to your Linux OSes has value. https://anapartner.com/2021/06/25/the-hidden-cost-of-entropy-to-your-business/

The hidden cost of Entropy to your business

On Linux OS, there are two (2) device drivers that provide entropy “noise” for components that require encryption, e.g. the /dev/random and the /dev/urandom device drivers. The /dev/random is a “blocking” device driver. When the “noise” is low, any component that relies on this driver will be “stalled” until enough entropy is returned. We can measure the entropy from a range of 0-4096. Where a value over 1000 is excellent. Any value in the double or single digits will impact the performance of the OS and solutions with delays. The root cause of these delays is not evident during troubleshooting, and typically there are no warning nor error messages related to entropy.

watch -n 1 cat /proc/sys/kernel/random/entropy_avail

The Symantec Identity Suite solution, when deployed on Linux OS is typically deployed with the JVM switch -Djava.security.egd=file:/dev/./urandom for any component that uses Java (Oracle or AdoptOpenJDK), e.g. Wildfly (IM/IG/IP) and IAMCS (JCS). This JVM variable is sufficient for most use-cases to manage the encryption/hash needs of the solution.

However, for any component that does not provide a mechanism to use the alternative of /dev/urandom driver, the Linux OS vendors offer tools such as the “rng-tools” package. We can review what OS RNGD service is available using package tools, e.g.

dnf list installed | grep -i rng

If the Symantec Identity Suite or other solutions are deployed as standalone components, then we may adjust the Linux OS as we need with no restrictions to add the RNGD daemon as we wish. One favorite is the HAVEGED daemon over the default OS RNGD.

See prior notes on value and testing for Entropy on Linux OS (standalone deployments):

https://community.broadcom.com/enterprisesoftware/communities/community-home/digestviewer/viewthread?GroupId=2197&MID=720771&CommunityKey=f9d65308-ca9b-48b7-915c-7e9cb8fc3295&tab=digestviewer

https://community.broadcom.com/HigherLogic/System/DownloadDocumentFile.ashx?DocumentFileKey=7747b411-2e1e-4bc2-8284-9b8856790ef9

Challenge for vApp

The challenge for Virtual Appliances is that we are limited to what functionality the Symantec Product Team provides for us to leverage. The RNGD service was available on the vApp r14.3, but was disabled for OS challenges with 100% utilization with CentOS 6.4. The service is still installed, but the actual binary is non-executable.

https://knowledge.broadcom.com/external/article/97774/ca-identity-suite-low-entropy-on-virtual.html
https://knowledge.broadcom.com/external/article/139759/ca-identity-suite-142-vapp-rngd-proces.html
https://broadcom-stage.adobecqms.net/us/en/symantec-security-software/identity-security/identity-suite/14-3/virtual-appliance/administering-virtual-appliance/using-the-login-shell.html

A new Virtual Appliance patch would be required to re-enable this RNGD on vApp r14.3cp2. We have access via sudo, to /sbin/chkconfig, /sbin/service to re-enable this service, but as the binary is not executable, we cannot progress any further. We can see the alias in the documentation still exist, but the OS alias was removed in the cp2 update.

However, since vApp r14.4 was release, we can focus on this Virtual Appliance which is running Centos 8 stream. The RNGD service here is disabled (masked) but can be re-enabled for our use with the sudo command. There is no current documented method for RNGD on vApp r14.4 at this time, but the steps below will show an approved way using the ‘config’ userID and sudo commands.

Confirm that the “rng-tools” package is installed and that the RNGD binary is executable. We can also see that the RNGD service is “masked”. Masked services are prevented from starting manually or automatically as an extra safety measure when we wish for tighter control over our systems.

If we test OS entropy for this vApp r14.4 server without RNGD, we can monitor how a simple BASH shell script that emulates a password being generated will impact the “entropy” of /dev/random. The below script will reduce the entropy to low numbers. This process will now impact the OS itself and any components that reference /dev/random. We can observe with “lsof /dev/random” that the java programs will still reference /dev/random; even though most activity is going to /dev/urandom.

Using the time command in the BASH shell script, we can see that the response is rapid for the first 20+ iterations, but as soon as the entropy is depleted, each execution is delayed by 10-30x times.

counter=1;MAX=100;while [ $counter -le $MAX ]; do echo "##########  $counter ##########" ; time dd if=/dev/random bs=8 count=1 2> /dev/null | base64; counter=$(( $counter + 1 )); done;

Enable RNGD on vApp r14.4 & Testing

Now let’s see what RNGD service will do for us when it is enabled. Let’s follow the steps below to unmask, enable, and start the RNGD service as the ‘config’ userID. We have access to sudo to the Centos 8 Stream command of /sbin/systemctl.

sudo /usr/bin/systemctl status rngd.service
ls -lart /etc/systemd/system/rngd.service
sudo /usr/bin/systemctl unmask rngd.service
sudo /usr/bin/systemctl enable rngd.service
cat /usr/lib/systemd/system/rngd.service
sudo /usr/bin/systemctl start rngd.service
sudo /usr/bin/systemctl status rngd.service
ps -ef | grep rngd | grep -v grep

After the RNGD service is enabled, test again with the same prior BASH shell script but bump the loops to 1000 or higher. Note using the time command we can see that each loop finishes within a fraction of a second.

counter=1;MAX=1000;while [ $counter -le $MAX ]; do echo "##########  $counter ##########" ; time dd if=/dev/random bs=8 count=1 2> /dev/null | base64; counter=$(( $counter + 1 )); done;

Summary

Aim to keep the solution footprint small and the right-sized to solve the business’ needs. Do not accept the default performance; avoid over-purchasing to scale to your expected growth.

Use the JVM switch wherever there is a java process, e.g. BLC or home-grown ETL (extract-transform-load) processes.

-Djava.security.egd=file:/dev/./urandom

If you suspect a dependence may impact the OS or other processes on /dev/random, then enable the OS RNGD and perform your testing. Monitor with the top command to ensure RNGD service is providing value and not impacting the solution.