Workaround instructions to address CVE-2021-44228 and CVE-2021-45046 in vRealize Operations 7.x

In this article i will go over one of the workaround instructions to address CVE-2021-44228 and CVE-2021-45046 in vRealize Operations 7.x. I have tested the workaround on vROPS 7.5 as its still shipped with VCF 3.x and i haven’t yet seen documentation on a workaround for this version. If you are looking for instructions for version 8.x consult kb article 87076. This has been tested on December 21 2021. Please check the official documentation or open a ticket for production usage.

Create a snapshot of the vROPS components to make sure we have something to revert to in case anything were to go wrong.

Log into the vROPS instance admin UI typically https://ip_address/admin and take the cluster offline. This applies to all nodes including but not limited to Analytic, Primary, Replica, Data, Remote Collectors and Witness nodes.

Give a reason and press ok

Verify the cluster is offline before continuing

Log in via ssh to a temporary path ex /tmp. Because vROPS 7.5 doesn`t come with the newer OpenSSL modules we need to find other means to get the files to the server without using a direct download method like wget.

In my case in put the code below in a file called vrops-log4j-fix.sh in my /tmp directory

#!/bin/bash

file=/tmp/impacted_jars.txt

echo "Searching for impacted .jar files. Please wait..."

find /usr/lib -type f -name "*.jar" -exec sh -c "zipinfo -1 '{}' | grep "org/apache/logging/log4j/core/lookup/JndiLookup.class" && echo {}" \; | grep "/usr/lib" > $file

line_qty=$(wc -l < $file)

if [ $line_qty -ne 0 ]; then
    echo "Found $line_qty impacted .jar files"
else
    echo "No impacted .jar files found"
    exit 0
fi

echo "Starting to patch impacted .jar files"

while IFS= read -r line;
do     
    echo "patching -> $line"

    own_user=$(stat -c '%U' "$line")
    own_group=$(stat -c '%G' "$line")

    zip -q -d "$line" org/apache/logging/log4j/core/lookup/JndiLookup.class
    if [ $? -ne 0 ]; then echo "ERROR: Fail to Patch $line"; fi

    chown $own_user:$own_group "$line"

done < $file

rm -f $file

Make the file executable by running chmod +x vrops-log4j-fix.sh

And then run the script by running ./vrops-log4j-fix.sh

The system will go through and find impacted .jar files and try to patch them. If successful we should end up with something like this

Next we will do the same with cp-log4j-fix.sh file

#!/bin/bash

#set -x


FAILURE="0"
WRAPPER_FILES="/usr/lib/vmware-vcops/user/conf/collector/wrapper.conf"
for f in $WRAPPER_FILES
    do
        last_idx=""
        if [[ -f $f ]]; then
            echo "********************************"
            echo "Updating file: $f"
            let last_idx=$(egrep "^wrapper.java.additional.[[:digit:]]+=" $f | cut -d= -f1 | awk -F '.' '{print $4}' | sort -n | tail -1)
            if [[ -z $last_idx ]]; then
                echo -e "ERROR: Failed to get JVM additional index"
                let FAILURE="1"
                continue
            fi
            ((last_idx++))
            echo -e "\n#Fixing Apache Log4j2 Remote Code Execution Vulnerability\nwrapper.java.additional.$last_idx=-Dlog4j2.formatMsgNoLookups=true" >> $f
            if [[ $? != 0 ]]; then
                echo -e "ERROR: Failed to update file: $f\n"
                let FAILURE="1"
            else 
                echo -e "Sucessfully updated file: $f\n"
            fi
        else
            echo -e "ERROR: file is not found: $f\n"
            let FAILURE="1"
        fi
    done


CASA_JVM="/usr/lib/vmware-casa/casa-webapp/bin/setenv.sh"
echo "********************************"
echo "Updating file: $CASA_JVM"
echo 'JAVA_OPTS="$JAVA_OPTS -Dlog4j2.formatMsgNoLookups=true"' >> $CASA_JVM
if [[ $? != 0 ]]; then
    echo -e "ERROR: Failed to update file: $CASA_JVM\n"
    let FAILURE="1"
else
    echo -e "Sucessfully updated file: $CASA_JVM\n"
fi

if [[ "X$FAILURE" == "X1" ]]; then
    exit 1
fi

exit 0

and lastly the data-rc-witness-log4j-fix.sh file

#!/bin/bash

#set -x


FAILURE="0"
WRAPPER_FILES="/usr/lib/vmware-vcops/user/conf/analytics/wrapper.conf
/usr/lib/vmware-vcops/user/conf/collector/wrapper.conf
/usr/lib/vmware-vcops/user/conf/gemfire/wrapper.conf
/usr/lib/vmware-vcops/user/conf/tomcat-enterprise/wrapper.conf"
for f in $WRAPPER_FILES
    do
        last_idx=""
        if [[ -f $f ]]; then
            echo "********************************"
            echo "Updating file: $f"
            let last_idx=$(egrep "^wrapper.java.additional.[[:digit:]]+=" $f | cut -d= -f1 | awk -F '.' '{print $4}' | sort -n | tail -1)
            if [[ -z $last_idx ]]; then
                echo -e "ERROR: Failed to get JVM additional index"
                let FAILURE="1"
                continue
            fi
            ((last_idx++))
            echo -e "\n#Fixing Apache Log4j2 Remote Code Execution Vulnerability\nwrapper.java.additional.$last_idx=-Dlog4j2.formatMsgNoLookups=true" >> $f
            if [[ $? != 0 ]]; then
                echo -e "ERROR: Failed to update file: $f\n"
                let FAILURE="1"
            else 
                echo -e "Sucessfully updated file: $f\n"
            fi
        else
            echo -e "ERROR: file is not found: $f\n"
            let FAILURE="1"
        fi
    done


CATALINA_FILES="/usr/lib/vmware-casa/casa-webapp/bin/setenv.sh
/usr/lib/vmware-vcops/tomcat-web-app/bin/setenv.sh"

for f in $CATALINA_FILES
    do
        if [[ -f $f ]]; then
            echo "********************************"
            echo "Updating file: $f"
            echo 'JAVA_OPTS="$JAVA_OPTS -Dlog4j2.formatMsgNoLookups=true"' >> $f
            if [[ $? != 0 ]]; then
                echo -e "ERROR: Failed to update file: $f\n"
                let FAILURE="1"
            else 
                echo -e "Sucessfully updated file: $f\n"
            fi
        else
            echo -e "ERROR: file is not found: $f\n"
            let FAILURE="1"
        fi
    done

if [[ "X$FAILURE" == "X1" ]]; then
    exit 1
fi

exit 0

It would look similar to this in the end

To verify that CVE-2021-44228 was applied run the following

ps axf | grep --color log4j2.formatMsgNoLookups | grep -v grep

Running ./vrops-log4j-fix.sh will also verify that there are no .jar files that need to be patched

Next bring the instance back online in the admin console

Please follow and like us:
Pin Share

Leave a Reply