Log4J Security Scan Hits Explanation
Overview
Syndeia Cloud (SC) is built on top of a software stack consisting of Cassandra (C*) + Janusgraph (JG) + Zookeeper (ZK) + Kafka. SC & Cassandra do not use Log4J but have SLF4J abstraction interface libraries to work with it- this is not the same as using Log4J directly. Certain 3rd-party software component dependencies of SC (ex: Janusgraph (JG) + Zookeeper (ZK)) may include Log4J but their usage of it can easily be remediated. Other 3rd-party software dependency components (ex: Kafka) do not or no longer use Log4J at all.
Applies to
Syndeia Cloud 3.5.X
Issue
If you are running a "security scan" for this, you may get hits for library files with the word log4j
. Some results (ex: for Janusgraph (JG) + Zookeeper (ZK)) may include Log4J- which can easily be remediated through the use of Reload4J, while others may be false positives depending on how you or your security software is performing the scan.
Reproduction Steps:
Ex: Here are results from one (naive) attempt to search for "Log4J" from a Linux SC 3.5-SP2 server:
joeadmin@sc-server.domain.tld:~$ sudo find / -type f -iname 'log4j.jar'
/opt/icx/syndeia-cloud-3.5-SP2.2023-10-30/web-gateway-3.5-SP2-patch/lib/org.slf4j.log4j-over-slf4j-1.7.32.jar
/opt/icx/syndeia-cloud-3.5-SP2.2023-10-30/teamworkcloud-impl-3.5-SP2/lib/org.slf4j.log4j-over-slf4j-1.7.32.jar
/opt/icx/syndeia-cloud-3.5-SP2.2023-10-30/confluence-impl-3.5-SP2/lib/org.slf4j.log4j-over-slf4j-1.7.32.jar
/opt/icx/syndeia-cloud-3.5-SP2.2023-10-30/collaborator-impl-3.5-SP2/lib/org.slf4j.log4j-over-slf4j-1.7.32.jar
/opt/icx/syndeia-cloud-3.5-SP2.2023-10-30/jama-impl-3.5-SP2/lib/org.slf4j.log4j-over-slf4j-1.7.32.jar
/opt/icx/syndeia-cloud-3.5-SP2.2023-10-30/gitlab-impl-3.5-SP2/lib/org.slf4j.log4j-over-slf4j-1.7.32.jar
/opt/icx/syndeia-cloud-3.5-SP2.2023-10-30/store-impl-3.5-SP2/lib/org.slf4j.log4j-over-slf4j-1.7.32.jar
/opt/icx/syndeia-cloud-3.5-SP2.2023-10-30/bitbucket-impl-3.5-SP2/lib/org.slf4j.log4j-over-slf4j-1.7.32.jar
/opt/icx/syndeia-cloud-3.5-SP2.2023-10-30/devops-3.5-SP2/lib/org.slf4j.log4j-over-slf4j-1.7.32.jar
/opt/icx/syndeia-cloud-3.5-SP2.2023-10-30/graph-impl-3.5-SP2/lib/org.slf4j.log4j-over-slf4j-1.7.32.jar
/opt/icx/syndeia-cloud-3.5-SP2.2023-10-30/web-gateway-3.5-SP2.bk/lib/org.slf4j.log4j-over-slf4j-1.7.32.jar
/opt/icx/syndeia-cloud-3.5-SP2.2023-10-30/artifactory-impl-3.5-SP2/lib/org.slf4j.log4j-over-slf4j-1.7.32.jar
/opt/icx/syndeia-cloud-3.5-SP2.2023-10-30/sysmlv2-impl-3.5-SP2/lib/org.slf4j.log4j-over-slf4j-1.7.32.jar
/opt/icx/syndeia-cloud-3.5-SP2.2023-10-30/jira-impl-3.5-SP2/lib/org.slf4j.log4j-over-slf4j-1.7.32.jar
/opt/icx/syndeia-cloud-3.5-SP2.2023-10-30/aras-impl-3.5-SP2-patch.bk/lib/org.slf4j.log4j-over-slf4j-1.7.32.jar
/opt/icx/syndeia-cloud-3.5-SP2.2023-10-30/restful-impl-3.5-SP2/lib/org.slf4j.log4j-over-slf4j-1.7.32.jar
/opt/icx/syndeia-cloud-3.5-SP2.2023-10-30/doors-impl-3.5-SP2/lib/org.slf4j.log4j-over-slf4j-1.7.32.jar
/opt/icx/syndeia-cloud-3.5-SP2.2023-10-30/testrail-impl-3.5-SP2/lib/org.slf4j.log4j-over-slf4j-1.7.32.jar
/opt/icx/syndeia-cloud-3.5-SP2.2023-10-30/aras-impl-3.5-SP2.bk/lib/org.slf4j.log4j-over-slf4j-1.7.32.jar
/opt/icx/syndeia-cloud-3.5-SP2.2023-10-30/windchill-impl-3.5-SP2/lib/org.slf4j.log4j-over-slf4j-1.7.32.jar
/opt/icx/syndeia-cloud-3.5-SP2.2023-10-30/aras-impl-3.5-SP2-patch/lib/org.slf4j.log4j-over-slf4j-1.7.32.jar
/opt/icx/syndeia-cloud-3.5-SP2.2023-10-30/github-impl-3.5-SP2/lib/org.slf4j.log4j-over-slf4j-1.7.32.jar
/opt/icx/syndeia-cloud-3.5-SP2.2023-10-30/auth-impl-3.5-SP2/lib/org.slf4j.log4j-over-slf4j-1.7.32.jar
/opt/icx/syndeia-cloud-3.5-SP2.2023-10-30/volta-impl-3.5-SP2/lib/org.slf4j.log4j-over-slf4j-1.7.32.jar
/opt/janusgraph-full-0.5.3/elasticsearch/lib/log4j-core-2.11.1.jar
/opt/janusgraph-full-0.5.3/elasticsearch/lib/log4j-api-2.11.1.jar
/opt/janusgraph-full-0.5.3/elasticsearch/modules/x-pack-core/log4j-1.2-api-2.11.1.jar
/opt/janusgraph-full-0.5.3/elasticsearch/modules/x-pack-security/log4j-slf4j-impl-2.11.1.jar
/opt/janusgraph-full-0.5.3/lib/log4j-1.2.16.jar
/opt/janusgraph-full-0.5.3/lib/slf4j-log4j12-1.7.12.jar
/opt/kafka_2.13-3.2.1/libs/kafka-log4j-appender-3.2.1.jar
/opt/zookeeper-3.6.3/lib/log4j-1.2.17.jar
/opt/zookeeper-3.6.3/lib/slf4j-log4j12-1.7.25.jar
/usr/share/cassandra/lib/log4j-over-slf4j-1.7.25.jar
joeadmin@sc-server.domain.tld:~$
Expected
No files with Log4J show up
Actual
Files with Log4J show up.
Explanation + Workaround/Resolution
A naive directory search for all files with the word log4j
in it will result in false positives. When performing a security scan for Log4J, one needs to apply a bit of insight to interpret the results.
An explanation of the results now for each component is as follows (Legend: = Safe: not used or harmless, Log4J used by 3rd-party component, *
= wildcard)
/opt/icx/syndeia-cloud-3.*/
: Log4j is NOT actually being used in the Syndeia Cloud services (there is no log4j-1.x or 2.x jar file results for Syndeia cloud). This is SLF4J, see A. below./opt/kafka*
:kafka-log4j-appender-3.2.1.jar
: See below for more info on Log4J Appenders in B. below/opt/zookeeper-3.6.3/*
:/opt/zookeeper-3.6.3/lib/log4j-1.2.17.jar
: this can be replaced by Reload4j jar file(s) (see Reload4J Usage below)/opt/janusgraph-full-*
:/opt/janusgraph-full-0.5.3/elasticsearch/*
: this is an optional 3rd-party indexing sub-component of JG which uses Log4J. As long as the JG was NOT setup to use Elasticsearch during setup, the entireelasticsearch
subdirectory can be safely deleted./opt/janusgraph-full-0.5.3/*
:/opt/janusgraph-full-0.5.3/lib/log4j-1.2.16.jar
: this can be replaced by Reload4j jar file(s) (see Reload4J Usage below)/opt/janusgraph-full-0.5.3/*slf4j*
: This is SLF4J, see A. below
/usr/share/cassandra/*
:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.25.jar
: This is SLF4J, see A. below.
Result Footnotes
A.
slf4j
: SLF4J is not the same thing as Log4J. The former is an abstraction for various logging frameworks, which includes the ability to interface with Log4J, IF used (which it doesn't based on the results). To state it another way, SLF4J is an interface for logging. It comes with an interface for log4j that allows applications to use log4j (for more details see: https://www.slf4j.org/legacy.html ).B.
*log4j-appender-3.2.1.jar
: This is a feature for streaming log4j logs to Kafka. Syndeia is not using this feature. It is available but unless there is a log4j jar included, it is not of much use (see Stream Log4j Logs to Apache Kafka | DevGlan & Log4j -- Log4j 2 Appenders - Apache Log4j 2 for more info)
Reload4J Usage
Reload4J is a drop-in-replacement initiative that was started by the community to address the security issues found in older versions of Log4J and continue maintenance of the code base for projects/software that aren’t ready to switch to newer versions of Log4J yet, ex: Log4J2.
A direct download for it can be found at the Maven Repository site: https://repo1.maven.org/maven2/ch/qos/reload4j/reload4j/. Select a version and then click on the link that says jar
(file size)
Q: For the patching of these vulns should we be removing and replacing the file in the directory with the reload4j-1.2.25.jar file or do we need to overwrite the existing file?
Unfortunately, there's no simple answer for this as there is no standard way to specify this. Most components come with custom scripts to build its Java -classpath
or -cp
of .JAR files. Some simply list a wildcard *
(for those I believe you'd be able to just remove the original log4j .JAR and drop in Reload4J's .JAR). Others hardcode the list of specific .JAR files (which is a bit more secure but would require manually editing the launcher script).
The easiest (but not cleanest) thing to do would be to do the latter (just overwrite the existing file) however that may lead to confusion later on if the same name is kept, but it can be done if absolutely necessary, or if you are unfamiliar with editing Java launcher scripts.
The harder (but cleaner and recommended) way would be to edit the individual component's launcher script(s) / classpath startup parameter.
Summary
SC 3.5(-SPX) does not use log4j v1 or v2 anymore and has SLF4J
Cassandra only uses SLF4J
Kafka is completely clean of Log4j v1.X and only has Log4J appenders IF log4j is used.
For Zookeeper (ZK), you will want to use Reload4J.
For Janusgraph (JG), you will want to just blow away the
elasticsearch
directory (it is unused by JG anyway, unless you told the SC JG setup script to use it) and use Reload4J
Related articles