log4j CVE-44228 | Scanning a Million Hosts in Less Than 30 Minutes

Scan one million Linux, Windows and macOS hosts in 30 minutes for all jar files open by any running process for the presence of JndiLookup.class

Our team has been working tirelessly for the past 11 days to ensure that our customers can detect and remediate the vulnerability and be prepared for immediate detection of post exploit activity. I am extremely proud of my team for the scale, speed and efficacy of our solution. I would like to share some of our observations.

How Are We Able To Do This in 30 Minutes or Less?

The architecture of osquery and Uptycs makes this possible. osquery checks in with the Uptycs cloud periodically (default value of 15 minutes) for configuration changes. Using this approach, osquery gets the latest configuration within 15 minutes of updating the configuration in the Uptycs SaaS platform. osquery immediately implements the configuration change. In this particular case the configuration change is to instruct osquery to run a querypack to collect information about the presence of JndiLookup.class file in jar files. The runs the query’s and immediately returns the data to the Uptycs SaaS platform. The query execution happens in parallel on 1 million hosts in a little over 15 minutes. The data from all of the million hosts is aggregated and presented in a report and dashboard in less than 30 minutes of configuring the querypack on Uptycs SaaS platform.

How Does osquery Detect The Presence of JndiLookup.class?

process_open_files - This table gives all of the running processes and the files they have open.

java_packages - Given a jar, war, ear file this table gives a list of the class files in the jar

Using data from these two tables osquery is able to figure out if a process has opened a jar file that has an embedded JndiLookup.class file.

As simple as that. Doing this quickly at scale of 1 million is a challenge Uptycs has solved for our existing customers.

Can Uptycs Look in a Jar Opened By Container Processes?

Yes, Uptycs osquery is able to look at all jar files that are opened by container and host processes.

What Types of Jar Files Can Uptycs osquery Look In?

Uptycs osquery can look in standalone Jar files, Uber Jar files and Shaded Jar files.

How Frequently Can We Get Updated Data?

You can run as frequently as you need to. For example the querypack can be configured to run once every 30 minutes. As your developers are fixing the jar files you can see a report of what is patched and what else needs to be patched.

Can osquery Detect Ongoing Exploitation Attempts of this Vulnerability?

Yes we can. One of the indications that you are being exploited is the presence of specific string patterns in your log files. osquery has the capability to perform yara scanning. Uptycs provides a query you can run in a querypack to scan all locations of log files periodically, say once in 15 minutes. When osquery is used with Uptycs you can collect any yara matches immediately on the Uptycs SaaS platform and alert on the attempted exploit.

Can osquery Detect Post Exploit Activity?

Yes it can. Uptycs provides built-in detection rules and yara rules to detect various APT’s and Toolkits most popular with malicious actors. Examples of yara rule detections we have out of the box : kinsing, Xmrig, Coinminer, Mirai_Strip, Flooder_dnsamp, Tsunami_log4j

Uptycs collects details of every process launched, every public ip address communicated with, every user login, every file changed (in FIM folders) and much more telemetry. The socket related information is very interesting for detecting post log4j exploits. As Uptycs records all of the socket activity for the past N days (number of days depends upon a tenant configuration), we are able to look for all socket activity on or after Dec 9th with the Socket activity post the announcement of log4j vulnerability. Uptycs is able to provide an instant report of remote addresses a host connected to after Dec 10th but did not communicate with in N number of days prior to log4j vulnerability discovery. This helps narrow down the number of ip addresses that need to be investigated.