Osquery and JA3: Detecting Malicious Encrypted Connections Locally

Network traffic encryption is increasing. This increase is driven by demand for privacy protection and the availability of great services for deploying certificates for free. According to Google’s Transparency Report, 88% of web traffic performed on Chrome for Windows is encrypted, and that number is higher for macOS, Android, and ChromeOS.

The encryption trend is even clearer when you look at the percentage of HTTPS browsing time in the Transparency Report. At the same time, malware is also following this trend, as the increased security allows attackers to evade some detection mechanisms.

Encryption of network traffic makes it more difficult to detect malware at the network level, as the visibility decreases. Decrypting traffic is an option in some environments, but there are performance and privacy impacts to consider.

For this reason, John B. Althouse, Jeff Atkison, and Josh Atkins at Salesforce created JA3, a method for profiling SSL/TLS clients, inspired by Lee Brotherston’s TLS FingerprinTLS tool. JA3 gathers information from fields that are not encrypted in the Client Hello packet—such as the version of SSL/TLS being used, the ciphers supported, extensions available—and concatenates it to generate a fingerprint of the SSL/TLS client.

We connected with John Althouse to discuss the why and how of the JA3 project. He explained:

"It's a game of cat and mouse between defenders and threat actors; historically, the defenders have been the mouse. With JA3, the tables have turned. Attackers are now on the back foot. That's exactly what we want."

The data used to compute a JA3 Fingerprint is exchanged between the client and server in clear text. The JA3 fingerprint is obtained by concatenating those fields together and hashing the result.

Because a lot of malware has a TLS implementation that is very different from a full browser, it’s possible to detect some malware via its JA3 fingerprint, at the network level, using tools like Zeek or Moloch.

But what if you don’t have network monitoring tools? What if people are working from home, connecting to the Internet without a VPN? Or what if you wanted to know not only which machine is exhibiting suspicious JA3 fingerprints, but the process that’s doing that?

This is where the power of endpoint monitoring becomes interesting. The traffic is generated by those endpoints, and with an agent installed on it, we have the most visibility into what process did so.

For that reason, Uptycs has added a table to osquery, called http_events. Recently, we also added JA3 support to this table. The table allows Uptycs customers to collect the following data via osquery:

• Information about the connection itself (source and destination IPs and ports)
• Information about SSL and TLS if the connection is encrypted (the metadata needed to compute a JA3 hash)
• The JA3 fingerprint, which is computed and available directly in the table when the connection is encrypted
• Other headers, if desired, though disabled by default as we don’t want to accidentally collect private information via headers

This allows our customers to look at the JA3 fingerprints of processes on their systems, and to correlate those with JA3 fingerprint indicators of compromise (IOC). It also makes it possible to create a list of allowed JA3 fingerprints and alert on the detection of any fingerprint not on the list. This is a great way to detect malware using Cipher Stunting, or to at least increase the cost of attacks by forcing them to completely simulate an already approved fingerprint.

During our call with John, we got his perspective on this new JA3-supported osquery table:

"JA3 is a great way to identify applications connecting on the network using encrypted connections. By using it on the endpoints, it's now possible to know exactly what process generates that fingerprint, a level of detail that can't be obtained at the network level."

HTTP_Events Table Now Available as osquery Patch

Uptycs customers have been using this table effectively for a few months, and we’re now happy to announce that we’re releasing it as a patch for the standard version of osquery so everyone can use it.

Along with the table, the patch introduces new flags to your osquery build:

• --enable_http_lookups - False by default, and when set to true, enables the capture of http_events
• --include_http_headers - Comma separated list of headers you want to capture

The new table was submitted to the official osquery project, but at this time, it has not been merged and the request has been closed. You can get the table from here though!

Applying the above patch to osquery is quite simple. To do so, follow these instructions:

1. Check out the master branch of osquery to your system
2. Navigate to the osquery directory
3. Apply the patch (provided above) to the current branch using “git apply http_events.patch”
4. Build osquery as per the official instructions

That’s it!

Stay tuned for more interesting detection methods using osquery, and feel free to request a demo to see the power of Uptycs for leveraging all of that data easily, with the APIs you need to enable automation within your organization.

Related osquery resources:

• Osquery: What it is, how it works, and how to use it