Effective Approach to Security Analytics & Big Data: 4 Essential Factors

Proper security analytics require big data—a fact that companies are increasingly starting to recognize. Nearly 30% of organizations claim they are collecting, processing, and analyzing significantly more security data than they did two years ago, and 42% acknowledge the future importance of leveraging big data for security purposes. But at the same time, only 13% of companies believe their IT security stack is up to the task of effectively collecting and analyzing data organization-wide.

So the question is: What’s the best way to approach data analysis across a large organization’s many operating systems and environments? What does your security team need in order to detect a minor incident before it potentially turns into a major breach? In choosing and deploying your big data security technologies, follow the four principles outlined below to devise a security analytics solution that provides access to hi-fidelity data across your operating environment while remaining flexible, fast, and insightful.

Watch this free webinar to learn about Osquery, the data it collects, and how it helps security teams leverage big data.

Security Analytics and Big Data: 4 Keys To An Effective Approach

Technology that lets you access and manipulate big data for security purposes will improve your ability to detect malicious activity, identify misconfigured or non-compliant assets, and hunt for threats, but it must be implemented thoughtfully to glean the full benefits. Many organizations use an ensemble of security tools to uncover meaningful data that helps teams identify vulnerabilities and defend against attacks intelligently.

1. Don’t let anyone hoard data.

Data hoarding—storing nearly all data for potential future use—can be detrimental to security efforts. (Tweet this!)

Not only is it expensive to store so much data, but it also makes it difficult to uncover what you’re looking to find (You can learn more about SIEM optimization here). In that case, the data becomes unusable for threat-hunting simply because there’s too much of it to query rapidly. A lot of commercial tools do collect and store a lot of data but don’t expose it in a way that makes it easy to use with other tooling.

When you do detect an issue, you want a solution with an open-data format, which lets you identify, and access, exactly the data you want to use. A solution like Uptycs, which uses osquery, is helpful here. Osquery exposes an operating system as a high-performance relational database, and uses SQL to enable the exploration of that operating system telemetry. Uptycs lets you get to that data easily, via a web console as well as through easy to use APIs.

You can also do snapshot queries or differential queries, so if you find an issue, you can then search for other systems where that same issue is present without logging huge amounts of data. For example, the software installed on servers does not change every few seconds or minutes, but you can query for a differential list of software installed every few minutes without needing to store anything if no new package was installed, and nothing was removed. This allows you to know exactly the status of systems, without having to log the entire output every time. Uptycs even makes it easy to go back in time and ask a question, to see what the result of that question was in the past.

2. Use a tool that easily exposes relationships in your data.

The difference between traditional and “big data” technologies is the ability to store large amounts of data in a way that can be indexed daily and searched through in a short amount of time. In recent years, new storage systems have come along—Hadoop, Google Bigtable, etc.—however, these are not relational databases, which can make it difficult to identify relationships quickly and contextualize your information.

With osquery, the breadth and richness of the system telemetry collected provides both event and context data, making it extremely useful for investigation purposes. Furthermore, osquery combined with Uptycs analytics and time machine functionality take the contextualization even further with time series data enabling complete state recreation at any given point in time. While Uptycs stores data in a “big data” environment, it also exposes it to you in an easy to understand format that can be queried with SQL!

3. Aim for real time access.

Your SOC will undoubtedly use a range of point solutions to gather data from network devices and help piece together what’s needed to understand the risk or severity of suspicious or malicious activity. However, you can’t query any of those sources in real time to learn more when you detect an issue.

Uptycs allows you to analyze threat intelligence data in real time. For example, say you’re using Splunk to gather data from your antivirus. If your team discovers an issue, you may need to investigate the endpoints—the servers or workstations—to get more data and find out if the problem is on any other machines in the environment. Uptycs gathers data from endpoints specifically. So when the security team finds an instance of a suspicious application on one machine, they can use Uptycs to dig into it further and confirm whether a threat exists, using historical as well as real time data.

4. Absolutely integrate with other tools.

Also important is the ability to integrate with other tools, so that data and insight can be used to inform cross functionally, activate other important workflows, or initiate remediation activities. For example, if you’re using Demisto or other SOAR tools, when properly integrated with your security analytics, data can be used to terminate a process, activate additional data collection for investigation purposes, trigger an alert, and more.

Uptycs offers two-way integration with tools in your existing security infrastructure. Using the example above, if an application is confirmed as malicious, security teams can use Uptycs in real time to find out if the application is on any other machine in the environment. Then, if a particular executable on a machine occurs, it can connect to your security orchestration system and trigger a specific playbook, perhaps sending the end user an email, disconnecting the laptop from the network, and creating a ticket for the service desk to reimage the machine.

Strengthen your company’s security profile with Uptycs.

Many security products produce data that can be useful, but collecting and analyzing big data sets for today’s complicated computing environments requires a thoughtful approach—one that makes it easier to optimize storage costs and minimize storage tradeoffs, simplify integrations across your security ecosystem, and extend to support your ever evolving cyber security program.

If you’d like to learn more about osquery’s modern approach to security analytics, take a look at our free, on-demand webinar, or visit our website.