Why telemetry storage matters for cybersecurity organizations

Telemetry storage is vital to cybersecurity firms for security as well as for operational reasons. It also represents a significant cost which is why LimaCharlie offers a full year of free telemetry storage to our users. LimaCharlie also enables organizations to route their data at the event level which means they can drastically reduce storage costs by only sending relevant data to high-cost security tools like Splunk, Elastic, Sumo Logic, or other SIEM and data analytics solutions.

In this post, we’ll talk about why it’s so important to have this kind of storage, and we’ll try to help different groups of stakeholders within security organizations understand each other’s telemetry storage needs more clearly.

Why demand for telemetry storage is increasing

Let’s begin with an overview of telemetry storage issues.

For anyone unfamiliar with the term, when we say “telemetry” in this context, we simply mean cybersecurity-relevant data from endpoints or applications. It comes from sources like:

• Windows Event Logs

• Endpoint Detection & Response (EDR) software 

• Captured network data packets

• Microsoft Office 365 and Slack

• 1Password, AWS, and Google Cloud logs

These are only a few possible examples…and the need to store new forms of telemetry data is increasing.

In part, this is due to attackers changing their tactics and techniques. For instance, there is a rising threat to CI/CD pipelines, which means companies now need to consider the security of code repositories in addition to traditional telemetry sources (so add GitHub audit logs to the above list).

There are also cultural and economic forces driving this trend. The shift to remote and distributed work, for example, has greatly expanded the attack surface available to bad actors, forcing security teams to monitor telemetry from a growing number of sources.

The upshot is that cybersecurity teams today need to store a lot of telemetry data—ideally in a format that simplifies correlation and analysis.

The business case for telemetry storage

We often think of telemetry as something for security teams to worry about, but business operations and legal teams need robust telemetry data storage too. For these folks, stored telemetry data helps to:

Meet compliance requirements:Legal and organizational requirements vary by industry and by region, but some level of security log retention is a must for nearly every company these days. Whether you’re an MSSP or part of an internal security team, it’s important to be able to offer compliance and reporting teams easy, reliable, and automated telemetry storage.

Simplify the eDiscovery process:When a company has to perform electronic discovery (eDiscovery) in the course of a lawsuit or investigation, it can be expensive and labor intensive. Collecting, analyzing, and preparing eDiscovery data is skilled work—work that’s often handled by an outside party that charges hefty hourly rates. If cybersecurity information is required during the eDiscovery process, access to stored telemetry data in a normalized, easily searchable format makes the work more efficient. 

Reduce spending on core security tools: Security information and event management (SIEM) platforms like Splunk are essential cybersecurity tools—but are notoriously costly. For anyone tasked with reining in SIEM spending, the most obvious solution is to send less data to the SIEM. But that creates a conundrum: deciding what data to collect and what data to leave out so that you don’t lose valuable information. If you have access to free or low-cost telemetry storage, however, this problem disappears. You can keep all of your telemetry data in your “cold storage” location, and only send what you think you really need to your SIEM. If some of the data that wasn’t sent to the SIEM is needed later on, it will still be there waiting in storage. This is a great way to reduce SIEM spending without worrying about whether or not you’ll lose telemetry data. 

The cybersecurity function of telemetry storage 

If you work on the business side of an organization, you may be wondering why your security engineers need stored telemetry data to do their work. It’s understandable: Many of us think of cyber incidents as things that happen in the moment and need to be detected and/or responded to instantly. But that’s not always the way it works in practice. Here’s how security teams use stored telemetry data in their daily work:

Historical threat hunting and incident response (IR):A big part of detecting a cyber threat is knowing what to look for—and that’s not always easy. For example, when a new zero-day exploit is discovered, it is (by definition) something that no one was aware of except for the bad guys. When organizations learn of a newly discovered zero-day threat, it’s imperative that they find out whether or not they’ve been compromised already. With access to stored telemetry data, this can be accomplished by running detection and response (D&R) rules against that historical data to search for known indicators of compromise. This technique is useful for both internal security teams as well as incident response (IR) specialists. 

Investigations and insurance claims: Stored telemetry data is also useful for internal investigations, or when gathering evidence for a cyber insurance claim or even a criminal investigation. This digital forensics (DR) use of historical data is becoming more and more relevant, especially in light of the growing awareness of insider threat. 

Learning more about telemetry storage with LimaCharlie

LimaCharlie gives cybersecurity teams predictable, usage-based pricing that always includes a full year of free telemetry storage—with all telemetry offered in a normalized, searchable data format.

To see how the platform works, try LimaCharlie or book a demo.

To learn more about the different types of telemetry data that can be ingested by LimaCharlie, please refer to the documentation for LimaCharlie Adapter (the software that provides real-time forwarding of logs and other telemetry to LimaCharlie platform).