Go on the offensive
Why your cyber test range should be built from the offensive perspective
This post explores security test ranges from the offensive and defensive side as a way to test if attacks and analytics are working. However, you should not use a test range as a real world environment for security assessments. You should (nearly) always do your security testing in production as any simulated environment - regardless of the time put into it - is not likely to perfectly replicate the networking, systems, resources and usage of production.
Since the dawn of time (queue dramatic music) defenders and hackers alike have wanted disposable test ranges - or practice environments - to trial their work.
The problem being solved is this: in order to not mess your own laptop up or irreversibly screw up a bare-metal server, you want a test environment which you can spin up and down without worrying about anything.
While I'm not sure if an accurate timeline exists for this problem or not, here's my accounting as I've watched it play out over the last 15 years:
Virtual machines, such as VirtualBox and VMWare, were used as test environments... but these used too many resources, requiring a beefy home lab to support them and they didn't scale well for representing many machines (such as a Windows domain).
Vagrant hit the scene to allow headless virtual machine usage, lowering the resources required and introducing more programmatic handling of test ranges. But this didn't solve the many-machine problem, it only delayed it.
Docker took the reigns from Vagrant by adding "containers" which did not require the overhead of a full operating system but instead allowed you to run the minimum number of processes required. For all intents and purposes, this solved the many-machine problem but it came at a price: realism. Without "real" operating systems, a test range in Docker wasn't helpful to a large population of (cybersecurity) users.
Cloud providers, such as AWS's Cloud Formation tool, started creating dynamic test ranges in their cloud, allowing users to offset the many-machine problem by shifting the resources offsite. For most people, this solved the problem entirely, with the only downsides being cost (which is minimal) and the lock-in effect to a cloud provider (which is no different than a lock-in effect to Docker, VirtualBox, etc.) when you think about it.
So with cloud providers supplying out-of-the-box ranges, why in 2021 are there so many people still building range solutions?
Cybersecurity is hard
The reason that engineers keep building range solutions for hackers and defenders is simple: cybersecurity is hard. It's not rocket-science but in order to build a good range for security testing, you must be an expert in three areas:
Infrastructure (i.e., how to build the range itself)
Offense (in order to replicate adversary behavior)
Defense (in order to apply analytics and defensive tooling)
These are three quite different skill sets.
Nearly all range builders come from a DevSecOps background and build their solution from that angle.
The vast majority of the cybersecurity workforce is on the defensive side. Because security is often thought of as a sunk-cost by businesses, the thought of hiring an offensive security staff is a luxury to most. Instead, organizations generally staff up defensive positions. I would guess that 95% of the industry is on the defensive side and only 5% on offense.
Since most people work in defense, building from this angle seems like a good idea, right? Wrong!
Step back and think about why you're building a test range to begin with:
If you're a defender, you want a range to test if your automated defensive tools (SIEM, EDR, antivirus, etc.) can detect and respond to attacks.
If you're a hacker, you want to test if your attacks work against various types of defenses.
Looking at this rationale, one thing should jump out: the range should be capable of swapping defensive solutions. This allows you to launch an attack, see if a solution detects or responds, and then iterate over more solutions to find gaps.
If you build a range around a specific defensive solution - such as a SIEM, like Splunk or Elasticsearch - you won't (easily) be able to swap out the solution to try alternative defenses.
The reason most cybersecurity engineers build a test environment is to see what defenses do/don't work. Pigeonholing to a single solution defeats the purpose.
The exception to this rule is, of course, if you are locked into using a specific defensive solution. For example, if your organization uses Splunk for centralized logging, you may feel inclined to pull a range option off the shelf which is based around it. The logic makes sense - but if you can prove by swapping out various SIEMs that there is a better option for your organization, don't you think that would be worth (at minimum) noting?
If you're an offensive/adversary emulation engineer, you may want to test out different autonomous red team tools, in which case, you'd want a tool which allows swapping offensive & defensive solutions.
Why is everything so complicated?
If you've used ranges in the past, you've probably been turned off. Not because of their effectiveness but because of their complexity. Most range set ups require significant installation steps, complex instructions to follow and a slew of side cases that are likely to bubble up because everyone's computer is different.
Here's a rule of thumb. If the instructions for using a range don't fit on your screen, it's too complicated. Or if you need to do more than 3 things to set up the range, it's time to shop for a new option.
So, are there any options?
Where does that leave you? As we stated earlier: nearly all security ranges are built from the defensive angle, centered around a specific blue team solution.
But not all of them.
The Operator cloud plugin allows you to spin up cloud hosts which include the following components:
A redirector to ensure your C2 can accept beacons from the internet (i.e., your cloud range)
Windows, Linux or MacOS hosts which are pre-compromised with Pneuma agents
The team is also working on an Operator Cyber Range, which consists of a:
Redirector
Splunk SIEM, stocked with Sigma and Security Content detection rules
Configurable number of compromised Windows hosts forwarding all logs into Splunk
The entire range is push-button, so you can just enter your cloud API keys and click "build". Phew, we kept it under 3 steps!
As you analyze security ranges and try to find one that best suits your needs, keep simplicity in mind.
It's too easy to settle for a complicated range set up just because the engineering team that built it struggled to simplify. Don't blame them - this is difficult - but you shouldn't have to settle.
Testing out attacks and analytics is one of the most enjoyable parts of the industry. So go, enjoy it, don't waste cycles waiting for your range to work.