Key[word]s to the Kingdom: Simple modular agents
A few things up front before getting into the meat of the content. First off, a writer I am not; I neither guarantee grammatical perfection nor am I verbose. Secondly, this will be a fairly technical post. I will try to keep good pacing and not go too deep down the rabbit hole. So without further ado...
The overview
For those of you who are not familiar with Prelude's Operator, this discussion will probably not make a lot of sense. So I'd recommend you at the very least download it, log in, and poke around the docs (the community edition is free and available on all major operating systems). Either way, here's a quick overview:
Operator is desktop application designed to be a one-stop shop for writing TTPs, controlling remote agents, going through up-to-date training on adversarial TTPs, building and launching custom adversaries powered by an automated planner. If that last bit sounds like Caldera and you know what that is, you're already familiar with David (@privateducky) and me (@khyberspache) :)
With a few days left in our sprint, I decided to go "all-in" on two areas that had been bothering me. Specifically, I wanted to improve our agent offerings and demonstrate the power and flexibility of the Operator platform beyond the existing shell command based TTPs. Our basic agent, Pneuma (http://github.com/preludeorg/pneuma) is a vanilla example of how you can design an agent, but it doesn't have much flexibility or extensibility.
Enter PneumaEX.
The problem(s)
There were few key problems I was trying to address within the Operator ecosystem:
Shell-based executors (sh, bash, PowerShell, osascript, cmd, etc) don't offer enough complexity for more advanced red, blue, and purple teams. We, as cybersecurity professionals, should be continually evolving our tradecraft over time, so new ways to streamline the testing more variations of TTPs leads to improved detection and response.
Advanced agents are difficult to write. There is a lot of boilerplate code that is needed to make even a basic-featured agent capable of communicating over a basic protocol like HTTP. And once you start talking about packaging more advanced capability, you run the risk of your agent being signatured and having to start from scratch. Obviously, there are ways around that, but in general, less is more in this case.
Operator users need a way to quickly implement TTPs as code in order to expand the scope of testing in their particular environment. From the red team perspective, Living off the Land is great, but sometimes it's just easier to bring along a tool. From the blue team perspective, maybe there is an API I should be monitoring that I'm not, and hunting for variations of a technique can help tune detections.
Operator's "brain" (automated planner) doesn't have a way to incorporate time or more specifically, expected knowledge, into its planning. It's trivial for a human to run a key logger in the background while working on other tasks, collect the data, exfiltrate and analyze it. However, breaking that down, it's a lot of coordination and timing. In order to replicate that, we need to have the planner make decisions based upon data it expectsto be available at a certain time while still proceeding with its other asynchronous tasking.
The idea
I had prototyped a few different concepts a couple weeks back with the general idea that I want to dynamically load in new "things" at runtime and have those "things" do "stuff." Oh, and have that dynamic piece be cross-platform. While that sounds ridiculously open-ended, that's kinda the whole point of Operator: give you the means to do "things" in a simple way. A modular architecture would keep complexity down while expanding capability, so it's ideal for this use case.
First thing I went towards was using C to construct SOs, DLLs, and DyLibs that could be loaded into a simple framework agent. Next, I decided what I wanted to do was craft standalone Position Independent Executable (PIE) blobs similar to this:
| // example pie_blob.c | |
| int f1(int v, void (* exit)(int)){ | |
| (*exit)(0); | |
| return v; | |
| } |
then compile those blobs and use objdump to cut out the .text segment. I could then load them into the main agent process like this:
| // Request the pie_blob module from the C2 server | |
| // mmap exectuable memory | |
| fptr = mmap(NULL, sb.st_size, PROT_READ | PROT_EXEC | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); | |
| // read the blob into memory | |
| result = fread(fptr, 1, sb.st_size, pBlob); | |
| // grab whatever libraries/symbols I want (or even better, just get pointers to dlsym/dlopen) | |
| handle = dlopen(LIBC_FILE, RTLD_LAZY); | |
| *(void**) (&ex) = dlsym(handle, "exit"); | |
| // then literally just call the function in pie_blob | |
| int i = 1; | |
| i = ((int(*)(int))fptr)(i); |
I could store all the loaded modules in the main agent process and call them as needed. Ultimately I chose not to go this route due to time constraints (I am a terrible C programmer) and it doesn't solve any of the immediate problems. A version of this is still on my TODO list - I think it's an incredibly powerful way to "patch" a binary.
Next, I poked at GOLang's built-in Plugin module. That was pretty much dead on arrival because: A) it doesn't support Windows, B) it is notoriously buggy according to anecdotes from other engineers using them, and C) Plugin support for Windows isn't even in the backlog (as of this writing).
After some more poking around, I came across Hashicorp's go-plugin (https://github.com/hashicorp/go-plugin). Turns out they built a plugin system for Go because Go's native plugin support isn't up to snuff - and they use it in Packer, Terraform, Nomad, and Vault. Aka it's a perfect solution for a quick implementation of a cross-platform module system. Not to mention, it's RPC/gRPC based, meaning I could write plugins in other languages!
The approach
The general idea is that we want to be able to have a simple TTP, when applied to an adversary, cause the agent to download a module, install it, then call a specific function inside the module with specific arguments.
TTP Definitions:
Looking at the TTP below we can see a few things:
Prelude Operator TTP Editor: Keyword executor for clipboard capture
We are using what is called a "keyword" executor 3 platforms (Windows and Darwin are visible, Linux is also there). "Keyword" is a way to tell the agent to execute a specific task that isn't the standard Shell executor mechanisms. For example, you could have keyword like "stop agent" you might do something like this:
| func RunCommand(message string, executor string, payloadPath string) (string, int, int) { | |
| if executor == "keyword" { | |
| switch message { | |
| case "stop agent": | |
| os.Exit(0) | |
| case "module": | |
| // do module stuff | |
| default: | |
| // do other stuff | |
| } | |
| } else { | |
| // run command using exec.Command("powershell.exe", ...) | |
| } | |
| } |
In this case we have the keyword syntax of:
| module.collect.captureClipboard |
which indicates which module (collect) and which function (captureClipboard) to call inside the module. The module itself is specified in the payload field for each platform as:
| #{operator.payloads}/path/to/payload/collect-windows.exe | |
| #{operator.payloads}/path/to/payload/collect-linux | |
| #{operator.payloads}/path/to/payload/collect-darwin |
The "#{operator.payloads}" is Operator's "fact" syntax, meaning the planner will automatically populate that location based upon where the payloads are stored.
Agent upgrades
Now that there is a simple syntax for referencing modules and functions, how does the agent actually work? The best place to start is with the module/plugin definition. Every module requires a very simple structure along with a base configuration. A basic module definition looks like this:
| package main | |
| import () | |
| var ( | |
| ModuleName = "collect" | |
| Functions = map[string]func(args []string) ([]byte, int){ | |
| "captureClipboard": captureClipboard, | |
| } | |
| ExecFunctions = map[string]func(args string) (){ | |
| "GoCapture": GoClipboard, | |
| } | |
| ) | |
| func captureClipboard(args []string) ([]byte, int) { | |
| // setup and do clipboard capture | |
| return []byte("We have captured all the things"), 0 | |
| } | |
| func GoCapture(args string) { | |
| // args is a string so you might need to parse if you're sending in multiple flags | |
| } |
As you can see, the keyword maps to the ModuleName and Functions. So "module.collect.captureClipboard" is actually going to end up calling the "captureClipboard" function and the results will be returned back to the C2 server! Woot! But what about that ExecFunctions? Well that comes into play with problem 4 above: time-based operations and creating "expected knowledge." ExecFunctions allow you to run the module as a standalone executable. For example, you can have long-running background tasks (like a key logger) while simultaneously informing the automated planner that the logger process will write a file and be done in X minutes.
Passing in arguments using the fact system is straightforward as well. You can see the "args" slice is passed through to each function. That maps directly to the TTP file as seen in our exfiltration module:
| command: | | |
| module.exfil.httpServer.["#{operator.http}", "#{file.T1056.001}", "#{agent.name}", "#{operator.session}"] |
These facts will be populated by the automated planner when they are discovered on target systems at which point they will be rendered into the command and passed through to your function as strings. Obviously, there is a typing issue in some cases, but you can use GOlang's reflect library and type casting to do most of what you'd want to do in advanced use-cases.
In terms of "what do I need to know to extend pneumaEX", that's all you need! All of the RPC communications and argument parsing is handled by the agent and a base file inside a plugin that you don't have to edit/change. Basically, you just add a function to the Functions map, write that function, compile, and stage the plugin somewhere for your agent to pull. As mentioned before, because this is RPC based, you also have the option of writing a plugin in any language as long as you handle the plugin setup.
How do you run the ExecFunctions? Inside the base class there is a helper function call RunStandalone that will handle it for you. Just call:
| RunStandalone("GoCapture", "C:\File\Path\To\Capture\into.tmp") |
There is a lot "more" happening behind the scenes, but that should cover the general idea. Using this approach you can quickly and easily add new modules into PneumaEX. Looking back at our problems:
We now support "keyword" executors that can load into modules that do arbitrary tasks, drastically increasing the possibilities of TTPs for red, blue, and purple team use cases.
The agent and module system complexity is hidden, while still providing extensive capability. Adding a new module requires 1 copy-paste base file and the above module configuration file. Users only have to implement (potentially) a single function, then use the build script to automatically construct all the modules.
Implementing a TTP-as-code can take a little as a single function and GOLang allows us to write cross-platform modules, including CGO modules. This means we can hit native APIs or potentially write plugins as DLLs/SOs/DyLibs that expose functions.
We can now send information to the automated planner ("brain") that includes timestamping for expected completion, which enables us to make assertions about potential future knowledge. Using that we can make predictions about future kill chains and queue up potential actions based upon what is expected to happen. This will also prove invaluable for handling what I dub the "lateral movement problem" in automated planning (that deserves an entire post in and of itself).
In addition to knocking out the core problems, we also gain some potential as this solution expands in complexity.
The future
I have many ideas for how I would improve this work, but here is a short list of areas I want to continue working on:
Time-based Operator TTPs.
Modules can run as entirely standalone executables which is great for creating time-driven TTP chaining inside Operator. A few of the plugins will create a temporary file and return a timestamp for when collection to that staging file is complete. With a few tweaks, Operator could schedule exfiltration TTPs to fire when that timestamp is reached.
gRPC.
Implementing gRPC in addition to the existing RPC communications simplifies the interfaces for building plugins in other languages.
In-memory only modules.
We can leverage the ReattachConfig structure as an argument to the plugin client configuration to attach to an already running process. This allows us to handle module execution separately, meaning we can can hot-swap in-memory loaders to stage our plugin, then attach to it when we want to use it.
Protocol negotiation and encryption.
Adding in a protocol negotiation helps with in-field upgrades and compatibility as the core agent is upgraded. On encryption (TLS): yes, I will implement it, no it's not a priority at this time. If your blue team is monitoring RPC between processes on a system (doubtful) then you should probably be using an approach similar to the one I outlined in the C code above.
Bi-directional communication.
While not critical for majority of tasks, there are benefits to enabling bi-directional communication between the plugins and the primary agent process. For example, a plugin is used to escalate privileges and remains running in that high security context while the main agent remains as is. The plugin could request plugin information and spawn that plugin in that higher security context. Bidirectional plugins will also enables the modularization of core C2 protocols within PneumaEX - meaning we can hot swap C2 modules within a fielded agent.