Ten years inside: how a Chinese APT group lived in Linux through a backdoor in PAM and OpenSSH

When Sygnia’s team began reconstructing the intrusion they would later call Operation Highland, the earliest artifacts dated back to 2016. They weren’t looking at a fresh breach — they were looking at nearly a decade of undetected presence inside an air-gapped critical infrastructure network. Velvet Ant, a China-nexus APT actor Sygnia has tracked across multiple investigations, didn’t exploit a vulnerability and didn’t leave obvious traces. It modified the programs that decide who gets to log in.

Velvet Ant replaced legitimate PAM modules and OpenSSH binaries with backdoored copies. Not new malware that scanners might catch — a modification of trusted system components the OS trusts by default. Passwords were reset; the backdoor stayed. Sessions were terminated; the group came back. Standard incident response playbooks didn’t work because the authentication mechanism itself was working for the attacker.

This isn’t the first time Velvet Ant has run this playbook. In 2024, Sygnia caught the same group abusing F5 BIG-IP appliances for persistence. Then exploiting CVE-2024-20399 in Cisco NX-OS to plant a backdoor directly on Nexus switches. Each time one foothold was discovered, the group went deeper — into less-monitored infrastructure. Operation Highland is the next step: there’s nowhere deeper to go. This is the authentication stack itself.

WHAT PAM IS AND WHY IT MATTERS

PAM (Pluggable Authentication Modules) is the Linux subsystem that handles authentication for essentially every service: SSH logins, sudo, su, the local console. When you type a password on a Linux server, PAM is what checks it. The key module — pam_unix.so — is responsible for standard password verification against /etc/shadow.

OpenSSH is the standard SSH stack on every Linux server: sshd as the server daemon, ssh as the client, scp for file transfer. Together, PAM and OpenSSH are the foundation of remote administration. Control them and you control everything: who gets in, what they see, which commands get recorded. All of it invisible in normal logs.

One important thing to understand: replacing pam_unix.so and OpenSSH binaries requires root on the target host. This isn’t an initial access technique — it’s a persistence technique used after root is already obtained. That’s exactly why Velvet Ant invested so much effort in the first two stages: get initial access, navigate to the isolated segment, get root — and only then touch the authentication files. For defenders, this means: controlling who gets root, and when, matters just as much as monitoring after the fact.

REAL-WORLD ATTACK CHAIN

The target network had no direct internet access, so the attackers had to build a path in stages. In the first stage, Velvet Ant established a foothold on internet-facing servers. They deployed a modified version of GS-Netcat — a tool from the public Global Socket Toolkit — renamed it auditdb, and dropped it into /usr/sbin/ among legitimate system utilities. To hide in process listings, it overwrote its own argv[0] with [khubd], mimicking a legitimate kernel thread. Persistence was handled through a systemd unit file on newer systems, or by appending a line to /etc/init.d/ scripts on older ones — in both cases disguised as a Chrome service.

Running in parallel: a SOCKS5 proxy written in Perl — a custom implementation of the public ssspl project — for traffic tunneling and lateral movement. The process masked itself as smbd -D. Each sample used a unique filename, process name, and listening port, making cross-host correlation significantly harder.

In the second stage, the group moved through the IT network into the isolated critical infrastructure segment. They modified a compromised nginx server to proxy requests for a specific URL to a backend, where a FastCGI wrapper (fcgiwrap) executed a custom binary named uptime. That binary established SSH connections to servers in the isolated segment and ran scripts there — driven by parameters from HTTP POST requests. No direct connection to the isolated network was ever needed: commands traveled over plain HTTP.

In the third — and central — stage, the attackers took over the authentication stack. On compromised hosts, pam_unix.so was replaced with a backdoored copy. Sygnia found nine distinct variants, each compiled in a separate build environment — a level of effort that points to a structured, well-resourced production pipeline. Two primary types: the first accepted a hardcoded backdoor password and bypassed standard verification entirely (the password was immediately zeroed out in memory after use, to avoid showing up in dumps); the second additionally recorded every real username and password to a hidden file at /usr/sbin/.ssh.log. In that variant, the backdoor password was stored in the binary in encrypted form.

The OpenSSH binaries — ssh, sshd, scp, and in some cases ssh-keygen — were also replaced. The modified sshd captured and stored credentials from all incoming connections in an encrypted file at /usr/share/man9/ph/.ph.man. Each entry recorded connection direction, authentication type, and success status. Both ssh and sshd also logged every command typed in a session: sshd wrote files to /usr/share/man9/ph/ named s{MD5(username+TTY)}, while ssh used c{MD5(UID+SSH_TTY)}. All created files and directories were timestomped to match the creation time of /etc/shells. The modified scp could disable SELinux when run as root. Its ! flag disguised the process as [kauditd] -sh.

One detail that reveals the operational discipline behind this campaign: the modified ssh included a custom -d flag that disabled its own credential and keylogging. When Velvet Ant operators worked through the backdoored binary, they activated that flag so their own activity left no trace. The third persistence layer was simpler but equally durable: the group appended their own public keys to authorized_keys on compromised hosts, giving them password-free access independent of everything else.

TIMELINE

The earliest keylogging artifacts in this environment date to 2016. They come from the older variant of the modified OpenSSH suite, which logged sessions to /var/lib/sam/ using filenames like sam_{timestamp} — less sophisticated than later versions, and that’s exactly what let Sygnia pin down when the group first appeared. The presence continued, undetected, for roughly a decade.

Sygnia tracks Velvet Ant across several independent investigations involving different victims. In 2024, the group was caught abusing F5 BIG-IP appliances for persistence inside a separate organization. Later that year, exploiting CVE-2024-20399 — a zero-day in Cisco NX-OS — to deploy the VELVETSHELL backdoor directly on Nexus switches. Sygnia published detailed writeups with IOCs for both.

The pattern is consistent: when discovered, Velvet Ant doesn’t leave — it moves to infrastructure that defenders monitor less. Proprietary network appliances get less scrutiny than Windows servers. Linux switches get less than appliances. The Linux authentication stack on the hosts themselves gets the least scrutiny of all. Operation Highland was published by Sygnia on June 11, 2026.

WHY IT MATTERS

The standard incident response playbook says: reset passwords, kill sessions, isolate the host. None of that works when the authentication mechanism itself is compromised. Resetting passwords is pointless if pam_unix.so accepts the backdoor password regardless of what’s in /etc/shadow. Killing sessions is pointless if the next legitimate admin login gets harvested by the modified module. EDR, IDS, and SIEM see nothing anomalous — the logins look exactly like normal administrative activity.

For any sysadmin running a production server with SSH access, this points directly to a blind spot: if an attacker had access to /lib/security/pam_unix.so or the OpenSSH binaries, checking logs won’t surface it. The class of attack Sygnia calls “authentication-layer persistence” requires a different detection strategy entirely — not log analysis, but integrity monitoring of the executables themselves.

Velvet Ant’s pattern holds: each time one foothold is found, the group already has the next one ready, one layer deeper. Operation Highland is the point where the attackers stopped controlling a specific service and started controlling the ability to authenticate to every server in the environment.

HOW TO PREVENT THIS ON YOUR SERVERS

The essential protection against this class of attack is File Integrity Monitoring on authentication binaries — not config files, not logs, the executables themselves. AIDE, auditd with write watches, or any FIM solution should cover: pam_unix.so and other modules under /lib/security/ and /lib/x86_64-linux-gnu/security/, PAM configuration in /etc/pam.d/, the OpenSSH binaries ssh, sshd, scp, sftp, and ssh-keygen, the sshd_config file, authorized_keys for privileged accounts, systemd unit files and SysVinit startup scripts. Any change to these files should trigger an alert that gets investigated — not just logged.

A quick integrity check against installed packages is available through dpkg --verify. On Debian and Ubuntu, it compares file checksums against the package metadata — empty output means the files match, a line starting with ??5?????? means the checksum doesn’t:

dpkg --verify openssh-server openssh-client libpam-modules

It’s a fast first check, but not a complete one: an attacker with root can tamper with the dpkg database too. A more reliable approach is to compare SHA256 hashes directly against a clean reference image of the same distro version — download the package from an official mirror on a separate clean machine, extract it, and compare hashes against the files on the server in question. A match is fine; a mismatch on an identical package version is a red flag.

Reducing the risk of root being obtained in the first place is where standard hardening also helps here: disable direct root SSH login (PermitRootLogin no in sshd_config), route all admin access through named accounts with tightly scoped sudo permissions. MFA should happen before the request reaches the host — a compromised PAM stack will simply ignore any MFA enforced inside it.

If a compromise has already occurred, the recovery sequence matters. Resetting passwords before replacing pam_unix.so accomplishes nothing — the modified module will capture the new ones on the very next login. The order: verify and replace authentication binaries first, then audit and clean authorized_keys, then rotate credentials. Sygnia specifically notes that any PAM module or OpenSSH binary replacement must be tested in an isolated lab against an identical configuration beforehand — an incompatible version or a missing dependency can lock administrators out of a live production server.

WHAT TO DO RIGHT NOW

Whether you have reason to suspect a compromise or just want to rule it out, check for the file paths Velvet Ant used in this campaign. These commands look for the directories and files specific to Operation Highland:

ls -la /usr/sbin/.ssh.log 2>/dev/null
ls -la /usr/share/man9/ph/ 2>/dev/null
ls -la /usr/lib/eth-scsi/ 2>/dev/null
ls -la /var/lib/sam/ 2>/dev/null

If any of these paths exist, that’s a direct IOC from this campaign. Also check authorized_keys for root and other privileged accounts — Velvet Ant added its own public keys as a third persistence layer. For root:

cat /root/.ssh/authorized_keys

For other users with login shells, check ~/.ssh/authorized_keys in their home directories. Any key that isn’t in your trusted key registry is a potential IOC.

Sygnia published the full list of indicators of compromise for Operation Highland as an appendix to the report. If your environment fits the described profile — critical infrastructure, isolated network, Linux servers with long uptimes and no regular binary auditing — integrity checks on PAM and OpenSSH should be at the top of the list.

CONCLUSIONS

Operation Highland shows where a sophisticated persistent attack ends up when defenders keep finding footholds: Velvet Ant kept going deeper until it reached the one layer most Blue Teams never check. PAM and OpenSSH aren’t “just packages.” They’re the mechanism that decides whether someone is who they claim to be. Compromise them and MFA, strong passwords, and log auditing all become irrelevant at the same time.

The practical takeaway for sysadmins: FIM on authentication binaries isn’t paranoia — it’s the minimum for any server running long-term administrative processes. For organizations with critical infrastructure or isolated networks: trusting a component simply because it’s a system component is exactly the threat model Velvet Ant has been exploiting for nearly a decade.