How I Would Turn the Investigation Into Permanent Monitoring

Because the best time to notice a new webshell is before the attacker has had three days to use it.

By SepedaTua — CrushEdge.com


After restoring a compromised server, I don’t want to spend the next six months manually running:

find
grep
stat
file

every morning.

That works during an incident.

It doesn’t scale as a habit.

What I want instead is a small amount of automation that tells me:

“Something changed that probably deserves your attention.”

Not:

“Here are 17,000 log lines. Good luck.”


1. Start with the simplest possible monitoring

I would begin with a daily filesystem report.

For each Virtualmin account:

find /home/customer/public_html \
-type f \
-newermt '24 hours ago' \
-printf '%TY-%Tm-%Td %TH:%TM:%TS | %u:%g | %s | %p\n' \
2>/dev/null |
sort

That gives me recently changed files.

For a busy WordPress site, there will be legitimate changes.

That’s okay.

I’m looking for unexpected changes.


2. PHP files deserve special attention

I would separately report new or modified PHP files:

find /home/customer/public_html \
-type f \
\( -name '*.php' -o -name '*.phtml' -o -name '*.php5' -o -name '*.php7' \) \
-newermt '24 hours ago' \
-printf '%TY-%Tm-%Td %TH:%TM:%TS | %s | %p\n' \
2>/dev/null |
sort

This is much more useful than scanning every file equally.

A new JPEG in:

wp-content/uploads/

is normal.

A new:

wp-content/uploads/cache123.php

is much more interesting.


3. I would specifically monitor uploads

Something like:

find /home \
-path '*/wp-content/uploads/*' \
-type f \
\( -name '*.php' -o -name '*.phtml' -o -name '*.php5' -o -name '*.php7' \) \
-print \
2>/dev/null

Ideally, the result should be empty.

If it suddenly produces:

/home/customer/public_html/wp-content/uploads/2026/08/x.php

I want an alert.


4. Fake images should also trigger an alert

The same file technique from the investigation can become a monitoring script.

Conceptually:

find image files
       ↓
file
       ↓
is it really an image?
       ↓
NO
       ↓
alert

This catches the exact trick that caused so much trouble during my investigation:

something.jpg

that is actually:

PHP script

5. Don’t alert on every suspicious PHP function

This is important.

If I monitor only:

eval()
base64_decode()
system()
shell_exec()

I’ll get false positives.

WordPress and plugins contain plenty of complicated code.

Instead, I’d give higher priority to combinations.

For example:

random filename
+
outside normal WordPress directories
+
recently created
+
obfuscated PHP

That’s much more suspicious than:

known plugin
+
old file
+
base64_decode()

Context wins.


6. Monitor the database accounts too

A simple daily snapshot:

mysql -NBe \
"SELECT User, Host FROM mysql.user ORDER BY User, Host;" \
> /root/mysql-users-current.txt

Then compare it with yesterday’s snapshot.

If a new account appears:

temporary_admin

you’ll know.

The same idea works for grants.


7. Monitor SSH keys

Create a snapshot:

find /root /home \
-path '*/.ssh/authorized_keys' \
-type f \
-exec sha256sum {} \; \
> /root/ssh-authorized-keys-current.txt

If the hash changes, investigate.

A changed authorized_keys file is not automatically malicious.

You may have legitimately added a key.

But at least you’ll know it changed.


8. Monitor listening ports

Once a day:

ss -lntup \
> /root/listening-ports-current.txt

Compare it with your known-good baseline.

If yesterday you had:

22
80
443
10000

and today you have:

22
80
443
10000
4444

I want to know what is listening on 4444.


9. Monitor cron

Save:

crontab -l \
> /root/root-cron-current.txt

And enumerate user crontabs if applicable.

Again, changes aren’t automatically malicious.

But unexpected persistence mechanisms should not silently appear.


10. Watch systemd

Similarly:

systemctl list-unit-files \
--type=service \
> /root/services-current.txt

A newly installed service may be perfectly legitimate.

But if nobody remembers installing it:

investigate.


11. Use hashes for stable WordPress core

This is where a WordPress-specific verification tool can help.

For standard WordPress core:

wp core verify-checksums \
--path=/home/customer/public_html

If WordPress reports modified core files, that’s useful.

But don’t blindly overwrite everything.

First understand whether the modification was intentional.


12. Don’t forget WordPress administrators

Once a week, I’d save:

wp user list \
--role=administrator \
--path=/home/customer/public_html

Then compare.

A new administrator should be explainable.

If it isn’t:

stop and investigate.


13. Plugin inventory is another useful baseline

wp plugin list \
--path=/home/customer/public_html

And themes:

wp theme list \
--path=/home/customer/public_html

A new plugin is not necessarily an intrusion.

But a new plugin installed at 3 AM by nobody you know?

That’s a very different conversation.


14. I would monitor web traffic for patterns, not individual IPs

This is another lesson from the incident.

Attackers use:

Cloudflare
VPNs
proxies
compromised servers
rotating VPSs
botnets

So I wouldn’t build my security model around:

“Block this one IP.”

That can help temporarily.

But it isn’t a defense.

Instead, watch for behavior:

POST bursts
upload attempts
wp-admin probing
strange PHP paths
plugin installation attempts
unexpected REST requests

Behavior survives IP rotation.


15. Don’t mistake scanners for successful attacks

This distinction is extremely important.

Suppose your log contains:

POST /xmlrpc.php

That doesn’t mean:

“The attacker compromised WordPress.”

It means:

“Someone sent a POST request to xmlrpc.php.”

You need more evidence.

Likewise:

GET /.env

doesn’t mean:

“.env was stolen.”

It may simply have returned:

404

Always look at the response.


16. Status codes matter

When reviewing logs:

404
403
401
200
500

mean very different things.

For example:

GET /random-admin-panel.php 404

is a probe.

While:

POST /upload.php 200

is much more interesting.

Even then, 200 doesn’t prove compromise.

But it tells you where to look next.


17. Response size can also be useful

Suppose the same request appears hundreds of times:

POST /xmlrpc.php 200 300

That doesn’t necessarily indicate successful authentication.

If the response size is identical every time, it may simply be automated background traffic.

Logs are evidence.

They’re not a verdict.


18. I would keep logs longer than the incident

This is one of the easiest improvements.

If your logs disappear after a short rotation period, investigating something that happened two weeks ago becomes much harder.

Keep enough history to answer:

What happened?
When did it start?
When did it stop?
What happened immediately before the first suspicious file appeared?

Disk space is usually cheaper than lost evidence.


19. But don’t keep everything forever

Log retention has a cost.

For a small server, I would choose a sensible period rather than keeping years of Apache logs.

Something like:

recent detailed logs
+
older compressed logs
+
important security events retained longer

is usually a reasonable compromise.


20. The monitoring system should produce a small report

This is the important part.

I don’t want:

12 MB log file

every morning.

I want:

Daily VPS Security Report

New PHP files:       2
Fake image files:    0
New executables:     0
New cron entries:    0
SSH key changes:     0
New MySQL users:     0
New WP admins:       0
New listening ports: 0

ATTENTION:
customer01/public_html/wp-content/uploads/x.php

Now I can decide whether I need to investigate.


21. Alert only when something changes

This is the philosophy I would use.

Don’t tell me every day:

“Everything is normal.”

Tell me:

“Something changed.”

That’s much easier to live with.


22. The monitoring script itself must be protected

There is an ironic problem with server monitoring:

If the attacker gets root, they can modify the monitoring system too.

So don’t rely on the compromised server as the only source of truth.

Important reports should ideally be copied somewhere else.

For example:

VPS
 │
 ├── security report
 │
 ▼
separate machine/storage

If the VPS is compromised, the historical reports still exist elsewhere.


23. Backups and monitoring solve different problems

It’s tempting to think:

“I have backups, so I’m safe.”

No.

Backups answer:

Can I recover?

Monitoring answers:

Did something change?

Security controls answer:

Can the attacker do it?

You want all three.

Prevention
    +
Detection
    +
Recovery

24. My preferred small-server security model

For my kind of environment, I’d rather have:

GOOD BACKUP
     +
GOOD ISOLATION
     +
LEAST PRIVILEGE
     +
PRIVATE DATABASE
     +
SSH HARDENING
     +
FILE MONITORING
     +
LOG RETENTION

than install twenty security products I don’t understand.

Simple systems are easier to maintain.

And if I can’t maintain it, eventually it stops being secure.


25. The one thing I would automate first

If I only had time to automate one thing, it would be:

new PHP files.

Something as simple as:

find /home \
-type f \
\( -name '*.php' -o -name '*.phtml' -o -name '*.php5' -o -name '*.php7' \) \
-newermt '24 hours ago' \
-print \
2>/dev/null

Then send the result somewhere I actually read.

That alone can dramatically shorten the time between:

attacker writes file

and:

I know something happened.

The real metric: time to detection

Before the incident:

attacker
   ↓
compromise
   ↓
files deployed
   ↓
...
   ↓
days later
   ↓
I notice

That’s bad.

A better system is:

attacker
   ↓
compromise
   ↓
file deployed
   ↓
monitor detects
   ↓
alert
   ↓
investigate

Even if the attacker still manages to get in, reducing that window is valuable.


And that changes how I think about security

I used to think:

“How do I make sure nobody ever gets in?”

Now I think in four questions:

1. Can they get in?

Reduce vulnerabilities and exposed services.

2. If they get in, what can they reach?

Use isolation and least privilege.

3. How quickly will I know?

Monitor meaningful changes.

4. If everything goes wrong, can I recover?

Maintain independent, tested backups.

That’s a much more realistic security strategy.


The complete model

                 ATTACK
                    │
                    ▼
             ┌─────────────┐
             │ Can enter?  │
             └──────┬──────┘
                    │
              prevention
                    │
                    ▼
             ┌─────────────┐
             │ Can spread? │
             └──────┬──────┘
                    │
                isolation
                    │
                    ▼
             ┌─────────────┐
             │ Detect it?  │
             └──────┬──────┘
                    │
                monitoring
                    │
                    ▼
             ┌─────────────┐
             │ Recover?    │
             └──────┬──────┘
                    │
                 backups
                    │
                    ▼
                  DONE

That’s what I want from a small-business VPS.

Not perfection.

Containment, detection, and recovery.

Because eventually something will go wrong.

The trick is making sure that when it does, it stays a problem you can fix rather than becoming the problem that shuts down everything you run.

No Comments

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.