The Restore Is Not the End of the Incident

What I checked after the websites came back online

By SepedaTua — CrushEdge.com


There is a very satisfying moment during a server recovery.

You start Apache.

You start MariaDB.

You open the browser.

And:

WordPress is back.

After two days of downtime, that feels like the finish line.

It isn’t.

It’s more like the point where I can finally start checking whether the recovery actually worked.


1. First: don’t immediately change everything

Once the restore is complete, I want a stable reference point.

Before installing random security plugins, changing ten PHP settings, and running twenty cleanup commands, I want to know:

What exactly did I restore?

That means recording:

date -u
hostname
uptime
ss -lntup

and keeping those results.

If something goes wrong later, I know what the server looked like immediately after recovery.


2. Check the files before opening the doors

If possible, I prefer to do the first inspection while the web server is still stopped.

For example:

systemctl status httpd
systemctl status mariadb

The exact service names depend on the distribution.

Then inspect the restored files.

The first thing I want to know is:

Did the suspicious files that caused the incident exist in this backup?

That question is much more useful than scanning blindly.


3. Compare suspicious filenames against the backup

If I previously identified something like:

bf6f03.php

I can check the restored filesystem:

test -e /home/customer/public_html/bf6f03.php \
    && echo "FOUND" \
    || echo "NOT FOUND"

If it says:

NOT FOUND

that’s good.

But it isn’t proof that the site is clean.

It’s simply one confirmed indicator that is no longer present.


4. Scan the restored site for PHP

This is my first broad scan:

find /home/customer/public_html \
-type f \
\( -name '*.php' -o -name '*.phtml' -o -name '*.php5' -o -name '*.php7' \) \
-printf '%TY-%Tm-%Td %TH:%TM:%TS | %s | %p\n' \
2>/dev/null |
sort

I’m not looking for a particular filename.

I’m looking at the shape of the filesystem.

Does PHP appear where I expect it?

Are there strange random filenames?

Are there PHP files buried inside image directories?


5. Then I check the contents

For suspicious PHP:

grep -RInE \
'base64_decode\s*\(|gzinflate\s*\(|gzdecode\s*\(|str_rot13\s*\(|eval\s*\(|assert\s*\(|shell_exec\s*\(|passthru\s*\(|proc_open\s*\(|popen\s*\(' \
/home/customer/public_html \
--include='*.php' \
--include='*.phtml' \
--include='*.php5' \
--include='*.php7' \
2>/dev/null

Again, this is a search, not a verdict.

A legitimate plugin can contain one of these functions.

The combination of:

random file
+
obfuscation
+
remote input
+
command execution

is what makes my eyebrows go up.


6. Then I check the images

This was one of the more interesting findings in the original investigation.

So I would absolutely repeat the test after restoration:

find /home/customer/public_html -type f \
\( -iname '*.jpg' -o -iname '*.jpeg' -o -iname '*.png' \
   -o -iname '*.gif' -o -iname '*.webp' -o -iname '*.ico' \
   -o -iname '*.bmp' -o -iname '*.avif' \) \
-print0 2>/dev/null |
while IFS= read -r -d '' f; do
    t=$(file -b "$f")

    case "$t" in
        *"PHP script"*|*"HTML document"*|*"ASCII text"*|*"Unicode text"*)
            printf '%s | %s\n' "$f" "$t"
            ;;
    esac
done

If the result is empty:

good.

If it returns something:

investigate it.


7. Don’t get fooled by legitimate text images

This is where context matters.

Some libraries contain files that file identifies as:

ASCII text

even though they are legitimate assets.

For example, documentation or generated image data can sometimes look unusual.

That’s why I don’t use:

ASCII text = malware

as my rule.

The useful rule is:

An image extension combined with executable PHP content is highly suspicious.

That’s different.


8. Check WordPress core

If WP-CLI is available:

wp core verify-checksums \
--path=/home/customer/public_html

For a normal WordPress installation, this can catch modified core files.

If it reports a mismatch, don’t immediately overwrite everything.

Look at the file first.


9. Check plugins

wp plugin list \
--path=/home/customer/public_html

Then ask:

Do I recognize every plugin?
Is it actually needed?
Is it current?

The same applies to themes:

wp theme list \
--path=/home/customer/public_html

10. Check WordPress administrators

wp user list \
--role=administrator \
--path=/home/customer/public_html

I want every administrator to have a reason for existing.

If there is a strange account, investigate:

When was it created?
Who owns it?
Was it legitimate?
What email address does it use?

Don’t simply delete an unfamiliar account without checking.


11. Check scheduled tasks again

A restored filesystem doesn’t necessarily mean the entire server is clean.

Check:

crontab -l

Then:

find /etc/cron.d /etc/cron.daily /etc/cron.hourly \
/etc/cron.weekly /etc/cron.monthly \
-type f \
-print \
2>/dev/null

And systemd:

systemctl list-unit-files --type=service

This is particularly important if the original compromise involved server-level access.


12. Check SSH keys

find /root /home \
-path '*/.ssh/authorized_keys' \
-type f \
-print \
2>/dev/null

Every key should be explainable.

If you have a known-good baseline, compare against it.

That’s much better than relying on memory.


13. Check MySQL accounts

After the restore, I would inspect this again:

SELECT User, Host
FROM mysql.user
ORDER BY User, Host;

Then inspect anything remote:

SHOW GRANTS FOR 'username'@'host';

This is where I want to answer:

Which database accounts can connect from outside the server?

Not:

“I think there was one remote account somewhere.”


14. Check whether MySQL is publicly listening

ss -lntp | grep ':3306'

If I don’t specifically need public MySQL access, I want something like:

127.0.0.1:3306

not:

0.0.0.0:3306

Then temporary applications can use an SSH tunnel instead.


15. Rotate credentials after recovery

This is one of the easiest things to postpone.

Don’t.

If the server was compromised, assume that credentials stored on it may have been exposed.

Depending on the environment, that includes:

WordPress passwords
database passwords
SSH credentials
Virtualmin credentials
API keys
SMTP credentials
cloud credentials
third-party service tokens

You don’t necessarily need to rotate every credential in existence.

Rotate the ones that could realistically have been accessed from the compromised environment.


16. Check outbound connections

After the site is live:

ss -tpn

Look for unexpected outbound connections.

Then:

ps auxww

Match suspicious network connections with processes.

If you see:

unknown process
      │
      └──> persistent external connection

that’s worth investigating.


17. Watch CPU after the restore

This is a simple but useful sanity check:

top

or:

ps -eo pid,user,%cpu,%mem,etime,args \
--sort=-%cpu |
head -30

A newly restored WordPress site shouldn’t suddenly have an unexplained process consuming all available CPU.


18. Watch the logs when traffic resumes

Once Apache/Nginx is running again, watch the access log.

For example:

tail -f /path/to/access_log

Then visit the site yourself.

You should see your request.

Now you have a live reference:

browser
   ↓
web server
   ↓
PHP
   ↓
WordPress
   ↓
database

If something unexpected happens, you’ll see it in context.


19. Don’t mistake bots for another compromise

Once the server is public again, the Internet will immediately start poking it.

You’ll probably see requests for:

/wp-login.php
/xmlrpc.php
/.env
/.git/config
/wp-admin/
/phpinfo.php

That is normal Internet background noise.

The question isn’t:

“Did someone request /wp-login.php?”

The question is:

“Did something successful happen after the request?”

That’s a much better signal.


20. The first few hours matter

After restoration, I would pay particular attention to:

new PHP files
new executables
new WordPress administrators
new plugins
new cron jobs
new systemd services
new SSH keys
new database accounts
unexpected outbound connections
unexpected CPU usage

If those remain quiet, confidence increases.


21. Then create a clean baseline

Once everything looks good:

ss -lntup \
> /root/baseline-ports.txt
systemctl list-unit-files \
--type=service \
> /root/baseline-services.txt
crontab -l \
> /root/baseline-root-cron.txt

For important websites, save plugin/theme/user inventories too.

This gives future-you something extremely valuable:

a known-good snapshot of what “normal” looks like.


22. The baseline is more valuable than another scanner

A scanner can tell you:

“This looks suspicious.”

A baseline can tell you:

“This wasn’t here yesterday.”

That distinction is incredibly useful.


23. And don’t forget the backups

Once you’re comfortable that the restored server is clean, make sure the new state is backed up.

But don’t immediately overwrite your historical backups.

Keep the old backups.

They are evidence and recovery points.

Your new clean backup becomes:

known-good-current

while the historical backups remain:

known-good-history

until you’re confident you no longer need them.


24. The recovery cycle I would use next time

If I ever have another incident, my workflow is now:

             SUSPICION
                 │
                 ▼
             CONTAIN
                 │
                 ▼
          PRESERVE EVIDENCE
                 │
                 ▼
           IDENTIFY SCOPE
                 │
                 ▼
          FIND CLEAN BACKUP
                 │
                 ▼
              RESTORE
                 │
                 ▼
         VERIFY RESTORE
                 │
                 ▼
        ROTATE CREDENTIALS
                 │
                 ▼
             HARDEN
                 │
                 ▼
             MONITOR
                 │
                 ▼
             BASELINE
                 │
                 ▼
               DONE

That’s much less stressful than trying to improvise everything while the server is down.


25. What “clean” actually means

I don’t think I’ll ever say:

“This server is 100% guaranteed clean.”

That’s too strong.

Instead, I’ll say something like:

“I restored from a backup created before the known compromise, removed the known indicators, verified WordPress core, checked persistence mechanisms, reviewed database and SSH accounts, restricted exposed services, rotated relevant credentials, and monitored the server after restoration.”

That’s a defensible statement.

Security isn’t about pretending uncertainty doesn’t exist.

It’s about reducing it systematically.


The final lesson from the restore

The restore itself was surprisingly simple.

The difficult part was everything surrounding it.

Before the incident, I thought:

backup = recovery

Now I think:

backup
+
evidence
+
verification
+
credential rotation
+
hardening
+
monitoring
=
recovery

That is a much better definition.

Because getting the homepage to load is easy.

Getting back to a state where I can reasonably trust the server again is the real job.

And once I have done that, I don’t want to spend another night thinking about the attacker.

I want to get back to writing code.

Preferably before my coffee gets cold again.

No Comments

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.