What a Ransomware Recovery Taught Me About Technology Leadership

I was brought in as a consultant to a small produce company after ransomware had already taken down nearly everything that mattered, at the tail end of a COVID summer that only made things harder. I stayed on, operating in a head-of-technology capacity, through the recovery that followed.

Within the first ten minutes, I knew we weren’t going to simply restore what had been there before.

The domain controller, storage, and print server were all encrypted. Desktops were compromised. QuickBooks was down. What backups existed were either encrypted too or so old they were nearly useless. The most recent were QuickBooks backups more than six months old, and desktop data wasn’t backed up at all. The infrastructure itself was old, unpatched, and in places no longer supported.

The previous IT person had said some of this was being handled. It wasn’t.

That mattered, because the ransomware was only part of the problem. The attack had exposed an environment that had been operating with too much accumulated risk for too long.

The company wasn’t especially large, but the work ahead looked like what a much larger organization would face after a serious incident. Identity, networking, endpoint security, backups, accounting, access control, monitoring, patching. None of it came with a large team.

That realization shaped nearly every decision that followed. We were going to start over, and we were going to keep it simple.

Treat the business like an emergency room

Once the decision to rebuild was clear, the next question was what to do first.

During a crisis, everything sounds urgent, and the easy mistake is to start restoring technology in whatever order makes the most sense technically. That’s not how I approached it.

I thought about it more like triage in an emergency room. What’s the current condition? What can be restored quickly? What will have the greatest business impact? What can wait? What will let the next part of the organization start functioning again?

Those questions drove the order.

Email came first, not because it was the most sophisticated system, but because it was a fast win with a large impact. The environment was hosted offsite, so webmail still worked. We reset passwords, enabled MFA, and reestablished a trusted channel to coordinate with employees, then customers. In a disruption like that, restoring trust and information flow matters almost as much as restoring applications.

Identity came next. The old Active Directory environment was compromised, so we didn’t try to preserve it. New domain, new credentials, password resets, MFA, new access rules, no unnecessary external access. The goal wasn’t to recreate what had failed, but to build a foundation that was easier to trust.

After that, accounting became the priority, collecting money, paying vendors and employees, sending invoices, understanding what was happening financially. These mattered more, in the moment, than systems that would normally rank higher on a traditional IT recovery list.

That sequence taught me something I still believe. Recovery priorities should be based on business capability, not infrastructure diagrams. The question isn’t “which server should come back next?” It’s “what does the business need to be able to do next?”

Rebuild with a different philosophy

The condition of the environment made the decision easier. Devices needed replacing anyway, servers were outdated, the access model was weak, and there was enough accumulated technical debt that preserving any of it meant rebuilding the same problems we were trying to escape.

So we treated the recovery as a chance to reset, not to redesign everything for elegance while the business was down (a crisis is not the time for architecture theater), but to make deliberate choices about what we wouldn’t bring back.

One of those choices was complexity. The company was small enough that every unnecessary variation created ongoing support work. Different computer models, configurations, security settings, user privileges. So we standardized aggressively. Same operating standards, endpoint protection, patching approach, backup expectations, monitoring. That may sound boring. It was exactly what we needed.

A small organization doesn’t get to skip enterprise-class responsibilities just because it has fewer people. It still has to secure identities, protect endpoints, maintain backups, patch systems, monitor threats, and recover from failures, just without a large team to maintain the complexity. A large enterprise spreads that work across dedicated teams; a small one might have one person responsible for nearly all of it. That means the right architecture isn’t the most powerful or customizable one, but the one that’s consistent, observable, and simple enough that one person can still understand it six months later.

Security itself needed reframing. If it makes every normal task harder, people find ways around it. The better approach is to design the environment so the secure path is also the easy path. We removed local administrative rights from every user, including me. My daily account had no admin access; I used a separate one when I needed elevated privileges. We rebuilt wireless around RADIUS authentication, with guest access walled off from the internal network. We replaced ad hoc password habits with a password manager. It makes the secure choice the easy one.

The same principle repeated everywhere. Endpoint protection ran without anyone thinking about it. Patching happened without depending on memory. Backups ran themselves. Alerts fired the moment something failed. Good security reduces the number of bad decisions people are asked to make in the first place.

Security became part of normal operations

Before the incident, many of the controls I’d consider foundational either didn’t exist or weren’t maintained consistently. That changed after the rebuild, but the addition that mattered most wasn’t any single control. It was visibility into whether the things we depended on were actually working.

One of the problems before the incident wasn’t only that controls were missing. It was that leadership believed things were happening because someone said they were. Good IT creates evidence. If backups are working, patching is current, or systems are protected, I should be able to show you, not just tell you. And if an important control fails, I should know before someone has to ask.

That’s not about mistrust. It’s about managing technology without depending entirely on verbal assurance.

Trust people. Verify systems.

The business crisis was bigger than the technology crisis

The ransomware incident would have been difficult enough on its own. It wasn’t happening in isolation.

The owners themselves were dealing with serious personal health emergencies, unrelated to the ransomware but very much affecting how available they could be. At the same time, the company’s banking was compromised through stolen mail and checks. Checking activity had to be frozen, new accounts established, positive pay put in place.

The organization was being hit from multiple directions at once, changing the nature of the leadership challenge. This was no longer just about restoring IT systems. The business was trying to reestablish normal operations while ownership was largely unavailable and major financial controls were being rebuilt at the same time. Decisions had to be made asynchronously through an employee who could coordinate with the owners, and I needed room to act without waiting on sign-off for every call.

That room didn’t come from a signed authorization. It came from phone calls and honest conversations, everyone involved wanting the same thing, for this to work. Trust built through character, not legalese, mattered more than any formal delegation would have. A crisis gets worse when every decision waits on someone unavailable, and the people closest to the problem need to be trusted to decide and know when to check in. Otherwise, the work stops while everyone waits.

The hardest decision wasn’t technical

The most difficult decision during the recovery was whether to pay the threat actor, and what to negotiate toward. There were no usable backups that could get the business back to where it needed to be. We brought in a firm that specialized in ransomware recovery and negotiation, though in practice we worked with a single person from that firm, not a team. The call had to be made quickly, and it was mine to make, backed by nothing more formal than the trust we’d already built.

It wasn’t a comfortable decision, and there was no waiting for perfect information. What surprised me was how transactional the process became once an agreement was reached. To the threat actor, this was a business. After payment, they were motivated to make the recovery work. It took several attempts, but they kept troubleshooting almost like a technical support organization. Some nights, an alert would come in and a decision had to be made immediately, because waiting could stall the process.

That reinforced something about leadership that goes well beyond cybersecurity. Sometimes you have to make an important decision with incomplete information, inside a defined risk boundary, and then own the result. You can get advice, gather evidence, ask experts. At some point, someone still has to decide.

What struck me, in a good way, was how well the team pulled together. They worked manually, slower and less efficient, but found ways to keep the business moving instead of panicking. I’ve seen that elsewhere too. Teams get remarkably effective when the priority is unmistakable. The challenge for leaders is bringing that clarity into normal operations without needing a crisis to create it. People perform better when they know the priority, who owns what, and that they have the authority to act. That shouldn’t require ransomware.

Maintenance is cheaper than heroics

The most important lesson came after the emergency was over.

Keeping technology healthy is boring. Patching. Replacing hardware before it fails. Testing backups. Reviewing permissions. Keeping documentation current. Monitoring alerts. Standardizing devices. None of it makes anyone feel heroic, and all of it costs far less than rebuilding an environment in the middle of a crisis.

Organizations delay maintenance because nothing appears broken. A server still runs, so replacement waits another year. Backups appear to exist, so nobody tests them. A system is out of support but still turns on every morning. A process depends on one person who’s still there. For now. The absence of failure creates a false sense of safety. The cost doesn’t go away. You’re just accumulating it.

That changed how I think about IT. The goal isn’t the minimum required to keep things running. It’s spending enough time maintaining and improving the environment that you’re not paying interest on old decisions. Replacing systems before they become emergencies, testing assumptions, automating what can be automated, removing complexity. That’s cheaper, and a lot less work.

Good IT should usually be boring

We eventually recovered the data and got the business running again. But what stayed with me was how much of the pain could have been avoided through ordinary, consistent maintenance.

Years later, that business is thriving. They’ve kept building on the core technology and processes we put in place, adding traceability along the way, and outgrew their old space enough to move into a larger building.

The recovery was dramatic. The work that prevents that kind of recovery is not. Good IT is usually quiet. Systems patched. Backups tested. Old hardware replaced on schedule. Accounts reviewed. Admin access limited to people who need it. Monitoring catching failures before anyone asks. The network segmented. The environment documented. People do the right thing because the right thing is easy. None of that makes for an exciting story. That’s exactly the point.

The best recovery project is the one you never need.

If I had to reduce the whole experience to one lesson, it’s this: build technology so the organization can operate securely without thinking about it every minute, keep the environment simple enough to maintain, and spend the boring time doing the work that keeps tomorrow from becoming an emergency.

Starting over after ransomware taught me a lot about incident response. It taught me even more about what good IT should look like before the incident ever happens.