An autopsy of the cyber casualty at Holby City

The popular BBC medical dramas Casualty and Holby City recently showed a special two-part crossover where the storylines of the two shows intertwined as a computer virus hit the hospital. So, for the first time in many years, I decided to tune in to watch (purely from a professional perspective obviously). Having survived two episodes relatively unscathed, here's my list of lessons learnt.

The first thing that struck me was that Charlie is still working, I’m pretty sure he must have been close to retirement when I last watched the show back in about 1995. Sadly, there was no Duffy though. Anyway, back to the plot. Within minutes of tuning in, one of the hospital staff was experiencing issues with their computer and the usual frustrations ensued. This is quickly followed by an alarm sounding on the machine of one of the patients and the doctors and nurses’ race to action.

The patient begins to come around during efforts to resuscitate and the medical team realised that the machine is giving false readings. It’s beginning to dawn on everyone that there’s something seriously wrong with the computer systems. Cue the first in several lessons businesses can learn from this fictional attack.

Lesson one

Computer systems are great, they make life better and easier, but only when they work properly. Computer readings need to be sense checked. If you have a critical system running on a network you can’t rely 100% on the output. An alert should always be confirmed by a second source, human or otherwise.

Back at Casualty, within 10 minutes of recognising an issue, a meeting of the department heads is called. This would seem to suggest that the Hospital has an incident response plan in place. Gold star for this. This crucial plan ensures the team is primed to respond quickly and with confidence that the processes of the business (in this case the hospital) can continue as smoothly as possible. Tell that to a panicked Charlie as the extremes of the situation are played out in thousands of living rooms up and down the country.

Lesson two

Most businesses have disaster recovery and incident response plans in place for fire, flood or even staff absence. It is vital that plans for computer failures or cyber-attacks are also established, after all the fall out of either of these is every bit as serious. A response plan must be agreed and led by senior level management. The IT department will be entirely focused on getting systems back up and running and won’t to be able to effectively manage this process. A nominated commanding officer needs to have responsibility for ensuring that the response plans are put in place to keep the organisation running as effectively as possible.

This being a TV show it is inevitable that things will not run smoothly. The incident response meeting is held in the reception area and it is announced that the system has been hit with a virus which means staff cannot trust any monitoring or access any patient data. Everything on the network has been affected.

Lesson three

At Holby, it appears that several critical and non-critical systems are all running on the same network. BIG mistake. Businesses really need to understand the importance of ‘segmenting’ networks. There should never be critical systems running on the same network as other not so critical systems. Had this been the case in Holby, then the impact of this cyber-attack would certainly have been lessened.

Meantime, there is another storyline running in which one of the paramedics hasn’t turned up for work and a concerned colleague goes to their house to find out what’s happening. When they arrive, the door is locked, and the paramedic finds a rock with the spare key to the house hidden in it. However, the ‘rock’ has a combination lock on it. It turns out that the combination is ‘3006’ which just happens to be the crew number of the missing paramedic.

Lesson four

This is essentially a form of social engineering. It can often be all too easy to guess pin numbers, combinations and even passwords by using information that is known about a person. Take pin numbers and combinations for example. If I wanted to ‘guess’ someone’s combination, the first things I would try would be the intended victim’s birthday followed by their partners and children’s birthdays. If none of them worked I would then try other numbers that may mean something to them, so maybe the year that their favourite football team was founded or last won a cup, or if they were a paramedic, their crew number. The lesson? Businesses must establish a password policy which is underpinned by staff training.

Back at the hospital, the staff are told that manual patient monitoring is the only option. This instruction is met with shock from most of the staff (except Charlie who predated any technology!) who are all sent off in search of pens. One medic admits that she has only once recorded manually before.

Lesson five

It might never happen, but organisations need to train their team on how to use manual systems before it is too late. Staff must understand that computer systems can go down, but that business must go on. We train our staff with regular fire drills and even make them run outside every six months to make sure that they know where the fire exit is. We hope that we will never need to evacuate in a real fire but we still run the drills on a regular basis. We should be doing the same with our ‘Cyber’ capability.

Making sure that everyone has pens may seem pretty basic, but this is the type of thing that needs to be considered in the incident response planning exercise. Back in August 2018, Gatwick Airport suffered an IT outage and very quickly staff were in possession of whiteboards and pens to display flight information. Although not ideal, at least they had a back-up plan in place, and it was implemented immediately.

As the story progresses it transpires that the initial infection happened due to a ‘bogus’ email which tricked a user in to clicking a link and submitting their password. The email was said to tell the user that they had been nominated for an NHS award.

Lesson six

Unfortunately, the vast majority of cyber-attacks begin with such a ‘bogus’ or phishing email. It is essential that staff are made aware of the dangers of such emails and how to spot them. The other point that needs to be addressed here is access privileges. At this stage, it appears that the attacker has managed to get access to all areas of the network from one set of captured credentials. Users should only be given access to areas of the network that they need access to. This can help to minimise the impact of any intrusion on a network.

As the program continues it becomes apparent that the incident response plan either does not exist or is not very robust at best. We hear staff suggesting that a whiteboard might be a good idea to establish which patients are being treated, the lines of communication between the staff and management do not seem to be very clear with little or no information filtering to the users, and someone eventually calls for ‘an emergency delivery’ for some supplies. All of this should really have been documented in a plan that could easily be implemented and acted upon when the incident first started.

We also witness staff trying to reboot computers and repeatedly attempting to use equipment that is known not to work.

Lesson seven

Sometimes ‘turning it off and on again’ does do the trick (ask any IT Manager!), but not in this case. During a cyber-attack it is important that the integrity of the data is maintained to help in the post-incident investigation. Repeatedly turning machines on and off may well destroy or corrupt important forensic data and should only be done with the blessing of the IT department.

Towards the end of the Casualty episode the fire alarm begins to sound, and staff are unsure if this is a real alarm or part of the cyber-attack. This shows the vulnerability of being too reliant on technology and again highlights the need to make sure that systems are segregated properly on the network. The IT team appear not to be anywhere near isolating the problem.

As if things couldn’t get worse, the episode ends with the lights going off and doctors being locked out of the new ‘Smart Theatre’.

It’s now Tuesday and there’s still no power, clearly, the disaster recovery plan hasn’t considered a backup. Thank goodness for pagers which were used in the show to make sure that doctors were where they were needed most.

Lesson eight

There is rapidly becoming a blur between traditional physical security and cyber-security.

When considering an incident like this sometimes older technology can really help. The NHS has been told to stop using pagers for communications by 2021, in order to save money. In the context of a backup plan, they’ll have their work cut outsourcing a newer technological replacement that works in all areas of a hospital when connectivity is compromised.

As the episode reaches its climax, we see 2 patients that need life saving surgery but only 1 working theatre. A decision has to be made on who lives. And 2 surgeons are battling it out.

Lesson nine

Although this might be an extreme case, businesses need to identify the most critical parts of the organisation and prioritise its return to operation. This includes ensuring that clear roles are defined and avoids human conflict.

In summary

Holby City was unprepared and had to improvise as the incident unfolded. The upside? Everybody lived to see another day and the hospital walked away with plenty of lessons learnt.

Plan for failure by following these three steps:

  • Prepare an incident response plan
  • Train staff on how to prevent and respond to an attack
  • Review and learn