Voir en

français

Technical troubleshooting at the CERN Control Centre

A team is on call 24 hours a day to monitor technical systems and quickly respond to alerts across CERN's accelerator complex

|

Technical troubleshooting at the CERN Control Centre

Chris Wetton of CERN's Technical Infrastructure Operation group keeps a keen eye on the monitors at the CERN Control Centre (Image: Stephanie Hills)

A longer version of this article first appeared in UK News from CERN.

If a technical alert goes off at CERN, it will almost certainly show up on one of Chris Wetton’s 15 screens in the CERN Control Centre. Wetton is a member of the Technical Infrastructure Operation group, an eight-strong team that works 24/7 to monitor all the services for CERN’s accelerator complex. A system of sensitive triggers throughout the complex sends an alert as soon as any deviation is detected so the team can react quickly.

The bank of screens is intimidating – there are screens with schematic diagrams and synoptics of the electrical network, and on others, lines of text in different colours indicate the priority level of an alert from zero (white/grey  – information only) to three (red  – requiring immediate intervention from the Fire Service). Fortunately there are no red lines – the screens are mostly yellow indicating level two technical issues, and after nearly 10 years in the job, Wetton quickly scans the screens to calmly assess what is going on.

“We monitor everything from the fire alarms, gas, heating and ventilation systems to the evacuation systems and automatic doors,” he says. “If an alert goes off, we identify the problem and then try to fix it.”

Wetton says that level two alerts are the most common during a technical stop when people are working on the machines. "We need to know who is working where, what they’re doing, and what alerts they might trigger. Most people will call us up before they start, but not always," he says. "When the accelerators are working it’s reasonably calm, but at the moment, we’re taking 200-280 phone calls every day.”

During CERN's first long shutdown (LS1) in 2013, a staggering 1.8M alerts were registered (doubling the total from 2012). Each one had to be dealt with. During normal operations, there is one person on shift, but for LS1, the team has needed extra resources and volunteers were recruited from other parts of CERN to assist the team. The experience has given them a valuable insight into the complexities of CERN and its operations.

Having such an in-depth knowledge of CERN means that the team is often seen as the first point of contact for troubleshooting any incident whether it involves the accelerator complex or not, and this can have its lighter moments; “A few days ago we took a call advising us that the sheep that graze some of the land inside the main CERN site had escaped. Sorting out livestock definitely isn’t our responsibility, but actually we do have the farmer’s phone number, and we did solve the problem!”

Aside from farming, Wetton and his colleagues come from a variety of professional backgrounds – cryogenics, electrical systems, cargo ships and submarines. He himself started out as an apprentice maintaining high vacuum, radiofrequency, cryogenics and metrology equipment in a semiconductor fabrication cleanroom for Philips. “CERN uses essentially the same technology, just on a huge scale!” he says. “It’s the hands-on interventions that I enjoy; we’ve been given more responsibility and more chances to go on site in LS1 so I hope that will continue.”