G0

Troubleshooting "How To" Manual for the SMS

The main troubleshooting methods all stem from the alarm control page in the control system computer. This document will serve to define what actions need to be taken when a certain alarm goes off on the "Alarm Control" panel on LookoutDirect running on the SMS control system computer. The shift crew will not have access to the Hall during beam time therefore this document is limited to what the crew can do remotely with the control system.

The control system is on the computer g0-sms.jlab.org. This system can be accessed remotely using virtual network computing (VNC) from a Java-enabled web browser. Go to http://g0-sms.jlab.org:5800 and use the generic G0 password to log on. A special VNC viewer client can be used instead.

As a general rule, a shift crew should not push any buttons or close/reboot any software without consulting a magnet expert first. The on-call magnet expert can always be reached via pager at number 584-5574.
The exception to this rule would be the buttons needed to navigate between Panels. (e.g. Main, Alarm Control, CDHXR, Interlocks, etc. located toward the top of the control system screen).
When an alarm goes off, and the problem has been solved, the alarm can be silenced by going to: Alarms (pull down menu) -> Acknowledge All. Then go to the Alarm Control page and press the Reset button for at least one second.

A quick way to find the relevant troubleshooting section is to ctrl-f (alt-f on unix/linux) and search for the alarm number



  • 1212 H2O Temperature A Low
  • 1222 H2O Temperature B Low
    This is the water cooling for the A and B leads. A low temperature warning indicates that the water in the leads near the top of the magnet may be in danger of freezing.

    1. Go to the Water page and observe the water temperatures in the appropriate strip chart.
    2. If the temperature drop was not a temporary event, then notify the on call magnet expert.



  • 1311 H2O Flow A Low
  • 1321 H2O Flow B Low
  • 1331 H2O Flow Jumper Low
  • 1341 H2O Flow PS Low
    This shows the status of the water flow through the Power supply, A and B leadds, and the Jumper leads. When the flow stops or is reduced below a certain setpoint, there is danger of overheating.

    1. This alarm should cause a slow dump of the current in the magnet. Verify that the slow dump has occurred by going to the Power Supply page and watching the current ramp down.
    2. Go to the Water page and observe the water flows in the appropriate strip chart.
    3. Notify the magnet expert.



  • 2111 LHe Inlet Pressure Low
    If LHe Inlet Pressure is too low, this could mean that there is reduced LHe flow going into the magnet. In time, this could cause the He in the reservoir to be consumed and the coil temperatures to rise.

    1. Go to the LHe Level page and observe the He inlet pressure in the appropriate strip-chart.
    2. If the He inlet pressure does not rise back to its original value within a couple of minutes or continues in a steep decline, then notify the on-call magnet expert.



  • 2131 LHe Reservoir Pressure Low.
  • 2140 LHe Reservoir Pressure High.

    1. Go to the LHe Level page and observe the He reservoir pressure in the appropriate strip-chart.
    2. If the He reservoir pressure does not return to its original value within a couple of minutes or continues in a steep decline/incline, then notify the on-call magnet expert.



  • 2211 CDHXR Temperature High.
    Once we are cooling with LHe near 4.5 K, we won't be using the Cooldown Heat Exchanger. This is used when we are cooling from 300K to ~100K.

    1. Ignore alarm, but make sure the magnet expert gets notified during normal working hours.



  • 2221 LHe Inlet Temperature High.
    This, along with "2111 LHe Inlet Pressure Low" could mean trouble with the LHe flow.

    1. Go to the LHe Level page and observe the He inlet temperature in the appropriate strip-chart.
    2. If the He inlet pressure does not rise back to its original value within a couple of minutes or continues in a steep decline, then notify the on-call magnet expert.



  • 2311 VCL A Flow Low
  • 2321 VCL B Flow Low

    1. Go to the Lead page and observe the lead flows in the appropriate strip-chart.
    2. If the lead flow does not immediately rise to its original value, then notify the on-call magnet expert.



  • 2401 LHe Level Low (Yellow)
    The reduced level of LHe that causes this alarm could be the sign of a problem somewhere in the cryogenic system or it could be due to a normal fluctuation of the PID loop.

    1. Go to the LHe Level page and observe the LHe level in the appropriate strip-chart.
    2. Monitor the level for several minutes.
    3. If the level does not increase above the threshold, then notify the on-call magnet expert.



  • 2402 LHe Level Low (Orange)
    Alarm 2401 should have signalled prior to this alarm. If the liquid level gets this low, then there is almost certainly a problem in the cryogenic system or the ESR.

    1. This alarm should cause a slow dump of the current in the magnet. Verify that the slow dump has occurred by going to the Power Supply page and watching the current ramp down.
    2. Go to the LHe Level page and observe the LHe level in the appropriate strip chart.
    3. Notify the magnet expert.



  • 3131 LN2 Reservoir Pressure Low
  • 3140 LN2 Reservoir Pressure High This alarm could indicate a problem in the liquid nitrogen circuit or it could arise from natural fluctuations in the PID control loop.

    1. Go to the LN2 Level page and observe the LN2 reservoir pressure in the appropriate strip chart.
    2. If the LN2 reservoir pressure does not return to its original value within several minutes, then notify the on-call magnet expert.



  • 3211 LN2 Bayonet Temperature High
    This can happen if the JT5 valve position is closed down (not necessarily to 0) As the valve is closed, the LN2 bayonet begins to warm up since there is reduced LN2 flow. This happens occasionally as the PID loop needs to close the valve in order to reduce the LN2 reservoir level.

    1. Go to the LN2 Level page and observe the LN2 bayonet temperature in the appropriate strip chart.
    2. If the LN2 bayonet temperature does not start to drop within a few minutes, then notify the on-call magnet expert.



  • 3400 LN2 Level Low

    1. Go to the LN2 Level page and observe the LN2 level in the appropriate strip chart.
    2. If the LN2 level does not begin to rise within a few minutes, then notify the on-call magnet expert.


  • 4111 Vessel TC Pressure High
  • 4121 Vessel CC Pressure High
  • 4211 Cryobox/LN2 Bayonet Pressure High

    1. Go to the Vacuum page and observe the vacuum in the appropriate strip chart.
    2. If the vacuum does not drop to normal levels within a few minutes, then notify the on-call magnet expert.



  • 25 Voltage Transition Lead A alarm (Yellow)
  • 28 Voltage Transition Lead B alarm (Yellow) These alarms occur if the voltage measured in the transition leads is slightly higher than the nominal value; indicating a potential quench situation.

    1. This alarm should cause a slow dump of the current in the magnet. Verify that the slow dump has occurred by going to the Power Supply page and watching the current ramp down.
    2. Go to the Quench Protection page and observe the lead voltage in the appropriate strip-chart.
    3. Notify the on-call magnet expert.



  • 26 Voltage Transition Lead A alarm (Orange)
  • 29 Voltage Transition Lead B alarm (Orange)
  • 31 - 38 Voltage Coil A-H Quench
    This alarm indicates that one or more of the coils or transition leads has gone normal, or quenched.

    1. This alarm should cause a fast dump of the current in the magnet. Verify that the fast dump has occurred by going to the Power Supply page and checking the value of the current.
    2. If the current has not gone to zero within a few seconds and the dump switch is still closed (as noted on the Power Supply page), then push the Open Dump Switch button to trigger a fast dump.
    3. Notify the on-call magnet expert.



  • 8210 Temperatures Not Updating
    This occurs occasionally with the temperature monitoring (Tempmon) software installed on the control system computer. Sometimes, the USB <-> Serial converter that the Temperature readings are fed through can stop. Thus the remedy is to reboot the USB <-> Serial converter. This process has been automated so generally no action will need to be taken.

    1. Acknowledge the alarm and reset as normal.
    2. If alarm occurs again within five minutes, then notify the on-call magnet expert.






    Send additions, corrections, and updates to
    Kaz Nakahara.