LN2
From tigwiki
This page describes the common operational features of the automated LN2 systems used at 8π and TIGRESS.
[edit] Overview
The all-attitude LN2 reservoirs on HPGe detectors at a variety of locations:
- 8π, at ILE1A in ISAC-I
- TIGRESS, at SEBT3A/B in ISAC-II
- Room 156 Detector Lab in ISAC-II
The reservoirs on these detectors must be re-filled three times a day at approximately equal intervals of 8 hours. This is accomplished through an automated fill system with local automated process control. There are essentially three layers to the system:
- Plumbing
- Process
- Remote
The plumbing layer delivers liquid to the reservoirs. It includes liquid supplies, valves, hoses, sensors, and the reservoirs themselves. A main supply dewar is connected to one or two manifolds, and each of these manifolds branches to up to 10 detectors. Solenoid valves control flow into and through the manifolds. LN2 flows into the reservoirs through the inner coaxial tube. Gas from evaporation or displacement is exhausted through the outer coaxial tube. When the reservoir is filled to capacity, liquid overflows out the exhaust. An LED sensor senses the presence of liquid in the exhaust.
The process layer controls the delivery of liquid to the reservoirs. It is entirely self-contained in a rack-mounted control box. It includes sensor signal conditioners, a PLC system and its state-machine firmware, a programmable dialer, ethernet hub, solenoid valve drivers, and front-panel indicators and switches. The PLC implements logic for scheduling next fills and filling reservoirs. Certain states indicate the termination of a fill process or the presence of error conditions, such as failure to fill or failure of sensors. The process layer includes a programmable dialer, that calls and delivers a voice message by telephone.
The remote layer provides both monitoring and control through TCP/IP. It is implemented through an EPICS interface to the PLC. EPICS GUIs provide a visualization of the system state. All front-panel process controls are functionally duplicated through the remote layer. In addition, the EPICS GUI provides configuration of process parameters such as fill intervals, sensor thresholds, and timeout limits. Background processes use the EPICS interface to provides e-mail services for notification of process termination (success or failure). The e-mail list is edited through the GUI.
At all times, there is a designated Cryogenics Officer (CO). The duties of the CO are posted on the Wiki. Normally these duties are rotated through research associate staff: the schedule is also posted.
[edit] Common Tasks
[edit] Checking Current Status Locally
[edit] Checking Current Status Remotely
[edit] Interpreting the E-Mail Messages
[edit] Changing Dewars
[edit] Blowing Out Lines
[edit] Changing Sensors
[edit] Updating E-Mail List
The basic procedure is quite easy, but you have to do it for each individual system.
[edit] Basic procedure
For each of the three systems:
- Log in to the LN2 system EPICS page.
- Click on the edit e-mail list button.
- Click on the Edit radio button at the bottom of the page.
- Edit the list in the text panel
- Click on Save.
- Close the window.
[edit] Separate lists for separate systems
You have to edit the e-mail list for each system separately at the moment, unfortunately. Furthermore, the State-of-Health machine has yet another, independent mailing list. If you are adding a long-term recipient to these lists, you must do it for each machine separately.
[edit] Trouble-Shooting: What to do if a fill fails
[edit] Frequently-Answered Questions
[edit] Who's responsible?
- The Cryogenics Officer in the LN2 Duty Roster
- Greg Hackman -- general questions, including adminstrative ones
- Randy Churchman -- general questions & debugging
- Dave Morris -- PLC controllers & ladder logic programming
- Jane Richards -- EPICS GUI interface, networking, IOC, et
[edit] Hardware-specific
[edit] What's an all-attitude reservoir?
I'll explain later.
[edit] Which green LEDs are you supposed to use on the 8π? on TIGRESS? in Room 156? How do I tell them apart?
Room 156 and TIGRESS use green LEDs that are available in stores as part number 3-3/4121. They are described in the stores catalog as LiteOn LTL-4231N or equivalent.
The 8π system uses LiteOn LTL-239. These are a discontinued item.
Operationally, the voltage drop across a chilled LTL-239 is much larger than for an LTL-4231 in an otherwise identical diode-resistor network. The TIGRESS system (designed from scratch) applies 24 V,while the 8pi system (as inherited from Chalk River) only applies 5 V. An LTL-4231 accidentally installed in the 8pi will never report "cold" because the voltage drop will never exceed the design threshold (typically 4.41 volts). An LTL-239 accidentally installed in TIGRESS or Room 156 will exceed the 10 V range of the sensor readouts, and as such will report as an open circuit rather than as cold.
The easiest (only?) way to test which one you have, is to chill it. Put the LED in series with a 6.8 kΩ resistor. Put a DVM in parallel to the resistor. Apply 24 V forward across the network. Check that the voltage drop across the diode as measured on the DVM is between 1.7 and 2.1 V. Dunk the LED in LN2. If the voltage on the DVM is between 5.5 and 9.5 V, it is LTL-4231 or equivalent. More importantly, it will work with the TIGRESS and Room 156 systems. If the voltage on the DVM is between 18 and 20 V, it is probably an LTL-239 and will work with the 8pi.
NOTE: The TIGRESS-style LTL-4231's should only be used if the cold voltage exceeds 6.66 V.
[edit] EPICS-specific
[edit] How do I get in?
- 8π: ssh -X tigln08@isacepics1
- TIGRESS at SEBT3 (i.e. on the beamline): ssh -X tigln01@isacepics1
- Room 156 lab: ssh -X tigress@sundance
[edit] I can't get in.
- The controls system is firewalled. You may have to log into trcomp01.triumf.ca, trcomp02.triumf.ca, or a similar on-site computer first. (♦SSH tunneling also works)
- Try changing -X to -Y.
[edit] What do all these buttons and squares on a fill line do?
Each fill line (typically) has a valve and a sensor associated with it.
Each valve has a cluster of controls associated with it. Most are self-explanatory. The square indicates the state of the PLC "enable" register for that valve. If the valve is enabled, the box is green, and the valve can and will be opened as part of the autofill process. If the box is lack, the valve is disabled. There is a pushbutton to enable or disable the valve through EPICS; this valve reports the state stored in the IOC which may not be synchronized to the PLC. The PLC value is correct.
The sensor has a rectangular indicator and a voltage readback. The rectangular sensor indicates black for room-temperature (~1.9 V), light blue for LN2-temperature (>4.41 V but less than open-circuit), red for short-circuit (0 V), and yellow for open-circuit (5 V on 8π, 10.24 V on TIGRESS).
There are also reset and on/off (1/0) buttons. The black box brings up a separate control and status window.
[edit] The valve is yellow/red. What's wrong?
The valve is interlocked due to a failure. Either the detector on that valve line timed out during a fill, or the sensor has failed. Check the fill time on that line, and check the voltage on the sensor. If it was a bad sensor, you will either have to replace the sensor or disable the fill line. Once the problem has been determined and (if applicable) fixed, the interlock can be cleared by hitting RST either on the main page or on the separate control window.
- An intermittent short or open will trip the interlock. The interlock will stay set even if the intermittent problem clears itself.
- Clearing the interlock doesn't physically fix anything. If your sensor is still bad, the interlock will return immediately.
[edit] The ERROR light is on. What's wrong?
Something has interlocked. Check for red or yellow valves.
[edit] I hit RST but the error light doesn't clear. What's wrong?
Something is still interlocked. Go back and check it.
[edit] Why is it that the error light is on and the exhaust valves are open?
This is the default failure mode. It allows the manifolds to vent safely.
[edit] Frequently asked but Unanswerable satisfactorily Questions
[edit] The button says "enabled" but the little box next to it is black. What's up with that? Can't you fix it?
As said above, the enabled/disabled button reports the status that the IOC thinks the PLC is in. The little box indicates the real PLC status. Don't trust the button, trust the indicator.
