BAS/EUVE/0091/95
1:0 Transition Plan
Version 3
2 Nov 1995

To: CEA Management
From: B. Stroozas and G. Kaplan
CC: G. Bevan, D. Biroscak, D. Christian, M. Eckert, F. Girouard, A. Hopkins, P. Jelinsky, F. Kronberg, M. Lampton, P. Ringrose, C. Smith, J. Vallerga, L. Wong, and C. Jones (file)

Introduction

This memo outlines the plan (Version 3) for the 1:0 transition, the purpose of which is to eliminate the need for a payload operator to perform routine daily "shift" work in the ESOC. This memo focuses on actual implementation and is a direct result of the internal CEA review that was held on 13 Oct 1995. At this review ISO presented the detailed Version 2 of the 1:0 plan (BAS/EUVE/0088/95); the attendees then discussed the concept of the phased transition and the various "needs" and "wants" associated with each Phase. In the end everyone expressed general agreement with the plan, the only major issue of contention being the timeline for the transition. Roger expressed a great desire to complete the 1:0 transition before the end of this year so as not to interfere with the SCOTT efforts; to do this he proposed a schedule of mid-Nov for Phase 1 and mid-Dec for Phase 2.

Since this review, ISO and ISD have had numerous discussions to outline all of the relevant issues in an attempt to meet Roger's proposed schedule. The short answer is that we can meet the Phase 1 deadline. However, due to its more complicated nature, Phase 2 can not be completed in any safe and robust manner by Dec; therefore, we have added a Phase 1.5 to be completed in this time frame. This memo outlines exactly what we plan to do -- in terms of primary and secondary "needs" and "wants" (PN/SN/PW/SW) -- to implement 1:0. (Remember that a PN is an issue that affects instrument health and safety (H&S), a SN is a non-H&S issue that would yield major cost savings, a PW is an issue that would yield minor cost savings and/or improve the science return, and a SW is an issue of a purely streamlining nature.)

Phase 1

As a reminder, Phase 1 is the "minimal" zero-shift scenario, the purpose of which is to automate all essential instrument health and safety (H&S) telemetry monitoring. The particular focus here is eliminating the need for an operator to visually monitor any real-time passes during nominal operations. In the Phase 1 scenario an operator will continue to be needed to handle the routine daily "shift" console support duties. The completion of Phase 1 is scheduled for mid-Nov.

The following is a list of the the software-related issues to be completed for Phase 1:

Eliminate the need for all routine daily operator console duty, i.e., the visual monitoring of real-time telemetry passes. (PN) -- In cooperation with the Duty Scientists, ISO has reviewed all of the engineering monitors not currently handled by eworks. The result is that no extra monitors are required for Phase 1; however, a small subset has been identified for Phase 2 (see below).
STATUS: Done.
Eliminate "bad data" in the monitoring and paging process. (SN) -- This issue will be addressed by changing the monitoring and paging from real-time data to post-pass real-time (pprt, which includes quality information). PACOR has already begun sending us pprt data on a routine basis; some minor software modifications are needed on our end to reconfigure the monitoring and paging process.
STATUS: Software changes nearly complete.

The following are the non-software-related issues to be completed for Phase 1:

Review the ACE system. (PN) -- ISO has begun reviewing the current ACE implementation to ensure that it supports a 1:0 scenario.
STATUS: Review in progress.
Ensure for adequate simulations. (PN) -- Since people will still remain in the ESOC on a daily basis after the Phase 1 transition, little is required in terms of a simulation. We need only verify that the monitoring is being done on the pprt data.
STATUS: In progress.
Update and augment all ESOC procedures. (PN) -- ISO has begun to review the procedures to ensure that they accurately reflect a Phase 1 1:0 scenario.
STATUS: Review in progress.
Streamline the anomaly response process by providing ACEs with remote access to the ESOC network. (PW) -- ISO is working with central to provide remote access to the ACEs, and is working an outline of how such access should be used (i.e., what is and isn't allowed).
STATUS: A usage outline is under development after which central will implement the remote access.

All those "needs" and "wants" from the Version 2 1:0 plan that are not addressed above will be folded into the SCOTT plan for future attention.

Phase 2

As a reminder, Phase 2 is the "operational" zero-shift scenario, the purpose being to automate all daily console support duties. The particular focus here is to eliminate the need for an operator to perform "shift work", defined to be various routine duties that are performed on a daily basis. It is in Phase 2 that we actually eliminate the need for daily shift support (including on weekends) for nominal operations. The completion of Phase 2 is scheduled for Spring 1996.

In order to partially meet Roger's proposed schedule we have instituted an intermediate "Phase 1.5" scenario to be completed in mid-Dec. Phase 1.5 will contain some of the Phase 2 functionality -- those tasks that are most easily implemented in such a short time frame (see details below) -- and will allow us to decrease the amount of shift support required on a daily basis (including weekends).

The following are the software-related issues that need to be completed for Phase 2 (those identified for Phase 1.5 are identified as such):

Eliminate the need for all routine daily operator console duty, i.e., the visual monitoring of real-time telemetry passes. (PN) -- This is a carry-over from Phase 1 and includes a small set of monitors that need to be included in eworks.
STATUS: Flowchart design beginning.
Automate the testing and validation of the eworks software. (SN) -- This is a carry-over from Phase 1 and is needed since we'll be modifying eworks.
STATUS: Design work has begun.
Automatically ensure that all critical ground systems are functioning at all times. (PN) -- ISO and ISD are working together to determine those systems that need to be monitored. A minimal version of this will probably be ready for Phase 1.5.
STATUS: Design work in progress.
Eliminate the manual daily review of limit transitions. (PN) -- ISD has discussed this issue with ISO and is currently developing software to implement this for Phase 1.5.
STATUS: Design work in progress.
Eliminate the manual daily review of command echoes. (PN) -- ISD is working with ISO on an implementation strategy to deal with this issue. A short-term solution for Phase 1.5 might be to route this material out of the ESOC to an individual's desk for review.
STATUS: Design work in progress.
Eliminate the manual daily review of detector plots. (PN) -- ISD is working with ISO on an implementation strategy to deal with this issue. It will probably involve routing the relevant plots out of the ESOC to the appropriate person's desk for daily review; this may be done as part of Phase 1.5.
STATUS: Design work in progress.
Eliminate the manual daily check for high count rate areas on the detectors. (PN) -- ISD is working with ISO on an implementation strategy to deal with this issue. A short-term solution for Phase 1.5 might be to route this material out of the ESOC to an individual's desk for review.
STATUS: Design work in progress.
Eliminate the manual daily review of "alert" conditions. (PN) -- ISD has discussed this issue with ISO and is currently writing software to implement this for Phase 1.5.
STATUS: Design work in progress.
Automate all routine commanding activities. (SN) -- Work on the ATC implementation continues, with a target completion date in time for Phase 1.5.
STATUS: Design complete; software development in progress.
Eliminate machine dependency on the ESOC network. (PW) -- Central and ISO are already working together to complete this for Phase 1.5.
STATUS: Central has implemented RAID box on the ESOC network and it is operational; additional functionality continues to be transferred to it.

The following are the non-software-related issues that need to be completed for for Phase 2:

Eliminate the manual daily shift report. (PN) -- ISO will be working up a plan to deal with this issue, the most likely scenario being by just incorporating the daily information into semi-weekly reports; this may be done for Phase 1.5.
STATUS: Not yet started.
Ensure for adequate simulations. (PN) -- A simulation phase will be necessary to ensure that proper testing is done of all new software components. In addition, it will be necessary to ensure that all procedural issues accurately reflect Phase 2 operations. ISO will work up a reasonable simulation scenario.
STATUS: Not yet started.
Update and augment all ESOC procedures. (PN) -- ISO will review all the procedures to ensure they are ready for both the Phase 1.5 and 2 scenarios.
STATUS: Not yet started.
Develop a training program to ensure that controllers retain operator proficiency with payload activities in a reduced support environment.
STATUS: Not yet started.

All "needs" and "wants" from the Version 2 plan that are not mentioned above will be folded into the SCOTT plan. The goal is to implement as much as possible for Phase 1.5 in order to allow for sufficient "shadow" time to properly verify everything before the actual Phase 2 transition.

Phase 3

There is no plan to implement a Phase 3 "full" zero-shift scenario before the SCOTT proposal takes affect. Therefore, all Phase 3 "needs" and "wants" will be folded into the SCOTT plan.

Summary

In summary, ISO and ISD will be working together to complete the 1:0 transition in a phased approach: Phase 1 by mid-Nov, an intermediate Phase 1.5 by mid-Dec, and Phase 2 by Spring 1996. A Phase 3 transition will be folded into the SCOTT plan, which will include all the unaddressed "needs" and "wants" outlined in the Version 2 1:0 plan that are not addressed in Phases 1 and 2.

By using such an incremental approach to 1:0, and remembering that the 3:1 took much longer than expected (the original problems were more difficult than expected and the proposed solutions were overly ambitious), software and procedural additions will be put into place each month as they become available. By doing the simple things first, we will gain tangible results quickly, but build an overall robust system in an organized manner. We will also be able to test, via an on-going simulation, the functionality of all the components before we begin to rely on them completely. We believe that by following the course outlined above we'll be doing 1:0 the right way -- one that is safe for the instruments and reasonable in relation to all other CEA work efforts.