HRMS Disaster Recovery Procedure

PeopleTools Application Administration

The following disaster recovery procedure is intended to be performed by a knowledgeable PeopleTools Applications Administrator. For more information involving these steps, refer to the private AppAdmin Tips&Techniques document index located at \\corpproj\data$\PsAdmin\HTML\tipsindx.html.

Recovery Procedure

  1. Set an appropriate message of the day for the hcm-prod application to inform users of the myHRInfo unavailability
  2. Request network services alter the DNS entry for the production database server to make the entry pointš to the development database server
  3. Request the css group recover the following ssh files˛ from the production database server's backup. Move aside (but preserve!) the development database server's original ssh files and replace with these:
    • authorized_keys
    • authorized_keys2
    • id_rsa
    • id_rsa.pub
  4. Shutdown all development databases and their related services on the development database server
  5. Ensure the current production software silo is distributed from psadm@boss to the development database server
  6. Clone a new WebLogic server definition called "PIA0" from the existing "PIA" server; this will act as the report repository webserver. Update the following configuration values so that it may co-exist with the primary PIAx server on the production webserver host in Physics:
    1. change the machine with which it's associated to be the production webserver host in Physics
    2. change the listen address to be the certificate name of the production webserver host
    3. change the http and https ports to the "other" domain values from our Ports Assignments document
    4. reset all Keystore, SSL and Start tab passwords
    5. ensure this new server definition is included in the Deployments
  7. Update the configuration.properties file for all production channels of the production webserver domain on the production webserver host in Physics so that the psserver parameter points to the development database server
  8. Update the SLB router so that only the production webserver host in Physics is OPERATIONAL
  9. Establish the required psconfig definition files for the database under /apps/psadm/version/{SID} on the development database server
  10. Update /apps/psadm/bin/localPSservicesList to add service calls for the production database SID
  11. Generate the production application server and process scheduler domains on the development database server
  12. Ensure the /apps/psadm/data/reports subdirectories on the production webserver host in Physics exist, have group ownership of pswls and group permissions of rws
  13. Test that user psadm on the development database server is able to ssh to all production ssh targets for interface purposes. See the HRMS project team for the list of internal and external targets
  14. Ensure the development database server is able to print to all production printers
  15. Ensure the firewall for the development database server will allow all appropriate remote production database hosts to connect via database link

Once the restored production database is available

  1. Restart the production webserver domain on the production webserver host in Physics so that the configuration changes take effect
  2. Start the PIA0 server definition to act as the report repository server
  3. Start the production application server domains on the development database server
  4. Notify the HRMS team that PIA is available, and that they must:
    1. disable any process scheduler jobs that should be prevented from executing
    2. notify the AppAdmins when the previous task is complete
  5. Sign-in to PIA, add a new process scheduler Report Node for the restored production database and production webserver host in Physics; assign the PIA0 definition for the URL, URI Host and URI Port
  6. In PIA, update the process scheduler Server Definition to associate the new report node
  7. If appropriate, change any unix environment settings required to ensure the process scheduler will scp interface files to the production file hub server
  8. When given the go-ahead from the HRMS team, start the production process scheduler domain on the development database server
  9. Run a test process to ensure the process scheduler is functioning appropriately and report distribution is successful
  10. Update or remove the message of the day as appropriate, and redistribute

Current Environment

At the time of this writing, the following values are in use:

Parameter Value
development database server dawn
production webserver host in Physics wally2
production software silo hcm13
production webserver domain hcm-prod
production file hub server dock


-- KevinOberle - 4 Dec 2008

Edit | Attach | Print version | History: r7 < r6 < r5 < r4 < r3 | Backlinks | Raw View | Raw edit | More topic actions...
Topic revision: r6 - 09 Dec 2008 - 20:07:21 - KevinOberle
 
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback