5.1 Introduction
_____________________________________________________________
5.1.1 Purpose
| During the Development Phase the Project Manager and the System Development Manager work closely together to develop, assemble, integrate, and test the AIS, and to update and finalize the plans for the deploying the AIS. |
During the Development Phase the Project Manager and the System Development Manager work closely together to:
a. Develop, integrate, and test the AIS,
b. Update and finalize plans to deploy the AIS, and
c. Complete business transition planning and initiate business transition activities.
5.1.2 Overview
The Development Phase is divided into two stages: Construction and Production Readiness. In the Construction Stage, the System Development Manager ensures that all AIS subsystems, modules, and components are fully documented, have been coded and tested, and that identified discrepancies have been corrected. The Construction Stage is completed when the Technical Review Board provides approval at Test Readiness Review., Earlier test results are reviewed and operational support arrangements are finalized during the Production Readiness Stage. The Production Readiness Stage ends with successful completion of the Production Readiness Review. This review affirms that Acceptance Testing has been successfully completed and that the AIS is suitable for production use.
5.1.3 Tasks
The tasks to be completed during the Development Phase are:
a. Allocate resources to support the current project plan.
b. Complete business transition planning, and begin business transition activities.
c. Perform component provisioning of reusable components.
d. Code software that is not commercial off-the-shelf (COTS) and is not generated by the CASE tools.
e. Perform module construction and application assembly of reusable components, and complete integration.
f. Complete acquiring equipment and approved COTS software for the AIS.
g. Complete AIS support documentation, including training.
h. Perform system testing activities to include unit and integration testing, performance, and stress testing.
i. Update the Configuration Management Plan.
j. Perform a Configuration Management Build.
k. Update the Economic Analysis.
l. Update the Project Management, AIS Development, and Quality Assurance Plans.
m. Update support plans.
n. Develop the Production Installation and Data Conversion Plans.
o. Update the technical architecture with details of the physical implementation.
q. Obtain approval to install the AIS into work areas.
r. Software development environment planning activities should be defined and submitted to OSDM/SDI.
5.1.4 Activities and Documentation
Development Phase activities and documentation requirements as summarized in the following table must conform to the indicated Technical Standard and Guidelines or other standards as noted. Work products included in the Product Baselines are identified under the “Baseline” heading in Table 5.1.4. Published standards and guidelines may be augmented by tailoring and documented in the Quality Assurance Plan.
| Work Product | TSG / Standard | Who’s Responsible | Must Create | Should Update | Must Update | Base- line | Must Complete |
| Business Case | | ProgramSponsor, Project Manager | | | | X | |
| Project Management Plan and supporting baseline project schedules | IT-212.2-01 | Project Manager, System Development Manager | | | X | | |
| AIS Project Quality Assurance Plan | IT-212.2-04 | Office of System Product Assurance | | X | | | |
| Data Management Plan | IT-212.2-05 | Office of Data Management | | X | | | X |
| Data Conversion Plan | IT-212.2-05 | System Development Manager | X | | | | X |
| AIS Configuration Management Plan | IT-212.2-06 | Office of System Product Assurance | | X | | | X |
| Concept of Operations | IT-212.2-11 | Program Sponsor, Project Manager | | | X | | X |
| Detailed Design | IT-212.2-12 | System Development Manager | | | X | | X |
| Economic Analysis | IT-212.2-13 | Project Manager, System Development Manager | | | X | | X |
| Business Transition Plan | IT-212.3-05 | Program Sponsor, Project Manager | | X | | | X |
| Requirements Specification | IT-212.3-10 | System Development Manager | | X | | X | X |
| Test Plan and Materials | IT-212.3-01 | Office of System Product Assurance | | | X | X | X |
| Test Specifications and Procedures | IT-212.3-01 | Office of System Product Assurance | X | | | | X |
| Training Plan and Materials | IT-212.3-02 | System Development Manager | | | X | X | |
| Requirements Traceability Matrix | IT-212.3-11 | Office of System Product Assurance | | X | | | |
Table 5.1.4 Development Phase Activities and Documentation Requirements
| Work Product | TSG / Standard | Who’s Responsible | Must Create | Should Update | Must Update | Base- line | Must Complete |
| Users Manual and/or Guide | IT-212.4-13 | System Development Manager | X | | | | X |
| Infrastructure Disaster Recovery Plan | IT-212.2-08 | Office of System Network Management | X | | | | X |
| Programmer Maintenance Manual | IT-212.4-15 | System Development Manager | X | | | | X |
| CM Build Instructions with Version Description Document | IT-212.2-06 | System Development Manager | X | | | | X |
| Production Installation Plan | IT-212.5-01 01 | System Development Manager | X | | | | X |
| Operational Support Plan | IT-212.5-01 | System Development Manager | | | | | X |
Table 5.1.4 Development Phase Activities and Documentation Requirements (Concluded)
5.2 System Development
_____________________________________________________________
5.2.1 Software Development and Construction
| Reviews, inspections and walk-throughs will be scheduled, conducted, tracked, and the results reported in accordance with the project's Quality Assurance Plan. |
Application code, databases, database conversion software, interfaces, and code to integrate COTS products into the AIS are constructed during the Development Phase. The following points apply to software development.
a. To improve software documentation and maintainability, whenever possible, changes to AIS application code will be implemented using USPTO standard CASE tool procedures. For later enhancement or maintenance, source code that had been automatically generated will be regenerated and not manually modified.
b. All code will be designed, documented, and developed using modern software development concepts, tools, and techniques. The supporting documentation will be reviewed by the Office of System Product Assurance for accuracy, efficiency, completeness, and ease of use.
c. Developers will thoroughly document and test all software that performs automated conversion of data from existing to new databases. The supporting documentation will be reviewed by the Office of System Product Assurance for accuracy, efficiency, completeness, and ease of use.
d. Using the component specification, the component provisioning task delivers an executing and tested implementation of the component. Each component implementation is designed, coded, and tested. For each reusable CASE component:
· The component is specified in the CASE encyclopedia.
· The component implementation model is constructed to meet the component specification model.
· The component executable is generated from the component implementation model.
e. The component publishing task places the component specification in a USPTO Enterprise Repository component catalog that should be browsed for review in future projects. Publishing produces an installation package to deploy the component into an AIS development environment. The entire model specification, including the data and behavior of the component, should be published for browsing.
f. TRM designated programming languages will be used in all USPTO multi-user applications. Nonstandard, high-order programming languages, as are commonly associated with COTS products, may be used for single user applications or where it can be shown to be more cost-effective for multi-user applications.
g. To help ensure code portability and scalability, developers will not use vendor-unique extensions unless it can be shown to be cost-effective over the life of the application. Waiver requests to use vendor-unique extensions must be submitted to the Technical Review Board for approval. Waivers are not required to use:
· COTS software packages and advanced software technology that is not modified or maintained by the USPTO and can operate on the USPTO information technology infrastructure.
· Programming languages that require installation of vendor-provided updates to COTS software. Use of such languages shall be restricted to implementing the vendor updates.
5.2.2 Integration
Hardware, software (including COTS software), and documentation components which have been individually tested are assembled into functional subsystems and systems in accordance with the project's quality assurance and configuration management plans. Application components must be integrated. Either part or all of an entire application may be built by assembling components. The component verification activity includes installation of components into the environment where they will reside when assembled into an application to verify that installation procedures work. The System Development Manager will complete and update all system design and support documentation in accordance with the applicable Technical Standards and Guidelines. Integration may be performed iteratively with increasingly larger and more complex combinations of components. The steps in which integration will occur shall be documented in the project's AIS project management and test plans. Integration is completed when the entire AIS has been assembled, integration level testing has been performed, and integration test results have been approved by the Technical Review Board at Test Readiness Review.
5.3 Testing and Reviews
_____________________________________________________________
Unit, integration, and independent acceptance testing activities are performed during the Development Phase. Unit and integration testing are performed under the direction of the Project Manager and System Development Manager. Independent acceptance testing is performed independently from the developing organization and is coordinated by the Office of System Product Assurance. The independent acceptance testing activities performed during the Development Phase are Formal Qualification Testing and Beta Testing. Defects uncovered by acceptance testing may lead to changes in the work products created in the Concept, Detailed Analysis and Design, and Development Phases. The technical standards and guidelines for those phases apply, as needed, during the Development Phase. As much as possible, acceptance testing will be performed in a test environment that duplicates the production environment. The Technical Review Board reviews the results of each test. Additional information on testing is provided in the Testing TSG, IT-212.3-01.
5.3.1 Design Characteristics
| Each program module will be separately tested and verified to ensure that all applicable business requirements and module interface specifications are addressed, and that all unit products conform to design specifications. |
Each program module will be separately tested and verified to ensure that all applicable business requirements and module interface specifications are addressed, and that all unit products conform to design specifications. Unit testing should verify that every logical path through the pseudo-code (or detailed module description) is implemented and functions as designed, and that no unit level code is accepted that is not designed and documented). All errors and discrepancies noted will be corrected.
5.3.2 Independent Security Testing
The Office of System Product Assurance and the Office of Information Systems Security will conduct independent system security testing during the Development Phase. System security testing includes both the testing of the particular AIS components, including both purchased and “in-house developed” components, and the testing of the entire AIS. Security management, physical facilities, personnel, procedures, the use of commercial or COTS products, the use of OCIO services (such as PTOnet services), and contingency plans are examples of security areas that affect the entire AIS. Because Project Managers or System Development Managers usually specify or determine requirements relating to these security areas before or after the Development Phase, additional security testing is usually necessary throughout the life of the system. The Operational Support TSG, IT-212.5-01, and the Testing TSG, IT-212.3-01, provide additional information about testing AIS security requirements.
5.3.3 Peer Reviews
Peer reviews will be conducted on designated critical AIS projects to identify and remove defects from the products early and efficiently. Peer reviews can be particularly effective when used to review the work products and documentation developed during the Detailed Analysis and Design and Development phases of the life cycle. AIS or infrastructure project development documentation will be reviewed in accordance with the Quality Assurance TSG, IT-212.2-04. For designated AIS projects, reviews, inspections and walk-throughs will be scheduled, conducted, tracked, and the results reported in accordance with the project's Quality Assurance Plan.
5.3.4 Test Readiness Review
This review is conducted at the end of the Development Phase to determine whether unit and integration testing has been conducted in accordance with the test procedures and that the tests are complete. The Project Manager reviews and approves test procedures, specifications, test procedure modifications (i.e., redlines), and the results of all integration tests before requesting Technical Review Board review and approval of integration testing at Test Readiness Review. A version of the Developmental Configuration is placed under formal configuration management after Test Readiness Review and is used for acceptance testing.
5.3.5 COTS Acceptance Testing
Acceptance testing of COTS products will be performed and all problems and defects will be corrected in accordance with the terms of the contract or other purchase agreement. Unit level testing of COTS products may not be required in circumstances where acceptance testing of the COTS product has been performed and documented. COTS acceptance testing procedures and specifications, and all results of COTS acceptance testing will be reviewed, and if accepted by the Project Manager, will forwarded to the Technical Review Board for review and approval at Test Readiness Review. The Office of System Product Assurance may perform independent testing of COTS products.
5.3.6 Integration Testing
Integration testing verifies the interoperability of AIS modules, components, and subsystems, up to the complete AIS, to ensure that all functional and technical objectives are achieved. Typically, test data must be specifically designed and developed to ensure full coverage of all requirements, and to ensure that all procedural and control options in the AIS code are tested. When possible, actual data should also be used in integration testing to ensure consistency between AIS specifications and the desired business process. The results of all integration tests must be recorded. Test Procedures are corrected by redlining the test procedures document to show changes.
5.3.7 Formal Qualification Testing (FQT)
Formal Qualification Testing is an independent acceptance testing activity performed on an approved Configuration Management Build by the Office of System Product Assurance. During this testing activity all AIS functional capabilities, including security measures, are independently exercised to verify that the AIS meets all functional and security requirements. Formal Qualification Testing is performed against the requirements identified in the project's Requirements Traceability Matrix in accordance with the redlined test specifications and test procedures developed during integration testing.
5.3.8 Beta Readiness Review
When a Beta Readiness Review is conducted, it verifies that formal testing is complete and that the system is ready to be tested by selected users, including employee union representatives. The Office of System Product Assurance will forward the results of Formal Qualification Testing to the Technical Review Board for review at the Beta Readiness Review. This review will be conducted in accordance with the project's Quality Assurance Plan and test plans. The AIS will not be installed in production work areas or operational facilities, or connected to production systems without the concurrence of the Project Manager.
5.3.9 Beta Testing
During Beta Testing selected users test routine features of the AIS to verify that the AIS performs as expected. Beta testing exercises the entire system as it is commonly used, rather than testing it extensively against formal statements of requirements. In preparation, the Beta testers will receive training on the AIS using the training material delivered with the AIS. The Project Manager selects test participants and establishes the schedule for Beta Testing.
5.4 Project Risk Management
_____________________________________________________________
The Project Manager will continue to evaluate risks to AIS project requirements, cost, and schedule. The most common risks encountered during the Development Phase relate to events that may prevent full development or implementation of AIS project requirements. Security requirements and controls, if not previously addressed, will impose significant project risk during the Development Phase. This is due to the potential cost and complexity necessary to retrofit these requirements and designs into the AIS late in the life cycle. Significant risks will also result if review and approval of designs and supporting plans, development tools, development skills, appropriate unit and integration testing, or specification, installation and testing of security controls are inadequate. AIS project managers must also evaluate the risks associated with recent changes and upgrades to the hardware configuration, operating system and system utilities, software development tools, and AIS COTS components. The Project Manager will advise the Program Sponsor and the CIO on matters of significant risk. The Project Manager will also coordinate risk management activities with key managers and analysts based upon their ability to control risk. The Project Management TSG, IT-212.2-01, provides additional guidance on managing risk.
5.4.1 Project Risk Assessment
The Project Manager and the System Development Manager will continue to analyze the existing project situation to identify, analyze, and mitigate significant AIS project risks. These managers will focus on ensuring the full implementation of AIS project requirements. AIS project management will also focus on providing appropriate review and approval of designs and supporting plans, and completing appropriate unit and integration testing that includes testing of security requirements. The Project Manager must also provide appropriate review and approval for changes in hardware, software, development tools, and COTS components. The Project Manager and the System Development Manager will ensure the complete documentation of information and activities that identify, estimate, and evaluate significant risks.
5.4.2 Project Risk Review and Mitigation
Risk assessment findings will be presented to the Program Sponsor, Technical Review Board, or the CIO, as necessary. These presentations will reaffirm the significance of risks and will solicit management support for addressing significant risks. Based upon the recommendations of the Project Manager, the Program Sponsor or the CIO may assign additional staff or allocate additional resources to control significant risks.
5.5 AIS and COTS Security
_____________________________________________________________
Security activities for an AIS may include:
a. developing AIS security aspects,
b. monitoring the development process itself for security problems,
c. responding to requirements changes from a security perspective,
d. monitoring security risks.
Security risks that may arise during the Development Phase include risks associated with code intentionally designed to defeat security measures and disguised as useful code, incorrect code, poorly functioning development tools, manipulation of code, and malicious insiders.
Security activities for COTS software may include monitoring to ensure that security is part of market surveys, contract solicitation documents, and evaluation of proposed systems. Many systems use a combination of “in-house” development and COTS acquisition. In this case all the security activities discussed above are applicable.
As the AIS is built, choices are made about the system, which can impact security including: selection of specific COTS products, AIS architecture, and, AIS site or hardware selection. In addition, security activities such as contingency planning, security awareness training, and security documentation must be conducted. The Operational Support TSG, IT-212.5-01 provides additional information regarding the operational aspects of AIS security. Security, like all requirements, must be anticipated from the beginning of the Initiation Phase.
5.6 Business Transition Planning
_____________________________________________________________
| Transition activities should be tested during the Development Phase to ensure that all personnel, business process, and organizational changes can be implemented with minimum disruption to ongoing production. |
Business Transition Planning, which began during the Detailed Analysis and Design Phase, continues through the Development Phase following the guidance contained in the Business Transition Planning TSG, IT-212.3-05. All significant changes to the business process made during the Development Phase must be reviewed by the System Development Manager, and reviewed and approved by the Project Manager. The Program Sponsor and Project Manager should coordinate significant business process changes with the employee union representatives.
5.6.1 Transition Activities
Transition activities should be reviewed and evaluated during the Development Phase to ensure that all personnel, business process, and organizational changes can be implemented with minimum disruption to ongoing production. Where feasible, interviews and surveys should be conducted to identify issues and determine the level of acceptance to changes in policies, procedures, and job descriptions. Additional changes in the Business Transition Plan may be necessary based upon feedback collected though this process. Moreover, specific presentations and specialized training sessions may be needed to further emphasize the intended goals and benefits of the process improvement activity, including AIS support. All transition and change management activities will be documented and tracked in the Project Management Plan according to IT-212.02-01. The Project Manager may brief the Technical Review Board on the expected impact the deployed AIS will have on their organization at Test Readiness Review.
5.6.2 Preparing the Organization for Change
End user involvement in the specification, design, development, and testing of transition activities is absolutely essential to promote broad-based process ownership and acceptance. During the Development Phase, end users and the employee union representatives should be heavily involved in transition activities which require pilot testing (e.g., workflow redesign, team-oriented examination) and should play a major role in reporting the results of pilot tests. Moreover, end users should be involved in updating the desired business process and in planning AIS deployment throughout the organization. Where feasible, end users should take the lead in validating revised policies and procedures, new forms and formats, job aids, and performance measures.
5.7 Production Readiness Review
_____________________________________________________________
| At the Production Readiness Review, the TRB will confirm that the AIS is complete, correct, fully tested, and ready for the Deployment Phase to begin. |
The products of the Development Phase, and updates to existing plans and specifications must be reviewed by the Technical Review Board. The Production Readiness Review will confirm that the AIS is complete, correct, fully tested, and ready for the Deployment Phase to begin. This review establishes the product and operational baselines and confirms that appropriate operational and maintenance support is available. The Product Baseline consists of the documentation describing all the necessary functional and physical characteristics of the elements that compose the system and the actual equipment and software. The Technical Review Board will confirm that the installation of the AIS into the production environment will have no adverse impact on other AISs or the PTO information technology infrastructure.
Deficiencies identified in unit, integration, Formal Qualification, and Beta Testing must be resolved, and changes generated by the review must be incorporated into the appropriate work products as directed by the Technical Review Board. Based upon a finding that the AIS is suitable for production use, the Technical Review Board will recommend to the Program Sponsor and CIO to the deploy the AIS in accordance with the Production Installation Plan.
[1] Formal Qualification Testing (FQT) exercises the entire AIS to independently confirm and extend integration testing results in an environment that more closely duplicates the production environment. OSPA will test all AIS requirements that can reasonably be tested. These requirements are identified in the current AIS requirements traceability matrix. See 5.4.6 below.
[2] Beta testing may be tailored in or out of the life cycle. End users perform Beta Testing in an ad hoc manner to confirm that the system will provide the expected functionality before placing the system into full service. This testing will simulate the production environment as closely as is reasonably possible. See 5.4.7 and 5.4.8 below.
No comments:
Post a Comment