Computer Software Validation News & Views Feature Article

By Alan Muirhead Alan Muirhead is a senior validation consultant with Taratec Development Corporation, a leading validation consulting firm. He can be reached at muirhead@voicenet.com. Originally published in News & Views March 2000 issue. Copyright 2000 STC-Philadelphia Metro Chapter. For permission to reprint this article, contact the Managing Editor.

We've all seen headlines like these: "Diet Drug Recalled; Heart Damage Cited;" "Drug Cocktail Suppresses HIV;" "Sildenafil Fails to Reduce Angina, But Is Reborn as Viagra" Have you ever wondered how pharmaceutical companies gather and analyze the data that lead to the successes and failures behind those headlines? How they know that their computer systems don't distort or lose information? How, in turn, the government knows that companies are reporting accurately? In the United States, the Food and Drug Administration (FDA) is responsible for ensuring the safety and efficacy of new drugs and rnedical devices. Pharmaceutical companies submit safety and efficacy data from clinical trials to the FDA in order to get approval to market new products. Reports of any "adverse events" encountered with drugs or medical devices are also submitted to the FDA. Companies retain lab tests and manufacturing batch data in case of any problems (remember the Tylenol poisonings a few years back). The FDA reviews supplied data and conducts on-site inspections; inspections include examination of the computer systems involved in gathering clinical data, ensuring production quality, and controlling medical devices. As part of their submissions and certainly during inspections, companies have to prove that their computer systems provide accurate record-keeping and analysis of the data required by the FDA. How do you prove that a large and complex computer system is working properly? For example, a clinical system can contain hundreds of pieces of information on each of several thousand patients, gathered from multiple remote sources over an extended period of time, with cross-links between clinical trial protocols, adverse event reports, and manufacturing quality records. A complete, direct test of such a complex system is impractical, if not impossible. Instead, you must validate the computer system. If people think about software validation at all. they usually think it simply means testing. While testing is an important part of the validation process, validation is in fact much more than just testing. The working definition of software validation cited by the FDA is "establishing by objective evidence that all software requirements have been implemented correctly and completely and are traceable to system requirements." This simple-seeming definition has several profound implications. First, "establishing by objective evidence" points toward the need for documentation, and a potential role for technical writers. Second, "implemented correctly" in practice means that the system must be developed according to quality software engineering principles, with documentation of development procedures, software design, and testing results. Third, the emphasis on requirements means that validation begins early in the project, with the definition of system requirements. In fact, validation - properly implemented - is an integral part of the life cycle of a computerized system. The development life cycle concept is not restricted to a single development methodology; organizations may use "waterfall," RAD, or prototyping, or whatever methodology they prefer, as long as the process is defined and then followed. Figure 1 illustrates the phases in a typical life cycle, together with the types of validation documents that might be associated with each phase and the group (users or I/T) primarily responsible for each phase. I'll discuss each phase in more detail in the following, pointing out how technical writers can be part of the validation effort. how the system will be developed and installed; the plan will likely need refining as the proj ect progresses. A "request for proposal" (RFP) may go to potential vendors; vendors could be outside suppliers, the in-house I/T group, or both. Vendors should be evaluated for their ability to provide both the functionality desired and to participate in the validation effort. Other than vendor ments must be clear, complete, and testable. A requirement such as "the system must be easy to use" gives no meaningful direction to developers and cannot be tested. I have found that my technical writing background helped a lot when writing functional requirements, since both writing skills and an understanding of the user point of view are needed. The validation plan or strategy document outlines how the system will be validated, including the development approach, the types of qualification testing to be performed, and what documents will be retained. One of the first things an FDA inspector looks for is to see that the validation plan was followed throughout the validation effort and that any deviations from the plan were documented. Figure 1: Phases in a typical system development life cycle Initiate Users and business management normally initiate the project to develop a new computerized system by preparing a proposal that outlines the capabilities and justification for the system. Depending on the company, I/T (the information technology or information systems group) may suggest a new or enhanced system, but ultimately it is the business that defines the need and provides the funding for major development projects. At this early stage a project plan will be written to outline audit findings, documents generated during the initiation phase are not often considered part of system validation; they may be retained for business purposes. Define Defining the system produces two critical documents: the functional requirement specification and the validation plan. Functional requirements should be based on the user's needs and describe what the system will do, not how it will do it. Functional requirements must be clear, complete, and test- able. A requirement such as "the system must be easy to use" gives no meaning-ful direction to developers and cannot be tested. I have found that my technical writing background helped alot when writing functional requirements, since both writing skills and an understanding of the user point of view are needed. The validation plan or strategy document outlines how the system will be validated, including the development approach, the types of qualification testing to be performed, and what documents will be retained. One of the first things an FDA inspector looks for is to see that the validation plan was followed throughout the validation effort and that any deviations from the plan were documented. Design The design phase defines how the system works. High-level design documents outline system architecture, data flows, and interfaces. Detailed designs include descriptions of individual program modules, data dictionaries, and perhaps pseudo code. Developers usually create these documents, but technical writers who are comfortable with the technical details may also have a role. Figure 2: Validation Deliverables Construct During the construction phase, developers time-shift into the dark hours and retire behind closed doors into which pizza boxes disappear and from which occasional screams or cries of glee are heard. Any specialized hardware components required for the system are assembled, interfaced, and configured. Developers write the unit test plans, which they then follow as they test individual modules. Unit testing is low-level testing, looking for correct data types and logic flows within single modules. Technical writers don't usually have much involvement in unit testing. (However, in my experience it has been difficult to get good documentation of unit testing.) Technical writers have more to contribute to manuals and online help, which should be developed during the construction phase. Integrate During integration, separate program modules are compiled into the complete system. Integration testing (sometimes called structural testing) should focus on how data flows from one module to another, overall program logic, and correct results of calculations. Formal testing plans and scripts are required; technical writers can help with their preparation. Integration testing is performed by developers, but users may be involved to review scripts and check results. Once the system is integrated together, it should be handled under change control; that is, any change that needs to be made to the system must be documented, the old code removed, and the new code compiled and retested as a system. Transition The transition phase includes all the steps necessary to move the new system from the development environment into production. A major feature of the transition phase is user acceptance testing (UAT). UAT is the part of validation that most people think of first, but as you can see, UAT is simply one part of the overall validation process. Before UAT can begin, however, the system goes through two more developer-run tests: installation qualification (IQ) and operational qualification (OQ). During IQ/OQ the system is installed in the user test environment, and all technical aspects of the system are tested. Users then execute the UAT, also known as performance qualification (PQ). Technical writers have much to contribute to this phase. IQ/OQ/PQ all require formal plans and scripts. Manuals, SOPs, and training materials must be complete and users trained on the new system before they can execute the tests. Finally, the results of all the validation activities must be summarized in a validation final report (VFR). The VFR also outlines support and maintenance for the production system, backup and recovery procedures, and archiving of the entire validation document package. Final management sign-off on the VFR gives approval to install the system in production. The VFR is the first document given to an FDA inspector, since it answers many of the questions an inspector will have about how the system was developed, tested, and installed. Figure 2 shows how all the validation documentation flows to this final report. Operate During normal operation, SOPs governing user procedures, system security, and the logging of errors and system change requests must be followed. If enough reasons are found to upgrade or enhance the system, a new development project can be justified, and the cycle will begin again. Otherwise, the system will eventually be retired or replaced. The retirement must be documented, together with a plan for how the information in the system will be archived or migrated into the replacement system. The role of the technical writer In the January 2000 issue of Intercom, Saul Carliner mentioned the opportunity for technical writers outside the normal "technology" areas related to computers; for example, the pharmaceutical, environmental, and agricultural industries. Validation is an example of such an outside opportunity. Validation is required not only in the pharmaceutical industry but also in other regulated industries such as utilities. But is validation the right thing for you? Working on a validation effort gives you the opportunity to develop and exercise many important skills: project management, to be sure that the validation plan is followed and all the validation documents are produced and approved on time interviewing, since you will be gathering information from both users and developers user advocacy and task analysis, to be sure that requirements accurately represent the user's business process and that testing scripts meaningfully reflect how the system will be used team-building, to help bring together users, developers, and management structured writing, since the form of most validation documents is defined by corporate SOPs and templates, and similar information appears in multiple documents As the validation person, you might also be called on to develop online help and training materials, since you will know as much or more about the total system as anyone. This happened to me at my last client. On the other hand, working on validation is not likely to put you on the forefront of the latest developments in web site design and embedded help. So you have a choice to make. I've listed several general sources on validation in the bibliography. The FDA site in particular has lots of relevant information, including applicable regulations, compliance policy guides, inspection guides, and guidance for industry documents. Fair warning: the FDA site is extremely large and complex, and finding exactly what you want can be frustrating. But at least for the pharmaceutical industry, it's the ultimate resource. One last word: not everyone appreciates validation. It requires extra work to document everything, and in a regulated industry, that documentation has to be carefully maintained, signed off, and archived - it's a legal document, just like a will or a mortgage. However, validation should be viewed not as a burden but as an integral part of quality assurance. Establishing a record of plans, specifications, designs, and tests helps to build quality into the system. Looking at validation this way may not make everyone happy, but it does explain why the FDA requires validation and how validation and the writer producing the validation documents add value to the system and the organization Bibliography FDA. "Draft Guidance for Industry: General Principles of Software Validation," 1997, www.fda.gov/cdrh/ode/swareval.html Scava, Frank. "Software Validation for Pharmaceutical and Medical Device Manufacturers," 1994, www.bssifirm.com/xval.htm Stokes, Teri, et al. Good Computer Validation Practices. 1994: Interpharm Press, Buffalo Grove, IL. Wallace, Dolores, et al. "NIST Special Publication 500-234, Reference Information for the Software Verification and Validation Process," 1996, hissa.ncsl.nist.gov/HHRFdata/Artifacts/ITLdoc/234/val-proc.html

Return to . . .

[News & Views] [STC-PMC Home] [STC Home Page]
Last updated: May 10, 2000 (mvh)