The first release into the system testing area is likely to be rather complicated and long winded. The effectiveness of this process will depend on a number of factors:
· Are all required deliverables itemised;
· Are all deliverables that are currently under source control effectively identified;
· Are the necessary development tools configured correctly;
· Are the necessary data files identified and ready for migration.
The release process should be effectively planned with defined activities for each of the following:
· Release technical architecture items and ensure that they are all correctly configured;
· Release all third party and tool specific deliverables;
· Release all data items;
· Release all the development deliverables.
If possible, temporarily secure all the development source deliverables. This will prevent any changes from being applied to the source that is about to be system tested.
It is important that the full contents of the system testing environment is documented. Where possible identify the version number of each item. Where version numbers cannot be identified then take note of the file size, date and time stamps.
On some platforms, DOS for example, you may be able to set the date and time stamp of all files to a predetermined value. This would enable one to easily identify items that were released as part of the normal system test release and those items that have been modified later.
Once the release process is complete, the run should be built within the system test area;
The task sound reasonably simple, but may prove to be very complicated. The build process may highlight one or more of the following problems;
· The environment has not been correctly configured;
· Some files may be missing or have the incorrect version;
· Source files may be missing;
· The build instructions may be incomplete.
It is amazing the number of problems that one encounters during this process.
Formalising release into system testing and producing the run time is very important for the following reasons:
· This process is likely to be performed many times per month during the system testing stage. Cracking the release process will remove the guess work out of the process and reduce the opportunity for errors;
· This will enable the project to fully understand the intricacies of the release and build process and hence enable effective release;
· Enable the project to repeat the exercise when additional environments are required, user testing, pre- production , ... etc.
· Enable the project to establish a link between the source and the run time.
The run time should always be produced either in the system testing environment or in a dedicated area within this environment. Run time should not be accepted directly from a development environment as the run time may contain code elements that are either not approved or not filed in the master library. Also there is always problems in identifying the exact make up of a run time module, i.e. what programs were used to produce the run time.
Where possible, appoint an independent developer to perform the release and build process. Some organisations employ “unskilled” operators to perform this task. The developers would be required to provide detailed release and build instructions that the operators would follow. Adopting this approach has a number of benefits:
· Forces the development staff to plan and define the build requirements;
· Highlights any deficiencies in the instructions, an indication of poor understanding of the process or poor documentation;
· Removes the scope for “Building by trial and error”.
· Improves awareness of the process and its complications.
The SCM system should not interfere or burden the system testing process; testers should be able to conduct their day to day work without any change in working practice. A number of rules should be adhered to however:
· No changes should be made to the structure or contents of the testing environment without raising the required change control forms.
· Any defects in the environment should be reported using the change control process.
· Developers should not be allowed to copy any form of files into the testing area.
The system testing process will identify defects in the system and these should be corrected within the boundaries of the development environment.
Faults should trigger the following processes:
· Fault management which aims to document, assess and recommend how faults should be corrected;
· Source control, to extract the affected items from the master library and into the development environment;
· Development activities to correct the fault;
· Approval within the development environment;
· Source control, to migrate the fixed code back into the master library;
· Release management to release code back into system testing;
· Build management to generate the run time;
The above process is very formal and can be time consuming. Organisations that expect each fault to be corrected and then released on a one to one basis may find the above process expensive and time consuming.
Organisations that adopt a cyclic approach to testing and those who are prepared to accept weekly releases into system testing will find the above process easy to operate and cost effective.
The benefits of the formal approach described above are as follows:
· Enables development to perform a full and thorough assessment of each fault. Rushed fixes generally introduce secondary faults.
· Enables the configuration librarian to maintain a full audit trail of all faults and how they were fixed.
· Enables the configuration librarian to maintain an up to date configuration index for the system testing area.
· Minimises change in the system testing area and hence reduce disruption to the system testing team.
System testing should be repeated until the system reaches an acceptable level of stability. When this is reached, all source should be baselined to document the system testing baseline.
System testing baselines should be created for the complete system, this is in sharp contrast to the approach adopted for items being unit tested.
The cycle for acceptance testing should be a repetition of that for the system testing process.
Fault management should be operated as with system testing. There is no real need for migrating items through system testing for faults detected during acceptance testing.
Once the acceptance testing is completed, the complete system should be baselined and promoted into production.
Parallel maintenance and development takes place when the development teams are involved in implementing changes to two or more system baselines. The most common example of parallel development is when development are working on a release of the system while implementing changes to previous release.
Parallel development may also take place when a project chooses to enter acceptance testing prematurely. In this case development staff may be expected to implement fixes to a version of the system that is undergoing system testing and another version that is undergoing acceptance testing.
Organisations should be very careful when parallel development is undertaken as this complicates the development process for the following reasons:
· Developers will need to segregate the different versions that are being worked on in the development area. This is best done by creating instances of the development environment for each strand of development work.
· Code belonging to the different versions will need to be segregated in the master library. This can be done by creating branches or variances within the master library or by creating several copies of the master library.
· Faults must clearly identify the version of the affected item. Resolution must also identify which versions are affected by each fault.
· Where a fault affects more than one version of the system, development will have to implement the changes to each affected version.
As a rule, avoid parallel development as the plague. If parallel development is required, and in many cases it will be, then try to restrict the parallel phases to two, one for developing new releases and one for maintaining previous releases.
To support parallel development, make sure that duplicate environments are created to support each phase.