Most commonly referred to as source and version control tools, these are the most common, essential and basic of all configuration management tools.
Most source control tools perform the following functions:
· Provide a configuration item repository. These tools are able to store the physical configuration items in a repository. Each time an item is added to the source control tool, a master copy of this item is created in the tool’s repository. Current source control tools are able to store all types of objects, ASCII or binary.
· Store all superseded versions of a given item. Each time an item is modified, the tool is able to store the current version and the previous. ASCII items are stored in delta format; this implies that the tool is able to store a complete copy of an item and generate all other versions by storing the differences between the master and the various versions. Generating delta format for binary files is not generally possible.
· Generate version numbers. Each time an item is modified, a unique version is generated and assigned to this new version. Most tools allow the user to override the default values that are being proposed by the tool. The format of the version number is generally tool specific and cannot be modified.
· Maintain an audit trail of all changes. These tools automatically generate an audit trail showing when an item was modified and who modified it. Most tools provide the user with an option to enter a description for each delta. This description can be used to document a more detailed audit trail by allowing the user to specify a cross reference to another process, change control for example.
· Synchronise development. Users need to check out items from the source control tool prior to modifying an item. These items are then checked back into the tool to generate a new version. Most tools can be configured to prevent more than one developer from checking out an item for modification.
· Streaming of development paths. This is generally referred to as branching or variant management. Most tools allow a developer to maintain two or more parallel variants of a given item at any one time.
The above represents the main features of source control tools. Most tools provide additional less important features. These include the following:
· Ability to compare different versions. Most tools provide a facility to enable one to compare two different versions of a given item or even different items. Whilst this feature is very useful for ASCII items, the reports generated for binary items are obviously totally unreadable.
· Ability to merge versions. This feature is normally used to merge two or more variants to produce a new revision. As with difference reports, this feature is only usable with ASCII items.
Most tools marketed as configuration management tools are essentially source and version control tools with additional marginal features. Source control tools range in price from a few hundred pounds per user to several thousand. There are also many good quality public domain tools, RCS for example, especially on the UNIX platform.
The following sections describe in more detail the various features that one must look out for when reviewing or assessing source control tools.
The term delta is used to describe the tool’s ability to store different versions of an item in a format that minimises the disc space requirement. A delta is produced by the tool each time a new version of an item is created.
A delta is essentially a description of the changes between two successive versions of an item. By documenting the changes between two revisions one is able to minimise the storage required to maintain copies of the two versions. Therefore, instead of storing a complete copy of each revision, the tool is able to store a complete copy of a single revision and then store the changes required to generate each of the remaining revisions.
There are basically two mechanisms for generating deltas. The first is referred to as standard delta and the second is referred to as reverse delta.
A standard delta is one where the tool stores a full copy of the first revision and change descriptions to generate all the remaining revisions. A reverse delta is one where the tool stores a copy of the latest versions and change descriptions to generate all the remaining revisions.
Generating a standard delta requires less CPU resource than generating a reverse delta. On the other hand, extracting the latest revision requires much less CPU resource for reverse delta format files than for standard delta files. Given that one is likely to access the latest revision most frequently, it follows that a tool that generates reverse deltas is likely to require less CPU resources than one that generates standard deltas.
Generating delta’s is subject to three major drawbacks:
· The reduction in required disc storage is determined by the format of the item. Disc savings are maximal for text items and minimal for binary items.
· The generation of deltas involves the modification of the items being stored and controlled. Any problem with the algorithm used to generate the deltas is likely to result in the corruption of the items being controlled.
· Certain incompatibilities may be encountered when controlling text items that contain certain control characters.
Many source control tools are able to store binary and text items. One however should note that such tools treat binary and text items differently. It is quite common for source control tools to store complete copies of binary revisions instead of generating standard or reverse deltas.
Given the low cost of magnetic media, the benefits of generating deltas are not as important as they used to be a decade ago. Also, given that most source control tools are unable to generate true deltas for binary files and given that many objects created by current development tools have non text formats, the benefits of generating deltas is further decreased.
Purists may argue that generating deltas should be avoided as this approach involves modifying items being controlled. The safest option would be to have a source control tool that provides a passive mechanism, i.e. items are stored without the generation of deltas, for storing revisions, hence minimising the opportunities for loss or corruption of deltas.
A source control tool provides storage for:
· The physical items being controlled, the program files for example.
· The audit trail and history record for each revision.
Different source control tools provide different mechanisms for storing the physical items and the audit history. Most tools make use of operating system files, referred to as archives or history files, to store the physical items and the audit trail. Some tools make use of operating system files for storing the physical items but provide a database to store the audit history. Finally, some tools make use of a database to act as a repository for the physical items and the audit trail.
Whilst the generation of history files is the most primitive form of storing items and their revisions, this provides the most robust and scaleable storage mechanism. Such tools are able to store very large numbers of items and are able to support access to a very large number of concurrent users with minimal degradation in system performance. With such tools, the performance of the repository is determined by the performance of the file system being used by the operating system.
Tools that make use of history files suffer from a number of drawbacks;
· Information distribution: given that each item has its own history stored in its own history file, it is not easy to view information relating to a large number of items.
· User queries: ones’ ability to generate complex queries against many history files is severely limited. Most tools that make use of history files do not enable a user to identify items of a given description.
· Security: given that the items are stored as operating system files, the security applied to such items is determined by the security features provided by the operating system. History files stored under the DOS FAT file system are completely unsecured for example.
Despite the limitation listed above, history file based source control tools provide exceptional performance and response times. Such tools are able to provide very responsive repositories for very large libraries; single libraries of up to several thousand items in a single directory are possible.
Source control tools that employ databases generally provide value added features that are not commonly found with tools that employ simple history files. Such tools may be used to document relationships between items and can generally provide very extensive query features.
Repository based source control tools have a number of major limitations:
· Performance: The performance of repository based tools is generally determined by the size of the repository and the number of concurrent users. Whilst this is not generally a problem on smaller projects, larger projects may experience severe problems. Such tools may impose restrictions on the size of directories and libraries; one vendor recommended that directories should not exceed 150 files each with a limit of about 500 files per library.
· Administration: Repository based tools may require extensive administration skills and overheads to operate correctly. Tuning the repository, archiving old revisions and restructuring of repository tables may require extensive effort and skill. This administrative overhead should be viewed as an essential part of supporting repository based tools. In the absence of such administration, the performance of these tools may degrade to the stage where the tool becomes unusable.
· Loading: Repository based tools will impose considerable loading on the workstation used to run the source control tool, the network and the central server used to store the repository. The number of users and the size of the repository also determine this loading.
· Cost: Repository based tools tend to be much more expensive than history file based tools.
This is a standard feature of all source control tools. A tool that does not provide this feature is not worthy of the “source control” title.
Check out is the mechanism by which a user obtains a copy of an item from the source control tool with the aim of making an update. Source control tools maintain a record of all checked out revisions.
Check in is the mechanism by which a user adds a new item or a new revision to the source control tool. Whilst users are able to create items on demand, users should only be allowed to check in a revision provided they had previously checked it out.
As a rule, source control tools provide the following check in/ check out features:
· Any revision can be checked out on demand. A user may check out the latest revision or may check out any intermediate revision.
· Different users can check out different revisions of a given item concurrently.
· Different users can check out the same revision concurrently.
· A given user can check out different revisions or the same revision concurrently.
When checking out and checking in revisions, a user is creating a series of revisions that are all related in one form or another. This relationship can be documented using a tree structure that has the following anatomical parts:
· Root: This is the first revision that is created for a given item.
· Trunk: This is the set of revisions that are created by checking out the latest revision of the items in a sequential manner. A given item has one trunk only.
· Branches. This is the set of revisions that have been created by checking out intermediate revisions.
· Tip. This is the last revision on a trunk or branch.
The following diagram shows a pictorial representation of a tree structure.
Source control tools will automatically maintain an audit trail of all revisions and how they are related. Once a revision is created then it cannot be modified.
All source control tools provide a feature for comparing two or more revisions. The compare function is generally used to help a developer to identify the differences between two or more revisions. Whilst this feature can be very useful, the usability of difference reports is subject to a number of limitations:
· Difference reports cannot be generated for files that do not have an ASCII format. Binary, bitmap or work processed documents cannot be compared using difference reports. Some tools will generate difference reports for binary files, but these are generally unreadable and quite unusable.
· Difference reports for large files can difficult to interpret.
· Difference reports of files that are generated by a code or form generator may not be meaningful.
Many vendors advertise this feature as being one of the most useful when using source and version control tools. Most vendors will point out functionality that can be used to merge revisions produced by concurrent update and many will sell this feature as being essential for large-scale development.
It is true that all source control tools are able to manage parallel and concurrent development. It is also true that many tools have features that will enable one to merge revisions together. However, the missing fact is that the merge feature is generally very primitive and is only usable on the simplest of changes.
Merging non-ASCII items is impossible and may be unpredictable. How does one determine how bits should be merged from a binary file?
Merging revisions that contain complex changes is also very difficult and time consuming. Given that the merge feature generally relies on the tool’s ability to generate difference reports, all the limitations of difference reports affect the usability of the merge feature.
As a rule, source control tools should be configured to avoid or prevent concurrent updates to a given item. Where concurrent updates are required, for an item that is designed to be used on two or more incompatible platforms for example, then concurrent updates should be initiated only after considerable thought and planning. It is also important to realise that once concurrent development is allowed to take place, the maintenance task and the overhead of tracking the various concurrent variants will increase significantly.
Most tools can be configured to prevent concurrent updates to a given item. Where this is not possible, then a process must be defined and put in place to enable developers to track who is modifying what, with the aim of synchronising parallel concurrent development.
All source control tools have one major drawback, once an item is extracted from the tool’s repository, then the identification of this item becomes impossible. Whilst one can run a query against an item to identify its version number and history, once the item is copied out of the source control tool, then identification becomes impossible.
Many users define naming conventions that would enable them to identify the individual items by viewing identification labels that are added to the body of the individual items. Items would typically contain the following labels:
· Item’s name.
· Item’s version number.
· Item’s status.
· Item’s author.
· Item’s create date.
· Item’s modify date.
· Last change author.
· Item’s change history.
Whilst adding the above information to each item represents good configuration management practice, maintaining the information and ensuring that it is synchronised with that stored by the source control tool can be a difficult and time consuming process.
Some source control tools provide a feature that allows one to automate the identification of items by using “keywords” or “meta data”. The owner of an item would imbed certain characters into each item and the tool would replace these with the relevant information when the item is checked out.
By adding keywords to an item, one is able to add the required identification information into items and is also able to automate the process of maintaining this information.
Using keywords is subject to a number of constraints:
· Many tools do not provide a keyword that is expanded to identify the item’s description.
· Keywords cannot be used with binary files, bitmaps for example.
· Files that have a file size check sum value are generally corrupted by the use of keywords. This is due to the change in file size that results from the expansion of a keyword with the appropriate information. A certain tool will replace the “Header” keywords with a string that is made up of the following:
· Item name
· Item version number
· Item modify date
· Item status.
· Some tools do not provide a keyword that translates to the history audit record of the item.
A number of tools provide a feature that enables one to generate fixed length keyword identifiers. With such tools, the user determines the number of characters to be assigned to the expanded keyword. This is done by padding the keyword with spaces, hence ensuring that the file size is not modified once the keywords are updated by the tool.
Keywords can also be used to add identification information into compiled run time modules. By assigning a keyword to a character variable, one is able to imbed the objects identification into the code and hence enable one to implant the program’s id into the program’s code. Once the program’s id is implanted into the run time then one is able to display it on the screen or a help menu. On some platforms, UNIX for example, imbedded keywords can be viewed using a keyword viewer, hence providing a sure method of identifying the components that make up each and every run time object.
Nearly all source control tools are designed to work with 3GL development tools, FORTRAN, Pascal and C for example. Nearly all source control tools are designed to manage and control items that appear as files under the operating system file system. Whilst this feature may not pose a problem to those who develop exclusively using 3GL tools, the average 4GL user is likely to experience some problems.
Many modern development tools store their own objects in repositories of one sort or another. Some tools do not even generate ‘source’; the program logic may be stored in the form of parameters in one or more tables. Such 4GL tools introduce technical challenges that may defer the user from employing a source control tool altogether.
A number of source control tools are provided with seamless interfaces with third party 4GL development tools. Such interfaces enable one to migrate ‘source’ from the 4GL environment and into the source control tool. Seamless interfaces of this type may offer limited functionality, however the benefits of such an interface should not be underestimated. A primitive interface may make the use of a source control tool acceptable as opposed to impossible.
Some vendors will provide assurances that their tool is able to manage any type of object regardless of its type. Whilst this statement is always true, users should investigate the type and extent of this interface. Will the source control tool enable one to access the individual ‘source’ components that are stored in the development tool repository? Or is the tool only able to control the items that are visible to the operating system? The key point here is the granularity of control. Controlling a 4GL tool at repository level offers limited benefits from the configuration management perspective.
A source control tool that is able to interface with all the development tools that are being used by an organisation is obviously much more useful than one that interfaces with a limited set only.
When assessing a source control tool for suitability it is important to review the capabilities of the development tools to establish the possibility of managing individual ‘source’ items even though the source control tool may not provide an obvious interface. Many 4GL tools provide an interface with the operating system that allows one to export individual source items into operating system files. Once an export feature is identified, the restrictions on source control tools are generally eliminated.
Following is a list of key features that a source control tool should have:
· Ease of use.
· Performance.
· Integration with 3GL and 4GL tools.
· Reliability.
Features offered by most tools:
· Delta generation.
· Compare features.
· Merge features.
· Ability to handle text and binary items.
These are tools that are used to automate the task of generating the run time from the source. Build management tools are essentially intelligent scripts that enable a developer to perform the following:
· Document relationships. Most build management tools enable the user to specify the required target, a program for example, and the components that are required to generate this target.
· Document the build commands. A build management tool enables the user to document the various commands that need to be executed to generate the run time from the source. In the case of a 3rd generation language, one would identify the pre-processor, compiler and linker commands.
· Intelligent compilation. At the heart of every build management tool is an engine that compares targets and dependencies to establish whether or not the target is out of date. Such tools compare the date stamp on a target and the items required to generate the run time and then determine the steps that need to be repeated and the targets that need to be re-generated.
Build management tools can also be used to generate simple impact assessment reports. One can typically find what actions will be taken if a given item is updated; for example one can find out what programs will be re-compiled if one or more source files are modified.
Build management tools can be viewed as intelligent scripting tools and are only suitable for automating a build process that is performed using command line directives or commands. Build tools are generally not suitable for automating the build process for 4th generation languages or 3rd generation languages that only have GUI commands.
The following sections describe in more details the various features that one must look out for when reviewing or assessing build management tools.
Build management tools, often referred to as Make tools, were designed to automate the generation of the run time for 3GL tools. This compatibility is driven by the nature of development within a 3GL environment. As a rule, 3GL developers need to perform the following each time they generate a run time object:
· Itemise the source files required.
· Identify the commands required to pre-process the source files.
· Identify the commands required to compile the source files.
· Identify the commands needed to link the compiled files.
With most 3GL tools a developer has to make use of several commands, run in a predetermined order using predetermined arguments. Most developers perform the above tasks by typing the commands at the operating system prompt.
Build tools allow one to encapsulate the knowledge needed to generate the run time in a script that can be run over and over again to generate the same run time with minimal user intervention.
Build tools are not designed to automate the build process of 4GL tools. In fact many build tool vendors will acknowledge this limitation even though the build process for some 4GL tools can be automated using a build tool.
4GL tools can be classified into two types when considering the usability of build management tools:
· Development tools that provide a command line interface.
· Development tools that do not provide a command line interface.
A development tool that does not provide a command line interface cannot be used with a build management tool. Integration between such a development tool and a build management tool is impossible.
Development tools that provide a command line interface may enable one to make use of a build management tool. The usability of build management tools is determined by the following:
· Does the compilation of the 4GL tool generate a run time component that is visible to the operating system?
· Does the 4GL tool provide commands to import/export ‘source’ from the 4GL’s repository to the operating system?
· Does the 4GL tool provide an object that can be treated as the source.
Integration of build tools and 4GL tools will always be subject to some constraints, however in certain situations build tools may be usable with limited benefits.
Build tools operate by allowing one to document the dependencies between the various items that make up the run time. These are physical relationships that are used by the tool to determine when a compilation is required.
All build management tools require the developer to specify the target to be generated and the dependencies needed to generate the target. For example, one may specify that program ‘A’ is made up of functions ‘a.c’, ‘b.c’ and ‘c.c Each of the functions must be stored in its own operating system file.
A: a.c b.c c.c
The exact nature of the relationship is not specified, i.e. one is not able to identify the precise logical relationship between the target and its dependencies. In the above example, program A is re-generated whenever a.c, b.c or c.c is modified.
Some build management tools offer integration with source control tools. With such tools, the build tool is able to extract the latest version of any required item from the source control tool.
This integration simplifies the build management task, as the user does not need to check for and provide the latest versions of the required files.
As the title implies, these are tools that can be used to manage and track change and problem reports. Change and problem tracking tools are new to the configuration management market and many are now supplied as add-in tools to complement source and version control tools.
Change and problem tracking tools are either simple databases for storing and reporting on changes and problems or may provide extensive workflow management features.
Simple change and problem tracking tools enable developers to register requests that are then progressed and their status updated on the tool. The tool normally provides facilities to enable one to document the impact of the change, the proposed resolution, the date the request was initiated and the date it was completed.
Work flow management tools are much more complex as these can be used to document a change management process by enabling one to document the states that a request must go through and the appropriate roles involved. Such tools have features to broadcast requests, forward them to nominated roles and enable one to progress a change through its states using electronic authorisations and signatures.
Some change and problem management tools provide integration with source and version control tools such that the audit trail stored in each is consistent with appropriate cross-references between the two tools. This enables one to view the history of a change and identify the items affected or view the history of an item to identify the changes that were applied and how they were authorised.
Change and problem management tools are used most commonly during system testing when they are employed to document software defects.
Help desk tools are not dissimilar from change and problem tracking tools and appear to perform the same function. In reality the help desk tools serve a different purpose to that served by change and problem tracking tools. The most fundamental difference between help desk and change and problem tracking tools is that help desk tools tend to contain extensive information about the end users who are being supported using the tool, e.g. name, address, hardware and software configuration being used, and may also contain special features to enable the operator to diagnose and resolve problems as they are being entered. Source and version control tools rarely contain this type of information.
A help desk tool provides the following functionality:
· Provide features to record help desk calls. These may be calls to report problems, requests for help or simple observations.
· Provide features to document the location of the user, his/her address and the configuration of hardware and software being used.
· Provide features to track help desk calls by generating reports or automatically escalating those outstanding calls.
· Provide a database for documenting the resolution of calls. This knowledge base can be used to assist in trouble shooting and in resolving similar calls in the future.
Help desk tools tend to be employed to support live applications. End users contact the help desk operators to report problems and enhancement requests. Problems that cannot be resolved by the operator may be assigned to other support or development staff.
Help desk calls that require resolution by modifying the application code may trigger the generation of change or problem reports on the change and problem-tracking tool. Such reports are then followed up by development or support staff resulting in a new release to the end users.
As the name implies, theses are tools that are used to document the inventory at a particular location or that is being used by a given user. Inventory management tools provide features to perform the following:
· Itemise the hardware configuration at a particular location. Each item would have a unique reference or asset number.
· Itemise the software configuration at a particular location.
· Itemise the applications installed at a given location.
Help desk tools provide additional features to document the following:
· Ownership.
· Price when purchased.
· Depreciated price.
· Reference to supplier, warranty and service agreement.
· Age and problem patterns.
Inventory management tools provide features to document information on the inventory and its configuration. These tools do not provide guidance and assistance in configuring the inventory.
Inventory management tools are normally integrated with Help desk tools as they assist the help desk operator to establish the platform against which the help desk call is being logged. This information may enable the operator to identify problems that are due to incorrect configuration.