Project Management of Large and Small
Multimedia Development Projects

These notes describe project management methods for large and small courseware development projects. They first describe the role of management in software and courseware development, and what activities this management involves. Three scheduling methods are then described and the use of software and schedules in project management. The relationship between the project manager and the development team is discused. Costing methods are then described in some detail, particularly the COCOMO method. Its adaptation to multimedia courseware development is described and then practical methods of costing and current rates are discussed. Finally, guidance on the management of small projects and teams is given.

1. Management in the development process

The fundamental activities in software and courseware development are analysis, design, production (including publication or installation) and evaluation-revision (including testing). The simplest model of courseware development is the familiar waterfall model which is a sequence of these activities (Figure 1); other versions add revision-evaluation at various points. Management is not a separate activity in the process, it is the control of the development process and each development activity.

------------------------------------------------------------------------------------------------

Figure 1

The waterfall model

Activity                                 product 

Analysis of requirements

V requirements specification

Design of software

V design specification

Production

V software

Testing

V working software installed

Maintenence

-----------------------------------------------------------------------------------

Project Management (PMT) can be contrasted with line management (Rickard 1994). Line management (so called because there is a clear line of responsibility) controls the continuous activities of an organisation, for example, human resource management or management accounting. In contrast, projects have a start and an end in time (even if the end is later than expected!). So PMT involves some different procedures and skills to line management, although it also shares many. It is a notorious fact that software development projects of all kinds often finish late, or over budget, or both. So PMT cannot be easy. It attempts to achieve the project objectives on time and on budget, by managing the development activities. PMT applies to any project. In software and courseware development projects the objective is installed, working software (and related products).

The three variables to be managed are

the quality of the product being developed (meeting the requirements)

the time scale (the delivery date)

the resources used (the cost).

These three are in tension - improving one tends to damage the other two. Higher quality costs more and takes longer. Reduced development time (faster delivery) could be achieved by using more resources (hiring more developers) or reducing quality (in functionality or robustness).

The appropriate PMT methods depend upon project size. The bigger the project the bigger is the management problem. A tiny project can be managed informally, for example, one person can manage the process of buying and cooking food for a meal without formal methods. Large projects such as getting a man on the moon or building the Channel Tunnel need project management as a major part of the total activity and cost, and use formal methods.

It is important to distinguish three fundamental roles in a project (Britton & Doake p. 169)

the client, (whose requirements are being satisfied, and who may be paying)

the developer (which may be a team of analysts, designers, programmers)

the project manager.

2. Management functions.

What is involved in the management of software development? Britton and Doake (1993) list the activities of

planning

estimating time, effort and cost

setting sensible tasks for the development team

using prior experience in scheduling work

using resources efficiently

monitoring and controlling progress

evaluating work and maintaining quality

identifying problems and adjusting plans

Sommerville (1989) presents a shorter practical list:

proposal writing (to get a project bid accepted)

costing

planning and scheduling

monitoring and reviewing

selecting and evaluating personnel

report writing and presentations

These lists are aimed at large commercial software projects. The activities of project management are similar in general software development and in non-computer instructional development. Foster (1993) suggests that even in an academic environment (where PMT is generally weak or absent) development of any course requires management as

Planning - in advance of development

setting objectives

programming (activities through time)

scheduling

developing procedures

budgeting

Leading - interpersonal activities

championing the vision of the course

taking responsibility for decisions

selecting team members

developing the team (training)

motivating team members

communicating well

Organising - structures

designing an organisation structure

allocating tasks and defining roles for team members

delegating responsibility and authority

creating an effective environment for team work

Controlling - keeping to the plan

developing standards for evaluating courseware design

measuring and evaluating the course against those standards

taking corrective action on shortfall

keeping to plans and schedules

Many of these activities are necessary in continuous line management too. All these lists assume a medium to large project with a team of several or many people. We will next consider methods specific to PMT.

3. Project Management methods

A schedule is the conversion of a project plan into a real timetable. Three methods are widely used in making schedules and keeping to them: project milestones, Gantt charts and critical paths.

Milestones

Project milestones are used universaly in monitoring and controlling projects. Because software is an intangible product, development progress cannot be seen or measured as it can with, for example, building a ship. Information on progress can only be provided through documents describing work carried out. When a project is planned a series of milestones are established and, at each one, a formal progress report is presented to management and possibly the client. The milestone will signal the completion of a development task, even if the task product is later revised after evaluation.

Each main phase of development (analysis, design, production) will have a milestone but there will probably be several for each phase, one for each smaller task within them (e.g. feasibility study, outline design, interface design). Milestones should not become too frequent or onerous, or producing them will become a major activity in its own right! A milestone every few weeks is typical.

As well as signalling progress in the schedule, the milestones are the opportunity for quality evaluation and review. In a commercial setting this means that the client can evaluate the products of development phases or tasks, and their 'sign-off' is part of the contractual relationship.

Figure 2 shows the development method used in Sanderson CBT which includes several sign-off points. Figure 10-7a from Meredith and Mantel (1989) shows an example project documentation. The project is a network of activities, including many sign-offs. The sign-offs are actually signed in the boxes at the top of the figure. For example, the bottom line of the figure includes ''wax sign-off" and this is signed in the (penultimate) box labelled "sculpting".

----------------------------------------------------------------------------------------------

Figure 2 Sanderson CBT
Life cycle using the Toolbook authoing tool

(analysis)

client call

initial analysis

prototype with Powerpoint

detailed needs analysis

report to client - sign-off

(design and production)

prototype developed with Toolbook

usability test

amendment cycle

sign-off

full content design in Toolbook

edit and test

Quality inspection

sign-off module by module

--------------------------------------------------------------------------------------------------

Milestones are easier to plan in a sequential development and their use is a major advantage of the waterfall model over more iterative prototyping models (Sommerville 1989). Nonetheless, if prototypes are being used they should still be the subject of milestones and sign-offs. Otherwise the project cannot be controlled.

Gantt charts

One of the oldest methods of representing project activities was invented by Henry Gantt in 1917. It shows the activities as bars on a horizontal time scale, both as planned and as actually achieved. It presents a clear picture of the current status of the project for monitoring purposes. It contains a lot of information but in an easily understood form.

Figure 8-18 and 8-19 from Meredith and Mantel (1989) show a Gantt chart of a small project at the start and then after 22 days with some activities achieved. They show activities labelled as a, b, c... vertically, and milestones and the current date horizontally.

Figure 8.3 from Rickard (1994) shows a Gantt chart for software development with milestones shown as 'reviews'. Only the planned activities, not the actual ones, are included.

Variations on the Gantt activity chart can be found. It is common to have layered activities so that a top level shows only the main phases but a lower level shows the detailed activities. Gantt charts do not show interdependencies between activities (one depending on a previous one being completed) or the resources being used.

Critical Path Method (CPM)

One important feature of scheduling is the dependencies of some activities on others, which must therefore precede them. The activities and their outcomes can be shown as a network (of nodes and arcs joining them) which displays these interdependencies. The project start is placed at the left and its end, at the right. All the activities are placed between them as nodes joined by arcs, for example figure 8.2 from Rickard (1994).

The nodes in the network can represent either the activities themselves or their outcomes (in which case the activities are represented by the arcs joining outcomes). Both are used. Activities-on-node charts are easier to draw, as in figure 8.2. Activities-on-arc networks are better at showing milestones, as outcome nodes.

The network chart has two main functions. Firstly, it displays the dependencies to help scheduling. Secondly it allows the calculation of which events are on the 'critical path'. Each activity has a planned time period to complete. Unless there is just a single sequence of dependent activities, some activities will be able to run in parallel to others. In this case, one activity x will take the longest time to complete and any delay in x will delay the whole project. Delay in the other parallel activities would not delay the project as x will still be running anyway. Extending this to a whole network it is obvious that one route of arcs from the project start to its end will involve the greatest total time and this is the 'critical path'. Delays in any activity on this path will delay the project end. Delays in activities off the critical path will not necessarily do so. These activities therefore have a 'slack' or 'float time'; the time by which they can be delayed in start or completion without having knock-on effects on the project deadline. It is possible to calculate the critical path by hand, but for large networks it is always done by software.

Note that the same information used in a network chart is used in a Gantt chart, and the one can be transformed into the other. Figure 8-13 from Meredith and Mantel (1989) shows a network for the same project as the Gantt charts 8-18, 8-19 above. It is an activity-as-arc network and the critical path is the top path. (EOT means earliest occurrence time of the event, when the activity ends).

Software support

The methods described can be performed by hand but for large projects they are now always performed by project management software. This was specialised and expensive until the 1980's but has become very common as part of a managers desktop support, for example, Microsoft's Project Manager and Computer Associates Superproject. They are often referred to as supporting PERT (Project Evaluation and Review technique) which is similar to CPM and often confused with it. The advantages of such packages include

speed of calculation and production of charts,

consistency of data between different charts and calculations as all are produced from the same data of activities, their time periods and necessary resources,

better monitoring and control as plans, calculations and charts can be updated simply by changing data on any of the charts,

automatic costing of projects as resources can be given unit costs,

integration between different projects using the same resources at different times, for example special equipment or personnel.

Using schedules

The project plan is written before development work can start and is its framework. It includes

project objectives,

assumptions made in estimating, including risks and contingency "buffer",

resource requirements and costs,

network and activity charts showing schedules, milestones, and resource allocations.

Scheduling is obviously essential to project management, both as initial planning and then as the yardstick for controlling activities to meet project objectives, deadline and budget. However, it is a management tool and should not become the manager. Steve Maguire (1994, chapter 5) describes the project management of the development of Microsoft's Excel for Macintosh. This two year project gradually slipped its schedule and the programming team became increasingly demoralised as milestones "slipped". Weekly status reports were written which gradually showed increased slippage. Weekly meetings concentrated on what was slipping and the time being wasted by some staff as they waited for others to finish work. A task list grew of bug fixes that were going to be needed before product shipping but could not be done yet if the schedule was to be met. The schedule drove the project and demoralised the team.

The surest way to mismanage a project and jeopardize the product is to put so much emphasis on the schedule that it demoralises the team and drives them to make stupid decisions despite their better judgements. (Maguire 1994, p.105)

Schedules are inevitably somewhat arbitrary so the real quantity and quality of the development work must not be damaged by attempts to meet the schedule. The wrong assumptions in Microsoft's use of scheduling on that project included.

all tasks for the two year project were known and on the schedule

each week each programmer would do 40 hours work on the scheduled tasks

the initial time estimates for task were correct

all features would be implemented perfectly.

Planning should be seen, not as a once-and-for-all activity, but as an iterative process. Change is a constant factor in the life of a project; it comes from the client, from the development team and from external circumstances. ..the project manager must be prepared to react to it in the way that brings the most long-term benefits ... to the project as a whole. (Britton & Doake 1993 p.179)

As well as using the schedule as a PMT tool rather than as the manager, long projects should be divided into subprojects. When these are completed as milestones there is something worthwhile to show, thus motivating the team. Successful PMT manages its most important resource: the human resource.

4. The project manager and the development team.

Structures

The project manager (PM) is not part of the development team, and his/her activities are outside the development process. What is their relationship with the development team? One model puts the PM as a co-ordinator between individuals, but this reduces the decision making by the PM and therefore their control of the project. After all, its successful conclusion is their responsibility. An alternative model is the line management view of the project with the PM as the top of a pyramid with clear lines of control down to the team and reporting up to the PM. Lines of communication are vertical. This mirrors the traditional and now largely abandoned view of organisational structure as a whole. All decisions are in principle made by the PM - PM as boss. But this is not a mass production factory! The expertise for many decision lies in the team members.

The best organisation structure generally is a compromise between these two (Rickard 1994). The PM has authority and responsibility and must exercise it (vertically) through, for example, milestone reporting, but the team members have expertise and initiative and must communicate (horizontally) with each other to co-ordinate and motivate their work. Perhaps this is just the general problem of management leadership in a project microcosm.

Golden rules.

The following have no scientific basis but are good advice from various sources

define specific project outcomes

schedule activities and milestones to allow sufficient time

establish rapport with the clients

plan effectively but appropriately, write a written plan and use it,

build and nurture a talented, motivated team; recruit or allocate staff so that existing expertise is used and train others as early as possible

communicate, communicate, communicate with clients and the development team

5. Costing methods and metrics

Different projects require different development efforts, resources and therefore costs. Costing is interesting for two reasons. Firstly, it is a practical need in project planning. A client agrees requirements and wants a firm estimate of cost. Secondly, in attempting to describe what variables affect development effort, we gain insight into issues of courseware development.

Metrics has been used in attempts at costing. Metrics in software engineering is the attempt to quantify (measure numerically), for any purpose, aspects of the development process, software product or related products. The best documented example is the size of software in number of lines of code, others are the person-days for development and readability indices of manuals.

Approaches to costing

There are several ways of estimating in advance the effort needed for developing a piece of software (Sommerville 1989, chapter 26).

Experts with relevant experience arrive at a consensus.

By analogy with similar projects already completed.

Parkinson's Law - work expands to fill the resource available.

Pricing to win - it costs whatever the client has to spend.

Top-down estimation, considering the overall properties of the requirements.

Bottom-up estimation, considering the cost of each component of the requirements and adding them.

Algorithmic cost modelling. A model based on historical data relates a software metric (e.g. size) to cost. The size is estimated and therefore the cost.

All are really concerned with estimating development effort before multiplying in the unit cost of different resources. Methods 1 and 2 are similar and are of no help unless you happen to be an expert and therefore do not need a method! Methods 3 and 4 are similar. Although perhaps cynical, in a commercial setting this may be realistic, and boils down to 'what can we do for £X,000 ?' Bottom-up estimation is a help if you can divide requirements (not software functionality or code - we don't have them yet) into components that can themselves be costed. Subdivision may help but we are left with the original problem for each component. Top-down is where we start from - given the overall requirements, what will it cost?

From the pessimistic tone of the above you may be putting hopes into algorithmic modelling, which will be described below. However, there is no escape from the basic problem that we are trying to predict the future and models are not crystal balls.

Metrics.

The best known software costing model is Boehm's COnstructive COst MOdel (COCOMO). It is an equation based on a mix of statistical data fitting and expert judgements. It exists in different degrees of complexity, some of which we do not need. At its simplest, it says

development effort = A (lines of source code) ^b

where A and b are constants for certain types of projects. In theory, b implies that effort increases faster than the number of lines of code - big programs require disproportionately large amounts of effort. In practice, though, value of b are close to 1.0 so that there is an almost linear relationship between the total effort and the number of lines of code. This is not surprising, as much of the development effort goes into writing them, and the rest is related to the analysis and design leading to the source code lines and the management of those activities. Sommerville discusses the empirical data on lines of code per programmer-month.

Three levels of project difficulty are described which have different values for A and b.

1. Small teams in familiar environments developing an application with which they are familiar.

2. Intermediate.

3. Projects with tight specifications, fitting closely into other hardware and software, regulations and procedures. Validation is tight. Developers are unfamiliar with such a specialised requirement.

The values for A and b are low for 1 and high for 3, leading to the effort/lines curve for 3 being steeper and more upward curving (see figure 3). Values of A and b have been found from empirical data (past projects).

There are many detailed problems with using COCOMO. The computer language being used will have a big effect on the actual values of A and b, and the model was developed with languages like FORTRAN and COBOL in mind. The constants would clearly be different for modern languages and 4GLs or authoring systems. Also, for different working practices and project management methods. In his own organisation, Boehm found values for intermediate projects of
Person-months = 3. (thousands of lines) ^1.12

As well as predicting the person-months, extensions of it also predict the months of development (and hence the persons needed).

The constants A and b in the model, and its predictive power, are going to vary with circumstance and in the next level of detail in the model Boehm described 15 factors which affected the constants. These include, for example, the required software reliability, database size, and software development tools. Each variable (or 'cost driver') is used as a multiplier of person-months. For example, different degrees of software reliability have multipliers from 0.75 for very low reliability, through 1.0 for normal to 1.4 for very high reliability. Combining the multipliers of all 15 cost drivers thus have a marked effect on person-months of development effort. Boehm's model thus becomes

person-months effort = A (lines of code) ^b . m(X)

where m(X) is the composite multiplier for the cost drivers. He used 63 projects to find values for the cost drivers.

To be useful, COCOMO obviously has to be tuned to the developer organisation using historical data on their past projects. Usually this is not available. However, as you may have realised, there is a more fundamental problem in using it for cost prediction in that it uses an attribute of the finished product (lines of source code) as its input. But until developed we do not know this! And there is no reliable way of predicting number of code lines from the requirement specification (Sommerville 1989). So although interesting in identifying variables affecting development effort it is of little direct use.

6. Costing multimedia development

an algorithmic cost model

Ian Marshall and his colleagues (1994a, 1994b, 1995) have adapted COCOMO for multimedia courseware development. He presents a version of the model

effort = a (average training delivery hours) ^b . CD(X)

a and b are constants and CD(X) is a composite of multipliers for as many as 24 cost drivers (compare with the algorithms above). These include number of course objectives, the existence of course content, the amount of animation, audio, and video, and so on (see Figure 4). These drivers are identified from accounts in the literature (not from empirical data on effort).

_________________________________________________________________

Figure 4 Cost drivers for multimedia development effort

Drivers are grouped into four categories. Each driver has about 5 increasing levels of multiplier.

1. course difficulty

number of course objectives (20 to 80+)

difficulty level of course objectives (concepts, principles, problem solving)

existence of course content (rewrite of CBT, of text, new course)

2. Interactivity

complexity of interface (text , graphical, windowing)

level of interactivity (linear, branching, adaptive)

question feedback (none, yes/no, remedial)

question style (multiple choice, words, free text, graphical)

graphics (none, existing, new artwork, complex artwork)

graphics density (less than 1 per 20 frames to more than 1 per frame/screen)

animation (none, existing, new, complex)

animation density (less than 1 per 20 frames to more than 1 per frame/screen)

audio (none, existing, new, complex)

audio density (less than 1 per 20 frames to more than 1 per frame/screen)

video (none, existing, new, complex)

video density (less than 1 per 20 frames to more than 1 per frame/screen)

simulations (none, existing, new, complex)

simulation density (less than 1 per 20 frames to more than 1 per frame/screen)

3. development environment

production tool (authoring system, authoring language, high, low level language)

instructional design (none, informal, traditional, modern)

development team size (15+, 10-15, 5-9, 2-4, 1)

subject experience (expert through to none)

multimedia development experience (extensive through to none)

4. subject matter expert

expert's multimedia experience (extensive through to none)

expert's availability (unrestricted through to restricted contact)

_________________________________________________________________

The model is similar to COCOMO in its structure and use of cost drivers. An important difference is that it is fundamentally driven by a metric that is often available from the client as part of the specification - the number of training hours - not one in the finished product. Thus, once constants are known for a, b and the cost drivers this promises to be of practical use.

In their initial study of 14 projects (Marshall et al. 1994a) using only the four cost driver aggregate groups, course difficulty explained 75% of the variation in effort. Adding development environment to course difficulty explained 85% of variation in effort. Interactivity had little effect because it was closely related to course difficulty.

Costing in practice

In practice, how do commercial CBT developers estimate costs? In some cases, Pricing to Win or Parkinson's Law are used. This is 'client pull' and needs no further comment. The alternative is 'developer push' where real costing must be attempted from the initial requirements, from delivery/training hours or learning objectives/instructional items.

For example, one medium sized U.K. CBT developer charges £15,000 per hour of training time (at 1995 prices). In the model above, this means a.(1)CD(X)= £15000 and b=1 (the cost relationship is linear. In terms of the cost drivers, this assumes one graphic on every screen, and one interaction on every other screen with about 90 screens per hour. It assumes a subject expert in the client organisation is readily available and at their cost. It assumes one or two learning objectives (concept or rule) per screen. Costing could be done per hour or per learning objective. If there are no graphics at all, a.CD(X)= £10,000.

If audio is used, the hourly cost is at least £16,000. If there is video involved, a.CD(X) = £20,000. So the cost multipliers for audio is about 1.1 and for video are 1.3 . Audio and video production are expensive. The audio for one hour's training might need a day's use of a sound studio, sound engineer and an actor, say £600. Video costs about £3000 for 5 minutes including camera, director and editor are involved. If video clips are provided by the client (say from a promotional video) the cost (of integration) is less.

Another much-used approach to costing based on final training hours is the ratio of development time to delivery time. The development time can then be multiplied by unit staff costs and depreciation of equipment. Marshall et al. (1994b) quote literature reporting between 50:1 and 800:1. Hawkridge, Newton and Hall (1988) report that Austin Rover quote a ratio of 100:1 for CBT but think that this is an underestimate, compared to 15:1 for conventional trainer-led training. Canale and Wills (1995) report a survey by Brown of Australian CBT development costs. While people in the CBT industry estimated 100:1, the data collected indicated 217:1, although less for larger projects. This resulted in costs of between (about) £10,000 and £20,000 in 1991. (Exchange rate is AU$2.2= £1 in 1995.) Other projects Canale and Wills report have costs per delivery hour of AU$33,000 to AU$50,000, and AU$35,000. If these seem high, they quote one interactive video program for bank managers at AU$100,000 per hour !

Tay Vaughn (1994) makes a number of practical points about costing multimedia production, including the difference between internal rates (costs) and external rates given to clients (pricing), the difference being profit. She quotes overall pricing rates of between $55 and $170 per development hour. At an optimistic 100:1 ration this is $5,500 to $17,000 per delivery hour. At a more realistic 200:1 it is $11,000 to $34,000 per delivery hour (exchange rate is $1.6 = £1 in July 1995). This is a little cheaper than the UK company quoted above.

The ratio of development to delivery time as a measure of productivity and method of costing is well established but can only be approximate. Marshall et al. (1995) reviewed the real factors affecting it, such as organisational context and software quality. They also noted inconsistencies in measurement of the variables involved, learner time and development effort. Development of a more accurate costing method continues.

Canale and Wills (1995) looked in particular at the anticipated and the actual costs of project management in multimedia courseware development. In developing multimedia courseware for managers they estimated 7% of total labour costs for project management. In fact it turned out to be 16%. The whole project ran over budget, so that the overrun on project management costs was actually four-fold in cash terms. Nonetheless, they insist that when projects fail it is usually because of project management rather than technical obstacles. The value of project management exceeds its quantifiable cost, which they recommend as 15% of labour costs.

7. Management of small projects

I have described project management in general and costing in more detail. Throughout, the assumption has been medium to large projects with teams of a few or many people. Some aspects of project management are not applicable or trivial. Based on the above, what practical guide-lines can be given for small projects with teams? (For example, one or a few people developing a multimedia application over a few months, for a client.) This assumes no financial transactions are involved and the only resource is staff time. Nonetheless this resource is limited, and a quality product is required and on time.

Referring to Foster's list above, many of the Planning and Controlling activities are relevant, and some aspects of Leading and Organising.

1. Planning is important and should produce a written plan.

• Objectives need to be stated in detail,
• make clear what development model is being used, that is, when evaluation and revision will take place,
• development activities (analysis, design, production, and evaluation) need to be scheduled and their products used as milestones.

2. Costing is not needed but the time resources required do need to be estimated so that the project being planned is realistic in the time available.

• Work on the assumption of a minimum of 100:1 development:delivery hours. This assumes about 90 screens in an hour and about the same number of instructional points, or rather more. Increase this significantly if
• any media are used in addition to text,
• if the course involves anything other than verbal information or simple concepts,
• if learner input and feedback are more than simple yes/no,
• if a programming language rather than an authoring system is being used,
• if the content and the subject expert are not immediately available.

3. Leadership is hardly needed but even with a client and one just one developer/project manager clear communication, rapport and positive motivation for the project must be maintained.

4. Unless the developer is expert in the subject and in using the media needed for the project, time must be allocated to training in these.

5. Organising activities will be few and authority is not relevant, but even with as few as two or three developers there must be a clear (written) and agreed allocation of responsibilities including project management.

6. Controlling is necessary if the project is to meet its delivery date.

• Evaluation plans must be made and appropriate standards used.
• Milestones must be monitored and if not met on schedule the reasons must be discussed. The effect on the final delivery date and product quality is what matters.
• A Gantt chart may be useful to show activities and their products, especially if they are overlapping rather than sequential. If the PM is not familiar with a project management package it is not worth learning to use one for this purpose.
• CPM is unlikely to be needed unless the production of media can proceed in parallel, in which case knowing the critical path may avoid delays.

If a project report is to be written, the project plan and management process will form part of it, and the milestone reports and development activity products will provide much of the final report content.

References

Britton, C. and Doake, J. 1993. Managing the Project . Chapter 7 in Software system development: A gentle introduction. McGraw-Hill, London.

Canale, R. and Wills, S. 1995. Producing professional interactive multimedia: project management issues. BJET 26 (2) 84-93

Foster, G. 1993 Managing course design. BJET 24, (3) 198-206.

Hawkridge, D. Newton W and Hall C. 1988 Computers in company training. Croom Helm , London

Maguire, S. 1994. Debuging the development process Microsoft Press, Washington

Marshall, I. , Samson, W. and Dugard, P. 1994a. Predicting the development effort of multimedia courseware. Information and software technology 36 (5) 251-258.

Marshall, I. , Samson, W. and Dugard, P. 1994b. Courseware - how much will it cost to develop? pp.1-14 in Proc. 2nd ALl-Ireland conference on teaching of computing. Dublin City University, Sept 1994.

Marshall, I. , Samson, W., Dugard, P. and Lund, G.R. 1995. The mythical courseware development to delivery time ratio. Computers and Education, submitted 1995.

Meredith J. and Mantel, S. 1989. Project management: a managerial approach Wiley, NY.

Rickard, S. Delivering on time. Chapter 8 in New frontiers of learning: Guidelines for multimedia course developers in Higher Education ITTI at Nottingham University

Sommerville, I. Software engineering 3rd ed. Addison Wesley, Wokingham

Appendix materials

Meredith J. and Mantel, S. , Figures 10-7a, 8-18, 8-19, 8-13.

Rickard, S. Figures 8.2, 8.3