An evaluation of three authoring tools:
PC-CAI, Authorware and Toolbook

Stephen Bostock, July 1994

Summary:

This evaluation of three authoring tools uses two devices: the production of a simple piece of courseware is used as a vehicle for evaluation, and a published framework for evaluation is used and discussed. The tools represent different programming paradigms. PC-CAI is a partially structured text language with specialist commands for tutorial courseware. Authorware Professional is a graphical programming environment with a flow chart interface. Multimedia Toolbook is an object-based, event driven language with an emphasis on screen displays and links, but no support for input analysis. The production of the same piece of courseware with each tool allowed a systematic evaluation of their functionality. Other criteria, such as the author interface and support for design, are also discussed.

Contents

~
Contents
	Introduction		1
	Evaluation Criteria	3
			Functionality	
			The author interface	
	Introduction to the three tools	6
	The example courseware	7
	Comments on PC-CAI authoring	7
	Comments on Toolbook authoring	8
	Comments on Authorware authoring	9
	Measures of  functionality	10
	Discussion		11
			Functionality
			The author interface
	References		14
	Appendices:
		evaluation checklists
		PC-CAI courseware documentation
		Toolbook courseware documentation
		Authorware courseware documentation

Contents
	Introduction		1
	Evaluation Criteria	3
			Functionality	
			The author interface	
	Introduction to the three tools	6
	The example courseware	7
	Comments on PC-CAI authoring	7
	Comments on Toolbook authoring	8
	Comments on Authorware authoring	9
	Measures of  functionality	10
	Discussion		11
			Functionality
			The author interface
	References		14
	Appendices:
		evaluation checklists
		PC-CAI courseware documentation
		Toolbook courseware documentation
		Authorware courseware documentation

Introduction

There are hundreds of computer programming languages both general and special-purpose, and many have been designed for the development of instructional software. Barker (1987), for example, listed over 60 'author languages'. Since then, with the standardisation on Graphical User Interface and the falling price of multimedia hardware, many new 'multimedia' authoring tools have appeared, and many evaluations of them. It is not the intention of this report to attempt to synthesise this large literature on specific tools. Instead, ideas from the evaluation literature are used in the evaluation of three authoring tools, using the implementation of a small piece of courseware as a vehicle to provide experience with the tools. This is a common device in evaluations (for example, Dean & Whitlock 1988, Barker 1989, Riley 1994). The process of authoring with the tools can thus be compared and reflected upon. The evaluation framework of MacKnight & Balagopalan (1989), in particular, will be used in evaluating the tools, and its usefulness is discussed.

It is worth clarifying some terms in an historical context. The history of tools used for the production of application software is as long as the history of computer hardware. Generations of programming languages have proceeded from binary code and assembler to 'high level' languages such as FORTRAN, BASIC and Pascal. Code at this level, of course, must then be translated into binary code for execution. Two important principles in the development of such languages are restricted control structures and functional modularity, which in its latest form of 'objects' encapsulates data and functionality. After these third generation languages (3GLs), 'fourth generation languages' (4GLs) allowed declarative rather than procedural specifications, particularly of database applications.

Software tools can crudely be characterised as being increasingly of a 'higher level'; the construction of the software can be done in terms related to the problem rather than the mechanism of its solution. With instructional software, this means being able to specify (in text or graphics) the output to the user by describing it interactively through editors, and specifying the processing of user input in terms of logical possibilities rather than details of code structure. In this sense, an interactive screen editor, for example, is a 4GL when compared to C or assembler code generating the same images. Three author interfaces for software construction are writing procedural text (3GL), menu option selection (4GL) and graphical manipulation of objects. Graphical interfaces may resemble either a 3GL or 4GL depending upon the freedom given to the author: graphical languages can be procedural (e.g. the ladder diagram coding of real-time control systems, the flowchart diagrams of some authoring tools) or declarative (e.g. graphical data flow languages, 'Upper CASE' tools which also generate C code) (Almgren 1994).

Authoring tools are languages or systems for producing instructional or educational software. Current authoring tools show a variety of these programming paradigms, and often more than one in the same package. For example, many provide graphical programming or menu selection specification, and resource editors, but also allow or require text code writing for additional functionality - 'scripting'. Nonetheless, tools can be broadly divided into:

• General purpose languages. Some are particularly suitable for applications such as courseware, having an emphasis on interfaces and user interaction, for example Toolbook, Visual Basic and Hypercard.
• Authoring languages with the structure of general languages but with specialist facilities for user interaction and screen displays, for example Tencore and PC-CAI
• Authoring systems. Fourth generation languages where text coding is not needed and menu option and other selection mechanisms specify the courseware, for example Tencore Producer, Mandarin, and the PC-CAI Create facility. Arguably Authorware, IconAuthor, Course Builder and other iconic systems could be placed here, but this will be discussed later.
• Finally, mark-up languages such as SGML, html (for World Wide Web) and Interactive Document Compiler (for Toolbook) can be used to add procedural code to text. They resemble text authoring languages but are more limited, being oriented to presentation rather than interaction although some produce hypertext.

Authoring is a vague term for software production. In the traditional development 'life cycle', authoring in a narrow sense is the equivalent of construction (coding), the implementation in software of a detailed design. With a language this requires programming skills. Authoring systems attempt to allow construction without programming skills . Some tools attempt to provide support upstream of construction, into the analysis and design processes, and authoring in a broad sense includes the whole software life cycle. The appropriate use of tools of different types is discussed below.

Figure 1 Pseudocode of example courseware

repeat 
	display first screen (three averages)
	display description of mean
	display numerical example overlayed
	display description of median
	display numerical example overlayed
	display description of mode
	display numerical example overlayed
	display problem data
	ask for numerical mean
	if correct (within a range) then praise
		else give correct answer
	ask for numerical median
	case answer of 
		correct as integer: praise
		correct as real: praise and give correct answer
		incorrect: give correct answer
	ask for numerical mode
		correct as integer: praise
		correct as real: praise and give correct answer
		incorrect: give correct answer
	display question about mean
	repeat
		if answer is mean then praise
		if answer is median or mode then try again
		if answer is other then advise one of three words
	until three goes or answer is mean
	display question about median
	repeat
		if answer is median then praise
		if answer is mean or mode then try again
		if answer  is other then give advice
	until two goes or answer is median
	display question about mode
	if answer is mode then praise else give advice
	display question asking for repeat
until user says quit

Evaluation Criteria

Evaluation, of course, must take place against criteria. The criteria for authoring tools can be divided into three broad categories (Barker 1989 p. 222, Hunka 1989 p. 13, Richards & Fukuzawa 1989 p. 28, MacKnight & Balagopalan 1989, Edwards, Howe & Smith 1993 p.5, see Appendix 1)

• Functionality addresses 'What they can do?', and is emphasised in many published evaluations.
• The author interface, in a broad sense, asks 'How do they allow the author to do it?' This is related to the question of how far 'upstream' in the development process the tools attempt to go. That is, do they assume, impose or support certain instructional designs?
• Wider questions concern support, cost, documentation, supported hardware platforms, run-time licenses and industry standards are outside the scope of this report.

Functionality

Functionality features can be listed in the following groups (based on Dean & Whitlock 1988, Richards & Fukuzawa 1989, and Hunka 1989 ).

(a) Presentation of text and questions

The most basic function is to present text in a frame that is either the whole screen or a window. Being able to clear frames is necessary. Variations include fonts and colours. A list of text editing functions is given by MacKnight & Balagopalan (1989) and in Figure 2.

(b) Acceptance and analysis of user input

The simplest input is characters, analysed as text or numbers, but graphical user interfaces also need to monitor pointer (mouse or other) movements and buttons. Analysis includes:

(i) structural aspects such as labelling responses as right or wrong for recording, and counting user tries;

(ii) Matching inputs. With text this includes dealing with blanks, capitals, alternatives, mis-spellings and matching various substrings. For numbers this includes accuracy ranges and types (interger/real). With a (mouse) pointer, the position, movement, button clicks, double clicks and buttonDown/buttonUp actions can be detected.

The list of features in MacKnight & Balagopalan (1989) listed under Instructional Strategy are mostly concerned with input.

(c) Filing of user responses details

Filing details of user software and actions can range from the simple, such as a bookmark for the following session, to detailed logs of questions and answers. Standard reports of these details will be needed by teachers.

(d) Control structures and other programming features.

Some method of conditional branching is necessary to act upon user inputs; in unstructured languages this requires statement like IF..THEN and GOTO Label. Structured languages allow multiple statements (blocks) within IF..THEN..ELSE and SELECT..CASE structures. Iteration could include REPEAT, WHILE and FOR loops. Iteration of user inputs could be dealt with automatically (counting tries). MacKnight & Balagopalan (1989) list different types of branching under Instructional Strategy features.

Modularity is desirable. Subroutines, functions or object classes are possible. There are then issues about parameters and the existence of local versus global variables.

Validation and debugging facilities are desirable. Validation of syntax at least is the simplest. Validation of sequence is desirable, such as eternal loops being created or subroutines not being used or exited. Validation of design features is possible, such as detecting excessive amounts of text in a frame or excessive iteration. Debugging tools are needed to fix errors in substantial courseware.

(e) Interfacing other programs

As no authoring tool will provide all the functionality required, at least not at an appropriate speed, it is necessary to be able to call other executables to perform special functions such as animation, data file analysis, or control special devices.

(f) Media editors

Traditionally, presentation was historically text, later supplemented by bitmap graphics. Now, it is normal to display a wide range of graphics formats, sound (e.g. from WAV or MIDI files), moving video (Full Motion Video, or within-window), and animation (path or cell). All of these resource types need their own editors, either within the authoring tool or compatibly linked with it.

(g) Authoring environment, in the terms of MacKnight & Balagopalan (1989), refers to general authoring facilities and includes 35 characteristics (Figure 3).

Author Interface

The second criterion of 'How they do it', their 'look and feel', is the most obvious distinguishing feature of different systems (Gardner & Munroe 1992). Here authoring systems can be arranged along two dimensions: the degree of programming needed (as text or graphics) as opposed to declaration or menu selection, and the dependence on typing text code as opposed to a graphical author interaction (Riley 1994).

( ü is a tick )

Figure 2 Tasks for text (after MacKnight & Balagopalan) creation Balogopalan 1989) Tasks PC-CAI Toolbook Authorware Set foreground ü ü ü colour Set background ü ü ü colour Set text size (2 sizes or as ü ü images) Set text spacing ü Set fonts (only images) ü ü Set style ü ü Highlighting ü ü blinking ü overlay graphics ü ü overwriting ü ü rotation pause texts ü ü ü foreign characters ü ü Static window boxes + cursor ü ü control dynamic window ü ü import files ü ü full screen erase ü ü ü part screen erase ü ü centering ü ü ü justification ü ü superscript ü ü subscript ü ü symbols ü ü text animation (as graphics) ? ü editor ü ü (poor) create text ü ü ü timed pause ü ü touch pause ü ü ü embedded variables ü ü Total: 10 / 29 26/29 25/29 Figure 3 Authoringenvironment (after MacKnight & Balagopalan) Authoring Environment (after MacKnight & Balogopalan 1989) Tasks PC-CAI Toolbook Authorware Create course ü ü ü Open/edit course ü ü ü Save course ü ü ü Delete course create frame ( cls before text)ü ü ü title frame ü ü rename frame ü ü edit frame ü ü delete frame ü ü combine frame ü ü expand frames ü ü list frames ü rename course ü ü create multiple ü ü course test course ü ü ü test part course ü text editor ü ü ü graphics editor ü ü video editor ü ?ü audio editor ü ü variable editor ü ü character set (windows)ü ü editor graphics library ü ?ü strategy library ü ü character library (windows)ü ü online help ü ü author/user not interactive ü ü up/download transport to Macü to Macü courseware configure hardware windowsü windowsü create student ü ü ü disk print course ü ü external link ü ü ü Total: 10/33 27/33 29/33

Introduction to the three Tools

PC-CAI 2.10 is shareware available from Texasoft. It runs under MS-DOS and was first available in 1983, so it is useful in the current context as an example of a traditional authoring language. It is a text authoring language (3GL) with a simple text editor, although any text editor will do, but no other resource editors. It also has a menu-driven authoring system called 'Create' (a 4GL) which generates text code. It can import and animate graphics and create simple sounds but has no facilities for creating graphics or handling video. The latest version was used.

Multimedia Toolbook from Asymetrix is the nearest MS-Windows equivalent to Hypercard on Macintosh (and Hypercard stacks can be largely converted automatically to Toolbook books). It most easily generates hypertext. A graphics editor creates screens (cards/pages) including Windows standard objects. No text code is generated for screens. It is event driven, with objects (both graphical and text procedures) receiving messages from user inputs and passing them to other objects or responding to them as determined by their script, or text code. The code must largely by typed by hand. Version 1.53 was used; a major upgrade to version 3 is due in 1994. It has facilities for graphics, animation and sound. It is a general Windows development tools, with no built-in facilities for input analysis but a good interactive screen editor, and a library of multimedia objects for customisation. The author interface is a page on which objects are placed, menus and dialogue boxes for selecting options, and a text editor for writing scripts to define object responses.

Authorware was used in the Working Model of Version 1 for MS-Windows, allowing only 50 icons/activities in an application, but the full version was observed at Keele. Authorware applications for Macintosh can be converted to WIndows, but not vice versa.Version 1 supports sound and graphics but has limited support for animation and video. Version 2 has enhanced support for multimedia, with a Media Manager maintaining a separate library of resources (Dean 1994). Macromedia Director is a version especially for authoring multimedia. Version 2 also permits hypertext links between objects. The Authorware interface is a flowchart on which icons are placed.

Toolbook and Authorware are amongst the handful of leading packages for the Windows environment (e.g. Reynolds 1994, Beilby 1992, Davies, Scarborough & Brailsford 1992, Riley 1994). They share some Windows interface features and access to Windows multimedia resources.

The example courseware

The example courseware was designed for its simplicity and small size, requiring only text displays so as not to prejudice the comparison by emphasising special effects which one tool did well. The courseware has simple displays and interactions about descriptive statistics. It consists of descriptions and illustrations of the different averages: mean, median and mode. It gives a numerical problem that requires each average to be calculated and entered, and it asks the learner to distinguish each from a description. Finally there is an option to repeat the lesson. Only text is displayed, and text and numbers are input. It is described as pseudocode in Figure 1.

Comments on PC-CAI authoring

PC-CAI is a traditional procedural language with few control structures (selection by IF and CASE but no iterative structures or modularity) but it does have commands designed to make screen displays of text and graphics, and analysis of user input, easier than in a general purpose language. It could be treated as a mark-up language, as tutorial text in the code is displayed on a scrolling text screen unless interrupted by embedded commands that begin with the '.' or '@' characters. To give a flavour of the language, the important commands used in the example code (Appendix 2) are described below.

The 'dot' commands can appear anywhere in the code.

.rem - a remark line to allow code documentation

.screen 0 - to select screen type. Only CGA modes are available.

.color 0,7 - to select foreground and background colours

.cls - to clear the screen. Otherwise the screen scrolls.

.skip 9 - to move 9 blank lines before continuing to display text.

.locate 24,1 - puts the cursor at the x,y character position before reading or writing

.pause - waits for an Enter key press and displays default prompt.

The '@' commands create the code structure. Upper case is used for documentation purposes.

@LABEL:START - line label for use with the commands @GOTO and @GOBACK

@BEGIN QUESTION - start a question section

@TRIES 1 - determine number of tries allowed

@READ(*What is the median value ?*) - get the user input

@RIGHT (*4*) EXACT - specify the correct answer 4 as text (various options.)

@WRONG(*4*) DECIMAL - specify a wrong answer, treated as a number.

@WRONG - start a section following a generic wrong answer.

@END QUESTION - finish a question section.

The two main parts of a CAI programme are the commands and text determining the display, and the QUESTION sections. Both right and wrong answers must be specified along with feedback text for each answer. The number of tries can be determined, and the following options used in answer matching.

keyword - the input must contain the specified word (substring) somewhere

exact - exact match

exact nocase - case is ignored. Nocase can be used with other options.

decimal - treated as a numerical value. A range of correct values can be used.

list - a list of correct values, in any order

soundex - accommodating misspellings.

force - the correct answer is required

PC-CAI is thus rather like the 'ideal' language HAL in Dean and Whitlock (1987), although it performs no validation and has no debugging facilities.

Writing PC-CAI code is simple in some ways: the text appears on the screen much as it is typed, modified by commands to control the cursor. However it suffers from two problems of procedural languages. Firstly, because the screen displays are not being created interactively, the programmer must count lines and character positions since the last .cls or .locate command. Even then, there is an element of trial and error when putting the finishing touches to the exact appearance of the screen if scrolling is to be prevented. Secondly, there is no help in debugging the structural '@' commands. For example, if an @END QUESTION command is omitted, the program runs straight into the following question. No debug help appears. For a long program this would be a serious problem.

However, if treated as a mark-up language PC-CAI is an easy way of quickly adding interactivity to existing text. Furthermore, the authoring system PC-CAI CREATE will generate source code following a question and answer session with an author, producing much of the code.

Comments on Toolbook authoring

The Toolbook example consists of both graphical objects and text code defining their responses to messages created by user input. It cannot easily be fully documented on paper, but the appendix includes some screen dumps and example object scripts. It is an object-based rather than an object oriented languages in the sense of C++ (Reynolds 1994).

The graphics editor makes it easy to design pages (frames), and the ready-made Button object makes it easy to move between them and to repeat the lesson. Positioning fields is interactive, and changing text characteristics is simple with menus. A graphics toolbox, grid snap and colour palettes provide allow for a professional screen design. A dialogue utility enables the creation of standard Windows dialogue boxes.

Objects include buttons, hotwords, graphics objects, page background, and the book (application) itself. Each can have a script consisting of one or more handlers for messages generated by the user or other handlers. There is a built-in hierarchy determining which object gets the chance to handle each message if it can. The syntax of Openscript appears English-like but has pascal control structures. It is also just as stringent on detail, and so must be learnt. The Openscript Help system is good. However, a syntax check will identify errors when writing scripts. A 'debug window' can be used to isolate and correct errors at run time. It allows breakpoints to be set, displays of local and global variables, and tracing of script execution and message passing. There is also a command window through which the author can directly manipulate objects, send messages and test scripts. Openscript statements are executed directly.

Creating separate fields for sections of text was repetitive but could be partially alleviated by Copy-and-Pasting them across pages. Page backgrounds can give standard layouts. There are no built-in functions for analysing user inputs, so this is done either by providing graphical buttons or writing Openscript code to match text answers. The most difficult code was that for allowing three answer tries; more so than in a procedural language. Openscript variables are not 'typed' and so numerical answers must be interpreted as text or numbers on use.

Comments on Authorware authoring

As the Authorware Working Model was being used the courseware had to have sections removed to keep within the 50 icon limit, but it illustrates all the aspects of the design. Flowcharts and example screens of representative sections are appended (Appendix 4). The present author had not used Authorware or anything similar before this task, and did not have access to manuals. About twice the time needed for either the Toolbook or the PC-CAI authoring was needed with Authorware. However, some help from colleagues was given through the module conference, and I had one discussion with a technician who had used Authorware to produce professional courseware. No doubt various small problems I encountered had simple solutions.

Authorware applications are constructed by dragging icons representing presentations, interactions and so on, onto a 'flowline'; a vertical line specifying the sequence from top to bottom. Control structures deviating from simple sequence must use either the selection or interaction icons to branch into alternative sequences, which cannot be shown at the same level alongside the main sequence. Branching inevitably requires the nesting of levels, each level being presented in a separate window. A major practical difficulty in using the 'flow line' window is that it does not scroll. On a VGA screen this means a maximum of about a dozen icons can be used before having to group icons into Map icons. As these icons had to include a Wait and an Erase icon for each frame, this means about three frames per flowchart. This led to an unnecessary degree of modularity, and of a hierarchy of levels in windows (see appendix 4).

An alternative way of producing a simple sequence of icons is to put them onto a Selection icon sequence with the select all options set. This ameliorates the problem of vertical spacing, but illustrates the sometimes illogical uses of the Selection and Interaction icons. For example, the repeat control structure also requires the Selection icon, with repeat..until options set, but the while structure is not possible. It is not possible to determine sequence other than through these icons, unless an unstructured GOTO is used.

The courseware is executed by Running the iconic program, and user displays and interactions occur in a separate Presentation window. It is possible to place start and end flags on the program to run only the specified section, but no other debugging tools seem to be available, and there is no on-line help.

The Erase icon illustrates a problem in having a completely graphical interface: it requires identification of what objects to erase, so the Presentation Window must contain those objects at the time the Erase icon is selected. This may mean having to Run the previous Display icon merely to specify its erasure. Screen objects in Authorware have no identification (unlike Toolbook where they have unique identifiers and possibly user defined names) so specifying an object must be done graphically, which in this case is tedious.

The Interaction icon is used for analysing user input. There is a good variety of options for the user inputs that can be required (which in Toolbook would be handled by object code). Common string matching options are available including capitalisation, spacing and substring matches. I had some difficulties with matching numerical input values. Attempts at writing expressions to deal with inputs failed; presumably a built-in function exists which allows a numerical interpretation of input characters.

The Restart command worked in the analysis of the last question. The alternative course of action, 'do nothing', required an empty remark statement to satisfy the syntax. This is an irritation in a system that attempts to minimise text coding.

Even allowing for much trial and error in the use of icons, the authoring process involved a great deal of dragging icons around and manipulating separate windows of flowlines for different levels.

Measures of Functionality

MacKnight & Balagopalan (1989) describe a detailed checklist of authoring facilities for evaluation (their 'Taxonomy of Tasks'), which are functional capabilities. They divide these into ten groups. The two most basic are Authoring Environment and Text Creation. These are the largest groups and the most relevant to this exercise (listed in Figures 2 and 3). Their Authoring Environment group does not correspond with the evaluation criterion 'author interface' being used here, being concerned with what can be done rather than how it can be done.

MacKnight & Balagopalan (1989) develop three measures from their checklists. Functionality is the proportion of possible Tasks a package possesses. Figures 2 and 3 list the tasks and Figure 4 shows the functionality indices. The measures for Toolbook and Authorware is very similar; they are both capable packages and share the facilities of MS-Windows. PC-CAI has many fewer facilities, notably lacking frames for text and editors for other resources. The difference reflects the development in authoring tools in the last few years.

Figure 4                                                                           
Functionality                                                                      
measure              Tasks           PC-CAI          ToolBook        Authorware    
                   (Maximum)                                                       

Text creation          29              34%             90%              86%        

Author                 33              30%             81%              88%        
Environment                                                                        

Flexibility (in their terms) is measured as the smallest number of steps needed to accomplish a task. Within a group of tasks the total number of tasks divided by the total number of steps needed to accomplish them is the flexibility measure. This was not measured because a moment's reflection shows it to be of little use when comparing such different interfaces. The number of keystrokes needed to type PC-CAI commands would be very large, so it would have a very low index. Toolbook would have fewer keystrokes as frame creation is interactive and some options are menu-selectable. However, text for input analysis of answers has to be hand-coded. Authorware would have fewest keystrokes and the highest flexibility index as very little text code is needed - strings for answer matching, a few Boolean expressions, and frame labels. The flexibility index values, then would be dominated by the degree to which text keying is necessary, so they would measure the degree of graphical input.

Productivity is the average time taken by an expert to perform these tasks: the total keying and thinking time to accomplish all the tasks in a group divided by the number of tasks accomplished. There is no attempt to measure Productivity in this narrow sense here. Firstly, the current author was unequally expert with the packages, having some knowledge of PC-CAI and Toolbook before. In practice, the Toolbook and PC-CAI applications both took one day to complete. The Authorware version also took a day but after an initial day of experimentation with the package. It seems likely, therefore, that for a simple courseware exercise such as this there would be little difference in productivity.

Discussion

Functionality.

The simplest evaluation is in terms of functionality, which was divided above into seven categories. For Text Creation, the two Windows tools are similar in their functionality index (Macknight & Balagopalan 1989) while PC-CAI is much more limited (Figure 4); similarly in the Author Environment functionality measure. In the second category, input analysis, PC-CAI provides good matching facilities for text and numbers. Toolbook allows for text and numbers and the detection of pointer actions, but no answer matching functions are provided - these must be hand-coded. Authorware provides many menu for answer matching, although not for numerical answers. In the third area of recording user responses, Authorware has good facilities and PC-CAI can also record and retrieve grade information at run-time. In Toolbook they would be hand-coded.

In the category of program structures, PC-CAI is unstructured (apart from within Question sections) so the author can produce any control structure, with some programming expertise, but with the usual risks of unstructured code. It has no subroutines although applications can call other CAI applications on disk. Toolbook has standard control structures. It is object-based and functionally modular. Authorware's control structures are sequence along the flow-line, condition only through the Selection icon, and iteration only by selecting repeat in the Interaction icon. There is no while loop. It is also possible to perform a jump GOTO a labelled icon, allowing unstructured programming. Apart from the GOTO option, iteration and selection are determined by the options within icons. This contrasts with Course Builder (another iconic authoring system) where the author graphically determines the sequence between frames, which could lead to chaotic structures. In Authorware the automatic generation of restricted control structures could be valuable in imposing logic, but unfortunately it is arbitrarily organised within icons. Using icons to produce iteration or selection automatically generates lower levels in the icon hierarchy. These were smaller than a more functional modularity required. In contrast, Course Builder allows and requires manual aggregation of icons into modules.

As to debugging tools, PC-CAI and Authorware have none, though this is less serious in the latter. Toolbook validates syntax at coding, and identifies the position of errors at run-time. But there are no trace facilities so a large application may be difficult to debug. Toolbook has good on-line help, the others have none.

PC-CAI cannot interface with other programs. Toolbook interfaces well with other Windows programs, supporting DDE, which Authorware does not.

As to media editors, the two Windows tools have toolboxes for graphics and text. They are available for audio and video for Toolbook and the Toolbook development environment is extensible with third-party extensions. Authorware can only be extended by calling lower level code at run-time. It does not have audio or video editors but Macromedia Director does and Authorware can 'play' its applications.

Macknight and Balagopalan (1989) provide a useful checklist of functions for authoring tools, although not as inclusive as the one used here, but their performance indices are of limited use. The functionality index is only a checklist (Figures 2 and 3). The flexibility index (number of keystrokes) is of little use when comparing the typing of commands with selection from menus or pointer actions. The productivity index is superficially attractive: the time taken by an expert to accomplish tasks with the tools. However, this must include 'thinking time', including such things as debugging and using on-line help, which begs questions of the author environment, programming skills and the scale of project for which a tool is appropriate. The limited usefulness of the functional approach of Macknight and Balagopalan is demonstrated by the similar values for efficiency measures it gives to Toolbook and Authorware despite their very different author interfaces and contexts for appropriate use.

Author Interface

The three tools reviewed here represent three paradigms for the creation of executable code. The two Windows tools represent the two directions of authoring systems described by Hunka (1989). Toolbook allows the author to create the presentation as the user would see it although, unlike Hunka's tool, no text code is generated for the presentation. Authorware represents the Course of Action paradigm where the author first determines the instructional sequence before creating the presentation of resources - icons must be placed on the flow-line before they can be edited. The flow-line is an iconic, high-level code and Authorware is essentially a graphical 3GL with menu options providing most additional specification, although variables and expressions must still be typed.

PC-CAI is a specialist but otherwise traditional 3GL. The source code must be edited directly. However, the Create program is a 4GL which will generate this code from author menus.

Is the choice of interface merely a matter of taste? Firstly, there are detailed features of Authorware that seem unnecessarily restrictive - the size of the flow-line widow being an obvious example. While there no doubt 'work-arounds' the need for these is itself a criticism of the interface.

To pass over the small irritations, there are two aspects to the question: the degree to which it is authoring rather than programming, and the graphical versus text nature of the programming. To consider the latter issue, it would be possible to design an iconic programming language with the flexibility of a general purpose 3GL text language. Authorware is not that language, it is more like an iconic version of a traditional, partly structured authoring language like PC-CAI. Built-in structures suitable for standard interactions are available and generally constrain program sequence, but it is also possible (and presumably sometimes necessary) to use unstructured GOTOs to create other sequences.

The issue of authoring or programming involves support for, or imposition of, instructional design. This can be viewed in two ways. If authoring is taken to be the whole development process, then current authoring tools (including these three) are clearly deficient in support for instructional design (Merrill 1985). Whether Merril's ID2 (Li and Merrill 1991) or other applications of artificial intelligence to the processes of instructional systems development (Spector et al. 1992) will augment authoring systems in future is beyond the scope of this report. In current authoring tools, instructional design knowledge is present as templates, restrictions or implicit paradigms that may improve productivity in some cases; a trade-off between productivity with a restricted set of designs and flexibility (Lewis and Mace 1988, p.3). In the three systems being reviewed it is present as implicit paradigms of branched programming learning or hypertext (Toolbook): 'existing software tools do not incorporate a methodology for courseware production, except by default' (Lewis and Mace, 1988, p.6).

Authoring languages need programming but give to the programmer the flexibility of supporting a variety of instructional designs. Authoring systems present templates, or at least a paradigm of instructional design, waiting for content to be added and options to be selected. Both Authorware and PC-CAI present in their icons and commands the paradigm of branched programmed instruction. Departure from this paradigm becomes increasingly difficult and, in the case of Authorware, needs work-arounds and coding. Toolbook, on the other hand, is a general purpose tool with an emphasis on graphical interfaces and built-in support for hyper-links. Its paradigm is hypertext, but as this paradigm is intrinsically more general-purpose, it can more easily be used for a variety of courseware types. The cost is in the necessity of text coding and programming skills.

In the selection of a tool, the desired functionality is a necessary but not sufficient criterion. Next, limitations in the author interface that will reduce author productivity may cause de-selection. Then, flexibility in the instructional designs easily implemented is important in the selection. An authoring system like Authorware will allow non-programmers to create multimedia courseware without investment in programming skills, as long as courseware fits the paradigm the system suggests. It will avoid the need for formal analysis and design for small projects. However, successful long term and widespread institutional use of a tool will not depend largely on initial ease of use, but on flexibility and extensibility, usefulness in the context of courseware teams and communication between its members, and software maintenance (Reynolds 1994). Here, Toolbook's more general hypertext paradigm more than compensates for its lack of input analysis. Its emphasis on interface design is of advantage in prototyping and its extensibility of editors makes it flexible for changing needs.

References

Almgren, R. 1994. The diagram paradigm. Program Now, p. 74, June 1994.

Barker, P. 1987. Author languages for CAL. MacMillan.

Barker, P. 1989. Authoring for multi-media CAL. In: P.BArker (ed) Multi-media Computer Assisted Learning, 210-224. London: Kogan Page.

Beilby M. 1992. Aymetrix Toolbook 1.5 The CTISS File 13, 16-19.

Davies, P., Scarborough, S. & Brailsford, T. 1992. Authorware Professional: multi-platform icon authoring. The CTISS File, 13, 7-11.

Dean, C. & Whitlock, Q 1988. Using an author language. In C.Dean & Q Whitlock, A handbook of computer based training 213-231. London: Kogan Page.

Dean, C 1994. An overview of authoring systems for CBT and Interactive Multimedia that are available in the UK. Sheffield: Dean Associates.

Edwards J., Howe G. & Smith F. 1993. Advice on choosing an authoring package. Educational Technology Service, University of Bristol.

Gardner J. & Munroe, G 1992. Authoring and authoring systems. The CTISS File, 13, 45-46.

Hunka, S. 1989 Design guidelines for CAI authoring systems. Educational Technology 29 (11) 12-17.

Lewis R. and Mace, T.D. 1988. Support tools for authoring - a seminar report. University of Lancaster, InTER/7/88.

Li, Z. and Merrill, M.D. 1991. ID Expert 2.0: Design theory and process. Educational Technology Research and Development 39 (2), 53-69.

MacKnight, C. & Balagopalan, S. 1989. A evaluation tool for measuring authoring system performance. Comm. of the ACM 32 (10) 1231-1236.

Merrill, M.D. 1985. Where is the authoring in authoring systems? J. Computer-Based Instruction 12 (4) 90-96.

Reynolds, M. (1994) KnowledgePro Windows : the order of merit? ALT-J 2 (1) 57-61

Richards, T & Fukuzawa, J. 1989. A checklist for evaluation of courseware authoring systems. Educational Technology 29 (10), 24-29.

Riley, F. 1994. Understanding IT: a review of hypermedia authoring packages. University of Hull.

Spector, JM, Goage, RM, Muraida, DJ & Dimitroff, WA 1992. Intelligent frameworks for instructional design. Educational Technology October, 21-27.

Appendix 2 PC-CAI screens and code