PCD Awards
SUBMISSION FORM - PCD Tools Category

 

Extraordinary PCD Tools are technologies (tools, products, protocol) that were not previously available commercially or otherwise.  Innovative PCD Tools and Techniques are things that facilitate embedding support into other software, act as agents to deliver decision support at the time of need, integrate disparate end-user software, help anticipate and resolve performance challenges, use general inference techniques to eliminate complexity, reduce the need for conventional IT support, replace more conventional means of support, and the like.  These can be new components of existing PCD Tools and Techniques if they are sufficiently unique.

Excellent examples of Extraordinary Performance Centered Design Tools are those that facilitate creating and integrating performance support environments, such as alternate interfaces, merging disparate systems (via interface integration, middleware, agents, and the like); capturing and disseminating business processes and best practices; improving business processes; embedding “wizards” and other productivity enhancers into existing systems; providing decision support and workflow support; providing resources to quickly troubleshoot problems, answer customer questions, and the like.  Any tool that clearly stands head-and-shoulders above the rest in its category - or that defines a unique category -  for enhancing the creation of EPSS / Performance-Centered Solutions is a candidate for an Extraordinary PCD Tool.  Hardware and hardware devices that fit this description are also candidates.




Entry Title

MAGIC – Metadata Automated Generation for Instructional Content. 

Submitted by:

IBM Corp.

Contact Name:

Galina Kofman

Phone:

914-784-6063

E-mail:

kofman@us.ibm.com

Address:

19 Skyline Drive, 1S-B59

Hawthorne NY 10532

Logo:

Corporate Logo:

IBM logo

 

Classification and state of deployment:

Please classify your entry:

 

Comprehensive performance-centered web-based portal development tool

Comprehensive performance-centered content/learning/knowledge management system development tool

 

Specialty performance support tool that address one or more elements of the PCD lifecyle:

Process/workflow modeling and/or simulation

User experience, interface development/generation, usability

Content, information, knowledge engineering

Real-time assembly of performance objects

Other specialty tools that address PCD lifecycle element(s) (describe)

Other (describe): The tool assists content authors and course developers in generating metadata for learning objects and information assets. The metadata enable wider reuse of these content objects across departments and organizations. 

 This entry is (check one):    In production (being used today in a live work setting)          In a formative stage (prototype, proof-of-concept, introduced a sample of its intended users)    

 

 

SHOW SCREEN SHOTS, ANIMATION (e.g., FLASH movie, animated gif or dynamic HTML) OR PROVIDE LINKS TO SUCH SAMPLES THAT SUPPORT YOUR RESPONSES.   You must provide the judges with as vivid a representation of the tool user's experience.  If you are providing links to a restricted site, please provide six (6) user names and passwords.

In addition, you may be asked to demonstrate your tool to the judges at a mutually convenient time via a Webex or similar session (scheduled by EPSScentral).

Product / Component Name & Brief Description:

The MAGIC (Metadata Automated Generation for Instructional Content) system assists content authors and course developers in generating metadata for learning objects and information assets to enable wider reuse of these objects across departments and organizations. Using the MAGIC system through a Web-based user interface, content authors review and edit automatically-generated metadata sufficient to register and describe their assets for use and discovery in current and future distributed learning applications. The metadata comply with the SCORM (Sharable Content Object Reference Model) standard. Course developers can use the system to assist in the conversion of existing courses to SCORM format or in developing new SCORM courses. The MAGIC system includes software tools (text analysis and video analysis technologies0 to analyze and extract descriptive metadata from instructional videos, training documents, and other information assets. The tools generate some of the most critical SCORM metadata completely automatically.  Benefits of MAGIC include easier reuse and repurposing, improved interoperability, and more timely registration of content for use by course developers.

 

Vendor:

Same as above

 

Operating Environment:

MAGIC is a hosted service available to users through Internet Explorer as GUI,  or through Web Services as programming interfaces.

 

Product/Component Detailed Description:

With the increasing amount of information that can be accessed online, users need help distilling vast amounts of accessed information into smaller sets that are relevant to their need of the moment. A variety of tools (search engines, alert systems, learning management systems) help search and filter information. These tools rely on metadata that tag and characterize content. But creating metadata and associating them with content is often still a manual task, very laborious and costly, and prone to inconsistencies.

 

MAGIC is a web-based tool designed to assist the process of metadata tagging. It automatically creates metadata -- a title, a description (or summary), and a set of keywords – and associates them with textual documents or videos.  It also automatically associates each piece of content with the appropriate topic(s) from a taxonomy of topics. For content that is long (which is usually the case with videos), MAGIC can be used to segment the whole into smaller thematic sections and tag each one.

 

MAGIC retrieves textual content published on the web. It can process HTML, Adobe PDF, and Microsoft Word documents. With the push of a button, MAGIC processes the document and generates metadata, using advanced text-processing capabilities and a general-purpose taxonomy of thousands of topics. The user can edit the metadata interactively. When done, the metadata are stored in a repository which can be sorted and searched.  MAGIC also packages the metadata with (or without) the original document content for distribution. It produces SCORM (Sharable Content Object Reference Model) packages for learning content, or HTML-based packages, to be viewed from a standard browser.

 

Figure 1 - Metadata Generation for a web page about Anthrax

 

Figure 2 - The Topic Taxonomy

 

Videos are also accessed via the web, but their processing takes place offline. Video segmentation is achieved with the use of advanced video analysis techniques that leverage audio and video characteristics, and combine them with text analysis of the video’s closed caption transcript. e-mail is sent to the user when segmentation and metadata generation are complete. Users can view the video segments and select segments for extraction.

 

 

Figure 3 - Browsing the results of video segmentation

 

Batch-mode processing is also available for handling large collections of materials.

 

 

Product use:

MAGIC was developed with funding from the US Department of Homeland Security (DHS), to help the agency's content authors, course developers and contributors with SCORM adoption and to enable more pervasive reuse of high value content of multi-agency archives.  The project was delivered to DHS in May 2006.

 

MAGIC is currently being evaluated by the University of North Carolina at Chapel Hill as a potential tool for use in their Open Video project http://www.open-video.org/project_info.php. “The purpose of the Open Video Project is to collect and make available a repository of digitized video content for the digital video, multimedia retrieval, digital library, and other research communities. Researchers can use the video to study a wide range of problems, such as tests of algorithms for automatic segmentation, summarization, and creation of surrogates that describe video content; the development of face recognition algorithms; or creating and evaluating interfaces that display result sets from multimedia queries.”

 

Deliverables provided and samples:

MAGIC automatically generates metadata fields – title, description, keywords, and taxonomy category – for textual documents, located on the Web (Fig 1 and 2).  In addition, the tool automatically breaks large video into segments – larger ones, related to single topic and smaller ones by their audio-video characteristics (Fig 3).  All the generated metadata are stored in a database, available for sorting and searching (Fig 4).

Figure 4 – Viewing the Metadata Repository

 

 

What difference does this tool make?

MAGIC is designed to assist in the preparation of learning material. With increased market pressures and accelerated business changes, instructional designers need to create learning rapidly, through reassembly from existing materials. They can use MAGIC to tag content and create standardized learning objects that they can later find and aggregate across repositories and organizations.

 

MAGIC can also assist in converting large amounts of existing content into small self-contained, context-free, learning objects, by segmenting videos into units which are cohesive and topic-specific.

 

By automating the process, MAGIC helps achieve substantial cost reductions.  For example, associating the right topic with a web page can take a human annotator up to ten minutes per page. The system processes 200-400 pages an hour, with similar or often superior accuracy.

 

Although designed specifically for learning, MAGIC can also be used to tag non-learning content.

 

What else makes this tool extraordinary?

The MAGIC system takes advantage of IBM Research content analysis tools, which include tools for extracting information from text, classifying text into categories, and segmenting videos.  The results of processing sample DHS and IBM material with these tools indicate that the automatically generated metadata are more consistent and complete than hand-coded metadata.

 

An informal evaluation was performed to determine a subjective level of satisfaction with the information extracted from text and the level of accuracy of the audio-video segmentation.

·        For the text analysis tools, an informal comparison between human annotated and automatically generated metadata indicated satisfactory results for most of the cases.  We believe that the cost of manually correcting the machine-generated metadata is significantly less than inputting metadata by hand. Furthermore, we believe that the automatic generation of metadata provides a level of completeness and consistency that is usually lacking from manual input.

·        For the text categorization tool, which provides automated classification with no human intervention, the results are also good.  The automatic classification removes the need for humans to browse and consistently identify categories in a taxonomy of several thousand nodes, which is extremely time consuming and can result in inaccuracies.  Human intervention is restricted to cases where the existing categories are not applicable.

·        For the video segmentation, the greatest value comes from creating annotations that accompany automatically generated audio-video segments. The system generates text notations (keywords) from the closed captions that accompany instructional videos and associates them with the segments.  Users can then locate pedagogically useful segments and manually extract them.