Implementation of Quality Information Systems for E-Learning Applications
Abstract:
Quality information systems (QIS) are providing all the quality-relevant information during the whole life cycle of a product to all the people involved. With regard to e-learning applications this means that a QIS should provide quality-relevant information to the authors of e-learning materials as well as to the instructors and the learners using them. In this work it is described how such a QIS can be implemented in the context of e-learning applications. In this connection a prototypic system design and strategies for the integration of a QIS into the information system infrastructure of e-learning providers are presented.
1 Introduction
Many concepts of quality management are not only applicable during the production and usage of material products, but also in the context of immaterial ones - like for example e-learning applications. Nevertheless quality management concepts are hardly used in this regard today. As a basis for quality management so-called quality information systems (QIS) can be used. They are providing all the quality-relevant information during the whole life cycle of a product to all the people involved. In the context of e-learning this means that a quality information system should provide all the quality-relevant information to the authors of e-learning materials, to the instructors and tutors, and also to the learners.
In this work after a look on the State of the Art some fundamental terms and ideas of quality information systems will be introduced, and it will be pointed out how they can be used in the special case of e-learning. In particular it will be described which information could be quality-relevant and how it should be prepared for the different users of the system. Then the focus of the work will be on the implementation of such a quality information system. After a detailed requirements analysis a system design will be presented. Moreover it will be shown how an information system like this could be integrated into the existing information system infrastructure of e-learning providers. Here the importance of standards for all the types of data involved, like for example IEEE LOM for the learning objects, will be pointed out.
The results presented in this work have been obtained in the context of the research project "ViKar - Virtual University Group Karlsruhe" [1]. This project is being financed by the state of Baden-Württemberg as a part of the research program "Virtual University of Baden-Württemberg" [2].
2 State of the Art
In the last few years a lot of endeavors have been made in several fields of e-learning research and development. In the beginning the authoring of e-learning materials was in the spotlight. As a basis for all further e-learning activities contents have been developed in a multitude of different technical formats. Since special programs for the authoring of e-learning materials have been missing, most of the authors used well-known software like word processing tools or HTML editors. Since these programs don't support any special requirements for the authoring of e-learning content at all, it's also hardly possible to support quality management concepts using these tools.
At the same time, while the first e-learning materials have been created, many so-called e-learning platforms have been developed to provide e-learning materials to the learners and for the handling of communication and organization. Comparisons of such e-learning platforms (like for example in [3]) show that there are wide differences in their functionalities, but nearly all of them have one in common: they don't force or support a continuous quality management.
Furthermore till now there is a big gap between the authoring tools used for the development of e-learning content and the e-learning platforms. Since most of the platforms don't support the authoring satisfactorily the authors have to use tools widely independent from the platforms, which obviously complicates quality management, too. A possible solution for this problem might be the usage of standards for the modeling of e-learning content such as LOM [4], but they are not used widely enough so far. In this situation the first step to a continuous quality management might be the implementation of a quality information system.
3 Quality Information Systems
3.1 Terms and Definitions
For the design and implementation of a quality information system it is necessary to understand what "quality information" really is. For this purpose some fundamental terms and definitions, which will be used in the succeeding sections, will be explained here.
The ISO 9000:2000 standard defines quality as the "degree to which a set of inherent distinguishing features fulfils needs or expectations that are stated, generally implied or obligatory" [5]. In accordance with this definition a quality information comes from the comparison of the actual and the nominal condition of a quality criterion [6]. Then a quality information system obviously is an instrument for handling such quality information. In many cases quality information is not available directly, but it has to be derived from other raw data. Therefore in the following data is regarded as quality-relevant, if it can be used to obtain quality information.
All these definitions can be transferred directly to the special case of e-learning. The most important expectation for an e-learning application and therefore the most interesting quality criterion should be that a certain subject can be learnt using the application. But different users of a quality information systems might have different other expectations: generally the authors of e-learning material for example want to improve the quality of their product and so they should be interested in many kinds of information about the usage of their material; and on the other hand for the management of a company using a certain e-learning application also cost savings may be of interest. For these reasons all the data that is quality-relevant for an e-learning application should be collected completely in a special quality information system.
3.2 Quality-Relevant Data in E-Learning Applications
To obtain a complete compilation of quality-relevant data all the different data types occurring in the whole life cycle of the application have to be considered. A detailed analysis of many different data types can be found in [7]. Figure 1 shows only a simplified overview of the principal constituents every quality information systems for e-learning applications should contain.

Figure 1: Quality-relevant data
The overview is structured according the different phases in the life cycle of the application: * In the analysis phase only general specifications of the planned e-learning application are generated. For example the learning objectives of the application and its target group should be defined here. * In the design phase the e-learning application is modeled in detail, in particular the course content and its structure has to be defined. It's very reasonable to specify all this data in a formalized way. Then it is obviously much more adapted for an automatic data processing than the implemented e-learning content itself. In later phases this data can be used to survey, whether the implemented e-learning application does suit its specifications or if it's used in a way according to them. * In the implementation phase the e-learning application has to be realized pursuant to the modeling in the preceding phases. The corresponding specifications should be transformed into the meta information of multimedia elements, modules and courses. Of course it makes sense to use a standard for e-learning meta information here. If the learning modules for example are written using XML only some special tags according to the standard have to be inserted.
It should be mentioned again that also the learning content itself is quality-relevant of course, but it's not as accessible for an automatic processing as its formalized meta information. Therefore it's not reasonable to store the whole learning content in a quality information system. * In the phase of integration and testing and in the phase of operation and maintenance essentially the same types of quality-relevant data can occur. Examples for these types of data are user-tracking data, results coming from direct questionings of the users and personal user data including the results of learner assessments. Obviously the learners are involved in the generation of all this data and therefore their privacy has to be taken into account. For this reason all the personal data in a quality information system should be anonymized or at least pseudonymized, but for a later only statistical analysis usually that's no problem. Some techniques respecting at least the German data protection acts have been described for example in [8].
All these types of quality-relevant data might be interesting separately, but they'll develop their significance only if they are associated with each other. If for example many users coming from the same module search for a certain word in the glossary, this might indicate that the learning module is ambiguous at a certain point. If this is not the case, some information about the learners might indicate that some knowledge the learners don't have has been presupposed in that learning module. As another example also questionings of the users will become more significant if they are related to certain parts of the content. If these relations are specified in a formalized way, an automatic analysis will be much simplified. Therefore in any implementation of a quality information system the different types of quality-relevant data and, what is more, their relationships have to be regarded.
4 Implementation Strategies
Generally a quality information system cannot be implemented strictly separated from the other information systems of an e-learning provider, but it's very important to consider how close the connection between these information systems should be. One extreme would be to consider the quality information system only as a view on the whole information system architecture, the other one would be to implement the quality information system widely independent from the other systems and to use only some small interfaces. Of course the decision which implementation strategy to use depends on the situation of the existing information systems, but there are some reasons for using only a loose coupling between these different systems.
This can be seen in the following comparison of two different approaches for the integration of a quality information system into an e-learning application. In Figure 2 an integration with a close coupling is shown.

Figure 2: Integration of QIS and e-learning application (I)
In this example the architecture is shown in a UML class diagram. It should be assumed that the classes Application and QIS are the "main classes" of the e-learning application and the quality information system respectively. The other classes should be assumed as interfaces to different types of quality-relevant data the quality information system is based on: * The Content (in general its meta information only) is used by the class QIS, * the personal data of the Learner is included in the analysis, * all the Tracks the learners left while using the e-learning application are available, * and the results of Polls are also included in the quality information system.
These relationships are shown in the UML diagram by associations between the class QIS on the one hand and the four classes standing for the different types of quality-relevant data mentioned above on the other hand. As shown in the figure also the e-learning Application needs to have associations to all these classes providing access to the quality-relevant data. Therefore this approach has some disadvantages: with regard to data protection issues for example it is critical that the personal data of learners can be accessed from the quality information system. The class QIS is also related to classes representing types of data that are not of interest to a quality evaluation (e. g. parts of the personal data or parts of the content, which are not accessible for an automatic evaluation). Furthermore the implementation of the e-learning application and the quality information system in a distributed way should be taken into account: using this architecture with its very close coupling a distributed implementation is made difficult.
For all these reasons an architecture with a looser coupling between the e-learning application and the quality information system, like the one in Figure 3, should be considered.

Figure 3: Integration of QIS and e-learning application (II)
Using an approach like this the e-learning application and the quality information system are much more separated from each other. Here the class QIS doesn't have access to all the personal data of the learners, since Anonymous represents an interface to some anonymized data only. Other types of quality-relevant data in this model, namely the results of polls and the tracking information, are only related to the anonymized learner profiles and not to the other personal data. In consideration of data protection that's a big advantage in comparison to the other approach. Since the associations between Application on the one hand and Poll and Track on the other hand can be realized using some small interfaces, the quality information system is much more independent from the e-learning application. Nevertheless there are some types of quality-relevant data that are also extremely important for the e-learning application, for example the Meta Information of the learning content. Here it's obviously not reasonable to introduce an artificial separation, since both parts need a close connection to the meta information, even though they need it for different purposes. But this quite close coupling here shouldn't be problematic, since in contrast to the other types of quality-relevant data the standardization of this data type has advanced significantly. If a standard for e-learning content and its meta information is used, like for example LOM [4], the data exchange between the e-learning application and the quality information system described above is quite uncritical. Therefore both systems are sufficiently independent from each other and may also be implemented in a distributed way.
5 Conclusion and Outlook
As described in the preceding sections a quality information system for an e-learning application should be designed very carefully. There are many different types of quality data occurring in the context of e-learning. To obtain a quality information system, which is usable during all phases of an e-learning application's life cycle, all the relevant data should be stored in one information system. In the long run there are many advantages, if both systems - the quality information system and the e-learning application - are working quite independently from each other. The implementation, the operation, and the maintenance of the systems will be simplified very much, if they have only some small interfaces realizing a standardized communication. This strategy can be used, if both systems are being designed from scratch, but it should also be considered, when it is planned to implement a quality information system in addition to an existing e-learning application. The increasing importance of e-learning standards is another factor supporting this approach.
The implementation of a quality information system includes further interesting problems of course, e. g. in the context of the analysis of quality data. There are some existing approaches, which could be transferred from the field of the evaluation of other web content, but nevertheless also special concepts for the quality management in the emerging area of e-learning will be developed to achieve a higher quality of e-learning.
References:
[1] ViKar - Virtual University Group Karlsruhe, retrieved August 7, 2002, from http://www.vikar.de
[2] Virtual University of Baden-Württemberg, retrieved August 7, 2002, from http://www.virtuelle-hochschule.de
[3] Landon, B.: Online educational delivery applications: a web tool for comparative analysis, retrieved July 25, 2002, from http://www.c2t2.ca/landonline
[4] IEEE Learning Technology Standards Committee: Standard for Learning Object Metadata, retrieved August 1, 2002, from http://ltsc.ieee.org/wg12
[5] International Organization for Standardization: ISO 9000:2000. Quality management systems - Fundamentals and vocabulary. Geneva: ISO, 2000.
[6] Scheer, A. W.; Trumpold, H.: Qualitätsinformationssysteme. Heidelberg: Springer, 1996.
[7] Sommer, D; Stucky, W.: Quality-Relevant Data in the Context of E-Learning Applications. Proceedings of the Second Research Workshop of the European Distance Education Network. E. Wagner; A. Szücs (ed.). Budapest: EDEN, 2002.
[8] Schaar, P.; Möller, F.: Der Hamburgische Datenschutzbeauftragte: Orientierungshilfe Tele- und Mediendienste, retrieved January 18, 2002, from http://www.hamburg.de/fhh/behoerden/datenschutzbeauftragter/veroeffentlichungen.htm