Quadrant of Difficulty-Usefulness (QoDU) as New Method in Preparing for Improvement of E-learning in Health College

Introduction

In the global era as it is today, information and communication technology (ICT) is commonly utilized by health institutions such as hospitals, community health centers, health clinics, health services, as well as educational institutions in health (health colleges).¹ Health colleges should not be left behind by the rapid development of the world of education which is the result of the development of ICT, in order to achieve progress far ahead, including the implementation of e-learning in health.¹ This is an urgent need because the use of e-learning is one of the strategic efforts to achieve competitive advantage. E-learning can increase the flexibility of the learning process, not too affected by time, place, busyness, physical presence, and stressful teaching-learning.^1–3 It has also been proven in previous research that the implementation of e-learning can increase student motivation.⁴

Currently, many health colleges in Indonesia are starting to implement e-learning, so efforts to improve quality are still very much needed. Most of the lecturers who act as managers of e-learning in their courses still face many technical obstacles, so the implementation of e-learning has not been optimal. Thus, students must use e-learning which is not ready to be used for the learning process.¹

One of the e-learning programs that is very often used in learning activities in Indonesia is Moodle. This program consists of several main elements, namely: learning design, handout, book, links to resources, discussion forum, chatting, assignment, feedback, quiz, and survey. Every lecturer should prepare and manage these elements well so that e-learning can be implemented ideally.⁵

Based on the consideration that most of the e-learning implementations in health colleges in Indonesia are still at an early stage, efforts to improve the quality of e-learning are urgently needed, especially for the performance of the 10 elements mentioned above. Because the number of elements that have to be improved in performance is not small, it is necessary to choose which elements are prioritized for performance improvement first.¹

In 2018, Bakri used the Urgency, Seriousness, Growth (USG) method to select prioritized elements or issues in a study at a public health center.⁶ The three attributes above are more general in scope, not specific to the elements of an information technology-based system. Thus, we need a method of selecting priority elements specifically for information technology-based systems. For this reason, it is necessary to explore the attributes derived from theories about the implementation of information technology-based systems.

Nugroho et al introduced the method of selecting elements based on priority order, using two attributes, difficulty and usefulness, known as difficulty-usefulness pyramid (DUP). These two attributes are adopted from theories relating to the implementation of information technology-based systems, namely the Technology Acceptance Model (TAM), which has been developed in three generations.^7–9 Then the difficulty and usefulness were assessed for each element. A combination of difficulty and usefulness values was used to arrange the bars for each element. Next, the bars were arranged sequentially starting from the widest in the lowest position, and ending with the bar that was the narrowest in the top position. The element that becomes the first priority was the bar which was at the lowest position, which indicated that the element had the highest level of difficulty and the level of usefulness.^1,10

The availability of various methods in order to select elements based on priority order is important. Therefore, with the same elements and attributes, it is deemed important to introduce ways to select prioritized elements, one of which is to group elements into four quadrants. So, a simple innovative method is introduced to select elements of e-learning in health that are prioritized for improvement and improvement, by grouping elements into four quadrants, based on difficulty and usefulness.

Materials and Methods

Time, Location and Type of Research

This research was conducted in the end of 2018 to 2019 at the Environmental Health School of Magetan, Indonesia, one of the institutions that produces health professionals, under the auspices of the Ministry of Health of the Republic of Indonesia. This research was a quantitative descriptive study, which was intended to introduce a simple innovative method for selecting elements of e-learning that will be improved, in the form of priority order, based on the difficulty and usefulness of these elements.

Population and Sample

The study population was all students of the Environmental Health School of Magetan, Indonesia in 2018, with a population size of 240 students. The specified error margin was 0.05. Next, the sample size was determined based on the Slovin formula, namely:

n = N/(1 + N.e²) = 240/(1 + 240 x 0.05²) = 240/1.6 = 150

Note: n = sample size; N = population size; e = margin error

The results of the above calculation were for the minimum sample size, and in this study, the sample size was enlarged to 200 students. The sample was selected by simple random sampling technique using a random number table.

Research Stages

This research was conducted in the following stages:

1. Determination of e-learning elements that will be sorted by priority

The determination of elements of e-learning in health college that would be prioritized referred to previous research, namely difficulty-usefulness pyramid (DUP)¹⁰ and difficulty-usefulness pyramid with weighting (DUP-We),¹ which consisted of 10 elements, namely: 1) learning design; 2) handout; 3) book; 4) link to resources; 5) discussion forum; 6) chatting; 7) assignment; 8) feedback; 9) quiz and; 10) survey. In previous studies, the determination of these 10 elements had been done carefully with the difference in time that was not too long, so the consideration was still relevant for use in this study. In order to ensure the accuracy of the selection of e-learning elements, the 10 elements based on previous research were further consulted with relevant experts from the Humanistic Network for Science and Technology (HNST) and the Alliance of Health Activists (AloHA).

2. Determination of the attributes used as a basis for determining priorities

The determination of these attributes also referred to the DUP and DUP-We methods, so the attributes used as the basis for determining priorities were difficulty and usefulness.^1,10 As explained in the two previous studies, the difficulty and usefulness were explored from the Technology Acceptance Model (TAM).^7–9 TAM was chosen because it was the most widely used information technology acceptance model.^11–15 As an information technology acceptance model, TAM was built by two important determinants, namely, “perceived ease of use” and “perceived usefulness”. As with the selection of elements, the two attributes based on previous research were also further consulted with relevant experts from the Humanistic Network for Science and Technology (HNST) and the Alliance of Health Activists (AloHA).

3. Measurement of difficulty and usefulness for each element

Referring to DUP-We, the term “ease of use” had been changed to “difficulty”, while “usefulness” was still used without change.¹ In this case, the two attributes were measured by completing a semantic differential questionnaire with a range of 0 to 10. For difficulty, the highest difficulty level was given a minimum score [0], while for usefulness, the highest usefulness level was given a maximum score [10] (Figure 1). The development of the questionnaire and scoring method was also further consulted with relevant experts from the Humanistic Network for Science and Technology (HNST) and the Alliance of Health Activists (AloHA).

4. The process of data collection

Figure 1 Questionnaires as instruments for selecting e-learning elements based on priorities.

The data collection process was carried out simultaneously at the same time and place, in order to facilitate the process of filling out the questionnaire and to anticipate difficulties in understanding and technical filling of the questionnaire by respondents. Thus, the possibilities for data collection bias could be minimized. Chronologically, the data collection steps in the field are as follows:

1. The research team obtained permission from the head of the Environmental Health School of Magetan to carry out all proposed research activities.
2. The research team trained enumerators who would act as data collectors. They were the lecturers at the college.
3. The enumerators provided explanations to the respondents about the objectives and research process, followed by signing the informed consent.
4. Respondents filled out a questionnaire guided directly by the enumerators, under the supervision of the research team.
5. The enumerators ensured that all respondents had filled out the questionnaire correctly and completely, before it was returned to the enumerators.

5. The method of selecting elements based on priority order

After the data had been collected and had been convinced that it was correct and complete (editing process), a descriptive analysis of the data was then performed, namely:

1. Calculated the mean score of difficulty for each element
2. Calculated the mean score of difficulty for all elements
3. Calculated the mean score of usefulness for each element
4. Calculated the mean score of usefulness for all elements

Furthermore, the mean score of difficulty and the mean score of usefulness in total was used as the basis for making the Y-axis and X-axis. Meanwhile, for each element, the coordinates were arranged based on the mean score of difficulty and the mean score of usefulness, so that the resulting quadrant image, in which there were elements with the quadrant location different ones. The first priority was the elements in Quadrant I, followed by the elements in Quadrant IV, Quadrant II, and the last was Quadrant III.

This method was also further consulted with relevant experts from the Humanistic Network for Science and Technology (HNST) and the Alliance of Health Activists (AloHA).

Ethical Approval

As a study involving humans as research subjects, this research has received ethical approval from the Health Research Ethics Committee of the College of Health Science “Maluku Husada”, with Number RK.05/KEPK/STIK/X/2018, 23 October 2018. In order to minimize the risk of conflict of interest related to ethical approval of this study, the research team chose to obtain ethical approval from the ethics committee outside the affiliation of one of the researchers. The ethics committee mentioned above was chosen because it has ethical examiners who already have an ethics examiner certificate issued by the Ministry of Health of the Republic of Indonesia. All matters relating to ethical issues such as informed consent, confidentiality, benefits, and so on have been reviewed, and it is declared that there are no ethical problems.

Results

The results of descriptive data analysis were the mean score of difficulty for each element, the mean score of difficulty for all elements, the mean score of usefulness for each element and the mean score of usefulness for all elements, which are shown in Table 1.

Table 1 Mean Score of Each Element

Furthermore, the mean score of difficulty and the mean score of usefulness in total, namely, 6.57 and 7.70 were used as the basis for making the Y-axis and the X-axis. Meanwhile for each element, the coordinates were arranged based on the mean score of difficulty and the mean score of usefulness, for example (9.03; 7.10) for element A, namely learning design. Next, the 4 quadrants produced are shown in Figure 2.

Figure 2 Analysis of the position of each element of e-learning in health in the four quadrants based on difficulty and usefulness.

Based on Figure 2, the order of health e-learning elements based on priorities were:

1. The first priority (Quadrant I) were assignment (G), quiz (I) and discussion forum (E).
2. The second priority (Quadrant IV) were feedback (H) and survey (J).
3. The third priority (Quadrant II) were link to resources (D) and book (C).
4. The fourth priority (Quadrant III) were learning design (A), handout (B) and chatting (F).

Discussion

The results showed that the order of elements of e-learning in health was arranged according to priority based on difficulty and usefulness, with the following explanation.

First Priority (Quadrant I)

The elements that are in quadrant I have a high level of usefulness (in the top position) but have a high level of difficulty to apply (in the left position). It may also be said to be very useful elements, but in fact, it is still very difficult to realize. Therefore, these elements need to be immediately improved in quality, so that they can quickly meet the needs of users. If it is successful, then these elements can move to quadrant II.

In the presentation of the results of the analysis, the elements in quadrant I are assignment (G), quiz (I) and discussion forum (E). With this position, it is clear that the three elements above are very useful for users, but they are disappointed because they are still difficult to realize, compared to other elements. Thus, these elements must be the first priority in efforts to improve the quality of e-learning in the health sector. If the improvement effort is successful, then of course the users will be satisfied, because something they find useful is easily realized.

Second Priority (Quadrant IV)

The elements that are in quadrant IV have a value of usefulness which is considered lower for the user (in the bottom position) and difficult to implement (in the left position). The elements in this quadrant need to be prioritized to improve their performance, but attention to this quadrant is still below quadrant I which is considered more useful by the user.

In the presentation of the results of the analysis, the elements that are in quadrant IV are feedback (H) and survey (J). These elements need to be managed very seriously because user disappointment usually starts from the elements that are in this quadrant.

Third Priority (Quadrant II)

The elements that are in quadrant II have a high level of usefulness according to the user (in the top position) and are easily applied or realized (in the right position). In other words, these elements are seen as very useful for the user and are also easily implemented or realized. These elements are the strength of the information system. Therefore, these elements need to be maintained (they can always be implemented or easily realized), and can gradually be improved.

In the presentation of the results of the analysis, the elements in quadrant II are the link to resources (D) and book (C). With that position, it is clear that the elements above are important from the point of view of the user, and also easy to implement when compared with other elements. This group of elements has the highest ability to satisfy users, so performance must be maintained. The quality of information systems is highly dependent on the four elements above; therefore, in addition to being maintained, these elements must also be improved gradually.

Fourth Priority (Quadrant III)

The elements in quadrant III have a lower level of usefulness when compared to other elements (in the bottom position), but the reality is easy to implement (in the right position). In other words, these elements are not very expected by the user, but instead, have good performance. Therefore, the performance of these elements is considered to have exceeded the standard.

At the presentation of the analysis results, the elements in quadrant III are learning design (A), handout (B) and chatting (F). These elements are seen as elements that are not too important for the user to be provided by the system manager, but in fact, the performance of these elements is relatively high. In a simple statement, we can say: “Why should we prioritize something that is currently considered less useful by users? Moreover, its performance is also good. Is not it better if we focus more on other elements that are considered more useful for the user? Moreover, elements that are still difficult to implement.” If we pay close attention to the questions above, then the elements in quadrant III should be the last order, in an effort to improve the performance of these elements, or even these elements need not be improved at this time.

Innovation

This research has produced a new method used for determining the initial steps for efforts to improve the quality of the management of e-learning in health college. The determination of e-learning elements is based on 10 elements used in popular e-learning programs, namely, learning design, handout, book, links to resources, discussion forum, chatting, assignment, feedback, quiz and survey, which are also used in DUP and DUP-We.^1,10

The attributes chosen as the basis for determining priorities are difficulty and usefulness. This refers to the results of previous studies, namely DUP and DUP-We.^1,10 “Difficulty” is a negative attribute, so the more difficult an element is implemented, the lower the score obtained. Meanwhile, “usefulness” is a positive attribute, so the more useful an element is, the higher the score obtained. If presented in quadrant form, then the difficulty is placed on the horizontal axis, so the more difficult an element is to be implemented, the position is on the left and the easier an element is to be implemented, the position is on the right. Meanwhile, usefulness is placed on the vertical axis, so that the more useful an element is to the user, then its position is at the top, while the less useful an element is to the user, then its position is at the bottom.

The use of this quadrant is intended so that the results of the analysis can be understood quickly and easily. In this case, quadrants can be created manually or using statistical software such as SPSS. With a quadrant like this, drawing conclusions can be done easily and quickly because the results of data analysis are presented visually in a quadrant based on difficulty and usefulness. In this case, the elements in quadrant I are the first priority, then sequentially followed by the elements in quadrant IV, quadrant II, and the last sequence is quadrant III. Recommendations can also be conveyed easily because it simply refers to the order of priorities that have been obtained in the conclusions of the study. Based on these conclusions and recommendations, the managers of the health information system can immediately draw up a plan to improve the elements according to the characteristics of the elements.

The entire process above is an effort focused on improving the quality of ICT-based information systems. The focus of this new simple innovative method is the visualization of the results of data analysis in the form of “quadrants” so that it is very easy to understand in a quick time. Furthermore, this quadrant is introduced with the name “Quadrant of Difficulty-Usefulness” which is called by the short name “QoDU”

It should be noted that in this study, “e-learning in health” is only an example of an object that has become the target of quality improvement efforts, through enhancing the building elements. Therefore, there are wide-open opportunities to implement the entire process above for other objects, such as health information systems, public health center information systems, hospital information systems, telemedicine, e-Health, and others. It should be noted that researchers must first make the selection of elements in accordance with the system that will be the object of study. In addition to referring to previous established literature, the elements to be investigated can also be explored by the users of the related system. Many ways to explore the elements of users, for example, through interviews, brainstorming or focus group discussions.¹

Limitations

As a method that has just been pioneered, QoDU still has a number of limitations, including the following:

1. In this first study, the number of elements is relatively small (10 elements), so that the usefulness value is not too big. However, if the number of elements is relatively large, this is very beneficial for managers, because the more elements, the more difficult it will be to choose which elements to prioritize for improvement.
2. In this study, difficulty and usefulness are considered to have the same weight, even though it is possible that the two attributes are weighted differently by respondents. It is more ideal if there is an additional item in the questionnaire about how much the importance of difficulty and usefulness is weighted. So that in data analysis, for each element, each score of difficulty and usefulness must be multiplied by their respective weights.

Conclusion

Based on the results of the study, conclusions can be drawn including:

1. The order of elements of e-learning at Environmental Health School of Magetan based on priority were assignment, quiz and discussion forum as the first priority; feedback and survey as second priority; link to resources and book as third priority; and learning design, handout and chatting as the last priority.
2. Through this research, an innovative and simple method is produced to select elements that are prioritized in efforts to improve e-learning in health.

Furthermore, several things are recommended, namely:

1. It is expected that the managers of e-learning and other systems use this method in order to initiate efforts to improve the quality of the system.
2. It is expected that the managers of information systems in general try to implement this method in order to initiate efforts to improve and improve the performance of various types of information systems, even other systems outside the information system, by first making some adjustments, especially the selection of system elements.