Construction and Validation of the Postgraduate Research Innovation Ability Scale (PRIAS): A Three-Dimensional Structural Model Based on Componential Theory of Creativity

Introduction

Creativity is considered an essential human resource in the 21st century and a powerful means of improving individuals’ quality of life.¹ Scientific research and innovation are closely connected, with scientific research being a tool for collecting and studying phenomena to enhance our understanding and approach to the nature. In the current era of rapid technological development, research is one of the most important logical resources for the advancement of human society and improvement of living conditions.² Postgraduates are the middle force between high-tech and scientific research fields. Scientific research plays an important role in postgraduates’ lives in universities. One of the core goals of postgraduate education in China is to cultivate students’ research innovation ability, teach students how to conduct scientific research, and arouse students’ motivation for academic research innovation.³ The quality of postgraduate training is the ultimate indicator for measuring a country’s educational level, and its core is the development of innovation ability.⁴ As the main foundation of scientific and technological personnel training, colleges and universities play a key role in the cultivation of research innovation ability among postgraduates. It is of great necessity and significance to conduct research to understand the research innovation ability of postgraduates.

Postgraduates’ research innovation ability is affected by many internal and external factors.⁵ Many researchers have studied the relevant external factors, such as supervisory styles, supervisor-postgraduate relationships, institution creative atmosphere, institution teaching methods, and other aspects. Chen⁶ found that project-based learning can motivate students’ innovative problem-solving abilities. Based on a questionnaire survey, Gu⁷ suggested that a directive supervisory style had a positive influence on postgraduates’ creativity. Peers, advisers, and experts are social resources that can independently and interactively influence postgraduates’ creativity.⁸ Cheng⁹ found that problem-oriented classroom teaching methods are more conducive to students’ creativity than traditional teaching methods are. Compared to research on the external factors of postgraduate research innovation ability, research on the relevant internal factors is scarce¹⁰ Factors such as intrinsic motivation, innovative self-efficacy, and personality are strongly associated with research innovation ability.^11,12

Other researchers have focused on the postgraduates’ innovation ability and tried to find ways to improve postgraduates’ innovation ability.^13–15 However, only a few studies discuss the measurement and structure of postgraduate research innovation ability. Yang¹⁶ used the Delphi survey to obtain three objective indicators to describe postgraduate research innovation ability: journal papers, participation in scientific research projects, and academic dissertations. Dong¹⁷ considered four elements of postgraduate innovation ability: ability to construct knowledge, ability to improve transformation, ability to discover problems, and problem-solving ability. Wang¹⁸ found that 10 personality traits (eg, pioneering spirit and strong self-confidence), 14 teacher-student relationship factors (eg, mentor morality and democracy), and 12 environmental factors (eg, teaching students according to their aptitude and research funding) could enhance postgraduates’ innovation ability.

Although there is abundant research on the composition of postgraduate innovation ability, we hope to understand graduate research innovation ability based on a mature theory. Componential theory of creativity is one of the most influential theories of creativity generation at present. In 1983, Amabile¹⁹ proposed the componential theory of creativity and held the view that the creation of creative products was the result of the interaction between three basic factors: domain-relevant skill, creativity-relevant skill, and intrinsic motivation. Domain-relevant skill refers to professional knowledge, technical skills, and talent in a particular field acquired by individuals through repeated practice.²⁰ Creativity-relevant processes refer to interdisciplinary skills, including individuals’ ability to use heuristic thinking to explore new cognitive pathways and ways of working, breaking the rules when attempting to understand and solve complex problems, and combining expertise, technical skills, and domain-related talent in novel ways. Based on componential theory of creativity, intrinsic motivation refers to individual actions that bring a sense of achievement, meaning, and value to the individual based on the action itself, rather than based on the influence of external factors, including individuals’ attitude toward the work (ie, the individuals’ cognitive evaluation of the work, their own preferences, and interests) and the individuals’ understanding of the reasons for completing the task (ie, obvious external constraints or not). Intrinsic motivation of postgraduate research is analogous to intrinsic motivation in the workplace. However, the teacher-student relationship in postgraduate stage is different from that in middle school, senior high school, and college. The mentor’s education of postgraduates is based on a certain organizational context. Mentors lead and manage postgraduates to achieve common goals through interactive activities. Therefore, it is more similar to the relationship between leaders and employees in the field of organizational management.Intrinsic motivation can not only significantly affect innovation ability, but also promote innovation ability jointly with exogenous motivation.²¹ Figure 1 provides a concise introduction of the componential model of creativity.

Figure 1 Componential Model of Creativity.

Several tools have been developed to measure creativity. These tests include divergent thinking tests, such as the Torrance Test of Creative Thinking (TTCT)²² and the Consensual Assessment Technique (CAT).²³ However, these assessments are time consuming. Many researchers have used questionnaires, such as the Kaufman Domains of Creativity Scale (K-DOCS)²⁴ and the Runco Ideational Behavior Scale (RIBS)²⁵ to measure some aspects of creativity. These are the simplest and easiest measures to implement. At present, there is a lack of a specific scale to assess postgraduate research innovation ability to help researchers understand the internal structure and current state of postgraduates’ research innovation ability.

Although undergraduates are involved in research activities, postgraduates are the main body of research work in China. Postgraduates do more innovative research under the guidance of their supervisors. Therefore, the research participants of this study are postgraduates.The purpose of this study was to construct and validate a scale to assess postgraduate research innovation ability based on the componential model of creativity. Based on the theoretical construction and research results of this study, research innovation ability is defined as an individual’s comprehensive ability to use relevant professional theories and skills to solve research problems, and generate innovative knowledge or achievements driven by personal innovation characteristics and motivation. Postgraduate research innovation ability is divided into three factors: domain-relevant skills, creativity-relevant processes, and intrinsic motivation.

The current research objectives were explored through two studies. Study 1 constructed a scale and determined its factor structure. Study 2 provided evidence for the construct and criterion validity of the scale. This study was conducted in compliance with the principles of the Declaration of Helsinki. The Biomedical Research Ethics Committee of Fujian Medical University approved the content and data collection procedures of this study. In the process of collecting data, authors went to several higher educational institution to ask for permission to conduct research in their places. They agreed to provide access to contact postgraduate tutors for interviews and distribute online questionnaires to postgraduates. Before the interview and the survey, the tutors and postgraduates were asked to read and fill out a consent form regarding their willingness to take part in the research.

Materials and Methods

Study 1: Development of Postgraduate Research Innovation Ability Scale

The purpose of Study 1 was to examine the psychological constructs related to postgraduate research innovation ability. Study 1 used NVivo software (version 11.0) to code analysis of interview texts and used SPSS 21.0 to analyze items of the preliminary test postgraduate research innovation ability scale by item, internal consistency, and exploratory factor analysis.

Participants and Procedure

Step 1: Identification of Domain

First, A qualitative research was conducted in study 1 to understand the structure and current status of postgraduate research innovation ability with a semi-structured interview hosted by a psychological professor and a postgraduate. Among seventeen postgraduate supervisors in this semi-structured interview, there were eight professors and nine associate professors from a university in southeastern China with an age range from 36 to 49 years old. To ensure the representativeness of the sample, the following screening criteria were adopted: (1) on-The-job tutors of postgraduates in university; and (2) serving as a tutor for postgraduates for more than three years. The participants voluntarily participated in this research. Two guided questions have been asked in semi-structured interview: (1) In the process of postgraduate education (ie, studying, doing experiments, reading literature, literature reports), how do you find the innovation of postgraduate in different situations? (2) What points do you think are emphasized more in the cultivation of postgraduate innovation? Based on the grounded theory paradigm of qualitative research, the interview content was edited into electronic text within 24 hours after the interview ended, which was analyzed and encoded by Nvivo 11.0 software. The interview content was edited into electronic text within 24 hours after the interview ended, which was analyzed and encoded by Nvivo 11.0 software. In the process of data analysis, we encoded the electronic text. Data analysis was carried out in the following steps. (1) Open coding: read interview materials repeatedly and extracted nodes; (2) Associative coding: according to the coding results of the first stage, different nodes are compared, and the interrelation of each node is established; (3) Core coding: analyzed the data and established core coding according to the class attributes. After the initial coding, the interview content under each node is extracted by using the classification function of the Nvivo 11.0. Then we modified and integrated the results of the initial coding on the basis of careful reading and thinking so as to finally determine the components of postgraduate research innovation ability. Two Interviewers integrated some inappropriate nodes and finally deleted the nodes with fewer than 50% sources in the process of revising and integrating.This Methods comes from Hill’s thesis. Hill et al²⁶ referred to the codes that all subjects chose as “common”, half or more chose as “representative”, and less than half as “incidental”. Therefore, in order to ensure the accuracy of the results. Three relational codes were extracted to generate an item pool. The internal structure of the scale consists of three elements: domain-relevant skills, creativity-relevant processes, and intrinsic motivation.Furthermore, an initial pool of items was developed by reviewing literature and consulting experts for their opinions. Initially, 13 items were developed, with a 5-point Likert scale with the options, 1 = strongly disagree, 2 = disagree, 3 = neutral/do not know, 4 = agree, and 5= strongly agree.

Step 2: Preliminary Test

In the second step, in order to analyze the psychometric properties of the 13-item postgraduate research innovation ability scale (hereafter, PRIAS), a preliminary test was conducted. The survey was conducted online on a Chinese questionnaire survey website (www.wenjuan.com). The questionnaires were completed voluntarily and anonymously. A total of 145 questionnaires were distributed and 125 valid questionnaires were recovered. Twenty students were excluded from the data analysis because they failed to pass a polygraph question. The final sample consisted of 125 postgraduates, including both men (n =47) and women (n = 78) from different universities in China. Items were subjected to item analysis, a corrected item-total correlation test (CITC), and exploratory factor analysis to remove unqualified items and screen the final items.

Study 2: Confirmation of Factor Structure and Validity

Study 2 aimed to confirm the factor structure of the PRIAS in an independent sample. It also aimed to provide evidence for the construct and criterion validity of the scale. Study 2 used SPSS 21.0 to analyze the items of the preliminary test PRIAS using item analysis, internal consistency analysis, and exploratory factor analysis (EFA). AMOS 24.0 was used in the confirmatory factor analysis (CFA). Participants’ data were randomly divided into two equal parts: one part for EFA and another for CFA. After testing, no significant differences were observed between the two parts for each index.

Participants

The study sample consisted of 330 postgraduates, including men (n = 112) and women (n = 218). Of the participants, 33.9% were male, 66.1% were female; 26.4% were in there first postgraduate year, 43.9% were in there second postgraduate year, and 29.7% were in there third postgraduate year. The sample consisted of participants selected from different universities in China. In addition, 56 postgraduates were selected as a test-retest reliability sample, and the same scales were distributed and recovered for data analysis 10 days after the initial assessment.

Instrument

Postgraduate Research Innovation Ability Scale (PRIAS)

A 13-item PRIAS was used in this study. It contains 13 items rated on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). Higher total scores reflect greater research innovation abilities. In Study 2, Cronbach’s alpha coefficient for the PRIAS was 0.89. Cronbach’s alpha coefficients for the three factors of the PRIAS were as follows: creativity-relevant processes (α = 0.87), domain-relevant skills (α = 0.80), and intrinsic motivation (α = 0.78).

Runco Ideational Behavior Scale (RIBS)

Runco Ideational Behavior Scale is prepared by Runco.²⁵ Zhang has translated and revised it into Chinese version. RIBS is a measurement of the generative expression of individual creativity ideas. It provides a new criterion for evaluating an individual’s creativity level. The scale does not contain intellectual or non-intellectual factors related to individual creativity, but only measures individual’s actual creativity concept generation performance, which clearly reflects individual’s creative tendency and skills in thinking. The 24 items were divided into three dimensions: fluency, originality, and flexibility.The Chinese version of the RIBS was used as an indicator of the convergent validity of the study.²⁷ The Chinese version scale contains 24 items on a 5-point Likert scale with 1 indicating “never” and 5 indicating “always”. Sample items include “I think about ideas more often than most people”, “Friends ask me to help them think of ideas and solutions”, and “Sometimes I get so interested in a new idea that I forget about other things that I should be doing”. In this study, Cronbach’s alpha coefficient for the RIBS was 0.90.

Results

Study 1

Item Analysis

Comparisons of the high and low groups and CITCs were used to check for discrimination and homogeneity. CITC is used to screen items (ie, the correlation coefficient between each item and the sum of all other items in the same variable dimension). If the CITC index was < 0.5, the item was deleted without any special reason. Thus, all items were retained because the difference between the high and low groups was significant. One item had a low CITC (r < 0.4). However, the subsequent EFA results show that this item had an indispensable contribution to the interpretation of the scale structure and variation rate. After in-depth group discussion, this item was retained. Study 2 was used to re-test the reliability and validity of the whole scale.

Exploratory Factor Analysis (EFA) and Internal Consistency

The factors were extracted using principal component analysis and rotated using varimax rotation. Items with initial eigenvalues above one was systematically assigned to a certain factor. Factor loads and common factor scores were then calculated. The criteria for removing the items were as follows: factor loading < 0.40, cross-loading < 0.20, less than three items in a factor dimension.²⁸ For the first round of EFA, the KMO was 0.80, and Bartlett’s test of sphericity was significant, χ2 (78) =570.89, p < 0.001. The results showed that a three-factor structure was formed. However, two items were deleted from the results. One item was removed because it did not retain its original component. The other item had a low extraction of common factor variance. After deleting this item, the explanation rate of the factor solution improved. Eleven items remained in the second EFA. For the second round of EFA, KMO was 0.78, and Bartlett’s test of sphericity was significant, χ2 (55) =463.09, p < 0.001. As shown in Table 1, the analysis indicated that each factor contained the expected initial items and the factor loadings ranged from 0.55 to 0.88. The PRIAS contained three factors. Eigenvalues were used to measure the quality of these factors. Multiple criteria were used to determine the number of factors. One of these is the eigenvalue, which is a good criterion for identifying a factor according to the Kaiser criterion. A factor is selected if its eigenvalue is greater than one and excluded if it is smaller than one. Factor 1 (creativity-relevant processes) contained four items, with an eigenvalue of 4.10, and explained 24.34% of the variance. Factor 2 (domain-relevant skills) contained four items, with an eigenvalue of 1.56, and explained 22.06% of the variance. Factor 3 (intrinsic motivation) contained three items, with an eigenvalue of 1.30, and explained 16.78% of the variance. Thus, a three-factor structure extraction was confirmed and the factor solution explained 61.29% of the variance. Internal consistency was measured to evaluate reliability using Cronbach’s alpha for the 11-item PRIAS (α = 0.84), and the sub-factors were creativity-relevant processes (α = 0.84), domain-relevant skills (α = 0.72), and intrinsic motivation (α = 0.67). Table 1 lists the rotated component matrix of the PRIAS.

Table 1 Rotated Component Matrix of Pre-PRIAS

Study 2

Item Analysis

Comparisons of the high and low groups and CITC were used to check for discrimination and homogeneity. The results showed significant differences between all the items in the high-score group and the low-score group, and all CITCs were above 0.5 (Table 2).

Table 2 Item Analysis

Test-Retest Reliability

The test-retest reliability of the PRIAS was 0.86 (p < 0.001). The scores of all three factors in the first test were positively associated with their corresponding scores in the second test: creativity-relevant processes (r = 0.77, p < 0.001), domain-relevant skills (r = 0.81, p < 0.001), and intrinsic motivation (r = 0.53, p < 0.001). The results indicate that the stability of the scale is acceptable.

Validity Analysis

Exploratory Factor Analysis (EFA)

A sample of 164 participants was selected for the EFA. The factors were extracted using principal component analysis and rotated using varimax rotation. Items with initial eigenvalues above one was systematically assigned to a certain factor. Factor loads and common factor scores were calculated. As shown in Table 3 and Figure 2, the results and scree plot indicated that a three-factor structure was formed. The KMO was 0.87, and Bartlett’s test of sphericity was significant (χ2 (55) = 892.97, p < 0.001). The analysis indicated that each factor contained the expected initial items and the factor loadings ranged from 0.51 to 0.85. Factor 1 contained four items, featured an eigenvalue of 5.26, and explained 27.30% of the variance. Factor 2 contained four items, featured an eigenvalue of 1.50, and explained 22.35% of the variance. Factor 3 contained three items, featured an eigenvalue of 1.01, and explained 21.12% of the variance. A three-factor structure extraction was confirmed and the factor solution explained 70.77% of the variance.

Table 3 Rotated Component Matrix of PRIAS

Figure 2 Scree plot of exploratory factor analysis.

Confirmatory Factor Analysis (CFA)

A total of 166 data points were used for the CFA. The model of the CFA for the PRIAS is shown in Figure 3. The preferred extraction method was used to estimate the structure and model fit indices. A structure factor analysis was conducted for each factor. Three-factor CFA yielded a good model fit [χ²(41) = 1.945, p < 0.001, GFI=0.950, CFI=0.916, RMSEA=0.076]. The comparative fit index (CFI) and goodness-of-fit index (GFI) values closer to 1 and root mean square error of approximation (RMSEA) values closer to 0 were used to assess the model fit. According to the criteria, the value of the chi-square for CFA should be non-significant, but it is nearly always significant due to the large sample sizes used in social sciences. To resolve this issue, the value of the chi-square is divided by the degree of freedom, and its value should be less than 3. Our results showed that the ratio of the chi-square and degree of freedom for the CFA was 1.945, which is in the range of the given criteria, and the results of the CFA were consistent with those of the EFA. The indicators of the PRIAS model were good, indicating that PRIAS had a three-dimensional structure, which is suitable for future research.

Figure 3 Measurement model fit of Postgraduate Research Innovation Ability Scale. Notes: Factor 1 = creativity-relevant process; Factor 2 = domain-relevant skills; Factor 3 = intrinsic motivation.

Criterion Validity

The Chinese version of the RIBS was used as the calibration scale for PRIAS. Table 4 shows the Pearson correlations among the variables. The results showed a significant positive correlation between PRIAS and RIBS (r = 0.53, p < 0.001). The correlation coefficients between the RIBS and the three subscales were 0.39, 0.57, and 0.44 (p < 0.05). The results indicate that the overall criterion correlation validity was satisfactory.

Table 4 Pearson Correlations Among the Variables (N = 164)

Discussion

This study aimed to construct and validate the PRIAS. Two studies in this research have verified that PRIAS has good reliability, structure validity and Criterion validity.Many scales have been developed worldwide to measure students’ innovation ability, but there is still no scale to measure postgraduate innovation ability in scientific research. Since postgraduates are the main group participating in scientific research and higher education, it was of great significance to measure their innovation with regards to scientific research. This research could help researchers understand the current state of postgraduate innovation and formulate measures to improve it. Therefore, the purpose of this study was to develop an effective and reliable scale to assess postgraduate innovation in scientific research.

Two studies were conducted in this present research. In step 1 of Study 1, the structure of the PRIAS was established and verified based on interviews, the component theory of creativity, and past literature. In step 2, a preliminary test was conducted. The 13 items generated in Step 1 of the PRIAS were administered. Based on the item analysis and EFA, two items were deleted. In Study 2, the factor structure of PRIAS was confirmed (Table 3 and Figure 2), and test-retest reliability, construct validity, and calibration validity were verified in an independent sample. In the pre-test and Study 2, the internal consistency of PRIAS had strong stability (pre-test: α =0.80; Study 2: α =0.89). The result of the test-retest reliability also indicated that the stability of the scale was satisfactory (r=0.86). In addition, the internal consistency coefficient and test-retest reliability of the creativity-relevant processes and domain-relevant skills subscales were good, while the test-retest reliability of the intrinsic motivation subscale was lower than that of the other two subscales, but still within the acceptable range. The reason for this might be that the intrinsic motivation items were fewer. Moreover, compared with the other two factors, intrinsic motivation is more easily affected by environmental factors such as academic pressure.²⁹ During the investigation period of this study, the postgraduates were taking exams, which might have affected intrinsic motivation.

The results of the EFA and CFA showed that postgraduate research innovation ability had a stable 3-factor structure, and a good model fit value was obtained for data with 11 items. This is a highly valid and reliable scale for measuring postgraduates’ research innovation ability (Box 1).

Box 1 Items of the Scale

Factor 1 (creativity-relevant processes), measured postgraduates’ logical thinking, innovative personality traits, innovative thinking, and problem-solving abilities. As the componential theory of creativity claims, the creativity-relevant processes include the ability, personality, cognitive style, and thinking strategy closely related to individuals’ performance of creativity. Several variables that are critical to research ability and innovation ability were considered for inclusion under this factor. For postgraduates, the creativity-relevant processes were the catalyst for postgraduates’ innovative achievements in scientific research as well as the processes to finding a solution to barriers in research. Factor 2 (domain-relevant skills) measured the level of postgraduates’ science literacy, professional skills, and professional knowledge. Domain-relevant skills could provide rich knowledge reserves, sufficient raw materials so individuals may think creatively, and enable individuals to seek the best solution from a wider knowledge base when facing a specific task. For postgraduates, domain-relevant skills are a basic requirement for scientific research. Finally, Factor 3 (intrinsic motivation) measured the level of postgraduates’ intrinsic motivation, including trait and state motivation. Trait motivation refers to the trait in which an individual has high intrinsic motivation that does not change dramatically due to the tasks and/or situations they face. State motivation refers to intrinsic motivation related to the current specific state and task, and is intrinsic motivation stimulated by a specific task.³⁰ Thus, there were two items in Factor 3 to measure trait and state motivation. The results showed that intrinsic motivation was well-fitted in the model. Domain-relevant skills determined “whether” an individual could create. Creativity-relevant processes determined “how” an individual could create. Intrinsic motivation provides the motivation to put ideas into action, and the mutual coordination of the three factors could better stimulate individuals’ creativity in research.³¹ Although it needs to be validated in more studies, the current PRIAS can be used to assess postgraduate research innovation ability. PRIAS can serve as a reference tool for supervisors to understand the research innovation ability of postgraduates. Factor 3 (intrinsic motivation) indicate the willingness and level to engage in research. Low scores in this factor indicate that postgraduates lack intrinsic motivation to participate in scientific research activities, thus affecting the ability of scientific research innovation. Factor 2 (domain-relevant skills) assess postgraduate knowledge, skills, and specific information necessary to tackle the research problem. High scores in this factor indicate that individual’s stock of domain-relevant skills is high, the likelihood of solving problems is higher; if not, the likelihood may be lower. Factor1 (creativity-relevant processes) measured postgraduates’ logical thinking, innovative personality traits, innovative thinking, and problem-solving abilities. High scores in this factor indicate that postgraduates can generate more ideas and improve possibility of solving the problem. Based on Componential theory of creativity, if the individual has better-developed creativity-relevant processes and stronger intrinsic motivation for the task, the number and novelty of ideas generated should be greater. Therefore, PRIAS can help postgraduates to know their deficiency of research innovation ability, so that they can adopt targeted strategies.

In China, most researchers have used external factors or aspects of organizational innovation to measure research innovation ability. This study focused on graduate students’ internal characteristics to measure postgraduates’ research innovation ability. It provides significant reference values and information for other researchers to explore postgraduate research innovation ability. Apart from enriching our understanding of postgraduate research innovation ability and opening avenues for further investigations in this field, the PRIAS might help verify the applicability of the componential theory of creativity among postgraduates in China. The PRIAS is a highly valid and reliable scale for measuring postgraduate research innovation ability.

Limitations

This study contributes to the theoretical and practical research on postgraduate research innovation ability and provides a new measurement tool for research on this topic. This study has several limitations. First, owing to the use of online platforms for testing, the sample size and representativeness of the samples was slightly insufficient, which might affect the validity of the scale. Researchers could conduct studies with larger samples to increase external validity or verify the effectiveness of PRIAS in different groups of graduate students in future study. Second, despite the high reliability and validity of the PRIAS, it is still unknown whether the measured level of graduate research and innovation ability was representative and discriminatory, and whether the PRIAS could be used to explore the psychological mechanism of postgraduate research innovation ability. Future study can apply PRIAS in the psychological mechanisms to test its validity and explore more factors affecting postgraduate research innovation ability.

Conclusion

Based on interviews and literature, this study constructed and tested the Postgraduate Research Innovation Ability Scale (PRIAS). The results showed that the PRIAS had good reliability and validity. The scale could be used in future investigations of postgraduate research innovation ability. PRIAS consists of three factors: creativity-relevant processes, domain-relevant skills, and intrinsic motivation, which cover the elements of postgraduate research innovation ability. Overall, we believe that the PRIAS is a reliable and valid measure for examining postgraduate research innovation ability. Future studies on the PRIAS with larger sample sizes in other locations are needed to demonstrate the generalizability of the scale.