Development of the Cultural Tightness-Looseness Orientation Scale for College Students

Introduction

As an effective way to promote college students’ all-round development, scientifically explaining and accurately predicting their psychology and behavior undoubtedly could provide a reliable guarantee for the healthy and sustainability of higher education development. Therefore, scholars in all kinds of fields or disciplines up to now have been tirelessly exploring college students’ psychology and behavior from a variety of perspectives such as personality,¹ attitude and motive,^2,3 growth background and macro environment,⁴ teachers’ feedback,⁵ and so on. Since social norms are well confirmed to play crucial roles in influencing human beings’ psychology and behavior,^6,7 the concept “cultural tightness-looseness”, which mainly depicts human beings’ strength of social norms for important behaviors and tolerance for behavioral deviance,^8–10 can be applied to college students, who are the fundamental audience as well as the vital consumers of higher education, and is more likely to become a new, important perspective to understand and foresee college students’ psychology and behavior. Moreover, a great number of the existing theoretical and empirical studies regarding the definition,^8–12 the influencing factors,^7,10,12–14 the operationalization,^{10,12,15–17} and the consequences of cultural tightness-looseness^16–33 have laid solid foundation for scientifical interpretation and accurate prediction of college students’ psychology and behavior from this new perspective.

Surely, we cannot directly use this somewhat “broad” concept to explore college students’ psychology and behavior since college students have their own particularities in terms of their cultural tightness-looseness. For example, college students not only live in a broad society but also spend most of their time learning in “class”, which is a dominant situation that college students stay in almost every day but has its own characteristics compared to the other situations such as the broad society surrounding them. In an effort to better explore college students’ cultural tightness-looseness, therefore, our study specifies the broad concept “cultural tightness-looseness” as a concrete construct “the cultural tightness-looseness orientation of college students” and defines it as college students’ cognition or perception about tolerance for non-learning-behavior in class, strength of learning-behavior norms in class, and strength of social norms in the generalized macro-context according to the well-established definition of cultural tightness-looseness.^8–10 Just like that cultural tightness-looseness in a broad sense exerts wide impacts on human beings’ work and life, the cultural tightness-looseness orientation of college students permeates their daily life and is often strengthened by a series of psychological processes which are necessary for them to adapt to the external environment.¹⁰ Due to the fact that college students usually have individual difference in the quality and quantity of knowledge and come from not-the-same backgrounds, they may differ individually in their strength orientation of social norms and tolerance orientation for deviance in both the macro and micro situations.^9,34 Consequently, it is necessary to conduct theoretical and empirical research on the cultural tightness-looseness orientation of college students.

As a Chinese saying goes, “sharpen tools make good work”, an instrument that could effectively evaluate “the cultural tightness-looseness orientation of college students” is surely crucial to scientifically facilitate the empirical research on college students’ psychology and behavior from this new perspective of cultural tightness-looseness and in turn will determine the reliability and validity of the empirical research conclusions. After having a systematic literature review, our study finds that the 6-item scale of cultural tightness-looseness developed by Gelfand et al in 2011¹⁰ is the main instrument that has been widely recognized, accepted, and used by scholars for empirical research at the micro level.^20–22,30 Besides, the data of 33 countries’ or the extended 57 countries’ cultural tightness-looseness relatively calculated by Gelfand et al in 2011 and Eriksson et al in 2021 using the above 6-item scale,^10,35 the cultural tightness index of 50 US states based on 9 indicators computed by Harrington and Gelfand in 2014,¹⁶ and the data of 68 countries’ cultural tightness-looseness calculated by Uz in 2015 using EWVS database,¹⁷ are the primary second-hand data sources for scholars to conduct empirical research at the medium and macro levels.^{23,25–29,31–33} While using the extant data of cultural tightness-looseness has the advantage of convenience, there is inevitably the limitation that the second-hand data could only capture static information of cultural tightness-looseness at a fixed point of time rather than the real-time dynamics. Besides, although human beings’ cultural tightness-looseness tends to be stable for a period of time, the volatility, uncertainty, complexity, and ambiguity of the current globe as well as its broad and deep integration to some extent make human beings’ cultural tightness-looseness become more and more dynamic. In this case, exclusively relying on the second-hand data to explore the possible causes or consequences is undoubtedly difficulty to meet the time-fit research aim, which thereby is likely to lead to the inaccuracy of research conclusions. Moreover, the extant data, whether it is the cultural tightness-looseness data of 33 countries,¹⁰ the extended 57 countries,³⁵ 50 US states,¹⁶ or 68 countries,¹⁷ almost fails to cover all countries or regions in the globe, which largely constrains the empirical research of cultural tightness-looseness for countries or regions that were not taken into consideration previously. Therefore, what our study greatly concerns is a reliable and valid CTLOS-S which can help researchers collect college students’ real-time cognition of cultural tightness-looseness anywhere and anytime. However, the existing 6-item scale of cultural tightness-looseness developed by Gelfand et al,¹⁰ which is the main instrument that has been widely recognized, accepted, and used by scholars for empirical research at the micro level,^20–22,30 merely reflects college students’ strength orientation of social norms in the generalized macro-context and cannot capture their tolerance orientation for non-learning-behavior and strength orientation of learning-behavior norms in “class” which is a dominant situation that college students stay in almost every day. So, the main aim of our study is to newly develop the items that could effectively measure college students’ tolerance orientation for non-learning-behavior and strength orientation of learning-behavior norms in “class”, which could combine with Gelfand et al’s 6 items assessing strength orientation of social norms and deviance-tolerance orientation in the generalized macro-context to constitute a reliable and valid CTLOS-S.

As we said earlier, the construct “the cultural tightness-looseness orientation of college students” is the specific application of the broad concept “cultural tightness-looseness” to college students. The relevant research consists of studies at the national level such as the comparison study of cultural tightness-looseness orientation between the Chinese and American college students, studies at the regional level such as the cross-region study of college students’ cultural tightness-looseness orientation between South China and North China, studies at the organizational level such as the study exploring why college students from Peking University and Nanjing University differ in cultural tightness-looseness orientation, and studies at the individual level such as the investigation aiming at the impact of college students’ cultural tightness-looseness orientation on their learning engagement and career decision-making. Certainly, there are sharp differences as well as mutually nested relations among these research levels. For example, the lower, micro levels of college students’ cultural tightness-looseness orientation not only depict the particularity of behavioral norms in the direct micro-situation such as “class” but also share the commonality of the relatively higher, macro levels such as the organizational, regional, or national level of cultural tightness-looseness. Since the cultural tightness-looseness orientation of college students at the higher, macro levels could be measured through the aggregation of that at the lower, micro levels, we mainly seek to develop CTLOS-S at the most micro level, ie, the individual level.

The remainder of the paper is mainly structured as follows. Section “Literature Review” mainly summarizes the relevant empirical studies in recent ten years (2012–2022) at great length to lay foundation for CTLOS-S development. Section “Participants and Methods” chiefly introduces participants, procedure, item preparation, criterion variable selection and instrument, and statistical analysis. Section “Results” primarily reports how the formal 14-item CTLOS-S was obtained from the initial 17 measuring items and the reliability and validity testing results of the formal 14-item CTLOS-S. Section “Discussion” has a brief summary and discusses the deleted items and main limitations of CTLOS-S.

Literature Review

After systematically reviewing the representative, empirical studies of cultural tightness-looseness in recent ten years (2012–2022), we summarize the main measurements or empirical data sources of cultural tightness-looseness, of which the results are presented in Table 1.

Table 1 Main Measurements or Data Sources of Cultural Tightness-Looseness (2012–2022)

As shown in Table 1, the empirical studies of cultural tightness-looseness published in recent ten years (2012–2022) could be classified into two subtypes according to whether the empirical data came from researchers’ own investigation or the extant data calculated previously by others: one subtype mainly relies on the first-hand investigation data and the other subtype primarily exploits the second-hand data. Besides, the 6-item scale of cultural tightness-looseness developed by Gelfand et al in 2011,¹⁰ is the main measuring-tool that has been widely recognized, accepted, and used by scholars for empirical studies at the micro or individual level;^20–22,30 the cultural tightness-looseness data of 33 countries and the extended 57 countries relatively calculated by Gelfand et al in 2011 and Eriksson et al in 2021 using the above 6-item scale,^10,35 the cultural tightness index of 50 US states based on 9 indicators computed by Harrington and Gelfand in 2014,¹⁶ and the cultural tightness-looseness data of 68 countries calculated by Uz in 2015 using EWVS database,¹⁷ are the primary second-hand data sources for researchers to carry out empirical studies at the state and nation levels.^{23,25–29,31–33}

As mentioned earlier, exploiting the existing data, such as 33 or the extended 57 countries’ cultural tightness-looseness data separately computed by Gelfand et al in 2011 and Eriksson et al in 2021,^10,35 50 US states’ cultural tightness index calculated by Harrington and Gelfand in 2014,¹⁶ and 68 countries’ cultural tightness-looseness data computed by Uz in 2015,¹⁷ almost fails to capture the real-time dynamics of human beings’ cultural tightness-looseness and is more likely to result in the inaccuracy of research findings. Hence, what our study is strongly interested in is exactly the 6-item scale of cultural tightness-looseness developed by Gelfand et al in 2011 which can achieve the dynamic, time-fit research aim.¹⁰ These 6 items assessing the extent to which social norms are clearly defined, pervasive, and reliably imposed within nations are ① “There are many social norms that people are supposed to abide by in this country”, ② “In this country, there are very clear expectations for how people should act in most situations”, ③ “People agree upon what behaviors are appropriate versus inappropriate in most situations in this country”, ④ “People in this country have a great deal of freedom in deciding how they want to behave in most situations”, ⑤ “In this country, if someone acts in an inappropriate way, others will strongly disapprove”, and ⑥ “People in this country almost always comply with social norms”.¹⁰ Obviously, these 6 measuring items have distinctive characteristics of the generalized macro-context, which to some extent ensures the universality of Gelfand et al’s scale. It is worthy to note that, in addition to the above 6 items measuring people’s strength of social norms in general situation, Gelfand et al developed 12 items evaluating people’s social norms strength and deviance tolerance in everyday situations that they inhabit in the local worlds. In this way, Gelfand et al, on the one hand, ensured the universality of the scale through the original 6 items assessing human beings’ strength of social norms in the generalized macro-context and, on the other hand, guaranteed the content validity of the scale through the additional 12 items measuring people’s strength of social norms and tolerance for deviance in the local, everyday situations.

It is a pity that scholars severely ignore to appraise respondents’ strength of social norms and tolerance for deviant behaviors in the local, everyday situations when they exclusively employ Gelfand et al’s 6-item scale focusing on the generalized macro-context to measure respondents’ cultural tightness-looseness orientation in their empirical research, particularly in their empirical studies on the consequences of cultural tightness-looseness at the micro level, which therefore tends to reduce the accuracy of research conclusions through a somewhat flawed instrument of cultural tightness-looseness with lower content validity. For example, when empirically examining the potential interaction of overcompensation attainability and cultural tightness-looseness orientation on consumers’ intention to voice complaint at the individual level, Li et al adapted Gelfand et al’s 6-item scale that exclusively assesses people’s social norms strength in the generalized macro-context, deleted 1 item with low item-total correlation, and used the left 5 items to measure experiment participants’ cultural tightness-looseness orientation.²⁰ These five items are ① “I have clear expectations of what behaviors are appropriate or inappropriate in most situations in our society”, ② “I have clear expectations of how people should act in most situations in our society”, ③ “I have clear expectations of what social norms are appropriate or inappropriate in most situations in our society”, ④ “I think people are supposed to follow the social norms in our society”, and ⑤ “I think people should not have a great deal of freedom in deciding how they want to behave in most situations”.²⁰ It is apparent that these 5 measuring items do not depict consumers’ tolerance for service suppliers’ deviant behaviors in the most service-consuming situations. For another example, when exploring the impact of school tight-loose culture on middle school students’ bullying behavior in China, Wang and Song directly adopted Gelfand et al’s 6-item scale aiming at the generalized macro-context to operationalize “school tight-loose culture”.³⁰ Their study not only confuses the specific concept “school tight-loose culture” with the broad concept “cultural tightness-looseness” but also neglects the direct influences of students’ social norms strength and deviance tolerance in the dominant micro-situation, ie, “school”, on their bullying behavior.

To sum up, pertaining to the operationalization of cultural tightness-looseness in recent empirical studies (2012–2022), researchers mainly adopted or adapted Gelfand et al’s 6-item scale to merely evaluate people’s cultural tightness-looseness in the generalized macro-context and severely neglected to assess people’s cultural tightness-looseness in the local, everyday situations, which is more likely to result in the inappropriate research findings. This is exactly the major limitation of extant empirical studies on cultural tightness-looseness, as well as the underlying reason why our study is committed to developing CTLOS-S rather than directly adopting Gelfand et al’s 6-item scale to appraise the cultural tightness-looseness orientation of college students.

Participants and Methods

Abiding by the fundamental principles of scale development,³⁹ our study at first pooled the initial 17 measuring items of CTLOS-S based on literature review, the open-ended interview, and experts’ suggestions; we then took the Chinese college students in real-life as sample to conduct the first-stage questionnaire survey using the initial CTLOS-S; we continued to perform extreme group test, item-total correlation analysis, Cronbach test, and exploratory factor analysis of the first-stage questionnaire survey data to obtain the formal CTLOS-S; finally, we implemented the second-stage questionnaire survey adopting the formal CTLOS-S to do a series of reliability, validity, and across-gender invariance tests.

Participants

Our participants are the Chinese college students in real life sampled from two universities in Jiangsu Province, ie, Jiangsu Normal University and Xuzhou University of Technology, through the cluster random sampling technique. Two stages of questionnaire surveys were both completed using the drop-off and pick-up approach to ensure a higher response rate.⁴⁰ First, we got the selected majors, grades, and classes from the above two universities through the cluster random sampling technique; second, students from these chosen majors, grades, and classes gathered in a classroom by class, were informed about the purpose, procedure, and confidentiality of the following two stages of questionnaire surveys, and decided whether to take part in our questionnaire surveys according to their own will; third, after obtaining the participation consent letter from the sampled students, each questionnaire was on-site allocated to each participant and was on-site picked up when it was fully filled.

The first-stage questionnaire survey, which mainly aims at investigating the cultural tightness-looseness orientation of college students with the initial CTLOS-S, lasted two months, ie, from early November to late December in Year 2021. As we said previously, the first-stage questionnaire survey data was primarily used for the extreme group test, item-total correlation analysis, Cronbach test, and exploratory factor analysis of the initial CTLOS-S. Three hundred questionnaires in all were dropped-off and picked-up on site, of which 264 were valid ones, with a valid response rate of 88.00%. Among 264 valid participants, 162 were females (61.36%) and 102 were males (38.64%); 56, 96, 78, and 34 separately came from grade one (21.21%), grade two (36.36%), grade three (29.55%), and grade four (12.88%).

The second-stage questionnaire survey, which primarily aims at investigating the cultural tightness-looseness orientation of college students with the formal CTLOS-S and college students’ learning engagement as the criterion variable, commenced two months later and lasted three months, ie, from early March to late May in Year 2022. To avoid the answering bias, the measuring items of CTLOS-S and learning engagement scale were randomly merged into a questionnaire. As we said early, the second-stage questionnaire survey data was mainly used to test the reliability, validity, and across-gender invariance of the formal CTLOS-S. Nine hundred questionnaires in all were distributed and collected on site, of which 755 were valid ones, with a valid response rate of 83.89%. Among 755 valid participants, 409 were females (54.17%) and 346 were males (45.83%); 170, 369, 67, and 149 separately came from grade one (22.52%), grade two (48.87%), grade three (8.87%), and grade four (19.74%).

Methods

Item Preparation

As aforementioned, “the cultural tightness-looseness orientation of college students”, in which our study is interested to develop an instrument, is mainly a construct at the individual level that depicts to what degree individual college student perceives the tolerance for non-learning-behavior in the local, micro-situation “class”, to what degree she/he perceives the strength of learning-behavior norms in the local, micro-situation “class”, and to what degree she/he perceives the strength of social norms in the generalized macro-context. Therefore, in terms of the measuring items, CTLOS-S should at least consist of the following three dimensions: “tolerance orientation for non-learning-behavior in class” dimension which mainly generalizes individual college student’s total perception of tolerance for deviance in class, ie, the non-learning-behavior in class, “strength orientation of learning-behavior norms in class” dimension which primarily focuses on individual college student’s overall cognition of learning-behavior norms in class, and “strength orientation of social norms in the generalized macro-context” dimension which mainly reflects individual college student’s general cognition of social norms in the generalized macro-context.

Regarding the operationalization of the third dimension “strength orientation of social norms in the generalized macro-context”, we still adopt the 6-item scale of cultural tightness-looseness developed by Gelfand et al in 2011,¹⁰ which is widely recognized and well accepted by scholars.^20–22,30 As for the operationalization of the first dimension “tolerance orientation for non-learning-behavior in class” and the second dimension “strength orientation of learning-behavior norms in class”, which are two powerful supplements at the local, micro-class level to the third dimension, we mainly rely on expert-interview, deduction, and induction to obtain some typical learning behaviors in class and some representative non-learning behaviors in class.⁴¹ After multiple rounds of summary, feedback, and re-summary, 7 representative non-learning behaviors in class and 4 typical learning behaviors in class were finally acquired. Accordingly, we add 7 new items measuring individual college student’s tolerance orientation for non-learning-behavior in class and 4 new items evaluating individual college student’s strength orientation of learning-behavior norms in class to Gelfand et al’s 6-item scale mainly appraising individual college student’s strength orientation of social norms in the generalized macro-context. As shown in Table 2, therefore, the initial CTLOS-S has 17 measuring items in all with 7-item subscale of individual college student’s tolerance orientation for non-learning-behavior in class, 4-item subscale of individual college student’s strength orientation of learning-behavior norms in class, and 6-item subscale of individual college student’s strength orientation of social norms in the generalized macro-context. It is worthy to note that we acquire the Chinese version of Gelfand et al’s 6-item scale through the back-translation procedure between English and Chinese.^42,43

Table 2 The Initial CTLOS-S (17 Measuring Items)

In the first-stage questionnaire survey, each college student obtained by the cluster random sampling technique was required to use the 7-point Likert rating method to evaluate these initial 17 measuring items (items TL1-TL17 in Table 2) based on her/his actual perception or feeling: 1-strongly disagree, 2-disagree, 3-slightly disagree, 4-neutral, 5-slightly agree, 6-agree, 7-strongly agree. Generally, the higher the scores of these measuring items, the tighter individual college student’s cultural orientation becomes; the lower the scores of these measuring items, the looser individual college student’s cultural orientation becomes. It should be noted that the 7 measuring items for the first dimension (items TL1-TL7 in Table 2) are reverse-scored.

Criterion Variable Selection and Instrument

It is well established that cultural tightness-looseness orientation influences human beings’ psychology and behavior.^16–33 For example, Gelfand et al’s study confirmed that the individuals in a tight culture tend to have a higher level of self-control or self-management compared to those in a loose culture,¹⁹ Wang and Song empirically revealed that the culturally tight schools could effectively decrease students’ bullying behavior in China,³⁰ and Noh and Cho’s study on the potential impact of intrastate cultural tightness-looseness on firms’ accounting conservatism found that the firms located in culturally tight states of the USA are more likely to boost accounting conservatism which in turn could decrease the uncertainty of future economic events.³³

As a construct that primarily depicts college students’ strength orientation of social norms and tolerance orientation for deviant behaviors, “the cultural tightness-looseness orientation of college students” will also profoundly affect their psychology and behavior such as “learning engagement” which mainly refers to a positive, fulfilling, and study-related state of college students’ mind and is primarily characterized by learning vigor, learning dedication, and learning absorption.^44,45 To test the criterion validity of CTLOS-S, we choose “learning engagement” as the criterion variable. We assessed individual college student’s learning engagement with the 17-item scale developed by Schaufeli et al (see in Appendix 1),^44,45 which includes 6-item subscale of learning vigor, 5-item subscale of learning dedication, and 6-item subscale of learning absorption. A representative item for learning vigor subscale is “When I get up in the morning, I feel like going to class”, a typical item for learning dedication subscale is “I find my studies to be full of meaning and purpose”, and an example item for learning absorption subscale is “When I am studying, I forget everything else around me”.

In the second-stage questionnaire survey, each college student selected by the cluster random sampling technique was asked to use the 7-point Likert rating method to evaluate these 17 measuring items of learning engagement (see in Appendix 1) according to her/his reality: 1-strongly disagree, 2-disagree, 3-slightly disagree, 4-neutral, 5-slightly agree, 6-agree, 7-strongly agree. The reliability testing results of the total scale and its three subscales are good as the corresponding Cronbach’s alpha coefficients are 0.95, 0.88, 0.90, and 0.89. Certainly, college students participating in the second-stage questionnaire survey as well were requested to use the same rating method to appraise the formal 14 measuring items of CTLOS-S (see in Appendix 2).

Statistical Analysis

Using the first-stage questionnaire survey data (N = 264), we run independent sample t-test of the score-difference between the high-scored group (the top 27% participants) and the low-scored group (the bottom 27% participants), item-total correlation analysis, Cronbach test, and exploratory factor analysis with SPSS software to screen the initial 17 measuring items and identify the internal structure of CTLOS-S. Then, employing the second-stage questionnaire survey data (N = 755), a series of confirmatory factor analyses were further implemented with MPLUS software to test the construct validity and across-gender invariance of the formal 14-item CTLOS-S; Cronbach test and correlation analysis were as well conducted with SPSS software to test the reliability, criterion validity, and content validity of the formal 14-item CTLOS-S.

Results

Extreme Group Test

Following the specific procedure of extreme group test,⁴⁶ first, we summed up each score of these 17 measuring items to acquire a total score of cultural tightness-looseness orientation for each participant based on the first-stage questionnaire survey data (N = 264); second, according to the total score, we ranked the participants from the highest to the lowest and classified the top 27% ones and the bottom 27% ones relatively into the high- and low-scored groups of cultural tightness-looseness orientation; third, we run independent sample t-test of the score-difference between the high- and low-scored groups, of which the results are shown in Table 3.

Table 3 Results of Extreme Group Test (N = 264)

As indicated in Table 3, except for item TL15 (the score-difference value = −0.06, p = 0.83, 95% CI = [−0.57, 0.46], including 0, not significant), the score-difference values of the left 16 items (items TL1-TL14, TL16, and TL17) and the total scale between the high- and low-scored groups are all positively significant at the 0.05 level, which suggests that these 16 items and the total scale could effectively distinguish college students’ tighter and looser cultural-orientation. Accordingly, we delete item TL15 from the initial CTLOS-S.

Item-Total Correlation Analysis

After deleting item TL15, we continued to perform item-total correlation analysis of the retained 16 items, of which the results indicate that these 16 correlation coefficients, ie, 0.61, 0.61, 0.49, 0.71, 0.64, 0.48, 0.65, 0.63, 0.50, 0.55, 0.59, 0.44, 0.41, 0.36, 0.31, and 0.37, are all greater than the criterion of 0.30⁴⁷ and are all significant at the 0.05 level. Therefore, the retention of items TL1-TL14, TL16, and TL17 is as well supported by the results of item-total correlation analysis.

Cronbach Test

Based on the 16 retained items (items TL1-TL14, TL16, and TL17), we further conducted Cronbach test, of which the results are presented in Table 4.

Table 4 Results of Cronbach Test for the Retained 16 Items (N = 264)

When comparing the Cronbach’s alpha values of CTLOS-S before (the first column in Table 4) and after deleting the focal item (the third and fifth columns in Table 4), we can find that if we delete item TL17, the Cronbach’s alpha value of CTLOS-S will soar up from 0.75 to 0.85. Hence, we delete item TL17 from the retained 16 items.

Exploratory Factor Analysis

For the 15 retained items after the deletion of items TL15 and TL17, we continued to run exploratory factor analysis (EFA) to screen the measuring items and identify the internal structure of CTLOS-S based on the first-stage questionnaire survey data (N = 264). To check whether our empirical data was suitable or not for EFA, we calculated the value of Kaiser–Meyer–Olkin (KMO) and the corresponding p value of Bartlett’s test of sphericity at first. As KMO value (0.84) is higher than the criterion of 0.70 and the corresponding p value of Bartlett’s test of sphericity (χ²(105) = 1585.14, p = 0.00) is lower than 0.05,^48,49 our empirical data (N = 264) was suitable for EFA. Table 5 summarizes the results of EFA.

Table 5 Results of EFA for the Retained 15 Items (N = 264)

As shown in Table 5, three factors, of which the eigenvalues (5.17, 2.12, and 1.36) are higher than 1.00 and the cumulative variance contribution rate (57.66%) is more than the acceptable criterion of 50.00%,⁴⁹ are extracted from the retained 15 items (items TL1-TL14 and TL16). As the loading value of item TL16 (0.48) is lower than the satisfactory criterion of 0.50,⁵⁰ we drop it. Finally, a 14-item CTLOS-S with 3-factor structure has been yielded. Drawing on our conceptualization of “the cultural tightness-looseness orientation of college students” and the items each factor contains in Table 5, factor 1 is named “tolerance orientation for non-learning-behavior in class” dimension, factor 2 is named “strength orientation of learning-behavior norms in class” dimension, and factor 3 is named “strength orientation of social norms in the generalized macro-context” dimension.

Reliability Test

Using the second-stage questionnaire survey data (N = 755), the reliability tests of the formal 14-item CTLOS-S and its three subscales (see in Appendix 2) were further performed with SPSS software, of which the results suggest that the corresponding Cronbach’s alpha coefficients are 0.85, 0.85, 0.83, and 0.76. Since they are all greater than the satisfactory criterion of 0.70,⁵¹ the formal 14-item CTLOS-S and its three subscales all have good reliability.

Construct Validity Test

Also, using the questionnaire data from the second-stage survey (N = 755), a series of confirmatory factor analyses (CFA) with MPLUS software were run to test the 3-factor structure of the formal 14-item CTLOS-S which was found by EFA early. Table 6 summarizes CFA fit results of the hypothesized 3-factor model and its possible, alternative 2-factor and 1-factor models, which indicate that the fit indices of the hypothesized 3-factor model (Model “F1, F2, F3”) are satisfactory with χ²(74) = 318.76, χ²/df = 4.31, CFI = 0.94, TLI = 0.93, RMSEA = 0.06, and SRMR = 0.04 and are significantly superior to all the alternative models at the 0.001 level.⁵² Besides, the standardized factor loading values of the hypothesized 3-factor model range from 0.51 to 0.84 with an average value of 0.69. Hence, the formal 14-item CTLOS-S has good construct validity.

Table 6 Fit Results of Confirmatory Factor Analyses (N = 755)

Criterion Validity Test

Based on the second-stage questionnaire survey data (N = 755), correlation analysis between the formal 14-item CTLOS-S (see in Appendix 2) and the 17-item scale of college students’ learning engagement (see in Appendix 1) with SPSS software was carried out to test the criterion validity of CTLOS-S, of which the results are presented in Table 7.

Table 7 Results of Criterion Validity Test (N = 755)

As suggested in Table 7, college students’ cultural tightness-looseness orientation is significantly, positively related to their learning engagement (r = 0.30, p < 0.01), learning vigor (r = 0.24, p < 0.01), learning dedication (r = 0.34, p < 0.01), and learning absorption (r = 0.24, p < 0.01). Moreover, (1) “tolerance orientation for non-learning-behavior in class” dimension is significantly, positively related to learning engagement (r = 0.19, p < 0.01), learning vigor (r = 0.15, p < 0.01), learning dedication (r = 0.23, p < 0.01), and learning absorption (r = 0.14, p < 0.01), (2) “strength orientation of learning-behavior norms in class” dimension is significantly, positively related to learning engagement (r = 0.30, p < 0.01), learning vigor (r = 0.24, p < 0.01), learning dedication (r = 0.37, p < 0.01), and learning absorption (r = 0.24, p < 0.01), and (3) “strength orientation of social norms in the generalized macro-context” dimension is significantly, positively related to learning engagement (r = 0.28, p < 0.01), learning vigor (r = 0.26, p < 0.01), learning dedication (r = 0.27, p < 0.01), and learning absorption (r = 0.25, p < 0.01). Accordingly, the formal 14-item CTLOS-S has good criterion validity.⁵³

Content Validity Test

As well using the second-stage questionnaire survey data (N = 755), correlation analysis between the formal 14-item CTLOS-S and its three subscales (see in Appendix 2) with SPSS software was run to test the internal content validity, of which the results are displayed in Table 8.

Table 8 Results of Content Validity Test (N = 755)

As shown in Table 8, the correlation coefficients separately between the total scale and its three subscales (0.90, 0.71, and 0.55) and between each two of the three subscales (0.43, 0.24, and 0.41) are all positively significant at the 0.01 level. Therefore, the formal 14-item CTLOS-S has good content validity.

Measurement Invariance Test Across Gender

Employing the second-stage questionnaire survey data (N = 755), measurement invariance test across gender was implemented to check whether the 3-factor structure of the formal 14-item CTLOS-S is established robustly across gender, of which the results are presented in Table 9.

Table 9 Results of Measurement Invariance Test Across Gender

Table 9 suggests that the 3-factor model’s fit indices for male sample’s cultural tightness-looseness orientation (N = 346) are good: χ²(74) = 239.05, χ²/df = 3.23, CFI = 0.91, TLI = 0.90, RMSEA = 0.08, and SRMR = 0.06. The 3-factor model’s fit indices for female sample’s cultural tightness-looseness orientation (N = 409) are as well good: χ²(74) = 196.88, χ²/df = 2.66, CFI = 0.95, TLI = 0.94, RMSEA = 0.06, and SRMR = 0.05. Hence, the formal 14-item CTLOS-S with 3-factor structure has a characteristic of invariance across gender.

Discussion

College students inhabit in not only the generalized macro-context but also the local, unique class-situation, which are distinct from each other yet have the relation that the local class-situation is closely nested in the generalized macro-context. College students’ strength orientation of social norms and tolerance orientation for deviance in both the generalized macro-context and the local class-situation therefore are strong forces resulting in their differentiated psychology and behavior.^7,10,54 Particularly, college students’ general perception of what they should do and what they are not allowed to do in the local class-situation will directly influence their learning behavior, learning engagement, and learning achievement, which thereby will further affect the talent-cultivation quality of higher education.⁵⁵ Hence, our study applies the somewhat broad concept “cultural tightness-looseness” to college students, puts forward a concrete construct “the cultural tightness-looseness orientation of college students”, and seeks to scientifically develop an instrument with good reliability and validity to lay foundation for further empirical research.

According to the existing definition of cultural tightness-looseness and related literature,^8–10 the construct “the cultural tightness-looseness orientation of college students” proposed by our study mainly involves college students’ tolerance orientation for non-learning-behavior in class, strength orientation of learning-behavior norms in class, and strength orientation of social norms in the generalized macro-context. Thus, the operationalization of “the cultural tightness-looseness orientation of college students” should comprehensively reflect college students’ cultural tightness-looseness orientation at both the macro-context level and the micro-class level. However, the extant research tends to measure human beings’ cultural tightness-looseness orientation exclusively at the macro-context level and severely neglects its operationalization at the micro-situation level.^20–22,30 Accordingly, adopting Gelfand et al’s 6-item scale, which merely focuses on the measurement of human beings’ cultural tightness-looseness orientation at the macro-context level, to operationalize “the cultural tightness-looseness orientation of college students” is undoubtedly biased. Hence, our study develops 11 new items to appraise college students’ cultural tightness-looseness orientation at the micro-class level based on literature review, the open-ended interviews, and experts’ suggestions, which combine with Gelfand et al’s 6 items measuring college students’ cultural tightness-looseness orientation at the macro-context level to form the initial 17 measuring items of CTLOS-S (items TL1-TL17 in Table 2). We performed the first-stage questionnaire survey (N = 264) using the initial CTLOS-S and conducted extreme group test, item-total correlation analysis, Cronbach test, and exploratory factor analysis to yield the formal CTLOS-S, based on which we continued the second-stage questionnaire survey (N = 755) to further test the reliability, validity, and across-gender invariance of the formal CTLOS-S.

As the score-difference value of item TL15 between the high- and low-scored groups in extreme group test is not significant at the 0.05 level, we delete item TL15; since the deletion of item TL17 in Cronbach test could raise the Cronbach’s alpha value of CTLOS-S from 0.75 to 0.85, we delete item TL17; because the loading value of item TL16 in exploratory factor analysis is lower than the satisfactory criterion of 0.50, we drop item TL16. The above three items (items TL15-TL17) deleted from the initial CTLOS-S all belong to the subscale of college students’ strength orientation of social norms in the generalized macro-context. As presented in Table 2, item TL15 mainly depicts college students’ opinion of “Most people in our society have a great deal of freedom in deciding how they want to behave in most situations”, item TL16 largely describes college students’ viewpoint of “In our society, if someone acts in an inappropriate way, others will strongly disapprove”, and item TL17 primarily reflects college students’ perception of “People in our society almost always follow social norms”. It is speculated by our study that the Chinese college students may have different understandings about “a great deal”, “strongly”, and “almost”. As they do not reach a consensus on the extent to what extent the freedom can be “a great deal” of freedom, the score-difference value of item TL15 between the high- and low-scored groups is not statistically significant; since they do not agree with the extent to what extent others’ disapproval can be “strong”, the loading value of item TL16 in exploratory factor analysis is lower; as they do not have a shared understanding of the extent to what extent the occurring frequency can be “almost” always, item TL17 makes little contribution to the reliability of CTLOS-S.

The confirmatory factor analysis results of the formal 14 measuring items support the hypothesized 3-factor structure of CTLOS-S. Specifically, the first factor primarily involves college students’ tolerance orientation for non-learning-behavior in class which contains 7 reverse-scored items (items TL1-TL7 in Table 5), the second factor mainly refers to college students’ strength orientation of learning-behavior norms in class which contains 4 items (items TL8-TL11 in Table 5), and the third factor largely encompasses college students’ strength orientation of social norms in the generalized macro-context which contains 3 items (items TL12-TL14 in Table 5). The positively significant results of correlation analysis between the total scale and its three subscales suggest good content validity of CTLOS-S. Good criterion validity of CTLOS-S is as well supported by the significant, positive correlation coefficients between “the cultural tightness-looseness orientation of college students” and “learning engagement”. Moreover, the testing results of measurement invariance across gender confirm that the 3-factor structure of the formal CTLOS-S remains robust across gender, indicating that there is no significant difference between Chinese male and female college students in their understanding of each measuring item in the formal CTLOS-S.

To sum up, the formal 14-item CTLOS-S with 3-factor structure developed by our study meets the requirements of psychometrics well and can be used as an effective measuring-tool to evaluate “the cultural tightness-looseness orientation of college students”. There are, however, two main research limitations that should be addressed in future. First, we need more empirical evidence supporting the deletion of items TL15-TL17 from Gelfand et al’s original 6 items measuring college students’ strength orientation of social norms in the generalized macro-context. Second, pertaining to the 11 measuring items (items TL1-TL11 in Table 5) we newly develop to assess college students’ tolerance orientation for non-learning-behavior in class and strength orientation of learning-behavior norms in class, we should continue to conduct more questionnaire surveys among college students from other countries or regions in the future to further improve the reliability and validity, since they are merely developed under the background of Chinese colleges and universities.

Conclusion

Our study applies the broad concept “cultural tightness-looseness” to college students and puts forward a new construct “the cultural tightness-looseness orientation of college students”. To lay foundation for empirical research, we develop a 14-item CTLOS-S (see in Appendix 2) after a series of works including literature review, the open-ended interview among experts, induction, deduction, extreme group test, item-total correlation analysis, reliability test, validity test, and measurement invariance test across gender. The 14-item CTLOS-S is confirmed to have good reliability and validity. Besides, it contains three subscales separately assessing college students’ tolerance orientation for non-learning-behavior in class, strength orientation of learning-behavior norms in class, and strength orientation of social norms in the generalized macro-context. Scholars attempting to conduct quantitative studies on the influencing factors or the potential consequences of college students’ cultural tightness-looseness could adopt this 14-item CTLOS-S.