The Effects of General Listening Anxiety and Listening Test Anxiety on Self-Perceived Listening Performance Among Chinese English Learners

Introduction

The relationship between anxiety and language achievement has attracted the attention of researchers for decades.¹ In language education, most of the foreign language anxiety (FLA) and foreign language listening anxiety (FLLA) research has focused on anxiety elicited in the classroom^1,2 and test situations.^3,4 The practical reason for this may attribute to the limited access to the target language outside the classroom.⁵ But there is insufficient research on FLA and FLLA in out-of-school exposure situations. However, with the development of technology and media, language learners have more opportunities to access English outside of class. Research into out-of-school English-as-a-foreign-language (EFL) learning has found that young children learned English outside the school due to the prominent role of English in the internet, audiovisual resources, and various media.⁶ Research also found that the pervasiveness of English exposure outside the school makes English play a dominant role in daily life in European countries.⁷ Maybe it is the influence of linguistic distance (European languages are typologically similar to the target language English) that contributes to the strong facilitative role of out-of-school exposure to EFL learning.⁷ Although Chinese is typologically distant from English, frequent and extensive exposure to English due to the audiovisual input-rich situations nowadays may compensate for linguist distance. Frequent exposure to English outside the school may diminish FLA in classroom learning and test taking. Consequently, previous research lacks exploration of FLA and FLLA in extended situations, which may provide greater flexibility for further research to bridge the gap.

Previous research conducted from the theoretical perspective has mainly focused on exploring the conceptual defining of FLLA, measurement of FLLA, and the mechanism of FLLA-and-performance relationship. However, it was found that the defining of FLLA was unclear-cut and non-transparent, and the measurement of FLLA was inaccurate and inconsistent to the conceptual definition to a great extent.⁸ Research into the mechanism of FLLA-and-performance has mainly examined the linear effect of FLLA on listening performance; however, findings concerning FLLA-and-performance are inconsistent. For example, a significant negative correlation was found between FLLA and self-perceived listening proficiency,⁹ whereas it was also found that FLLA was not a predictor of listening performance.¹⁰ Likewise, research found that listening test anxiety was negatively related to listening performance;¹¹ on the other hand, the non-significant result of the effect of listening test anxiety on listening performance was detected by previous research.³ Against this backdrop of inconsistent findings, it is necessary to distinguish different types of FLLA because the conceptual confusion and contradictory of FLLA may introduce a major threat to the validity of the measure of FLLA; it is of great importance of developing a new scale to precisely assess FLLA according to the theoretical defining; also there is an urgent need for tapping into a unifying mechanism of FLLA-and-performance relationship based on exploring the complex relationship among different types of FLLA and listening performance.

As mentioned above, the present research aimed to explore FLLA from various situations, including in and outside the classroom, and the impacts of different types of FLLA on listening performance. We undertook two studies for the purposes of the FLLA dimension identification and measurement, and FLLA-and-performance relationship research. In Study 1, we firstly conducted semi-structured interviews to explore the potential anxiety-arousal situations and relationships between different types of FLLA, and further to develop a new scale for measuring FLLA from the situational perspective. In Study 2, using the newly developed scale, we verified the hypothesized relationships among different types of FLLA and self-perceived listening performance by employing structural equation modeling, and compared different effects of listening anxieties on listening performance among Chinese English major and non-English major learners. The main contributions of this research may lie in offering a new perspective to reveal the effect of FLLA on listening performance and deepening the understanding of the intensity differences of the impacts of different listening anxieties on listening performance.

Literature Review

Defining Foreign Language Listening Anxiety

The situation-specific perspective claimed that FLA was not a composite of other anxieties, but a distinct set of feelings of tension, apprehension, and nervousness limited to the language learning situations.^12,13 It can be seen as trait anxiety restricted to a given situation;^14,15 in other words, language learners had the trait of feeling state anxiety when learning and/or using language.¹⁶ However, a criticism of situation-specific perspective was that the situation was not precisely and clearly defined.¹⁴ Previous research has showed that language classroom and test situations were two situations that were especially prone to elicit language anxiety,^12,17 and FLLA evoked in classroom was considered general listening anxiety.¹⁷

Based on Complex Dynamic Systems Theory (CDST), general listening anxiety and listening test anxiety differed in the strength of attractor states.¹⁸ General language anxiety was considered learners’ self-perceived unpleasant attractor state in various situations¹⁹ that was a rather stable tendency or outcome for the dynamic system.¹⁸ While high-stakes test situations may contribute to the perturbations with the overall stable system, thereby changing the strong unpleasant attractor state into a weaker attractor state.¹⁸ In addition, general listening anxiety and listening test anxiety differed in the effects on listening performance. General listening anxiety was elicited in various situations (such as in daily communication situations⁵) in which listening comprehension was non-task-oriented and associated with anxiety-free responses.²⁰ It indicates that general listening anxiety may not have a direct effect on listening test performance. However, listening test anxiety directly and adversely affected task-oriented listening performance in high anxiety-eliciting test situations.^21,22 Therefore, the dynamic approach based on CDST¹⁹ captures the dynamic characteristics of FLLA and reframes different types of FLLA. In this study, general listening anxiety refers to an unpleasant emotion associated with listening in a foreign/second language in various situations that tends to settle in over time; listening test anxiety refers to a feeling of tension in listening comprehension in tests that constantly fluctuates due to the importance of tests, high-stakes situations, and interactions between these factors.

Furthermore, the dynamic approach emphasizes a contextualized perspective to construct FLA. According to CDST, FLA was influenced by both learner factors and contextual effects,^16,18,19 and subtle or strong effects of situations can exert great impacts on FLA.²³ It was proposed to identify FLA in most typical situations in learners’ past learning experiences because situational contexts have been confirmed to have unintended effects¹⁸ and FLA may interact with the situation of which FLA was a part. Moreover, attention should be given to the evolution of language learning from classroom situation to situations on a daily basis²⁴ because with the development of technology and media learners currently have more opportunities and resources available on various media to learn a foreign language beyond academic settings.^25,26 In other words, general listening anxiety should extend from classroom to a dimension of anxiety aroused in a wide range of situations.

In sum, the situation-specific perspective on FLLA emerged as trait-focused characteristics at the beginning, and then the dynamic perspective transformed it into dynamic and context-dependent construction, foregrounding the importance of situations where language learning takes place and thereby distinguishing general listening anxiety from listening test anxiety. In this sense, CDST has provided evidence that identifying different types of FLLA can help to specify different levels of anxiety, leading to different effects on listening performance.

The Effects of Foreign Language Listening Anxiety on Listening Performance

Research over the past two decades has found that FLLA was negatively related to a variety of objective measures of foreign language listening achievements, such as listening proficiency tests,^17,27,28 course grades,^29,30 and self-designed listening comprehension tests.^10,31,32 However, results from the meta-analysis revealed that the strongest relation between FLA and language achievements was observed for self-perceived measures.^33,34 In FLLA research, it has been proved that FLLA and self-perceived listening proficiency were negatively related,⁹ and FLLA was a powerful negative predictor of self-rated foreign language listening proficiency.³⁵ These findings suggest that FLLA may affect learners’ self-perceived assessment of the performance gap between an ideal score and the actual obtained score, and this subjective test performance can be assessed by self-perceived test performance.

With regard to listening test anxiety, evidence showed that the intensity of listening test anxiety was significantly higher than general listening anxiety;¹⁷ it indicated that language learners experienced a higher level of anxiety in high-stakes situations,²² which echoes the assumption that the level of anxiety depends on the situation.³⁶ However, despite the significant negative relationship between test anxiety and achievement measures, non-significant results of some studies deserve more attention. It was found that test anxiety did not influence average academic achievement³⁷ or listening test performance.³ These results may attribute to the delayed distribution time of listening anxiety scales. The long duration (eg, two weeks, a month, or a semester) between the listening anxiety tests and listening comprehension tests made it difficult to capture authentic feelings of listening test anxiety.^3,38 Instead, what a delayed distributed scale measured is the respondents’ global self-perception of FLLA in various situations. It suggests that test anxiety should be measured right after the test, and the measurement of FLLA in different situations is of great significance because it can more effectively distinguish listening test anxiety from listening anxiety in other situations.

The Measures of Foreign Language Listening Anxiety

Currently the most widely adopted or modified scale is Kim’s⁹ Foreign Language Listening Anxiety Scale, followed by a scale developed by Elkhafaifi (2005).³⁰ However, the validity of these two scales was questionable due to the inconsistency between the definitions and measurements.⁸ However, Horwitz, et al’s ¹² Foreign Language Classroom Anxiety Scale (FLCAS) and Chang’s¹⁷ Listening Anxiety Questionnaire are the scales that were developed based on corresponding conceptual definitions. But FLCAS focused on overall language skills, serving as a macro scale to identify FLA; it seemed inappropriate to use FLCAS to measure FLLA. Chang’s¹⁷ scale was situation-specific-based in terms of both the conceptual defining and scale development. Two situations were identified, named the general situation and the listening test situation. Nevertheless, the general situation was not specified. Few words may shed a dim light on the explanation of the term of the general situation, which was equal to classroom. However, currently classroom may not represent all situations in which listening anxiety prevails. Therefore, to ensure the accuracy of the measurement, FLLA should be measured based on various situations.

When academic majors are taken into consideration, mixed results of measurements of FLLA are obtained from previous studies. Research has found that humanities and non-humanities majors did not differ on the level of FLLA.⁹ On the other hand, significant differences were found between social science majors and math majors in FLLA, with math majors being more anxious than social science majors.³⁹ However, the measurement of different intensity of FLLA between English and non-English majors has been neglected. Previous studies have given great attention to the differences between English and non-English majors in motivation,^40,41 engagement,⁴² and learning environment.⁴³ These differences may reveal different learning styles of EFL learners; thereby we believed that there may be some differences in FLLA between English and non-English major students.

Based on the above discussion, this research aimed to investigate the following research questions: (1) How is FLLA measured from the situational perspective? (2) How and to what extent does FLLA affect EFL learners’ self-perceived listening performance? (3) How do Chinese EFL learners in different academic majors differ in FLLA? Two studies were developed for inquiry into the above three research questions.

Study 1

Research Objectives

To answer the first research question, Study 1 was conducted in the following three steps. The first step was to conduct semi-structured interviews that tapped into various anxiety-arousal situations to form an initial questionnaire. The second and third step involved an Exploratory Factor Analysis (EFA) and a Confirmatory Factor Analysis (CFA), respectively, which were conducted to testify reliability and validity of the newly developed FLLA scale.

Participants

Study 1 included three independent samples: the interviews sample, the EFA sample, and the CFA sample. In the semi-structured interview sample, 23 university students (12 English majors, 11 non-English majors) from grade one to grade four aged from 18 to 24 (M = 21.15, SD = 1.713) were recruited based on purposive sampling.⁴⁴ During the interview process, English majors and non-English majors were interviewed, respectively. It was found that no new theme can be generated after analyzing the interviews with 9 English majors; then three more interviews were conducted to achieve data saturation based on the guideline by Francis et al.⁴⁵ Likewise three more interviews were conducted after no additional issue was identified from 8 non-English majors’ interviews. Among all the participants, 10 were males (43.48%) and 13 were females (56.52%); their listening proficiency levels differ from high-intermediate to low-intermediate. They had completed at least six years of English studies before entering the university. None of them had ever visited or lived in any English-speaking counties. The interview process revealed the universality of the anxiety-arousal situations encountered by English and non-English majors. In other words, both English and non-English majors identified similar general situations; they only differed in the degree of sensitivity of generating these anxiety-arousal situations, with English majors more sensitive to anxiety-arousal situations. In the EFA sample, 215 English majors from grades two and three in a university in a city in central China participated in this study. Among them, 22 were males (10.23%); 193 were females (89.77%); they ranged in age from 18 to 22 years (M = 19.63, SD = 0.953). Because English as a foreign language has been designated in the curriculum, English courses are compulsory from junior middle school to university (some primary schools begin to teach English as a foreign language from Grade three). Therefore, they had received at least 8–12 years of formal EFL education. In the CFA sample, 208 English majors aged from 18 to 22 years (M = 19.80, SD = 0.844) participated in the study. They had similar EFL learning backgrounds to that of the participants in the pilot EFA study. Among them, 12 were males (5.78%) and 196 were females (94.23%). Snowball sampling was applied for the data collection process of CFA. The questionnaire was first distributed to English majors from two universities in central China, and then distributed to other English majors through invitations given by the initial respondents. The questionnaire was distributed in the form of network questionnaire through social media and group distribution. The quality control audit function was adopted in the questionnaire distribution; in other words, participants with missing data were automatically excluded.

Procedures

The semi-structured interviews adopted the situation-specific perspective on conceptualizing FLLA; more specifically, based on Chang’s binary model,¹⁷ FLLA was identified as general/classroom listening anxiety and listening test anxiety. FLLA in this study was defined as worry and apprehension of listening comprehension in the classroom, tests, and other out-of-school English exposure settings. To generate new items for developing FLLA scale, the structure of the interviews was developed based on previous research,^12,17 focusing on FLLA in the classroom, tests, and other potential situations. The interviews were conducted with individual participants, lasting 30–45 minutes each.

According to the participants’ responses in the interviews, the initial 25 items were generated and then evaluated for appropriateness by three experts. The items with similar or ambiguous meanings were eliminated. After the initial piloting, the retained 18 items that consisted of four factors were used to measure FLLA. A Likert response format was adopted consisting of a 7-choice response scale with the highest number 7 corresponding to strongly agree and the lowest number 1 corresponding to strongly disagree.

Data Analysis

The data analysis consists of three major stages. First, according to Hesse-Biber’s coding method, a top-down thematic analysis coding was employed to analyze the transcribed texts.⁴⁶ The analysis of the interview data progressed in three major steps:^46,47 (a) firstly each transcription was analyzed independently as raw materials; (b) subsequently patterns were identified by comparisons of the coding of raw materials; (c) then themes were generated from the coded patterns. Thus, an initial pool of scale items was formed based on the results of the interviews. Second, a total of 215 valid cases were coded and processed with SPSS,⁴⁸ and an EFA with the extraction method of principal axis factoring and rotation method of Oblimin with Kaiser Normalization was conducted to determine the component structure of the newly-developed scale. Third, a total of 208 cases were processed with AMOS⁴⁹ and a CFA was performed to evaluate construct validity and to testify whether the higher-order construct of general listening anxiety was reasonable.

Results

The Results of the Semi-Structured Interviews

Through the top-down template analysis process, four major thematic patterns were identified from the interview transcripts. The coding results of the semi-structured interview are presented in Table 1. Among them, two a priori themes detected from previous literature were listening test anxiety and listening anxiety in classroom, and two newly emerged patterns were listening anxiety in daily usage and listening anxiety in media learning. The results of interviews also revealed that the above anxiety-arousal situations are the representatives of general situations that may largely elicit FLLA. Although the participants differed in the degree of experiencing listening anxiety in these situations, they agreed that the above situations can provoke their FLLA if they were exposed to these situations.

Table 1 Coding Results of the Semi-Structured Interviews

Listening test anxiety referred to listeners’ fear of failure in tests. Listening anxiety in classroom referred to EFL learners’ feelings of uneasiness and nervousness in classroom performance. These results concurred with previous research^17,30 that classroom and test situations were commonly identified as the main anxiety-provoking situations. Participant 17 said that:

Sometimes I get worried before the English listening class because I was afraid to answer questions in English listening class.

Listening anxiety in daily usage referred to obstacles influencing successful receiving and decoding of information when using English on daily basis. Almost half of the participants reported that they were overly concerned about their reception of listening information when listening took place in daily communication, such as answering phone calls from college English teachers and foreign teachers, and attending social gatherings. The results indicate that situations that can elicit FLLA extend from traditional classroom and test situations to a great variety of situations in which EFL learners have daily contact with English. Participant 9 felt nervous when receiving information in English. He said:

I worry about not understanding the speaker who speaks English fast in a social gathering. Sometimes it is hard for me to keep up the speed. If I cannot receive information fluently, I will get worried and I can hear my heartbeat. If I cannot follow the conversation, I will be disappointed with my listening comprehension.

Listening anxiety in media learning referred to a frustrating feeling that EFL learners experienced when they engaged in informal learning on media, such as watching English movies and videos, and listening to English songs and news. The interviews showed that engagement in out-of-school English contact was unintentional language learning or listening practice; however, EFL learners were aware of potential benefits of listening gains from these activities. This finding was in line with the result found by Muñoz and Cadierno (2021) that out-of-school English exposure predicated listening improvement.⁶ However, when EFL learners perceived the gap between ideal improvement in listening comprehension and the actual experience of listening comprehension when engaging in out-of-school English contact, they became sensitive to FLLA. Participant 20 said about FLLA she experienced when she watched English movies:

I like watching English movies and videos on media, and I think it is good for my English learning because I can learn the cultural, know English with different accents, and practice listening. But when I fail to understand what the characters say in movies, I will become nervous and upset. And it makes me frustrated when I have to over-rely on Chinese subtitles to figure out the plot of the movie.

The results of interviews also revealed the relationship between general listening anxiety and listening test anxiety. To a great extent, FLLA elicited in situations like classroom, daily situations, and media learning can be considered as the representatives of general listening anxiety. In this study, general listening anxiety referred to the apprehension that occurs when EFL learners are exposure to a set of anxiety-arousal situations demanding listening skill. On the other hand, listening test anxiety referred to worry and tension over the comprehension of listening material experienced at a particular moment in test situations. In addition, the findings of the interviews suggest that general listening anxiety affects listening test anxiety. Many of the participants believed listening anxiety they experienced in general situations influenced listening anxiety in tests. For example, Participant 5 described the influence of general listening anxiety on listening test anxiety as follows:

After a period of listening practice on online APP learning, if I don’t gain improvement in my listening, I will put pressure on myself and I feel anxious. The more anxious I feel in listening practice by myself, the more anxious I feel during the listening test.

Although many previous studies dealt with general listening anxiety^11,30 and listening test anxiety,^3,17 respectively, there is no study examining the relationship between them. Our interviews suggest that general listening anxiety has an effect on listening test anxiety, which needs to be further tested.

The Results of EFA

The EFA results showed that Kaiser-Meyer-Olkin (KMO) index for Measure of Sampling Adequacy was 0.913, and Bartlett’s test of Sphericity was significant at the 0.001 level. These results indicated the data were appropriate for factor analysis. The 18 items revealed highly related four factors, which supported the interview results. Only one item was discarded for not loading on any factor. The remaining 17 items of the scale are detailed in Table 2. All of the loadings met the cutoff values proposed by Hair et al⁵⁰ (See Table 2). Finally, four distinct factors with 17 items emerged from the EFA, accounting for 76.827% of the variance. EFA generated four factors, named Listening Test Anxiety (LTA), Listening Anxiety in Daily Usage (LADU), Listening Anxiety in Classroom (LAC), and Listening Anxiety in Media Learning (LAML).

Table 2 The Scale Items and the Results of Exploratory Factor Analysis of the Scale

The Results of CFA

A CFA was performed on the second batch of the data to assess the four-component structure yielded via the EFA. It is suggested that the root mean square error of approximation (RMSEA) should be in the range between 0.05 and 0.08 (with the lower bound of 90% confidence interval less than 0.05 and the upper bound less than 0.10), and standardized root mean square residual (SRMR) should be less than 0.08.⁵¹ The results of the fit indices χ² = 201.503, df = 113, p < 0.001, χ²/df = 1.783, RMSEA = 0.062 [0.048, 0.075], SRMR = 0.041, together with the value of comparative fit index (CFI) 0.966, suggested that the model was reasonable fit, meeting general guidelines by Hair et al.⁵⁰

Convergent validity and discriminant validity tests were conducted to assess whether the model had an adequate level of validity. Convergent validity tests were examined by assessing standardized factor loadings, average variance extracted (AVE) values, and construct reliability (CR). According to the recommended thresholds suggested by Hair et al, ⁵⁰ the results showed that standardized factor loadings were higher than 0.7 and significant (p < 0.001); the AVEs that ranged from 0.598 to 0.781 were all greater than the recommended 0.5 threshold, and CR values (ranging from 0.882 to 0.934) were larger than 0.7 threshold (See Table 3). In CFA, discriminant validity is to estimate the extent to which a construct is different from another, which can be tested by Fornell-Larcker Criterion.⁵⁰ Fornell-Larcker Criterion is the positive square root of the AVE for the each construct that is higher than the highest correlation with constructs. Table 4 details the results of discriminant validity, indicating that the four dimensions were distinct from each other. In sum, the results of convergent validity and discriminant validity tests suggest the scale was of good construct validity.

Table 3 Results of Confirmatory Factor Analysis

Table 4 Results of Fornell-Larcker Criterion Test for Confirmatory Factor Analysis

Theoretically, LADU, LAML, LAC, and general listening anxiety (GLA) constituted a second-order construct that was a reflective-reflective type of construct. Based on the method of identifying and testing of a higher-order construct proposed by Doll et al, ⁵² four plausible alternative models composed of LADU, LAML, LAC, and GLA were compared and tested the fit of each hypothesized model. The results suggest that both model 3 and model 4 show better model-data fit than model 1 and model 2. The results of alternative models for the FLLA scale are presented in Table 5. Based on guidelines by Doll et al, ⁵² model 4 with three first-order factors (LADU, LAML, and LAC) and one second-order (GLA) provided higher validity than other alternative models, which is of greater theoretical interest than other models.

Table 5 Alternative Models for the Foreign Language Listening Anxiety Scale

Study 2

Research Hypotheses

In study 2, we examined the relationship between general listening anxiety, listening test anxiety, and self-perceived listening performance through structural equation modeling (SEM). Previous research has yielded mixed results on the relationship between FLLA and listening performance. For example, several studies have found a significant negative relationship between FLLA and listening achievement;^28,53 whilst some studies have not detected any significant correlation between FLLA and listening performance.^10,54 These incongruent outcomes may partially attribute to unclear distinctions between different types of FLLA. Given CDST has endeavored to explain the stability and instability of behaviors, which provides strong evidence for the distinction between general listening anxiety and listening test anxiety. According to this theory, when the system is more unstable, situational factors can have a great influence.⁵⁵ It suggests that listening test anxiety tends to be more heavily influenced by external situational factors (such as the high-stakes of test situations) than general FLLA does. Thus, we hypothesized that general listening anxiety and listening test anxiety have different effects on self-perceived listening performance. Specifically, the following hypotheses were formed:

H1: General listening anxiety has a positive effect on listening test anxiety.

H2: General listening anxiety has a negative effect on self-perceived listening performance.

H3: Listening test anxiety has a negative effect on self-perceived listening performance.

H4: General listening anxiety, mediated through listening test anxiety, influences self-perceived listening performance.

Based on previous research, it was found that the intensity of FLLA varied considerably due to different majors.³⁹ For the situation-specific perspective on FLLA discussed in this study, we were more concerned about the differences in general listening anxiety between English and non-English major students; thereby we put forward the following hypothesis:

H5: There is a significant difference in general listening anxiety between English and non-English major students.

Besides the possible differences in general listening anxiety, there may be differences in the strength of the impacts of general listening anxiety and listening test anxiety on listening performance between different majors, even though there is few research involving this issue. Therefore, we put forward the following research hypotheses:

H6: There is a significant difference in the impact of general listening anxiety on listening test anxiety between English and non-English major students.

H7: There is a significant difference in the effect of general listening anxiety on self-perceived listening performance between English and non-English major students.

H8: There is a significant difference in the effect of listening test anxiety on self-perceived listening performance between English and non-English major students.

Participants and Procedures

As mentioned in the literature review, to collect the feelings of self-reported listening anxiety, the scale should be distributed immediately after the listening test. In this study, the survey was conducted right after the listening course tests. We recruited all the senior English majors in a university in a city in central China as the participants (not the same cohort of participants in Study 1). A total of 163 English majors took the exam, and 152 participants voluntarily took part in the survey and finished the questionnaire. Among the included 152 participants, 13 (9%) were males and 139 (91%) were females. The participants ranged from 20 to 24 years of age (M = 21.28, SD = 0.743). All the participants had learned English as a foreign language from 10 to 14 years, and none of them had experience living in any English-speaking country. A total of 228 non-English majors participated in the survey and completed the questionnaire. Among them, 68 (30%) were males and 160 (70%) were females, and they were aged between 18 and 23 (M =19.55, SD = 0.793). All these participants, prior to college, had learned English as a second language for at least six years. At the college, the College English course is compulsory for these non-English major students for the first- and second-year of study.

Instruments

Foreign Language Listening Anxiety Scale

We employed the newly developed FLLA scale to measure listening anxiety. The scale consisted of 17 items that comprised four factors evaluated by EFA and CFA above. The items were answered on a 7-point Likert scale, ranging from 7 (strongly agree) to 1 (strongly disagree).

Self-Perceived Performance Assessment

The self-perceived performance referred to test takers’ self-assessments of the current listening test performance, and the full score of the self-assessment was 100. This self-assessment score was not the estimation of the actual score in the test, but the self-assessment of the performance of this test that whether they have outperformed or underperformed in this test.

Data Analysis

SmartPLS 3.3.2⁵⁶ was used for data analysis, because it has more advantages in dealing with a single item structure and small samples than covariance-based structural equation modeling, and it can better deal with high-order models. The data processing involved the following three procedures. First, measurement model testing was used to evaluate whether the reliability and validity of construct measurements met the requirements. The two-stage method⁵⁰ was employed for the analysis, because general listening anxiety in the model was a second-order construct that consisted of three first-order constructs as a new reflective-reflective construct. In the first stage, the repeated indicator approach was used to examine the measurement models of the first-order constructs (LAC, LADU, and LAML). In the second stage, the latent variable scores of the three first-order constructs were obtained (LAC_LV, LADU_LV, LAML_LV), which served as the indicators of the higher second-order construct (GLA).⁵⁰ Structural model testing and mediating effect testing were then conducted to test the hypotheses of this study.

Results

Assessment of Measurement Model

The assessment of the measurement model in PLS-SEM involves testing internal consistency reliability, convergent validity, and discriminant validity by judging the indicators of factor loadings, composite reliability (CR), Cronbach’s α, average variance extracted (AVE), and Fornell-Larcker Criterion. The results showed that the measurement models had good internal consistency, reliability, and convergence, meeting guidelines by Hair et al.⁵⁰ The specific statistics of factor loading, CR, Cronbach’s α, and AVE are provided in Table 6. The Fornell-Larcker criterion test indicates that the measurement model has good discriminant validity (see Table 7 and Table 8).

Table 6 Results of the Test of Measurement Models

Table 7 Results of Fornell-Larcker Criterion Test of the First-Order Constructs

Table 8 Results of Fornell-Larcker Criterion Test of the Second-Order Constructs

Measurement Model Invariance Testing

The measurement variance of composite models (MICOM) was used to assess the measurement model invariance. Based on the guideline of Hair et al, ⁵⁰ a three-step analysis was performed and the results showed that all the three constructs established configural and compositional invariance but the constructs did not have equal mean values and variances, indicating that partial measurement invariance was confirmed. The partial measurement invariance suggests that the comparison of differences between groups in models can be analyzed but data of different groups cannot be pooled together. The results of the equal mean assessment also showed that there were significant differences in general listening anxiety intensity between the two groups, suggesting that H5 was supported. Specifically, the findings revealed that non-English major students’ general listening anxiety was higher than that of English major students. Results of invariance measurement testing are presented in Table 9.

Table 9 Results of Invariance Measurement Testing

Structural Model Testing

To test the relationships among the constructs, bootstrapping with 5000 subsamples was run to assess the significance of path coefficients. The results showed that general listening anxiety had a significant effect on listening test anxiety, and listening test anxiety had a significant effect on self-perceived listening test performance. However, no significant relationship between general listening anxiety and self-perceived listening test performance was found. The summary of the indices for the testing of the structure model in Table 10 suggests that H1 and H3 were supported but H2 was rejected. In terms of evaluating mediation analysis, results showed that the mediation role of listening test anxiety was established (see Table 10 for details). According to the mediation analysis method of Zhao et al, ⁵⁷ it was found that listening test anxiety played a full mediation effect on the relationship between general listening anxiety and self-perceived listening performance, indicating that H4 was supported. In terms of coefficient of determination, Chin suggested that R² values higher than 0.67, 0.33, and 0.19 indicate a strong, moderate, and weak coefficient of determination, respectively.⁵⁸ As summarized in Figures 1 and 2, the models of the two groups moderately explained the variation of listening test anxiety (R²_EMS = 0.385, R²_non-EMS = 0.530) and weakly explained self-perceived listening test performance (R²_EMS = 0.208, R²_non-EMS = 0.225). In terms of model fitting, the value of SRMR of the two groups was 0.072 and 0.071, respectively, which is less than the recommended cut-off value of 0.08,⁵⁰ indicating that the model fitting was acceptable.

Table 10 Results of the Test of Structural Models

Figure 1 Results of the structural models analysis for English major group. Abbreviation: ns, not significant. Note: ***p < 0.001.

Figure 2 Results of the structural models analysis for non-English major group. Abbreviation: ns, not significant. Note: ***p < 0.001

Multi-Group Analysis

As detailed in Table 11, the results of the multi-group analysis showed that there was no significant difference between English and non-English major students in all three paths of the model. These findings showed that the structures were the same in the two models. Thus, H6, H7, and H8 were rejected.

Table 11 Results of Multi-Group Analysis

Discussion

The present study has taken a closer look at the distinction between general listening anxiety and listening test anxiety and the relationship between these listening anxieties and listening performance. The results of this research revealed that general anxiety-arousal situations involved classroom, English daily usage contexts, and media learning settings. The componential factors of foreign language listening anxiety included two factors, one having to do with general listening anxiety; the other with listening test anxiety. Specifically, general listening anxiety was typically represented by three sub-components: listening anxiety in classroom, listening anxiety in daily usage, and listening anxiety in media learning. The results also found that general listening anxiety had no direct effect on self-perceived listening performance; while listening test anxiety negatively affected self-perceived listening performance, and meanwhile listening test anxiety played a full mediation role in the relationship between general listening anxiety and self-perceived listening anxiety. In addition, a significant difference in the level of general listening anxiety was found between English and non-English majors; however, no significant difference in anxiety-performance linkage was found between the two groups from the multi-group analysis.

With regard to research question one concerning the measurement of FLLA, the results of the qualitative study found the emergence of four factors of FLLA: listening test anxiety, listening anxiety in daily usage, listening anxiety in classroom, and listening anxiety in media learning. And further the quantitative study has provided robust evidence for the distinctness of two types of listening anxiety: general listening anxiety and listening test anxiety. Findings across the studies supported the separation of listening test anxiety from general listening anxiety. These findings aligned with the results of studies that test anxiety should be independent of language anxiety,¹⁵ yet diverged from Horwitz et al claim that test anxiety was one of the componential factors of general foreign language anxiety.¹² Differentiating between listening test anxiety and general listening anxiety has expanded knowledge of the phenomenology of listening anxiety and helped identify potentially stressful situational characteristics as sources of listening anxiety. Furthermore, the empirical results of this research have supported the theoretical distinction between general listening anxiety and listening test anxiety under the framework of CDST. The CDST postulates that attractor state describes a dynamic system moving from one state to another as a result of the interactions between the individual factors and the nature of the situation.¹⁸ The nature of a listening test situation that characterizes as the time pressure for receiving, decoding, and processing transient spoken information⁵⁹ and highly emphasizes task-oriented performance²⁸ may act as a disturbing force that can change the original stable attractor state into an increased intensity of tension and worry.

In addition, evidence combined with the semi-structured interviews and qualitative analysis revealed that general anxiety-arousal situations included classroom, daily communication contexts, and media learning settings. This finding was not consistent with the result from previous research that general language anxiety was confined to classroom.^1,17 It indicates that new learning contexts can arouse unique general listening anxiety, which was congruent with the finding that general language anxiety was affected by contextual influences.^16,18,19 The result of the existence of one second-order factor and three first-order factors suggests that there exists a relatively stable general listening anxiety across several situations. This result concurred with the study by Hwang et al in that general English anxiety cannot be reduced by merely providing more English exposure in mobile-assisted language learning contexts.⁶⁰ The results also suggested that general listening anxiety used to situated in the static classroom¹⁷ has evolved into various context-dependent situations. With the development of technology and media, an array of out-of-school learning activities expanded language learning from the classroom context to daily and online learning settings;⁶¹ thereby it was necessary to explore language leaners’ emotional experiences beyond the classroom.²⁴

In respondence to research question two about the anxiety-and-performance relationship, results from the structural equation modeling confirmed different effects of listening anxieties on listening performance. Results found that only listening test anxiety negatively affected self-perceived listening performance (R² _EMS = 0.208). This finding was consistent with the quantitative research that test anxiety negatively influenced performance.⁶² However, general listening anxiety had no direct effect on self-perceived listening performance. Combined with the results from qualitative analysis, the empirical results of this study offered greater support for the need to separate listening test anxiety from general listening anxiety.⁸ This finding may help explain why no significant relationship between listening anxiety and listening test performance was detected in previous research,¹⁰ and why intervention in extensive listening failed to lower listening anxiety.⁶³ Because unawareness of the distinctions between general listening anxiety and listening test anxiety causes the mixed use of these two listening anxieties to tap into the causal relationship between general listening anxiety and listening test performance; however, general listening anxiety does not have a direct effect on listening test performance, which was not congruent with the findings of by Elkhafaifi (2005),³⁰ in that general listening anxiety was a strong predictor of listening test performance. The findings of this study indicate that a critically important distinction between different types of listening anxiety, which was overlooked in previous research, is a particularly useful path for anxiety-and-performance research.

In terms of the structural framework of listening anxieties and listening performance, the SEM results provided empirical evidence for the full mediation role of listening test anxiety on the relationship between general listening anxiety and self-perceived listening performance. Notably, this finding was inconsistent with previous research^17,27,53 in that empirical evidence was not found for the direct adverse impact of general listening anxiety on listening performance. It suggests that the separation of general listening anxiety and listening test anxiety may provide precise and accurate evidence for the detection of the relation between listening anxiety and performance, because it has been proved that general listening anxiety indirectly affected listening test performance through the full mediation effect of listening test anxiety. The result also suggests that the inconsistency in argumentation in previous research^3,9,10 concerning the relationship between FLLA and performance can be resolved by separating different types of FLLA and then exploring different effects of listening anxieties on performance. In addition, this empirical finding may support the theoretical assumptions in CDST that attractor states are assumed as a result of self-organizing, and external sources from environment can disturb the stability of the system.¹⁸ General listening anxiety can self-organize and self-regulate to settle in a relatively stable attractor state, which will not directly influence listening test performance. However, the test situation can perturb the stable attractor state into an anxious state that in turn directly influences listening test performance. Nevertheless, caution is required when generalizing from the main findings of this study, because learners with different cultural backgrounds may have multiple types of out-of-school target language exposures, which might contribute differently to the effect of general listening anxiety on listening test anxiety.

With regard to research question three concerning the difference in general listening anxiety between English and non-English major students, the results of MICOM of this study showed that a significant difference between the two groups was found, with non-English majors’ general listening anxiety higher than that of English majors. To explain this finding, two considerations are put forward. One possibility is that language proficiency plays an important role in the effect of general listening anxiety on learners with different majors. Given the fact that the language proficiency of English majors is higher than that of non-English majors, and the settled issue that proficiency is also regarded as a cause of anxiety,¹⁹ it is possible that English majors are not as strongly influenced by general listening anxiety as their non-English major peers. A second possibility is that the amount of out-of-school exposure to English listening may have effects on the general listening anxiety between different majors. Non-English majors may be less frequently exposed to out-of-school incidental language learning settings. It can be supported by the results gained from the semi-structure interviews that non-English majors were not as sensitive as English majors in generating anxiety-arousal situations; in other words, when non-English majors were prompted to identify possible anxiety-arousal situations, they responded that they experienced listening anxiety in these situations to a certain degree. Insensitively realizing the existence of these situations may suggest that non-English majors are not frequently exposed to out-of-school English contact situations. However, results obtained from the multi-group analysis showed that English majors did not differ from non-English majors in the effect of general listening anxiety on listening test anxiety, in the effect of general listening anxiety on self-perceived listening test performance, and in the effect of listening test anxiety on self-perceived listening test performance. The results revealed that the structure of listening anxiety affecting listening performance proposed in this study was relatively stable between English and non-English majors. For the two groups, the mechanism of anxiety-and-performance relationship was consistent.

Conclusions and Implications

The present research investigated foreign language listening anxiety both in and beyond the classroom setting and, for the first time, verified the component structure of foreign language listening anxiety. The analysis of the newly developed scale provided evidence in support of a two-component structure of foreign language listening anxiety: general listening anxiety and listening test anxiety. Further structural equation modeling analyses showed that general listening anxiety positively influenced listening test anxiety; listening test anxiety negatively influenced self-perceived listening test performance, and general listening anxiety indirectly affected self-perceived listening performance through the full mediation effect of listening test anxiety. For the group comparison analysis, results showed that non-English majors’ general listening anxiety was higher than that of English majors; however, the two groups did not show a difference in the intensity difference of the impact of general listening anxiety on listening test anxiety, and in the intensity differences of the impacts of listening anxieties on self-perceived listening performance.

Several implications for teachers can be drawn from this study. First, teachers should recognize the increasing importance of foreign language listening practice in out-of-school settings because lowering listening anxiety in general situations may effectively reduce listening anxiety in tests and improve test performance accordingly. Second, there is a need to reduce listening test anxiety as it influences listening test performance directly. Thereby, teachers should help students to regulate their worry and apprehension with effective strategies.

This research expands upon the findings of previous listening anxiety studies; however, it is not without limitations. One is the unbalanced proportion of male and female samples, because the overwhelming majority of English majors in universities in mainland China are female. In future studies, the samples can be enlarged with more male participants participating in the research. Second, in this research we only focused on English as a foreign language; however, we do not have evidence to show whether the newly developed foreign language listening anxiety scale is applicable to other types of foreign languages. Future research needs to validate the structure of foreign language listening anxiety in learners with various learning contexts. Third, although the results of the present research have shown that general listening anxiety had a significant effect on listening test anxiety, other factors may have a direct influence on listening test anxiety. For example, Chang (2008) has found that worrying about test difficulty was the source of listening test anxiety.¹⁷ Therefore, future studies can deepen the research by explicitly identifying various factors that influence listening test anxiety.