This prospectus was completed 2016-11-30 for the course, IDS 7501: Issues & Research in Education, building upon my 2016-10-20 annotated bibliography and 2016-11-07 instrument search, as part of my coursework in the Education Ph.D. program at University of Central Florida.
This brief literature review and research proposal synthesizes the research on threshold concepts as applied to doctoral students, pioneered by Drs. Erik Mayer and Ray Land. Please feel free to cite this review. Feedback is welcome, via blog comments, my contact form, or Twitter.
Prospectus for Survey Research on Threshold Concepts and the Doctoral Process
University of Central Florida
This initial research prospectus first reviews the existing, primarily qualitative research on threshold concepts and doctoral students. Threshold concepts are skills that may be difficult to acquire, but their acquisition is transformative—much like learning to ride a bicycle. Frequently, threshold concepts such as designing a research study, writing a research report, and conducting a literature review are not explicitly taught to doctoral students. In fact, academics may not even consciously consider them, or may be dismissive toward students who have not yet acquired them. In this prospectus, the existing literature is used to guide the design of questionnaire-based research that quantitatively and qualitatively assesses threshold concepts in a broad sample of doctoral students at the University of Central Florida. Ultimately, this will contribute to our understanding of threshold concepts, the doctoral process, and the interaction between the two; guide further research including the development of a validated quantitative threshold concepts instrument; and perhaps suggest practices and workshops that may be implemented to reduce periods of uncertainty (liminality) among doctoral students, improve well-being, encourage productivity, and prevent attrition.
Keywords: threshold concepts, conceptual thresholds, doctoral studies, scholarly research, higher education, PhD students, liminality, troublesome knowledge, higher-order thinking skills, cognitive strategies, doctoral attrition, ontology, epistemology, beliefs
Prospectus for Survey Research on Threshold Concepts and the Doctoral Process
Threshold concepts (or, learning thresholds) are skills or ideas that may be troublesome to acquire, but doing so is often transformative—much like learning to ride a bicycle (Meyer & Land, 2005). The threshold concept framework (TCF) emerged in 2003 from Meyer and Land’s work on the U.K. national research project, Enhancing Teaching–Learning Environments in Undergraduate Courses, and was refined by their research on economists, which revealed that understanding certain concepts, such as opportunity cost, is pivotal to progressing in the field (Flanagan, 2016). Acquiring a threshold concept can be counter-intuitive and messy, causing both cognitive and affective stress not unlike a rite of passage (Rhem, 2013), with a liminal space in between that can be long and uncertain (Meyer & Land, 2006), particularly because instruction may address it only tacitly, peers and overseers may not know or recall what not “getting” a threshold concept feels like, and the endpoint of liminality is only clear in hindsight.
In particular, Ph.D. students are supposed to be our next generation of academics. They are expected to have acquired extensive technical and soft skills even at the outset of their programs, particularly for programs requiring a Master’s degree. Unfortunately, many have not acquired skills which appear to be threshold concepts, such as advanced writing and research skills (e.g., Kiley, 2015; Johnson, 2015). Further, there may be other threshold concepts for doctoral students that the limited body of primarily qualitative research has not revealed. Little quantitative research has been employed, and no quantitative instrument has been validated for the threshold concepts field in general, even after 13 years of research and hundreds of academic journal articles. Moreover, in the context of doctoral students, the beliefs of people who have achieved or not achieved threshold concepts has not been systematically compared. This proposed exploratory study will form a portion of the groundwork toward these ends.
Compared to students seeking professional graduate degrees (e.g., J.D., M.D., and terminal Master’s or specialist degrees), students seeking research doctorates receive a much larger investment of public funding in the United States, presumably because of their potential value to scientific progress and the research output and prestige of our nation as a whole. However, if these students do not achieve the requisite skills, they may fail to complete their programs, or graduate without essential skills, which can discourage them from pursuing a career in research and negate or reduce taxpayers’ return-on-investment. It can also leave them feeling inferior and demoralized. While it is tempting to conclude such students simply lack aptitude or perseverance, a more nuanced view is that in the doctoral process, certain skills such as conducting literature reviews (e.g., Wisker, 2015) and designing research (Exner, 2014) have a portal-like quality: those who achieve the skill are transformed, while those who do not remain in a “stuck” or liminal state, often due to no fault of their own. This liminality might be pictured as a wheeled cart being stuck behind an elevated threshold. The skills to be acquired may be troublesome—integration may require accepting counter-intuitive premises and overturning existing beliefs (Meyer & Land, 2003). This premise of threshold concepts may yield important insights into the journey toward “doctorateness” (Trafford & Leshem, 2009).
In exploring research on doctoral attrition, Breckner’s (2012) dissertation is of note because it was a phenomenological analysis on attrition among nine Counselor Education Ph.D. students, meaning he qualitatively interviewed students who left their programs. Because conducting exit interviews with attriting graduate students is rare, these kinds of insights are hard to come by. Like with Bair and Haworth’s (1999) meta-synthesis, Breckner found no single predictor of attrition: participants diverged when asked to identify their “biggest obstacle” (p. 57)—only two of nine cited the same reason: attrition of advisor. However, this phenomenon may align with the threshold concept framework: Wisker and Robinson (2013) purport that doctoral “orphans,” that is, those whose advisors have either moved to another institution or simply failed to fulfill their responsibility as an advisor (see also Ismail, Majid, & Ismail, 2013), are more likely to experience demoralization, liminality, and failure to cross conceptual thresholds. Another of Breckner’s (2012) subjects cited “professor leaving” (p. 57), which could similarly inhibit conceptual threshold crossing. Moreover, other students’ big issues could be characterized as threshold concepts: “technical writing” relates to writing ability as identified by Johnson (2015), “online classes” could relate to information illiteracy (e.g., Hoffer, Townsend, & Brunetti, 2012; Townsend, Brunetti, & Hofer, 2011), and “lost passion” and “program mismatch with identity” may relate to liminality, which can impede necessary shifts in ontology, epistemology, and personal identity (Keefer, 2015; Kelly, Russell, & Wallace, 2012).
Although threshold concepts research is eclectic, drawing insights from many disciplines, we can broadly organize the research on doctoral students into three categories: Skills and Strategies, including information literacy and research skills; Self-Beliefs and Epistemology, including meta-awareness, growth mindset, and personal conceptual frameworks; and External Influencers, including doctoral supervisory practices and supervisory attrition (see Appendix A).
Overall, we see that the numerous factors that contribute to doctoral attrition (for a somewhat dated meta-synthesis, see Bair & Haworth, 1999) may have a common thread binding them together—they may be more readily clustered and attacked through the lens of the threshold concept framework. Broadly, this framework has already yielded epistemological insights (Meyer & Land, 2005), design implications (Land, Cousin, Meyer, & Davies, 2005), and a pedagogical synthesis (Meyer, 2016). To our benefit, a small but promising corpus of qualitative research applying the framework to the doctoral process has emerged, in part revealing that doctoral students commonly struggle with writing (Humphrey & Simpson, 2012), research design (Exner, 2014), meta-awareness (Harlow & Peter, 2014), and feelings of stuckness (Kiley, 2009). However, this body of research is in need of replications, quantitative elements, and larger numbers of subjects to determine the existence, salience, and commonness of various threshold concepts. The present research prospectus is a step toward these ends.
While doctoral attrition can be an end result of frustrations in the doctoral journey, problems may start far earlier. Surveying doctoral students about concepts they find counter-intuitive or difficult to master may yield insights into how these areas can be further examined and ultimately improved, and provides the opportunity to examine threshold concepts in the context of development of scientific expertise. This area is highly relevant because it is abundant in troublesome knowledge—even experienced researchers may remain in liminal states with respect to elements of their practice (Kiley & Wisker, 2009).
For doctoral students, skillsets such as designing a research study, writing a research report, and conducting a literature review are often considered baseline prerequisites, but this assumption is often untrue and unhelpful. The threshold concept framework can help identify and explain the transitional challenges doctoral students face, as well as suggesting focal points for academic faculty and staff. Existing research has involved interviewing doctoral students with particular threshold concepts in mind (e.g., Hofer et al., 2012), or interviewing doctoral supervisors (e.g., Ismail et al., 2013). This mixed-methods study will survey doctoral students directly, with the purpose of gaining a window into the salience of known thresholds and discovering new thresholds, filling a gap in the research base.
These research questions have been established for the proposed study:
1. What academic and non-academic skillsets are important to being a successful doctoral student and can be considered conceptual thresholds?
2. How do doctoral students who have achieved a threshold concept differ from those who have not on beliefs about importance, counter-intuitiveness, personal aptitude, and others’ expectations regarding the concept?
To satisfy RQ1, participants will be asked to rate skillsets posited as threshold concepts by prior researchers, such as information literacy, meta-awareness, and supervisory relations, with respect to importance, difficulty or counter-intuitiveness, and their aptitude and achievement in these areas. For RQ2, based on self-reported achievement, students’ responses on other questions pertaining to particular threshold concepts will be compared, via descriptive and inferential statistics.
This study will employ a non-experimental, cross-sectional design using mixed-methods survey research, with no comparison group (Gall, Gall, & Borg, 2007). Many questions will require quantitative responses on a Likert-type scale, while other questions will elicit qualitative, typewritten insights or feedback which will be thematically coded to yield insights about pre-established threshold concepts and perhaps reveal new ones. Additionally, certain quantitative questions will be analyzed via correlation, with the purpose of discerning whether participants who have achieved particular threshold concepts differ in regards to their beliefs about the concept from those who remain stuck in a liminal state. This mixed-methods design provides flexibility and additional insights beyond using quantitative or qualitative methods alone.
Population and Sampling
The population of interest broadly consists of all doctoral research students in general. The sample, while based on convenience, purposefully targets doctoral research students. With approval, all doctoral students at University of Central Florida (UCF) will be solicited via email to participate in a questionnaire about their experiences and challenges in their doctoral studies. UCF is a large research university with 586 doctoral students as of Fall 2016 (www.admin.graduate.ucf.edu/admissionsreport/), potentially providing ample statistical power. UCF has 31 doctoral programs (www.graduatecatalog.ucf.edu/programs/), including 27 doctor of philosophy (Ph.D.) programs, two doctor of education (Ed.D). programs, one doctor of physical therapy (DPT) program, and one doctor of nursing practice (DNP) program. All 31 programs, with the possible exception of the DNP program, require students to conduct research and are thus relevant to the proposed study. UCF has a heavy focus on engineering, computer science, and other sciences, with 17 Ph.D. programs in these domains. Overall, 21 of 31 of UCF’s doctoral programs are approximately five years in length and require only a Bachelor’s degree, with six of these 21 having viable options for students holding Master’s degrees (viz., reduced program length). Ten doctoral programs require a Master’s degree as a mandatory prerequisite. The proposed study will ask questions that are at a sufficiently abstracted level to apply to this diverse sample that includes students from various fields with zero to five years of graduate-school experience, possibly with some differentiation of questions (e.g., doctoral students vs. candidates), based on responses to demographic questions at the beginning of the questionnaire.
Data Collection Procedures
Data will be collected via a Qualtrics questionnaire. The UCF Institutional Review Board (IRB) will be asked to review the proposed research, and is anticipated to classify this as exempt human subjects research with minimal risk. Participants will be adults of age 18 or older, who will consent and be informed of their rights prior to completing the questionnaire. Data will be de-identified, which prevents follow-up studies but protects the privacy of the participants. Though demographic data will be collected, when it is presented or published, it will be made sufficiently vague to prevent individual students being personally identified (for example, statistics on categories with fewer than five students will be omitted).
I will seek cooperation with the UCF College of Graduate Studies to solicit participation through mass email to all doctoral students at the university, including a follow-up email two weeks later to students who have not responded. If such a university-wide email solicitation cannot be achieved, I may be able to perform the study on a subset of doctoral students with help from faculty or administrators I am acquainted with in the College of Education and Human Performance, the College of Engineering and Computer Science, and the Department of Psychology (College of Sciences), who may distribute the questionnaire advertisement through official email lists or other channels. If a university-wide solicitation can be achieved, it is anticipated that at least 150 of 586 will complete the questionnaire.
Participants will be incentivized with a $5.00 Amazon gift card code, delivered digitally. In order to encourage participants to provide thorough submissions including responses on a majority of qualitative portions, participants will be required to respond to a minimum of 90% of Likert-type questions and 80% of typewritten questions to receive the gift card.
Instrumentation and Data Gathering
The quantitative, Likert-type questions in the questionnaire will unfortunately be non-validated and will have to be created by the author, because no such questions could be found, except Manyiwa (2006), who wrote non-validated Likert-type questions and administered them to students of marketing research. Questions will be organized around three themes: Skills and Strategies, Self-Beliefs and Epistemology, and External Influencers (see Appendix A). Short-answer questions and exploratory free-response questions will be used to potentially reveal new threshold concepts.
Manyiwa (2006, p. 6) created the following question template, administered on a five-point Likert-type scale ranging from “strongly disagree” to “strongly agree”:
1. The understanding of [concept] is very important for gaining new insight
2. I understand [this concept] very well
3. Previous knowledge is required to grasp [this concept]
4. The knowledge I gained [previously] prepared me for understanding [this concept]
5. On the face of it, [this concept] seems to be counter-intuitive
Manyiwa administered these questions for 35 separate concepts, including causal research design, qualitative research methods, normal distribution, hypothesis testing, and marketing research process, yielding 175 total items. Using only descriptive statistics, he identified, based on mean scores, 15 of 35 concepts as likely threshold concepts. While my email to Manyiwa on November 7, 2016 has not received a response, I believe his questions can be adapted to the present study without permission, but with credit given.
The present study will look at a smaller number of concepts in greater depth, perhaps examining 15 concepts with five quantitative (Likert-type) and one qualitative (typewritten) question per concept, plus additional open-ended questions at the end of the questionnaire. These questions will encompass:
- The importance and counter-intuitiveness of a concept (e.g., Manyiwa, 2006) [RQ1]
- Aptitude and achievement regarding the concept; expectations [RQ1, RQ2]
- Qualitative questions on liminality and attitudes [RQ1, RQ2]
- Qualitative questions on other potential threshold concepts [RQ1]
Aptitude and achievement are distinguished because one could have a growth mindset for a concept, meaning they believe effort is more important than inborn aptitude, or they could have a fixed mindset, meaning they believe others blessed with higher inborn aptitude are eternally privileged (Boyd, 2014; Thripp, 2016). Self-beliefs about aptitude may be correlated with achievement, but are very different constructs.
The concepts evaluated will be informed by the literature, and will include writing ability (Humphrey & Simpsons, 2012; Johnson, 2015), literature review skills (Wisker, 2015; Wisker & Robinson, 2009), information literacy skills (Hofer et al., 2012; Townsend et al., 2011), research skills (Rowe & Martin, 2014; Exner, 2014), understanding scientific theories (Kiley, 2015), meta-awareness (Harlow & Peter, 2014; Kiley, 2009), developing personal conceptual frameworks (Berman & Smith, 2015), liminality (Adorno, Cronley, & Smith, 2015; Keefer, 2015; Kelly et al., 2012), and interactions with supervisors (Johnson, 2014; Kiley & Wisker, 2009; Ismail et al., 2013; Wisker & Robinson, 2013). For doctoral candidates, there will be additional questions about the transition to candidacy and writing a dissertation (e.g., “doctorateness,” Trafford & Leshem, 2009).
Qualitative questions at the end of the questionnaire will ask “Are there any other difficult topics, areas, or concepts you find difficult? What makes them difficult? How could the learning process be made easier?” These questions may help us find other potential threshold concepts and may inform future research.
Data Analysis Procedures
Descriptive statistics will be used on the quantitative portion, including measures of central tendency and mean scores per question, per concept, and per question type (across concepts). Where sufficient amounts of data are present, inferential statistics such as correlation will be used to compare students’ responses for individual concepts based on self-reported achievement. These procedures will allow us to 1.) see which concepts are more salient as threshold concepts, and 2.) see if students who claim to have achieved a particular concept have different beliefs about its importance and counter-intuitiveness, personal aptitude, and others’ expectations regarding the concept.
Tentatively, qualitative (typewritten) data will be analyzed with thematic coding, guided by directed content analysis (Hseih & Shannon, 2005), with initial coding categories derived from existing research and modified or expanded on an ad-hoc basis, both incrementally and iteratively, as data analysis proceeds.
This study will shed more light on how threshold concepts apply to doctoral students, building upon past qualitative research while adding a quantitative element.
Obviously, survey research from a convenience, self-selecting sample is a limitation. Also, due to the limited research in this area, no validated instrument is available for threshold concepts. Studying all doctoral students at UCF is a broad scope, which could be both a strength in an eclectic field like threshold concepts, and a limitation due to the questions being generic rather than tailored to a particular program or department. Some of these limitations may be addressed by preceding this study with a small, pilot study, and the proposed study itself might be split into multiple studies looking at particular types of students (e.g., doctoral students vs. candidates; social sciences vs. “hard” sciences), and using different methods (e.g., qualitative vs. quantitative).
We will gain new insight into how threshold concepts shape the doctoral journey, where they lie and which are salient, and if the inward and outward beliefs of those who have achieved thresholds differ from those who have not. The proposed research adds a quantitative element that was not present in past research, will be valuable for having a larger sample size, and may help replicate findings from other research in a relatively new setting (the United States), since the majority of existing research has been conducted in Australia, New Zealand, and the United Kingdom. This study may suggest future quasi-experiments, suggest areas in which doctoral students should be provided additional scaffolding, and inform the development of threshold concepts instruments. Potentially, the development of workshops addressing particular threshold concepts may be informed by the results of this study.
Adorno, G., Cronley, C., & Smith, K. S. (2015). A different kind of animal: Liminal experiences of social work doctoral students. Innovations in Education and Teaching International, 52, 632–641. http://doi.org/10.1080/14703297.2013.833130
Bair, C. R., & Haworth, J. G. (1999, November 20). Doctoral student attrition and persistence: A meta-synthesis of research. Paper presented at the annual meeting of the Association for the Study of Higher Education, San Antonio, TX. (ERIC Document Reproduction Service No. ED437008).
Berman, J., & Smyth, R. (2015). Conceptual frameworks in the doctoral research process: A pedagogical model. Innovations in Education and Teaching International, 52, 125–136. http://doi.org/10.1080/14703297.2013.809011
Breckner, J. A. (2012). A phenomenological study of doctoral student attrition in counselor education (Doctoral dissertation). Retrieved from http://trace.tennessee.edu/utk_graddiss/1510/
Boyd, D. E. (2014). The growth mindset approach: A threshold concept in course redesign. Journal on Centers for Teaching and Learning, 6, 29–44.
Exner, N. (2014). Research information literacy: Addressing original researchers’ needs. Journal of Academic Librarianship, 40, 460–466. http://doi.org/10.1016/j.acalib.2014.06.006
Flanagan, M. (2016). Threshold concepts: Undergraduate teaching, postgraduate training, professional development and school education. Retrieved from http://www.ee.ucl.ac.uk/~mflanaga/thresholds.html
Gall, M. D., Gall, J. P., & Borg, W. R. (2007). Educational research: An introduction (8th ed.). Boston, MA: Pearson.
Harlow, A., & Peter, M. (2014). Mastering threshold concepts in tertiary education: “I know exactly what you are saying and I can understand it but I’ve got nowhere to hook it.” Waikato Journal of Education, 19(2), 7–23. http://doi.org/10.15663/wje.v19i2.95
Hofer, A. R., Townsend, L., & Brunetti, K. (2012). Troublesome concepts and information literacy: Investigating threshold concepts for IL instruction. Portal: Libraries and the Academy, 12, 387–405. http://doi.org/10.1353/pla.2012.0039
Hsieh, H.-F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15, 1277–1288. http://doi.org/10.1177/1049732305276687
Humphrey, R., & Simpson, B. (2012). Writes of passage: Writing up qualitative data as a threshold concept in doctoral research. Teaching in Higher Education, 17, 735–746. http://doi.org/10.1080/13562517.2012.678328
Ismail, H. M., Majid, F. A., & Ismail, I. S. (2013). “It’s complicated” relationship: Research students’ perspective on doctoral supervision. Procedia: Social and Behavioral Sciences, 90, 165–170. http://doi.org/10.1016/j.sbspro.2013.07.078
Johnson, E. M. (2014). Doctorates in the dark: Threshold concepts and the improvement of doctoral supervision. Waikato Journal of Education, 19(2), 69–81. http://doi.org/10.15663/wje.v19i2.99
Johnson, E. M. (2015). “He just told me to get on with it”: Insights into transforming doctoral writing development. Collected Essays on Learning and Teaching, 8, 147–152.
Keefer, J. M. (2015). Experiencing doctoral liminality as a conceptual threshold and how supervisors can use it. Innovations in Education and Teaching International, 52, 17–28. http://doi.org/10.1080/14703297.2014.981839
Kelly, F., Russell, M., & Wallace, L. (2012). Trouble in mind: Supporting the transition to graduate research in English. Arts and Humanities in Higher Education, 11, 418–433. http://doi.org/10.1177/1474022211416779
Kiley, M. (2009). Identifying threshold concepts and proposing strategies to support doctoral candidates. Innovations in Education and Teaching International, 46, 293–304. http://doi.org/10.1080/14703290903069001
Kiley, M. (2015). ‘I didn’t have a clue what they were talking about’: PhD candidates and theory. Innovations in Education and Teaching International, 52, 52–63. http://doi.org/10.1080/14703297.2014.981835
Kiley, M., & Wisker, G. (2009). Threshold concepts in research education and evidence of threshold crossing. Higher Education Research & Development, 28, 431–441. http://doi.org/10.1080/07294360903067930
Land, R., Cousin, G., Meyer, J. H. F., & Davies, P. (2005). Threshold concepts and troublesome knowledge (3): Implications for course design and evaluation. In C. Rust (Ed.), Improving student learning diversity and inclusivity (pp. 53–64). Oxford, England: Oxford Centre for Staff and Learning Development.
Manyiwa, S. (2006). Threshold concepts in teaching and learning undergraduate marketing research. Middlesex University Business School, 40. Retrieved from http://eprints.mdx.ac.uk/10040/1/Manyiwa.pdf
Meyer, J. F. (2016). Threshold concepts and pedagogic representation. Education + Training, 58, 463–475. http://doi.org/10.1108/ET-04-2016-0066
Meyer, J. H. F., & Land, R. (2003). Threshold concepts and troublesome knowledge: Linkages to ways of thinking and practising. In C. Rust (Ed.), Improving student learning: Theory and practice ten years on (pp. 412–424). Oxford, England: Oxford Centre for Staff and Learning Development.
Meyer, J. H. F., & Land, R. (2005). Threshold concepts and troublesome knowledge (2): Epistemological considerations and a conceptual framework for teaching and learning. Higher Education, 49, 373–388. http://doi.org/10.1007/s10734-004-6779-5
Meyer, J. H. F., & Land, R. (2006). Threshold concepts and troublesome knowledge: Issues of liminality. In J. H. F. Meyer & R. Land (Eds.), Overcoming barriers to student understanding: Threshold concepts and troublesome knowledge (pp. 19–32). New York, NY: Routledge.
Rhem, J. (2013). Before and after students “get it”: Threshold concepts. Retrieved from https://teachingcommons.stanford.edu/teaching-talk/and-after-students-get-it-threshold-concepts
Rowe, N., & Martin, R. (2014). Dancing onto the page: Crossing an academic borderland. Waikato Journal of Education, 19(2), 25–36. http://doi.org/10.15663/wje.v19i2.96
Thripp, R. X. (2016). The implications of mindsets for learning and instruction. Retrieved from http://thripp.com/2016/05/mindsets-education-lit-review/
Townsend, L., Brunetti, K., & Hofer, A. R. (2011). Threshold concepts and information literacy. Portal: Libraries and the Academy, 11, 853–869. https://doi.org/10.1353/pla.2011.0030
Trafford, V., & Leshem, S. (2009). Doctorateness as a threshold concept. Innovations in Education and Teaching International, 46, 305–316. http://doi.org/10.1080/14703290903069027
Wisker, G. (2015). Developing doctoral authors: Engaging with theoretical perspectives through the literature review. Innovations in Education and Teaching International, 52, 64–74. http://doi.org/10.1080/14703297.2014.981841
Wisker, G., & Robinson, G. (2009). Encouraging postgraduate students of literature and art to cross conceptual thresholds. Innovations in Education and Teaching International, 46, 317–330. http://doi.org/10.1080/14703290903069035
Wisker, G., & Robinson, G. (2013). Doctoral ‘orphans’: Nurturing and supporting the success of postgraduates who have lost their supervisors. Higher Education Research & Development, 32, 300–313. http://doi.org/10.1080/07294360.2012.657160
Appendix A: Concept Map for Threshold Concepts Literature Review
Note: Several sources (e.g., Breckner, 2012) were added after this concept map was constructed.