Symbiotic Space Exploring the Nexus of Rigor , Problems of Practice and Implementation

Across the United States, doctoral programs in education are in deep reflection about their purpose, content, and expected outcomes for graduates. Many are in the throes of redesign and testing to better differentiate between two pathways to the doctorate—one for scholars preparing for research and academic roles and one for scholar-practitioners who wish to focus on problems of practice and implementation. This paper explores a major source of challenge among faculty as they engineer both research-oriented and practitioner-oriented degrees, and that is the interconnections among the constructs of rigor, problems of practice, and implementation .  The paper examines the tension between the “gold standard” of random assignment studies and causal inferences, and the more recent frames of inquiry in which research is translated to practice through shared pathways that engage multiple stakeholders. Several questions are explored: Are “rigor” and “relevance” opposing constructs? What is rigor and should there be a new standard for scholars navigating within a symbiotic space – the arena in which they seek to impact highly complex social problems within authentic field-based settings? Is rigor subjective, or can it be measured? These questions are examined within the context of implementation science and through the lens of the lived experience of Education faculty who find themselves in a creative crucible as they navigate the redesign process.


INTRODUCTION
The so-called dichotomy between research and practice is a common theme in the field of education.Doctoral programs in education in the U.S. and around the world are in deep reflection about their purpose, content, and expected outcomes for graduates.Many are in active redesign and testing to better differentiate between two pathways to the doctorate-one for scholars preparing for research and academic roles and one for scholar-practitioners who wish to focus on problems of practice and implementation (Reardon & Shakeshaft, 2013;Walker et al., 2009;Zambo et al., 2014;Zusman, 2013).Programs, enrollments, and degrees in new professionalpractice doctoral fields have bourgeoned over the past decade, adding over 500 programs with more than 10,000 degrees awarded in 2011-12 and 35,000-40,000 students enrolled (6% of all U.S. doctoral degrees) (Zusman, 2013).
Our experience navigating the dual implementation of both a Professional Practice Doctorate and a new PhD program within the environment of traditional EdD degrees presents an opportunity for close examination of the construct of "rigor" within each program structure.It may be instructive to others in similar situations.How faculty communicate, debate, and cooperate to resolve the collateral tensions represents a primary challenge as practitioner-oriented doctoral programs expand.

THE CARNEGIE PROJECT AND OUR PARTICIPATION
The Carnegie Project on the Education Doctorate (CPED) is an organization dedicated to ensuring high-quality EdD programs.For 10 years it has explored the purpose and nature of the doctorate in education and encouraged alternatives to the traditional dissertation (e.g., the "dissertation in practice").Doctoral programs concerned about relevance and impact are based on a set of principles aimed at graduating scholars who can integrate theory and practice as well as address critical problems of practice.Six working principles were established by CPED member institutions to ensure excellence in educational doctorate preparation programs.According to the principles, the professional doctorate in education: (1) is framed around questions of equity, ethics, and social justice to bring about solutions to complex problems of practice; (2) prepares leaders who can construct and apply knowledge to make a positive difference in the lives of individuals, families, organizations, and communities; (3) provides opportunities for candidates to develop and demonstrate collaboration and communication skills to work with diverse communities and to build partnerships; (4) provides field-based opportunities to analyze problems of practice and use multiple frames to develop meaningful solutions; (5) is grounded in and develops a professional 7 knowledge base that integrates both practical and research knowledge and links theory with systemic and systematic inquiry; and (6) emphasizes the generation, transformation, and use of professional knowledge and practice (Carnegie Project on the Education Doctorate, 2009).Over the past three years, in the Graduate School of Education and Human Development, our faculty participation with the Carnegie Project on the Education Doctorate has stimulated an appraisal of our EdD Programs-the intellectual content, rigor, and relevance to our consumers as they prepare to face the dominant challenges in education today in the U.S. and internationally.

Outcomes of Our Participation in the Carnegie Project
A faculty learning community was formed in 2013 to identify core features of effective doctoral programs and to work toward purposeful transformation as we rethink the foundations of our programs.Such transformation has not meant reinvention for the sake of change, but rather faculty recognized the need to nurture a new generation of courageous transformative scholars who could translate research and impact practice.They realized that new standards of rigor needed to be defined for those working within a symbiotic (mutually beneficial) space between the University and the field-"transformative" scholars who are change agents seeking to impact highly complex social problems within authentic field-based settings.
We currently offer two doctoral degrees, the Doctorate in Education (EdD) and the Doctor of Philosophy (PhD).EdD degrees can be obtained in several subfields, including Educational Leadership, Education Policy, Higher Education Administration, Human and Organizational Learning, Curriculum and Pedagogy, and Special Education.PhD degrees are offered in Education and in Counseling.A research-focused PhD program in Education has recently been approved and a faculty-launch committee formed to implement the program.Features of the PhD program include a modified admission process for applicants, a research course sequence that differs between the EdD and PhD and research apprenticeships that begin in the first semester of the program.
Frameworks for a third degree-the Professional Practice Doctorate (PPD)-are under development in Education Policy, Educational Leadership, Special Education, and Higher Education Administration.These programs are characterized by a shorter yet intense program and field-based apprenticeships in which scholarpractitioners draw on their coursework to define and examine significant problems of practice.Capstone projects are designed to be of direct benefit in impacting a school system, organization, governmental agency, or community organization.

Dimensions of Culture Change
As faculty navigate the redesign, they have begun to experience a wholesale culture change that includes aspects of faculty work, behavior, values, and attitudes.First, faculty expressed concern about whether resources would be shifted away from the EdD degrees (or new PPDs) to the PhD program, as the latter would admit only full-time students who rely on full fellowships.The PhD program was long fought for over a three-year period, and therefore stakes are high for its success.It is important that faculty and the finance director of the department or unit closely observe the patterns of change in resource distribution among the programs in the subsequent years, with attention to issues of equity and the sustainability of both types of programs.
Second, the redesign of academic programs in a school of education challenges well-established ideology and traditions of many faculty.For example, for faculty who have deep-seated beliefs that the PhD is appropriate only for those aiming for careers in research and academia, advanced research courses and extensive research apprenticeships are viewed as a central part of the program and the program is expected to take 5 years or longer.Other faculty are interested in advancing the development of the professional-practice doctorate designed for applied research, field-based apprenticeships that prepare students for leadership roles, and can be completed in 3-1/2 years.It was vital to the success of the PPD to support those faculty members who were excited by alternative models and willing to work together to create new programs.They came to represent a new "force" within the School and were encouraged by the Dean and academic leadership.
Third, balance in teaching, research, and advising loads are a paramount concern in an environment with both PhD and practitioner-based EdDs or PPD.PhD programs tend to have smaller cohorts than the PPD programs.However the PPD is equally as intensive, as faculty load is increased by time spent mentoring students as they create projects of impact, guiding them as they negotiate with partner schools or organizations, and closely supporting their work with stakeholders in the field.A major source of cultural dissonance, however, occurs as faculty works to ensure rigor in the curriculum, the field experiences, and the dissertation within the two programs.

Rigor vs. Relevance
Relevance for practice and for implementation of research have also become an important criteria for rigor.Regardless of how technically sound research projects are, if they do not address issues or questions of concern to practitioners and policymakers, it is unlikely that the research will be used (Bradford, 2015).However, despite considerable focus on the research-to-practice gap and on identifying evidence-based practices as a means to bridge it, there is little evidence that suggests that the gap has been meaningfully narrowed (Cook & Odom, 2013).The implementation of research, therefore, becomes an important focus for understanding the problem.Fixsen et al. (2005) defined implementation broadly as''activities designed to put into practice an activity or program" (p.5).In the inaugural issue of Implementation Science, Eccles and Mittman (2006) defined implementation science as "the scientific study of methods to promote the systematic uptake of research findings and other evidence-based practices into routine practice" (p. 1).Evidence-based practices were required to be supported by multiple, high-quality, experimental or quasi-experimental studies demonstrating that the practice has a meaningful impact on subjects (e.g., student outcomes) (Odom, 2005).However, implementation and translation of research findings to practice-"wicked problems," according to Fixsen, Blase, Duda, Naoom, and Van Dyke (2009)-represent the most perplexing aspects of evidence-based practices.Implementation conditions are always in flux and engage multiple layers of infrastructure of a system that has been referred to as "contested spaces," in which key stakeholders can either support or obstruct transformation (Gutiérrez & Vossoughi, 2010;Penuel & Spillane, 2014).Gutiérrez and Penuel (2014), therefore, argue for a reconceptualization of rigor that requires sustained, direct, and systematic documentation of what takes place inside programs in order to document how systems being stu-died change and adapt interventions in their interactions with each other in relation to their dynamic local contexts.

Rigor, Problems of Practice, and Implementation Science
The aim of the Educational Sciences Reform Act in 2002 was to call for scientifically based research that would "apply rigorous, systematic, and objective methodology to obtain reliable and valid knowledge relevant to education activities and programs" (Pub.L. No. 107-279, p. 116).However, the field continued to express concern about the narrow set of criteria used to define "rigor," and the establishment of a "scientific culture" that relied primarily on the "gold standard" random assignment studies of program effects as the remedy for the failures of education research to offer credible guidance for policy and practice (Gutiérrez & Penuel, 2014).At the time, Erickson and Gutiérrez (2002) (Gutiérrez & Penuel, 2014) As a result, the Institute for Education Sciences (IES) began research programs that focused on problems of practice and for which relevance for practice and policy was an explicit criterion for judging the quality of research proposals (Easton, 2013).Examples of these programs include Continuous Improvement Research in Education and the Researcher-Practitioner Partnership in Education Research.These programs were not intended to promote translation of research to practice in a unidirectional manner, but rather through a more reciprocal pathway (Easton, 2013).Criteria for judging the "relevance" of research proposals were required to include the following:

•
Documentation that the problem of focus is perceived by multiple stakeholders to be significant, persistent, and worthy of investigation

•
Evidence provided by researchers that they have engaged in a process to negotiate the focus of their joint work and which documents how participation in the process was structured to include diverse education stakeholders • Specific methods for bringing relevant stakeholders together and deliberating about the problems that can and should be addressed through research and development

•
Engaging the multiple layers of a system's infrastructure that have accumulated over time (contested spaces) so that they can support rather than obstruct transformation (Gutiérrez & Vossoughi, 2010;Penuel & Spillane, 2014) These criteria include important aspects such as 'problems worthy of investigation,' 'multiple stakeholders,' 'negotiating the focus,' 'participation in the process,' 'deliberation about problems,' and 'engaging layers of infrastructure.Following Gutiérrez and Vossoughi (2010), rigorous and consequential study of efficacy of educational interventions involved sustained first-hand observation, sharing in the action and cognition of practitioners, studying side-by-side in symbiotic space, and being jointly engaged in work to transform systems.These conditions are more likely to produce sensitive and robust measurement and ecologically valid accounts of institutional change.
Educational interventions have been viewed as contested spaces that are filled with tension and resistance from a range of stakeholders.Supporting and engaging more diverse stakeholders in defining the focus of research and development requires that researchers rethink the nature of educational interventions.Approaches to interventions can be viewed as dialectical systems that are subject to revisions, disruptions, and contradictions (Gutierrez & Vissoughi, 2010).Therefore, research into the "social life of interventions" demands more open-ended, socially embedded studies that involve mutual engagement, and the identification of contradictions within and across the levels of the system under study.The contested spaces, then, become symbiotic spaces.

Differences of Opinion About Research Preparation
The definition of 'rigor' as a key standard for innovative PPD programs actually encompasses a set of transformative developmental skills that reach beyond the aims and capacities of most traditional research oriented programs.These transformative skills enable scholars to harness their creativity to solve crucial and highly complex social problems.Transformative programs create transformative scholars who can redraw or expand the boundaries of practice and policy because they have developed the commitment and identity of a change agent, are guided by a powerful vision of the future, and can translate that vision into reality regardless of the environment (Nisan & Pekarsky, 2009).Transformative scholars envision, in the field of education, that which has never been, and they do whatever it takes to make it happen.In short, they are prepared for high-impact action in the world.

Rigor and the PhD
In recent years, academics have debated the value of the EdD and considered plans to improve it, and to raise its stature to that of the PhD (Basu, 2012), or to offer both degrees.University program redesigners tend to distinguish the degrees by candidates' role aspirations, but do not attempt to define 'rigor' in relation to the two degrees.For example, in 2012, Harvard University eliminated the EdD which it had offered for 90 years and replaced it with a PhD The Academic Dean, Hiro Yoshikawa, explained that the EdD program was already a research-based degree, and so to eliminate the confusion between the two degrees they moved to the PhD for those preparing for research roles.The EdD was replaced by a shorter, practice-based program designed for those preparing for leadership in education.The goal was to equate such a degree with other professional degrees rather than with the PhD.
The Rossier School of Education, University of Southern California, distinguishes the PhD program (a 44-month program) as "research-oriented," whereas the EdD programs (32-33 months) are directed toward educational "practice" and the application of theory and research.The EdD is considered equal in rigor but different in substance from the PhD (USC, 2017).The PhD student is typically a person who (a) anticipates a faculty career or an area of practice that demands research expertise, (b) is excited by theory and conceptual analysis, and (c) intends to participate in research and has the potential for primarily advancing the theory in the field rather than implementing its practice.The EdD student is a person who (a) is planning to work in the field as a practitioner, (b) is particularly interested in developing new practical or technological capabilities, and (c) is interested in research that emphasizes development, evaluation, or field-based projects (USC, 2017).
Concerns about creating and sustaining rigor represented the greatest challenge for the quality and coexistence of the two programs.A review by the current author of the content and requirements of 25 PhD programs yielded the following criteria by which quality of the programs is judged: intensity of coursework; high level of critical thinking; a common domain of knowledge in the discipline that all students should master; advanced knowledge of the research methodologies of the discipline; original research contributions that build on current research available in the field of study; and the stature of the journal or other media for publications.Requirements in these programs for candidates who complete a doctoral degree, either the PhD or the EdD (in the 20 programs that offered both), should demonstrate the following skills:

The Crucible of Implementation: Rigor and Laboratories of Practice
Definitions of rigor and program quality require re-imagination in the context of authentic field-based settings.The previous discussion about the meaning of rigor and relevance leads to other questions that are equally as provocative in the context of program design and expectations for candidate performance.Can a practitioner-oriented program achieve the same level of "rigor" as a traditional research doctoral program or should the construct be redefined?Does "rigor" encompass a special set of skills that are linked with problem solving and field impact?Program "rigor" as defined within a practitioneroriented program affects matters of program status and value, faculty identity, student identity, student performance expectations and experiences, program content, and distribution of resources.
CPED held a discussion about this topic in its November 2012 UCEA meeting in Pittsburgh, offering the following observation Because the professional practice doctorate in education prepares graduates to engage in a type of work fundamentally different from a research-heavy PhD, CPED asserts that the standards for the two degrees should display strikingly different definitions of excellence and quality.Reframing the question of what constitutes rigor, based on a criterion unique to the EdD, rather than a standard borrowed from the PhD offers an important opportunity for the development and longevity of a true professional practice degree in education.To that end, perhaps we should be asking: What is the IMPACT of our programs?(Perry, 2012) Other discussants suggested that quality be defined as the degree of impact our programs have on the profession we serve.The challenge is, however, to reach consensus on how quality should be defined and how impact should be determined and measured in terms of the leadership performance of our graduates.Several dimensions of quality and rigor can be defined as core elements of "laboratories of practice" and authentic field-based settings (Shulman, 2005) in which theory and practice inform and enrich each other and allow students to address complex problems of policy and practice.

A Vignette: Research Reciprocity and Impact
The following vignette illustrates the process of interweaving professional and personal development through the curriculum, experiential activities, development experiences, and dissertation.This vignette reflects the development and dissertation work of the current author's recent doctoral graduate.
The choice.Angela H. had examined several doctoral programs that offered PhDs.However, she was concerned about a "traditional" program, in which she may have to fit into existing research orthodoxy and faculty agendas rather than forge her unique path.As a school psychologist, she believed that research should be conducted in collaboration with the "researched" and in real-world settings.She was inspired by the idea of embedding her research in the professional community of which she was a part and creating a context for shared inquiry.When she was interviewed for the program, she was asked, "How do you want to make a difference in the world?"She had never been asked this question and was intrigued by the idea of having the freedom to envision what had never been and then do whatever it took to make it happen.Faculty knew that Angela was self-motivated, inspired, and committed, and could be guided to translate her vision to practice.
The preparation.Angela was well prepared for the dissertation stage as a result of her coursework and related development experiences.Her program plan combined courses in special education leadership, systemic change and consultation, qualitative and quantitative research tools, legal issues and public policy, preparation for the professorship, and coursework that bridged neuroscience with child development and education.She had also participated in two field-based research internships under the direction of faculty leaders.Each student was required to examine a topic of interest related to a specific population of individuals with particular neurodevelopmental disorders requiring educational interventions.The requirements for the student's project portfolio included information related to prevalence and incidence data, assessment approach, social profile (e.g., a picture of the social conditions for the population in terms of access to education, technology, economic wellbeing and employment, postsecondary education, and independent living), an individual case example, overview of research, implications for educational research, and recommendations for further research.For example, one of the projects engaged her, along with other doctoral students, in collaborative exploration of the neuroscience of attachment disorders in adolescents.The team provided a history of reactive attachment disorder (RAD), diagnostic criteria, definitions, and behavioral manifestations and symptomatology.They distinguished among different types of RAD, closely examined the DSM-IV classifications, and critically examined underlying theories of RAD, particularly those of Bowlby (1983), Ainsworth (1978), andBlount-Mathews andHartenstein (2005).A compelling exemplar case study captured not only the behavioral manifestations, but also the impact on the family and on schools attended.They provided available prevalence and incidence data and explained the limits of research literature on the subject.Definitions of RAD by the American Psychiatric Association, World Health Organization, Centers for Disease Control, and Administration on Children, Youth and Families were presented.
The team presented a review of the social conditions for the population, addressing factors such as self-concept and mental health, success in employment and postsecondary education, independent living, and community participation.Outcome data on the population of children were compared with that of children without the disorder.A framework was provided for understanding RAD, including impacts of early care-giving and brain development; effects of trauma and abuse on self-regulation, communication, and cognition; and psychobiological relationships.The team discussed emerging efforts to understand the role that early negative experiences have on the neural system and brain structures, aspects of early brain development, stress responses, and neurobiology and neurotransmitters.They also addressed interacting neurobiological systems (sympathetic and parasympathetic systems, limbic system), genetic predispositions, and temperament.
The project closed with the framing of important research questions, with examples of emerging and promising research on the topic.A summary was provided on what is known about the effects of RAD on cognition and learning, the challenges to educating children and youth with the disorder, and identified interventions in practice with the population.
An important part of this kind of project is the practice of "assumption hunting" (Brandenburg, 2008), in which students examine the complexity of a problem as well as the beliefs and assumptions at its root.The grounding assumptions of contemporary interventions were critically analyzed in the light of new neuroscience knowledge (e.g., limitations in effectiveness of medications) and recommendations were made.Another important element of the project was the hunting of former theories of change associated with previous interventions into the problem.While the team concluded that the disorder is "entirely preventable," their course professor queried whether enough is known to date to be certain that attachment disruptions would not occur even in healthy caregiver relationships.
During this team project, Angela was challenged to think deeply about the kind of impact she wanted to prepare for and create.
Mentoring for impact.Angela's program included mentoring and guided processes in which students clarified their personal and professional identity, examined their motivations for their commitment to change, and reflected on the importance of culture and diversity in their decisions and actions.Angela spoke about her concern with the rising number of older children with math disabilities and attentional problems, and wanted to design an intervention that she could test.Doctoral programs that design for impact require intensive faculty time devoted to advising the student, guiding the study design, and coaching for the negotiation of the field-based project implementation.Angela viewed the dissertation as an opportunity to design and measure an intervention with a population of children for whom she and school professionals were concerned.She also hoped that a successful intervention could be replicated in the future with a larger population in her school district.She wanted to achieve a goal greater than that of individual personal and professional development-to in fact transform her commitment to relevance and impact in her work.She and her peer cohort had become aware that they had stepped outside the narrow band of advancing their selfinterests as scholars and had embraced the transformative scholars' doctrine: impact the world and work deeply in the community.In summary, Angela was oriented early to a critical interrogation of her beliefs as well as of current practices.She strengthened her ability to express her values and talents and thus to actualize her identity as a skilled professional.She reflected on the personal and professional transformations that contributed to her identity as a change agent and on the value-commitments that guided her actions.(Heishman, 2015).As a school psychologist in rural Pennsylvania, she was aware that children with learning disabilities and attention deficit hyperactivity disorder (ADHD) struggle daily and are at risk for poor long-term outcomes.She was interested in emerging evidence suggesting that working memory (WM) may improve by adaptive computerized working memory training.She enlisted 23 school-aged children (11 females), from the 5th grade to the 12th grade, who had both math difficulties and ADHD to participate in a quasi-experimental, repeatedmeasures study in a school to investigate transfer effects of working memory training (Cogmed RM) on math achievement, fluid reasoning, and memory and learning tasks.As part of a pilot, the Cogmed Progress Indicator (CPI) was used to measure transfer effects on working memory, following directions, and math challenge through Statistically significant improvement on the CPI was found on the following-directions tasks, and on indices measuring verbal memory, visual memory, verbal working memory, symbolic working memory, attention/concentration, working memory, general memory, and fluid reasoning 4 weeks post-intervention.Statistically significant differences were also found at the 4-month follow-up period in math fluency, applied problems, and math calculation.Working memory, inhibition, organization, and the Behavior Rating Index scales of the BRIEF were found to be statistically significant at the 4-month posttest, and ADHD showed significant improvement.Angela concluded that the results were very promising, though additional research was needed to address the limitations of the study.
Angela knew from the beginning that her research needed to be relevant to those in the research environment.In the initial stage of her research she identified shared interests with the school community and engaged multiple stakeholders in her research design and its implementation, including her participants and their families.She worked with teachers and school administrators to identify what they needed and expected from her research and learned of their interest in using the results of her study to improve instruction with students and to integrate the practices into the curriculum.She also asked parents and teachers what they believed needed to be explored and how they might contribute to the study.

Lessons about relevance and stakeholder perspectives.
Acting on the ethic of relevance and reciprocity, Angela considered how the different groups in the study would use the work.She explored cultural issues that might shape the perspectives on the problems of children with ADHD and math difficulties.She asked the parents, teachers, and administrators what they were interested in seeing as a result of their participation, and she provided a thorough report of processes and results.This report resulted in the development of additional professional development materials for teachers, and recommendations for wider use of the Cogmed instructional strategies.Angela worked continuously to adjust her study to ensure that she was exploring questions that were relevant and important to the community she was studying.In the process, she became an advocate for the student participants and their families.Collaboration and reciprocity meant viewing research as a relational endeavorasking questions that mattered to the community and engaging them in interpreting the meaning and the results.

Lessons about reciprocity in research.
The notion of reciprocity regarding the dissertation elicited provocative questions from Angela about who owns a research project.Her advisor reframed her more general questions as follows: if the research is contextual, engages multiple stakeholders, is dependent upon these relationships, and the interpretation is context-sensitive, then who ultimately owns the work (Kochhar-Bryant, 2016)?She concluded from the question that it is not a dichotomy between the researcher and the researched, but it is both-and.She also learned that the dissertation in practice that embraces reciprocity leads education researchers to a broader conceptualization of evidence, one that expands the transformative potential of collective work (Trainor & Bouchard, 2013).
Reflecting on barriers and threats in the process.Angela reflected on many challenges and barriers in the process of preparing for and negotiating the research project in an authentic setting.Echoing Schein (2004), she learned that for a culture to effectively change within an organization, the staff not only had to learn something new, but also had to "unlearn" something.Angela also realized that for transformative change to occur there needed to be "disconfirming data" that suggested the inadequacy of former strategies.It was important that she gathered and presented that data.Third, Angela learned that the school culture was perpetuated through a process of socialization that involved the transmission of values and information.It was, therefore, important to ensure that the senior staff received continued training to be able to communicate and reinforce these concepts effectively with new staff (Van Maanen & Schein, 1979).Angela came to understand that the perfect reform initiative does not exist and that the focus on reform distracts educational leaders from focusing on the school culture and the elements of that culture that must be understood and nurtured for the reform to succeed (Ross, 2010).
Fourth, Angela noted several factors that may have affected the outcomes of the behavioral checklists.One factor in particular that may have affected teachers' observations of students' behaviors may have been the excessive demands placed upon them that essentially led to a negative school climate.Teachers reported feeling overwhelmed during the school year in which the research study was conducted, particularly with changes in the teacher evaluation system whereby their students' performance on standardized assessments could affect their ratings.Additionally, teachers expressed concerns about pending changes taking place in the curriculum due to requirements to meet Common Core standards.Adding to these new demands, teachers were faced with changes in the assessment of their students, and they expressed some concern about students participating in computerized interventions during the resource period in the day instead of receiving additional homework support.These stressors on teachers may have produced negative attitudes affecting the ratings.In other words, bias against the participants' involvement in the computerized working-memory training program needed to be considered, particularly as the teachers were well aware of the training program.It is plausible to think they were resentful as they were further frustrated by a system that gave them less autonomy in how they structured their lessons and met the needs of their students.
A fifth serious threat to the successful implementation of the initiative was the inadequate time for planning, sharing information, and working through implementation problems.Finally, the shrinking pool of financial resources available for staff training and for implementation required creative thinking about sustaining the project through alternative funding.
Lessons about bridging knowledge across disciplines.Angela raised an important concern that many stakeholders in education are unaware of the knowledge from in the field of neuroscience explaining the learning processes involved in reading, writing, and mathematics.They could be at risk of employing instructional practices and using expensive materials that are not appropriately addressing the developmental needs of many of the students in the classroom.This is of particular concern when working with struggling learners who may have specific cognitive deficits that contribute to their inefficiency in learning.By not considering the neurodiversity that exists among learners, educators could clearly miss opportunities to make significant changes in young people's lives.One of the strengths of this research, as well as others on computerized working-memory training, is that it provides additional translational research through the examination of cognitive underpinnings linked with learning and tests whether specific targeted interventions may lead to improvements in neuropsychological processes and academic skills of struggling students.
Angela asserted that probably the most important group of stakeholders that needed to be aware of the research connecting the fields of neuroscience and education were teachers.One of the most powerful concepts that teachers should understand is that the brain is malleable, and not fixed as many educators were once trained to believe.Angela had reviewed the research on the plasticity of the brain and the discovery that reducing the negative effects of certain variables, such as poverty and stress, can improve specific cognitive processes that directly impact school performance.Therefore, she surmised, translating this knowledge to teachers may help them to feel more empowered about their crucial role in shaping young children's current and future neuropsychological development.

SUMMARY AND CONCLUSIONS: COMPARISON OF THE PHD AND PPD IN RIGOR AND RELEVANCE A Quantitative PhD Study on Working Memory
The following summary of a quantitative dissertation study by Southard (2014) is presented here as a comparison with the previous study in order to discuss distinctions between the criteria for rigor and the construct of relevance between the PhD and PPD programs.
The Southard quantitative study investigated the relationships among working memory, creativity (measured as divergent thinking and creative achievement), and nonverbal intelligence.The study also examined the roles of working memory and intelligence in the creative process.In order to examine this, participants were evaluat-ed using a variety of cognitive tasks that included the Alternative Uses Test, the Consequences Task, the Creative Achievement Questionnaire, the Alloway Working Memory Assessment, and the matrix test from the Wechsler Abbreviated Scale of Intelligence.
The sample for this study consisted of 166 students enrolled at the University of North Florida.Of the students in our sample 29 were Freshmen, 21 were Sophomores, 65 were Juniors, and 51 were Seniors.A large portion of our sample (42.77%) were between 18 and 20 years old, 30.72% were between 21 and 23 years old, 9.64% were ages 24-26, and 15.66% were 27 or older.The majority, 53.61%, of the participants' ethnicity was Caucasian; 18.07% were African American, 13.25% were Hispanic, 7.23% were Asian/Pacific Islander, and 7.83% identified their ethnicity as Other.The results of this study indicate that verbal working memory was related to divergent thinking over and beyond intelligence and creative achievement (Southard, 2014).

Standards for Rigor in the Scientific Process
Many academicians equate good (i.e., rigorous) research with a positivist perspective and recommend an emphasis on quantitative skills and methods (Torff, 2011), proposing a hierarchy of data collection and analysis techniques, some more "rigorous" than others.For example, doctoral programs funded by the Institute for Educational Sciences (2008) to increase the supply of researchers in education give preference to rigorous education research preparation for the conduct of efficacy and scale-up trials.IES refers to the "methodological superiority of randomized trials for drawing causal claims in areas in which outcomes are affected by many variables" (pg.10).The National Academies of Science (2002) concludes that nonrandomized studies are weaker in their ability to establish causation than are randomized field trials (National Research Council, 2002).
The practice of scientific research-whether quantitative, qualitative, or mixed method-demands that the researcher's work be intellectually rigorous, accurate on all points, ethical, and dispassionately executed (Bradford, 2015).Rigor in quantitative research is judged by how narrow, concise, and objective the design and analysis techniques are and how strictly the rules have been adhered to and applied to all decisions (IES, 2008).The steps involved in the scientific process vary slightly depending upon the field; the following is an outline of the general approach: • Make an observation or observations.
• Ask questions about the observations and gather information.
• Form a hypothesis-a tentative description of what's been observed and predictions based on that observation.

•
Test the hypothesis and predictions in an experiment that can be reproduced.
• Analyze the data and draw conclusions; accept or reject the hypothesis or modify hypothesis as necessary.
• Reproduce the experiment until there are no discrepancies between observations and theory (Bradford, 2015;Kantrowitz, 2014).
However, there are differences of opinion about what the preparation should look like for doctoral students.Kantrowitz (2014) argues that researchers need to be cautious about making claims that some data collection or analysis techniques are "more rigorous" than others, and furthermore, that researchers should stop associating standards and rigor only with confirmatory and hypothesis-driven research.She also advises caution when claiming that fixed standards are needed for particular methodological techniques.Methodological techniques share a common set of properties but their power lies in the ability to adapt a specific research technique to different research contexts (Kantrowitz, 2014).Standards of rigor can be established for exploratory and descriptive research.

Rigor and the Problem of Practice Dissertation
Problems of practice-of implementation and translation of research findings to practice-have been referred to as "wicked problems" (Fixsen et al., 2009).They represent complex problems that occur in implementation conditions ("contested spaces") that are in flux and that engage multiple layers of infrastructure of a system where various stakeholders can either support or obstruct transformation (Gutiérrez & Vossoughi, 2010;Penuel & Spillane, 2014).Such conditions require a set of skills in the researcher that enables him or her to engage in a process with diverse stakeholders to define the problem of focus as being significant, persistent, and worthy of investigation.A reconceptualization of rigor for studies in these conditions requires sustained, direct, and systematic documentation of what takes place inside programs to document how systems being studied change and adapt interventions in interaction with each other in relation to their dynamic local contexts.
Scientific rigor associated with qualitative research (nonexperimental) is defined as being open to the data, scrupulously adhering to a specific philosophical perspective, and thoroughness in collecting data (Guba, 1981).It is also judged by the logic of the emerging theory and whether the findings are contributing to what is already known about the phenomenon under study.Criteria for rigor and trustworthiness of the data included the following:

•
Truth value / credibility of the study: degree to which the researcher has established confidence in the truth of the findings for the subjects and the context in which the study was undertaken

•
Applicability: degree to which the findings can be applied to other contexts, settings, or groups • Consistency: extent to which the findings would be consistent if inquiry were replicated with the same subjects or similar context • Neutrality: degree to which the findings are a function of the subjects/informats and condition of the research and not of other biases, motivations, and perspectives (Lincoln & Guba,1986) Ensuring a standard of rigor within a problem of practice dissertation requires that the program provide candidates with the following: 1.An opportunity to envision solutions to significant problems within the symbiotic space between the university and the stakeholder community 2. Challenging opportunities to express one's values and talents, and thus to actualize one's identity as a skilled professional 3. Opportunities to define and reflect on personal and professional transformations that contribute to one's identity as a change agent, and commitment to a larger community impact 4. An opportunity to construct a theory of change that is grounded in theory and understanding of the history of previous interventions into the problem in the past; 5. An opportunity to develop a sense of mastering a challenging situation and, through this, to feel a sense of responsibility for future outcomes 6.The challenge of exercising leadership for social change through professional and applied research activities that result in appreciable improvements on processes, policies, or social conditions within the community, particularly for vulnerable populations 7. Opportunities to use data to understand the effects of innovation, and ability to gather, organize, judge, and analyze situations, literature, and data through a critical lens (Shulman, 2005) 8.The shaping of a perspective that views cultural and linguistic diversity as an asset 9. Mentoring that engenders a sense of deep commitment to a research problem that is compelling for the individual and for the community or organization in which it occurs The dissertation, therefore, is viewed as having the aim of achieving a goal that is much greater than the individual's personal and professional development.Rather, it becomes an instrument for transformative development within the research context, and for the shaping of both a personal and a shared identity for change and a commitment to making a difference or effecting a change.
The shared, negotiated process becomes an enriching aspect of the dissertation for the researcher and his or her relationship with the researched.Departing from the model of 'get the data and run, the researcher develops a reciprocal relationship (Fine, Weis, Weseen, & Wong, 2003;Trainor & Bouchard, 2013;Zigo, 2001), ensuring that the subjects benefit from the study.

Closing
With several practitioner-oriented doctoral programs in embryonic stages, along with the concurrent implementation of a new PhD, our faculty are navigating the tensions of past practices and behavioral and value sets, and the demands of a new conceptual frame.They are faced with disquieting issues related to the balance of resources among programs, equity in the balance of faculty load, challenges to program traditions, threats to faculty and student identities, and dynamic standards of rigor.High-impact projects are rigorous, powerful, inspiring, bold, and often dramatic.Deliberate and sustained efforts within and outside the academic unit to promote understanding and positive communication about PPD contributions and expectations for candidates can add abundant reputational value to the institution.

•
Broad and advanced knowledge within the discipline • Successful use of a range of methodologies of the discipline • Independent performance of original or applied research • Effective communication, written and oral • Performance as a professional in the discipline Finally, she learned what it meant to construct a theory of change that was grounded in an understanding of the history of previous interventions into the problem of interest.The dissertation: Rigor and relevance.Angela's dissertation is titled the Effectiveness of Computerized Working Memory Training on Math Achievement and Other Transfer Effects in Children with ADHD and Math Difficulties out the training.Cogmed Working Memory Training is a softwarebased program reported to improve working memory and designed for children and adolescents.It combines cognitive neuroscience with computer game design and professional support for teachers.Standardized instruments were administered at baseline and at 4 weeks and 4 months post-interventions.Teachers and students completed the Conners-3 to assess ADHD and teachers completed the Behavior Rating Inventory of Executive Function (BRIEF) to measure executive functioning.The Cogmed Coach discussed with parents and students the concept of working memory, how it impacts academics and functioning in daily life, and how Cogmed working training works.
argued that rigor in studies that aimed to draw causal inferences about policies, programs, and practices required in-depth qualitative research-an understanding of the Why, How, and Under what conditions programs and policies work, including the following: