Distance Education as a Pilot Activity

The EdTech Center@World Education encourages states and programs to consider their first attempts at implementing distance learning programs for adult education students as a pilot activity. Pilot activities are experimental in nature and allow an organization the opportunity to explore a new approach on a small scale. They leave room for trial and error and encourage people to move in new directions. Pilot activities are distinct from the more established programs offered by an organization; although if they succeed, they may become incorporated into the organization’s regular course offerings. As noted earlier in this Handbook, teaching at a distance is dramatically different from teaching in a classroom. Organizations will need to experiment to learn what works best for their staff and students.

Typically, the EdTech Center recommends that organizations start with a small pilot. However, this has most likely not been feasible for many organizations that needed to develop or upscale distance education programs because of the pandemic. Although starting off with one or two teachers and classes may not be currently feasible, it is still possible to utilize approaches similar to those used with a smaller pilot.

It is recommended that administrators follow these pilot principles, regardless of whether you are developing, upscaling, or refining a distance education program, we recommend that you follow these pilot principles:

Spend time planning all of the components of your distance education program.

Even if you are already offering remote classes , it is still critical to evaluate each major component of the distance education program to determine what, if any, changes need to be made. IDEAL Consortium resources such as the IDEAL 101 course, the Distance Learning Site Plan, and additional online resources can help organizations consider how they will recruit, assess readiness, and orient learners; prepare teachers; provide instruction; assess learners; and evaluate the success of the program.

Expect uncertainty and changes throughout the pilot.

When piloting distance education, there is bound to be uncertainty from both teachers and learners. Administrators can support teachers by acknowledging the uncertainty that exists when trying something new and understanding if first attempts do not match expectations. Teachers’ apprehension or reluctance may be reduced if they know their administrator does not expect them to become instant experts in the technology, curriculum, and delivery method. In turn, teachers can be transparent with learners that they are both learning something new while piloting distance education, which may support students’ willingness to participate and persevere.

Determine your measures of success, which includes student, teacher, and program outcomes.

Before a pilot begins, you should determine what success looks like. Consider defining success as learning more about the approaches that worked, as well as those that are problematic. While positive student outcomes, such as learning gains and goals met as measured by the National Reporting System (NRS), are important for adult education, there are other measures related to teaching and learning that can be considered during a pilot. These might include recruitment of new students from a different population, improvement in student digital literacy skills, increased student persistence, increased teachers’ confidence in using technology for instruction, new  instructional materials, and the development of new partnerships. Administrators will want to work with their staff to determine what measures beyond NRS outcomes you will use to measure success.

Create an environment that encourages experimentation.

Pilot programs are most effective if the participants—that is, the organizations, administrators, teachers, and learners implementing them—perceive themselves to be innovators and experimenters. To do this, participants must be willing to try new approaches, take risks, and think creatively. For many educators, this involves developing a new mindset and acting outside the established norms of the field, which can be challenging in today’s accountability-driven climate. The administrator at each organization, in conjunction with people at the state and federal levels, must create an environment in which distance educators are comfortable with the risk-taking and creative thinking that accompanies all innovations.

Administrators may need to remind participants, over and over again, of the experimental nature of the project. This is a novel idea for many participants, and it may take time for them to accept the message. For example, it took three to four months before Pennsylvania pilot sites were willing to share with others the problems they experienced and the approaches that did not work. It took time for experimenters to fully grasp that the focus was on accumulating knowledge and that their efforts to try new things were among the most highly valued components of the project. 

Identify what works and what does not work.

Stress that the goal of the project is to accumulate knowledge about both what does work and what does not work. Help participants understand that in pilot projects, we learn as much from apparent failures as we do from apparent successes. Encourage participants to try new and creative ideas rather than limiting themselves to strategies they already know.

Be aware of the approaches you choose and the rationale behind those decisions. Understanding the antecedents of success is critical to replicating that success (Reeve, 2016). As you work to implement your distance education program, be sure to build in ways for staff to reflect on what is working and what is not, as well as the related reasons why. Teacher reflection logs, supervisor check-ins, and professional learning communities are all ways that can help you identify what is working and what needs to be revised or abandoned. Some organizations have found that regular team meetings focused on successes and challenges were important to identify best practices and areas that needed either more attention or a different approach. Having such meetings will allow organizations to use a systematic approach to maintaining and expanding their programs in a more efficient and effective way.

Pilot activities should help organizations determine both whether distance education is a viable option for targeted learners and, if so, how organizations can best facilitate their distance education program. Distance education may work better for some organizations than for others, just as distance learning is better suited for some learners than for others.

Setting a Vision and Developing a Plan

Setting a vision and developing a realistic plan with timelines for the distance education program can create buy-in from staff as well as alleviate concerns that may arise when trying something new. Many administrators have found it helpful to develop the distance education vision and plan with a team of stakeholders. This ensures that multiple perspectives are considered and empowers program staff in the planning process. Some possible questions to consider when setting a vision and developing a plan include:

• What is the vision behind adding a distance education program? Do you want to reach new learners, increase the intensity of instruction by offering a blended model, improve students’ technology skills, prevent students from dropping out from the program when they can no longer attend face-to-face instruction, improve outcomes, or a combination of these areas?
• Since this is a new initiative, what are the goals for the pilot? What are the expectations at the end of the pilot?
• How can the distance education program build on your organization’s strengths? How can it support achieving funders’ expectations, such as meeting the Workforce Innovation and Opportunities Act (WIOA) performance standards?
• How can leadership for the distance education program be shared?
• What is a reasonable timeline for implementing the distance education program?

Organizing the Distance Education Program

Each state and/or organization must choose the distance learning model(s) (as described in Chapter 1, “Setting the Stage”), instructional materials, and technology to deliver distance education that will best meet the needs of its learners.

Distance Learning Model(s)

The amount of face-to-face interaction that is required for distance education programs may vary widely. Some organizations choose to offer a blended distance education program, where face-to-face instruction and online learning are combined to increase the intensity of instruction for students. Other distance programs have the majority of the instruction occurring online using asynchronous tools like online curriculum and activities and/or synchronous instruction using webinar or video chat software. Some distance programs have open enrollment, where learners can start on any day, while other programs have found that some form of managed enrollment, where a group of students all begin distance learning together, creates efficiencies. HyFlex classes provide flexibility for the learner to choose between in-person instruction, joining the class remotely, or doing online asynchronous work. The format of your program will depend on the learners and their goals and the vision and goals for your program. The EdTech Center can provide technical assistance to help states and/or organizations explore what distance education model best fits programs’ and learners’ needs.

Instructional Materials

A second decision involves selecting instructional materials. Many organizations choose to use a publisher-developed curriculum as their core instructional resource when first beginning a distance program. Teachers can then identify or create supplemental activities to fill in gaps and further address skills. Your state will provide guidance on what curricular options are available for teaching at a distance. As noted earlier, in order to count distance learners in the NRS, states must submit a Distance Learning Policy to OCTAE. The acceptable curricula for distance learning must be specified in the state policy, if a program is planning on collecting proxy contact hours. However, during the pandemic, OCTAE has granted flexibility for states to use products for proxy contact hours even if they do not have a policy in place.

States may allow currently funded Adult Education and Family Literacy Act (AEFLA) grantees to administer distance learning programs, even if the state does not have an established distance learning policy in place. However, OCTAE urges states to put a policy in place (or change current policy) as soon as feasible. Statewide distance learning policies help address how the state intends to collect instructional hours (if it chooses to do so), as well as convey its policies on student assessment. State policies may also convey important information about distance learning curricula that local programs can use to provide distance education. See Program Memorandum OCTAE 20-3.

Digital Literacy, Access, and Equity 

A third consideration involves exploring the ways technology can support the expansion of services and what type of technology to use. Technology can be used to reach more learners as well as to motivate them, provide greater instructional flexibility, and increase resources for teaching and learning. Programs should consider what technology students have access to and what technology skills they need for their future employment and postsecondary education goals. For example, smartphone ownership is becoming more prevalent for all demographics (Pew Research Center, 2019). Slightly over two-thirds of adults with less than a high school diploma or who make less than $30,000 own a smartphone (Pew Research Center, 2019).

Some students may go online only using a smartphone or tablet because they do not own a computer.

U.S. adults with less than a high school diploma are most likely out of all educational attainment levels to indicate that they own a smartphone but don’t have access to a high-speed internet connection at home (Anderson, 2019). The COVID-19 pandemic amplified the need to address the lack of devices and high-speed internet at home that many of our learners face. Adult education organizations found creative ways to address this digital divide by partnering with K-12 school districts, libraries, government entities, and grant-making organizations. There are also programs, such as EveryoneOn and the Affordability Connectivity Program, that provide free or low-cost access to computers and high-speed Internet for adult learners.

One of the goals of distance education is to provide easier access, so the technology selected should not itself become a barrier. Some programs have also looked at non-tech ways to provide instruction to students, such as textbook drop-off locations throughout the community or mailing resources to students. There are also some instructional applications, such as Cell-Ed, that work on any mobile device and do not require a smartphone. These low- or no-tech methods make educational services more equitably accessible.

Once you select a model and decide on your instructional materials and technology, you will start focusing on planning in the five areas discussed throughout this Handbook: (1) recruitment, (2) assessing readiness, (3) orientation for learners, (4) teaching at a distance, and (5) assessment of distance learners. In the final activity in the EdTech Center’s online course, IDEAL 101, each organization completes a Distance Education and Blended Learning Site Plan for its distance or blended learning pilot. We strongly recommend that a staff team, composed of the administrator and two teachers, complete it. Developing this plan as a team has several advantages:

• All parties involved in the distance education pilot programs have the opportunity to participate in the design and development of the experimental program. This not only brings a broader range of expertise to bear on program development, it also helps all participants feel a sense of ownership for the pilot.
• Team planning provides administrators with a fuller understanding of what the teachers in their organizations will be doing and the types of supports they will need.
• Developing the plan as a team helps create a cohesive, experimental mindset.

Budgeting

Administrators adding a distance education component need to consider costs related to the instructional approach, instructional materials, communication tools, and staffing. Being strategic in your selection is important. Administrators will also want to ensure that they allow enough staff time for planning, professional development, teaching, and reflecting on the pilot activities. Staff will need time to learn new technology, become familiar with the curriculum, and organize instructional materials for students. Distance teaching time may not equate to teaching time in the face-to-face classroom. Distance teaching can require more time for communication, instruction, and progress monitoring, all of which may need to occur individually with learners or outside of synchronous instruction.

Some questions to consider: What are the factors that determine the format of the delivery service? When might an organization use a safe socially distanced activity versus a group or individual online activity? How can technology be used to increase organizational efficiencies in communication, instruction, and program management? When might free open educational resources (OERs) be used, or when is a purchased product necessary? Are there ways to more efficiently replicate and scale your program, such as creating an online course template that all teachers use so they do not need to spend time working on formatting a course in a learning management system?

Identifying and Supporting Teachers

Identifying Teachers

Distance teaching requires different skills from classroom teaching skills. (See Appendices C and D for resources to measure teacher readiness.) Some excellent classroom teachers make the transition well, while others are not comfortable in this new environment. Successful distance teachers are innovative, creative, and flexible. They are open to new experiences, willing to explore multiple pathways to reach an endpoint, and able to try new strategies for meeting students’ needs. Successful distance teachers need to be technologically adept, knowledgeable about the curriculum, and able to establish rapport with their students at a distance. It also helps if teachers are excited about the opportunity for professional growth and about what distance learning can offer their students. 

Thus, just as distance learning is not for every student, distance teaching is not for every teacher. Whenever possible, you should ask teachers to volunteer or to try distance or blended teaching; this increases the likelihood that the teachers will bring the constellation of characteristics described above. A teacher with no distance experience and little interest in innovative educational practices is not likely to be successful. Because distance and classroom teaching are so different, distance teachers need additional training and openness to new educational approaches if they are to be successful.

Some organizations have found ways to creatively leverage their staff’s strengths during the pandemic. For example, teachers who may not have felt comfortable teaching online focused on reaching out to students over the phone or texting to maintain open lines of communication. Team teaching allows teachers with less digital literacy skills to learn from more experienced teachers. More experienced teachers may design online lessons that other teachers use for teaching.

Several administrators have also pointed out the need to consider digital literacy skills when hiring teachers, since all teachers may need to deliver some type of online instruction. IDEAL Consortium states are collaboratively looking at frameworks that can be used to evaluate staff’s digital literacy skills to identify opportunities for professional development and growth.

Supporting Teachers

Administrators need to understand and be prepared to support the additional responsibilities that teachers will assume as well as prepare teachers for the new roles they will fill when teaching at a distance. Data from studies in several states indicate that, at the start of a distance education pilot program, only about half of teachers’ time is spent actually teaching; the other half is devoted to the activities necessary to recruit learners, develop partnerships with other organizations, orient new distance students, and plan for new ways of interacting with and motivating learners.

Many of these activities, particularly recruitment, are not typically a part of a classroom teacher’s job, but they tend to fall to the distance teachers in pilot programs.

In addition, teachers in pilot programs assume a dual role: they are teachers, but they are also researchers collecting data about the pilot program. Teachers will often need to complete forms, keep records, and collect data to provide insight into program implementation and effectiveness.

Teachers need to understand the reasons for the data collection, feel confident using the data collection tools, and appreciate the importance of their role as experimenters. This data collection can be time-consuming and needs to be figured into teachers’ time allocations. If both teachers and administrators are aware of these additional roles, it will help all participants appreciate the time demands the program places on staff.

It is also important that teachers be knowledgeable about the technology needed to teach at a distance or in blended learning classes. Because many distance programs have an online or computer-based component, distance teachers need to be technologically savvy. They must not only understand how to use the delivery modality of their curriculum, but also be able to act as a technology support person to help students resolve their technical problems. Recognizing this need, you may wish to survey teacher technology competencies and organization technology capabilities as part of the selection process for pilot sites. (See Appendix D.)

Professional Development for Teachers

Good teaching is at the heart of effective distance education for adult learners, and distance teachers need a variety of support mechanisms as they make the transition from classroom teaching to distance. Providing teachers with professional development, recognition for their efforts, financial compensation, and the opportunity to interact with peers teaching at a distance are among the many ways organizations can make this transition easier for teachers. IDEAL Consortium states recommend that you do the following:

• Provide professional development opportunities for teachers preparing to teach at a distance. This Handbook provides an introduction to the main concerns and is a good starting point, particularly when used with IDEAL 101. Some states have developed their own training protocols for distance education, and commercial resources are available as well. See the EdTech Center website for professional development opportunities, such as webinars on blended and mobile learning. Regardless of the training approach and tools utions-of-distance-educationsed, teachers will need additional training if they are to be as effective at teaching at a distance as they are in the classroom.
• Provide mentoring groups in which experienced distance teachers can support and guide new teachers. This provides an opportunity for teachers to work together to address challenges and creates an environment that encourages professional growth. Texas and California have extensive, formally organized mentoring programs for their distance educators. Teachers learn from the experiences of their colleagues and become part of an active community of practice. For example, the Outreach and Technical Assistance Network in California runs a Digital Leadership Academy that brings groups of teachers together and matches them with a coach.
• Recognize that making the change from classroom teaching to distance teaching is a major transition for teachers. Create an institutional climate that supports them in making this transition. Provide supports, such as conference calls, online chats, and websites, for teachers where they can ask questions to help them think through the many issues they will encounter.
• Understand that to teach effectively, teachers must be intimately familiar with the instructional resources. Because distance education programs may be individualized, students can enter the program at any number of points. Thus, the teacher cannot simply stay “one day ahead” of the class and be able to meet the students’ needs. Provide curriculum training and planning time for teachers.
• Provide financial compensation and/or release time from other duties for teachers working with experimental distance education programs. Consider providing flexible working hours for distance teachers and compensation for the nontraditional hours they are likely to work. It is unreasonable to expect teachers to assume a task of this magnitude during the normal working day or on top of a full workload and be able to flourish as distance teachers.

Monitoring Achievement and Evaluating the Pilot Process

In distance education and blended learning pilot programs, data play a critical role. While data regarding enrollment, hours of instruction, and outcomes may not be the primary focus of the distance education pilot, they are still important measures to track. This quantitative data along with the qualitative reflections of the pilot staff can be useful for monitoring achievement and evaluating what worked and what can be improved.

Accountability 

In an ideal situation, states would release organizations from their customary accountability requirements for the first phase of any new pilot program. We believe that distance education for adult learners is so different from traditional classroom programs that it is equivalent to “reinventing the school.” It requires that organizations look for different students and find new ways to teach and interact with them. It clearly takes an extended effort as well as a period of trial and error to determine best practices (Askov, Johnston, Petty, & Young, 2003, p. 31).

For example, in some states, such as Pennsylvania, certain pilot sites were exempted from some of their usual accountability requirements to encourage experimentation. Sites were required to provide a count of the number of students their Workplace Essential Skills distance education programs served, but they did not need to provide evidence of educational gains or progress. This was important for several reasons. It further reinforced the pilot program’s experimental nature, encouraged sites to actively try new approaches, and allowed both the sites and the state a longer period of time to deal with the unique set of issues related to assessing distance education students.

Other factors may also affect accountability. For example, the U.S. Department of Education Office of Career, Technical, and Adult Education (OCTAE) determined it will not make any determinations of performance success or failure based on 2019 program year performance data because of the wide, sweeping impacts of COVID-19 on adult education services (Stump, 2020).

Data monitoring 

Regardless of how the accountability of distance education pilot programs is measured, data monitoring is a key component of the pilot. Organizations and states will want to determine what data will be collected and how often it will be reviewed. For example, Arizona Department of Education staff met with pilot programs twice a year to review student and program data as well as to discuss successes and challenges of the pilots.

Administrators can work with the pilot team to determine how distance learners will be assessed. Administrators need to ensure that their organization’s assessment plans are aligned with those set out in the state distance learning policy. They will need to work closely with both state- and organization-level data staff to make sure that the appropriate information about distance learners can be captured in the data systems. Administrators will also need to train teachers about the assessment and data reporting policies and requirements.

The pilot team can also determine if any other data might be helpful. Some organizations have had distance students participate in focus groups or complete surveys to provide additional feedback about the program. The Virginia Adult Learning Resource Center has created an Adult Education Indicators of Quality Online Courses rubric that could be used to identify strengths and areas of improvement.

A Note about Data Security and Confidentiality

It is important to maintain confidentiality and data security practices with distance education programs. Whether staff are working remotely from their home or in the office, it is important for administrations to ensure clear expectations and procedures are put in place to secure students’ personal information. If staff are working from home, it is also important to protect their personal information. For example, staff could set up a Google Voice phone number for students to use so staff do not have to give out their personal cell phone numbers.

Moving Beyond the Pilot

Pilot programs have a limited life span and at some point are likely to be replaced by a larger scale version. Although the growth of the distance education program clearly depends on state policies and support, the local organizations are where the changes are typically implemented. At the local level, the goal is to provide distance learning as one of the available options for adult learners. A good place to begin is to create organization-wide awareness of the distance education program and how it can serve students. Many organizations find that it is helpful to combine the recruitment, determining readiness, assessment, and orientation of distance students with those same functions for classroom students. This not only reduces the demands on distance teachers, but also serves to legitimize distance learning within the organization. Some examples of how organizations have integrated distance learning into organization-wide activities and services are:

• Including distance learning as an available option on all recruiting materials, such as websites, brochures, and fliers
• Training intake staff to identify students for whom distance learning might be a good fit 
• Supporting the professional development of teachers interested in distance education

Changes in the delivery of education are not going to be easy or swift. A popular misconception about distance education is that it can be implemented with little change in the way an organization is structured, the way teachers teach, or the way learners learn (Moore, 1993). Research on K–12 curriculum innovations, for example, suggests that, even with all the right conditions in place, it may take three to four years for teachers to adopt, adapt, and reinvent how they teach (Askov et al., 2003; Hall & Hord, 1987). Therefore, adding distance education to an organization’s spectrum of services should be viewed as an “organizational change” effort.

First and foremost, if you are considering adding distance education as a delivery mode, you must base your decision on the educational principles and issues that form the foundation of your organization. Thus, you will ensure that your decision is rooted in the mission of the organization and, therefore, will help make its addition to the organization smoother and more likely to succeed.

Experience in the IDEAL Consortium states suggests that the of the following approaches may be useful to organizations moving from an experimental to a programmatic implementation of distance education:

• Capture the lessons learned during the pilot phase and use them as a basis for future planning. Keep the practices that worked well and drop or modify those that did not. (See Appendix E for a detailed description of how to use webinars to reflect on different phases of the pilot.)
• Write down how the procedures have evolved and the rationale behind the decisions to make changes. This helps to formalize the process and ensures that all participants have a shared understanding of the organization’s approach to distance education.
• Create an action plan with strategies to help participants move from the idea stage to the implementation stage.
• Write job descriptions for the key players. This may include teachers, organization administrators, technical support people, recruiters, and others involved in making the organization’s distance project a reality. Keep in mind that the nature of distance education may require some flexibility in job roles and assignments.
• Get involved with people at the state level interested in distance education and make policy recommendations based on participants’ experiences.

Connecting Distance Education with Workforce Innovation and Opportunity Act (WIOA) Outcomes

The Workforce Innovation and Opportunity Act (WIOA) describes the performance outcomes for adult education organizations that receive funding through this federal legislation. All workforce development and adult education partners funded through WIOA share the same performance outcomes: job attainment, job retainment, increase in average earnings, secondary school and postsecondary credentials attainment, measurable skill gain, and effectiveness in serving employers. Here are some of the ways that distance education can help organizations meet these performance outcomes.

• Increasing student persistence and preventing student stop-out
• Increasing skill attainment necessary for work and postsecondary education
• Modeling and building digital literacy and independent, lifelong learning skills and mindsets needed for the workplace and postsecondary education
• Incorporating academic skills with a training program to offer an Integrated Education and Training (IET) model
• Customizing instruction to provide sector-specific activities that prepare students for the workplace

Support for Distance Education and Blended Learning

The EdTech Center is available to provide support to you and your program staff as you pilot distance education and work to integrate it into your program services. A remote instruction observation tool is available for administrators and teachers. The EdTech Center blog also includes useful articles and resources.

Administrative Considerations and Strategies

Note that in the course, IDEAL 101: Foundations of Distance Education and Blended Learning, this prompt is expanded into fully developed collaborative activities for your team to complete together.

References

Anderson, M. (2019, June 13). Mobile technology and home broadband 2019. Washington, DC: Pew Research Center. : https://edtechbooks.org/-Ayo 

Askov, E., Johnston, J., Petty, L., & Young, S. (2003). Expanding access to adult literacy with online distance education. Cambridge, MA: National Center for the Study of Adult Learning and Literacy. https://edtechbooks.org/-qCSw 

Hall, G. E., & Hord, S. M. (1987). Change in schools: Facilitating the process. Albany, NY: SUNY Press.

Horrigan, J. B., & Duggan, M. (2015, December 21). Home broadband 2015. Washington, DC: Pew Research Center. https://edtechbooks.org/-UNZg 

Moore, M. G. (1993). Editorial: Is teaching like flying? A total systems view of distance education. The American Journal of Distance Education, 7(1), 1–10.

Moore, M. G. (2002). Editorial: Learning the necessary principles. The American Journal of Distance Education, 16(3), 129–130.

Pew Research Center. (2019, June 12). Mobile fact sheet. Pew Research Center. 

https://edtechbooks.org/-PHtc 

Reeve, D. B. (2006). The results paradox in the learning leader. Alexandria, VA: Association for Supervision and Curriculum Development.

Stump, S. (2020, March 27). Adult Education and Family Literacy Act and COVID-19–Frequently asked questions. U.S. Department of Education. https://edtechbooks.org/-BRDm 

U.S. Department of Education, Office of Career, Technical, and Adult Education. (2020) Policy Guidance and Policy Memoranda. https://edtechbooks.org/-GqYI