log inhelp

Get your own free workspace
View
 

Draft Performance Analysis

Page history last edited by PBworks 4 years, 7 months ago

  

Final Performance_Analysis_090907.doc  

 

 

 

Thanks team for all your suggestion, comment and correction. I will send an email to Brenda too. Rahel

 

 

 

I noticed a couple of typos and included my edits in the document that Kim A. just emailed to everyone.  I also added an introductory sentence to the Potential Solutions section for flow.  Rahel -- I know this took a lot of time to pull together.  Thanks for doing such a great job!  ktm

 

Performance_Analysis_090907_KM.doc

 

 

 

Hi Team, please review the final PA and give me any comment by 7:00 pm. you can send me email to rahelya@yaho.com  or make the editing on the document using MS review and attach it.  In addition, I will be on skype after 7:00. Thanks Rahel

edit_PA_intros_DriveBarr_ActOpt.doc

 

The introductory paragraphs that were added to the "Drivers and Barriers" section and the introductory paragraphs added to the "Actuals and Optimals" section are good additions to the performance analysis. Thanks to whoever put that together!

 

I edited the introductory paragraphs in these two sections, however, to address a few problems concerning the citation of a quote versus the use of a paraphrase as well as the clarity of the writing. I think it is better to use a paraphrase for the drivers and barriers explanation, because the information in Rosset's glossary is poorly edited, garbled, and/or not writtten in complete sentences.

 

For the actuals and optimals, I think it's best not to present info from Rosset's glossary as an indented quote, because, it is actually content pulled together from two different entries in the glossary, not a continuous passage of text.

 

I also think it makes more sense to use bullets to indicate the three focus areas, because each is so complex that when you combine them into a sentence, they kind of blend one into the other.

 

See the attached Word doc (above) showing my reworking of the introductory paragraphs discussed above. Please make these changes if you think they make sense.

 

Thanks. Good work, team. Matthew

 

 

Performance_Analysis_090907.doc    

 

 

 

 

 

 

 

 

 

  1. Here is a copy of the 2005-2006 Immersion Team's PA (ktm): performance_analysis_revised_fall2005.doc

 

 

 

Background and Introduction

 

The Virginia Department of Education’s (VDOE) Technical/Training Assistance Center’s (T/TAC) mission is “to improve educational opportunities and contribute to the success of children and youth with disabilities (birth - 22 years).” (http://www.ttaconline.org)

 

T/TAC has eight regional centers across the state of Virginia that provide training and technical assistance to school personnel in support of the nearly 175,000 students served under the Individuals with Disabilities Education Act, Part B, as of the 2005-06 school year (Digest of Education Statistics 2006, p. 83).  Assistance ranges from consultations with school teams to a lending library of multimedia, resources, and technology.

 

According to the United States Bureau of Labor Statistics, as of May 2006, employment in education-related occupations in Virginia included: 

  • Special Education Teachers     16,100
  • Education Administrators        10,290
  • General Education Teachers    97,350

       (U.S. BLS)

 

To reach this large population of geographically dispersed educators, T/TAC developed a website called T/TAC Online (http://www.ttaconline.org/).  It provides:

·         Information on Virginia Alternative and Alternate Assessments Program (VAAP)

·         Differentiated lesson plans for the Standards of Learning exam (SOL)

·         Resources including links to websites that provide additional information

·         Links to upcoming events

·         Training through webshops

 

T/TAC Online is available 24/7 for teachers, other professionals working with students with disabilities, and the parents of children with disabilities.  It is meant to be a “one-stop-shopping” source where educators can quickly find the information that they need to serve their students.

 

Since the fall of 2000, T/TAC and the George Mason University Instructional Design and Development Immersion Program have collaborated on various projects in support of T/TAC and T/TAC Online.  Immersion projects have ranged from the development of T/TAC Online prototypes to the development of specific resources posted on the T/TAC Online website in support of T/TAC professionals and teachers.

 

The George Mason University Instructional Design and Development Immersion Program in the fall semester of 2007 is conducting a performance analysis and needs analysis to evaluate the opportunities for supporting the staff and stakeholders of T/TAC. 

 

This document is the performance analysis.  It includes information gathered from interviews with various T/TAC staff members as listed in the "Identification of Information Sources" section.  A description of findings based on the interviews is included.  Recommendations derived from an analysis of opportunities as a result of the findings are also included.

 

 

Clients and Stakeholders

 

Dr. Patricia Abrams—Associate Director, Special Education, VDOE

Dr. Michael Behrmann—Professor, Director of Helen A. Kellar Institute for Human disAbilities (KIHd), George Mason University

T/TAC Personnel

Special education teachers and general education teachers

School administrators, principals

Parents

Students

 

 

 

Data Collection and Information Sources

 

 

I. Identification of Information Sources:
 

Human Sources:

  • Dr. Michael Behrmann -- Professor, Director of the Helen A. Kellar Institute for Human disAbilities, George Mason University
  • Dr. Shuangbao Wang- Research faculty at Kellar Institute, George Mason University.
  • Dr. Amie Fulcher – Director of Academic Operations for the College of Education and Human Resources and Technical Support Specialist of Helen A. Kellar Institute for Human disAbilities.
  • Mary Wilds - Statewide Coordinator for Distance Education, Old Dominion University
  • Nancy Andersen -- Special Education Policies and Procedures Coordinator
  • Clare Talbert -- T/TAC Online Coordinator for the Virginia Department of Education (VDOE)
  • Soojin Jang -- Assistive Technology Coordinator
  • Seunghun Ok -- T/TAC Online Manager
  • Dr. Harolynn (Lynn) Wiley-- Director, VDOE Training & Technical Assistance Center (T/TAC, Region 4), Kellar Institute for Human disAbilities
  • Margaret (Kay) Klein--Assistant Director, VDOE Training & Technical Center (T/TAC, Region 4), Kellar Institute for Human disAbilities
     

Content Sources:

The Helen A. Kellar Institute for Human disAbilities

ohttp://kihd.gmu.edu/

T/TAC Online

ohttp://www.ttaconline.org

VDOE Website

ohttp://www.pen.k12.va.us/

GMU immersion project Website

ohttp://immersion.gmu.edu/immsite/

 

II. Data Collection

1. Presentation by Dr. Behrmann , Dr. Wang and Dr. Fulcher

2. Interview with Nancy Andersen, Clare Talbert and Soojin Jan

3. Interview with Mary Wilds and Seunghun Ok

4. Interview with Dr. Lynn Wiley and Kay Klein

5. Interview with Dr. Behrmann

6. Data from past immersion T/TAC project website

 

 

 

1.Presentation by Dr. Michael Behrmann, Dr. Shuangbao Wang, and Dr. Amie Fulcher

 

Dr. Behrmann indicated a need to determine whether applying new technologies can enhance T/TAC Online. He expressed particular interest in technologies that were not available at the inception of the site. He wants to make T/TAC Online more user friendly, to make information easier to find,  and to increase the customization of site content according to the varying needs of its users.  He noted that an automated means for analyzing data on site visits would be especially useful to get "...feedback on who’s using what, and how they’re using it, and how they feel about it..., so that we could build the system and we could adjust to where the demand is."

 

Dr. Fulcher discussed her doctoral dissertation, "The Diffusion of an Educational Technology Innovation: Which Opinion Leaders are Most Likely to Accelerate the Adoption Of T/TAC Online Among Virginia Educators?". She found that school principals were stronger opinion leaders than teachers. As part of her study, Dr. Fulcher sent out an electronic newsletter called "Fast Facts". She noted that principals used email as an information dissemination tool. In addressing research implications for professional practice, she listed options such as:

  • offering subscriptions to the "Fast Facts" eNewsletter
  • providing podcasts of recorded state conferences
  • delivering RSS feeds as a news service and to apprise subscribers of T/TAC Online updates immediately
  • listing job openings for special education teachers on T/TAC Online.

 

Dr. Shuangbao Wang gave an overview of collaborative and data-driven aspects of Web 2.0 technologies. He touched on Rich Internet Applications, Web feeds, and content management systems. With respect to T/TAC Online, he noted challenges regarding the inability to bookmark dynamically generated content or to highlight content for future reference. He mentioned the importance of ensuring data security in applying Web 2.0 technologies. He pointed out potential for applying these technologies to T/TAC Online in some areas:

  • Resources and Events: RSS feeds
  • Webshops: Podcasts
  • User Management and Progress Tracking: Wiki and CMS

 

 

2. Interview with  Clare Talbert, Nancy Anderson, Soojin Jang

 

These SMEs say that the biggest strength of T/TAC Online is providing a wealth of useful resources and information that are easy to find for time-pressured teachers. They report challenges in developing effective content for online training. They would like to make  webshops more interactive. They also note that T/TAC staff can post webshops without prior approval from VDOE. Doing a webshop requires a visitor to log in, so some data on the use of these webshops is available, including evaluations done upon completion. The SMEs stressed the importance of achieving accessibility of resources available on T/TAC Online, including captioning where appropriate.

 

They described significant venues of information exchange for T/TAC professionals:

  • Priority project teams that address major priorities identified by VDOE
  • Annual or biannual T/TAC directors meetings
  • Meetings of special education directors in the region
  • Informal networking
  • Use of Polycom

 

They also reported that a survey will soon be included under the SOL Enhanced tab of T/TAC Online to determine how that section of the site is being used.

 

 

 

3. Interview with Mary Wilds and Seunghun Ok

 

Mary Wilds and Seunghun Ok view the evaluation of the effectiveness of T/TAC online as a higher priority than any need for a technological aid to enhance collaborative communication. A major shift seems to have occurred regarding the permissibility of tracking the browsing behavior of individual visitors to T/TAC Online. They seem to agree that an optional login for site visitors that would enable logged in users to be tracked as individuals is fine, because this is necessary for the collection of meaningful evaluation data. Mary Wilds stressed the importance of obtaining evaluation data to justify the continued funding of T/TAC Online.

 

 

The implications of an optional login for T/TAC Online users appear profound, as Seunghun Ok explains them. Tracking individual users makes individual customization of content possible using Web 2.0 technologies and simultaneously enables collection of quantitative and qualitative evaluation data.  An optional user login allows for analysis of individual browsing behavior and could thus enable very targeted sollicitation of user feedback on the value of the specific resources accessed in past visits.

 

 

Mary Wilds described her vision of a T/TAC Online that serves as a "one-stop shop".  She acknowledged the value of finding or developing a simple, collaborative communication tool that could be seamlessly integrated with the familiar T/TAC Online interface. Such a tool would have to be part of a closed system, accessible only by T/TAC professionals and other members of Priority Project teams. Mary Wilds noted that one Priority Project team is experimenting with the use of Blackboard as such a tool, even though it is an environment distinct from T/TAC Online, and she cited prior Statewide training on Blackboard as a driver toward successful use of it. She also observed that channels of communication are flowing better than ever now because of the interaction of Priority Project teams. Email seems to be a preferred mode of communication for most T/TAC stakeholders including T/TAC professionals across regions, within Priority Project teams, and between T/TAC professionals and local school officials and teachers.

 

 

Seunghun Ok and Mary Wilds described a few instances where T/TAC attempted to use collaborative communication tools, including an electronic  bulletin board, a listserv, and a free tool available through Yahoo. All these attempts failed. Mary Wilds noted that Priority Project teams  include only 10 to 15 people, they rely heavily on email to communicate, and this apparently works fairly well.

 

 

Mary Wilds expressed interest in updating templates for the delivery of online training through T/TAC Online, but she also related that she has received some positive feedback on the simplicity of the current templates. The level of computer skills of teachers varies dramatically as does their access to current software and hardware and broadband Internet access. She commented that much use of the site occurs in the evenings, after school hours, so many teachers presumably gain access using whatever computer resources they have at home, rather than school computer resources.

 

 

The interview also yielded information on the "train-the-trainer" model of service that T/TAC is now pursuing. T/TAC professionals train teams of people from different areas of the state who then assist teachers in those areas.

 

 

Mary Wilds reported that plans to send "Fast Facts" eNewsletters are under way. Initial recipients will be past users of T/TAC Online webshops. She also confirmed the importance of the Section 508 accessibilty of resources offered on T/TAC Online, pointing out that pdf and Microsoft Word documents on the site are somewhat deficient in this regard.

 

 

4. Interview with Dr. Lynn Wiley and Kay Klein

 

Lynn Wiley and Kay Klein said that T/TAC provides effective professional development around initiatives defined by the state performance plan and the priority projects.  T/TAC wants to emphasize the instructional opportunities for teachers that will help the local regions meet performance plan objectives and priority project goals.  

 

They confirmed that T/TAC staff heavily rely on email and expressed some concerns that email is not a complete communication solution for collaborative work activities. They believe a collaborative workspace integrated into T/TAC Online could greatly benefit the work of T/TAC staff and priority project teams. To be effective, such a workspace would need to be very easy to use and should facilitate interaction regarding specific tasks or topics. Better communication on policy matters, especially interpretation of existing policy, could result from the use of collaborative communication tools.

 

Lynn Wiley expressed her belief that T/TAC Online should provide resources that are specific to the Commonwealth of Virginia. T/TAC should not develop webshops that are duplicative of training resources available elsewhere.

 

 

5. Interview with Dr. Michael Behrmann

 

Dr. Behrmann reviewed draft opportunities and solutions of the T/TAC iTeam (2007-2008 GMU Immersion Team) and provided feedback. He noted that he had purchased much collaboration enhancing software, but this software is not yet being used extensively. The assets he described include Web conferencing and video conferencing software. Specifically, he mentioned Codim IP VCR, MOVI Server, and Adobe Connect.

 

Dr. Behrmann indicated that the use of collaborative tools—including collaborative workspaces—by T/TAC staff is fine as long as access is strictly limited to staff. He pointed out a need to facilitate use of existing collaborative conferencing software through the staff-only area of T/TAC Online. He envisions a virtual conferencing center, the use of which should not require significant staff training. He believes that such a conferencing center would facilitate the development of training materials that take advantage of newer technologies. Dr. Behrmann stated that whatever the iTeam develops should be a "push into the future" towards Mobile Web 2.0.

 

By November of 2007, all visitors to T/TAC Online will be able to register to receive the new Fast Facts electronic newsletter. Dr. Behrmann discussed the implications of such a login with respect to studying individual user behavior on T/TAC Online and customizing site content for logged-in users. He also pointed out the online evaluation instrument that is currently used by persons who complete online training through T/TAC Online.

 

 

6. Data from past immersion T/TAC project website (2003- 2006)

 

Brief summary of actuals and optimals identified by past immersion groups as it relates to our current project.
 
 
2003- 2006 immersion group—Actuals identified:
-         Teachers do not have sufficient access to resources
-         Teachers collaboration outside classroom is limited
-         Teachers are busy and constantly need to monitor students.
-         Teachers did not have the experience and/or the time to aligning lesson plans to new standards
-         Schools have very high staff turnover
-         There is a decline in attendance at T/TAC Training Seminars & Conferences
 
2003- 2006 immersion group—Optimals identified:
-         Teachers want to be familiar with state and district wide assessment
-         Teachers need to have knowledge of learner's skill level, needs, and characteristics
-         Teacher need access to best practices for instruction
-         Teachers need ways to customize lesson plans for student
-         Schools need cost-effective method to continue to provide training
-         Public tool for discussions is needed for different stakeholders
-         Facilitate sharing and creation of lesson plans
-         Reduce staff travel while improving service capabilities
-         Compiling workshop content from the whole state is value.
-         Expand sources of content to nationally recognized experts

 

Drivers & Barriers

 

*10/3: Here is an rtf file of the most current Drivers/Barriers, in non-table form: Drivers_Barriers_for_table.rtf

* 9/30:  I teased out the Drivers and Barriers for each category.  Rahel -- I'm not sure if that changes any of the suggestions on your document.  Either way, I think it would be good to have your feedback on the other sections as well as the skills and motivation if you have the opportunity.  ktm

 

*I folded in the additional information from the 9/27/07 SME interview.  I will tease out the Drivers and Barriers under each category sometime this weekend.  ktm

 * KM, I tried to look through your driver/barrier- Skills and Motivation section in detail. I think some of the skills you mentioned are environment or processes. I made some suggestion on word document. If you find it useful, please let me know and I will continue with the last two section. Drivers_Barriers KM - Rahel Recommendation.rtf   Rahel

 

 

 

Here is a WordPad file of the Drivers and Barriers: Drivers_Barriers.rtf

 

Drivers and Barriers

 

  Drivers Barriers
Access to content on T/TAC Online/Ease of Use

Overall, it is easy for T/TAC Online stakeholders to find information (teachers) and to post content (T/TAC staff)


Stakeholders like consistency and simplicity

 

Stakeholders want T/TAC Online to be more flexible, friendly

 

T/TAC staff want T/TAC information to be seamless for regional users (joining T/TAC sites with T/TAC Online)

 

Stakeholders want customized content tailored to their interests

Teachers do not have time to search for information on T/TAC Online


Some stakeholder are not tech savvy/do not want to embrace technology, they do not have access to advanced technology, or they are not shown how to use technology

 

Not all content on T/TAC Online is accessible (captioning for webshops, PDFs, PowerPoints)

 

T/TAC Online is not device independent

 

There is no means to automate the analysis of data of T/TAC Online site usage (who is using it, when they are using it, what resources are they using)

 

Unable to bookmark or Google search T/TAC Online due to dynamic content
Communication of information among stakeholders/Harnessing collective intelligence

Information sharing among T/TAC staff and to stakeholders occurs through face-to-face interaction and electronic means (email, T/TAC Online, e-community, Polycom, BlackBoard)

 

T/TAC staff and stakeholders prefer email to communicate and some are familiar with BlackBoard

 

Information sharing occurs through Priority Projects and T/TAC Staff and Regional Special Education Directors determine how and what information is shared in their regions

 

A priority for T/TAC is to assist those schools that do not make AYP

 

T/TAC Online’s should be the “one stop shop” for all information and resources

 

It is assumed that T/TAC staff are responsible for posting content to T/TAC Online

 

Areas of T/TAC Online that receive the most user traffic are dependent on AYP, influence of school opinion leaders (principals), marketing of information and resources (Fastfacts), event registration, and activities promoted by VDOE (such as Virginia Assessment)

 

Top down communication occurs on regulatory issues (DOE/VDOE)

 

More information sharing occurs now more than ever before among T/TAC communication channels (informal networking)

 

Belief that VDOE has control over content posted to T/TAC Online

 

Successful online collaborative communication among T/TAC staff needs to be task specific

 

Stakeholders do not like using list servs, bulletin boards, blogs

 

It is assumed that T/TAC staff are responsible for posting content to T/TAC Online.  There are no incentives for T/TAC staff to post content to T/TAC Online

 

VDOE will only allow controlled or closed collaboration/communication in an online setting

 

No online environment available for T/TAC staff to work collaboratively and synchronously within T/TAC Online

 

Communication barriers limit information exchange between T/TAC staff and T/TAC Online users (information from T/TAC Online may not reach teachers, difficult to get feedback from teachers about T/TAC Online)

 

Belief that VDOE has control over content posted to T/TAC Online

 

T/TAC staff would like to see an increase in teacher-to-teacher communication and training, but there are people who are teaching who are not professionally trained to teach

 

T/TAC staff want feedback on effectiveness of T/TAC Online without overwhelming stakeholders

 

Do not have staffing to monitor open communication in an online environment

 

Past efforts to implement collaborative communication tools suggest that T/TAC staff do not want to use separate, additional tools to communicate. It is unclear if staff would use new communication tools even if they were seamlessly integrated into the familiar T/TAC Online interface.
Capacity building among T/TAC staff and among T/TAC stakeholders

T/TAC staff are local experts to their regional school districts with special needs students as it relates to Priority Project goals

 

T/TAC staff build the capacity of local teams by providing resources and posting content to T/TAC Online

 

Federal indicators and state mandates drive the work plans of Priority Projects and T/TAC activities and ultimately impact classroom instruction

 

Teachers need information on Enhanced Scope and Sequence (ESS+), VAAP, and differentiation

 

T/TAC staff need information on VDOE policies and procedures

 

Continual funding of T/TAC depends on measurable impact of T/TAC Online activities

 

All teachers need training on items related to Priority Projects

 

Regional needs are different

 

Users want more resources available on T/TAC Online

 

New technologies need to be used to enhance content

 

Special Education teachers cannot get out of their classrooms for training.

 

Climate of education is focused on accountability (AYP)

 

T/TAC is very autonomous with regards to T/TAC Online content

 

Priority project teams have their own culture, processes, and approaches

 

T/TAC Online helps people work smarter, not harder

 

Need to look at creating guided, state sponsored online activities for teacher certification

T/TAC staff devote many hours vetting webshop content created by outside sources.

 

Transfer of face-to-face trainings to create dynamic and collaborative webshops has been difficult

 

Stakeholders prefer to view webshops in print form

 

Dissemination of information (via T/TAC Online) and teacher training impacted by high teacher turnover rate

 

T/TAC Online not taking advantage of existing e-learning resources

 

Majority of teachers at schools not meeting AYP are long term substitutes, not certified, or not teachers

 

Regional needs are different

 

Special Education teachers cannot get out of their classrooms for training

 

Teachers don't have any way of showing their accomplishment related to webshop completion

 

Climate of education is focused on accountability (AYP)

 

Policy changes can occur mid-stream effecting resources

 

Priority project teams have their own culture, processes, and approaches

 

T/TAC regions want to stay autonomous

 

No incentives for teachers to use T/TAC Online

 

 

ACTUALS AND OPTIMALS

 

The actuals and optimals explain what is currently existing and what the ideal situation should be.

 

The word version of the actuals and optimals is: Actuals and optimals 1.doc

 

 

 

Actuals

Optimals

Access to content on TTAC online/ ease of use
  • TTAC online users have very little time and want TTAC online to serve as a “one-stop-shop” where they can get all the information they want.

 

  • Some parts of the site are not easily accessible especially by people with disabilities (for example Word, PowerPoint, and PDF files).

 

  • Web shops are not dynamic for the users and some users are not technically savvy.

 

  • TTAC online has no mechanism in place for tracking and collecting feedback on its users.

 

 
  • In order to save time for users of TTAC online, any additional development of  resources should be made available through TTAC online.

 

 

  • The site should be easily accessible by people with disabilities and meet the Web Accessibility requirements.

 

  • Have very simple templates in order to have interesting, short and interactive web shops to make them more attractive.

 

 

  • Be able to have a means of customizing user information in order to track as well as collect user feedback.

 

 

Information sharing among stakeholders/Harnessing collective intelligence
  • There is no means of getting information on new resources to teachers instantly.

 

  • Information sharing and collaboration among TTAC online users as well as TTAC staff is limited. The methods used right now include face-to-face meetings, presentations, emails, newsletters, conferences etc.

 

 

 

 
  • Availability of a means to enable teachers to receive instant information on new resources.

 

 

  • Have a variety of options available for collaborative communication using available and emerging technologies to improve the flow of communication channels and information sharing among T/TAC staff and T/TAC Online users.

 

 

 

 

 

 

 

Capacity building among TTAC staff and stakeholders

·        Teachers do not have incentives to use T/TAC Online.

 

·        Special Education teachers are very busy and thus are not able to leave the classrooms for training.

 

·        Regional needs vary hence TTAC regions prefer to stay autonomous.

 

 

 

  • The provision of incentives for teachers that use TTAC online.

 

  • More training needs to be made available through T/TAC Online so that Special Education teachers can have the opportunity  to participate asynchronously. 

 

  • A need for a collaborative environment to increase interaction among T/TAC staff and reduce duplication of efforts and reinforce the "one-stop-shop" among T/TAC regions.

 

 

 

 

 

 

 

 

 

 

 

Opportunities and Potential Solutions

 

Take advantage of emerging technologies to harness collective intelligence of T/TAC community by analyzing communication channels (create a "web" of communication channels among T/TAC staff and Priority Project Team members).

Examine formal and informal communication channels or “information flows” between T/TAC staff and Priority Project Team members.

Based on findings concerning information flows, develop plans to support and enhance existing communication channels or foster the development of new channels for the beneficial flow of information between individuals within T/TAC staff and Priority Project Team members. 

Explore the affordances of existing (e-Meetings Blackboard, Polycom, e-community) and emerging (Web 2.0 such as blogs, wikis, Mobile Web 2.0) technologies to seamlessly integrate information sharing and collaborative interaction to T/TAC Online.  Determine the most effective ways of using technology to share resources and explore task oriented and topic specific collaboration.

Customize T/TAC Online content for T/TAC Online users. 

Examine the possibility of customizing user experience by tracking individual behavior and interaction with content on T/TAC Online.  Explore possibilities of customization with and/or without a user login.  Based on findings, determine the best way to customize site content for users. 

Explore the feasibility of establishing usage histories for individual, so users have easy access to content from past visits.

Analyze the potential for implementing mechanisms that allow T/TAC Online users to specify their interests.  Data collected on users’ interests will be captured and used to customize user experience on T/TAC Online.

 

Create processes for evaluating the effectiveness of T/TAC Online.

Examine means of obtaining targeted feedback on individual resources from T/TAC Online users.

Explore means of evaluating collected feedback to measure the effectiveness of activities and resources provided by T/TAC Online with respect to meeting performance indicators.

Develop evaluation mechanisms that minimize burden on users (“two clicks” principle, optional feedback).

 

 

Priorities and Recommendations

 

Based on our data collection, initial findings, analysis of drivers and barriers and actuals and optimals, and feedback from Dr. Michael Behrmann, we recommend analyzing the feasibility of the following solution systems through a more thorough needs analysis.

 

1. Develop a virtual collaboration center that would enable T/TAC staff to take advantage of existing but currently under-used conferencing and collaborative software resources. The virtual center would reside within the staff-only section of T/TAC Online. It would enhance productive communication and collaboration among T/TAC staff. The center might also:
  • Provide a collaborative workspace for Priority Project teams
  • Serve as an archive of past video conferences, creating the potential to repurpose recordings as training content
  • Support collaborative development of training materials
  • Enhance collaboration in other areas depending on the results of deeper analysis of existing information flows and communication and collaboration needs.

 

Conduct needs analysis to determine how to use the available collaborative technology resources to maximize benefit.  Determine how best to enable use of collaborative technology resources (job aids, training, etc.).

 

2. Create resources to support T/TAC staff development with respect to their ability to use newer training creation tools, such as Flash, Camtasia, Articulate, Captivate, podcasting, accessibility tools, annotated Powerpoint presentations, or others.

 

The resources developed might be housed within the virtual collaboration center. Some resources might be deployable via mobile devices and could thus serve as test cases for the applicability and usefulness of this mode of training delivery for the T/TAC staff audience.

 

3. Customize T/TAC Online content for individual users

 

Explore how implementation of the optional login to occur by November 2007 might create opportunities to enhance customization of content. Examine the possibility of customizing user experience by tracking individual behavior and interaction with content on T/TAC Online.

 

Explore the feasibility of establishing usage histories for individual users, so users have easy access to content from past visits.

 

Determine the best way to customize site content for users.  Consider whether data mining can be used to track site usage and to evaluate T/TAC Online.

 

 

 

Team Members Roles and Responsibilities

Primary Responsibility areas:

Kim Amin: Background and Introduction

Rahel Woldehanna and Matthew Mariani: Data Collection and Information Sources

Kim Monti: Drivers and Barriers

Afua Gyasi: Actuals and Optimals

Matthew Mariani: Priorities and Recommendations

Rahel Woldehanna: Final editing of Performance Analysis

 

Group Responsibility areas:

Clients and Stakeholders

Opportunities and Potential Solutions

Priorities and Recommendations

 

 

References

 

Rossett, A. (1999). First Things Fast: A Handbook for Performance Analysis. San Francisco, CA: Jossey-Bass/Pfeiffer.

 

T/TAC Online. Retrieved September 20, 2007 from http://www.ttaconline.org/staff/s_home.asp.

 

Instructional Design and Development Immersion Program.  Retrieved September 20, 2007 from http://immersion.gmu.edu/immsite/student/ttac.html.

 

Digest of Education Statistics 2006. U.S. Department of Education, July 2007.

 

U.S. Bureau of Labor Statistics online database for the Occupational Employment Survey. Retrieved September 18, 2007 from http://data.bls.gov/oes/search.jsp?data_tool=OES

 

 

 

Appendix: Findings

 

Word file of Findings: findings_appendix.rtf

 

Skills, Knowledge, and Information:  Capacity - Individual's "ability to do what is needed and to know what he or she needs to know" (Rossett, 38)

 

 

Access to content on T/TAC Online/Ease of Use

 

 

Findings:

  • T/TAC Online lets stakeholders find the information faster.  It is easy to navigate and has a wealth of information
  • It is easy for T/TAC staff to post resources and information on new events to T/TAC Online
  • 50-70% of T/TAC staff are tech savvy when it comes to posting general information and content to T/TAC Online
  • Teachers do not have a lot of time to get in, get information and get out - Teachers should have access to information in no more than "two clicks" 
  • 30-40% of T/TAC staff are tech savvy when it comes to building webshops
  • Still have some teachers without technology skills or access to more advanced technology

 

 

 

Communication of information among stakeholders/Harnessing collective intelligence

 

 

Findings:
  • Information sharing among T/TAC staff and disseminated to stakeholders through face-to-face meetings, email, T/TAC Online, and other technologies (e-community, Polycom)
  • T/TAC staff, school opinion leaders, teachers prefer to communicate via email
  • T/TAC staff familiar with BlackBoard since most Virginia state universities use Blackboard. 
  • Collective intelligence of T/TAC professionals evident in collaborative efforts (i.e. differentiated section in lesson plans found under SOL resources)

 

 

 

Capacity building among T/TAC staff and among T/TAC stakeholders

 

 

Findings:
  • T/TAC professionals are experts in assisting children with special needs
  • T/TAC staff become local experts to their region based on the Priority Project that they are involved with
  • T/TAC strategy is to focus on capacity building of local teams
  • Resources are available through T/TAC Online for teachers, administrators, Special Education Directors, etc.
  • T/TAC staff responsible for posting content to T/TAC Online

 

 

 

    Online Training/Webshops

 

 

    Findings:

        - T/TAC staff may post webshops without having them vetted.

        - T/TAC staff devote many man hours vetting webshop content created by sources outside of T/TAC

        - E-learning concept has been hard idea for people grasp when transferring content from a face-to-face workshop to an online webshop.  Information needs to be presented in a dynamic             way that allows opportunities for collaboration.

        - Many stakeholders do not view webshops online.  They prefer to print out webshop materials.

 

 

 

Motivation: Individual's ability to acknowledge the reasons for doing something; confidence (Rossett, 39)

 

 

Access to content on T/TAC Online/Ease of Use

 

 

Findings:
  • Keep it simple - Stakeholders like consistency and simplicity
  • The more technology savvy people are using T/TAC Online the most
  • Still have people who do not want to embrace technology
  • Technology exists in the schools, but teachers are not shown how to use it

 

 

Communication of information among stakeholders/Harnessing collective intelligence

 

 

Findings:
  • Information sharing happens through Priority Projects
  • As information comes down from VDOE, it is up to the regional T/TAC staff to disseminate the information in their region.  Special Education Directors in the region then decide how information in disseminated to local school districts.
  • Much of the communication occurs between T/TAC professionals and administrators of schools not meeting Adequate Yearly Progress (AYP)
  • T/TAC Online is the vehicle for T/TAC and VDOE staff to provide training, share information and resources, and communicate initiatives.  Goal is to make T/TAC Online a "one-stop" shop of information and resources
  • T/TAC Online is a way for T/TAC staff to funnel information and simplify their jobs.  It is assumed that it is part of T/TAC staff's job to post information to T/TAC Online.
  • Users want more customized content on T/TAC Online (such as information by disability type)
  • Visits to SOL area of T/TAC Online impacted by the need for schools to meet AYP (25% of hits last year)
  • School opinion leaders (principals) introduced to T/TAC Online impacted first time use of T/TAC Online by teachers
  • The registration for events has also increased awareness of T/TAC Online
  • Fastfacts newsletter highlighted online training and as a result the number of hits for online training increased
  • Believe that Virginia Assessment gets lots of hits, esp. in the coming months, due to increased awareness of the assessment instructional material.

 

 

 

Capacity building among T/TAC staff and among T/TAC stakeholders

 

 

Findings:
  • VDOE provides a list of schools not making Adequate Yearly Progress (AYP). Those schools not meeting AYP are assisted by T/TAC
  • State performance plan is based on federal indicators.
  • T/TAC Priority Projects are the process to ensure the indicators are being met.  Each Priority Project has a work plan that is based on the federal indicators and state mandates
  • Not only one Priority Project meets a specific indicator. Indicators span across Priority Projects
  • T/TAC staff becomes specialists on Priority Project items and will be the first point of contact on those issues for their region
  • What the teachers need: Information on Enhanced Scope and Sequence (ESS+), VAAP
  • Need an area on T/TAC Online that focuses on VDOE policies and procedures
  • Differentiation is a hot button issue because so many children have different abilities that all teachers in all classrooms have to take into consideration.
  • Teacher turnover high, which impacts the ability to get information out about T/TAC Online
  • Teachers need to change the direction of their teaching based on the state indicators
  • Some resources are aligned differently in the different regions.
  • Evaluation is the issue.  Continual funding depends on measurable impact of T/TAC Online activities based on Priority Project indicators
  • Getting the tools in the hands of the teachers for them to use in the classroom so that it can impact state indicators. T/TAC Online should stress the instructional opportunities teachers have that will impact achievement of indicators
  • Need to encourage more stakeholders to use T/TAC Online
  • Differentiation is a hot button issue because so many children have different abilities that all teachers in all classrooms have to take into consideration.

 

 

 

    Online Training/Webshops

 

 

    Findings:

        - All teachers (general education included) need training on Priority Projects because they all have children the projects target in their classrooms

        - T/TAC has evaluation data from users who participate in webshops

        - Not sure why T/TAC should re-create the wheel when it comes to webshops.  Links should be made to webshops that already exist at other sites.

        - All teachers (general education included) need training on Priority Projects because they all have children the projects target in their classrooms

        - Not sure why T/TAC should re-create the wheel when it comes to webshops.  Links should be made to webshops that already exist at other sites.

 

 

 

Environment, Tools, Processes: Organizational issues (Rossett, 41)

 

 

Access to content on T/TAC Online/Ease of Use

 

 

Findings:
  • T/TAC Online needs to be more flexible and friendly
  • Folding regional T/TAC sites into T/TAC Online and making it seamless for regional users
  • Access to T/TAC Online needs to be device independent
  • T/TAC Online needs to be accessible (captioning for webshops, PDFs, PowerPoints)
  • T/TAC Online regional users should be able to log onto site and see information about their view information that is customized to their interests
  • Currently there is no means to automate data of T/TAC Online site usage to identify who is using it, when they are using it, and where they are going on the site
  • Users are unable to bookmark dynamic content provided on T/TAC Online

Communication of information among stakeholders/Harnessing collective intelligence

 

 

Findings:
  • Top down communication occurs from DOE and VDOE on regulatory issues
  • Horizontal communication occurs across T/TAC professionals and across Priority Projects
  • T/TAC Online is also where VDOE shares information on mandates
  • VDOE will only allow controlled collaboration/communication in an online setting
  • No online environment to work collaboratively and synchronously
  • Information disseminated by T/TAC Online staff may not always reach teachers.
  • T/TAC staff have difficulty getting feedback from teachers on T/TAC Online initiatives

 

 

 

Capacity building among T/TAC staff and among T/TAC stakeholders

 

 

Findings:
  • T/TAC staff provides the tools and let's the local team take care of implementation
  • T/TAC operationalizes policies and procedures in their region
  • Regional needs are different
  • Majority of teachers at schools that are not meeting AYP are: long term substitutes, not certified, not teachers
  • Anecdotal information from users indicate that T/TAC Online is meeting user needs, but users still want more resources
  • Turnover of teachers is high and as a result, teacher pool is constantly evolving.  T/TAC staff trains teachers and then the teachers leave
  • Majority of teachers at schools that are not meeting AYP are: long term substitutes, not certified, not teachers
  • Regional needs are different

 

 

 

    Online Training/Webshops

 

 

    Findings:

        - Need to simplify tools that allow T/TAC staff to upload webshop content

        - Need to use new technologies to enhance T/TAC Online content, specifically webshops

        - Special Education teachers cannot get out of their classroom for training.  There are no substitutes available.

        - Teachers don't have any way of showing their accomplishment related to webshop completion

 

 

 

Incentives: Organizational culture (Rossett, 43)

 

 

Communication of information among stakeholders/Harnessing collective intelligence

 

 

Findings:
  • Informal networking often occurs among stakeholder groups
  • Communication channels - not all communication starts with VDOE
  • T/TAC staff feel there is more information sharing now than ever before amongst the various communication channels
  • Belief among some staff that VDOE has control over content posted on T/TAC Online
  • VDOE may approve a closed community of learning
  • Current collaborative communication seems very task specific.  Focus should be narrowed to one topic a week. Communication can flow if it is focused on a task and moderated T/TAC staff serves as gatekeepers of information, designers, and marketers of T/TAC online
  • Efforts to more effectively market T/TAC Online will increase site usage (current marketing - Fast Facts, newsletters, conferences, workshops)
  • T/TAC staff would like to see an increase in teacher-to-teacher communication and training, but there are people who are teaching that don't know how to teach
  • Biggest challenge is trying to collect feedback on effectiveness of T/TAC resources without overwhelming stakeholders
  • Belief among some staff that VDOE has control over content posted on T/TAC Online
  • Open forums will not be sanctioned by VDOE unless someone is monitoring it every day, but T/TAC does not have the staffing to do that
  • No incentives for T/TAC staff to post to T/TAC Online, it is not part of their job description.  As a result, some T/TAC staff are not utilizing T/TAC Online to channel information and resources.

 

 

 

Capacity building among T/TAC staff and among T/TAC stakeholders

 

 

Findings:
  • Climate of education in the US is focused on accountability, so it is important to evaluate how site resources are being used to meet AYP
  • Policy changes can occur mid-stream, which effects resources provided to T/TAC stakeholders
  • T/TAC is very autonomous. VDOE does not need to know everything that T/TAC is doing
  • All of the Priority Project teams have their own culture, processes, and approaches.
  • Regional staff located at local universities, which eliminates territorial issues. 
  • T/TAC Online is meant to help people work smarter, not harder
  • Difference between AT (Assistive Technology) people and IT (Information Technology) people
  • VDOE may want to see what Best Practices look like in terms of webshops outside the T/TAC site (state sponsored online activities).  Create guided activities to help teachers achieve certification
  • Policy changes can occur mid-stream, which effects resources provided to T/TAC stakeholders
  • All of the Priority Project teams have their own culture, processes, and approaches.
  • Each T/TAC region wants to stay autonomous. Each region has the university to answer to and most regions have a Principal Investigator (Mike Behrmann equivalent) too
  • Difference between AT (Assistive Technology) people and IT (Information Technology) people
  • No specific incentives created for teachers to use T/TAC Online

 

 

 

 

 

Comments (0)

You don't have permission to comment on this page.

PDF version