Giz's Instructional Design Blog

Ongoing Project Wrap-Up

2007-12-02T11:24:00.000-08:00

The Technology@WFU program, a component of freshmen orientation at Wake Forest, has been redesigned twice in the 12 plus years since its inception in August of 1995. The original format was hands-on, face to face, classroom instruction that focused on basic computing skills. The program was adapted to an arena-style format that focused on computing in a community environment in August 2005. The shift to an arena-style, sage on the stage format came out of a need to use fewer resources and move the program from a weekday to a weekend event as the orientation period shrank by a day. Research of other institutions showed this arena-style format was common, and survey results from the first year showed it had similar customer satisfaction to the classroom based instruction. Now, three years into the arena-style format, it is once again time to evaluate this program, this time considering a move to an online format. For August 2008, the program will be offered both arena-style and to a pilot group online. An assessment will be conducted of students from both groups in order to determine retention and an evaluation will be done of both programs to determine both customer satisfaction and the efficacy of both the online and face to face formats.

A committee has been formed by the Manager of Technology Training in the Z. Smith Reynolds Library to create, implement and assess the online and face to face Technology@WFU program for 2008. This committee is composed of the Manager of Technology Training, the Training Specialist, the Instructional Design Librarian, the Library webmaster, and one of the Instructional Technologists from an academic department. This committee will report to the Training Task Force and work in conjunction with the WFU Orientation Committee to design and deliver Technology@WFU 2008.

An online alternative to Technology@WFU has been discussed since the program’s inception, but it was not until the alignment of numerous elements of the program happened that this option was truly considered. In previous years the Training Task Force had seen a migration to an online format as a “bridge too far” It was not until the elements of Instructional Design and the consensus building efforts of the Manager of Technology Training came together that this option began to be seriously considered. For 2008, both formats will include assessment and a more comprehensive evaluation process. Additionally, prior to designing the program more effort will be placed on needs assessment and the creation of goals and objectives, all in an effort to create a more effective and engaging learning experience for the students.

In MBA School one of the key tenants is “You Manage What You Measure”. In many ways, Instructional Systems Development (ISD) follows this mantra as well. By creating a prescription for effective instruction, it becomes possible to measure needs with needs assessment and task analysis. It also becomes possible to measure efficacy with assessment and evaluation. The ADDIE model gives instructional designers and instructors the framework to create and assess effective instruction consistently in a way that can be replicated in multiple incarnations over time. While examining the Technology@WFU program using the ADDIE model, it became apparent it was weak in the areas of assessment and evaluation.

A needs assessment of the participants of the Technology@WFU program showed that while the Help Desk gets many support calls on basic information covered in the program, when surveyed many students believe they already know this information. This disconnect was startling to the designers of the program. As a result, the decision was made to frame all the content in terms of “why you need to know this”. For example when the program covers what to do when a user spills liquids on the computer, the topic will be prefaced with the statistics on how often this issue occurs and how much it costs the student. Fortunately this program has a very homogenous group of learners, however while they are all 17-19 year old freshmen at WFU, what is not homogenous is there level of computing skills. This program must take into account those varying skill levels and find a way to engage both expert and novice users while teaching both groups about the basics of computing in a community environment at Wake Forest University.

Fortunately, task analysis and goals and objectives have been constants in the history of the Technology@WFU program. Each year the Training Task Force examines the content covered and discusses the goals of the program, however, mapping out objectives has been a stepped that was skipped in the past and will add much to the program in 2008. Additionally, the online pilot will introduce new technologies into the program, with the pilot students participating online via a web based program and follow-up assessment. In 2007 an audience response system was used with a portion of the students, while the ZSR Library has been successfully using “clickers” in the Information Literacy program for over a year, their use in the Technology@WFU program was not readily received by the students. In the end “clickers” turned out to be a poor method of assessment for this size group, and a technology that did not engage them in this format.

The most significant change to the Technology@WFU program coming from this project will be the changes to assessment and evaluation. Assessment has been non-existent since the migration to an arena style program, primarily due to the logistics of administering an assessment to 500+ students per session. Both the online pilot and the face to face program will have an assessment module, but only a random sample of the 1190 freshmen will receive this assessment, with incentives offered to those who complete the assessment to encourage participation. Evaluation has previously been based on feedback of the Resident Technology Advisers, and the Help Desk. This will continue, and be expanded to include more directed questions on the Freshman Orientation Survey.

The Technology@WFU program for 2008 will still resemble the previous program, but will include the elements of instructional design that will help the designers properly assess the program and give the designers the information they need to make well-informed decisions about the future of the program.

Assessment & Evaluation

2007-11-27T20:11:00.001-08:00

In the past, the Technology@WFU program for freshmen has not involved any formal assessment of the students. Evaluation of the program was based on feedback from the Resident Technical Advisors (RTAs) who live in the residence halls, and feedback from the Information Systems Help Desk.

The August 2008 Technology@WFU program will include a pilot program of 50 students who will participate in an online version of the program, while the other 1150 students participate in a face to face program. The members of the pilot and a random sample of students in the face to face program will take an online an assessment two weeks after the program to gauge the effectiveness of the two formats.

In an ideal world, all participants in the Technology@WFU program would participate in an assessment, but the logistics of assessing 1150 students in a meaningful way are proving to be a challenge. The committee developing the online pilot of the Technology@WFU program and assessment will be considering the assessment issue for the face to face program. This committee is composed of four Library staff with instructional design and web design experience and a member of the instructional technology group with instructional design experience. The charge of this committee is to develop the Technology@WFU online pilot and the student assessment to evaluate the program.

Based on the assessment results, the Training Task Force will make a decision to move the Technology@WFU program to an online format or keep the current face to face model. Additionally there will be questions about the Technology@WFU program in the Freshman Orientation survey given to a sample of 250 freshmen. These survey questions will be used to evaluate the Technology@WFU program.

Instructional Strategies for the Technology@WFU Program

2007-10-26T13:38:00.000-07:00

Goal One: By the end of the program, the freshmen who attend will know how to successfully care for ThinkPad.

Procedure Strategy

Knowledge Structure:

Transporting the ThinkPad

Proper shut down prodedure to avoid hard drive damage

Proper suspend procedure to avoid hard drive damage

Proper transporting with padded case to avoid impact damage

Common Care Issues

Avoiding ThinkPad contact with food or drink

Proper response for contact with drink

Proper response for any incidence of damage

Presentation:

Presentation Tools

While the content delivered in both the face to face and online versions will be aligned to ensure students receive the same content, the face to face program, delivered in a large auditorium to 500 + students per session, will focus on students introducing the topics, giving a brief introduction via a PowerPoint presentation projected on a theater size screen at the back of the stage. The videos are embedded in these Powerpoint presentations. After each video, the student presenters will solicit questions from the audience on the material presented.

Transporting the ThinkPad

Lenovo/IBM video of mock CSI episode where the “victim” is a ThinkPad and the ”perp” is a student. The vignettes portrayed in the video illustrate the proper transport of the ThinkPad and the repercussion of improper transport. Video is introduced by a Resident Technology Adviser who introduces the video and frames it by explaining how many times a year ThinkPad are broken.

Common Care Issues

WFU video vignette where a student describes the repercussion of improper care and explains how easy proper is. Video is introduced by a Resident Technology Adviser who introduces the video and frames it by explaining how many times a year ThinkPad are damage by food and drink and the expense to the student.

Goal Two: By the end of the program, the freshmen who attend will be able to use the ThinkPad in a community computing environment while avoiding common issues.

Procedure Strategy

Knowledge Structure:

Computing in a Community Environment

Connecting to Wireless Networks

On-campus

Off-campus

Network security and VPN

Getting regular WFU computing updates

Avoiding Common Issues

Using:

Pure Message

Sophos

Symantec

Spybot

Ad-aware

Thunderbird Junk Mail controls to avoid

Avoiding:

Spamming

Spoofing,

PhishingSpyware

Adware

Backup your data regularly

Presentation:

Presentation Tools

Again, these videos are embedded in these Powerpoint presentations. After each video, the student presenters will solicit questions from the audience on the material presented.

Computing in a Community Environment

Student created video on the importance of backing up data and possible repercussions as well as using the wireless networks. Video introduced by students.

Avoiding Common Issues

Student created video of common scams and how to avoid them. Student led program on Importance of backing up data and possible repercussions. Video introduced by students.

Goal Three: By the end of the program, the freshmen who attend will know how to find and use technology support resources on campus.

Procedure Strategy

Knowledge Structure:

Find and Use Technology Support Resources on campus.

Presentation:

Presentation Tools

The skits are introduced in these Powerpoint presentation. After each skit, the student presenters will solicit questions from the audience on the material presented. PowerPoint will also be used to review the key points discussed in the skit.

Student led skit demonstrating help resources available at WFU.

Exploration, Practice and Learner Guidance are the same for all three goals

Exploration:

Currently we have no model for students to play and explore. The videos show the consequences, but I’m not sure how to offer them play opportunities

Practice:

Immediately upon leaving the program students will be expected to perform these tasks.

Learner Guidance:

Resident Technology Advisers in the residence halls offer support and guidance as students perform these tasks.

Follow Up/Assessment:

A subset of the freshmen will be given an incentive to take a follow-up assessment three weeks into the semester to gauge the efficacy of the presentation.

Project Alignment Diagram

2007-10-22T19:28:00.002-07:00

Click the diagram for a full-size image of the DRAFT Project Alignment Diagram for the Technology@WFU Program.

Goals and Objectives for the Technology@WFU Program

2007-10-11T12:12:00.000-07:00

Goal

By the end of the program, the freshmen who attend will know how to successfully care for ThinkPad.

Objectives:

Demonstrate ability to care for the ThinkPad, including how to logon, shutdown, and transport the hardware.

Goal

By the end of the program, the freshmen who attend will be able to use the ThinkPad in a community computing environment while avoiding common issues.

Objectives

Use the applications Pure Message, Sophos, Symantec, Spybot and Ad-aware, Thunderbird Junk Mail controls to avoid Spamming, Spoofing, and Phishing scams as well as Spyware and Adware
Connect to the WFU Wired and Wireless Networks
Connect to other unsecured wireless networks and secure them with VPN
Backup your data regularly
Know the circumstances and consequences of illegal computing activities including internet gambling and downloading pirated media files.
Demonstrate a knowledge of classroom technology etiquette

Goal

By the end of the program, the freshmen who attend will know how to find technology support resources on campus.

Objectives

Demonstrate ability to effectively access technology help resources on campus when needed

ABCD Model

Audience-The learner, WFU freshmen

Behavior LWBAT-The Learner will demonstrate an ability to accomplish each goal.

Condition- Attendance at the Technology@WFU session

Degree-95% of the time users will be able to accomplish each goal when the situations requires it.

2007-10-07T09:37:00.000-07:00

Task Analysis for Technology@WFU Program

Mega Create graduates of WFU with technology skills they can leverage in their daily lives to be more productive and successful members of society.

Macro Create technology savvy WFU students who can successfully compute in a community environment and use those technology skills to be more effective learners.

Micro -Know how to use the WFU issued ThinkPad
-Understand how to successfully compute in a community environment
-Understand the personal responsibility of computing in a community environment

Task Inventory

Know how to use the WFU issued ThinkPad
-Know how to Care for the ThinkPad (including know the ThinkPad Insurance and deductibles)
-Know the Login and password conventions and rules
-Understand how to avoid Spamming, Spoofing, Phishing scams and avoiding Spyware and Adware
-Know how to use Pure Message, Sophos, Symantec, Spybot and Ad-aware, Thunderbird Junk Mail controls

Understand how to successfully compute in a community environment
-Know how to get ThinkPad software updated by regularly rebooting on the wired network
-Know how to access the WFU campus wired and wireless network
-Know how to access other wireless networks
-How to access Registration in WIN

Understand the personal responsibility of computing in a community environment
-Back up your data regularly.
-Understand legal consequences of Downloading and Copyright
-Understand the illegal nature of Internet Gambling.
-Know the WFU Computing Policies
-Follow basic classroom technology etiquette
-Be aware of any campus-wide issues that have developed.
-Getting Help from the Resident Technology Advisors in the Residence Halls
-Getting Help from the Help Desk

Task Selection – All task identified need to be included in the Technology@WFU program at the present time.

Task Analysis

Know how to use the WFU issued ThinkPad

Know how to Care for the ThinkPad (including know the ThinkPad Insurance and deductibles)

Place ThinkPad in Standby (FN+F4 or Shutdown prior to transporting the computer.

Protect the ThinkPad from drops, impacts and spills

Deductibles of $500 for each occurrence of damage and $750 for theft.

Know the Login and password conventions and rules

In the User Name field, enter the first four letters of your last name, your first initial, middle initial (all without capitals), followed by 7. For example, if your name is Jane M. Austen, you would enter austjm7.

In the Password field, enter the last 4 characters of your User Name (above) plus your birth date - month and day only (2 digits for the month and day). Using the example above and Jane's birthday as May 3, Jane's password would be tjm70503 (0503 being her birthday month and day).

Passwords must be changed every six months, include a combination of letters and numbers, and have a total of at least six characters.

Understand how to avoid Spamming, Spoofing, Phishing scams and avoiding Spyware and Adware

Identify and delete messages that are spam or phishing scams

Identify software downloads that may contain spyware and adware

Know how to use Pure Message, Sophos, Symantec, Spybot and Ad-aware, Thunderbird Junk Mail controls

Allow Symantec to scan hard drive

Download, Install, and Run Spybot

Download, Install, and Run Ad-Aware

Teach the Thunderbird Junk Mail Controls

Understand how to successfully compute in a community environment

Know how to get ThinkPad software updated by regularly rebooting on the wired network

Plug the ThinkPad Ethernet cable into a live Ethernet jack and reboot the computer. Live jacks are located in Residence Halls, the Library Electronic Classrooms, and Help Desk Lite in the ITC Computer Lab.

Know how to access the WFU campus wired and wireless network

Use ThinkVantage Access Connections Software instead of Windows Wireless Networking Control Panel to the Student Wirless Network

Know how to access other wireless networks

Use ThinkVantage Access Connections Software instead of Windows Wireless Networking Control Panel to find and connect to Wireless Networks using the Guest profile or by creating a profile.

How to access Registration in WIN

Go to http://win.wfu.edu/, login with WFU login and password and access the “Virtual Campus” tab.

Understand the personal responsibility of computing in a community environment

Back up your data regularly.

Back up data prior to visiting the IS Help Desk

Use Start > Backup > RecordNow Data to backup data to CD-R or DVD-R

Understand legal consequences of Downloading and Copyright

The Record Industry Association of America has subpoenaed ?? WFU students

The Record Industry on average collects $3000-$4000 per settlement.

If approached by an official about illegal downloading tell the truth, as we all should do always J

Using legal downloading options like Ruckus or iTunes

Understand the illegal nature of Internet Gambling.

Title VIII of the Security and Accountability For Every Port Act of 2006 is also known as the Unlawful Internet Gambling Enforcement Act of 2006. This title (found at 31 U.S.C. § 5361–5367) prohibits the transfer of funds from a financial institution to an Internet gambling site, with the notable exceptions of "fantasy" sports, online lotteries, and horse/harness racing.

Know the WFU Computing Policies

http://www.wfu.edu/technology/reference/thinkpad/guide/r52/policies.html

Follow basic classroom technology etiquette

When using the ThinkPad in class, avoid emailing, instant messaging and surfing the web

Cell phones and other personal communication devices should not be seen or heard in the classroom and should be muted or turned off and stowed for class.

Be aware of any campus-wide issues that have developed.

Check and Read Email daily

Check http://techtalk.wfu.edu/ regularly

Getting Help

Contact Resident Technology Advisers in the Residence Halls

Contact IS Help Desk

Visit http://help.wfu.edu/

Submit issues to http://resolveit.wfu.edu/

Learner & Contexual Analysis

2007-09-30T08:43:00.000-07:00

The Technology@WFU Program is a 75 minute program that is held on the Saturday morning of Freshman Orientation at WFU, opposite University testing. This means that half the freshmen attend Technology@WFU (575 students) while the other half is in University Testing, then the two groups swap locations and we train the second half of the Freshmen.

Learner
This means our learners are somewhat of a demographically homogenous bunch,18 year old college freshmen who have all received a WFU issued Lenovo ThinkPad two days prior. What are not homogenous are the technology skills they bring. The goal of the Technology@WFU program is to give the students the information they need and teach them the skills they need to be successful computing in a community environment. Our current face to face arena style model does not offer us the flexibility to adjust for varying skill levels. The freshman orientation survey results show that while many find Technology@WFU useful, many also feel they already know the content provided in this program.

Context
Currently, this training is offered in the midst of one of the most chaotic, change-riddled weeks of their lives. Often the approach of freshman orientation feels like giving someone water via a firehose. Like other freshman orientation programs we believe our information cannot wait. Additionally, the program must address the group environment and peer pressure where no one wants to admit they don’t know something. While we are currently evaluating the program and the delivery means, it is also an opportunity to consider if an online model would offer a more individual, receptive context for the information.

If the program continues to be a face to face model, one change that is under consideration is to preface all the information provided with “why” we are covering this material. For example, some student think it is silly that we talk about “Care of the ThinkPad” but perhaps if information on the large number of expensive damages that occur was presented, the students would find this content more relevant.

Partial Data for ID Needs Assessment

2007-09-26T12:33:00.001-07:00

I managed to get a couple of questions about "Technology@WFU" added to the Freshman Orientation Survey. While I couldn't get as specific as I would have liked due to the limitations on the number of questions, I was able to gather some data that might help in considering the format (online or face to face) and what content they found useful or unnecessary. Here is a link to the PDF file with the questions and their responses.

Technology@WFU Program Questions from Freshmen Orientation Survey 2007

This is only a small portion of the data required to design this program. Feedback will be solicited from Information Systems, Resident Technology Advisers and other constituencies.

ID Needs Assessment

2007-09-23T13:16:00.000-07:00

Needs Assessment in your blog entry:

1. Explain what Needs Assessment model you used and why.
ADDIE and Shepherd — Ragan Model
It emphasizes identifying the important goals for students to achieve and meeting the needs of students.
The Technology@WFU program is designed to meet the technology needs of students new to campus and usually new to computing in a community environment.

Not only do we need to ask the students what they need to know about computing at WFU, we also need to ask the Help Desk, the Faculty, and the Resident Technology Advisers who support technology in the Residence Halls before we can get a complete understanding of the needs.

2. List the performance goals in a situation you have identified
-Care for the ThinkPad in order to avoid damage, downtime and $charges.
-Know the ThinkPad Insurance deductibles for damage and theft in order to avoid $charges and encourage proper care of the Thinkpad.
-Know the laws and consequences regarding illegal downloading in order to avoid honor code violations and legal violations. This includes knowing about legal alternatives of music downloading provided by the University.
-Know the laws and consequences of Internet Gambling in order to avoid honor code violations and legal violations.
-Know the WFU Computing Policies, in order to avoid honor code violations.
-Understand and exhibit knowledge of Spamming, Spoofing, Phishing Scams, Spyware, Adware
-Use Pure Message, Sophos, Symantec, Spybot and Ad-aware, Thunderbird Junk Mail controls to avoid scams and viruses.
-Follow the requirements for computing in a community environment, including:
--Regularly rebooting and getting updates on the wired network
--Regularly changing passwords
--Regularly backing up your data
--Successfully accessing the WFU campus network
-Follow proper classroom technology etiquette in order to succeed academically.
-Successfully register for classes in WIN through Banner.
-Use the Technology Help Resources that are in place to support users.

3. Validate the instruction need using data. Data can be either quantitative or qualitative.
Feedback from the IS Help Desk, the Faculty, the RTAs, the WFU staff and the students themselves in the Freshman Orientation survey all reflect the need for this instruction.
(As part of the Freshman Orientation survey I included three questions about the Technology@WFU program to solicit feedback from the students. I asked them if the program provided them with the information necessary for computing at WFU, I asked their opinion of an online option instead of a face to face program, and gave them an open-ended option to give any feedback about the program or how it is delivered.)

4. Point out where there are needs for instruction and non-instructional interventions.
The IS Help Desk sees many cases that are a need for instruction, our program attempts to reduce the number of cases. The RTAs in the residence halls also see the need for instruction when making support calls to resolve issues that could have been avoided with more effective instruction.

5. Describe the learning environment.
The current environment is an orientation session held face to face in a recital hall with +550 students. However this environment is under negotiation and may be an online environment for next year.

6. What are the characteristics of the teachers/trainers who will be using your instructional unit/materials? (Maybe it's you.)
As the lead trainer of this program, I would describe my characteristics as engaging, energetic and excited about the material. The characteristics of the lead trainer would need to be expressed if the format of this instruction moves online.

7. Are there existing curricula into which their piece of instruction must fit? If so, what is the philosophy, strategy, or theory used in these materials?
No, this is a free standing orientation program. It is not for credit, it exists simply to prepare our users for computing at WFU.

8. What hardware is commonly available in the potential learning environment(s)? Are computer workstations available? If so, what kind, how many, and in what configurations and networks? What about other technology (modern or antiquated)--e.g., slide or overhead projectors? What software and other materials are available?
This is where it gets interesting. All of our students have a ThinkPad, making an online option more feasible than in some environments. Additionally, we have used an audience response system in this program with some success as well. If held in the auditorium, projection, networking, audio, is all available.

9. What are the characteristics of the classes and facilities that will use the new instruction?
While this material is exclusively for this orientation program, if migrated to online it would also be available outside the classroom via the student’s ThinkPad. The classes are completely composed of freshmen students.

10. What are the characteristics of the environment (school, library, workplace) in which the new instruction will take place?
Not sure how this differs from above.

11. What is the philosophy and what are the taboos of the larger community in which the organization or school system exists?
The greater University philosophy with regards to Freshman orientation is to exceed the needs of the students (my words, not the University’s) Thus a taboo would be to reduce the content delivered, unless that content can be proven extraneous.

12. Describe the learners.
The learners are traditionally 18 year old WFU freshmen with almost no exceptions.

2007-09-15T07:33:00.001-07:00

Proposal for Instructional Design Project

Our current model for Freshmen Technology Orientation (called "Technology@WFU") is a 75 minute presentation in a recital hall. Students attend this event two days after they receive their WFU issued Lenovo ThinkPad. We have two sessions as only half the freshman class can fit in the hall at a given time. We have 34 staff and 15 students who work this event. Seven of these people help give the program on stage and the rest help with the event logistics including loading and unloading the room.

This program contains a wide variety of topics in a wide variety of formats. For that past three years we have followed this "arena style" model. For two of those years, we have ended with a "Jeopardy" session with volunteers from the audience playing Jeopardy on stage. This past year we gave up Jeopardy and instead passed 60 "clickers" through the audience and during the session we had students answer questions and pass the clicker to the next student.

The outline for the session is as follows:

-Introductions
-Show the "ThinkPad Investigation" video that parodies CSI and covers "Care of the ThinkPad"
-ThinkPad Insurance and deductibles information
-Downloading and Copyright (Using the new "Campus Downloading" DVD -see http://www.campusdownloading.com/) We will be encourage the use of Ruckus (see http://www.ruckusnetwork.com/)
-Spamming, Spoofing, Phishing Spyware, Adware
-Pure Message, Sophos, Symantec, Spybot and Ad-aware, Thunderbird Junk Mail controls
-Internet Gambling
-Regular rebooting and getting updates on the wired network
-Logins and passwords
-Backing up your data
-Accessing the WFU campus network
-WFU Computing Policies
-Classroom technology etiquette
-What you need to know NOW(registration and WIN)
-Getting Help

For my project, I would like to apply the principles of Instructional Design to this program and consider creating it as an online program. If this program could be available in an online format, the University could consider making it available to the students over the summer prior to arriving on campus, and it would be available anytime of year for students who miss the session, or transfer students who arrive in January. Additionally we could reduce the need for staff time on a Saturday during the busiest time of year.

New Blog for ID Class!

2007-09-03T13:17:00.000-07:00

If you want to see my personal blog, see http://gizwomack.blogspot.com