[atug] letters between Desire2Learn and OSU, Winter quarter accessibility testing

[Ken Petri]

Hi ATUG Membership,

I thought I'd let you all in on something that may involve you next quarter.

As you know, we have been working through the Desire2Learn Consortium
at the University of Illinois Urbana-Champaign to try to get
Desire2Learn (D2L) to focus a bit more attention on accessibility. D2L
(locally known as Carmen) presents quite a number of problems for
users with mobility impairments and, especially, for users who rely on
screen readers to access the web.

This Wednesday (December 18) a small group of people involved with the
Consortium (Beck Andre, Alan Foley at Wisconsin, and Mark Felix at
Arizona, Don Amos at Pellissippi, and me) are meeting via
teleconference with Janna Cameron, the usability specialist at D2L,
and one of the D2L engineers to hash out plans for
usability/accessibility testing of version 8.2 of the product. (OSU
currently deploys 7.4 of D2L. 8.1 is under examination for roll out.
8.2 is a (we are told) significant upgrade to 8.1.)

The email thread below is a back and forth between Joanne Dehoney at
TELR, John Baker, the CEO of Desire2Learn, Beck Andre, and me. It
starts (reading from the bottom) with a letter from Beck that presents
the results of a session she and I had working with Steve Kuusisto to
see if it was yet possible for him to be able to use Carmen for his
classes. Joanne sent that letter to John Baker. Joanne's note and
John's reply are also included. The topmost response is from me and
refers to a testing session conducted by Beck, Kelvin Trefz, and me in
which we try out version 8.1.

I'm hoping these notes bring you up to speed on where we are in this
process and give you an idea of what testing will be occurring in the
Winter. I am hoping ATUG will be a good place to recruit some folks
for testing. Please let me know if you would like to be a part of
this--either in testing or in analysis and collation of results.

Happy Holidays!
ken

---------- Forwarded message ----------
From: Ken Petri <petri.1 at osu.edu>
Date: Dec 16, 2006 8:48 AM
Subject: Re: d2l letter
To: Joanne Dehoney <dehoney.1 at osu.edu>
Cc: Rebecca Andre <andre.1 at osu.edu>, kuusisto.1 at osu.edu, Kelvin Trefz
<trefz.1 at osu.edu>

Hi Joanne,

On Thursday, Beck, Kelvin Trefz, and I did a short walk through of D2L
version 8.1, the version OSU is considering for the next upgrade. We
used JAWS 8.0 on both Internet Explorer 7 and Firefox 2 (which now
works quite well with JAWS). I drove.

While there seem to be minor improvements in this new D2L version,
they are not very significant, and implementation of features
conducive to accessibility is highly uneven.

For example, the skip navigation link at the top of pages still does
not work. All of the pages we tried had no headings. The frames are
still improperly named (there are two "navigation" frames and all
frames are called "frame," so JAWS reads "contents frame frame,"
etc.). On one of the minor administrative interfaces, we found labels
improperly used and one case in which a label for an input was read
aloud but it was the wrong label--a bad association, which, of course,
would cause the wrong information to be entered by a screen reader
user. In one of the forms for adding a discussion forum, the date
fields were not unlabeled--there was a label for the start of the
month, day, year triplet, but no individual labeling of the fields
themselves, which again would add confusion to an already overly
complicated interface. There were still a number of missing alt
attributes in images, and a wealth of links called "hide" and
"show"--hide what? show what? These are problems that at this stage of
the game, after D2L has received two yearly reports from the WAC and
other intermittent fixes and recommendations, should certainly no
longer be present.

After creating a forum topic (for any user an overwhelmingly
non-intuitive operation that ought to be revised and streamlined), we
then returned to the discussion page and tried to read the forum
topic, itself. The discussion page has no headings. So this was one
hoop to have to find a way to jump through. We skipped to the
"contents frame frame" (via the the "m" shortcut key in JAWS) and then
started reading. The organization of the interface elements forces one
to read through all of the function buttons first--"add a topic,"
etc.--before getting to the clickable forum topic itself, which is
buried in nesting created solely via presentational rather than
structural markup. Had there been headings as way points and the forum
been implemented as, say, a definition list or series of headings with
unordered lists or a well structured table below, there might have
been a way for a screen reader user to determine the structure of the
discussion forum page. But the HTML used is geared overtly toward
presentation rather than structure, making screen reader navigation
overly difficult and frustrating.

I want to stress that the problems that are apparent in this version
of D2L, as in previous versions, are definitely not irresolvable. In
fact, most of the fixes require only a common sense notion of
usability and accessibility and a decent working knowledge of HTML
structures and when they ought to be used.

That is what we found with only a very surface perusal of the application.

It is apparent some minor improvements have been made--labels added
(though not always properly), etc. But overall these don't have much
positive impact on accessibility for the screen reader user. I think
both Beck and Kelvin would concur.

As you know, the D2L Consortium has been arranging to perform
accessibility testing of the 8.2 Desire2Learn release, which, we are
told by Janna Cameron and others at D2L, has significant redesigns of
various widgets and interfaces. Janna has said that there a many
changes between 8.1 and 8.2. The testing proceeds in two parts.

Part one is a per page view analysis of a number of pages. The matrix
into which expert testers (me, Alan Foley at Wisconsin, and Mark Felix
at Arizona are the initial group) will be entering our results is
geared toward functional accessibility--not checking off of 508
guidelines or whatever but analysis of page structures that provide
functional accessibility for screen reader users, users with
low-vision, and users with mobility impairments.

After that phase of testing, we will be doing usability testing with
disabled users. Steve Kuusisto has volunteered, along with Michele
Solomone (another JAWS user). I imagine Pete Bossley will be involved
as well, and we will be leveraging ATUG, Unity, and ODS to gather a
small core of testers with a range of disabilities. We also hope that
we get some testers from other universities involved. We are in the
process of developing formal protocols for the testing and we will be
recording screen capture and audio (and in some cases video) feedback
from testers. We will also be using special software to analyze use
patterns (number and region of clicks, time on task, etc.).

My sense is that after this round of testing--which will take up a
large chunk of the Winter quarter--we will have given D2L a very
precise set of strategies and fixes for making the product accessible.
We should also have a clear sense of their real commitment to
implementing changes and a time line for that.

Though my office and the good people involved in ATUG have certainly
been learning a lot about the process of testing, working with
vendors, and the intricacies of LMS systems, there will come a point
of diminishing returns. I'm not sure how to express that to the people
at Desire2Learn. I hope it will become clear to them in our testing
over the Winter that, one, there are serious problems with their
product's accessibility that go beyond simple, check-point driven
patching and filling, and, two, that OSU will not wait indefinitely
for real solutions to be delivered.

Rgrds,
ken

On 12/14/06, Joanne Dehoney <dehoney.1 at osu.edu> wrote:
> Here is John's response. Ken & Steve, we do have 8.1 MR 1 up on a
> server to begin the various tests and explorations that are required
> to make a major upgrade. I suggest that we complete accessibility
> testing on it as a first priority. Can you give me a sense of the
> timeline for this?
> Joanne
>
> Begin forwarded message:
>
> > From: John Baker <john at desire2learn.com>
> > Date: December 13, 2006 1:11:22 AM EST
> > To: Joanne Dehoney <dehoney.1 at osu.edu>
> > Cc: Susan Metros <metros.1 at osu.edu>, Dana Frigula
> > <dana.frigula at desire2learn.com>, Kenneth Chapman
> > <Kenneth at desire2learn.com>
> > Subject: RE: d2l letter
> >
> > Hi Joanne,
> >
> > I think you'll find our product management team (Ken and others) at
> > our company willing to work very closely with you on these items.
> >
> > I know our usability specialist is already working very closely
> > with your university and others on testing the 8x framework.  We
> > also have put in place strong commitments from our Dev and QA teams
> > to ensure we address all P1, P2 issues and are working hard to
> > ensure we cover as many P3's.  Most are already addressed.
> >
> > Ken and his team will be working with you to constantly improve the
> > system and I believe you will see very significant strides forward
> > in our 8-series releases.
> >
> > We take these concerns very seriously and that's why we have put so
> > much effort into making improvements in 7-series and major steps
> > forward with accessibility built into the UI components in 8-
> > series.  We look forward to continuing to work collaboratively so
> > narratives like the ones below will be very quickly a thing of the
> > past.
> >
> > John
> >
> > -----Original Message-----
> > From: Joanne Dehoney [mailto:dehoney.1 at osu.edu]
> > Sent: Tuesday, December 12, 2006 6:38 PM
> > To: John Baker
> > Cc: Susan Metros; Dana Frigula; Kenneth Chapman
> > Subject: Fwd: d2l letter
> >
> > Dear John,
> >
> > I'm forwarding you an interesting narrative from one of our
> > instructional designers, Beck Andre, who recently attempted to assist
> > a blind professor who wished to use the current production instance
> > of Carmen, D2L 7.4.3 MR1. As you will see, he was unable to use the
> > system. Notably, it took ~40 minutes~ to navigate the MyHome page to
> > the point of opening the correct folder for Winter quarter classes.
> > Once he accessed a class, it was prohibitively difficult to do basic
> > setup.
> >
> > JAWS itself is somewhat finnicky about tuning, so we did have our
> > accessibility expert, Ken Petri, on hand to assure that this aspect
> > was completely supported.
> >
> > We have been working closely with D2L on accessibility since the
> > beginning of our implementation, and I would expect us to be further
> > along by now in addressing the needs of the entire learning community.
> >
> > We will begin an accessibility audit of the 8 platform in January.
> >
> > Thanks,
> > Joanne
> >
> >
> > Begin forwarded message:
> >
> >> From: Rebecca Andre <andre.1 at osu.edu>
> >> Date: December 12, 2006 5:53:32 PM EST
> >> To: 'Joanne Dehoney' <dehoney.1 at osu.edu>
> >> Subject: RE: d2l letter
> >>
> >> Dear Joanne:
> >>
> >> I'm writing to relay recent experiences gained during an attempt to
> >> use Carmen with JAWS 8, popular screen-reading software. The bottom
> >> line is that Carmen does not appear meet the minimum needs of those
> >> who use screen readers.
> >>
> >> Ken Petri, Director of Web Accessibility Services, and I met with
> >> Professor Steve Kuusisto, a professor of English who is blind and
> >> interested in using Carmen. Professor Kuusisto is agile with
> >> technology and appreciates many products that enable him to
> >> communicate and share documents with students. He wishes to utilize
> >> Carmen for many reasons, including multiple ways to communicate with
> >> students and time-saving options such as publishing grades directly
> >> to the Registrar's database.
> >>
> >> During our meeting, we began with a discussion about what tools or
> >> features Steve was most interested in using. Like many faculty, he
> >> expressed interest in sharing his syllabus and other documents such
> >> as assignments, the News tool, discussions, and the gradebook.
> >>
> >> Ken and I needed a demonstration of Steve's process. So we sat behind
> >> Steve as he expertly navigated JAWS in an effort to access Carmen.
> >> JAWS reads the ALT attribute of all images, all page text, and can
> >> move between elements such as headings, frames, and links via menu
> >> systems that the user can call up or special keystrokes. Users can
> >> also use JAWS to search for key bits of text within pages, moving to
> >> the text found. Typically, a user will navigate through the page
> >> using these cues, keystrokes, and menus, listening for an action to
> >> take or some significant text-a link to click or a box in a form to
> >> edit. The process was quickly bogged down by either redundant or
> >> overly generic ALT attributes, poorly titled frames, and a lack of
> >> headings to use as waypoints.
> >>
> >> In about 40 minutes, with a few verbal prompts from us and a lot of
> >> trial and error, Steve was able to locate and click the plus sign
> >> next to winter quarter to expand the list of his courses. Steve
> >> entered one winter course shell and JAWS located no language to
> >> assist the next step, such as "activate" or "set date" under the Edit
> >> Course button. When Ken prompted, Steve attempted to add dates to the
> >> fields, but because the end date was not clearly identified to him,
> >> the system locked up a couple of times when he tried to choose a date
> >> from the drop-down menus. As mentioned, JAWS reads most D2L links
> >> twice (icon plus the item link itself), forcing Steve to tab through
> >> all the items on the screen to locate the "save" button. Tabbing
> >> through JAWS "form field" option finally yielded the "save" function.
> >> Several places weren't consistent with earlier discovered patterns so
> >> Steve could neither rely on his memory of patterns or JAWS yielding a
> >> usual set of options from which to choose.
> >>
> >> After about 90 minutes, we took a collective breath, sat back in our
> >> chairs and began to strategize about other options. In the short-
> >> term, Professor Kuusisto will again rely on a department support
> >> person to build his course websites outside of Carmen, where he can
> >> at least navigate around and locate course materials. Long-term ideas
> >> included: 1) exploring the next D2L version, 2) exploring other LMS-
> >> type software, such as Moodle and A-Tutor, 3) meeting in person with
> >> D2L staff (there or here), and 4) assigning an eLearning staff or
> >> student employee to work closely with Assistive Technologies,
> >> developing solutions and documentation.
> >>
> >> Collectively, we thought it was a worthwhile exploration and wanted
> >> to share our concerns. We are interested in expanding the discussion
> >> with you soon and into the future.
> >>
> >> Thanks,
> >>
> >>
> >>
> >> Rebecca Andre, Ph.D.
> >> eLearning Manager and Adjunct Assistant Professor
> >>