1. Institute of Fundamental Sciences, Massey University, New Zealand
  2. Department of Statistics, University of Auckland, New Zealand
  3. Department of Computer Science, University of Birmingham, United Kingdom

Keywords: accessibility, exploration, interactivity

Introduction

Descriptions of graphs using long text strings are difficult for blind people and others with print disabilities to process; they lack the interactivity necessary to understand the content and presentation of even the simplest statistical graphs.

My work on the BrailleR package started about five years ago.

library(BrailleR)

My stock standard example:

attach(airquality)
MyHist <- hist(Ozone, xlab="Ozone (ppb)")
The histogram

The histogram 

VI(MyHist)
This is a histogram, with the title: Histogram of Ozone
"Ozone" is marked on the x-axis.
Tick marks for the x-axis are at: 0, 50, 100, and 150 
There are a total of 116 elements for this variable.
Tick marks for the y-axis are at: 0, 10, 20, and 30 
It has 9 bins with equal widths, starting at 0 and ending at 180 .
The mids and counts for the bins are:
mid = 10  count = 37 
mid = 30  count = 34 
mid = 50  count = 14 
mid = 70  count = 15 
mid = 90  count = 9 
mid = 110  count = 4 
mid = 130  count = 2 
mid = 150  count = 0 
mid = 170  count = 1
detach(airquality)

N.B. the hist() function is included in BrailleR to do a few extra little jobs that collect the necessary bits and pieces that form the plot. It still calls the base R graphics::hist() function.

BrailleR::hist
function (x, ...) 
{
    MC <- match.call(expand.dots = TRUE)
    MC[[1L]] <- quote(graphics::hist)
    Out <- eval(MC, parent.frame())
    if (length(MC$main) > 0) 
        Out$main = as.character(MC$main)
    if (length(MC$sub) > 0) 
        Out$sub = as.character(MC$sub)
    if (length(MC$xlab) > 0) 
        Out$xlab = as.character(MC$xlab)
    if (length(MC$ylab) > 0) 
        Out$ylab = as.character(MC$ylab)
    Out = Augment(Out)
    return(invisible(Out))
}
<environment: namespace:BrailleR>

Until very recently, R has been the only statistical software that has any capacity for offering the print disabled community any hope of support with respect to accessing graphs.

SAS launched its tool called the graphics accelerator in March this year. It purports to be able to handle any graph found on the web that was generated by SAS. The user clicks on the SAS graph and opens it in a new webpage with the accelerator wrapped around the graph. It is undoubtedly a major step change in the world of access to graphs for blind and other print disabled people.

You can look on YouTube for a demo video introduced by a blind SAS employee (Ed Somers).

I would argue that its main focus is to support blind people viewing the work of others, but there is scope for it to assist print disabled SAS users.

In 2016, another tool was released for MS Excel users wishing to create and explore graphs. It is delivered to end users as an add-on for the open source screen reading software known as NVDA. Many blind people use this software, but it is only a solution for Windows users. Similar attempts to create tools for the commercial screen reader known as JAWS have come and gone without as much success.

My personal focus has always been to improve the educational and employment prospects of blind people. In an information age, there is an expectation that people will process information presented in a variety of forms, and to a lesser degree, be able to create that same content. I want a solution that is:

We have levered off the ability to create text descriptions of graphs and the ability to create interactive web content for chemical diagrams to offer a new user experience.

In February 2016, I was in the audience for a keynote presentation by Volker Sorge on his work to create accessible chemical diagrams. That solution had an interface that was:

I set about finding out how I could do work with Volker and what I’d need to get from R to make it all happen. I ended up spending two days with Volker when we visited a blind computer scientist in Dublin during December. Our host (Donal Fitzpatrick) deserves a mention for his ability to get me to start speaking the right language to survive my encounter with computer scientists.

The building blocks

We will present the necessary tools that (1) produce the desired graph in the correct form of a scalable vector graphic (SVG) file, (2) create a supporting XML structure for exploration of the SVG, and (3) the javascript library to support these files being mounted on the web.

Scalable vector graphics (SVG)

SVG is a very powerful means of presenting graphs produced in R, but not all SVGs are equal!

There is a standard for SVG developed by the World -Wide Web Consortium (W3C). Unfortunately, not all SVGs created in R meet this standard svg(); while others meet the letter of the standard, but arguably do not meet the intent of the standard; while still more do meet the intent of the standard `ggsave(), but do not create the SVG content we need. Enter Paul Murrell.

The gridSVG package and other supporting packages create the hierarchical structure that provides the semantics we need to be able to add content to the graph, or to extract the details of the rendered graph needed in our post-processing described below.

The result is a file that is longer than other SVG, but it is much richer.

To make the SVG of a graph created using the graphics package e.g. boxplot(), hist(), plot() means use of Paul’s gridGraphics package to convert a base graphic to an identical image built using the grid system.

Paul also had to perform some non-invasive surgery on the gridSVG package to make some of our examples work.

XML

BrailleR will build an XML file to sit alongside the SVG for the graph.

We build a hierarchical structure for the user to “walk” for their exploration. Each node can have one parent, children, and siblings, walked using the up, down and left/right arrow keys respectively. Pressing the up arrow key several times takes the user back to the head node.

Each node has two different text strings attached to it that are the feedback you will soon see and hear. Scope exists for altering the text for braille displays but this is not yet fully implemented. This text builds on the text given earlier.

This tree structure works for most statistical graphs, but future development will look at other data displays that do not lend themselves to the current hierarchy, such as Venn diagrams and directed networks.

javascript?

The .js file is shipped with the current release of the BrailleR package. It controls the information and consequences of key presses when viewing the graph in a web browser.

It is not open source, but as part of any webpage it is open access.

gluing it all together

The three components (SVG, XML, and js) are then combined into an HTML file for viewing. Viewing is done interactively if the R session is interactive.

Playtime

Demonstration of how a blind user can explore the graph by “walking” a tree-like structure will be given. A key enhancement is the ability to explore the content at different levels of understanding; the user chooses to hear either the bare basic factual description or a more descriptive layer of feedback that can offer the user insight.

The additional work is fairly small:

attach(airquality)
MakeAccessibleSVG(MyHist <- hist(Ozone, xlab="Ozone (ppb)"), file="OzoneHist")
detach(airquality)

and let’s see what happens!

Problems

The work ahead

We need to: