Enhancing accessibility in written communications: A review of forms and letters for the Social Security Tribunal
Dr. Julie Macfarlane for the National Self-Represented Litigants Project
June 17, 2021
Acknowledgements
This project was worked on by a number of members of the NSRLP team.
NSRLP Research Assistants Negin Shahraki and Silvia Battaglia, supervised by NSRLP Communications Manager Moya McAlister, drew a sample based on SST stipulations from the NSRLP’s National SRL Database and promoted participation in the study on our social media channels. Negin and Silvia then recruited participants to the study and acted as their liaison and point of contact throughout the review process.
Dayna Cornwall (NSRLP Project Manager) and Moya McAlister conducted the readers’ focus group.
Under my supervision, Research Assistant Hannah Thackery did a terrific job of crunching the data and collating and setting out the quantitative results of the reader reviews, and Research Assistant Charlotte Sullivan ably coded the reader comments from both the checklists and the focus group. I wrote the Draft and then the Final Report with input from Dayna and Moya. All remaining mistakes are my responsibility.
As always, this was a team effort!
From inside the Tribunal, we were facilitated by Glenn Ng and encouraged throughout by Paul Aterman, whose support we greatly appreciate.
Finally, our grateful thanks to those (former) self-represented individuals whose willingness to work on making the experiences of navigating the legal system better for those coming after them makes this work possible.
Julie Macfarlane Kingsville
June, 2021
Executive summary
Those with prior experience of representing themselves in either courts or tribunals were asked to review 8 letters and 2 forms (equivalents in both English and French) that the Social Security Tribunal regularly sends to claimants. The focus of their review was the user-friendliness, completeness, and overall accessibility of both the letters and forms in order to better serve the approximately 75% of claimants who come to the Tribunal without legal counsel. To that end, readers were asked to complete an individual checklist of both ratings and threshold questions on each letter and form, and to add additional comments and suggestions for enhancements.
A total of 15 readers reviewed the forms and letters in English and 6 in French. There were challenges recruiting French readers due to the relatively small numbers of Francophone SRLs currently in the NSRLP SRL Database. We also experienced some overall attrition among those initially recruited who found the task time-consuming and challenging. A focus group was held to enable further discussion of the forms and ways to improve their accessibility.
The readers’ response to the letters and forms was overall a positive one, with many commenting that these documents were clearer and easier to follow than those they had been confronted with when they made their own prior court or tribunal application. Both the letters and the forms scored highly on the explanation of legalese and other unfamiliar terms (although we note there were some concerns about the everyday meaning of “Navigator”), and organization and structure. Both the letters and the forms scored less well on the question of “reasonable length,” with the letters in particular singled out as overly wordy and lengthy by 20% of readers in both languages (interestingly, there was greater acceptance of the length of the forms, possibly reflecting a prior expectation and/or experience with court and tribunal forms).
The most negative comments made about the letters, in both languages, related to the outstanding questions a lay reader might have. Readers made many specific suggestions for improvements, including links to more information on the SST website in order that a reader could click through if they needed more explanation. This was also the area of most negativity in relation to the forms, with even higher numbers of readers saying that they had unanswered questions. These readers suggested drop down menus, sample completed forms for illustrative purposes on the website, and perhaps the addition of a “Live Chat” function to assist self-represented claimants.
The “soft” aspect of the forms and letters – visual appeal and accessibility, and the overall tone of the communications (letters only) – also attracted some negative reviews and many suggestions for improvements. Many readers pointed to what they felt was the bureaucratic and even “cold” tone of the letters. Many also proposed the greater use of visuals including flow charts, infographics, and short videos. The addition of such devices – which are widely popular among self-represented litigants in our experience – would reflect the inevitable range of both educational level and learning styles among self-represented claimants.
The relatively recent innovation of the Navigator program may assist in resolving some of these questions for self-represented claimants (obviously a Navigator was not available to these readers). Notwithstanding, the detailed results and comments set out in this report highlight ways in which the Tribunal might further enhance the effectiveness of its arms-length communication with self-represented claimants, and resolve some issues before they need to be brought to a Navigator. For some claimants, contact with an individual specialist such as a Navigator may be a more attractive option than for others, again depending on learning style as well as access to the internet and overall confidence and familiarity using online resources.
1. Study Methodology
1.1 Background
The Social Security Tribunal ( SST ) has undertaken a number of steps to enhance its accessibility for those coming without representation, who comprise approximately 75% of those applying to the Tribunal. These include the establishment of a Navigator program in November 2019 for claimants without counsel in the General Division and in November 2020 for claimants in the Appeal Division, which is currently being independently evaluated by Professor Laverne Jacobs and Professor Sule Tomkinson; an internal evaluation by the SST reporting shortly; a plain language revision of letters, forms and website materials, which began in June 2019; and a review of SST processes using the Department of Justice’s A2J Index tool.
This review of the accessibility and clarity of selected forms and letters continues this commitment to enhancing the experience of self-represented claimants ( SRC s) at the SST. In particular, this review connects to the earlier readability assessment by an independent plain language expert, which has already resulted in significant revisions and changes in standard letters and forms.
The forms and letters selected for this evaluation are representative of a wide range of such documents that claimants will typically encounter as they progress through the Tribunal process. This evaluation brought in the additional and critical perspective of members of the public who had also been self-represented litigants, as well as members of the NSRLP team who have deep experience working with the public and developing materials on legal topics that are accessible, clear, and welcoming (see for example NSRLP’s Primers series). Review from actual or potential lay claimants is important to supplement professional reviews including those by legal professionals familiar with this material, to catch any lacuna in explanations of legal terms, missing process information, and to provide detailed comments on the general style of writing and engagement from the perspective of the public.
Note on terminology: the term self-represented claimants ( SRCs) refers to applicants and claimants specifically in relation to the SST. We use the broader term, self-represented litigants ( SRLs), when referring to those people using the justice system without a lawyer in general, and to all participants who took part in this evaluation, who may or may not have had a specific experience at the SST.
1.2 Evaluation questions and scope of work
The purpose of this evaluation is to assess the accessibility and “ SRC-friendliness” of a selection of documents, including standard form letters and forms developed by the Social Security Tribunal. While navigating a tribunal or court process is challenging without expert legal representation, the goal as NSRLP understands it from the SST Chairperson, Paul Aterman, is to make the application and appeals process as manageable as possible for those without counsel. The letters they receive from the Tribunal and the forms they must complete are obviously a critical part of this.
This work has sought to build on work already done and continuing at the SST, especially the readability assessment. It is, however, limited to external assessments, from SRLs/former SRLs and others ( NSRLP team members) without familiarity with SST processes, and does not include further evaluation from internal staff.
1.3 Evaluation methodology
1.3.1 Participant recruitment
In order to capture as much feedback as possible, NSRLP undertook an extensive recruitment exercise using the National SRL Database and our social media platforms to solicit participants. As agreed with SST, a small number of NSRLP staff and research assistants were also recruited to the sample.
The National SRL Database has the capacity to sort and identify entries according to a range of demographics, and we used the preferred parameters provided to us by the SST. These were former or current self-represented litigants (Canada-wide but with a primary focus on Ontario, Quebec, and BC), both with and without direct experience in administrative tribunals. The participant group was drawn to ensure a wide range of ages, ethnicities, education, and income levels, as well as retired persons and persons with disabilities.
Once an individual agreed to participate, they were sent a package which included a short explanation of the project, its goals and what they would be asked to do (in either French or English, according to their preference). This was developed in co-operation with the communications team at the SST. This package also included details on who at NSRLP would be a point of reference and support for the participant throughout the project (one of two research assistants assigned to the project) and an explanation of how to access the relevant documents on Google Drive (where participants found this onerous, we sent them the documents directly).
The recruitment process was challenging for several reasons. One was that some SRLs signed up to participate, but clearly felt a little overwhelmed by the task once they confronted it (reviewing 8 letters and 2 forms in either English or French). Some did not complete, despite our efforts to follow up and support them.
Second, this project underscored the lacuna of francophone SRLs in our National Database. We made many additional efforts to recruit francophone participants, but some of the access to justice agencies to whom we turned expressed a similar view about the onerous nature of the task. NSRLP is undertaking a targeted outreach in Quebec and to Francophone Canadians, especially in Ontario, which we hope will address this problem in the future.
Despite these challenges, a total of 21 individuals (henceforth “readers”) completed the review of the letters and forms (15=English, 6=French1) and provided a great deal of both quantitative and qualitative data.
1.3.2 Checklist design
In order for readers to assess the assigned letters and forms, we designed two checklists (which were as far as possible mirror images of one another, see Appendices A & B).
The checklists collected a combination of numeric/quantitative data and qualitative data, via space to add written comments.
Quantitatively, the checklists employed two discrete approaches to assessment.
- “Threshold” questions in each case asked for a yes/no response and also provided space for further commentary which we then coded for themes. The threshold issues tackled in this way were:
- Comprehensibility – that is, were all legal terms and expressions explained, and in a way that made sense to the reader?
- Structure – that is, was the letter or form set out, organized, and structured in a way that was easy for the reader to follow?
- Length – that is, was the length of the letter or form and amount of text reasonable, or was did it feel too long and/or too unnecessarily wordy?
- The Forms checklist included an additional unique threshold question, i.e.:
- Ability to complete – that is, were there any parts of the form that the reader found they could not complete and/or did not understand how to complete?
- Each threshold (yes/no) question included a space for readers to add comments, and these responses were coded and themed (see detailed findings below at Appendix E).
- “Rating” questions asked the reader to rate the letter or form as “Excellent,” “Good but,” or “Poor” in relation to:
- Answering questions that a self-represented person might have (some examples of typical questions were provided). Suggestions for unanswered questions were invited.
- The visual layout of the form. Suggestions for improvements were invited.
- Overall clarity and accessibility (summative evaluation).
The Letters checklist included an additional unique threshold question about the tone of the communication and whether it was welcoming and encouraging.
Each rating question included a space for readers to add comments, and these responses were coded and themed.
Finally, the Forms checklist asked an additional question about the length of time it had taken for the reader to complete the form. These estimates were then banded and coded (see detailed findings below).
1.3.3 Review of letters
The SST provided 8 letters, identical in English and French, that they wished to be reviewed. Using the checklist (above), 15 readers reviewed all the letters in English and a further 6 readers reviewed all the letters in French.
1.3.4 Form completion
The SST provided 2 forms, identical in English and French, that they wished to be reviewed (one for the General Division and one for the Appeal Division). Using the checklist (above), 15 readers reviewed the forms in English and 6 in French.
We asked readers to time how long the completion of each form took them. This gave us a sense of how much time it took to complete the form and we have provided the average time in each case (General Division, Appeal Division, French, English). It should be noted however that readers did not complete those sections of the form that related to the facts of an actual case, since obviously this was hypothetical. While we have no way of knowing this, it should be assumed that completing these sections also would have taken considerably more time.
1.3.5 Focus group
We invited those who had completed the forms to attend a focus group to describe their experience in more detail. While 5 SRLs signed up to attend, unfortunately only 3 participated, but they provided a great deal of rich additional detail.
The focus group questions are at Appendix D.
These discussions were contemporaneously noted by a research assistant and the data coded for themes and patterns. These are discussed at (4) below and with the remainder of the qualitative data.
2. Reader reviews of the letters
In this section, we present the results of the reviews of the 8 letters (in both French and English) that the SST provided to us.
In breaking down and presenting each layer of data, we have made the assumption that the most useful data for the Tribunal is information that highlights where further improvements in their written communications can be made. Therefore, we have mostly focused on question topics and letters that attracted the most negative appraisal.
2.1 Threshold questions
Threshold questions asked readers to respond either “yes” or “no” to questions about the terminology, structure, and length.
2.1.1 Aggregated overview of results
We first present the responses to the “threshold” questions as aggregated data (all results for all letters in both languages). This provides an initial overview of how far our readers assessed the letters as meeting these important thresholds.
The letters generally scored well on all the threshold questions, with at least 80% of readers answering a positive “yes” to each of the threshold questions on terminology, structure, and length. The results for each question follow.
Question 1: Are legal and other unfamiliar terms, abbreviations, and acronyms explained?
Text version
Question 1 – All letters, both languages
Yes: 84%
No: 16%
Question 3: Is the letter organized and structured in an order that would make sense to the reader, using appropriate headings, etc.?
Text version
Question 3 – All letters, both languages
Yes: 87%
No: 13%
Again, a very positive score on organization and structure.
Question 4: Is this letter a reasonable length – not too long and unnecessarily wordy, but also not too concise to be useful?
Text Version
Question 4 – All letters, both languages
Yes: 80%
No: 20%
On the question of length, 80% of the reviews indicated satisfaction with the letters. However, at 20% this question drew the most negative (“no”) responses.
2.1.2 Question-by-question analysis disaggregated by language
In both French and English, the question that drew the most negative response (the most “no”s) was the question relating to length (question 4: “Is this letter a reasonable length – not too long and unnecessarily wordy, but also not too concise to be useful?”)
Comments included the following:
“As there is a navigator available, the [letter] could be pared down to basic information, such as timelines and expectations, and the rest left to the navigator to provide.”
This is obviously a difficult balance to strike, since there is a certain amount of critical information, which the Tribunal wishes to convey, while avoiding overwhelming the reader. Further detail follows below at section 4.3.
2.1.3 Letter-by-letter analysis disaggregated by language
Finally, we have isolated the specific letters, in both English and French, that drew the most criticism in relation to each question, because we assume that the most useful data for the Tribunal is information that highlights where further improvements in their written communications can be made.
Threshold question | Most negative score: English | Most negative score: French |
---|---|---|
Question 1 (terminology) | Leave NOH teleconference | Acknowledgement of complete leave to appeal, late |
Question 3 (organization) | Navigator call follow up | [No negative scores] |
Question 4 (length) | Decision letter Appeal Division | Acknowledgement of complete leave to appeal, late; Leave NOH teleconference |
2.2 Ratings questions
Ratings questions asked readers to respond either “excellent” or “good but needs improvement” or “poor” to our enquiries about how well the letter answers questions SRLs might have, visual appearance and format, overall clarity and accessibility, and tone.
2.2.1 Aggregated overview of results
We first present the responses to the 4 “ratings” questions (how well does the letter answer the questions SRLs might have, visual appearance and format, overall clarity and accessibility, and tone) as aggregated data (all results for all letters in both languages). This provides an initial overview of how well our readers scored the letters on these important aspects of communication.
Question 2: How well does the letter answer the questions that you imagine a self-represented person might have?
Text Version
Question 2 – All letters, both languages
Excellent: 55%
Good but: 35%
Poor: 9%
N/A: 1%
This question attracted the most negative (“poor”) responses from aggregated French and English readers in their evaluation of the letters (12%).
Comments included the following:
“I had so many questions! How do you ask a Federal Court to review? What is judicial review? Why would I ask for a review?”
Question 5: How would you rate the visual layout of the letter?
Text Version
Question 5 – All letters, both languages
Excellent: 63%
Good but: 32%
Poor: 5%
There were many suggestions made for improving the visual impact and accessibility of the letters, which are detailed in Appendix E. A number of readers singling out the usefulness of the flow chart/infographic attached to one of the letters.
Question 6: Overall, is the letter written clearly and in a way that is accessible?
Text Version
Question 6 – All letters, both languages
Excellent: 79%
Good but: 16%
Poor: 4%
N/A: 1%
This was the question that received the most “excellent” scores from aggregated English and French readers.
Question 7: Finally, does the letter have a welcoming, encouraging tone?
Text Version
Question 7 – All letters, both languages
Excellent: 47%
Good but: 40%
Poor: 10%
N/A: 3%
There were many comments made about the bureaucratic feeling and formal tone of the letters. All such comments and suggestions for improvements are coded and included in Appendix E.
Typical comments included:
“It is never welcoming or encouraging to make people work harder than necessary to understand what they need to know.”
“No greeting or conclusion, the letter begins and ends very abruptly.”
“This continues to be the main problem here. Just the way it reads (and does not have to read) seems like you don’t really care — just trying to follow procedures.”
2.2.2 Question-by-question analysis disaggregated by language
In English, the question that drew the most negative response (the most “poor” rankings) was the question relating to how well the letter answered the questions that a self-represented person might have (question 2).
The areas that readers felt were insufficiently explained in order to address SRL concerns are coded and set out in full detail in Appendix E.
An example comment:
“Really could do a much better job of explaining all the [terminology], how to appeal, what the appeals process will look like. If it would make the letter too long to include all this, the letter should link out to this information.”
In French, the question that drew the most negative response (the most “poor” rankings) was the question relating to tone (question 7).
Some of these comments included: (also above)
Readers generally liked the overall clarity and accessibility of the letters (question 6), and both English and French readers gave this question the most “excellent” scores.
2.2.3 Letter-by-letter analysis disaggregated by language
Finally, we have isolated the specific letters, in both English and French, that drew the most criticism in relation to each ranking question. While this highlights negative comments, we assume that the most useful data for the Tribunal is information which identifies where further improvements in written communications can be made. Further detail follows in section 5 below and in our final recommendations.
Ranking question | Most negative score: English | Most negative score: French |
---|---|---|
Question 2 (SRL questions) | Acknowledgement of complete leave to appeal, late | Decision letter Appeal Division |
Question 5 (visual) | Acknowledgement of notice of appeal; ESDC docs and next steps in appeal; Navigator call follow up; Notice of hearing teleconference |
Acknowledgement of notice of appeal |
Question 6 (overall clarity) | Leave NOH teleconference | [No negative scores] |
Question 7 (tone) | Acknowledgement of notice of appeal | Decision letter General Division; Appeal Division form; Acknowledgement of complete leave to appeal, late |
3. Reader reviews of the forms
In this section, we present the results of the reviews of the 2 forms in French and in English that the SST provided to us.
Just as with the letters, we have assumed that the most useful data for the Tribunal is information that highlights where further improvements in their written communications can be made. Therefore, we have mostly focused on question topics and forms that attracted the most negative appraisal.
3.1 Threshold questions
Threshold questions asked readers to respond either “yes” or “no” to questions about terminology, structure, length, and their ability to complete the entire form.
3.1.1 Aggregated overview of results
We first present the responses to the “threshold” questions as aggregated data (all results for all forms in both languages). This provides an initial overview of how far our readers assessed the forms as meeting these important thresholds.
Question 1: Does the form explain any legal and other unfamiliar terms, abbreviations or acronyms, or other references?
Text Version
Question 1 – All forms, both languages
Yes: 51%
No: 49%
This is the first area of evaluation in which we see a significant dissatisfaction and some confusion among readers, with almost half commenting that the two forms they reviewed (either in French or in English) contained terms that they did not understand and/or were not adequately explained.
Some illustrative comments included:
“‘Navigator’ is needlessly complicated and repeatedly obfuscates the role of the individual. A more common word is ‘facilitator’ or ‘appeals assistant’ or ‘guide’ or ‘coordinator.’”
“All the terms used as to the grounds for appeal are not known by everybody unless they know something about Administrative Law — Procedural Fairness, error of jurisdiction, error of law, error of fact…”
Question 3: Is the form structured and organized in an order that would make sense to the reader, using appropriate headings, etc.?
Text Version
Question 3 – All forms, both languages
Yes: 86%
No: 14%
A largely positive result, with just 14% saying that they thought that the organization and structure of the form could be improved.
Negative comments included:
“Critical information is not properly emphasized. For example, the navigator should be introduced as Navigator: X; Contact info: X, etc. Don’t make people search through pages of text for the most important points. What comes next?”
“Attachments is noted but not how many, or the # pages, doing so would ensure the person knows they got the complete intended correspondence.”
Question 4: Is the form a reasonable length – not too long and unnecessarily wordy?
Text Version
Question 4 – All forms, both languages
Yes: 89%
No: 11%
Interestingly, there was less negativity about the length of the forms than the letters (see above section 2.2.1).
Question 7: Finally, was there any part/section of the form that you did not know how to complete?
Text Version
Question 7 – All forms, both languages
Yes: 29%
No: 71%
Almost one third of readers reported (“yes”) that there was a part of the form that they could not complete.
Comment examples included:
“Did not know what the term leave means or summary dismissal decision and whether it applied to my situation.”
“Mostly it was about justifying the appeal or its lateness — fuller understanding of what might make the application valid or the lateness acceptable would help. I am not sure for example of how my late application might disadvantage others.”
3.1.2 Question-by-question analysis disaggregated by language
Almost three quarters of the readers (73%) of the English forms said they were able to fully complete the form, compared to 60% of the French readers. (However, note the smaller sample in French, n=5).
The question attracting the most positive “yes” scores for the French forms were Questions 1 (“were terms explained?”) and 3 (“is the form well organized and structured?”) with all 5 readers answering “yes” to each question. English readers were also very positive in relation to Question 3 (with 83% answering “yes”) but significantly less positive in relation to Question 1 (with just 31% answering “yes”).
See full details at Appendix E.
3.1.3 Forms disaggregated by language
Threshold question | Most negative score: English | Most negative score: French |
---|---|---|
Question 1 (terminology) | General Division form | General Division form |
Question 3 (organization) | General Division form | [No negative scores] |
Question 4 (length) | General Division form | [No negative scores] |
Question 7 (completion) | General Division form | General Division form |
3.2 Ratings questions
3.2.1 Aggregated overview of results
Question 2: How well does the form provide answers to questions that you imagine a self-represented person might have?
Text Version
Question 2 – All forms, both languages
Excellent: 46%
Good but: 46%
Poor: 5%
N/A: 3%
This question elicited mixed results for the forms, with almost half the sample suggesting a need for improvements. A specific suggestion made by several readers was that contact information for where to go to ask for further information should be easily accessible by link, rather than at the bottom of the form. Other comments included:
“The form tries, but I think people are likely to still have questions and confusion over how to write about their reasons for requesting an appeal. It would be so helpful if the form linked to a resource that explained in more detail the four reasons, and gave more examples of circumstances where they would apply.”
“The form assumes everybody is familiar with words like videoconference, high speed internet. I assume many of the people filling out this form are senior citizens and are not familiar with such terms.”
“It wasn’t really clear what the length of the accommodations could be.”
For detailed comments, see Appendix E.
Question 5: How would you rate the visual layout of the form?
Text Version
Question 5 – All forms, both languages
Excellent: 77%
Good but: 20%
Poor: 3%
This aspect of the forms received quite positive rankings, but there were still some suggestions for improvements (see Appendix E). Comments included:
“Where there are choices to be filled out, there is no obvious place or way to select them. A box to check in front of each choice would help.”
“Being visually challenged, I find many legal documents are difficult to read once printed.”
Focus group participants suggested the inclusion of a checklist of “to do” steps for each form that users could complete as they went along.
Question 6: Overall, how would you rate the clarity and accessibility of this form?
Text Version
Question 6 – All forms, both languages
Excellent: 51%
Good but: 43%
N/A: 6%
Again, some mixed results, with some specific suggestions for improvements included in comments (see below at Appendix E). Comments were often positive, and recognized the efforts that had been made to increase the user- friendliness of the forms. For example:
“The forms were great, overall, they were great.”
One participant in the focus group who had been an SRC in the Tribunal in 2012 commented that the user-friendliness of the forms had really improved since then.
But again, there is always room for improvement, as comments indicated:
“An attempt has definitely been made to make the form more user- friendly. But it would be significantly improved by the inclusion of more and more detailed plain language definitions, examples, etc. Again, linking to further resources would be very helpful.”
“Even if the forms are not written by lawyers – but in consultation with them – you can tell that they are written by those who come from privilege… you can tell that they’re written for that level of understanding.”
3.2.2 Question-by-question analysis disaggregated by language
In English, the question that drew the most negative response for the forms (the most “poor” rankings) was question 2, “How well does the form answer the questions an SRL might have?”
Comments included:
“In the ‘reasons’ for appeal, it seeks to define what the reasons might be with examples. This may not be very helpful as it doesn’t show the real meaning of those reasons. A hyperlink to a resource explaining what things are (like what procedural fairness is) would be a lot more helpful than making one single example.”
“Having to scroll to the bottom of the form to get contact info is inaccessible. It should just be a link – it should be an easy back and forth. All of these things matter to a self-rep, who is trying to learn these things for the first time in their life.”
In French, none of the readers gave any “poor” rankings to questions regarding the two forms, but the question that drew the most negative response (the least “excellent” rankings) was question 6, on overall clarity and accessibility.
English readers gave the highest scores for the forms to question 5 (visuals) with several referencing the use of flow charts, and in French, to question 2 (SRL questions). This means that the French result was the opposite of the English readers assessment who scored question 2 lowest, above: but note this was a very small sample.
3.2.3 Forms disaggregated by language
Ranking question | Most negative score: English | Most negative score: French |
---|---|---|
Question 2 (SRL questions) |
Appeal Division form | [No negative scores] |
Question 5 (visual) | General Division form | [No negative scores] |
Question 6 (overall clarity) | Appeal Division form | General Division form |
3.3 Form completion times
We asked readers to record how long it took them to fill in each of the two forms (application to the General Division and application for leave to appeal to the Appeal Division). There are two important limitations to note for this data:
- There are very small numbers on the French side (only 2 readers gave us a form completion time).
- Completing the form did not, naturally, include completing sections about details of personal case specifics and arguments, which are probably the most time-consuming sections in practice.
From the available data (n=17):
- The average time to completion for the General Division Income Security application was 18 minutes for English, and 24 minutes for French.
- The average time to completion for the leave to appeal to the Appeal Division was 16 minutes for English, and 50 minutes for French.
4. Qualitative data
4.1 Coding system
All French and English qualitative data (including comments on the checklists and the focus group discussion) was reviewed carefully and then coded using the following categories:
- Potential for reader error
- Technology
- Language (including jargon)
- General comprehension
- Missing information for an SRL (includes suggestions of missing questions that would have elicited missing information)
- Translation or grammatical errors
- Visuals
- Legibility
- Organization/layout
- Length
- Too long
- Too short
- Tone (general)
- Bureaucratic/cold
All specific suggestions, in both French and English, regarding improvements and enhancements made by readers are coded and included in Appendix E.
4.2 Themes specific to letters
While the letters generally scored well (and significantly better than the forms) on the explanations of legal terminology and reducing “jargon” (Question 1 “Does the form explain any legal and other unfamiliar terms, abbreviations or acronyms, or other references”) there were nonetheless some lacuna highlighted (see the specific suggestions in Appendix E). Frequent examples included: “parties,” “appellant,” “testimony,” and “evidence.” Ironically, one of the terms singled out by several readers was “Navigator,” with some complaining that the description of this role was unnecessarily complex and could be framed in more familiar terms:
“‘Navigator’ is needlessly complicated and repeatedly obfuscates the role of the individual. A more common word is ‘facilitator’ or ‘appeals assistant’ or ‘guide’ or ‘coordinator.’ A navigator is someone who provides direction, if you’re not offering advocacy or legal representation then it’s not the word you want to use. A facilitator is someone who helps another find where they need to go. A coordinator is one to takes direction from others and organizes it to achieve everyone’s objectives.”
“The Navigator role is a novel approach to make the letter more welcoming, but I am not sure if Navigator is the right word. Assigned Agent? Case Worker? Assigned Tribunal Advisor?”
Some readers also suggested that the letters referring to the Navigator should provide a name and phone number and that this might reduce the amount of information that would be needed in the letter.
There were a number of suggestions relating to the overall visual appeal and structure of the letters. These focused on suggestions for the inclusion of more flow charts (infographics are usually popular), and the use of headings and bullet points to break up the text and make it easier on the eye. Specific suggestions for particular letters are described in Appendix E.
The letters checklist asked one unique question (Question 7: does the letter have a welcoming, encouraging tone?”) that was not asked about the forms. This data suggests that there are significant improvements could be made in tone. While the French comments are divided equally between “excellent” and “good but,” 25% of the English readers described the tone as “poor” in relation to being welcoming and encouraging. Specific detailed comments in relation to each letter can be found at Appendix E.
4.3 Themes specific to forms (including focus group discussion)
The response to the forms was positive overall, with a number of compliments made about the efforts to make the forms user-friendly and accessible. However, there were still many concrete suggestions for improvements.
Two aspects that attracted a lot of comment related, first, to the use of language, and second, to the visual presentation and layout of the forms. In relation to Question 1 (“Does the form explain any legal and other unfamiliar terms, abbreviations or acronyms, or other references?”) only half of the form readers (aggregated in both languages) agreed that all such words and phrases were sufficiently explained. This suggests that there is still some way to go to ensure that those without legal training will be able to comprehend and correctly/accurately complete the forms. Several readers reflected that despite their own high level of education they still had difficulty with some of the language:
“(As a social worker with a Master’s degree) (I)f I can’t understand it, how can someone with more barriers understand it?”
The 8 letters scored significantly better than the forms on this assessment, with 84% of readers agreeing that legal terms were properly explained.
The visual layout of the forms drew many suggestions for improvements, not limited to the possible simplification and expansion of the use of flow charts/infographics (see above at 3.2.1), and extending to detailed suggestions about font type, font size, use of bold type, etc. For example, one focus group participant noted:
“Shading, color coding, fillable boxes, bolding, asterisks, and even mandatory fields could be used to indicate importance.”
There were also a number of thematic comments related to Question 2 (“Does the form answer questions that SRL s might have?”) which focused on more use of drop-down menus offering explanations of the process, “how to” information (such as examples of what types of documents would be useful in support of the application), and a number of comments highlighted the importance of clarifying and explaining the process for applying for accommodations (this is a concern we hear often from litigants in the courts).
The forms checklist asked one unique question (Question 7: was there any part/section of the form that you did not know how to complete?) that was not asked about the letters. As noted above at 3.1.2, almost three quarters of the readers (73%) of the English forms said they were able to fully complete the form, compared to 60% of the French readers. Specific detailed comments in relation to each letter can be found at Appendix E.
5. Recommendations
- Given the detailed analysis and commentary provided by each of the English and French readers, it is important that their detailed comments on each form and letter (see Appendix E) are read carefully and where possible responded to. For example, there are many signposts to particular words and vocabulary that could be clarified, and much of this is consistent. Despite the generally positive assessment of the efforts made in these letters and forms to be accessible to those coming to the Tribunal without legal training, there are inevitably still some areas for improvement, and language and the use of language and terminology also needs to be kept under constant review (see also Recommendation 10, below).
- One aspect of the added value provided by user assessment (which complements the work already conducted by the Tribunal on readability) is the identification of additional information that users may need to be able to both comprehend and follow instructions in both letters and forms. These issues are highlighted by Question 2 (for both forms and letters) and Question 7 (for forms only). One of the readers (a focus group participant) highlighted the fact that even if the forms were developed by plain language specialists who are not lawyers, their level of education and would be very different from many SRLs. Moreover, only users who are “in process” at the SST are able to identify the specific difficulties they might face in that position, exacerbated by their personal stress as SRLs. This points to the need for ongoing review and evaluation, discussed further at 10 below.
- A number of readers commented on the usefulness of additional informational resources including infographics and video. We have found that among self-represented litigants at large, visually attractive and accessible resources are very popular.
- Many readers commented that it would be useful if both the letters and forms could be hyperlinked to further resources on the SST website. Drop down menus with these options were suggested for the forms. These links could provide (for example) a glossary of terms, and more detailed explanations of processes possibly using flow charts or other visuals, posted on the SST site.
- Some readers and focus group participants suggested that sample versions of the forms, filled out with hypothetical circumstances and posted on the website, would be very useful to give people a sense of what they “should” look like.
- Some readers also discussed the idea that the Tribunal might offer a “live chat” option on the website for claimants with questions or concerns about completing the forms. This would help claimants in real time with quick and immediate answers to their questions. Given the anxiety that we know form completion causes those without legal representation, and the potential for unintentional error causing future loss of Tribunal time/resources, this would add value to the overall user experience. This may be seen as an aspect of the role of the Navigator, but experience may show that some questions are repetitive and could be answered this way without using the time of the Navigator. Working the “live chat” may also prove to be a useful preliminary training ground for Navigators. It is recommended that this idea be re-visited once the independent evaluation of the Navigator program is completed.
- Some of those in the focus group expressed a need for clarification on the security of personal information when this is sent electronically to the Tribunal. We recommend that a short note be added to the forms and on the website explaining the steps the SST has taken to ensure the security of claimants’ information.
- The issue of tone (Question 7, unique to the letter reviews) stood out to us as something that can be easily overlooked but remains extremely important. There is a relationship between tone and accessibility that reflects the importance of keeping the lay reader encouraged and moving on through the unfamiliar information and terms contained in both letters and forms. Some readers repeated a comment about an unfriendly, officious and even overbearing tone several times over. This is another aspect of creating user-friendly forms and letters that requires the personal experiences of SRLs who are invested in their own matter and may have a heightened reaction to what is probably an unintended coldness in the tone of the letters. We recommend that some thought is given to reducing the formal and somewhat cold tone of some of the letters.
- There was some interesting reaction to the new SST position of “Navigator” and the choice of this term to describe this role. While this expression is becoming familiar to many of us working to improve the level of SRL assistance in courts and tribunals, it seems less intuitive to system newcomers. Without changing the name, it is important to consider how the descriptions provided of the Navigator position can be made clear, and critically, welcoming and encouraging (“here to help!”) to SRLs, rather than simply providing one more “jargon” term that they are unclear about and possibly confused by. This recommendation should be re-visited following the independent evaluation of the Navigator position.
- Evaluation of forms and letters by users is an ongoing process. We experienced some difficulties in maintaining the interest of our reader recruits to this task, since reviewing all 8 letters and 2 forms was quite time consuming and laborious. We would recommend that consideration be given to establishing a permanent SST working group involving SRCs in regularly reviewing new communication resources.
Another idea is to offer those coming self-represented to the Tribunal the opportunity to complete their forms at a particular time and place, and offering a staff person or persons to answer questions about what the forms require (limited to legal information). This would be a way of both offering some orientation to new SRCs at the Tribunal, and collecting further information about those aspects of the form completion process with which they struggle, and ways in which the forms might be improved.
Appendix A: Letter review checklist
Please complete one checklist per letter that you review. Please put your initials at the end to enable any follow up questions and discussions.
Some questions have yes or no answers.
For some questions, you are asked to please rate the document and score it as either:
- “Excellent” (meaning that you see no obvious enhancements)
- “Good but needs improvement” (self-explanatory)
- “Poor” (needs many enhancements, currently inadequate)
E
GB
P
If your answer is “no”, or if your score is “good but needs improvement”, it is really important to give us your feedback on what would make the document better. There is some space under each question to write your specific comments and we will read all these carefully.
First, which letter are you reviewing?
- Acknowledgement of Notice of Appeal
- ESDC documents and next steps relating to appeal
- Navigator call follow-up
- Notice of hearing – Teleconference
- Decision letter General Division
- Late application for appeal
- Leave to appeal granted and notice of teleconference
Decision letter Appeal Division
1 – Are legal and other unfamiliar terms, abbreviations and acronyms explained?
Yes
No
If no, please tells us what terms/words/language references are unclear and need further explanation:
2 – How well does the letter answer the questions that you imagine a self-represented person might have? (If you think someone might have unanswered questions, please note these in the comments section.)
E
GB
P
3 – Is the letter organized and structured in an order that would make sense to the reader, using appropriate headings etc.?
Yes
No
If no, please tell us what could be improved in terms of structure, headings, and order:
4 – Is this letter a reasonable length – not too long and unnecessarily wordy, but also not too concise to be useful?
Yes
No
If no, please tell us more about your reaction to the length of the letter:
5 – How would you rate the visual layout of the letter? Do you have any suggestions for how to make the letter more visually pleasing and easier to “digest” the information? (For example: format, font, style, spacing, and layout.)
E
GB
P
6 – Overall, is the letter written clearly and in a way that is accessible?
E
GB
P
7 – Finally, does the letter have a welcoming, encouraging tone?
E
GB
P
Other comments, both general and specific:
Your initials: ______
Thank you so much for helping to make the Social Security Tribunal better for other SRLs.
Appendix B: Form review checklist
Please complete one checklist for each form you complete. Please put your initials at the end to enable any follow up questions and discussions.
Some questions have yes or no answers.
For some questions, you are asked to please rate the document and score it as either:
- “Excellent” (meaning that you see no obvious enhancements)
- “Good but needs improvement” (self-explanatory)
- “Poor” (needs many enhancements, currently inadequate)
E
GB
P
If your answer is “no”, or if your score is “good but needs improvement”, it is really important to give us your feedback on what would make the document better. There is some space under each question to write your specific comments and we will read all these carefully.
Are you completing (please check one):
The application to appeal to the General Division
The application to appeal to the Appeal Division
1 – Does the form explain any legal and other unfamiliar terms, abbreviations or acronyms, or other references?
Yes
No
If no, please tell us what terms/words/language/references are unclear and need further explanation:
2 – How well does the form provide answers that you imagine a self-represented person might have? (For example, about disability accommodations, or how a virtual hearing will proceed.) If you think someone might have unanswered questions, please note these in the comments section.
E
GB
P
3 – Is the form structured and organized in an order that would make sense to the reader, using appropriate headings, etc.?
Yes
No
If no, please tell us what could be improved in terms of structure, headings, and order:
4 – Is the form a reasonable length – not too long and unnecessarily wordy?
Yes
No
If no, please tell us more about your reaction to the length of the form:
5 – How would you rate the visual layout of the form? Do you have suggestions for how to make the form more visually user-friendly/pleasing? (For example, format, font, style, spacing, and layout.)
E
GB
P
6 – Overall, how would you rate the clarity and accessibility of this form?
E
GB
P
7 – Finally, was there any part/section of the form that you did not know how to complete?
Yes
No
If yes, (i.e.: if there was a section you could not complete) please tells us what, and why not:
Please provide your time log for the time you spent completing the form: _________________________
Any other comments, both general and specific:
Your initials: ______
Thank you so much for helping to make the Social Security Tribunal better for other SRLs.
Appendix C: Focus group questions
- Overall, how would you rate the clarity and accessibility of the forms you reviewed?
- How well do the forms you reviewed provide answers to questions that you imagine a self-represented person might have? (For example, on disability accommodations, or how a virtual hearing will proceed.)
- Did the forms explain any legal or other unfamiliar terms, abbreviations acronyms, or other references? If not, what were the terms you were unfamiliar with?
- How would you rate the visual layout of the forms you reviewed? Do you have suggestions for how to make the forms more visually user-friendly/pleasing? (For example, format, font, style, spacing, and layout.)
- Finally, were there any parts/sections of the forms that you did not know how to complete? If yes, what were they?
Appendix D: Coding system
- Potential for reader error
- Technology
- Language (including jargon)
- General comprehension
- Missing information
- Translation or grammatical errors
- Visuals
- Legibility
- Organization
- Length
- Too long
- Too short
- Tone
- Too cold/bureaucratic
Appendix E: Complete coding of all comments by letter and form
Code | Comment |
---|---|
Question 1: Does the form explain any legal and other unfamiliar terms, abbreviations or acronyms, or other references? | |
1.1 |
|
1.2 |
|
1.3 |
|
2 |
|
4.2 |
|
4.1 |
|
Question 2: How well does the form provide answers to questions that you imagine a self-represented person might have? (For example, about disability accommodations, or how a virtual hearing will proceed.) If you think someone might have unanswered questions, please note these in the comments section | |
1.1 |
|
1.2 |
|
1.3 |
|
2 |
|
4.2 |
|
6 |
|
Question 3: Is the form structured and organized in an order that would make sense to the reader, using appropriate headings, etc.? | |
1.3 |
|
Question 4: Is the form a reasonable length – not too long and unnecessarily wordy? | |
1.2 |
|
2 |
|
5.1 |
|
Question 5: How would you rate the visual layout of the form? Do you have suggestions for how to make the form more visually user-friendly/pleasing? (For example, format, font, spacing, style, and layout.) | |
4.1 |
|
4.2 |
|
Question 6: Overall, how would you rate the clarity and accessibility of this form? | |
1.1 |
|
1.2 |
|
1.3 |
|
2 |
|
3 |
|
4.1 |
|
4.2 |
|
5.2 |
|
Question 7: Finally, was there any part/section of the form that you did not know how to complete? | |
1.3 |
|
2 |
|
4.2 |
|
Question 8: Any other comments, both general and specific? | |
1.1 |
|
2 |
|
3 |
[Translation] Throughout the document, « la ou le membre » is an error. Replace with just « le membre » or « un membre ».
|
4.1 | Overall, very easy to read, nice font size. |
4.2 |
|
Code | Comment |
---|---|
Question 1: Does the form explain any legal and other unfamiliar terms, abbreviations or acronyms, or other references? | |
1.2 |
|
2 |
|
4.1 |
|
Question 2: How well does the form provide answers to questions that you imagine a self-represented person might have? (For example, about disability accommodations, or how a virtual hearing will proceed.) If you think someone might have unanswered questions, please note these in the comments section. | |
1.1 |
|
1.2 |
|
1.3 |
|
2 |
|
Question 3: Is the form structured and organized in an order that would make sense to the reader, using appropriate headings, etc.? | |
4.2 |
|
Question 4: Is the form a reasonable length – not too long and unnecessarily wordy? | |
1.2 |
|
5.1 |
|
5.2 |
|
Question 5: How would you rate the visual layout of the form? | |
4.1 |
|
4.2 |
|
Question 6: Overall, how would you rate the clarity and accessibility of this form? | |
1.2 |
|
2 |
|
Question 7: Was there any part/section of the form | |
1.2 |
|
1.3 |
|
2 |
|
Question 8: Any other comments, both general and specific? | |
1.1 |
|
1.3 |
|
2 |
|
4.2 |
|
Participant | Comment | |
---|---|---|
Question 1: Are legal and other unfamiliar terms, abbreviations and acronyms explained? | ||
1.1 |
|
|
1.2 |
|
|
2 |
|
|
Question 2: How well does the letter answer the questions that you imagine a self-represented person might have? | ||
1.2 |
|
|
1.3 |
|
|
2 |
|
|
Question 3: Is the letter organized and structured in an order that would make sense to the reader, using appropriate headings, etc.? | ||
2 |
|
|
4.1 |
|
|
4.2 |
|
|
Question 4: Is this letter a reasonable length – not too long and unnecessarily wordy, but also not too concise to be useful? | ||
1.2 |
|
|
2 |
|
|
5.1 |
|
|
Question 5: How would you rate the visual layout of the letter? | ||
4 |
|
|
4.1 |
|
|
4.2 |
|
|
5.1 |
|
|
Question 6: Overall, is the letter written clearly and in a way that is accessible? | ||
1.2 |
|
|
2 |
|
|
4.2 |
|
|
Question 7: Does the letter have a welcoming, encouraging tone? | ||
1.3 |
|
|
6 |
|
|
6.1 |
|
|
Question 8: Other comments, both general and specific? | ||
1.2 |
|
|
2 |
|
|
3 |
|
|
6.1 |
|
Code | Comment |
---|---|
Question 1: Are legal and other unfamiliar terms, abbreviations and acronyms explained? | |
1.2 |
|
2 |
|
Question 2: How well does the letter answer the questions that you imagine a self-represented person might have? | |
1.2 |
|
1.3 |
|
2 |
|
Question 3: Is the letter organized and structured in an order that would make sense to the reader, using appropriate headings, etc.? | |
1.2 |
|
2 |
|
4.1 |
|
Question 4: Is this letter a reasonable length – not too long and unnecessarily wordy, but alos not too concise to be useful? | |
2 |
|
5.1 |
|
Question 5: How would you rate the visual layout of the letter? | |
4 |
|
4.1 |
|
4.2 |
|
6 |
|
Question 6: Overall, is the letter written clearly and in a way that is accessible? | |
2 |
|
4.2 |
|
Question 7: Does the letter have a welcoming, encouraging tone? | |
6.1 |
|
Question 8: Other comments, both general and specific? | |
1.1 |
|
2 |
|
3 |
|
4.1 |
|
4.2 |
|
6 |
|
6.1 |
|
Code | Comment |
---|---|
Question 1: Are legal and other unfamiliar terms, abbreviations and acronyms explained? |
|
1.2 |
|
2 |
|
Question 2: How well does the letter answer the questions that you imagine a self-represented person might have? | |
1.2 |
|
1.3 |
|
2 |
|
6 |
|
Question 3: Is the letter organized and structured in an order that would make sense to the reader, using appropriate headings, etc.? | |
1.3 |
|
4.2 |
|
5.1 |
|
Question 4: Is this letter a reasonable length – not too long and unnecessarily wordy, but also not too concise to be useful? | |
1.2 |
|
2 |
|
5.1 |
|
Question 5: How would you rate the visual layout of the letter? | |
1.3 |
|
4 |
|
4.2 |
|
Question 6: Overall, is the letter written clearly and in a way that is accessible? | |
1 |
|
Question 7: Does this letter have a warm, encouraging tone? | |
6.1 |
|
Question 8 : Other comments, both general and specific? | |
1.2 |
|
1.3 |
|
2 |
|
3 |
|
Code | Comment |
---|---|
Question 1: Are legal and other unfamiliar terms, abbreviations and acronyms explained? | |
1.1 |
|
1.2 |
|
1.3 |
|
2 |
|
4.1 |
|
4.2 |
|
Question 2: How well does the letter answer the questions that you imagine a self-represented person might have? | |
1.2 |
|
1.3 |
|
2 |
|
5.1 |
|
Question 3: Is the letter organized and structured in an order that would make sense to the reader, using appropriate headings, etc.? | |
4.1 |
|
5.1 |
|
Question 4: Is this letter a reasonable length – not too long and unnecessarily wordy, but also not too concise to be useful? | |
5.1 |
|
Question 5: How would you rate the visual layout of the letter? | |
4 |
|
4.1 |
|
4.2 |
|
5.1 |
|
Question 6: Overall, is the letter written clearly and in a way that is accessible? | |
1.2 |
|
5.1 |
|
Question 7: Does the letter have a welcoming encouraging tone? | |
English version | |
1 |
|
6 |
|
6.1 |
|
Question 8: Other comments, both general and specific? | |
1.1 |
|
2 |
|
5.1 |
|
6.1 |
|
Code | Comment |
---|---|
Question 1: Are legal and other unfamiliar terms, abbreviations and acronyms explained? | |
1.1 |
|
1.2 |
|
2 |
|
4.2 |
|
Question 2: How well does the letter answer the questions that you imagine a self-represented person might have? | |
1.2 |
|
1.3 |
|
2 |
|
4.1 |
|
4.2 |
|
Question 3: Is the letter organized and structured in an order that would make sense to the reader, using appropriate headings etc.? | |
1.3 |
|
2 |
|
4.1 |
|
4.2 |
|
6.1 |
|
Question 4: Is this letter a reasonable length – not too long and unnecessarily wordy, but also not too concise to be useful? | |
2 |
|
4.2 |
|
5.1 |
|
5.2 |
|
Question 5: How would you rate the visual layout of the letter? | |
2 |
|
4 |
|
4.1 |
|
4.2 |
|
Question 6: Overall, is the letter written clearly and in a way that is accessible? | |
1.2 |
|
2 |
|
5.1 |
|
Question 7: Does the letter have a welcoming, encouraging tone? | |
6.1 |
|
Question 8: Other comments, both general and specific? | |
1.2 |
|
1.3 |
|
2 |
|
3 |
|
5.1 |
|
6.1 |
|
Code | Comment |
---|---|
Question 1: Are legal and other unfamiliar terms, abbreviations and acronyms explained? | |
1.2 |
|
2 |
|
Question 2: How well does the letter answer the questions that you imagine a self- represented person might have? | |
2 |
|
6 |
|
Question 3: Is the letter organized and structured in an order that would make sense to the reader, using appropriate headings, etc.? | |
2 |
|
4.1 |
|
5.2 |
|
Question 4: Is the letter a reasonable length – not too long and unnecessarily wordy, but also not too concise to be useful? | |
1.2 |
|
1.3 |
|
2 |
|
5.1 |
|
5.2 |
|
6 |
|
Question 5: How would you rate the visual layout of the letter? | |
4 |
|
4.1 |
|
Question 6: Overall, is the letter written clearly and in a way that is accessible? | |
English version | |
1.2 |
|
2 |
|
Question 7: Does the letter have a warm, encouraging tone? | |
1.3 |
|
2 |
|
6.1 |
|
Question 8: Other comments, both general and specific? | |
6.1 |
|
Code | Comment |
---|---|
Question 1: Are legal and other unfamiliar terms, abbreviations and acronyms, explained? | |
1.2 |
|
2 |
|
6.1 |
|
Question 2: How well does the letter answer the questions that you imagine a self-represented person might have? | |
1.2 |
|
1.3 |
|
2 |
|
Question 3: Is the letter organized and structured in an order that would make sense to the reader, using appropriate headings, etc.? | |
4.1 |
|
4.2 |
|
5.1 |
|
Question 4: Is this letter a reasonable length – not too long and unnecessarily wordy, but also not too concise to be useful? | |
1.3 |
|
2 |
|
5.1 |
|
5.2 |
|
6 |
|
Question 5: How would you rate the visual layout of the letter? | |
4 |
|
4.1 |
|
4.2 |
|
Question 6: Overall, is the letter written clearly and in a way that is accessible? | |
1.2 |
|
1.3 |
|
4.2 |
|
5.1 |
|
Question 7: Does the letter have a welcoming, encouraging tone? | |
1.2 |
|
5.1 |
|
6.1 |
|
Question 8: Other comments, both general and specific? | |
1.1 |
|
1.3 |
|
2 |
|
Code | Comment |
---|---|
Question 1: Are legal and other unfamiliar terms, abbreviations and acronyms explained? | |
1.2 |
|
2 |
|
Question 2: How well does the letter answer the questions that you imagine a self-represented person might have? | |
1.2 |
|
1.3 |
|
2 |
|
4.1 |
|
Question 3: Is the letter organized and structured in an order that would make sense to the reader, using appropriate headings, etc.? | |
4.2 |
|
5.2 |
|
Question 4: Is this letter a reasonable length – not too long and unnecessarily wordy, but also not too concise to be useful? | |
2 |
|
4.2 |
|
5.2 |
|
6.1 |
|
Question 5: How would you rate the visual layout of the letter? | |
2 |
|
4 |
|
4.1 |
|
4.2 |
|
5.2 |
|
Question 6: Overall, is the letter written clearly and in a way that is accessible? | |
2 |
|
5.2 |
|
Question 7: Does the letter have a welcoming, encouraging tone? | |
2 |
|
5.2 |
|
6.1 |
|
Question 8: Other comments, both general and specific? | |
1.2 |
|
2 |
|
4.2 |
|
6.1 |
|
Appendix F: Forms reviewed (in French and in English)
Appendix G: Letters reviewed (in French and in English)
- Acknowledgement of Notice of Appeal
- Accusé de réception de l’Avis d’appel
- ESDC documents and next steps regarding the appeal
- Documents reçus d’EDSC et prochaines étapes du processus d’appel
- Navigator call follow-up
- Suivi de la conversation téléphonique
- Notice of hearing by teleconference
- Avis d’audience par téléconférence
- Decision letter – General Division
- Lettre de décision – Division générale
- Acknowledgement of complete Application for Leave to Appeal
- Accusé de réception - Demande de permission d’en appeler - Semble être hors délai Division d’appel
- Leave to appeal granted and notice of hearing by teleconference
- Permission d'en appeler et avis d'audience par téléconférence
- Decision letter – Appeal Division
- Lettre de décision – Division d’appel