Methodology
This Comprehensive Methodology
provides a structured tool for
assessing the quality of implementation
of a Right to Information Law.
Central Measures
This section of the methodology looks at how the RTI law is implemented at the central level. Central institutions serve two main functions: processing appeals and undertaking promotional measures.
Institutional Measures
This section looks at what individual public authorities have done to implement the RTI law. In any particular jurisdiction, there will always be a great number, normally hundreds and sometimes thousands, of different public authorities of different types and levels.
Proactive Disclosure
This section looks at proactive disclosure or the release of information by public authorities without an RTI request. This is often done through websites and annual reports.
Reactive Disclosure
Whereas proactive disclosure looks at whether public authorities make information available even in the absence of a request for it, reactive disclosure is about how public authorities respond to RTI requests.
Methodology Roadmap
1. Assess Feasibility of Doing Assessment
2. Put in Place the Governing Structure
3. Develop an Implementation Plan
4. Apply the Assessment Tools to Collect Information
5. Analyse Information and Score Results
6. Draft Report and Recommendations
7. Publish Report
8. Conduct Follow-up Advocacy and Activities
Applying the Methodology
The Comprehensive Methodology evaluates the implementation and performance of a jurisdiction’s RTI laws by its central authority and public authorities. In any given country, there will be 100s or even 1000s of authorities, and it will not be possible to assess all of them. The Methodology, therefore, calls for a minimum of ten public authorities to be selected for assessment. The central and public authorities are evaluated at the institutional level. The public authorities are further assessed based on their performance of proactive and reactive disclosure requirements. According to a three-part scale, both the jurisdiction and the individual institutions are assigned grades: poor, mediocre, excellent.
The following constitutes an overview of the methodology. The complete methodology is available for download here.
Central Measures
This section of the methodology looks at how the RTI law is implemented at the central level. Central institutions serve two main functions: processing appeals and undertaking promotional measures. In almost every country where an independent oversight body (often called an information commission) exists, it bears primary responsibility for the first function. In many countries, the information commissions both process appeals and are the main official body that is responsible for promotional measures. However, in others, the oversight body is supported by a nodal body—either internal or external to the government—that undertakes the promotional functions, including assisting individual public authorities. For the purposes of this Methodology (available for download in full here), the information commission and nodal body (if any) should be evaluated as one.
Assessment
In this section, assessors evaluate whether the central authority has successfully implemented the RTI law at an institutional level, for example, by establishing an independent oversight body and whether the institution fulfills its RTI obligations in practice. In most cases, responsibility for the former lies with the government or parliament. The latter focuses on the effectiveness of the oversight body in discharging its responsibilities.
Structural |
Performance |
|
|
The following assessment tools should be deployed:
- Key informant interviews
- A self-assessment by the public authority
- A desk-based literature review
- Another desk-based review of actual decisions on complaints/appeals
The central authority’s institutional measures are graded through five binary (0, 1) questions and eight ternary (0, 0.5, 1) questions that are then averaged. A data entry spreadsheet (available for download here) can be used to facilitate the grading process.
The five binary (yes-no) questions are:
- Has funding been allocated (i.e. to the body)?
- Does the body recruit its own staff (as opposed to this being done by the government, for example)?
- Are the body’s appeals decisions available online?
- Has the body produced and published an annual report for both of the last two years?
- Has the body published a guide for requesters?
The eight more qualitative questions are:
- Have the members been appointed?
- Are the members of the body independent and effective?
- Is the funding provided to the body reasonably sufficient for it to discharge its functions?
- Does the body decide appeals in a timely fashion?
- Are the due process rights of parties respected during appeals?
- Has the body made reasonable efforts to raise public awareness?
- Have effective measures been taken to provide training to officials?
- Has the body made a reasonable effort to comment on draft laws which affect the right to information?
Here, assessors are asked to determine whether the system performs Strongly (1 point); Partially (0.5 points) or Weakly (0 points).
The overall point score is calculated by taking the 13 individual scores and averaging them. This score should then be converted to a final colour grade using the following table:
| Red | Yellow | Green |
| 0-0.33 | 0.34-0.66 | 0.67-1.0 |
Public Authorities
This section looks at what individual public authorities have done to implement the RTI law. In any particular jurisdiction, there will always be a great number, normally hundreds and sometimes thousands, of different public authorities of different types and levels. Acknowledging that it is unrealistic for assessors to review every single authority, the Methodology requires the Project Lead to select a subset of authorities representing the whole. In this subset, the Project Lead should only include public authorities that receive a volume of RTI requests that makes proactive disclosure—which this methodology measures—reasonable. This subset of authorities will be reviewed across all three assessment areas: Institutional Measures, Proactive Disclosure and Reactive Disclosure.
Institutional Measures
In this section, assessors evaluate whether the public authority has successfully implemented the RTI law at an institutional level, for example, by hiring a public information officer, and whether the institution fulfills its RTI obligations in practice.
Structural |
Performance |
|
|
The following assessment tools should be deployed:
- Key informant interviews
- A self-assessment by the public authority
- A desk-based literature review
Each public authority’s institutional measures are graded on ten binary (0, 1) questions and six ternary (0, 0.5, 1) questions that are then averaged. A data entry spreadsheet (available for download here) can assist with this calculation.
The ten objective evaluations are:
- Has an IO been appointed?
- Has the IO formally been given terms of reference or a job description?
- Has the IO been provided with training?
- Has an overall implementation plan or set of standard operating procedures (SOPs) been adopted?
- Has a set of guidelines for how to process RTI requests been adopted?
- Is it possible to lodge requests electronically? Is it easy to obtain an RTI request form? Is it easy to find the contact details of the IO? (YES is given for two or more positive answers, NO for one or less)
- Has a person who is different from the IO been appointed to deal with internal complaints?
- Did the public authority publish an annual report for the last two years?
- Has the public authority conducted any public awareness-raising activities over the last year?
- Has the public authority put in place any system or taken any action to improve its records management?
For each of these evaluations, one point is given for a YES and zero points are given for a NO. In some cases, these questions do not generate simply YES-NO answers but the assessor should nevertheless allocate binary scores here.
The six qualitative evaluations are:
- Does the IO have appropriate qualifications for the job and has he or she been allocated time to do the job?
- There is no political pressure on the IO that makes it difficult for him or her to do the job properly.
- How strong is the overall implementation plan or SOP?
- How strong is the annual report?
- How extensive are the awareness-raising activities?
- How effective are the measures taken to improve records management
For each of these evaluations, an evaluation of STRONGLY, PARTIALLY or WEAKLY is awarded, depending on how well the assessor believes the public authority has done.One point is awarded for STRONGLY, one-half point for PARTIALLY, and zero points for WEAKLY.
The final point score for the jurisdiction as a whole should be calculated by averaging the results of all public authorities.
The 16 point scores for each public authority are then averaged (added up and divided by 16) to get a final point score for each authority. These averages by public authority are then averaged again to get the final point score for the jurisdiction. The final point scores for each public authority and for the jurisdiction should be converted to a final colour grade using the following table:
|
Red |
Yellow | Green |
| 0-0.33 | 0.34-0.66 |
0.67-1.0 |
Proactive Disclosure
This section looks at proactive disclosure or the release of information by public authorities without an RTI request. This is often done through websites and annual reports.
Formally, public authorities’ proactive publication obligations are limited to what the RTI law requires them to do. In other words, performance should be assessed against the list of proactive publication obligations set out in the law. The types of documents that are required to be disclosed can be listed, and the assessment can be conducted against that list. Where the law is vague, the assessors should make and report on any required interpretations.
RTI laws are, however, slow to change. Therefore, it is recommended that the assessors evaluate the public authorities against a set of better practice standards. The final report should delineate between the requirements prescribed by law and those by generally accepted standards.
Structural |
Performance |
|
|
By definition, information made available on a proactive basis should be relatively easy to access either in person via the public authority’s office or website. The following assessment tools should be deployed:
- Desk-based review of proactive disclosure
- Office visits
- Key informant interviews
- A self-assessment by the public authority
- A desk-based literature review
Proactive disclosure is graded on two scoring criteria. The first—a compliance score—involves measuring the public authority’s compliance with each legally prescribed proactive disclosure requirement against a quinary scale (0, 0.25, 0.50, 0.75, 1.0). In other words, the assessment of proactive disclosure involves making a list of what should be disclosed proactively (according to the law) and then seeing if it is in fact available. For each item on the list, the public authority should be given one of the following evaluations: Full, Full to Partial, Partial, Partial to None or None.
It is not possible to provide precise instructions as to how to allocate evaluations here, given the wide range of different types of information covered. Care should be taken to be sufficiently stringent in allocating evaluations. Thus, a FULL evaluation should only be given if all of the factors are met.
Factors to consider here include:
- Is the information which is available complete, in the sense of including everything in the relevant category
- Is the information easy to find
- Is the information up-to-date, taking into account how frequently that type of information changes
Points should then be assigned according to the below scale:
|
Full |
Full to Partial | Partial | Partial to None | None |
| 1.0 | 0.75 | 0.50 | 0.25 |
0 |
The results of each legal requirement are then averaged.
The second criteria—a convenience score—is comprised of five ternary (0, 0.5, 1) questions that are then averaged. The five issues that should be assessed here are:
- The extent to which the website is WCAG 2.1 compliant.
- The extent of the efforts the public authority takes to disseminate information other than simply via its website.
- The extent to which the public authority makes use of social media and other means to draw the attention of the public to its proactive publications and to disseminate information proactively.
- The extent to which the public authority makes an effort to create understandable versions of at least the most important documents (such as its budget).
- The extent to which it is reasonably easy to find specific information from among all of the information that is being published online.
For each of these issues, an assessment of STRONGLY, PARTIALLY or WEAKLY is awarded, depending on how well the assessor believes the public authority has done. One point is awarded for STRONGLY, one-half point for PARTIALLY, and zero points for WEAKLY. The point score by public authority for this (second) part of the proactive area is the average of these point scores (add them up and divide by three).
Each public authority’s final score is comprised of 3/4 compliance score and a 1/4 convenience score.
The final point scores for each authority are then averaged to get a final point score for the jurisdiction as a whole.
The final point scores for each public authority and for the jurisdiction should be converted to a final grade using the following table:
|
Red |
Yellow | Green |
| 0-0.33 | 0.34-0.66 |
0.67-1.0 |
A data entry spreadsheet (available for download here) can be used to assist with recording and averaging point scores.
Reactive Disclosure
Whereas proactive disclosure looks at whether public authorities make information available even in the absence of a request for it, reactive disclosure is about how public authorities respond to RTI requests.
- Is only the minimum information required by the law demanded when making an RTI request or is other (additional) information demanded?
- Can RTI requests be made in commonly used local languages or only official languages?
- Is it easy to submit RTI requests (electronically, in other ways)? Do you need to use the form? Is the form easily available? Do you need to prove citizenship? If so, is this easy to do?
- Is assistance provided when needed?
- Is a receipt provided when an RTI request is lodged?
- If the public authority does not hold the information, do they transfer the RTI request to the public authority which does hold it (or at least refer the requester to that authority)? In a timely manner? Are transfers made which the law does not authorise (i.e. where the request should not be transferred because the original public authority holds the information)?
- How long does it take to process RTI requests? Are responses provided as soon as possible? Within the maximum time limits? Are any extensions legitimate in terms of the rules in the law for this? Where extensions are claimed, are responses provided within the extended period?
- Is information provided in the format stipulated by the requester? If not, are reasons for this given? Are these reasons in line with the law (i.e. in line with the conditions regarding not respecting the requester’s preferred format set out in the law)?
- Are only reasonable fees charged for RTI requests (i.e. in line with what the law and any rules on this allow, including no fee for lodging the request)?
- If an RTI request is refused, is appropriate notice in line with the legal requirements provided?
- Are claims for exceptions reasonable or overbroad (this is a subjective issue but can be assessed both directly by reviewing these claims and also by looking at the percentage of the appeals which are based on refusals that the authority loses, if that information is available)?
- Are any guidelines adopted by the public authority followed when RTI requests are processed?
- To the extent that this can be determined, do responses to RTI requests appear to vary as a function of the gender of the requestor? Where this is the case, this should be reflected and analysed in the final report, including any statistically relevant information on this.
The primary assessment tool here is to make several actual RTI requests to test the public authority’s response. The other assessment tools are:
- RTI request testing
- A desk-based literature review
- Key informant interviews
- A self-assessment by the public authority
Each RTI request is graded on two scoring criteria. The first type–a processing score—is made up of three binary (0, 1) sub-scores that are then averaged. The three sub-scores are as follows:
- A receipt score (based on whether a receipt was provided)
- A timeliness score, based on whether the request was answered within the statutory time limits, with any extensions being assessed for both compliance with the legal rules and reasonableness
- A fee score, based on whether any fee charged was in line with the legal requirements
The second type of scoring criteria for each RTI request—a results score—is comprised of two binary (0, 1) sub-scores and one ternary (0, 0.5, 1) sub-score that are then averaged. The sub-scores are as follows:
- For full disclosure of the information, which is always valid, one point is given.
- For oral refusals or mute refusals (failures to respond at all within the time limits), as well as for cases in which it was not even possible to lodge the RTI request in the first place, all of which are never valid, zero points are given.
- Three other responses – a written refusal (in whole or in part, with partial information having been provided), transfer of the RTI request or referral of the requester to another authority and indicating that the information is not held – may be more or less valid, depending on the circumstances. The assessor should decide whether these responses are LIKELY, MAY BE or is UNLIKELY to be valid, based on all of the circumstances. One point is given for LIKELY, one-half point for MAY BE and zero points for UNLIKELY. Providing only part of the information and not indicating that the rest of the information is either not held or is exempt is at least partially invalid and so should never receive full points. Where such a response is deemed to be more in error than a deliberate strategy to deny access to the rest of the information, one-half point may be awarded.
The final score of each RTI Request is comprised of 1/3 processing score and 2/3 results score.
The final point score for each public authority is calculated by averaging the results of the RTI Requests made to it.
The final point score for the jurisdiction as a whole should be calculated by averaging the results of all RTI Requests.
The final point scores for each public authority and for the jurisdiction should be converted to a final grade using the following table:
|
Red |
Yellow | Green |
| 0-0.33 | 0.34-0.66 |
0.67-1.0 |
Final Grading
Scoring Instructions
Final grades may be assigned in a number of different areas including to the jurisdiction as a whole, to each individual public authority overall, overall in each of the four assessment areas and to each public authority in each assessment area.
Each public authority should have been assigned three final point scores, one for Institutional Measures, one for Proactive Disclosure and one for Reactive Disclosure. To calculate the overall grade for each authority, simply average the three final point scores from each assessment area and then convert to a final grade using the following table:
|
Red |
Yellow | Green |
| 0-0.33 | 0.34-0.66 |
0.67-1.0 |
For the jurisdiction overall, there should have been four final point scores in each assessment area (i.e. one each for Central Measures, Institutional Measures, Proactive Disclosure and Reactive Disclosure). The overall final point score should be calculated by averaging these four final point scores and then converting this to a final grade using the following table:
|
Red |
Yellow | Green |
| 0-0.33 | 0.34-0.66 |
0.67-1.0 |