You can quickly rate each article with “include” and “exclude” and proceed to the next. All other buttons are optional.
Include – labels an article as being of interest to the research topic, and proceeds to the next article. Any assistants enabled will try to use this knowledge to present similar documents. This button will appear ticked, if you review a previously “included” article.
Exclude – labels an article as not relevant, and proceeds to the next article. Any assistants enabled will try to avoid showing you these kinds of documents, unless there are no others left, or during randomisation. This button will appear ticked, if you review a previously “excluded” article.
Refs, Interest checkboxes – additionally labels the article with these extra flags. During initial screening you must toggle these before clicking either “include” or “exclude”. If you revisit an already rated article, you may change these flags and they are saved instantly.
Next Unrated Article – This can be used to resume from where you got to in the normal flow of presentation. This is useful if you have used the alternate navigation options instead of “include” and “exclude”, but now want to continue the primary screening process. Note: you don’t need to use this button during the initial screening process, as clicking either “include” or “exclude” will automatically proceed to the next unrated article.
Prev/Next Included Article – This allows you to re-review the set of articles you have already labelled as “include”.
Prev/Next Article – This takes you through the articles by the order they were presented to you from the article set, or by any assistant.
(Jump) This is useful if you want to go directly to a specific article, by index number. Comments – Each collaborator may comment on an article. Your comments are initially not visible to other collaborators, but become visible later during rating disagreement reviews. (Initially you need to enter the comment first, then click “include” or “exclude” to save the comment. If you revisit an already rated article, you may edit the comment, and then click SAVE to store it. )
Progress Bar and Status Page – There is a visual indication of your progress through rating the articles along the top of the screen. If you reach 100% or click the bar at any time, you will be taken to a summary report of your screening progress, including a reference list of the articles you have “included”.
Screening Criteria Checklist
a screening criteria checklist allows you to present a set of criteria the collaborators should consider during screening before “including” or “excluding” an article. You can set up a plain list, or create a checklist, which the reviewer can then record their decisions. Ticking off the list can be made optional or mandatory depending on your needs.
If you wish to use an online criteria checklist, the project administrator must set it up before screening can begin.
To prevent inconsistencies the list must be finalised and locked to prevent any changes before screening can start, and it may not be unlocked after screening has started.
The process of setting up and using a criteria checklist is as follows:
- The list options can be found at the bottom of the project page. For the collaborators this section appear as just information, but the administrator also sees the editing controls.
- The administrator chooses the type of list required plain list, or mandatory or optional checklist.
- The administrator enters the list items, ordering them appropriately.
- At this time, with the list enabled but unfinished, the screening review buttons are disabled. This is to ensure the administrator has actively thought through the criteria and finalised them and that screening cannot accidentally start until the list is explicitly finished.
- The administrator locks the list. Screening now can begin.
- The collaborators begin screening and see the criteria list appear in the page margin next to each article they review. They can fill in their choices. If the administrator set the list to “mandatory” the reviewer must fill in the entire form before they can click the rating buttons.
- Note after screening has started, the list cannot be unlocked nor edited. This is to prevent inconsistent data being captured.
Once screening is finished and all the criteria choices have been made there are two ways they can be used.
- Data export – each collaborator’s criteria choices are included individually in the screening progress export spreadsheet files. You will see extra columns to the right of the spreadsheet containing the criteria and each persons selections in the fields below where Y = Yes, N = No, D = Don’t know, and blank was unanswered.
- Disagreement review – the criteria choices are summarised in a table format during screening of a disagreement article set. This allows the reviewers to see what choices all collaborators chose during their own screening. You may also re-enter criteria during the disagreements re-screening (agreement process). These criteria selections can be exported from the disagreement set, under the username of the person logged in when doing the agreement process.
Future criteria features.
It can get complicated when combining criteria selections with split sets, disagreement, re-agreements and rejoining sets. SYRAS tries to give you all the components to make your own workflow as appropriate to your process needs. Hopefully all the pieces are there, but there may be a gap in these new tools for some workflows.
Please contact us via syras.org If you wish to send us a suggestion of how you would like to use criteria in the later stages.