Double screening, Disagreements, and exporting

Double Screening example

Under the Assigning Work Guide we explained how to set up the team structure for a double-screening review, where two people (the project administrator and one collaborator) both screen the same article set. This allows you to cross check their judgement of included and excluded articles, and perform a disagreement review cycle. This guide uses this scenario to demonstrate how you can use Syras to manage multi-phase reviews.

By this time you have each performed your screening of the ALL ARTICLES set, as covered under the Screening Tools Guide.

Blinding note – during the parallel review process, the collaborators cannot see each other’s results. Each can enter comments which are kept private for the moment.

As project administrator, once both reviews are complete, you can now proceed to analyse any disagreements.

Go to the Project Dashboard and open the ALL ARTICLES set. Click the CREATE SUBSET button.

In the pop-up dialogue box, select the Disagreements tab. Click GO which will run a scan of the collaborators ratings and generate two new sub-sets.

Under the Article Sets section, there will be a new set called “Disagreements in ALL ARTICLES”, with the count showing how many articles had conflicting screening results. Note, if there were no disagreements no set is generated.

There will also be a new set entitled “Agreed INCLUDE articles in ALL ARTICLES”, listing all the articles which both reviewers chose to “include”. This will be used later in the final step.

You may now screen the disagreements according to your protocol. Typically you would assign it to the collaborators who disagreed and have them review it together again, discussing criteria and why they thought differently. This can be done face-to-face with just one user logged in, or remotely with both of them screening separately again.

Combined screening results – the disagreements set is unique in the way it aggregates all the comments and ratings from the team. While screening a disagreement set, you will see a list of any comments per article on the right hand speech bubble, with timestamps and ratings included. (Any criteria choices are similarly aggregated – see the separate guide).

Proceed to screen this set in the same way, choosing include or exclude, until complete. You are then ready to export the chosen articles.

Finalising the Export

To help you produce a single set of exportable references, Syras has a Merge Set tool, which enables you to combine the “include” articles from the various stages. In this example you now have two sets of article references which you would presumably want to export for full-text document purchase and to continue your review beyond Syras. To achieve this you first need to produce the “agreed includes” from the disagreements review, which can be done by repeating the agreement/disagreement scan as before.

Go to the Project Dashboard and open the “Disagreements in ALL ARTICLES” set. Click the CREATE SUBSET button.

In the pop-up dialogue box, select the Disagreements tab. Click GO to scan and generate the new set.

Under the Article Sets section, similar to before, there will also be a new set entitled “Agreed INCLUDE articles in Disagreements in ALL ARTICLES”. Presumably the disagreement review was completed perfectly, and therefore there should not be another “disagreement set”.

You can now merge the two “agreed include” sets using the merge button. This will pop up a new dialogue, in which you can select both sets using the pair of drop-down lists. You may name this set e.g. “final export”, otherwise Syras will generate a name for you. The generated name is logical, but can become a little long and cumbersome, as it contains all the names of the combined sets to record the history of the union.

You now have a final set of all articles everyone has decided to include in your study. You may export this into RIS or Endnote, using the export tools.

Was this guide useful? Let us know what it’s missing, or what features would help make Syras more useful for you.