Once you have completed your search strategy you will need to remove both internal and external duplicate citations from your search results. Many tools can assist you in this process. Here, we will demonstrate how to use EndNote to find and remove duplicate citations from your search results. At this point in the systematic review process, we are only removing duplication citations. Some citations may share the same data, but still be unique citations. These citations should remain in your search results to be assessed during the study selection process.
Track your search strategy, search dates and duplicates with the Systematic Review Search Tracking Spreadsheet. Ultimately this information will be recorded in the PRISMA Flow Diagram.
Set Your Preferences in EndNote
Before you begin removing duplicates with EndNote, you will need to set your preferences to optimize EndNote's native Find Duplicates tool.
1. Start by modifying your Duplicates field. Go to Edit, Preferences, Duplicates uncheck Reference Type and Author. This will increase the accuracy of the Find Duplicates tool in EndNote.
2. Next, modify your Display Fields. Go to Edit, Preferences, Display Fields. Reorder your display fields to show the Record Number, Author, Year, Pages, Title and Name of Database. These fields can be easily scanned for additional duplicates.
You will need to create a group for each database that you search for your systematic review. By creating groups, we can organize our search results by database and easily find internal duplicates.
1. To create a group, go to Groups, Create Group. Organize your groups alphabetically in order of importance. For example, A-Medline, B-Embase, etc. You should also create a group for your duplicates titled Z-Duplicates. This will be important for the deduplicating process.
You will need to import your search results one database a time into separate groups. You will start with TWO databases only. Brining in more than two databases at a time will disrupt the deduplication and tracking process. Once you have removed internal and external duplicates from these first two databases, you will export your next set of search results from your third database.
1. Start with Ovid Medline. You can only export 1000 search results at a time from the Ovid platform (this includes Medline and Embase). In the Range field, enter 1-1000, choose Export, Export to EndNote, Complete Reference, and Export Citation(s). If you have more than 1000 results you’ll need to select the next range (i.e. 1001-2000) and export those until all search results are in your EndNote Library. You should add the Name of Database to your imported results before importing your next set of database results (see below).
Adding the Name of Each Database to your EndNote records will help you through the deduplicating process and ensure that you will know where each reference came from. Some databases will be named automatically when you export into EndNote. Some databases, like PubMed, will leave the Name of Database field empty. Adding the Name of Database will assist the deduplication process.
1. When you bring in a set of results, immediately add the Database Name to the Name of Database field. You can do this by highlighting all of your imported references, select Tools, Change/Move/Copy Fields. Select Name of Database, Replace Whole Field With and name your database.
2. Once you have added the Name of Your Database, select all of your references (Ctrl + A) and drag them to the appropriate group (i.e. A-Medline, B-Embase, etc).
2. Repeat this process with Embase until you have two groups, “A-Medline” and “B-Embase”. Remember, you will add a third database only after completing the deduplication process for the first two databases.
Now that your preferences in EndNote are set, your references are exported to EndNote, you have named your databases, and placed the references into their groups, you are ready to remove internal duplicates.
1. Select your first group of results (which should be titled A-Medline) and find internal duplicates: References, Find Duplicates. In EndNote, select cancel. Notice that a separate field of duplicates is created. Look through the found duplicates to ensure that they are indeed duplicates. Move these duplicates to a duplicates folder titled Z-Duplicates.
2. Next, sort your display fields by page number and review your results manually to make sure you do not have additional duplicates. Move any additional duplicates you may find to your Z-Duplicates folder. When you are done, record the number of internal duplicates (in your Z-Duplicates folder) in your spreadsheet. Finally, copy all of your files in your Z-Duplicates folder to your separate Duplicates EndNote Database. Empty your Z-Duplicates folder into the trash. Empty your trashcan.
3. Repeat this process for your next set of results (which should be titled B-Embase).
Once internal duplicates (duplicates within a database) are removed and recorded on your spreadsheet, search for external duplicates (duplicates between databases).
1. Select References, Find Duplicates. Look through the found duplicates to ensure that they are indeed duplicates. Move these duplicates to your Z-Duplicates Folder. Sort the remaining results by Pages and manually scan them for additional duplicates. Move any additional duplicates you may find to your Z-Duplicates folder. When you are done, record the number of internal duplicates (in your Z-Duplicates folder) in your spreadsheet under External Duplicates. Finally, copy all of your files in your Z-Duplicates folder to your separate Duplicates EndNote Database. Empty your Z-Duplicates folder into the trash. Empty your trashcan.
2. At this point, you can now export your next set of search results to EndNote. These may come from PubMed, Cochrane, Web of Science or any number of databases. Remember that you should add only one database at a time to ensure the accuracy of your duplicate count.