The Importance of Re-Executable Publications

While information is good, exactly how we use our information is just as important. Discovering, sharing and reusing software, data and computational environments, as enabled by NITRC, establishes a level playing field for each of these research objects.

NITRC as a Vehicle for the Good, the Better, the Best Practices…

NITRC is pleased to be part of the neuroinformatics infrastructure. With over 34-thousand registered users and more than 11-hundred tools/resources, NITRC has really emerged as the ‘go-to’ place for neuroimaging informatics. While information is good, exactly how we use our information is just as important. Discovering, sharing and reusing software, data and computational environments, as enabled by NITRC, establishes a level playing field for each of these research objects (Kennedy, 2019). Working with our colleagues at the International Neuroinformatics Coordinating Facility (INCF), we continue to strive to establish standards and best practices for the use of these research objects. In addition to ‘standards’ for data formats and representations (i.e. NIfTI and BIDS), NITRC seeks to promote best practices in resource utilization and management. We take this opportunity to review some of these issues here.

Software Citation

Software should be treated as a first-class research object, as detailed in the report by Smith, et al, 2016. As such, it should be treated and credited with as much scientific rigor and case as the research publication. Specifically, this requires both the publication of software and the citation of software. The authors established a set of ‘Software Citation Principles’ that embody:

• Importance
• Credit and Attribution 
• Unique Identification
• Persistence
• Accessibility
• Specificity

NITRC supports credit and attribution (and unique identification) for software through the support of the generation of digital object identifiers (DOIs) for NITRC-hosted objects such as software. NITRC supports persistence and accessibility through our long-term commitment to high-reliability uptime, content indexing, and long-term data plan. The community is urged to take software citation to heart and follow the principles established by this document.

Data Citation

In a fashion very similar to that outlined above, Altman and Crosas, 2014, established the Data Citation Principles’, again with the objective of elevating data to a first-class research object status. Similar features for data citation are elaborated upon in this document:

• Importance
• Credit and Attribution
• Evidence
• Unique Identification
• Access
• Persistence
• Specificity and Verifiability
• Interoperability and Flexibility

As above, NITRC supports credit and attribution (and unique identification) for data through the support of the generation of digital object identifiers (DOIs) for NITRC-hosted objects. NITRC supports persistence and accessibility through our long-term commitment to high-reliability uptime, content indexing, and long-term data plan. We would like to urge the community to take data citation to heart and follow the principles established by this document.

Re-Executable Publication

Putting it together, in support of enhancing the overall reproducibility of the published literature, we are supporting an emerging ‘best practice’ of generating publications that are completely re-executable. In addition to the words of a publication, a re-executable publication (aka ReproPub, Kennedy, 2019) includes the specification of its data sources, analysis workflow, computational environment, and complete results. Ideally, these software and data elements are attributed according to the data and software citation principles, enumerated above. The benefit of the ReproPub is that it forms the basis of a systematic exploration of the generalizability of the publication claims, but starting from the exact basis of the original claims, and supporting exploration of claims in the context of changes in data and analytic approach. Such a systematic approach is needed in order to interpret the mounting conflicting claims that are accumulating in the current literature.