Mettre en acte l’universalité de la science des images / Reproducible Research in Signal, Image, and Geometric Processing

Séminaire organisé par Jean-Michel Morel et Nicolas Limare (CMLA – ENS Cachan, France) du 22 au 27 juillet 2013

Participants

Luis Alvarez, Pablo Arias Martinez, Juan Cardelino, Miguel Colom, Mauricio Delbracio, Gabriele Facciolo, Raffael Grompone, Bertrand Kerautret, Daniel Kondermann, Nicolas Limare, Jose Luis Lisani Roca, Enric Meinhardt-Lopis, Sheila Miguez, Jean-Michel Morel, Laurent Oudre, Benoît Petitpas, Agustin Salgado de la Nuez, Haldo Sponton, Victoria Stodden, Eva Theumann-Wesfreid

—

Abstract

This workshop gathered representatives of groups leading various initiatives toward reproducible research in the fields of sound, image and geometric processing, by fostering online publication and execution of research code, peer review evaluation of algorithms and code, and shared datasets. The workshop examined all aspects of the new sort of online journal, a long list of considerations that are listed below. The workshop also designed the generalization of the concept of journal exemplified by Image Processing on Line to other fields (sound processing, computer graphics) and gave a precise road map for these extensions. It also gave a road map on the ways to build on the current journal readership success to make it financially durable, and to expand the number of contributors, as the publication of this rich content remains a challenge for newcomers.

—

Compte-rendu

Much of the current research in computational sciences aims at designing algorithms. New advances are communicated to the scientific community through peer-reviewed publications in conferences and journals. The established peer reviewed publication system, which dates from the 18th century, considers only the publication of a manuscript to be printed on paper. Current technology calls for updating the publication system in the field of computational sciences, making it more appropriate for the publication of algorithms and computer code, and providing researchers with tools to easily reproduce the work and experiments of their peers.

This workshop gathered representatives of groups leading various initiatives toward reproducible research in the fields of sound, image and geometric processing, by fostering online publication and execution of research code, peer review evaluation of algorithms and code, and shared datasets.

The initiatives are:

RunMyCode (www.runmycode.org) publishes online executable code focusing on finance and econometrics, but with a call to extend the procedure to all applied science code.

IPOL (Image Processing On Line, www.ipol.im) is an online journal publishing image processing and geometric processing reproducible papers. Each paper consists of a detailed algorithm description, a source code, an online execution facility, and an archive of experiments.

J-RASP (Journal of Reproducible Algorithms for Sound Processing), is an initiative with goals similar to IPOL but intended for sound processing.

These recent initiatives have permitted to gain insights and expertise into the technical challenges and possibilities of online reproducible research. Building on these experiences and contrasting their advantages and drawbacks, the meeting focused in solving several editorial, scientific and technical problems that arise from this new kind of publication model. In addition, the discussion also considered the challenges faced by the different initiatives in making the scientific community adopt the new publication system. The workshop helped establishing strong personal and institutional links, and advance on the following technical issues: software, libraries, online execution, journal strategy, journal formatting, relation with the community, institutional support, perpetuity.

This two pages report is necessary limited to a summary of the topics discussed in the meeting and presents the main conclusions and actions to be taken in the near future.

The questions addressed in the meeting were:

• How to introduce new authors to IPOL?
• Special issues proposed to foster the development of the journal in several areas of crucial interest
• Improving documentation and communication
• External impact study
• Article-level statistics
• Demos for anonymous submissions to journals/conferences
• Extension of the software rules to allow for Matlab submissions
• Web interfaces for multimedia data visualization
• New pre-submissions procedure
• Reader feedback mechanism
• Adding browsable code with articles
• Making explicit citation instructions in demo page
• Including watermarks in the output images
• IPOL partnerships and links
• A project of modification of the author and reviewer forms
• More readers and more money
• Towards IPOL Recognition: road to the impact factor
• Institutional affiliations
• Operating costs
• European projects
• NSF projects (USA)
• Demo group conclusions
• Audio/sound group conclusions
• J-RASP (Journal of Reproducible Algorithms for Sound Processing).

A detailed 27 pages report is available on request to Jean-Michel Morel. This report, elaborated during and after the meeting, contains all details on the above topics and on the decisions taken on them to develop two online journals (IPOL and J-RASP).