Comments on: “It’s not only peer-reviewed, it’s reproducible!”

By: Panton Fellowship Wrap-Up | Science and the Web

Panton Fellowship Wrap-Up | Science and the Web — Mon, 17 Nov 2014 10:39:41 +0000

[…] contain and how to use them. In my role as an advocate for reproducibility I wrote a blog post on why reproducibility should become a quality criterion in science. The post sparked a lot of discussion, and was widely linked and […]

By: Aime tes données… et permets à autrui de les aimer aussi | Science ouverte

Aime tes données… et permets à autrui de les aimer aussi | Science ouverte — Thu, 16 Jan 2014 15:42:27 +0000

[…] de répéter ou même de statistiquement vérifier une étude présentée. Cela a un nom : la recherche reproductible. Nous avons tous entendu parler du résultat choquant de l’étude de Glenn Begley de 53 […]

By: Love your data – and let others love it, too › The Aggregator

Love your data – and let others love it, too › The Aggregator — Thu, 16 Jan 2014 15:39:35 +0000

[…] the impossibility to repeat or even statistically verify a study being presented. This has a name: reproducible research. We have all heard about the shocking outcome of Glenn Begley's survey of 53 landmark cancer […]

By: Peter Kraker

Peter Kraker — Thu, 05 Dec 2013 14:27:24 +0000

Thanks, Matt! I agree, investing time into reproducibility will pay off in the long run. In the short term, however, it may seem that you are losing time. That's why I think that if reproducibility was seen as a quality standard, it would be easier to get people to commit to it.

By: Matt Fenwick

Matt Fenwick — Wed, 04 Dec 2013 20:02:34 +0000

Nice article! As a reproducibility advocate, one of the criticisms I've often run into from people is that making sure things are reproducible takes time away from "getting real work done". I hope these people realize how critical reproducibility truly is -- without it, what is science? We throw away so much information about how our data analysis was performed it's sickening!

By: Peter Kraker

Peter Kraker — Mon, 25 Nov 2013 13:52:31 +0000

Sounds like an exciting project! I think I need to read up on your latest publications :)

By: Fleur Jeanquartier

Fleur Jeanquartier — Fri, 22 Nov 2013 17:38:52 +0000

@Peter:
First and foremost thank You for linking to the open access version of the paper! 😉
To your question:
Science mapping is surely another interesting example usage and I like the idea of visually approaching the question of mapping science!
By doing this visually, we can reduce complexity with this approach. Currently many research is published on the topic of time visualization and this could be a benefit for the science mapping research too. I also recently came across certain research questions to model evolution. One of the latest case studies I participated in was dealing with the question how (biological) model visualizations evolve over time. This is also a really exciting part for new project ideas! 😉

By: Peter Kraker

Peter Kraker — Tue, 19 Nov 2013 11:10:03 +0000

Thanks Fleur for this valuable insight into the information visualization community! As I am working on science mapping myself, I noted that there is a trend towards making not only evaluations comparable but also the maps themselves: http://knowescape.org/standards-for-science-mapping-and-classifications-workshop-at-the-issi-2013/
What do you think about that?

P.S.: Here is a pre-print of the first paper that Fleur mentioned: http://hcil2.cs.umd.edu/trs/2004-30/2004-30.pdf

By: Fleur Jeanquartier

Fleur Jeanquartier — Mon, 18 Nov 2013 19:51:45 +0000

I also like the idea of promoting "reproducibility" as new important criterion for dissemination quality. I'd also underline the importance for this kind of quality metric with regard to information visualization. F.i. Plaisant, Fekete et al. (2008) promote development of benchmarks to facilitate the comparison of certain visualization techniques. North (2006) and also others already try to introduce and describe methods to produce comparable visualization evaluation results. I also underlined the importance of repositories for comparison in one of my latest publications. Thank you for introducing this term as a quality metric and reminding me of its' importance!

By: Peter Kraker

Peter Kraker — Mon, 18 Nov 2013 11:10:07 +0000

Limor, thanks for pointing out your comprehensive review. It was interesting to learn that the data curation process in your repository is driven by replication requirements. I am especially impressed that you check whether the submitted materials can actually be used to reproduce the results from the papers.