RepliCHI 2013 - Proceedings
The RepliCHI 2013 Proceedings are now available online.
Didnt submit, but want to talk about RepliCHI?
We are meeting informally for drinks between the workshop and dinner (Dinner is TBA), in the Concorde LaFayette Hotel Foyer Bar, 5:30pm - 7pm on Saturday 27th April 2013. Meet us there to talk about RepliCHI.
RepliCHI 2013 (Draft) Schedule
|Times||Day 1 - Sat 27th April 2013||Day 2 - SunSun 28th April 2013|
|09:00 - 10:30||Session 1 - Framing the Workshop:
||Session 5 - Tandem Talks:
|11:00 - 12:30||Session 2 - Differing Demographics:
||Session 6 - Tandem Talks & Planning:
|13:30 - 15:00||Session 3 - Things Change Over Time:
||Session 7 - Discussions:
|15:30 - 17:00||Session 4 - New Methods:
||Session 8 - Planning for the future:
|~17:30 - 19:00||
at LaFayette Hotel Foyer Bar
at location TBA
RepliCHI 2013 - Accepted Papers
- Replicating and Extending Research on Relations between Visual Aesthetics and Usability
by Noam Tractinsky
- Newscube Replication: Experience Report
by Sidharth Chhabra and Paul Resnick
- Do lab effects transfer into the real-world? And should we care?
Petr Slovak, Paul Tennent and Geraldine Fitzpatrick
- Teaching HCI Methods: Replicating a Study of Collaborative Search
by Max L. Wilson
- Replicating an International Survey on User Experience: Challenges, Successes and Limitations
by Carine Lallemand, Vincent Koenig and Guillaume Gronier
- Replicating and Extending a Facebook Uses & Gratifications Study: Five Years Later
by Tasos Spiliotopoulos and Ian Oakley
- Re-testing the Perception of Social Annotations in Web Search
by Jennifer Fernquist and Ed H. Chi
- Challenges of Replicating HCI Studies with Children
by Quincy Brown, Lisa Anthony, Robin Brewer, Germaine Irwin, Jaye Nias and Berthel Tate
- Replicating Residential Sustainability Study in Urban India
by Mohit Jain, Yedendra Shrinivasan and Tawanna Dillahunt
- Replicating and Applying a Neuro-Cognitive Experimental Technique in HCI Research
by David Coyle
- Replicating Two TelePresence Camera Depth-of-Field Settings in One User Experience Study
Jennifer Lee Carlson, Mike Paget, Tim McCollum
|Challenging RepliCHI||< < > >||Supporting RepliCHI|
|Is replication important for HCI?
by Christian Greiffenhagen and Stuart Reeves
|RepliPRI: Challenges in Replicating Studies of Online Privacy
by Sameer Patil
|Avoiding "It's JUST a Replication"
by Bonnie E. John
Forms of Replication
Four common notions of replication, which have emerged from RepliCHI events and discussions so far, and from other disciplines, include:
Direct Replication consists of attempting to entirely replicate a study or system, using the same format and with the same tools, and experimental protocol. The aim of direct replications is often to replicate a specific finding. Direct Replication is often driven by the aspirations of strong science to confirm that results are true, are not created by an unseen bias, or that they apply in different contexts (geographic, cultural, topic, task) to the original study . This method is often used as a teaching method for postgraduate students.
Conceptual Replications are systems and studies that focus on a certain principle or phenomenon and confirm findings using alternative methods. Of the three approaches, this is most common in HCI, in that multiple studies demonstrate the principles of direct manipulation. Many instances, however, are post-hoc reflections of their findings in the context of prior work. Through this approach we surmise heuristics about best practices for design or for evaluation.
Replicate+Extend is a common research method in which people first reach the level of prior research before investigating it further. This may involve reproducing a phenomenon before specifically investigating it further, or by building on the findings of the study. This form of research is often essential in understanding a form of interaction, after learning about the limitations of an initial approach, for example. However, Replicate+Extend is associated with the high risk of being described as ‘incremental’, and being rejected from prestigious peer-reviewed venues.
Applied Case Studies - One common form of replication is application – a special instance of conceptual replication. If HCI research produces a finding, and its application in real world contexts confirms it, then case studies are a form of replication. It is perhaps often one of your core desires that research findings are applied in real world contexts. Case studies of applying research findings, by practitioners in real world settings, are highly desirable, and this is an opportunity to engage with practitioners.
Reasons for Replication
For Research – Part of the maturity of a community involves reflecting and examining what has come from within it. We are currently driven towards novelty and impact, but part of impact is validating and understanding contributions. Having an archive of research findings that reflect directly on prior work would be highly valuable for our community.
For Practitioners - With an ever growing focus on integrating research and industry in within the HCI community, we can increase confidence in HCI findings by validating findings, whether our own or the findings of others. We can increase industry confidence, and thus our perceived value from outside our own community, of the work we together perform.
For Teaching – Replication of studies is commonly used in other disciplines as a teaching method for undergraduates or new postgraduate students. Trying to replicate a published study a) helps students to learn what is involved in performing a proper study, and b) reinforces the notions of properly reporting methods in papers (as they learn what is not reported in the papers describing the studies they have to replicate).
Workshop Key Dates
- Submit: 1811 Jan 2013, 11:59pm PST
- Notification: 8 Feb 2013
- Camera-Ready: 18 Feb 2013
- Earlybird Registion: 14 Mar 2013
- Workshop: 27th & 28th April 2013
Register & Attend
RepliCHI 2013 is a workshop at CHI2013 in Paris
Find out more about registering and attending on the CHI2013 Attending Page.
If you have any more questions about attending, please contact Max.
Repli CHI2013 is a 2-day workshop. The schedule is TBA, but we currently intend to focus on experience reports on the first day, and then go meta on the second day.
Call for Papers - CHI2013
We are inviting papers that describe experiences surrounding attempts to replicate HCI research. We believe that novel research findings should be published at venues equivalent to the magnitude of the results and their contribution. Experience Reports may be about a) previously published, b) unpublished, c) teaching, d) industry replications. Authors will retain copyright such that they can publish unpublished studies described in RepliCHI experience reports at a later time. The aim of these reports, therefore, is to help document the nature of replicating HCI systems and research, and provide a starting point for further discussions.
Consequently, we are primarily inviting experience reports (in 3-5 pages of ACM Extended Abstract format) that elaborate on:
- the challenges you faced,
- your successes,
- the limitations of your replication
- whether the results confirmed or challenged original findings
- whether any differences in experimental protocols may have accounted for any differences in results.
We will also accept detailed well-considered position papers about the replication of HCI research, which contribute to our community’s understanding of the topic.
Papers will be reviewed according to
- the insights into the challenges of replicating HCI research
- the objective nature with which they discuss the possible reasons for differences in results
- the professional tone in which they discuss the original work being replicated.
3-5 pages - not anonymous - ACM Extended Abstract Format
Position papers will be discussed when we go meta in the second day
Experience reports may be about a) previously published papers, b) unpublished studies, c) teaching replications, d) applications of HCI principles in the real world, e) or any so-far unmentioned form of replication
Publishing experience reports will not prevent you from publishing the actual research elsewhere