Figure 1: A research contribution by a citizen.
Part 3: Course of the project, experiences and interim conclusion.
Anna Luise Kiss reports from her film studies citizen research project.
1 November 2020
In the second blog entry I presented the project The cinematic face of the city of Potsdam and showed how I implemented central requirements for citizen research projects. Another important element in a citizen research project is evaluation. In its conception, different paths can be taken and different criteria can be used (see e.g. here and here). I have decided to use a formative evaluation, i.e. interim results for the entire project and its sub-projects are proactively compiled in the course of the project according to defined criteria, so that corrections can be made if necessary. On the basis of a first interim evaluation, I can give a partial insight into how the citizen research project actually went and formulate initial responses. Hopefully, these can be useful in the conception of a citizen research project as an addition to the basic requirements listed in the second entry.
Survey methods – theory and practice
As described above, I have provided three ways to report filmic finds. The lowest participation was via Instagram, followed by the app (Epicollect5) which was found to be only moderately acceptable to participants. E-mail was the most used way of reporting, along with the verbal report of cinematic finds, a way of communication I had not anticipated. The contact via e-mail did not result in the processing of the questions I had presumed would be processed. Instead, as with verbal references, invitations were extended to come by and take photos. These e-mail invitations and the verbal references now confront me with an evaluation difficulty: should such contacts be considered a research contribution by a citizen? A hotel owner, for example, invited me to his hotel restaurant because there are film posters hanging there and a film camera has been set up. He told me about his plans to update the filmic references in his hotel. Is it citizen research if a workshop participant does not become active him/herself, but “only” points me to a location where he/she found an artefact?
Figure 2: In a hotel in Potsdam, a stylized film strip on the wall in the restrooms. It shows photos of cinema lettering. After an invitation to the hotel in question, I took the photo myself.
From this I conclude that citizens have expressed a clear wish not only to participate in preparatory events, deliver data and participate in a process of joint analysis, but also to use the data collection as an opportunity for personal interaction, informal discussion and clarification. This is at least true on a regional level, if not more generally.
In the light of this it would have been more appropriate to offer joint field research from the beginning. In a similar project I would prioritize this method of data collection, if only because the personal encounters have resulted in valuable impulses for the evolution of project. An example: the owner of a guesthouse let me photograph a collection of autographs in the reception area. She told me that the collection was unfortunately no longer up to date. Prominent actors and directors were no longer willing to offer autographs for inclusion in the collection, but would only offer to shoot a selfie together with the guesthouse owner. It was only through this conversation that I realized that media change and the ongoing history of such change can be directly read off of filmic artifacts in urban space. The person in question would not have come to my planned analysis workshop. Only through the on-site discussion could this thought be incorporated into the project.
Reflecting on the role as communicator in advance
The role that scholars play in citizen research projects has hardly been reflected upon and certainly not been scientifically investigated. Susanne Hecker and Nina Wicke therefore make an important contribution to the reflection of role constellations in citizen research projects.[i] They have analyzed the roles of citizen science protagonists from the “political actor” perspective. The authors were able to show that political papers on citizen science focus on traditional role assignments rather than innovative approaches to breaking up existing power structures, hierarchies and role specifications.[ii]
In this context, the question of the personal communication skills that a scholar should bring with him/her to citizen research projects is too little discussed. In many projects online communication with the fellow researchers is all that is needed. Such communication allows one to enter into simultaneous relationship with many participants over considerable distances. Online communication can be controlled and easily traced. If, however, a project is based on the knowledge of local citizen researchers, personal contact with such researchers is unavoidable, either because it is provided for in the project or is subsequently demanded by the citizen participants. In my case, I had initially planned personal exchange only in specific formats (among others, public kick-off, site walk with the participants, public workshop). This resulted in communicative interaction with strangers, a situation which could not be planned for or controlled, despite a measure of preparation. Such a possibility should be carefully examined in preparation for a citizen research project. One should ask oneself if, as a researcher, one has the ability to deal with such situations in a confident, appropriate and fulfilling way for all participants, at once flexible and goal-oriented and, above all, free from fear and a feeling of reticence among inexperienced participants. The direct exchange with students and with colleagues takes place in established settings to which researchers have access as a matter of course. Within these settings (e. g. seminars and conferences), which almost always include an accepted and familiar framework, a series of unspoken rules apply. However, these communicative situations differ from those required for the success of citizen research projects. Such projects require a local focus that includes face-to-face interaction with members of the public. There are no established rules on which the participants can rely. Even the general reference to pedagogical skills that should be available for conducting citizen research projects[iii] is of limited help, because the pedagogical tools acquired in an academic context cannot simply be transferred to the communicative dynamics of a citizen research project. In my project, for example, I followed up on requests for additional unplanned direct exchange and had many conversations on film topics that were not directly relevant to the project. I also had to deal with criticism, because, for example, my project could not include memories associated with bygone experiences of filming in Potsdam. In existing publications on citizen science projects it seems to be assumed that all scholars will be able to handle the direct exchange with citizen researchers. In regard to quality assurance, however, it is in my opinion absolutely necessary to reflect very precisely on one’s own role in a citizen research project in advance. Admitting that one is not made for such communicative situations can mean, for example, that others are assigned the communicator’s role. In these cases, it can be useful to schedule a professional facilitator as project member. For me, communicative situations have so far been a matter of a hands-on interaction with the impulsive immediacy of events in the midst of the evolution of the project, an experience at once unpredictable and personally enriching.
Number of members in the research team and risk management
How many citizen researchers must actually be part of the research team for a project to be successful? I was advised by Rosy Mondardini, the managing director of the Citizen science Center Zurich, that only a few citizens can be expected to participate in humanities projects. However, a look at the high number of registrations for some projects on the citizen research platform Zooniverse gave a completely different impression. There, humanities projects with hundreds or thousands of citizen researchers are listed. Impressive numbers, which can be off-putting for those who plan a small-scale citizen research project.
Figure 3: Impressive numbers of participants: In the project Scribes of the Cairo Geniza 4,972 “volunteers” have registered (as of April 6, 2020). A project that has set itself the goal of deciphering one of the largest archives of the Middle Ages.
With the help of Montserrat Prats López’s dissertation (2017) on Quality Assurance in citizen science, I was able to explain the discrepancy between Rosy Mondardini’s prediction and the numbers on the platforms. The author has shown that signing up for a project is one thing – and active research is something else. The majority of citizens do not participate in the research process at all or only once after registration. However, in citizen science projects, there are individual, particularly committed researchers who submit most of the data, write most of the transcripts and/or participate in analytical or interpretative processes. To illustrate this, López lists one transcription project, among others, for which 3,000 people had registered, 400 of whom transcribed a manuscript in part or in full, while only 11 people continuously translated several manuscripts.[iv] A study by Henry Sauermann and Chiara Franzoni also shows that even projects from the natural sciences face the challenge of finding citizen researchers: the number of registrations may be very high, but on closer inspection, most citizens participate only once, while most of the research work is done by a few “top contributors”[v].
It seems important to me to emphasize that prospective initiators of citizen research projects should not be put under pressure by the number of registrations on citizen research platforms. There is no blanket answer to the question of how large citizen science teams need to be. Only the intended data corpus and the intended evaluation procedure are decisive. If, for example, a clearly limited corpus of film-related materials is to be translated in co-operation with citizen researchers, then the pages to be processed can be used to estimate, approximately, how large the team should be to complete the translation and analytical discussions within a certain period of time. For a high quality result in such a case, a very small group of dedicated team members can be the decisive factor.
My project is one of qualitative data collection carried out through so-called theoretical sampling up to the theoretical saturation of categories to be worked out in succession. I did not set an ideal number of citizen researchers, and it was not important to me how many filmic finds were submitted. For me, it was important that among the artifacts submitted there should be those that I probably would not have found myself and which also provide impulses for category formation and thus for theoretical insights. In my risk management – i.e. the consideration of possible pitfalls that may occur in the project and the determination of necessary back-up strategies – I assumed that it should be possible, even with a small number of citizen researchers, to find at least initial artifacts that provide starting points for further data collection and category building. I had taken into account an extension and/or repetition of the survey period for the citizen researchers and, above all, the supplementation of the initial artefacts with own data collections. Consequently, the call for participation was preceded by an initial collection in order to facilitate the development of the theses and for the testing of data collection methods. A first extension of the survey period and a supplementary collection could then be carried out. Due to the Corona crisis, however, neither a further data collection nor an on-site review or supplementation by me is currently possible. This is (still) ok, because I can take advantage of the flexibility of working from home to undertake the processing of the data in Google Map and for the creation of first categories.
The attendance figures so far: A total of 24 people have participated in the preparatory events. So far, 15 people have participated as citizen researchers. Not counted were entrepreneurs who invited me and for whom I did the documentation myself. Most of the citizen researchers have reported at least one finding place, sometimes with several artefacts (9 persons); but there were also up to 5 finding places, which were submitted by one person. 70 artifacts were added to the database by the citizen researchers, 8 of which I classified as initial artifacts. I was lucky that two persons from Potsdam actually turned out to be “top contributors”, who also became active as communicators of the project and are already thinking about a further citizen research project on the topic of film.
Figure 4: An initial artifact. Research contribution by a citizen.
One important lesson I have learned from the process is that in the run-up to a project one should carefully consider whether the investment of a lot of time and effort justifies the recruitment of what might turn out to be only a core team. One should ask oneself whether the discrepancy between effort and number of team members can be justified by the quality of the data and the joint data analysis. In addition, risk management must be taken very seriously, for example by taking into account extension and repetition times and designing follow-up instruments.
Interim conclusion – Try again, fail again, fail better
Actually, some things went wrong in the unfolding of my project: The App Epicollect5 was generally ignored. Although some important contributions have occurred by hotel and guesthouse owners, I am unable to list them under citizen research. Due to the Coronavirus (COVID-19) crisis it is currently unclear when and how the next steps – joint analysis, data collection and derivation of practical proposals for action for urban development – can be realized. It all feels a bit like failure at the moment. And yet I draw a positive interim conclusion. It is precisely because my data collection methods were not accepted as I had planned that the project has been enriched with important but fortuitous insights. The collected data is already very extensive and can be used to form categories. Because of the opportunity afforded by working from home, I can take enough time for this and make the further data collection and evaluation even more effective. I have found great supporters among the people of Potsdam, who have committed themselves to the project and contributed important artifacts. It seems that, by the end of the project, both sides will have deepened their knowledge and that, once the Corona crisis is over, this project will lead to another citizen research project initiated entirely by the citizens of Potsdam. And finally, I was able to put theoretical knowledge about open science and citizen research into practice. Therefore, I would rather speak of a measure of failure, but, also, in Beckett’s words, of failing better. Perhaps this is a process that participants in citizen research projects have to prepare for anyway: not everything works as theoretically conceived, but it is precisely from this mismatch between theory and practice that valuable impulses for a project and for follow-up projects can arise.
Every citizen research project must be individually designed and equipped, so it makes little sense to take only one project as a model – “there is no single “right” model”[vi]. It is always necessary to conduct comprehensive research and adapt suitable elements from existing projects. I would be pleased if my interim report on the “cinematic face of the city of Potsdam”, despite the challenges I have described, would encourage the implementation of further film- and media-science citizen research projects. After all, Film and Media Studies could be seen as predestined, by their very nature, for citizen research. For many people the whole range of media is an integral part of their lives. The relevance of scientific questions for the everyday life of potential citizen researchers does not have to be derived but is obvious. I am convinced that Film and Media Studies could also play a role in resolving problematic aspects of citizen research. Citizen science projects often address well-educated and technically equipped milieus, and it can be observed that predominantly well-educated males from the USA and Northern Europe participate in them.[vii] Citizen science is therefore still practiced under a principle of exclusion. The topics covered, the use of technology, and even the choice of language and communication channels need to be reconsidered in the light of this. Film and Media Studies – with it’s emphasis on, and knowledge of, communicative processes, it’s thorough integration into innumerable aspects of everyday life, it’s role in individual socialization, and its gender sensitivity – can make a contribution to ensuring that citizen research really deserves to be listed under the “umbrella term”[viii] open science. In principle, however, it is important to ensure that the appropriation of citizen research into one’s own research repertoire does not, inadvertently, restrict the freedom of science, by limiting the topics addressed to those in which citizens have a current and direct interest. Smolarski and Oswald counter this concern by stating that citizen science should not be about subjecting itself to trends in topics. This implies that one should not only determine the value of one’s own knowledge within one’s own discipline, but also try to think it more comprehensively and at larger scales.[ix]
If there are any readers who have already carried out or are planning to carry out a Film or Media Studies citizen research project or simply wish to exchange views further on this topic, I would be pleased to receive an e-mail: a.kiss@filmuniversitaet.de.
[i] Susanne Hecker, Nina Wicke, “Roles of actors in citizen science in international policy documents” in Thomas Bartoschek, Daniel Nüst, Mario Pesch (eds.), Forum Citizen Science 2019: Die Zukunft der Bürgerforschung (Münster 2019), 5–15, 6.
[ii] See ibid., 12.
[iii] See Daniel Wyler, François Grey, Citizen science at universities: Trends, guidelines and recommendations, (Leuven 2016) 4 and 10 and European Citizen Science Association (ECSA), Ten Principles of Citizen Science, September 2015.
[iv] See Montserrat Prats López: Managing Citizen science in the Humanities: The challenge of ensuring quality (Amsterdam 2017), 7–8.
[v] Henry Sauermann, Chiara Franzoni, “Crowd science user contribution patterns and their implications” in PNAS, 112(3) (2015), 679–684, 680–681.
[vi] Andrea Wiggins, Kevin Crowston, “Surveying the citizen science landscape” in First Monday, 20(1), 2015.
[vii] See Mordechai (Muki) Haklay, Suvodeep Mazumdar, Jessica Wardlaw, “Citizen science for Observing and Understanding the Earth” in Pierre-Philippe Mathieu, Christoph Aubrecht (eds.), Earth Observation Open Science and Innovation (Cham: Springer Open, 2018) 69–88, 74 and Wyler, Grey, Citizen science at universities, 10.
[viii] Núria Bautista-Puig, Daniela De Filippo, Elba Mauleón, Elías Sanz-Casado, “Scientific Landscape of Citizen Science Publications: Dynamics, Content and Presence in Social Media” in Publications, 7(1), 12 (2019), 1–22, 1.
[ix] See René Smolarski, Kristin Oswald Einführung: Citizen Science in Kultur und Geisteswissenschaften, in Oswald, Smolarski (eds.), Bürger Künste Wissenschaft, 9–27, 14.
Anna Luise Kiss is a PostDoc Researcher at the Film University Babelsberg KONRAD WOLF and currently heads the BMBF-funded research project The cinematic face of cities. She is editor of the anthology Jede Menge Perspektiven. Der Regisseur Herrmann Zschoche (CineGraph Babelsberg, 2014) and – together with Dieter Chill – of Pathenheimer: Filmfotografin. DEFA Movie Stills (Ch. Links Verlag, 2016) and co-editor of the current issue of the FFK Journal (Avinus, 2020). Her dissertation Topografie des Laiendarsteller-Diskurses – zur Konstruktion von Laiendarstellerinnen und Laiendarstellern im Kinospielfilm (Springer VS) was published in March 2019. Also recently published was “Eine Medienwissenschaftlerin und eine Schauspielerin unterhalten sich über Performanz in ihrem beruflichen Alltag. Eine Text-Performance“ in Thomas Etzemüller (ed.) Der Auftritt. Performance in der Wissenschaft (transcript, 2019).
Disclaimer: The IAMHIST Blog is a platform that offers individual scholars the opportunity to present their work and thoughts. They alone are responsible for the content, which does not represent the view of the IAMHIST council or other IAMHIST members.
