Its Time to Re-Evaluate

This week’s readings all focused on the importance of evaluation. A feature emphasized throughout the semester to create successful digital public history projects, the readings this week got more in depth in exactly how to go about evaluating successfully.

There are many different types of evaluation. In evaluating the success of your project one important step is to analyse your information through analytics. The Tate Museum in the UK focused on the importance of this type of analysis to ensure that you are reaching your audience, achieving your goals, and managing the project in an appropriate way interanlly. The Tate recommends using Google’s free analytics service to see your effectiveness and connect all of the elements of your web presence (social media, emails, website) to see exactly how effective your message is.

However analytics alone cannot determine a project’s success. Both “Leveling Up” and “Museum Evaluation without Borders” recommend moving beyond analytics alone. This is because While they argue that analytics are one important tool, they show its weaknesses since it only quantifies how many users are being reached, instead of qualifying their experience. They recommended the evaluation methods of paper and wireframe testing, play testing, soft launching, surveys, interviews and analysing responses, linking program activities with intended outcomes and impacts, taking a systems-oriented evaluation approach, using affirmative data collection approaches, and daring to have courageous conversations with visitors and staff.

The “Experiencing Exhibitions” article would lead you to believe that museums are not evaluating the visitor experience at all, calling for more cross-disciplinary studies of visitor experiences which do not focus on linear ideas of knowledge. However all of the readings from this week show that is not the case. The next steps called for in this article are actually achieved by the Tate’s evaluation process with its emphasis on experience over education. But perhaps the authors are just trying to call for more emphasis on evaluation. In that case the resources of the Institute of Museum and Library Services (IMLS) would be a great place to start.

The IMLS provides assistance for creating evaluation models under the broad categories of general guides for program evaluation and outcome monitoring, project planning tools for museums and library services, common evaluation methods and terms, measuring outcomes in museums and libraries, and network associations that provide evaluation resources.

Browing these resources, I found the Shaping Outcomes site within the Project Planning Tools section to be most helpful. In looking at Outcome Based Planning and Evaluation (OBPE) gives a step by step process on how to create a project, identify goals, build toward those goals, and help both staff and visitors achieve them. Creating a model similar to Content Strategy, this was helpful in setting up not only the steps for good evalution, but also what makes up each of the components of those steps so that you could clearly define your own needs, unlike the Templates for Creating Logic Models which assumed a certain amount of preexisting knowledge for evaluators.

Evaluation of your project, goals, resources, tools, and outcomes before, during, and after your project has launched is the only way to ensure continued success.


The Possibilities of Mobile Content for Museums

This week focused on the importance of content accessible for mobile devices. Creating mobile friendly content is important because the majority of adults around the world not only own cellphones but will use them as the primary source of internet access in the coming decades. This is a great way to take advantage of tools which the public already has, not requiring the institution to invest in tools. Even though smartphone ownership is a barrier to the content, this barrier is actually relatively low, making mobile content a good way to reach large audiences who may not have access to this information otherwise.

The CHNM project Mobile Museums attempts to help museums create cellphone specific content with small budget and small technical knowledge. With a focus on the needs of art museums, this site encourages museums to move beyond in-gallery displays alone, to make all current and future websites mobile friendly, and to create content designed for mobile web browsers generally instead of specific operating systems to help prevent sites from being too connected to transient technology. Understanding that many institutions do not have the technical skills to create these projects, Mobile Museums provides links to plugins, code, and conversations going on in the field on the topic to encourage collaboration.

A few of the projects which use mobile applications to move beyond in-gallery experiences alone are PhillyHistory, PhilaPlace, the Kew Botanic Gardens app, and the International Spy Museum GPS game.

The PhillyHistory website and corresponding mobile component have the goal of increased access to archival information. Using “mobile augmented reality,” the app helps to achieve this goal by reaching out to younger and more diverse audiences who are savvy smartphone users. In creating their application PhillyHistory struggled with using Layar technology for the project, ultimately deciding to create their own application instead, one that worked across various smartphone platforms, consistent with the recommendations of CHNM.

While PhillyHistory showed the trials and tribulations of working with different technologies, PhilaPlace looks at how to successfully expand an application from one specific project to other history and educational projects. With the central interpretive idea that “place is an important touchstone for memory, history, and culture” the PhilaPlace project used collective access and Google Maps API technology to provide primary source information to residents and organizations within specific Philadelphia communities. By providing the raw historical data, the PhilaPlace project then used collective local knowledge to create a rich history about the place over time. The project creators believe that the technological framework of this would be easy to expand to other projects and this is important because “by exploring the memories and records of place, we educate the public about the past to promote and protect neighborhood spaces, sites, and stories that hold meaning in the present.” The main hurdle to move from PhilaPlace to AnyPlace has to do with staffing and content control. The Kew developers challenge this idea of a universally useable application, holding an application can only be universal amongst those institutions with similar audiences and user needs found through extensive user testing.

The Kew Botanic Gardens Delightfully Lost project focuses on a different kind of mobile app which is actually less place based than either PhilaPlace or PhillyHistory. Instead of creating a narrative, the developers of the Kew app decided to provide information through maps and QR codes that was focused on giving information to visitors as they request it rather than designing a narrative. This was because during user testing they discovered that many visitors use their site not for learning but for social and personal experiences. This shows the importance of getting to know your audience, something which we discussed in our class on Digital Exhibits. However, once up and running continued user testing showed that once the app was available that users would use it in a traditional place based, wayfinding way. This shows the expectations that are inherent with this kind of application, the usefulness of continued user testing, and the ways that the app can be improved in the future to be more user friendly to the novice.

The Spy in the City GPS game run by the International Spy Museum featured on the MobileMuseum site uses two FBI cases as the basis for a walking tour around Washington, D.C. This project jumped out at me because of our Mobile Mall project. Part of D.C.’s appeal has to do with the power, intrigue, and mystery of the government leaders and agencies in the city, something which the Spy Museum uses to its advantage to teach people about international affairs in a fun way. While the very high tech aspects of their app may not be for our project, or for any project with a smaller budget, it does show appealing and engaging possibilities for our future content. It was also nice to see what can be done with a larger budget, since limited resources are so often a factor in museums.