Building a Data Intensive Infrastructure for evaluating the role of Universities in Society

Dr Richard Hosking1

1Curtin University, Perth, Australia


It is distinctly a human response to optimise for, or even game any system of measurement. In this work we begin to question if our current metrics of academic scholarship tell incomplete stories or align with what we truly value. We consider the idea of an Open Knowledge Institution, and how open access rates, the diversity of hiring, policy direction, and research impacts are tracking across Australia, and also globally.

This talk focuses on our methods, infrastructure, code, and practices for data collecting to begin addressing these questions. From the use of Cloud Computing to dynamically scale and schedule our harvesting, to our partnerships for obtaining large data collections, and the ongoing challenges of working with incomplete, disjointed and often disagreeing data.

To date we have collected a multi-billion point dataset covering multiple decades, which extends well beyond Australian shores; linking funding, publications, citations and other forms of alternative impacts. To contextualise this BigData we have also collected deeper qualitative metrics of universities.

A indisputable conclusion of the work thus far is the wide scope of questions we have yet to answer, and the desire for further information to tell the full story. But there is scope to extend this framework: to collect more data, link more sources, and contextualise what we collect through further qualitative enquiry. But not all of this can be done without reaching out and working openly. We end by posing the question: what could an open digital scholarly observatory in the 21st century look like?


Richard is a Senior Data Scientist working at the Curtin Institute for Computation. He holds a PhD in Computer Science from the University of Auckland. He has worked in both Academia and Industry in the areas of Machine learning and Data Intensive Systems. This project is a collaboration with Centre for Culture and technology exploring the place of Universities in Society.


AeRO is the industry association focused on eResearch in Australasia. We play a critical coordination role for our members, who are actively transforming research via Information Technology. Organisations join AeRO to advance their own capabilities and services, to collaborate and to network with peers. AeRO believes researchers and the sector significantly benefit from greater communication, coordination and sharing among the increasingly different and evolving service providers.

Conference Managers

Please contact the team at Conference Design with any questions regarding the conference.
© 2019 Conference Design Pty Ltd