Vivian Qiang's ARIA project: Evaluating the Efficacy of AI Ethics Frameworks: A Case Study
A short outline of your ARIA research project.
Artificial Intelligence (AI), defined as a software that “learns” from data without explicit programming, holds incredible potential to aid and automate decision-making. AI technology also introduces numerous ethical concerns including fairness and bias, privacy, security, transparency, and more. This project aims to analyze the efficacy of existing AI ethics toolkits by applying their guidelines to a health tech start-up’s AI-powered product.
In the past four months, we have reviewed and categorized 38 AI ethics frameworks. Then, we selected four relevant “toolkits” and applied them through conducting interviews with stakeholders and reviewing company policies. Our expected results will include: a comparison of the two distinct forms of toolkits (checklists and processes), content analysis of the toolkit reports, and evaluation of the implementability of the selected frameworks based on personal experience. We also plan to brainstorm solutions for the discovered ethical risks with the company.
Tell the reader why you were interested in an ARIA project.
I am thankful to ARIA for providing the privilege of working from home. ARIA offered monetary and academic support, and opportunities for scholarly mentorship so I could continue actively learning and engaging in research despite the current circumstances. Having only conducted observatory research, I wanted to gain exposure to a different form of study that critically analyzes the role of science and technology. Throughout my interdisciplinary degree, I realized my interest in taking a “humanities perspective” on the sciences, and ARIA afforded the flexibility to tailor the research topic to my goals.
What were your learning objectives?
I hoped to participate in qualitative research and gain experience in managing my own project. I also strived to develop skills in building a professional partnership with the start-up and collaborating with company stakeholders. As I had minimal experience in the domain, I aimed to develop an understanding of the fundamental concepts of AI and its broader societal impact. Furthermore, I was interested in learning about the development and enforcement of regulations in the technology sector. More specifically, I wondered how AI systems fit within the current landscape of technology ethics and regulations. Therefore, we developed a project that applied academic ethical analysis to a real industry case study.
What were some of the highlights? Give some examples.
Despite the remote nature of the project, we fostered a welcoming and supportive online work environment. The team had weekly lab meetings where we shared our progress and discussed readings, building academic and personal relationships within the lab. We also held Zoom work sessions to encourage and hold each other accountable during the long online workdays.
I also enjoyed interviewing participants over Zoom, connecting with people from various backgrounds to develop an intersectional perspective on technology ethics. In addition to developing great scientific innovations, the company also demonstrated genuine efforts in helping their users and prioritizing ethics, which was fascinating to learn from.
What were some of the challenges that you encountered? How did you overcome them?
The most significant challenge involved an unforeseen delay with the Research Ethics Board approval for the study’s participant phase. As a result, we extended our schedule and used the extra time to construct a more comprehensive literature review. As we could not conduct any interviews until late July, I compensated for lost time by using company policies and reports to apply the frameworks which led me to form better informed interview questions.
As many others also experienced, it was difficult maintaining motivation and productivity during quarantine. To overcome screen fatigue and distractions, I set daily goals and established a consistent work schedule. Dr. Moon and our other lab members also helped keep me on track by regularly checking in and offering support. Our lab culture of working together even while apart helped alleviate the oftentimes isolating nature of online research.
How do you think ARIA has or will shape your future career and education path?
ARIA has provided a great opportunity to explore my interests with expert supervision and excellent support. I have gained clarity in the areas I hope to pursue academically and in my future career — health, technology, and policy. This summer, with Dr. Moon’s guidance, I grew confident in my ability to self-teach research skills and acquire knowledge in an advanced field. As a minority, I am inspired by and grateful to work in an environment with fellow women of colour in STEM. I am learning from their experiences and paths in this field, and how we can make academia a more inclusive space for all. I intend to continue working with this lab in the upcoming school year with Dr. Moon as my honours supervisor. I would like to thank the Arts Student Employment Fund for their generous support of my ARIA project.
