-
Understanding accountability in algorithmic supply chains
Authors:
Jennifer Cobbe,
Michael Veale,
Jatinder Singh
Abstract:
Academic and policy proposals on algorithmic accountability often seek to understand algorithmic systems in their socio-technical context, recognising that they are produced by 'many hands'. Increasingly, however, algorithmic systems are also produced, deployed, and used within a supply chain comprising multiple actors tied together by flows of data between them. In such cases, it is the working t…
▽ More
Academic and policy proposals on algorithmic accountability often seek to understand algorithmic systems in their socio-technical context, recognising that they are produced by 'many hands'. Increasingly, however, algorithmic systems are also produced, deployed, and used within a supply chain comprising multiple actors tied together by flows of data between them. In such cases, it is the working together of an algorithmic supply chain of different actors who contribute to the production, deployment, use, and functionality that drives systems and produces particular outcomes. We argue that algorithmic accountability discussions must consider supply chains and the difficult implications they raise for the governance and accountability of algorithmic systems. In doing so, we explore algorithmic supply chains, locating them in their broader technical and political economic context and identifying some key features that should be understood in future work on algorithmic governance and accountability (particularly regarding general purpose AI services). To highlight ways forward and areas warranting attention, we further discuss some implications raised by supply chains: challenges for allocating accountability stemming from distributed responsibility for systems between actors, limited visibility due to the accountability horizon, service models of use and liability, and cross-border supply chains and regulatory arbitrage
△ Less
Submitted 22 May, 2023; v1 submitted 28 April, 2023;
originally announced April 2023.
-
Out of Context: Investigating the Bias and Fairness Concerns of "Artificial Intelligence as a Service"
Authors:
Kornel Lewicki,
Michelle Seng Ah Lee,
Jennifer Cobbe,
Jatinder Singh
Abstract:
"AI as a Service" (AIaaS) is a rapidly growing market, offering various plug-and-play AI services and tools. AIaaS enables its customers (users) - who may lack the expertise, data, and/or resources to develop their own systems - to easily build and integrate AI capabilities into their applications. Yet, it is known that AI systems can encapsulate biases and inequalities that can have societal impa…
▽ More
"AI as a Service" (AIaaS) is a rapidly growing market, offering various plug-and-play AI services and tools. AIaaS enables its customers (users) - who may lack the expertise, data, and/or resources to develop their own systems - to easily build and integrate AI capabilities into their applications. Yet, it is known that AI systems can encapsulate biases and inequalities that can have societal impact. This paper argues that the context-sensitive nature of fairness is often incompatible with AIaaS' 'one-size-fits-all' approach, leading to issues and tensions. Specifically, we review and systematise the AIaaS space by proposing a taxonomy of AI services based on the levels of autonomy afforded to the user. We then critically examine the different categories of AIaaS, outlining how these services can lead to biases or be otherwise harmful in the context of end-user applications. In doing so, we seek to draw research attention to the challenges of this emerging area.
△ Less
Submitted 2 February, 2023;
originally announced February 2023.
-
Towards an accountable Internet of Things: A call for reviewability
Authors:
Chris Norval,
Jennifer Cobbe,
Jatinder Singh
Abstract:
As the IoT becomes increasingly ubiquitous, concerns are being raised about how IoT systems are being built and deployed. Connected devices will generate vast quantities of data, which drive algorithmic systems and result in real-world consequences. Things will go wrong, and when they do, how do we identify what happened, why they happened, and who is responsible? Given the complexity of such syst…
▽ More
As the IoT becomes increasingly ubiquitous, concerns are being raised about how IoT systems are being built and deployed. Connected devices will generate vast quantities of data, which drive algorithmic systems and result in real-world consequences. Things will go wrong, and when they do, how do we identify what happened, why they happened, and who is responsible? Given the complexity of such systems, where do we even begin?
This chapter outlines aspects of accountability as they relate to IoT, in the context of the increasingly interconnected and data-driven nature of such systems. Specifically, we argue the urgent need for mechanisms - legal, technical, and organisational - that facilitate the review of IoT systems. Such mechanisms work to support accountability, by enabling the relevant stakeholders to better understand, assess, interrogate and challenge the connected environments that increasingly pervade our world.
△ Less
Submitted 1 June, 2021; v1 submitted 16 February, 2021;
originally announced February 2021.
-
Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems
Authors:
Jennifer Cobbe,
Michelle Seng Ah Lee,
Jatinder Singh
Abstract:
This paper introduces reviewability as a framework for improving the accountability of automated and algorithmic decision-making (ADM) involving machine learning. We draw on an understanding of ADM as a socio-technical process involving both human and technical elements, beginning before a decision is made and extending beyond the decision itself. While explanations and other model-centric mechani…
▽ More
This paper introduces reviewability as a framework for improving the accountability of automated and algorithmic decision-making (ADM) involving machine learning. We draw on an understanding of ADM as a socio-technical process involving both human and technical elements, beginning before a decision is made and extending beyond the decision itself. While explanations and other model-centric mechanisms may assist some accountability concerns, they often provide insufficient information of these broader ADM processes for regulatory oversight and assessments of legal compliance. Reviewability involves breaking down the ADM process into technical and organisational elements to provide a systematic framework for determining the contextually appropriate record-keeping mechanisms to facilitate meaningful review - both of individual decisions and of the process as a whole. We argue that a reviewability framework, drawing on administrative law's approach to reviewing human decision-making, offers a practical way forward towards more a more holistic and legally-relevant form of accountability for ADM.
△ Less
Submitted 10 February, 2021; v1 submitted 26 January, 2021;
originally announced February 2021.
-
Monitoring Misuse for Accountable 'Artificial Intelligence as a Service'
Authors:
Seyyed Ahmad Javadi,
Richard Cloete,
Jennifer Cobbe,
Michelle Seng Ah Lee,
Jatinder Singh
Abstract:
AI is increasingly being offered 'as a service' (AIaaS). This entails service providers offering customers access to pre-built AI models and services, for tasks such as object recognition, text translation, text-to-voice conversion, and facial recognition, to name a few. The offerings enable customers to easily integrate a range of powerful AI-driven capabilities into their applications. Customers…
▽ More
AI is increasingly being offered 'as a service' (AIaaS). This entails service providers offering customers access to pre-built AI models and services, for tasks such as object recognition, text translation, text-to-voice conversion, and facial recognition, to name a few. The offerings enable customers to easily integrate a range of powerful AI-driven capabilities into their applications. Customers access these models through the provider's APIs, sending particular data to which models are applied, the results of which returned. However, there are many situations in which the use of AI can be problematic. AIaaS services typically represent generic functionality, available 'at a click'. Providers may therefore, for reasons of reputation or responsibility, seek to ensure that the AIaaS services they offer are being used by customers for 'appropriate' purposes. This paper introduces and explores the concept whereby AIaaS providers uncover situations of possible service misuse by their customers. Illustrated through topical examples, we consider the technical usage patterns that could signal situations warranting scrutiny, and raise some of the legal and technical challenges of monitoring for misuse. In all, by introducing this concept, we indicate a potential area for further inquiry from a range of perspectives.
△ Less
Submitted 14 January, 2020;
originally announced January 2020.
-
The Security Implications of Data Subject Rights
Authors:
Jatinder Singh,
Jennifer Cobbe
Abstract:
Data protection regulations give individuals rights to obtain the information that entities have on them. However, providing such information can also reveal aspects of the underlying technical infrastructure and organisational processes. This article explores the security implications this raises, and highlights the need to consider such in rights fulfillment processes.
To appear in IEEE Securi…
▽ More
Data protection regulations give individuals rights to obtain the information that entities have on them. However, providing such information can also reveal aspects of the underlying technical infrastructure and organisational processes. This article explores the security implications this raises, and highlights the need to consider such in rights fulfillment processes.
To appear in IEEE Security & Privacy
△ Less
Submitted 30 May, 2019;
originally announced May 2019.
-
Reclaiming Data: Overcoming app identification barriers for exercising data protection rights
Authors:
Chris Norval,
Jennifer Cobbe,
Heleen Janssen,
Jatinder Singh
Abstract:
Data protection regulations generally afford individuals certain rights over their personal data, including the rights to access, rectify, and delete the data held on them. Exercising such rights naturally requires those with data management obligations (service providers) to be able to match an individual with their data. However, many mobile apps collect personal data, without requiring user reg…
▽ More
Data protection regulations generally afford individuals certain rights over their personal data, including the rights to access, rectify, and delete the data held on them. Exercising such rights naturally requires those with data management obligations (service providers) to be able to match an individual with their data. However, many mobile apps collect personal data, without requiring user registration or collecting details of a user's identity (email address, names, phone number, and so forth). As a result, a user's ability to exercise their rights will be hindered without means for an individual to link themselves with this 'nameless' data. Current approaches often involve those seeking to exercise their legal rights having to give the app's provider more personal information, or even to register for a service; both of which seem contrary to the spirit of data protection law. This paper explores these concerns, and indicates simple means for facilitating data subject rights through both application and mobile platform (OS) design.
△ Less
Submitted 14 September, 2018;
originally announced September 2018.
-
Decision Provenance: Harnessing data flow for accountable systems
Authors:
Jatinder Singh,
Jennifer Cobbe,
Chris Norval
Abstract:
Demand is growing for more accountability regarding the technological systems that increasingly occupy our world. However, the complexity of many of these systems - often systems-of-systems - poses accountability challenges. A key reason for this is because the details and nature of the information flows that interconnect and drive systems, which often occur across technical and organisational bou…
▽ More
Demand is growing for more accountability regarding the technological systems that increasingly occupy our world. However, the complexity of many of these systems - often systems-of-systems - poses accountability challenges. A key reason for this is because the details and nature of the information flows that interconnect and drive systems, which often occur across technical and organisational boundaries, tend to be invisible or opaque. This paper argues that data provenance methods show much promise as a technical means for increasing the transparency of these interconnected systems. Specifically, given the concerns regarding ever-increasing levels of automated and algorithmic decision-making, and so-called 'algorithmic systems' in general, we propose decision provenance as a concept showing much promise. Decision provenance entails using provenance methods to provide information exposing decision pipelines: chains of inputs to, the nature of, and the flow-on effects from the decisions and actions taken (at design and run-time) throughout systems. This paper introduces the concept of decision provenance, and takes an interdisciplinary (tech-legal) exploration into its potential for assisting accountability in algorithmic systems. We argue that decision provenance can help facilitate oversight, audit, compliance, risk mitigation, and user empowerment, and we also indicate the implementation considerations and areas for research necessary for realising its vision. More generally, we make the case that considerations of data flow, and systems more broadly, are important to discussions of accountability, and complement the considerable attention already given to algorithmic specifics.
△ Less
Submitted 15 November, 2019; v1 submitted 16 April, 2018;
originally announced April 2018.
