Introduction
I want to talk about how we are discussing AI. We are living in a narrow set of borrowed opinions, and they aren’t particularly good ones. I’m building on a previous argument I made relating to AI and values. We all understand the security situation isn’t improving. The industry talks about how data breaches are on the rise or that ransomware payments are increasing. It seems like we struggle to deal with the current set of problems let alone start thinking about dealing with new ones. The current paradigm we use to address these is ineffectual. So, how does perpetuating existing thinking into new technology help us? I’m not convinced that it does and the more I think about it, I’m not sure it ever will.
There is some level of hyperbole associated with the term AI although in reality much of it is an extension on what already exists. There are some distinctions to outline upfront. I am not talking about AI systems with a pre-defined task that could be achieved via other existing means such as OCR, voice recognition, alerting and detection. What I am talking about here is where we have a system that takes some form of input, makes and evaluation, and gives and outcome or decision based on that information, perhaps generative AI and as we approach general AI this becomes a more pronounced problem. But in short, we are talking about a synthetic substitute for human decision-making capacity.
With each publication of industry checklists, frameworks, and risk management regimes relating to AI, I move closer to despair. We are repackaging lists of existing controls that have been around forever and applying them to the shiny new toys. A mapping exercise does not constitute meaningful thought on the subject. Does this set upon ourselves a mark of Cain? Have we dispensed with the able? I imagine you will have enough familiarity with these frameworks and check lists to see why I won’t need to address this point in extenso. But I will treat myself to revel in the moment and highlight the irony that the industry is reluctant to shift left into an area of discussion that would be truly useful. But there is a view in the industry that we are making good steps forward. Merely reading some of this rubbish will serve a prophylactic for that mental disorder.
It’s an obvious problem.
If a business has a strategy, then it is the business processes that achieve that strategy. We might want to be concerned with those over the technical interfaces. These processes are the mechanisms in the business that create value to the business. So how does an AI support or uphold the business processes that achieve the strategy? What problems does it introduce to that business process? Those would be better questions to start with, but we need to go further. We might consider that there are decision points within a process and those decisions are currently made by humans. You might replace that decision making with some kind of AI. Then what?
Typically, when we talk about AI risks, threats, and security, the conversation degenerates to technological concerns. The click, click, whirly, beep, beep of protocols, and interfaces. You might hear of ‘prompts’ being sent to generative AI which leads to ‘poisoned data’, ‘hallucinations’, and other ‘miscellaneous technical shenanigans’. Buzzwords are judiciously scattered around like participation trophies on a primary school sports day. ‘Technology leaders’ speak in an authoritative way about the problems of fairness, bias, inclusion, transparency, accountability, or equity in AI technologies but aren’t taking the additional step to reach the crux of the issue. The real problem is that discussions are at the surface level with the exception of some enclaves of insight outside the sphere of security practice.
But these problems of fairness or bias are based on what exactly?
Discussing these concepts at any worthwhile level means you are engaged in a conversation about political philosophy. At the point you mention fairness or bias then you are forced to acknowledge that the definition of this word not only matters, but it differs based on politically anchored axioms. IEEE provided a statement that summaries this reasonably well.
If machines engage in human communities as autonomous agents, then those agents will be expected to follow the community’s social and moral norms. A necessary step in enabling machines to do so is to identify these norms. But whose norms?
Who’s norms? That’s a good question and it really hits at the heart of the problem, and that this we talk about values, morals, and ethics yet we are seemingly held in a state of paralysis when thinking about our own, or those of the organisations we operate in. There is an additional dimension to this conversation, the EU AI Act which has introduced fundamental rights into the conversation. It requires that developing and securing AI systems uphold these rights . . . this seems like that is a point of note. How can you uphold something like fundamental rights if you don’t understand the basis on which they are predicated? The short answer is you can’t, and it turns out definitions do matter.
There aren’t universal definitions for these attributes, they are subjective. The understanding of these concepts is predicated on political philosophy and should be understood in the context of values. This is a problem of values and a conversation of political philosophy regardless of if you like it or not. At the point you are invoking statement around fairness, bias, or anything of that ilk, then you are already having a conversation about political philosophy and you are required to have a reasonable understanding of these terms to engage with it in a meaningful way.
And that is the problem, we aren’t going far enough in conversations to understand what it truly is we are dealing with. We aren’t performing the requisite due diligence on the synthetic decision makers we place into our businesses.
Fairly straightforward.
Let’s pick at fairness a bit and which might illuminate why the current thinking is superficial. A number of big tech companies would determine that fairness can be achieved if aspects like bias, inclusion, accountability, equity are addressed. The WEF say that “the notion of fairness itself is fluid and requires a multi-stakeholder consultation” and say that this is addressed by removing bias. This is incorrect, obviously. There is no need to consult on the definition of fairness if you are clear about the underpinning values. It is somewhat concerning there is a discussion in this respect but it’s understandable why there is.
Fairness in the liberal tradition would generally be characterised as “equality of opportunity” meaning all persons have the same available opportunities. Perhaps a reasonable definition might be “the consistent application of process under the same rules and parameters”. However, the outcomes may lead to underrepresentation as individuals from different backgrounds have the liberty to make different choices. The problem is that “equality of outcome” is the metric under which determinations of fairness are made. So, it’s taken that disproportionate representations are evidence of bias or unfairness, but these fail to acknowledge an individual’s liberty and assume that a person from any given demographic will exercise their liberty in the same way as those from a different demographic. A disproportionate outcome does not mean that it is an unfair one, yet this is the conclusion many would lead us to believe. This is why tests for fairness primarily orient around testing such as demographic parity. It’s a common conclusion but is a non sequitur, and a conclusion that is inconsistent with a liberal value base.
There are a couple of very concrete examples we can point to. As we come into an election year we will see all manner of vote forecasting. It’s generally accepted that older age demographic will vote in different ways to younger ones i.e. tend towards conservatism. This is just accepted because it’s consistently seen. We might see acceptance rates for life assurance decrease at higher age demographics but does this constitute systemic injustice? Of course not, it would be a non-sense to suggest such a thing. Equality of outcome or demographic parity is a poor proxy as a measurements of fairness.
John Rawls outlines principles of justice as fairness, but a key point he outlined was that there is a priority order. These principles say nothing about the fundamental definition of fairness, rather these principles are contingent on context of a justice system, but they have utility in demonstrating some of the emergent complexity. The first principle takes priority over the second principle. This means there is a hierarchy of those principles and it’s not a requirement that the subsequent principle is upheld in all cases.
First Principle: Each person has the same indefeasible claim to a fully adequate scheme of equal basic liberties, which scheme is compatible with the same scheme of liberties for all;
Second Principle: Social and economic inequalities are to satisfy two conditions:
a) They are to be attached to offices and positions open to all under conditions of fair equality of opportunity;
b) They are to be to the greatest benefit of the least-advantaged members of society (the difference principle).
When discussing values, we have to consider how important they are to us relative to our other values. Values do not hold moral equivalence, yet we have a tendency to flatten them in such a way that removes nuance and understanding. We are left with sound bites and check lists.
This has consequences if we are trying to establish if an AI is fair or unfair. How we make determination as if this is true can be a nuanced and subjective process as the measures need to be calibrated to your definition of fairness. Even looking at something as seeming straight forward as fairness reveals that the question is not a straightforward one and requires introspection on the part of organisations developing or consuming AI technology. Failing to do so presents challenges to it’s stability if this technology is being used to execute critical business processes.
The AI Fifth Column.
Let’s consider what the training of an AI might entail. The training of an AI would consist of supplying it data, curating that data to remove undesirable elements, and validating the output to ensure it conforms to expected parameters. A human agent is making a decision to omit data or include data into the training of the AI. What data is being provided or how it is being curated is not neutral, selection bias comes into play. It’s important to differentiate between cognitive bias and the type of bias commonly referred to in the context of AI. The latter tends to be a thick concept pertaining to a particular set of views such as ‘historical systemic injustice’. It is the claim of historical injustice that forms the basis on which data of the past is altered, supplemented, or removed, to train the AIs of today. Data that is altered to remove bias becomes a form of bias at the point the first alteration is made.
This will get a little bit controversial, it might make you somewhat uncomfortable. Prejudice is considered to be negative and something that is to be removed. Edmund Burke presented a counterpoint to enlightenment values and to paraphrase a perspective he offered, ‘prejudice is the solution to problems we have forgotten’. Prejudice is predicated on the existence of a continuum or continuity of a cultural or ethic group and this can be described as a form of cultural memory.
Prejudice renders a man's virtue his habit; and not a series of unconnected acts. Through past prejudice, his duty becomes part of his nature.
But as we seek to remove prejudice, then what other aspect is societal norms will be lost. If, as the IEEE states, AI should conform to societal norms then it isn’t a ‘point in time’ perspective, it has to be a longitudinal one. The existence of a shared history means that it is possible that the erasure of data is the erasure of cultural memory. To dispel any ambiguity, I’m not arguing for prejudice, I’m merely making a point on the temporal aspect of societal norms.
So, by manipulating data sets for training purposes with the intent of removing bias, unfairness, prejudice, or any number of undesirable aspects, we are curating a view of the past and providing that to AI systems as their basis of decision-making. At the worst extremes we might have to consider that these processes to train AIs become a Ministry of Truth viz:- “Who controls the past controls the future. Who controls the present controls the past”.
Testing or human facilitated learning can fall into a similar trap, especially where supervised or reinforcement learning is concerned. Judgements of right or wrong are subject the whims of the trainer. This talks to deontological norms that inform obligation, permission, and prohibition. These set the boundaries of behaviour for the AI, what it must do, what it can do, and what it can’t do. Additional constrains might also be applied after the training and testing, akin to Robocop’s prime directives but the main thrust will be determined by how it’s trained and what feedback it gets when it’s tested.
As we start to move toward general AIs, conversations about the fundamental values need to be addressed or we will be subject to the whims of the select few corporations with the capital to create these technologies.
There’s not an obvious solution.
Organisations take prompts from the WEF Global Risks Report and will likely have positions on climate change, perhaps implementing net zero policies. That is easy to take a position on, it doesn’t require much thought and its low reputational risk with no real immediate commitment that generates good PR in the hands of a competent marketing department.
Yet the same risk report with risks identified by the same people considers that ‘extreme weather events’ as the highest risk in the next two years, also identified ‘AI generated misinformation and disinformation’ and ‘societal and/or political polarisation’ as the following two highest risks. So, if you believe that misinformation or political polarisation is a risk, then you have to take a position predicated on values.
What is the solution here?
Well, the EU AI calls for a fundamental rights impact assessment. But what about the values of the organisation consuming an AI service or tool? Wouldn’t it be prudent to make a determination about the organisation providing such a tool and the training provided to the tool to ensure that it is aligned with your values? Norbert Wiener wrote “we had better be quite sure that the purpose put into the machine is the purpose which we really desire”. As we desire that an AI is fair or that outcomes are consistent with our values, we need to have checked that it aligns to our values in the first place.
There has been some discussion as to a solution to this problem. Proposals for Ethics by Design have posed. In a paper by Brey and Dainow, they make the observation that “design choices are not morally neutral”. And this is really where security needs to be on the ball.
Is this even a problem for security?
Yes, it is. Well, we talk about insider threats a lot, don’t we? And what are we doing with this technology? We are introducing synthetic decision-making capability or content generation into organisations. We might say something like we can use ‘human in the loop’ or labour the point of human supervision, but then how effective will this be on a protracted timeframe and would there even be appetite to supervise this far into the future.
If our primary purpose is to protect the organisation, then that applies to the mechanisms that organisations depend on to exist as sustainable entities. We can’t silo ourselves away from difficult questions because they aren’t related to technology and can’t be addressed by a check list. It would be negligent for a security practitioner to ignore the fundamental considerations that will inform how this technology develops, a failure to execute the first responsibility.
You might not agree, or like what I have to say on AI, and that’s fine. But there is a big part of the conversation relating to values and morals of AI that remains unaddressed by the security industry. We can flirt with the idea of principles by nominally exploring what is fed to us by tech companies and standards bodies. That comes in easy to consume frameworks and check lists . . . but then we are failing to apply critical thinking and not exploring the assumptions and premisses that really talk to the foundation of the problem.
When you start writing bad cheques, they don’t start bouncing right away.