Review: Human Compatible by Stuart Russell

Stuart Russell is a very influential figure in the Artificial Intelligence (AI) community. He is co-author of a widely studied text book on the subject, Artificial Intelligence: A Modern Approach, that was key to moving the subject from being dominated by a formal logic approach to one that focused communities of artificial agents operating and cooperating to maximise rewards in uncertain environments.

His most recent book, Human Compatible, is written for a general audience. It helps if you are a reasonably informed member of the general public but the book rewards the effort with not only an overview of the current successes of AI but also addressing the technical and ethical challenges with novel, constructive  proposals. It is highly recommended.

Russell takes the challenges to human purpose, authority and basic well being seriously without being unnecessarily alarmist. Even with current and immanent applications of AI there are threats to jobs but also real gains in efficiency and cost effectiveness in medicine and production.

The aim of what Russell calls the standard model of AI has been defined almost since its inception as

Machines are intelligent to the extent that their actions can be expected to achieve their objectives.

However the major challenge comes with an anticipated major step forward. That is the creation of  general-purpose AI (GAI). In analogy with general purpose computers, that is one computer can carry out any computational task,  GAI will be able to act intelligently, that is plan and act to achieve what it needs to achieve, on a wide range of tasks. This would include creating and prioritising new tasks. Currently AI produces highly specialised agents but many of the algorithms  can be put to diverse uses; learning different skills but once learned that skill is exercised narrowly. For example, Deepmind's AlphaGo can play the board game Go very well, better than any human, but that is it. Related underlying techniques, however, can be applied to a number of applications such text interpretation, translation, image analysis and so on.

The major breakthrough that would enable the AI to escape from the constraint of narrow specialism has yet to be made. As indicated above, GAI would be a method that is applicable across all types of problems and would work effectively for large and difficult tasks. It’s the ultimate goal of AI research.  The system would need no problem-specific engineering and could be asked to teach sociology or even run a government department. A GAI would learn what from all the available sources, question humans if needed, and formulate and execute plans that work. Although a GAI does not yet exist, Russell argues that a lot of  progress will results from research that is not directly about building "threatening" general-purpose AI systems. It will come from research an the narrow type of AI described above plus a breakthrough on how the technology is understood and organised.

It is the mistake to think that what will materialise is some humanoid robot. The GAI will be distributed and decentralised; it will exist across a network of computers. It may enact a specific task such as cleaning the house by using lower level specialised machines but the GAI will be a planner, coordinator, creator and executive. Its intelligence would not be human it will have significant hardware advantages in speed and memory it will also have access to an unlimited number of sensors and actuators. The GAI would be able to reproduce and improve upon itself. It would do this at speed. 

And the role for the human? None that the GAI requires. The GAI would be able to construct the best plan to tackle climate change, for example, but on its terms  and in a way that maximised its utility. And to maximise its utility it will find ways to stop itself being switched off.

This danger, according to Russell, comes from the aim that has guided the whole research programme so far. That is, to give an AI objectives that become the AI's own objectives. A GAI could then invent new objectives such as self replication or inventing better information storage and retrieval to speed up its own actions. A GAI could then be said to have its own preferences and effectively is a form of person. It need not be malevolent but equally it need not be motivated by benevolence to humans  either.

In short, the solution proposed by Russell to the problem of human redundancy is to engineer the GAI so it cannot have preferences of its own. It is engineered to enact preferences of individual humans. The final third of the book is devoted to exploring the ramification around a set of principles for benevolent machines:

1. The machine’s only objective is to maximise the realisation of human preferences. 
2. The machine is initially uncertain about what those preferences are.
3. The ultimate source of information about human preferences is human behaviour.

These principles are there to inform design but also regulation and the formulation of research programmes. Russell deals with these and the societal efforts and cooperation that will be required to deal with unintended consequences as well as the actions of some less than benevolent humans who may want to create a GAI that has a goal other than to satisfy human preferences.  The discussion addresses the challenges of multiple artificial agents and multiple and incompatible human preferences. Of course Russell does not definitively solve any of these but indicates routes to resolution.

It is important that this book shows there is an approach in which humanity can reap the benefits of AI research without being subjugated to a superior intelligence or being forces to implement an AI ban. It is a positive vision that I hope informs and shapes our approach to this technology. But there is urgency, because we do not know when or where the break through to GAI will take place. If it happens within the current standard model pattern of research and application then disaster threatens.

Choosing a voting system

Voting rules range form the very simple such as the plurality rule (commonly known as "first past the post") to a plethora of more or less complex schemes. These schemes attempt to capture notions of fairness such as proportional representation and and majority preference. Voting takes place for many purposes; even choosing a voting system. Whatever the more general position may be, if there is one, proportional representation has been Liberal Democrat policy on voting reform at all levels of government for so long it has become an entrenched view. In its defence, the policy is well worked out in the sense that there is a specific actionable proposal but the major problem has been in getting wider agreement on implementation. The Liberal Democrats went into the 2010 election with a specific position on electoral reform. The 2010 manifesto promised to:

Change politics and abolish safe seats by introducing a fair, more proportional voting system for MPs. Our preferred Single Transferable Vote [STV] system gives people the choice between candidates as well as parties. ...

Is there a liberal counter view. Yes, and it is embedded in a liberal classic. It is to be found in Karl Popper's major contribution to political philosophy "The Open Society and its Enemies". In this book Popper strenuously, but only briefly, defends the two party system and simple majority voting (plurality rule). In 1988 the Economist invited Popper to return to these themes in   The open society and its enemies revisited. In this piece he expands on his defence of the two party state and majority voting with detailed objections to proportion representation. Here I will engage with the presentation of the same arguments in an update provided by David Deutsch in his wide ranging book The Beginning of Infinity. Although, I will conclude that the criticism of proportional representation is not as conclusive as Popper and Deutsch would claim and that they underplay the weaknesses of "first past the post" or plurality voting, the critical engagement with their arguments brings out some points that strengthen the case for a proportional system.

Choices

In Chapter 13 of "The Beginning of Infinity" Deutsch devotes much space to working through examples to illustrate just how difficult it is to design a voting system that all parties agree to be fair. He then moves on to to more formal account using the setting of social choice theory.  This leads him to present and discuss the Arrow no-go theorem.This states that there is no rule that maps the preferences of the individuals in a group on to the preferences of the group as a whole that can satisfy a complete set of five intuitive, desirable and rational properties. These desirable properties (axioms) are:

1. The rule should define a group’s preferences only in terms of the preferences of that group’s members.
2. The rule must not simply designate the views of one particular person to be ‘the preferences of the group’
3. If the members of the group are unanimous about something – in the sense that they all have identical preferences about it – then the rule must deem the group to have those preferences too.
4. If a given definition of ‘the preferences of the group’, the rule deems the group to have a particular preference – say, for A over B, then it must still deem that to be the group’s preference if some members who previously disagreed with the group (i.e. they preferred B) change their minds and now prefer A too.
5. If the group has some preference, and then some members change their minds about something else, then the rule must continue to assign the group that original preference.

Remarkably, Arrow proved this set of 5 axioms is logically inconsistent. That is, no voting system can satisfy them all. This is a blow for a rational foundation to social choice theory but that has not stopped research and the development of a variety of voting rules.

In the UK the Electoral Reform Society (ERS) accepts the theorem but recognises the need to get on with voting system definition and evaluation, otherwise how would representational democracy work? An alternative, and more radical, reaction is to reject the social choice setting of the problem. This is implicit in "Open Society and its Enemies" but is made explicit by Deutsch.

The ERS introduce or exploit criteria such as locality and proportionality to rank and make trade-offs between voting systems. As could be anticipated from this approach, the ERS has come out in favour of compromise that includes an element of proportionality and and an element of locality, this is the Single Transferable Vote system, as adopted by the Liberal Democrats. In this system the more proportionality gained the less locality and vice versa, with the number of seats per constituency as the free parameter. In practice, the final position chosen will be a pragmatic trade-off with public opinion and wider political support.  

Proportional representation

After presenting Arrow's no-go theorem Deutsch attacks rather than discusses proportional representation (PR) voting systems. It is evident that PR shares weaknesses with all voting rules that are formulated in a social choice setting. Most of Deutsch's specific objections are effectively, if not absolutely conclusively, answered by the ERS in its pamphlet PR Myths. The first objection Deutsch makes is, however, not addressed in the pamphlet. This is:

... the ‘More-Preferred-Less-Seats paradox’, in which a majority of voters prefer party X to party Y, but party Y receives more seats than party X.

Deutsch neglects to mention that this also a weakness of the plurality rule. The system that avoids this particular paradox is Condorcet ranking, of which there are a number of variants, just as there are for STV. It would take us too far from the present discussion to examine Condorcet methods in detail. It has a formal weakness that the originator discovered, which is, basically, that a ranking is not guaranteed to exist. However this risk has recently been mitigated by factoring in preference structures in realistic population models. This recent work was carried out by Partha Dasgupta and Eric Maskin in a very formal version of social choice theory. However, what this objection highlights is that STV and other systems with similar ranking rules will be in conflict with an alternative view of fairness to PR, which is that preference should be given to option X over Y if  the majority prefers X to Y.

Beyond Social Choice

At this point Deutsch makes an interesting move against social choice theory itself that gains support from Popper's wider philosophy:

It [social choice theory] conceives of decision-making as a process of selecting from existing options according to a fixed formula (such as an apportionment rule or electoral system). But in fact that is what happens only at the end of decision-making – the phase that does not require creative thought.

The creative aspect is what happens before a set of choices are put before the electorate. Arguments, explanations, economic theories, values, public opinion and so on, all contribute to formulating the set of options. It is argued that the quality of the options on offer is more important than the voting rule that provides the preferences. Deutsch argues further that the weakness of the social choice setting and the paradoxes associated with it means that a more fundamental and wider criterion for voting is required. The more fundamental criterion is, according to Deutsch:

Popper’s criterion that the system facilitate the removal of bad policies and bad governments without violence.

Whether this is the only or dominant criterion, it is clearly a valuable one. Deutsch adopts Popper's arguments that a plurality voting rule matches this criterion better than any proportional rule. The ERS pamphlet PR Myths seeks to address the Popper objection directly. They frame it differently  as  "PR doesn't let you kick out an unpopular government", which moves it back into the social choice setting but it is close enough to the criterion for the purpose of the present argument. The counter evidence presented by ERS is straight forward observation. Countries that practice PR do not only get changes of government but it is not even a rare occurrence. This is followed up by the further observation that plurality rules have historically failed to remove unpopular governments and removed popular ones. Popper himself originally had the excuse that these observations were not available at the time of the first edition of the "Open Society" but not in 1988 and it is certainly not the case for  Deutsch. It is understandable that in the 1940's that the USA and UK were taken a the outstanding examples of the stable and effective liberal democracies but since then the problems with the two party system underpinned by a plurality voting rule have become evident.

Seen from the stand point of Popper's criterion both plurality and PR can provide in practice adequate mechanisms for removing policies or governments without resorting to violence but in both cases the importance of other institutions, constitutional checks and balances, and a fertile public sphere of debate and ideas is required. What this discussion is leading towards is the need to manage expectations of what can be achieved merely through adopting one voting rule rather than another. Behind Popper's criterion there is a philosophy closely related to his theory of knowledge. This requires a diversity of theories, conjectures, ideas and, in the political context, policies. This is facilitated by a system that encourages the growth of a diverse number of groupings that can formulate, criticise and propose solutions to the challenges in society. The plurality voting rule tends to lead to two major party blocks. Other sources of ideas and opinion can be safely ignore by these two major groupings. In the past, during the formative years of liberal democracy, the major groupings have been the Conservatives and the Liberals but for the last seventy years it has been the Conservatives and Labour. What also happens is the absorption by the major parties of more extreme positions, as can be seen clearly in the current situation in the UK and to some extent in the USA. In the UK the Conservatives contain a substantial and influential group of English nationalists and currently the Marxist faction leads the Labour party. So, there is some internal diversity but it is a management issue for the main parties not a constructive component of the national debate. Smaller parties are traditionally ignored except when there are small majorities or hung parliaments.

For a well functioning constitutional democracy it is this suppression of opinion and diversity that is the damning consequence of the plurality voting rule. PR is much stronger in nurturing small parties and opinion groups. What is needed is further reform of institutions that make sure that this diversity leads to more robust debate and effective policies.

Plurality voting rules should go, but this leaves Condorcet methods as an alternative to PR and there are numerous versions of PR. It is easy to see that Condorcet methods will tend to correct for any formulation of two polarising groups but whether these methods provide for and nurture diversity of opinion awaits further analysis. In a social choice setting the fairness encapsulated in the Condorcet mechanism is just as valid as the proportionality notion with which it is in conflict. Consideration should be given in a case by case basis as to which is the appropriate voting rule; whether in general elections, internal voting in Parliament and other bodies, and at different levels of government. Not only theory but the evidence from practice shows PR to have an established advantage in the creation of a diverse, multi-party public sphere, which together with the right institutions should provide a robust foundation for a liberal order. 

A critical rationalist approach to policy creation

The starting point for this piece is "What use is Popper to a politician?" by Bryan Magee. Magee is a long standing advocate for Popper and therefore the philosophical position known as Critical Rationalism (CR). Without wishing to take away from Poppers contribution, I prefer to use CR and make clear that this is not advocating the opinions of a person but to argue for a constructive philosophical approach to policy formulation. 

Magee is that rare thing in the UK a public intellectual who was an elected politician. First as a Labour MP then for the newly formed SDP. The article referred to above was written some 10 years following his career as an MP but is strongly coloured by his social democratic position. He seems blind to the non-social democratic liberal position which is mine. However this is not too grave as the philosophical stance being advocated is available to anyone who is open to rational argument and evidence.  Magee himself mentions Margaret Thatcher as the sort of radical conservative who could open to this approach.

In much of what follows I will follow Magee quite closely but my formulation is adapted to the creation policy proposals and amendments. The sort of work that takes place prior to a party conference. One of my motivations for this piece is dissatisfaction with both the process and with much of the output from this process.

CR itself is a subject with its own extensive literature, but you should be able to pick up the essential points relevant to policy formulation in what follows. CR was developed via a critique of positivism and induction in the natural sciences. It presents the scientific enterprise as an exercise in problem identification and resolution with the important caveat that the solutions are provisional and need to be subjected to continuing critical review. Policy formulation is not a natural science but the recognition of the fallibility of proposed policy proposals should be evident.  

So, in policy too, first identify and formulate the problem with care. This means not jumping to solutions or using the issue to display indignation or personal virtue. The articulation should be as clear and jargon free as possible. For example, in the case of health consequence of diet, it is necessary to formulate the problem, if there is one, as objectively as possible. If people are eating too much; that is their concern. If the are eating too much and damaging their health; that too is their concern. If over eating is leading to strains on the health service, leading to higher taxation, then that is potentially a real policy problem and it is possible to start to address it. But even here we do not stop. Having formulated the problem better we can now quantify it. This is not just looking at the evidence but looking at the quality of prognostics and the assumptions made. There are always assumptions.

The next step is to formulate policy proposals. It is the creative step, but based on best available knowledge. This is not just data but economic theory, philosophy, science and knowledge of how government works, and whatever else can be brought to the task. Other softer and more value oriented considerations should not be neglected. For example, ask whether it would be legitimate for free individuals to be constrained through a proposal that addresses a problem by managing a statistical distribution across the whole population. It is not possible to derive a solution for the body of knowledge, hence the creative element. In getting proposals formulated anything goes: debate with yourself, with others, writing Op-Eds and getting feedback, etc. The outcome should be a proposal or set of proposals that is clearly articulated, defendable and actionable. By actionable is meant that a policy is a solution to a problem and therefore if acted upon will, or at least intends, to solve that problem.

Done? No. The proposal needs to be subjected to a further critical analysis. An important mechanism is to try out the proposals against implementation scenarios. This is motivated by the recognition of unintended consequences of a policy. That is, the solution may not be robust to  small changes in the implementation scenario or it may have negative impact if implemented in a certain way. The outcome should be a ranked set of actionable policy proposals with supporting explanations and evidence. But not always.

It is quite possible that after much hard work and critical analysis no actionable policy proposal emerges. So, have we been too critical. Very unlikely. Remember that fallibility at each step is never eliminated.The problem as formulated may not have an actionable solution or the actual problem has not been identified after all. To return to the impact of diet on the health service;  is the problem perhaps with how the health service is structured. For example the health service has no, or weak, personal responsibility mechanisms. The other realisation is that in the end it may be better to do nothing. The process will not have been a waste because you will know why you are proposing no action. However in many and I would anticipate the majority of cases the approach outline here will indeed help in identifying strong and defendable policy proposals.

My main motivation for writing this is the often dismal to poor quality of policy proposal writing.  If in turn you are critical of the CR approach or my formulation of it then you too are participating.  Thank you.

Sustainable Energy - without the hot air: 10 years on

The free to download ebook version of Sustainable Energy — without the hot air was last issued on the 3rd of November 2008. Ten years have passed since this important and influential science based look at sustainable energy was updated. I am sure that if it was not for the early death of David MacKay, he would have built upon this achievement and updated and evolved his position. There have been calls for a collective effort to keep this book up to date, notably from Chris Goodall. This has not gained very much traction but books do tend to need a high level of personal ownership to advance.

Let's look at the approach MacKay took as it has been particularly robust to intervening events. Rather than focus on what 2008 technology could do he used physics to bound what is in principle achievable. He attempted to answer the following question as honestly as possible: how much energy is there to be harvested practically and in a sustainable form? Questions of social, economic and wider environmental impacts were not ignored. These factors lead to MacKay to propose a number of plans, of which the feasibility and affordability will depend on technical and behavioural developments. They each add up to 70 kWh per person per day (estimated consumption in 2008 was 125 kWh per person per day) and are shown below.

David MacKay provided a ten page summary and the details of the figure above are explained there. The presence of Nuclear and "Clean Coal" will not please environmental purists but MacKay's approach is pragmatic and clear that dealing with climate change will require major sacrifices, of which green sensitivities are not the most onerous. For any of these plans to be feasible MacKay's analysis points to a 25% improvement in heating efficiency and a 50% improvement for transport and other efficiency savings that must reduce overall energy consumption to about 60% of 2008 levels.. His analysis does not show that it will be easy to achieve this nor that it is achievable then or now through established technology.  He does argue persuasively that there no barriers in physics. This analysis lays bare, although largely implicit in the book, the need for political leadership and imagination that will galvanise humanity to meet the challenge.

Let's pause to reflect what an achievement the book was, how well it stands up against developments and what a loss David MacKay's early death was. Since 2008 the acceptance of the evidence for climate change and its human generated causes has gained much wider informed acceptance. The case for a technically grounded response to the challenge has never been stronger and MacKays example needs to be followed and acted upon. Science does not only provide the analysis but also the foundation for the technology that the solution to the climate change challenge will require.

Reclaiming and renewing liberalism: Free Trade

Free Trade as manifested in the repeal of the Corn Laws was a founding event for the Liberal Party. There are signs however that liberals are becoming wary of the phrase. This a clear symptom of the malady that the The Orange Book was seeking to address fourteen years ago. The ailment is that key liberal ideas have been appropriated and distorted by the Conservative Party and in the case of the phrase "Free Trade" by the right wing of that party. So, by association in the minds of many with a weak grasp of history the phrase has become in the UK a right wing Conservative slogan and tainted for that reason.

The phrase "Free Trade" on its own is open to a number of interpretations and requires context to clarify which interpretation is meant. On the Conservative right the freedom being emphasised is freedom from regulation. Liberals have long since moved on from a 19th century laissez-faire philosophy. Freedom should be understood as the liberty of all parties to enter into an exchange or not. This means commerce and trade is in harmony with the core liberal value of individual liberty. Domestically, it has long been recognised that the rule of law is required for competitive markets to flourish. Internationally a system of rules is required to allow trade between and across nations to be effective. Agreements that include harmonisation of regulations are necessary but we should be limiting them where possible to concerns like safety in the food chain or of electrical products, for example. Regulations should not be used to skew advantage to one party or a particular group.

Liberalism is a dynamic and adaptable philosophy. Today's goods and services are of a complexity that makes a laissez-faire approach, suitable for corn, unfit for the exchange of contemporary goods and services.  It is this dynamic and forward looking liberalism that is presented in A manifesto for renewing liberalism. But here too there a sign that the phrase "Free Trade" is not to be used and "Open Markets" used as a replacement. Are we to give in to the loud and strident Conservative right and cease talking directly about Free Trade and its benefits?

No, while looking forward, it is important to mark and take pride in key historical achievements. To the fore among these is the establishment of Free Trade as the engine of growth in the 19th century and then its resurgence after the 2nd World War based on a system of agreements and rules that recognise and protect human freedoms and rights.  So let us not be bullied out of using the phrase and support Free Trade as the strongest economic mechanism yet discovered for fostering growth and relieving poverty.