'Look at it this way. News is what a chap who doesn't care much about anything wants to read. And it's only news until he's read it' (Evelyn Waugh, Scoop, p.66).

'Journalists often end up in jail because of their commitment to reveal important matters that those in power want kept hidden' (Tim Dean, in The Guardian, 14th May 2005).

Two fairly extreme versions of the journalistic calling; the negative and the positive faces of that central and demanding task, which is to surprise the reader or viewer. Because journalism as a profession largely exists to surprise. Something that attracts the attention of a chap who doesn't care much about anything requires some professional skill in its presentation; more seriously, the liberating surprise of uncovering what too many people want hidden is – potentially – a moment of real moral change and needs some quite substantial resources to make it happen. The personal courage and commitment of certain journalists in the service of such moral change and vision is indisputable; and last week's award to Frank Gardner of the BBC is testimony to this.

The difference between the positive and the negative is something like this. A journalist may want to pursue surprise because he or she assumes that where most people are starting from is boredom – and so the surprise has to be at some level entertaining. Or they may start from the assumption that the real problem is not boredom but the fact that certain people have decided what's good for you to know (and are therefore quite happy that you should be alternately bored and entertained); what needs to be challenged is such people's right to decide for others.

Hence my title for this lecture. One of the most powerful defences the media can offer for controversial actions is, of course, public interest. The concealment of this or that set of facts damages that shared space in which we all combine to find ways of acting on our common concerns. Hiding something is in the interest of a particular person or party, giving them unfair advantage. Uncovering these facts restores a balance, even conveys a power. So revelation in the public interest ought to be the same as working for the common good – the journalist in the service of active democracy. That picture, of the media serving the creation of a genuinely and rightly questioning public, sometimes at considerable risk, is a deeply attractive image (and self-image), very hard to quarrel with. Which is why the 'public interest defence' is at first blush unanswerable.

What I shall be arguing is that this really is a proper aspiration for journalists, a justification for the whole idea of a journalistic profession and a necessity for mature democracy; but that it has itself to be looked at very critically. The assumptions of the way public interest is often appealed to in the present climate look less impressive under scrutiny. If the profession is to perform its necessary job, some aspects of current practice are lethally damaging to it, and contribute to the embarrassingly low level of trust in the profession (especially in the UK) shown in most opinion polls.

Let me start with two fairly basic (and in fact interconnected) points which relate primarily to British journalism, though there are some resonances elsewhere.

First: as some recent studies have emphasised (I'm thinking especially of John Lloyd's What the Media Are Doing to our Politics), there is a difference between exposing deceptions that sustain injustice and attacking confidentialities or privacies that in some sense protect the vulnerable. If we begin by assuming that the question to ask almost anyone – not just politicians – is the immortal 'Why is this bastard lying to me?' the effect is to treat every kind of reticence as malign, designed to deny other people some sort of power. Exposing what is for any reason concealed becomes an end in itself, because the underlying reason for all concealment is bound to be corrupt and mystificatory. The political culture of 'transparency' and the magic word 'accountability' reinforce an already sufficiently powerful trend. And there is the further problem of an unblinking determination to find buried (and probably discreditable) agendas in every public statement or decision. As Peter Wilby writes about the parliamentary lobby (New Statesman, 13 June 2005, p.23), it 'allows no political event...to have meaning in itself, like a piece of poetry in a postmodern university literature department...What does an NHS reorganisation or an "initiative" on behaviour in schools mean for doctors, patients, teachers or children? The political journalists cannot tell you. They can tell you that this is a Blairite or Brownite idea, that it shows the minister is "getting a grip" or losing it, that it will pacify backbenchers or enrage them.' Parallels beyond the world of parliamentary journalism are not hard to find.

But a moment's reflection ought to remind us that the templates at work here are inadequate. Various kinds of investigative process (including the actual processes of journalistic enquiry) require confidentiality and therefore concealment in order to guarantee fairness; certain things cannot be said while legal proceedings are in train; there is a convention about what can be said or shown about minors, especially the children of public figures. Even on the other end of the judicial system, when papers publish or threaten to publish the addresses of convicted paedophiles, most of us feel uneasy. It exposes individuals to mob law and does nothing at all to protect children. Medical and psychological records are confidential. Sensitive material around national security is confidential. Concealment isn't by definition unfair; it may be part of a system guaranteeing fairness. Which of us would happily contemplate our guilt or innocence being assessed by a casual majority poll or our medical records being public? And which of us relishes any actions or words of ours being subjected to exhaustive interpretation to reveal their true agenda? As even Freud said, sometimes a pipe is just a pipe...

Obvious enough, and of course most of us would agree so far. The point was eloquently made by Onora O'Neil in her Reith Lectures three years ago. But push it just a little further. There are undoubtedly facts which would be of huge interest to a certain sort of public, but are not by any stretch of the imagination matters of public interest in the sense that not knowing them creates or prolongs a seriously unjust situation. But in a culture where conventions of ordinary privacy and modesty have been massively undermined, it is hard to set any defensible conventions that restrict what is fair game. Human beings have always been fascinated by gossip about private lives (Suetonius's Twelve Caesars – written nearly two thousand years ago – is a good corrective to anyone who thinks we are a uniquely prurient and sex-obsessed civilisation); what is important is not to dress this up as something essential to democracy. Arguably, democracy guarantees not only access to significant information but also some sorts of defence of the personal realm and its rights.

However, this is not quite the main point here. What I am interested in is the presupposition that the area covered by inviolable professional confidentiality is very small and that nearly all concealment is therefore dishonest. Consider situations in which the general reporting for an uninstructed public of views or proposals during a period of difficult negotiation will undoubtedly skew or wreck the negotiating process – not too remote or difficult a scenario. Is anything owed to professional restriction of information in such a context? Since this is a routine political phenomenon, how just is it to assume that there are no boundaries?

I'm not claiming that the media invariably do act as if there were no boundaries (legal restriction still bites); yet high levels of adversarial and suspicious probing send the clear message that any kind of concealment is assumed to be guilty until proved innocent. But this case needs more than just assumptions to be morally persuasive. And having referred to 'professional' restrictions, I want to pass on to my second basic point about the difficulties of current journalistic practice, a point which is slightly more complex.

Implicitly defining public interest as a right to know any kind of information that is being withheld works with a picture of a mass of undifferentiated 'members of the public', who have no other social and corporate identity; and for them, any information at all is going to be in some significant sense empowering. For the purposes of media reporting, there are only information processors and information recipients. Whether a media outlet is basically oriented to the left or to the right, it still 'generalises' its public in this way, by working with the model of a totality of consumers with common concerns.

A public is a necessary fiction. If a journalist or broadcaster, or of course, rather more significantly, a proprietor wants to secure consumers, a sense of solidarity and loyalty has to be built up; and it is built up very effectively by two complementary strategies. One is to communicate as if every reader or consumer shared the same fundamental values and preferences and anxieties. The other is to communicate as if these fundamental values and so on were the natural moral world of everyone with a brain or a conscience. The calculation of what will surprise (or better still, shock) the public is based on a careful assessment of what is unassailable and utterly taken for granted by that public. The left wing press needs to know that 'Secret Government memo reveals plans to restore death penalty' will attract attention. The right wing press needs to know that 'Secret Government memo reveals plans to make national anthem illegal' will have the same effect. The public is assumed to be homogeneous; and this particular public is assumed to be representative of the real moral life of society.

This is how news is inevitably written; and it is written on the assumption that knowing about secret Government memos conveys to people some sense of increased power – if only in terms of warning about impending disaster. But the shadow side of this needs to be brought out. Not even the most loyal readers or viewers in fact belong exclusively to the imagined class of 'the public' for this newspaper or that programme; their identities are more of a patchwork. And this means that they can in no sense be – simply as the public for this or that outlet – representative of something called The Public at large.

Actual human discourse happens within a number of contexts, not in some sort of unified public forum. Actual human learning about most things that matter happens in overlapping sets of relations and conversations. In human life generally, information, significant and otherwise, is shared in such overlapping networks, and absorbed at different levels over time. The journalistic assumption, though, follows a market pattern, in which a product is refined and distributed to a public defined for these purposes as concerned only to acquire it. And where that product is 'information', the model is particularly problematic.

So there is a tension at the heart of the journalistic enterprise. Its justification is that it promises to deliver what other sources can't, information that is needed to equip the reader or viewer or listener for a more free and significant role as a human agent. But at the same time, it is bound to a method and a rhetoric that treats its public as consumers and the information it purveys as a commodity – which is therefore selected, packaged, and, to that degree, inevitably slanted. This unavoidable 'marketising' of the process has the effect of creating yet another interest group, the professional producers of information, whose power as suppliers in the market restricts the freedom of others.

Awareness of this paradox – explicit or implicit awareness – is part of what has generated and encouraged the world of 'new news, exploiting the once unimagined possibilities of the electronic media. It is the world of the weblog and the independent media centre; it is interactive, restlessly conscious of its own transient nature. If the classical journalist just occasionally nurtured the illusion of writing or speaking for posterity, no such fantasy is possible in the electronic world. In one way, it is the reductio ad absurdum of marketised information, indiscriminate information flow. From another perspective, the user's immediate access to both the producer and the rest of the audience radically undermines some of the power of the producer. Classical media outlets claim to serve democracy but often subvert the possibilities of an active, critically questioning public by assuming the passive undifferentiated public we have been thinking about. The drift in some quarters to near-monopolistic practices, the control of the product by careful monitoring of response and periodic re-designing – these evaporate when we turn to internet journalism. Ian Hargreaves, in his excellent Journalism: Truth or Dare, gives a sharp account of the difference made by these developments; surely this is the context in which genuinely unpalatable truths can still be told, 'unsullied by the preoccupations of the mainstream media' (p.259)?

Yes and no. Unwelcome truth and necessary and prompt rebuttal are characteristic of the web-based media. So are paranoid fantasy, self-indulgent nonsense and dangerous bigotry. The atmosphere is close to that of unpoliced conversation – which tends to suggest that the very idea of an appropriate professionalism for journalists begins to dissolve. Many traditional newspapers and broadcasters now offer online versions of their product and many have allowed interactive elements to come into their regular material, for example by printing debates conducted on the web. But they have not thereby abandoned the claims of professional privilege. The question that seems to pose itself is whether a balance can be struck between the professionalism of the classical media and the relative free-for-all of online communication.

Onora O'Neil spoke about 'assessable communication' as the ideal. This means incorporating into what is communicated some of the material you might need to judge its reliability: 'showing your workings', distinguishing more sharply between report and comment, allowing some ways of evaluating reported reactions to something (is this from a person or body who represents anything serious? Is this comment there simply because it is obligatory to have at least one really hostile voice, never mind its credibility?). A couple of weeks ago, Alan Rusbridger wrote for Newsweek a comment on that journal's recent troubles over the imperfectly confirmed story about the treatment of the Qur'an in US detention centres; his point was that media admissions of fallibility or provisionality could have the paradoxical effect of strengthening trust. Admit that what is written or broadcast is a highly provisional construct, produced (often) by non-experts under pressure, and this realism might offset the deep cynicism that is generated by a marked habit of reluctance to apologise or explain. It will be interesting to see if the spread of the online culture into the mainstream media by way of online publication and internet debate will move mainstream journalism as a whole towards this provisionality, towards a more general notion of 'assessable communication'.

What I have said so far boils down roughly to this. We need to deflate some of the rhetoric about the media as guardians and nurturers of democracy simply by virtue of the constant exposure of 'information', and we need to be cautious about a use of 'public interest' language that ignores the complexity and, often, artificiality of our ideas of 'the public'. We need to recognise that there is a difference between concealment that is corrupt and designed to exclude or disadvantage those who have a legitimate interest, and boundaries that are properly patrolled by professional systems of accountability and gain nothing from being opened to universal – potentially demagogic – scrutiny. It is a very difficult discrimination – it can be used easily enough as an excuse for avoiding proper questioning - but it helps simply to acknowledge that there is a discussion to be had, and that 'public interest' is not too readily to be identified with the majority prejudice of a particular readership. And finally under this head, we need a form of self-regulation that admits provisionality and provides means of assessment. We need journalistic work that equips its own critics.

The difficulty that surrounds these matters is compounded by a world of communication in which uninterrupted and instantaneous information flow is the norm. 'Breaking news' we read at the bottom of the screen, and we know that someone is being made ready to produce an instant reaction. When the pace of events slows, but the situation remains critical, there is a real practical problem (the last days of Pope John Paul): uninterrupted coverage with no significant change for long periods. But the point is about how the media constructs and manages time. Urgency is all; and when urgency is an inappropriate or inadequate response to a situation, the risk is either distortion for the sake of a quick story or of attention being shifted because a process is not moving at media pace. This in fact relates to a point touched upon briefly earlier on. We learn significant things in varieties of overlapping communities; and we learn them at different paces. Some things can be mastered quickly, almost instantaneously, some take significant time. And I suspect that the difficulty most of the modern media finds in handling religion is not simply some sort of hostile bias to belief as such, but the extreme difficulty of representing in an 'urgent' medium experience or awareness that is apprehended in common practice over time. Which is why, incidentally, the recent BBC series, 'The Monastery', succeeded in such a remarkable way; it was about what can be known only by taking time, in company. Perhaps observers of religious broadcasting should concentrate not on the time or space given to simple and static representations of religious views and activities but on how this method of following the 'real time' of religious knowing and experiencing can be fostered. The recent speech by the BBC Director General, Mark Thompson, to the Churches' Media Conference seemed to endorse very clearly the significance of this dimension to religious broadcasting – allowing religious knowledge to be complex and engaging in the way any serious human knowing must be.

But this is not only a question about religious broadcasting or religious journalism. Christian belief takes as fundamental the idea that humans are created for communication; they are gifted with language. They are designed to speak to God and to each other and to give names to the things of the world around them. They are who they are in and through how they communicate. There is quite a bit in the New Testament from Jesus and St Paul and St James on the dangers of 'idle' speech, speech that debases the currency because it is inflated, untruthful, aggressive, contemptuous or salacious. Corrupt speech, inflaming unexamined emotion, reinforcing division, wrapped up in its own performance, leaves us less human: fewer things are possible for us. Bad human communication leaves us less room to grow. So the question that a religious believer, a Christian in particular, might want to be pursuing here is what the responsibility of the media is for the quality of communication in a society.

I am not talking about the charges of 'dumbing down'; that's a different problem. Nor am I talking about indecent language – again a different problem. The bigger question is about what is made more possible or less possible by what is said. What is the measure of the human that is shown in styles of communicating? The kinds of corrupt speech I have mentioned assume certain things about what it is to be human that are not self-evident, however strong the evidence. Manipulating fear. Exhibiting violent conflict between people for entertainment. Living off internal feuds and dramas between members of the profession. These not wholly unfamiliar elements in our current media culture take for granted a number of things about what is humanly natural and important in a way that, left unchallenged, closes down areas of the imagination.

And the trouble is that in the world of uninterrupted and instantaneous communication, these are more than ever the easy options, because they deal with surface dramas. The degree to which material is produced with a tacit slant towards these unexamined responses is the degree to which communication is 'shutting down the plant'. It may be true, as Steven Johnson argues in his recent book (Everything Bad is Good For You), that much material now being broadcast or published requires a quicker intelligence than comparable material from twenty years ago. But a quicker intelligence doesn't of itself guarantee an imaginative depth, a sense of knowledge as tied in with processes that take time.

Serving democracy and nourishing the common good is, for the media, something that requires not only attacking corrupt secrecies in a society, but also defending non-corrupt communication. And this defence of non-corrupt communication has something to do with a point discussed earlier. Journalistic communication is bound to a market model, whose ambiguities we have looked at; it is not going to change overnight by moral exhortation. But it is genuinely a 'parallel universe' when set against the actual ways in which people learn. This does not go unnoticed, and it contributes to the exasperation and scepticism with which so much of the media world is treated. The only thing that could in some ways offset this is a sense that journalists were sensitive to the varieties of actual communities in which information was processed and understood. And this is not an easy sense to maintain when there is a dominance among commentators – and columnists in particular – in the national media of people whose main or exclusive experience is urban, usually metropolitan. Once again, whether the journalist is professedly on the right or the left matters less than their location in a particular kind of elite. It makes it a good deal harder for them to be successful in facilitating conversation between the actual diverse groups in a society.

Ian Hargreaves (op.cit. pp.229-30) notes some internationally based research on journalists which offers an interesting profile of the profession – predominantly male, young, drawn from the majority ethnic group in their society and university educated. In the UK at least, we could probably add, for the national media, that their professional experience has been largely London-based. If it is a significant part of the profession's justification that it helps to equip a maturely questioning democracy, it is unfortunate if its profile suggests a strong tribal identity which may be pretty far removed from the specific local and civic loyalties that form the raw material of serious 'discursive politics', to use John Lloyd's phrase. And that suggests in turn that the profession has to ask some questions about how it works to help interaction and argument between real local and civic communities, resisting the temptation to apply metropolitan templates as the obvious frame of reference. My own sense of the risks here was intensified by the nature of some of the national media coverage of the foot and mouth epidemic a few years ago, which revealed some disturbing gaps of information, let alone empathy, in regard to rural affairs. In respect of religious communities of all kinds, the problem seems endemic.

My argument is that 'public interest' if it is understood as the process of opening up conversation and debate between the real communities of learning that make up a society, is a real and crucial priority for a society's health, for the common good. It is too important to be reduced to a battleground where information is dragged out of reluctant and secretive powerholders (secretive powerholders, that is, other than media magnates), or to a gladiatorial spectacle staged by an unelected political opposition. There is a real task which certainly involves unwelcome questioning of unaccountable power. But this will only retain credibility if it shows more awareness of its own limited and therefore compromised position; a task which involves brokering understanding between actual groups and their views rather than simply assuming that there is some kind of homogeneous public of information consumers waiting to be serviced. So it is also a task which entails taking responsibility for the quality of communication. If the other tasks are to be performed in a way that is morally credible – and that is essential to a healthy and properly critical common life – there must be some sense of journalists working for the sake of a humanity that needs not just a supposedly liberating flow of information but a wide imaginative horizon, in which cynicism is checked and facile emotion challenged.

To conclude: good journalism is one of the models of good conversation and communication in the wider social context. That is, it may be and should be at times argumentative and one-sided; but it must leave room for reply and even provide material for reply. It must work with a sharp sense of what it is that different kinds of community know and how they know it. Without this, it will move constantly further into its parallel universe. And so long as there is real work in a real world to be done by the news media, this movement into a parallel universe would be a disaster.

We need people who are recognisably professionals in facilitating exchange and mutual critique between the worlds people (actual three-dimensional people) inhabit; it is an aspect of the common good for which 'public interest' is often an inadequate paraphrase. Common good requires public space. But public space is a good deal more than a market, in information or anything else; and it has to be a space that doesn't demand that every speaker before entering the discussion, be reduced to an abstract member of the public, a consumer of general information.

A flourishing, morally credible media is a vital component in the maintenance of genuinely public talk, argument about common good. Such talk is not in rich supply just now, and it is only fair to ask what share of responsibility the media has for this. But it is not fair to treat media as a scapegoat. The relation with the wider society is mutual; societies to some extent have the media they deserve and license. Can a more realistic, less fevered, more modestly provisional journalistic practice recover a sense of how to nurture public conversation in a mature democracy – even of a truth that sets people free?

© Rowan Williams 2005