IN the brilliant, taut film Zero Dark Thirty, about the hunt for Osama bin Laden, there is the controversial suggestion that information obtained by torture may be useful.
The film's lengthier demonstration of the centrality of phone intercept and phone-tracking technology has surprised no one. Today we stand on the threshold of losing that ability and massively empowering terrorists and criminals as a result.
The Parliamentary Joint Committee on Intelligence and Security is about to make recommendations on changes to national security legislation. Most of the attention is rightly focused on the proposal for mandatory two-year data retention by telcos and internet service providers.
The committee has received all its submissions and has one more meeting, scheduled for March 1, before it finalises its report and recommendations. It is the most credible of all parliamentary committees. In Kevin Rudd, John Faulkner and Philip Ruddock, it has three veterans of cabinet's National Security Committee. Its other members are thoughtful, experienced people from both sides of politics, as well as independent Andrew Wilkie.
While no one from the committee will say what it will recommend, I believe it will favour some sort of data retention regime.
I also think there is already some draft legislation somewhere inside the Attorney-General's Department, which, though it may need modification in light of the committee's recommendations, is just about ready to go.
This is the most important law enforcement and counter-terrorism action proposed in the past 20 years. Without it, we are disarming ourselves in the face of our enemies. Every police force and every intelligence agency in Australia wants this regime enacted because they know how badly their ability to do their jobs will deteriorate without it.
Nonetheless, there will be howls of protest from a weird alliance of the Greens, the far Left and some in the Liberal Party who have civil liberties concerns. Those concerns are not trivial, but they can be accommodated within a data retention regime.
This whole area can become a technological wilderness of mirrors, but the basic points are easy enough to grasp. Until fairly recently most telephone calls were made on fixed lines through one provider. For billing purposes, the details of which numbers were rung by which other numbers were kept by the phone company and available to police if necessary. Details vary across jurisdictions but mostly police need a warrant to tap someone's phone calls. But they generally don't need a warrant to look at what is called metadata; that is, data about phone calls - how many calls you made, what numbers you rang, what numbers rang you. While a warrant is not necessary, a serious suspicion of criminal activity and a high level of internal police authorisation are needed to access such records.
You would be astonished at how central this kind of data is to almost all successful policing these days. Indonesia has been able to try and convict hundreds of terrorists, some of whom murdered Australians in Bali and many of whom wanted to murder more Australians, essentially because of the mobile phone intercept technology the Australian Federal Police has provided them with, and helped them operate.
A huge proportion of crimes that are solved - drugs deals, murders, kidnappings, frauds and the whole gamut - are basically the result of the use of phone tapping and access to metadata.
But now the world is changing radically because of technology. There are many providers of telephony and many of them do not keep such records. British police already find that something like a quarter of their requests for such information produces no result because the information does not exist, and for the AFP the figure is more than 15 per cent.
Increasingly, people are making phone calls over the internet. Internet providers charge by the quantity of data used rather than the phone calls made, so there is no record for the police to access.
Internet communications of all kinds are also now central to police and security agencies.
Unless internet service providers are required by law to keep this information, they won't do so. One estimate goes like this: if current trends continue and there is no new legislation, the investigative capacity of all agencies will fall by almost 10 per cent in a year and 50 per cent in a decade. This would be the greatest boon we have seen to all levels of criminality, and to terrorists.
Michael Danby, a committee member and chairman of parliament's foreign affairs committee, speaking as an individual and not foreshadowing the committee's verdict, told me: "I think the situation described by the director general of ASIO makes it imperative for the government to legislate for both data retention and strong parallel privacy breach penalties."
It's important to be clear about what the police and security agencies want access to here. It is not the content of telephone or internet communications. It is simply the fact of who communicated to whom.
It is not the police seeking a new power. It is a desperate effort to keep up with the way serious criminals, organised crime, cyber enemies and terrorists communicate with each other.
Let me make a prediction. Without this legislation there will inevitably be a mass terrorist event in Australia and then the legislation will pass in 12 hours.
Many nations are grappling with these issues. The parallel British parliamentary inquiry has just recommended a data retention regime.
There would be financial costs, easy to exaggerate, and very strict privacy legislation would be necessary. But the alternative is very bad.
There have been some disappointing elements of the debate. Some Liberals have been hysterical in their response to it. There is a disturbing drift among numerous Liberal politicians not to take national security agencies and their advice seriously. If Labor politicians were making such comments against these agencies with the Liberals in power they would be savaged.
Former attorney-general Nicola Roxon comprehensively messed up the process by not providing a draft bill. Her hyper-partisan politics also engendered needless hostility. Her replacement, Mark Dreyfus, can do much better. And he needs to, because this is business of the first importance.