Oct
28
Are You Joking, Minister?
Filed Under B2B, Big Data, Blog, data analytics, data protection, Industry Analysis, internet, privacy, Publishing, Search, Uncategorized, Workflow | Leave a Comment
Are we sitting comfortably? Then lets begin on a story of determination matched by delay, obstinacy about the exercise of rights matched by obfuscation in public office. Here, in my view you see the dandruff of bureaucracy, the manipulation of regulatory documentation, used to demonstrate the passive aggression of civil servants forced to maintain and defend their jobs by doing the opposite of what government and society have asked them to do. And in this instance, it affects the very ground beneath you feet – and the very foundations of your home.
The Coal Authority (TCA) is a very English institution. While it is designated as a part of the Department of Energy, it is in tortured fact a “non-departmental” body, subject to the Treasury, but not covered by regulations that affect departments of state. Its function is to license mining operations in the UK. Its residual power comes from its data – it holds records on every mine in the UK and can tell you where and when shafts were drilled. It sells this data to a licensed distributor who resells them to lawyers doing property searches. Tidy little earner, nice little data monopoly.
But then the world changed. We now know about the value of data. Memorably Gordon Brown had dinner with Tim Berners Lee and the beginnings of a release government information for commercial re-use turned from a dribble to a flood to a European regulation. A new regulator (sans teeth) was created from the ashes of HMSO (Her Majesty’s Stationery Office) and thus the Office of Public Sector Information (OPSI) was born. I served on its Advisory Panel (APPSI) for the first five years of its existence, so I was not too surprized when I interviewed a private sector data player having difficulty with TCA this week. If his evidence is typical though we need an urgent review of OPSI whose limited powers are clearly insufficient, and someone needs to call TCA to account.
But back to the story. What my interviewee wanted to do was to license TCA data and put it with other data to create a value-added service. Not content with finding out whether there was a mine shaft under the house you were about to buy, he wanted to be able to tell you whether the soil around your foundations made a flooded shaft and tunnels a risk. In other words he wanted to link the TCA data to third party data and increase its value and get a return on that effort. A man then in the spirit of the age, who should get a medal from the ODI (Open Data Institute). And how has he done? Well, less than impressively. First there was a great hassle about getting a licence to use TCA content. But TCA subscribe to the OPSI Fair Trader scheme (IFTS) so he was eventually able to do this, and his intended service was duly written into his licence agreement. Then at the end of last year 80% of the data arrived. So far so good, and work began on building the new service. But then Data delivery stopped and a letter arrived saying that TCA had reviewed its position and had now decided that further supply contravened its public task. But its public task is to license mining, not sell data at a profit. I smell budget cuts, job uncertainty and an agency seeking to augment what it gets from the taxpayer via government with what it can squeeze from a data monopoly. But am I right?
My patient interviewee was less suspicious. He went to OPSI and asked for his agreement to be fulfilled. Nine months have passed and no satisfactory action has taken place to regulate the matter. Rather than a regulatory enquiry on the breach of the agreement they have insisted that a formal complaint to TCA be made before any review can take place. A formal complaint that could take years to process and, in my own view, effectively allows the regulator to avoid regulating. Rather than asking for the TCA’s CEO to account for their actions, they seem to have passively accepted the fact that a “non-departmental” agency is a rogue elephant who can ignore them with impunity. There the matter rests.
But my informant is a man of determination, as I have said. So what does he do next to implement license agreements he has already with a data supplier who fears a fall in revenues if a better service which enhances the usability of the data for public use is allowed to intrude on the monopoly? Well, he can complain to his MP, or invoke the European Parliament and the Commission, whose original directive bringing public sector data rights of access for private sector re-use is at the base of all of this. But I was there on the Advisory Panel when the Directive became UK law, and saw how the intentions of the European legislators were softened and undermined by a statutory instrument which allowed government to load the dice in its own favour (“We have not included a right of appeal,” said the Civil Servant driving the legislation “since it is hardly likely tour own government will act against your interests!”).
So who knows what will happen next? In the US , where rights of access are much clearer, this sorry tale would be in the courts already. In the UK, with Sir Humphrey at work, we move through a fog of legal uncertainty in which the amounts that need to be deposited with the court before action can take place effectively deter SMEs from complaining, even if they could afford the legal fees. As we talked I went to the TCA website. Goodness me, it has a policy for relationships with data re-users: http://coal.decc.gov.uk/en/coal/cms/publications/data/data.aspx
Lets click it: I have tried several times today: “We’re Sorry – this page is not available!”. If you have read this far, have a go yourself. My hunch is that Sir Humphrey has fixed it for his own side, and as a result we have a case study of how the public will for better value data services for all of us is being frustrated. We will never build the information society this way!
PS. I have removed my informant’s name to avoid other data relationships that he has with government departments from being affected – though of course your own Government would never act prejudicially against you! But OPSI will know – and must act if the interests of SMEs are to be protected.
Sep
22
Goodbye Native Websites!
Filed Under B2B, Big Data, Blog, data analytics, data protection, Education, eLearning, Financial services, healthcare, Industry Analysis, mobile content, privacy, Publishing, Search, semantic web, social media, STM, Uncategorized, Workflow | 3 Comments
It was the second afternoon of the last EASDP annual conference, last Friday in Amsterdam. The Big Business of the day was said to be over, in that at their General Council EASDP, representing Europe’s directory publishers, had voted to merge with EIDQ, Europe’s directory enquiries services. Sic transit the glory of the yellow page players. I was sad – EASDP in its heyday ran some of the most entertaining meetings in Europe. I was happy, since I had lost a night’s sleep en route to Amsterdam and was approaching going home time. And then he threw this thunderbolt across the stage and rocked us all back in our seats, “You may never visit a native website again!”
The line had added impact in that it came from a former CEO of Experian’s B2B services, Phil Cotter. He was speaking for BIIA and his own consulting interests, and addressing the issues posed by predictive analytics. And he was skilfully piling up the arguments around a machine-to-machine future, the role of the intelligence in the network, the ability to track and map our activities as predicted by the past activities of ourselves and others like us. And suddenly, all of the chat about behavioural targeting and the future of advertising on the web crumbled into dust for me. The website now becomes a totally different proposition. This is not the display table, advertising driven, designed to bring users to your goods and services. This is the storehouse of your advanced metadata and this the key to your discoverability. Mostly you will get discovered by machines, so you need to be very aware of how to tell them who you are and what you are about in language they can understand and use. As it happens I am moderating a session at the Frankfurt Book Fair (http://en.book-fair.com/fbf/programme/calendar_of_events/detail.aspx?PageRequestId=6ea4655a-3dd4-4209-872a-fcd3a6240b02&a1850834-d682-44a4-9b98-1ff33a3bcb5c=72b77c9a-c2af-4cca-a94f-268d1d3987ed) where some of the best brains in STM will address this issue: yet Phil makes me realize that this is not just an issue for the advance guards of science and technology publishing: it is about to crash, with frightening speed, on your shore as well.
Later in the session, as Phil was explaining the way in which the LAPD use predictive analysis to create patrol patterns for police cars, a hand shot up. “If patterns of crime exist so that you can say where the next lookalike crime will occur, after a few nights the cars will be entirely in the wrong places” Phil explained gently that this was why the analysis was run every day, and thereby gives me a second insight into what is happening. We are still thinking at our own speed about real world cycles of change. It does not matter to the machine that we are so slow to process: predictive analysis can be run repeatedly to catch nuanced change in activity if that activity is important enough to justify it. Then again, most of the apps that run predictive analysis are going to lodged, for consumers and for commercial users, on the advanced smartphones of the future. There the emphasis is likely to be upon rapid decision-making in an increasingly time-constrained society. Predictive analysis only needs to be “right enough” to allow a decision to be made.
And, of course, intelligent predictive analytics software is everywhere you look. SAP and SAS have history here: IBM and Oracle have serious offerings: TIBCO and Orange have activity here. But have a look at WEKA from Waikato in New Zealand (http://www.cs.waikato.ac.nz/~ml/weka) for some fascinating stuff on machine learning, and kick the tyres of specialist players like Foresee (www.foresee.com) or Absolutdata.com. This is a fast-changing world and the time between research lab and application grows ever shorter. Meanwhile I heard a good interview on the radio last week. An independent television producer was complaining that the advertising muscle of major agencies like WPP was being used to compel the co-financing of the TV they wanted – no shared deal equals no advertising was the implication. And we were expected to disapprove of the power of advertising being used in this way. But what if the agencies simply have a realistic view of the future of advertising and want their business to migrate to different places in the value chain. They will discover in time that content production is not the route to riches, but maybe they have already worked out that advertising is unlikely to go to the networks without being wholly changed – by predictive analysis, by recommendation engines, by community buying and countless other network driven expedients. Once again, the power migrates in the network to the user.
Then Laurie Kaye, now at Shoosmiths as their lead man in media legal pyrotechnics, came on stage and told us about the “right to be forgotten”. Not a good day for advertising and lead generation – in a conference dedicated to advertising-based directories and marketing services. The world is moving too fast to allow for the realtime re-calibration of the trade associations.
« go back — keep looking »