Wolfram Alpha – how not to make friends and influence people!

Hmmm…this is becoming WA corner recently – take a look at my previous piece here.  I was less than impressed with the technology and considered it either over-hyped or released too early in to the world.  However, I did hope that as time progressed there might be improvements in the results set returned and, more importantly from a developer point of view, an API published that would allow developers to build new applications to stretch and maybe improve WA.

So, this week an API was announced for Wolfram Alpha on the company’s Blog and I was pretty excited about the prospect of trying out a few things.  Despite my grumbles about the results returned, I was hopeful that with a suitable API encouraging third party developments, the underlying technology and data sets at WA might see an improvement.  My hopes survived for as long as it took me to start reading the small print – in particular this little document, the price list.  Now, I’m aware that WA has cost money to develop but to charge for developers to make use of teh API seems to be one of the dumbest and most counter-productive things they could do.  There are some ‘pioneer grants’ available for the developers, but I get the impression that these are still likely to involve shelling out money.

Google do not charge developers for use of the API until you start using the API in ‘closed’ systems and with a large number of calls.  They certainly don’t charge you during the development cycle – they have more sense.

Now, let’s assume I wanted to develop an API based application for WA – what we in the trade call a ‘proof of concept’ model – i.e. something that proves whether or not the bright idea that we sketched out on the back of a beer-mat in the pub will actually work.  How many requests might I get through to develop such an application?  Well, the other day I wrote some code to retrieve data from a Postcode / Geocode system’s API.  Now, this was a VERY simple application – send a Postcode, retrieve a list of addresses, send a code number, retrieve a full street address with map reference.  Let’s say 2 calls to the remote API for something very straight forward.  During code development and ‘in house’ testing I made about 30 or 40 API calls.  Now, during more formal testing on the client site that’s going to increase somewhat – probably in to the low-hundreds.  And this is for a problem with a well defined structure, with a finite returnable answer set – i.e lists of addresses, a single address or nothing at all, all in a set, predictable format. 

By the very nature of the sort of problem that WA has been set up to deal with, the problems passed up via an API are unlikley to be as well defined and the results set returned is also unlikely to be as simple to deal with as my addresses.  When I did some API work with Google for a client I found I was generating hundreds of API calls and responses during development, let alone testing.  For WA, I’m looking at $60 for 1000 API requests, and $0.08 for each additional request beyond the thousand I initially pay for.  Obviously, I can buy a bigger bundle, but the inference is clear – it ain’t gonnna be cheap developing for the WA API. 

API developments typically involve a learning curve for the API syntax and methods of use.  This is par for the course and to be expected.  However, when the API is interfacing to a curated data set like WA, we have an additional problem of whether the data set will actually contain the sort of data that we’re wanting to get back.  And whether it will be available in the sort of format we’re interested in.  And whether the curated data is timely compared to the data that is being made available through non-curated data sets like those available via Google – or other APIs, for that matter.  Clearly, if your problem space IS covered by WA and the data set WA has available contains what you want in the format in which you want it, then perhaps the API fee is worthwhile.  But for those developers wanting to try something new out, they’re most likely to look to free APIs to test their ideas, and spend time and energy working the wrinkles out in an environment that isn’t costing them pennies for the simplest query.

I’m afraid WA have dropped the ball big time here; by charging for ALL development use of the API they’ve alienated a large source of free development and critical expertise.  Look at how Google has benefited from the sheer number of developers doing ‘stuff’ with their various APIs.  Can you imagine that happening had they charged all the way?  Hardly likely. 

If WA were to make a limited  ‘sandbox’ set of data available for developers via a free of charge API, that would at least allow the developers to get the wrinkles out of their code.  The company could then charge for use of the ‘live’ WA datasets, and would have the additional advantage of the code being run against the live system being reasonably bug free.  By charging from the first line of code written, they’re restricting the development of their own product and driving people in to the arms of Google, Amazon, Bing and the like.  WA doesn’t appear to be offering a lot that is truly revolutionary; so-so natural language query interface against a curated data set.  I doubt it will be long before third party developers start producing the same from Google.

Book Review – ‘Mere Christianity’

Mere Christianity cover from Wikipaedia

Mere Christianity cover from Wikipaedia

When I used to commute between work and office I used to do a regular(ish) item on here called ‘The Bus Book’ in which I reveiwed the book I’d been reading whilst on the commute.  One book I intended to review as part of that series, but never managed it because the commuting finished, was C. S. Lewis’s ‘Mere Christianity’.

C. S. Lewis is probably best known for his children’s classic ‘The Lion, The Witch and The Wardrobe’, part of the ‘Narnia’ series of stories about a fantastic land in which magic has true power.  The books are also deep Christian allegory, reflecting Lewis’s great abilities as a writer on the topic of Christianity and Christian apologetics.

‘Mere Christianity’ grew out of a series of radio lectures that Lewis was asked to do in the Second World War.  The BBC approached a large number of writers and artists to develop radio programmes in the war – Orwell and Priestley were amongst Lewis’s fellow contributors to the literary war effort – and Lewis contributed a series of programmes describing the ‘guts’ of Christianity – the common issues that the Christian Faith of all denominations has to deal with.  And these programmes, after the war, became the basis of ‘Mere Christianity’.

I’ve often commented that the mental processes that led to my eventual Confirmation in to the Church of England were started by two men – Johnny Cash and C.S. Lewis – both of whom came to their belief via what’s best described as a ‘non-standard’ route – Cash through feeling the presence of God when he’d decided to give up and die in a cave, and Lewis coming back to belief after many years as an Atheist.

‘Mere Christianity’ is a relatively slim book, but heavily laden with ideas.  Stylistically it hasn’t aged well in the 60 years since the material was originally written, but that’s not necessarily a bad thing.  The style is best described as ‘no-nonsense’ and the book approaches Christianity from, in my opinion, a very Anglican perspective, although the theses within are applicable to all Christian denominations.  The Anglican faith is often said to be based on three cornerstones – Faith, Tradition and Reason – and it is this statement that Lewis uses as the basis of his ideas in the book.

The book is split in to 4 sections –

  1. Right and Wrong as a clue to the meaning of the universe
  2. What Christians believe
  3. Christian behaviour
  4. First steps in the doctrine of the Trinity

Central to the arguments of the first part of the book, where Lewis puts the case for Christianity, is the idea that there exists a general ‘law of morality’ – a rule about right and wrong known almost implicitly by all humans.  Whatever our beliefs, most people would argue that the Holocaust was wrong at any number of levels, that child-murder is abhorrent, etc.  (This was written 60 years ago – I guess it says a lot about the changes in morality in the last 60 years that I had to think hard when writing that last sentence!)  Lewis argues that for such a universal rule of right and wrong to be known to people irrespective of culture, there must be something above and beyond us to impose such a rule.

Lewis then posits what is now known in theological circles as the ‘Lewis Trilemma’ – an argument that is now a little dented by modern theological studies but that stated that Jesus was either divine, lying, or insane.  As His behaviour didn’t seem to indicate insanity, and his works did not indicate the moral turpitude associated with lying, Lewis was left with the conclusion that Christ was indeed divine.

He explores the virtues and the sins – I have to say that on reading this book for the first time the idea of  ‘pride’ being a sin – maybe THE sin -came as something of a shock to the system but when Lewis explores the idea that extreme pride is often at the back of the other sins, such as gluttony and lust – then perhaps it’s not such a long shot.  He then points out that Pride was what separated the Devil from God in the first place, so that rather put the hat on it!

Lewis’s exploration of virtue, sin and morality from a Christian perspective are interesting and well grounded.   He states very clearly that his intention with the book is to bring people who might be intrested in becoming Christians in to a sort of spiritual ‘waiting room’ where they can determine which particular branch of Christianity their calling will be for.  And it works very well on that level.  he does not intend the book and the ideas within it to be a doctrine of their own.

I think the only issue I woudl take with the book is the language and general style – it’s a little ‘stuffy’ and in a couple of places distinctly politically incorrect – and whilst that doesn’t bother me one jot I can see some people being put off.  My advice would be to persevere – the book was written 60 years ago by an upper-middle class male academic, but the issues it deals with are eternal.

I agree wholeheartedly with Anthony Burgess’s comment about the book : “…the idea persuader for the half-convinced, for the good man who woudl like to be a Christian but finds his intellect getting in the way.”  It’s a great and useful book – I wish I’d come across it earlier in my personal spiritual journey.  An excellent companion for Lewis’s religious novel in ‘letter’ form, ‘The Screwtape Letters’.

Google and ‘The Dead Past’

Earlier this year we saw the launch of Google’s Street View system – here – and with it came a plethora of complaints about the invasion of privacy implications.   I was one of the happy complainants – Google had a view right through my house, showing people in the house.  To be honest, their reaction was swift and the imagery was removed, but it was an invasion of privacy and I’m still to be convinced that there is any long term gain to be obtained from the system.  yes, I’m aware of all the ‘well, you can see what a neighbourhood’s like before buying a house there’ arguments, but if you do all your checking out of the largest investment you’ll ever make on the Internet then you deserve to find yourself living between a Crack Den and a student house.

Enough…step back and breathe…the title of this piece is ‘Google and ‘The Dead Past’ – now what on Earth do I mean by that?

Science Fiction afficionados amongst you may recognise part of the title as coming from an old story by Isaac Asimov, in which a researcher develops a time viewer to look in to the past, only to eventually realise that the past starts exactly a fraction of a second ago – for all practical, human purposes, the past, to his machine, is identical to the present.  He’s accidentally invented the world’s finest surveillance machine.  As a character says at the end of the story – ‘Happy goldfish bowl to you, to me, to everyone, and may each of you fry in hell forever.’

Now, there’s a looooong way to go between Google and eternal damnation through surveillance, but as is often pointed out, the road to Hell is firstly paved with good itentions and always starts with a single step.  Let’s do soem of that old style extrapolation, though, and see what we’ve got coming up in our future.  Here are a few things that have been posited and talked about as being part of our online future,  some of which are already here, some of which are extrapolations, all of which are technically feasible, if not yet politically acceptable.

  1. Decreased latency between changes in the online world and those changes turning up in Search Engines.  At the moment we might expect a day or so even on busy sites regularly trawled by search engines – a possible future might be that items get folded in to search space within hours.  We’re also already heading towards Tweets being searchable – perhaps future APIs will allow combined searches of facebook, Twitter and general webspace all in one shot?
  2. Use of  ‘mechanical turk’ approaches in encouraging people to use their spare time to classify images, scan online video, etc.  to tag media that are currently not searchable by search engines in their raw form. Imagine that being idone in near real-time.  DARPA are already researching tools to extract context out of text and digitised speech; perhaps some degree of automated scanning of video will follow.  And it’s not outlandish to suggest that what might be useful for the military will sooner or later find its way into civillian online life.
  3. The possibilities inherent in IP Version 6 for a massively enlarged Internet Protocol addressing space make it easier than ever to ensure that everything that can have a separate IP address will have a separate IP address.  Combine that with the geolocation capabilities that come with reduced cost GPS chip sets – many phones now have GPS built in – and the tracking of devices (and their owners) in real time or near real time, sold to us as extensions of the social media experience, becomes a reality.
  4. The increasing usage of ‘Cloud’ computing where everything about you is stored not on your computer or phone but on a ‘cloud’ storage system run by your phone company (T-Mobile?), software supplier (Microsoft?), media seller (Amazon?) puts all your digital life in to teh network – where it can be scanned and examined in transit or in storage.

Add to the technical advances the willingness for peopel to share their activities via Social media (or eventually the commoditisation of their activity patterns and media interests, as ISPs and phone companies realise that people will give up a lot of privacy for cheaper connectivity) and we are perhaps heading towards the science fiction scenario described above.

If people were concerned about the impact of Street View on their lives – a single snapshot taken as a one off – imagine the possible impact of your real-life world being captured as a mosaic by different sources and then being rendered and made searchable by interconnected search tools.  A phone call positions you in one place, photographs taken on the same phone and geo-tagged by the software are sent to a searchable social media site and so identify who you were with and when.  You show up in other photos,  as a recipient of a call from another phone, and so on.  The other evening I was asked ‘Who doesn’t want to be tagged in these photos?’ – the new social nicety for people who are concerned over the privacy of their friends.   Sooner or later I’m certain that nicety will slip by the wayside, and it will be up to us to police our own image online.

A recent business enterprise where people are being asked to monitor CCTV cameras in their spare time  – Internet Eyes – may be regarded as distastefully intrusive, but I do wonder whether it’s the start of a whole range of ‘mechanical turk’ type activities where people are encouraged to act as high-tech lace-curtain twitchers.  That past is not looking as dead anymore.

Are you feeling spied on yet?  If not, I’m sure you soon will be.

Death of a celebrity

This weekend the singer Stephen Gately died at his residence in Majorca.  At the time of writing, the cause of death is unknown but suicide,  foul play and drugs abuse are not being suggested.  I was provoked in to making this post by the reaction to the death that I noticed from various friends and acquaintances who took teh death quite hard but who also commented on the ‘gallows humour’ and apparent indifference of people to the fellow’s passing.

Mr Gately was clearly well loved by friends, family and fans.  I have to say that he meant little to me – a passing aquaintance with his name on the news – but unfortunately those who live as celebs must die as celebs, and part of that is the sick jokes marking their passing.  Since the widespread uptake of email, and especially since the web, this sort of humour has followed celebrity death as quickly and inexorably as paparazzi photographers and ambulance chasing lawyers.  Before electronic media, one at least had to wait for the jokes to appear in the newspapers / magazines or be passed from people who’d heard them from a friend who in turn heard them from a guy who knew the gardener of the dead celeb.

It’s rarely anything personal – it’s a coping mechanism, perhaps some of the milder jokes even provide the 21st Century version of marking the death of someone by printing the borders of the newspapers in black.  As some of you will know I was Admin on Sheffield Forum for a couple of years.  How to handle posted ‘dead person humour’ was an ongoing problem.  I used to apply the rule of 24 – within the first 24 hours it’s not nice – after that, it happens.  It may not be nice but it’s a byproduct of being in the celebrity food chain.  When you stop swimming in the media seas, your body sinks and the local bottom dwellers come and dismember the body, so to say….

One comment made stuck with me; imagine going to bed at 33 years old and not waking up.  When I was a kid I lost a friend who died at age 11.  As a younger man I lost a friend who died at 21.  Every morning in the developing world people in their 30s don’t wake up because they’ve died in the night of malnutrition, AIDS, Malaria, Cholera.  At the risk of sounding callous, I’m afraid that death is not the preserve of the poor, the sick, the elderley and the nobodies in the world.  It’s pretty Catholic in it’s tastes and can strike out at anyone – not just people who immediately surround us, and those of our modern pantheon of celebrities that our media choose to inform us are worthy of dying publically.  Don’t get me wrong; I’m not hypocritical enough to comment that I feel the death of total strangers in the developing world at all in my life – I don’t – but neither am I willing to go to serious grief over a celebrity who I didn’t know from Adam and who doesn’t even know I personally exist, except as part of a demographic.

I’m willing to admit to being sad at the deaths of three celebs in particular – John Peel, Joe Strummer and Johnny Cash.  I grew up with their music playing an important part of my life to varying degrees, so can empathise with people who’ve felt the loss of Mr Gatley as a figure in their musical upbringing – and especially those who’ve actually met the fellow.  Whilst we can all reflect on John Donne’s words about ‘ask not for whom the bell tolls, it tolls for you’  it’s worth also reflecting on whether your feelings are genuinely inspired by the death, or inspired by the media scrum surrounding the death suggesting how we should feel.

Meanwhile, back in June…..

Originally a Facebook Note, June 8th 2009, after the EU Elections….

The problem with democracy is that sometimes it allows people to vote for folks that you personally don’t want to gain any sort of power. Unfortunatley, that’s democracy for you. She can be a total bitch. FWIW, I voted for the Socialist Party inspired ‘No2EU – yes to democracy’.

There is an old latin saying – Ut sit magna, tamen certe lenta ira deorum est – the wrath of the gods may be great, but it is slow – that we can perhaps borrow and replace gods with people. To everyone saying how disgusted they are with their regions, their countrymen, stating people are idiots, etc. I ask them to think about the following.

From 2003 (beginning of the Iraq war) through to this weekend, the major parties in the UK have singularly managed to ignore or disregard the concerns and criticisms of voters. The Government has thundered on, ignoring calls for inquiries on numerous issues, ignoring the fears and concerns of voters on a number of issues, whilst keeping their “heads down in the pig bin, saying ‘Keep on digging'”, in the words of Pink Floyd. I’ve lost count of the number of people and groups who’ve asked the Government to reconsider their policies on things like immigration, ID cards and personal privacy, civil rights and freedom of expression in the UK. Phillip Pullman put it better than I can… http://www.joep.communityhost.org.uk/?p=71

We’ve recently had the financial debacle and then the site of MPs ignoring repeated requests for over 18 months to release details of their expenses.

And people wonder why voters voted the way they did? After the Government and major parties have acted with such hubris and contempt?

To be honest, we’re lucky this morning that we don’t have a handful of BNP and far-right groups holding seats all over Europe.

Voters used the only power left to them to get the attention of their leaders – the one thing that, as yet, New Labour haven’t removed from us. The power of the ballot. And when people used it they no doubt considered how they’d been ignored, taken for granted, treated as idiots and generally regarded as sheep who would quite happily vote for the famous ‘rosette on a dog’ representing the major parties.

Guess what – they didn’t. They said ‘Listen. We will not go this way again with you. You repeatedly ignore our concerns. You treat as as children and with contempt. Listen. We’re going to take the one course of action that will get your attention. We will vote for the parties that you and all those with a vested interest in the current system don’t want us to vote for’.

And that’s what they’ve done. Vox populi – the voise of the people. Keep on ignoring that voice – so apparently quiet in Westminster and Islington and the in inner circles of New Labour and the other major parties – and this will keep happening.

My final question – what are WE going to do about it? Many of you will know I’m a Libertarian – I believe in small government, and maximum involvement of the people in that governance. Wearing badges and shouting slogans and signing petitions is not enough. Wherever you live there are going to be issues in YOUR community that need tackling – social and political issues that left a mess for long enough will provide more grist to the mill of those on the extreme right and left who want to remove freedoms from us all.

Get out there and start fixing YOUR community and YOUR society. Listen to the people who’ve said ‘Enough’s enough.’ Work with them to address those issues that they’re concerned about and maybe, just maybe, we can collectively remind all politicians that they’re there because we give them the permission to be there.

If you need to blame anyone, need to feel ashamed or disgusted with anyone – just look to Westminster. Briefly – don’t dwell on it. Then look back to wherever you are and start fixing this mess.

Wolfram Alpha – too early released or over-hyped?

In case you’re saying, “Wolfram what?”, here’s a little reading:

http://www.wolframalpha.com/

http://www.bbc.co.uk/blogs/technology/2009/05/does_wolfram_work.html

http://news.bbc.co.uk/1/hi/technology/8052798.stm

http://www.guardian.co.uk/news/blog/2009/may/18/wolfram-review-test-google-search

http://www.theregister.co.uk/2009/05/19/dziuba_wolfram/

http://www.theregister.co.uk/2009/03/17/wolfram_alpha/

http://www.theregister.co.uk/2009/05/18/wolfram_alpha/

 

OK – I’ll start by announcing a vested interest here.  I occasionally write software that attempts to make sense out of straight English questions and phrases, and then by cunning trickery makes the response from the program appear ‘sensible’ as well.  So I know something about how to make software appear smarter than it actually is.  And I’m afraid that at first glance I regard Wolfram Alpha as over-hyped, under-delivering and pretty much unsure of it’s position in the world.

But, the folks at Wolfram Research score highly for getting the coverage they’ve managed!

WA is described as a Computational Knowledge Engine, rather than a search engine.  However, it’s raison d’etre is to answer questions, and nowadays any piece of software on the internet that does that is always going to be regarded by users as some sort of search engine, and the ‘Gold Standard’ against which all search engines tend to be judged is Google.  So, first question…

Is it fair to compare WA and Google?

Not really, and Wolfram himself acknowledges this.  WA is regarded by the company as a means of getting information out of teh raw data to be found on the Web, and it does this by having what’s called ‘curated’ data – that is, Wolfram’s team manage sources used for the data and also the rpesentation of the data.  This makes it very good at returning solid factual and mathematically oriented data in a human readable form. 

Whereas Google will return you a list of pages that may be useful, WA will return data structured in to a useful looking page of facts – no links, just the facts.  And a list of sources used to derive the infromation. The results displayed are said to be ‘computed’ by Wolfram Research, rather than just listed as is the case of a search engine.

Is it a dead end?

WA relies on curated data – that is, a massaging and manipulation process to get the existing web data in to a format that is searchable by the WA algorithms and that is then also presentable in a suitable fomat for review.  This is likely to be a relatively labour intensive process.  Let’s see why…

In a perfect world, all web data would be tagged with ‘semantic tagging’ – basically additional information that allows the meaning of a web page to be more explicitly obvious.  Google, for all it’s cleverness, doesn’t have any idea about the meaning of web page content – just how well or poorly it’s connected to other web pages and what words and phrases appear withjin the page.  They do apply a bit of ‘secret sauce’ to attempt to get teh results o your search closer to what you really want, assuming you want roughly the same as others who’ve searched the Google search space for the same thing.  Semantic tagging would allow a suitably written search engine to start building relationships between web pages based on real meaning.  Now, you might just see the start of a problem here…..

If a machine can’t derive meaning from a web page, then the Semantic tagging is going to have to be human driven.  So for such a tool to be useful we need to have some way of ensuring as much web data as possible would be tagged.  Or, start from tomorrow and say that every new page should be tagged, and write off the previous decade of web content.  You see the problem.

What the WA team have done is taken a set of data from the web, and massaged and standardised it in to a format that their software can handle, then front-ended this system with a piece of software that makes a good stab at natural langauge processing to get the meaning of your question out of your phrase.  For example, typing in ‘Compare the weather in the UK and USA’ might cause the system to assume that you want comparative weather statistics for those two countries.  (BTW – it doesn’t, more on this later)

The bottom line here is that the data set has had to be manually created – something that is clearly not posisble on a regular basis.  And a similar process would ahve to be carried out to get things semantically tagged.  And if we COULD come up with a piece of sofwtare that could do the semantic analysis of any piece of text on the web, then neither of tehse approaches would be needed anyway.

In a way, WA is a clever sleight of hand; but ultimately it’s a dead end that could potentially swallow up a lot of valuable effort.

Is it any good?

The million dollar question.  Back to my ‘Compare the weather in the UK and US’ question.  the reason I picked this was that WA is supposed to have a front end capable of some understanding of the question, and weather data is amongst the curated data set.  I got a Wolfram|Alpha isn’t sure what to do with your input. response.  So, I simplified and gave WA : “Compare rainfall london washington” – same response.  I then went to Google and entered the same search.  And at the bottom of Page 1 found a link : http://www.skyscrapercity.com/showthread.php?t=349393 that had the figures of interest.  Now, and before anyone starts on me, I appreciate that the data that would have been provided by WA would have been checked and so would be accurate.  But I deliberately put a question to WA that I expected it should be able to answer if it was living up to the hype.

I then gave WA ‘rainfall london’ as a search and got some general information (not a lot) about London.  Giving ‘rainfall london’ to Google and found links to little graphs coming out of my ears.  A similar search on rainfall washington to Google gave me similar links to data on Washington rainfall.

WA failed the test, I’m afraid. 

Will it get better?

The smartness of any search tool depends upon the data and the algorithms.  As we’re relying on curated data here, then improvements might come through modifications to data, but that might require considerable effort.  If the algorithms are ‘adaptive’ – i.e. they can learn whether answers they gave were good or bad – then there might be hope.  This would rely on a feedback mechanism from searchers to the sofwtare, basically saying ‘Yes’ or ‘No’.  If the algorithms have to be hand crafted – improvement is likely BUT there is the risk of over-fitting the algorithms to suit the questions that people have asked – not the general searching of what MAY be asked.

And time passes…

As it turned out, this post never moved from ‘Draft’ to ‘Published’ because of that thing called ‘Life’.  So, a month or two have passed, and I’ve decided to return to Wolfram Alpha and see what’s changed….

Given the current interest in the band Boyzone, I did a quick search.  WA pointed me to a Wiki entry – good – but nothing else.  Google pointed me to stacks of stuff.  Average rainfall in London got me some useful information about rainfall in the last week.  OK….back to one of my original questions ‘Compare rainfall London Washington’ – this time I got the London data with the Washington equivalent on it as well – sort of what I wanted.  Google was less helpful this time than back when I wrote this piece.

So…am I more impressed?  Maybe a little.  Do I feel it’s a dead end?  Probably, yes, except in very specific areas taht might already be served by things like Google and Wiki anyway.

Do I have an alternative solution for the problem?

If I did, do you think I’d blog it here and expose myself to all that criticism? 🙂

Are such things done on Albion’s shore?

I don’t often have much to say in favour of Phillip Pullman, but this is a fantastic article, IMO.  He encapsulates so much of how I feel about the gradual erosion of our rights in the UK.  Now…and disturbingly…this piece was ‘vanished’ from the Times’ website for a while – no explanation, no reason. It eventually re-appeared. Concerning, no?

I believe this to be one of the most important articles I’ve seen online for some time, and feel stronlgy that it should be widely read.  So, just in case it disappears again, I’ve taken the liberty of posting the full item as a Blog post here.  As an intresting aside, as I’m typing this I’m watching an episode of ‘The X Files’ in which Mulder says “…too many others know what’s happening out there. And no one, no government agency has jurisdiction over the truth. ”

PHILIP PULLMAN:

Are such things done on Albion’s shore?

The image of this nation that haunts me most powerfully is that of the sleeping giant Albion in William Blake’s prophetic books. Sleep, profound and inveterate slumber: that is the condition of Britain today.

We do not know what is happening to us. In the world outside, great events take place, great figures move and act, great matters unfold, and this nation of Albion murmurs and stirs while malevolent voices whisper in the darkness – the voices of the new laws that are silently strangling the old freedoms the nation still dreams it enjoys.

We are so fast asleep that we don’t know who we are any more. Are we English? Scottish? Welsh? British? More than one of them? One but not another? Are we a Christian nation – after all we have an Established Church – or are we something post-Christian? Are we a secular state? Are we a multifaith state? Are we anything we can all agree on and feel proud of?

The new laws whisper:

You don’t know who you are

You’re mistaken about yourself

We know better than you do what you consist of, what labels apply to you, which facts about you are important and which are worthless

We do not believe you can be trusted to know these things, so we shall know them for you

And if we take against you, we shall remove from your possession the only proof we shall allow to be recognised

The sleeping nation dreams it has the freedom to speak its mind. It fantasises about making tyrants cringe with the bluff bold vigour of its ancient right to express its opinions in the street. This is what the new laws say about that:

Expressing an opinion is a dangerous activity

Whatever your opinions are, we don’t want to hear them

So if you threaten us or our friends with your opinions we shall treat you like the rabble you are

And we do not want to hear you arguing about it

So hold your tongue and forget about protesting

What we want from you is acquiescence

The nation dreams it is a democratic state where the laws were made by freely elected representatives who were answerable to the people. It used to be such a nation once, it dreams, so it must be that nation still. It is a sweet dream.

You are not to be trusted with laws

So we shall put ourselves out of your reach

We shall put ourselves beyond your amendment or abolition

You do not need to argue about any changes we make, or to debate them, or to send your representatives to vote against them

You do not need to hold us to account

You think you will get what you want from an inquiry?

Who do you think you are?

What sort of fools do you think we are?
The nation’s dreams are troubled, sometimes; dim rumours reach our sleeping ears, rumours that all is not well in the administration of justice; but an ancient spell murmurs through our somnolence, and we remember that the courts are bound to seek the truth, the whole truth, and nothing but the truth, and we turn over and sleep soundly again.

And the new laws whisper:

We do not want to hear you talking about truth

Truth is a friend of yours, not a friend of ours

We have a better friend called hearsay, who is a witness we can always rely on

We do not want to hear you talking about innocence

Innocent means guilty of things not yet done

We do not want to hear you talking about the right to silence

You need to be told what silence means: it means guilt

We do not want to hear you talking about justice

Justice is whatever we want to do to you

And nothing else

Are we conscious of being watched, as we sleep? Are we aware of an ever-open eye at the corner of every street, of a watching presence in the very keyboards we type our messages on? The new laws don’t mind if we are. They don’t think we care about it.

We want to watch you day and night

We think you are abject enough to feel safe when we watch you

We can see you have lost all sense of what is proper to a free people

We can see you have abandoned modesty

Some of our friends have seen to that

They have arranged for you to find modesty contemptible

In a thousand ways they have led you to think that whoever does not want to be watched must have something shameful to hide

We want you to feel that solitude is frightening and unnatural

We want you to feel that being watched is the natural state of things

One of the pleasant fantasies that consoles us in our sleep is that we are a sovereign nation, and safe within our borders. This is what the new laws say about that:

We know who our friends are

And when our friends want to have words with one of you

We shall make it easy for them to take you away to a country where you will learn that you have more fingernails than you need

It will be no use bleating that you know of no offence you have committed under British law

It is for us to know what your offence is

Angering our friends is an offence

It is inconceivable to me that a waking nation in the full consciousness of its freedom would have allowed its government to pass such laws as the Protection from Harassment Act (1997), the Crime and Disorder Act (1998), the Regulation of Investigatory Powers Act (2000), the Terrorism Act (2000), the Criminal Justice and Police Act (2001), the Anti-Terrorism, Crime and Security Act (2001), the Regulation of Investigatory Powers Extension Act (2002), the Criminal Justice Act (2003), the Extradition Act (2003), the Anti-Social Behaviour Act (2003), the Domestic Violence, Crime and Victims Act (2004), the Civil Contingencies Act (2004), the Prevention of Terrorism Act (2005), the Inquiries Act (2005), the Serious Organised Crime and Police Act (2005), not to mention a host of pending legislation such as the Identity Cards Bill, the Coroners and Justice Bill, and the Legislative and Regulatory Reform Bill.

Inconceivable.

And those laws say:

Sleep, you stinking cowards

Sweating as you dream of rights and freedoms

Freedom is too hard for you

We shall decide what freedom is

Sleep, you vermin

Sleep, you scum.

Philip Pullman will deliver a keynote speech at the Convention on Modern Liberty at the Institute of Education in London tomorrow

http://www.modernliberty.net

Am I a twit not to Twitter?

OK….I remember a year or so ago saying i’d never join Facebook, and then making myself look a pudding within a month or so when i started using Facebook to keep me in touch with friends after I stopped using another online service.

Now, around the same time I also made a brief investigation of the Twitter service – some more information here.  Whilst I can’t argue that it’s popular, and has attracted a vast amount of traffic and interest, including being used in the Australian bushfires and the Mumbai terrorist attacks, I’m still yet to be convinced of the value of telling the world precisely what I’m doing in 140 byte chunks.

Let’s face it, I’m too busy / idle to maintain my Facebook status more than once a day on average, so the idea of me managing to ‘tweet’ happily several times a day on the Twitter system is probably minimal.  And I’m not convinced of the overall value of most of the content that seems to be generated on Twitter; allow me to explain.

Too short!

To begin with, 140 characters is shorter than an SMS message, and unless you’re skilled at putting highly informative short messages together, the informational content of such messages is limited purely by the size of the message, unless you send a string of such messages.

Too distracting!

We then move on to whether Tweeting encourages the attention span of a boiled potatoe; it’s a disruptive technology in all the wrng ways – it simply disrupts your attention by a string of pointless inanities appearing in your Phone, Twitter client or web browser.

What does it do that other media doesn’t?

In terms of brevity you have SMS messages or Facebook statuses.  In terms of information content you have Email, blogs or Forum posts.  Tweets are ephemeral – they’re not naturally persistent and are as short lived as real birdsong.

So, what the Hell is it all about?  I’m aware of the use of this sort of technology in crisis situations but is this genuinely making appropriate use of the available technology?  I’m yet to be convinced that Twitter is anything but another toy for the technorati, and one whose lifespan in it’s current form is probably going to be limited by the emerging financial realism in the world.  I’ve heard of alternative uses – people using hardware to automatically place Twitter messages in to the ‘twittersphere’ form such things as potted plants and the old standby of IT departments, the drinks machine.  These messages are then picked up by a piece of software listening on Twitter for ‘tweets’ from the appropriate account.  This is nothing different to using UDP packets, for example, but at least there’s a more easily accessible interface here.

But I’m not convinced – someone, anyone, convince me of the value of this application, PLEASE!

You pays peanuts…..

And you get monkeys.

I assume most of us have heard this phrase. It’s become almost a mantra with me in my professional life because the last 6 months have exposed me to an interesting aspect of the freelance world that I’ve not been aware of until now; the fact that there are a Hell of a lot of people out there expecting a lot of work for next to nothing!

Allow me to elaborate…I get most of my work through ‘word of mouth’ – this has always been the way and after 20 odd years in IT it seems to have worked well. But I still like to chase the odd new client – after all, nothing wilts faster than laurels that have been sat on, as they say. In many ways, the availability of Internet web sites that allow people wishing work to be done to advertise their requireents for people like me to pick up the jobs should have ade things easier, but it hasn’t.

In fact, I’m beginning to regard such sites as one of the worst things that has happened to ‘professional’ freelancers and contractors, because they have totally distorted the market. Don’t get me wrong; I’m a firm believer in market forces but these sites are actually pushing the markets for freelance development work to the brink of extinction. And this isn’t going to be a rant about out-sourcing…

My concern is that people are posting requests for work like the following:

“Develop a highly interactive and very aesthetic media review website. A good example is Yahoo! TV. The site is going to cater for commercial considerations i.e web ads. Want a site that would load fast as well.
Hence, beautiful but efficient. Must do the job. “

This is a real advert, tweaked for punctuation and spelling in two places.  Now – this isn’t a hobby site, it’s not a charity.  The poster is open in that there will be advertising and will be catering for ‘commercial considerations’.  That’s the full ‘job brief’ against which people are expected to bid, by the way.  Now, let’s assume that we can put something together like the Yahoo TV site – here and ignore the content and imagery side of things for now.  It’s got forums, photo galleries, all sorts of cute stuff.  I wouldn’t even want to try tackling it – a wise man knows his limitations, after all.  But I can guess the sort of development time – you’re looking at the minimum of 2-3 man-months here, I’d estimate.  

And the suggested budget?  £250.  Yes, Two Hundred and Fifty Pounds.  No missing zeroes.

I cannot imagine the most desperate out sourcer being willing to work for that sort of money, let alone a programmer in the UK, US or Europe.

Oddly enough I came across this today:

http://technology.timesonline.co.uk/tol/news/tech_and_web/the_web/article5483244.ece?token=null&offset=0&page=1

An article in the Times dealing with Amazon’s Turk’ project which harnesses the available time of people to do online jobs of various sorts.  Where you might be expected to work for a couple of pence an hour, if that.

Digital exploitation?  You betcha.  There are projects that rely on the good nature of people to get things done – projects where the bottom line is a better, publically and freely available service, rather than profits to corporations who can already dictate terms to much of the online world.

Some years ago I was involved in film making and there was a very rich culture of ‘No-budget’ filming, where productions were put together with no budget except for the essentials of film stock or tape – everything else was borrowed, begged or blagged.  But part of the contract was that anyone involved would get a copy of the material for their own portfolio and an on-screen credit – ‘Credit and VHS’ – as well as being fed and watered on set.  This model could, of course, be exploited but rarely was, because the world of film making was relatively insular and someone pulling a fast one would immediately find it difficult to crew-up next time around.

Perhaps we need to start being similarly watchful in the information marketplace?

 

Everybody Hurts

For various reasons, I started really thinking about the REM song ‘Everybody Hurts’ today.  Just in case you’re not au fait with it – here are the lyrics…

When the day is long and the night, the night is yours alone,
When you’re sure you’ve had enough of this life, well hang on
Don’t let yourself go, ’cause everybody cries and everybody hurts sometimes

Sometimes everything is wrong. Now it’s time to sing along
When your day is night alone, (hold on, hold on)
If you feel like letting go, (hold on)
When you think you’ve had too much of this life, well hang on

‘Cause everybody hurts. Take comfort in your friends
Everybody hurts. Don’t throw your hand. Oh, no. Don’t throw your hand
If you feel like you’re alone, no, no, no, you are not alone

If you’re on your own in this life, the days and nights are long,
When you think you’ve had too much of this life to hang on

Well, everybody hurts sometimes,
Everybody cries. And everybody hurts sometimes
And everybody hurts sometimes. So, hold on, hold on
Hold on, hold on, hold on, hold on, hold on, hold on
Everybody hurts. You are not alone

“Everybody hurts.  You are not alone.”  So very true, and also so  difficult to remember when you are in any sort of pain.

Continue reading