*Update – NY Judge has delayed hearings in the Google Book Deal – read all about it*

Much of humanity’s knowledge is in the process of being digitized, and a new set of rules are currently in draft-mode concerning how this wealth of knowledge may (or may not) be made available to the public.
If you’ve ever read a book, or wanted to, if you’ve ever searched Google, or dreamed of it, you might want to catch up on the latest developments in our civilization’s fight for access to information. More recently, as physical copies of literature have become less relevant to the public, being able to effectively access their digital counterparts becomes increasingly important. This is an opinion piece, and all of ‘em are mine, the author, so don’t go blaming anybody else if your sensibilities are touched. With that disclaimer made, on with it!

Walking down the street the other day a friend of mine turns to me and asks: Hey, so you work with Google or whatever, what the hell is the big deal? Why am I constantly hearing about this ridiculous company with a childish logo, I mean, it’s just a freaking search box isn’t it?

My response was to pause for a moment in order to consider how I could express to this pre-Internet generation chum what the hell the big deal was. Eventually I responded: Well, I think that search box will come to represent, in a very real way, either human freedom, or a lack thereof.

My friend looked at me, head tilted to one side, incredulity spurting out of the corners of his squinted eyes, and said: never mind. In a manner that clearly portrayed the underlying sentiment: Naoise, you’re an idiot who never learned to answer simple questions with anything but pedantic, complex answers. And while that may be so, in my defense let me just say: c’mooooon, strue! Freedom of search really is related to the larger concept of the freedom of civilization.

There is a real technological divide that meters how freedom of search is doled out to the world’s citizens, and we, as a generation, may be leaving the future of this freedom in the hands of a small group of private, for-profit companies: Google, Microsoft, and quite possibly, Facebook.

Google’s search box, simply because Google themselves seem to be the only ones taking up the reigns and ‘organizing the world’s information’, is quickly becoming our gateway to information and new knowledge. Microsoft wants to compete on the same grounds with a universal search engine, and Facebook wants to redefine how search is perceived, utilizing personal social graph information in order to make that happen.

The Digital Divide, Search & Creating

The most basic staple of modernity, electricity, is an infrastructure issue where the poor are the last to receive access. One of the most basic staples of freedom, access to information, is fast becoming a technology issue, where once again your class dictates both your physical ability to access the information, and often the quality of that access. What it doesn’t limit, for a large portion of what we consider the ‘free world’, is the pool of information which you are allowed to access, that slushy digital sea of knowledge that sits behind the protective, omnipotent, yet wholly opaque search box.

The socioeconomic access-to-information differences don’t only manifest in the extremes – one finds it easy to imagine the poor of India having trouble accessing the Internet – but also in modern western splits. In Canada, 91% of people earning more than $90k per year access the Internet regularly, while only 47% of people earning less than $24k per year (a little above the Canadian poverty line) do the same (source: Statistics Canada). That’s a significant difference in a fully modern first world country. Even in the US, low-income American’s broadband connections decreased 3% from 2007 to 2008, which is hardly a sign of convergence in the digital divide.

A survey taken in May 2008 (warning: PDF file) revealed that 49% of Internet users use a search engine ‘on a typical day’. This is an increase of 69% from January 2002, when the data was first collected. By contrast, the percentage of people who use e-mail ‘on a typical day’ grew only 15%, from 52% to 60% over the same time-frame. Search is used on a daily basis almost as commonly as e-mail, and is growing at a much, much faster rate. We’re addicted to search, and that addiction has yet to even approach maturity.

These increases reflect greater general access to search, greater awareness of search in demographics that already had access, and a habituation of searching amongst those who already have access and awareness. In all cases, search is working its way deeper and deeper into people’s lives, but again, not in an equitable way.

Almost 50% of the American population’s Internet users search on ‘a typical day’, but that number is 66% for college graduates, and only 32% for those who never attended college. The number is 62% for people who earn more than 75K per year, but only 36% for those who earn less than 30K. Part of this is the quality and immediacy of access, which shows people with broadband at home to be significantly more likely to have ever tried using search engines at all, when compared with dial-up users.

The younger generation, of course, is also more habituated to integrate searching online into their daily lives, with 55% of those 18-29 years old searching on a typical day, but only 27% of those 65 or older doing the same. The children are our future, and our future looks pretty saturated in search queries.

So where does this leave us? The concept of search in 2009 is different from our old concepts of information retrieval, and I fear we are moving dangerously close to a common acceptance of the search box as being not only the primary, but the best method to discover knowledge.

I believe it is inevitable that a larger and larger portion of the population will not only integrate search engine use into their daily lives, but invest in search mentally, as a means to knowledge discovery of all types. Knowledge discovery, via access to information, is of course the basis upon which new knowledge is created, and fundamental to the advancement of a civilization. The personal freedom to create is in my opinion (which, hey! I created) an essential aspect of freedom, and is fundamental to the freedom of civilization as a whole to progress in anything that could be considered a natural way.

Within this quest for knowledge lies the almost inconceivable potential of the Internet as we know it, to provide. But it is far from guaranteed that the information the populace at large is given access to will be a full index, free of an intermediary opinion based layer of adulteration (and I do not mean in a ranking sense, but in an inclusion in the index sense). If the index is maintained by a for-profit company, is it unrealistically cynical to think that they will consider their index their property, and appropriate it according to their will, now or in the future? If you managed to come to the conclusion that no, it is not unrealistic, ask yourself this: is it naive to trust the future will of any for-profit company on the basis of their current ideology?

How to Save a Civilization: The Public Library, Unabridged

The idea of harboring or safeguarding the canon of a civilization’s knowledge being essential to the maintenance and cohesion of that civilization is certainly nothing new, and what naturally follows is the necessary freedom to access that information being granted to the populace. Governments have, for many generations, taken responsibility for their citizen’s access to information, if only to guarantee an educated and productive public. For the same reasons inverted, knowledge has long been suppressed by governments, ruling groups and private groups alike in an effort, beyond the obvious economic gains available to those who control production methods, to keep the public passive and unquestioning.

In modern western culture circulating libraries receive public funds to catalogue and make accessible as much common and accepted knowledge as is possible. When new entries to the greater canon of human knowledge grew beyond books and evolved to the point where the ideas only ever knew a digital life, libraries adopted and helped to evolve the early Internet. One of my first real jobs, back in 1997, was for a government program designed to teach people how to use the Internet and how to use search engines effectively, via free access at the public library.

If the world evolves to the point where physical access may eventually fade in importance as ubiquitous access to the Internet becomes the standard, the issue then becomes the nature of the body of information that we do have access to. While it may remain difficult to control access to websites if one knows the URL, it needs to be understood that, essentially, the only form of online information discovery outside of visiting sites you already know (which then link directly to new knowledge), is via search engines.

And what happens when the all-powerful little search box becomes the gateway to more than just the Internet as we currently understand it? When the true canon of human knowledge migrates to ‘The Cloud’, and books are no longer printed and sent to the public library for anyone to access, that little search box takes on a whole new level of importance.

Concentration and Ownership of Knowledge

Google has a stated goal to “organize the world’s information and make it universally accessible and useful”, which taken optimistically, seems almost noble – an image Google has always tried to create for themselves (do no evil = be noble). What is at issue is whether or not “universally accessible” means equitable and identical access without regard for any characteristics of the searcher, be it nationality, location or socioeconomic status. Nobility may be a relative thing.

It is quite obvious that Google does not, in fact, mean this, as they have already capitulated to the Chinese government and censored their search results for Chinese citizens. There is no hope that Google has any intention of being noble in achieving its mission; they have already failed in the name of profit. In a much larger sense, if their mission is to make the world’s information accessible, in suppressing it, they have failed civilization.

While the suppression of human freedom for profit is a tough one to justify, Google has produced rhetoric to do just that. Regardless of their optimism in this being a short-term solution with a goal of greater good in the long run, the suppression of information concerning human suffering only helps to extend and proliferate that suffering, now, in the present – if things change in the future Google will at least have profited in the Interim.

Google are not alone in their transgressions, and in fact are much more up-front about their opinion on the matter, and much more transparent to users about changes to their index that affect search results than their major rivals. Microsoft has also come under fire for similar issues, but in typical Microsoft fashion, they’re just too big and monolithic to really care about the world around them. Recently (june of 2009), Microsoft altered search results clear across the one billion person country of India, to exclude by default sexually explicit content. This is not the same as not displaying sexually explicit content to those who do not request it, but actively blocking it from those who have in fact requested it.

This is the modern world’s version of book burnings and bannings – it takes the form of incomplete, abridged, or adulterated indexes of data behind a corporate search box.

A Brave New Digital World: The Google Books Debacle

Earlier I speculated as to what may happen if the true canon of human knowledge – at the moment more confined to paper and ink than bits and bytes – were to migrate to ‘The Cloud’. If the current store of humanity’s books does go digital it may simply become another catalogue of bytes, which will almost inevitably be controlled by an entity, if not outright owned by it. That human civilization’s knowledge would then also become subject to annihilation by something as ridiculous as the EMP effect of some future solar storm is disconcerting enough. Even if that may be a little far-fetched, it’s certainly not far-fetched to ask the question: if the country’s electricity grid fails, or if brown-outs and blackouts significantly affect our lives, do we want our history, along with news and the ability to search our current world to become something we have technical problems accessing? Or to become something we need to appropriate access to physically, once again with all of the classic problems it presents for equitability?

Beyond the big picture problems of appropriating access-to-(way more)-information to a private company, the Google book deal is riddled with problems. A couple of years ago Google started to develop the concept of digitizing books which have fallen out of copyright protection. This, once again, taken optimistically seems quite noble. But the truth of the matter is that Google has tried to jump the gun. The Authors Guild and the Association of American Publishers both sued Google back in 2005, claiming copyright infringement for their initial digitization and book search efforts. In disentangling itself from these issues Google actually converted the former adversaries to allies, and announced a deal. A new deal. A stunner.

Not only will Google digitize out of copyright books, but also out of print books that are still under copyright – even when those holding the rights to those books don’t specifically agree to Google’s plan. Keep in mind that lots of books go ‘out of print’ just a few years after they’re first published. The reason Google has carte blanche to sell these books is because the Authors guild filed a class-action lawsuit, which would require you as an author to actively opt-out of the settlement if you didn’t like it. By default, if you do nothing, Google can sell your books. Entire sovereign nations have complained, with Germany and France claiming broad copyright issues. So what happens? Google will likely be forced to hide these books from European searchers, once again closing the canon of human knowledge selectively, based on who you are.

An organization called the Open Book Alliance (OBA), along with numerous other entities such as the Electronic Frontier Foundation (EFF), have taken up the fight against Google. Here’s what the OBA had to say about Google’s attempts:

“The members of the Open Book Alliance recognize the tremendous value that the mass digitization of books can bring to consumers, libraries, scholars and students. Making books searchable, readable and downloadable promises to unlock huge amounts of our collective cultural knowledge for a broader audience than was ever possible.

But, as we’ve noted, this settlement is the wrong way to go about making this promise a reality. The current settlement proposal would stifle innovation and competition in favor of a monopoly over the access, distribution, and pricing of the largest collection of digital books in the world, and would reinforce an already dominant position in search…”

Google has paid more than lip service to the concept of accessibility, in that they plan to provide terminals to all public and university libraries in order to access the Google Book repository free of charge, but as The New Yorker points out:

“The out-of-print books Google has digitized come from nonprofit institutions that built their collections as a public good. In return for pocket change—Google will contribute $125 million to create a nonprofit rights registry—these public treasures will now be monetized for the benefit of a private corporation.

True, Google will give every public and university library one terminal where readers can access its entire collection. But these machines won’t be able to download or print texts—and you can imagine the lines. Libraries that want full access to all the books in Google will have to pay for the privilege, as well as for every download.”

While there is plenty to think about in terms of access to information for the public at large, the Electronic Frontier Foundation (EFF) is also concerned about your rights as a consumer – in this case, a reader. In the initially proposed Google Book deal, there were absolutely no guarantees that your personal privacy regarding what you read and purchase would be in any way secured. Here’s a quote from a group led by the EFF:

“Google Book Search and other digital book projects will redefine the way people read and research,” said Lethem, winner of a National Book Critics Circle Award. “Now is the moment to make sure that Google Book Search is as private as the world of physical books. If future readers know that they are leaving a digital trail for others to follow, they may shy away from important intellectual journeys.”

While for you and me, today, this may not be so much of a concern, put yourself in the position of an oppressed Chinese or poor Indian person who may very well shy away from searching and reading specific things for fear of being personally identified in the act, and persecuted for their thoughts and beliefs. This already happens, but it’s only going to get worse if what people are denied access to includes all of the books in the world, not just all of the blogs.

There is no point in ignoring the fact that humans will eventually, inevitably, digitize all past and current knowledge. The problem is it’s happening too fast, while the technology is still misunderstood by our governments, the very entities who should be helping ensure equitable access to information, while the resources to digitize are only truly in the hands of the private corporations of the world, and while the laws of copyright have not yet caught up to the digital age, allowing those corporations to take advantage of, potentially control, potentially privatize, and most certainly profit from, something so sacred as the canon of knowledge of human civilization.

It’s a big deal.