Cloud vs Internal storage

I have written a few times about the lack of trust that corporate IS groups have for data stored in the cloud.  Much of this is based around FUD (fear uncertainty and doubt) injected through the media, but also through a lack of trust of anything that cannot be directly controlled.  Whilst corporate IT often feels it does a better job that a nameless cloud provider is this really the truth?

For example I am often told how unsafe data is when stored within the Google Apps services, that Google indexes your data and use that company confidential data for nefarious reasons etc.  On the other hand I have sat in Googles offices talking to Google themselves and they tell a very different story about their data security processes.  The differences are profound and I  been wondering for some time whether the FUD is actually a tactic by other vendors aimed at ensuring that they have time to prepare their own cloud services.  Google must have encountered this concern a number of times now and have produced the video below to give an insight into how their data centres are managed.

Now it is worth noting that this video is obviously a propaganda video aimed at dispelling something disadvantageous to Google however it does show a well organised and thought through approach to data in their care.  I know some will doubt the truth of this however I have no reason to doubt that the video is not a demonstration of the processes that they have in place.  Now lets compare that to corporate IT, many of the best corporate IT groups use the ITIL service management system to ensure a consistent approach to the IT services they provide.  Part of this approach advocates something called a DHS (Definitive hardware store) where all equipment not currently in use is stored so that it is ready for re-use.  Recently I had the chance to take a tour of a DHS and saw stacks of used servers; storage arrays and individual hard drives just sitting within the store waiting for possible re-use.   The building containing the DHS itself stands within a business park that has a small amount of security present, and both it and the DHS have locks but this level of security is not even close to that demonstrated on the video above.  I am pretty sure that the disks in this store are not tracked individually through the CMDB so their last use is unknown and thus the contents of the disks could be anything.  Not only that the data on those devices is in an unencrypted form so this could be accessed if the device fell into the wrong hands.  Even disposal of the disks must be handled elsewhere, usually by the vendor that offers the company the lowest price.  At this point I want to stress that his is in a corporate that handles IT pretty well, some older style organisations will most likely not be anywhere near this organised. So my question is where is the basis for the assertion that corporate IT looks after the data better than the cloud vendor?  Not only that the corporate IT department presents itself to the business departments in a manner not unlike the cloud vendor i.e. they are dealing with a remote centralised organisation that claims to look after its data.  For them would using a cloud service directly be any different to using corporate IT storage?   I suspect only the size of files being worked on and the cost of comms links is holding them back....for now.

The forecast is briefly clear followed by heavy cloud

The weather in England has been unusually sunny for April, with not a cloud in the sky but it was not the only place lacking clouds last week as for 36 hours the cloud services giant Amazon disappeared from the web ( see here ).

I know what will now happen as I've seen it before with managers that found access to Google mail was offline for a few hours a couple of years ago.  They will state in their old school IT manager way that this just proves the cloud to be unreliable "we are safer to have the equipment in our own server room because then we have control".

I say they are wrong, I've worked in IT for a long time and in that time seen a lot of major outages and they almost always take more that a couple of hours to fix often multiples of days.  This asscertion is particularly true when it is loss of a service such as GMail, I can guarantee that corporate email services are unavailable more often that in most companies.   Corporates often have to do work to particular sites or servers that require this equipment to be taken offline, but even if the servers did run continuously there will be people that loose connectivity to the servers and this will be perceived as an email outage.  Not only that can people really not live without email for a couple of hours?  Personally I did not see the GMail outage at all probably because I was in a meeting or something for its whole duration.

The Amazon outage is more serious, many businesses have used these facilities to produce online ordering sites vital to their business and there is no doubt shutting up shop for 36 hours will have hurt them.  At this point though we must consider the costs of running these sites the old fashion way.  We not only need to put up enough servers to run the maximum load, we must duplicate this to at least one other site possibly even across regions.  This will require investment in infrastructure and communications not to mention a tribble load of techies in each location to keep it all running sweetly.  None of this will guarantee correct set-up and the avoidance of an outage at some future point, all it will do is ensure you are in control of that outage and entirely culpable.

Compare this to what has actually happened with the Amazon outage.  Users are pretty tolerant to web site failures and most will simply return the following day to place their order.  This is especially true when it wasn't the fault of the vendor to whom they are loyal but of some corporation on which they rely.  Indeed its now less than a week since the issue and yet there are no stories of this in the IT press, it has been forgotten and consigned to the past.  The biggest looser in all of this will have been Amazon who will have not only lost sales but also will probably have to compensate against SLA's for some of those sites, a good incentive to track down the issue and make sure it never happens again.

I don't believe any of those companies will be planning to implement their own internal architecture to ensure that they can avoid this rare occurrence, though some of the more affluent ones may consider implementing their sites using multiple cloud services.

Remember errors are not completely undesirable they are necessary to progress and as long as they are not too major or too frequent they make us stronger.  If they become either of these for Amazon people will move to another service and Amazon know it. Not only that all of their competitors know it and will be making sure they to learn from it and through the one incident the whole cloud market becomes stronger.

Computer viruses and you

Computer viruses are a problem for everyone, even if you do not realise it yet!  For the pedantic I am lumping up all forms of Malware into that as most non geek people do not distinguish.  Make no mistake nowadays these things are out for your credit card number, if you have a disk failure or virus outbreak that suggests that you need to “buy the pro version” to fix or eradicate it then you are almost certainly interacting with some malware.  Give them your credit card number at your peril.

It occurred to me that people try to think about these things like the biological equivalent but I think that bacteria may be a better model.   For example if someone handed you a lovely piece of chocolate you would look at its brown loveliness and eat it, gaining the bit of happiness you expected.  What would you do if someone handed you that bit of chocolate with the same promise of happiness but it was covered in a blue mold?  Would you eat anyway in the hope of a bit of happiness?  Would you be more or less likely to eat this if it was a good friend that gave it to you?  What about if it was in a box with your good friends name on it?  Would you ignore the blue and eat?

I hope you see what I am getting at here, if you replace the chocolate with a file representing “this will make you laugh” you start to realise the problem, you cannot see the computer virus but that file may well in reality be covered in mold.  The escalating situations with the moldy chocolate represent the escalating suspicion with which you would treat it. if I had used cheese instead of chocolate for my example and a friend offered it directly you would eat it, though probably hesitantly, because you trust your friend but you’d be unlikely to do so if it was offered in a dark alley by a stranger.

The avoidance of bad food is something we are all trained into from a really young age and we tend to stick with stuff we know and not experiment on our own.  This keeps us safe in general, although not without the occasional jippy tummy.  IT professionals and IT aware people are also pretty good at avoiding malware in much the same way just because they recognise more clearly the circumstances in which it may occur and avoid them.  Those that do not recognise it but flag unusual behaviour to someone that may know more can also avoid malware more often than not.   Even these people though will end up with the malware equivalent of a jippy tummy now and again (ref: my sons currently infected profile on his computer that I keep caged to study ;-)  )

And like bacteria there are lots and lots of different kinds of malware and growing numbers of writers of this type of software.  You cannot assume that Anti-virus vendors are capable of holding back the tide, defend yourself and your company by assuming that everything is out to get you until proven otherwise.

Trial and Error

As well as selection of over complicated technology products there are other factors that may well contribute to the slow deployment of technology. As we have aspired to IT professionalism we have also potentially lost some of the factors that made us agile.

We have decided that we should get everything perfect first time, after all we specialise in this stuff don’t we? This is reinforced by the way we are treated by our businesses if even the smallest thing goes wrong. As a breed nothing is every good enough for an engineer in whatever discipline they work. We can always improve something and will do so if we are allowed to, even if that means starting again from scratch and missing deadlines. Add to this the threat of bite back should anything fail and projects take too long as we try harder and harder to make them perfect.

The evolutionary process that has lead to our own existence on this planet and then to all of our achievements makes use of the trial and error process. You try a few things discard the failures then keep the successes and try a few more things based on them. To improve our delivery timescale we need to be able to work in ways that allow application of trial and error.

In software development we have methodologies such as agile that allow us to use this process quite effectively. I have seen a number of significant successful software development projects use this to great effect. In infrastructure though this does not seem to be the case as large investments in equipment that don’t do what was expected is never going to be popular. Once again though there is hope on the horizon in the form of Cloud compute.

Spinning up a computer in the cloud to try out your new service costs you only the computing power that you use. If the service does not work then just turn it off again and stop paying, suddenly you are free to take risks. This potentially allows you to become much more agile in your approach to the creation of infrastructure. Not only that but successful trials can be scaled up within the same service into a live system, or depending on the services you are creating can be moved onto on premise equipment.

Of course software is a factor in here that currently confuses the picture. Whatever software you are putting on the Cloud compute infrastructure will need to be licensed correctly. This may well be a driver that helps certain open source products to proliferate as their licensing may better suit this agile environment, until the large software vendors work that out at least.

What do Cloud Computing and Farming have in common?

Many people cannot see a connection between these two things as they would seem to be a world apart but actually the connection is not in the work being done but in what it enables.

Farming is not always given the credit it deserves for what it gives us but think about what it would mean if it did not exist.  How many acres of land and head of livestock would you have to manage to feed your family for a year?  You may feel the once a week trip to Tesco's is arduous but without farmers doing this for you you would spend the majority of your life just arranging for the food you need to survive.  This is especially true if you assume that you have no access to other machinery, such as the tractor in the picture, to help you achieve your goals.

Why would this machinery be missing?  Well if you are constantly working in the fields you have not got time to invent and build a tractor.  Build from that and you realise that you have not got time to even sit and read a book and learn something, all you know is toiling in the fields.

Farmers doing this for us frees us to do all the other things we do, and some of those things help the farmers to get a bit of free time too.  None of this would work however if we did not trust that the farmers would deliver the food to Tesco's so that we could trade tokens for it despite having never set foot in a field.

So where is the connection between this and Cloud Computing?  The connection is the trust, we do not trust the cloud computing vendors to keep on doing this for us therefore we keep on doing it ourselves.  Keeping the systems working then takes up all our time and we have little time for working out how to do new things to improve the business.

I often talk to people about "the cloud" and it's clear there is a lot of confusion about this, what it is and what it can do for your company.  Much of this is led by the fear of your company data in a place that you don't own and control.  I must stress here that I am talking about true cloud services and not old style computing equipment renamed as "private cloud".

10 years ago I worked on a project to consolidate a computing environment, at the time people had departmental file servers and found this move to location based servers to be a loss of control.  Then some years later as computing power improved and in particular network reliability increased we moved to a truly centralised store, again this was met with some fear and suspicion about lack of control. Moving computing to the cloud is only a matter of taking the final logical step, yet this for many companies proves hard to achieve.

I believe that this step will not be willingly taken and that market forces will have to act to make larger companies utilise the cloud.  This has begun on a cost saving basis but even so it is not yet a big enough driver for larger businesses to leave the safety of familiar IS provision.  My feeling is that smaller companies that have currently started with most of their services provided by the cloud will begin to grow, putting them in a position to compete with the larger established companies.  They will however be operating with a much lower cost base courtesy of cloud computing. Eventually the only way to compete will be to adopt these same cloud systems.
Of course there will be other drivers, as companies begin to deliver necessary software using only a cloud model (so they can control access to their technology via subscription) this will force adoption of these systems.

So why haven't larger companies implemented cloud services?  Here are some suggestions

• The fear of lack of control of their data, or some form of data leakage.
• Internal subversive IS pressure to ignore this in order to preserve careers, normally by contributing to the fear.
• Concerns that all your eggs in one basket makes you vulnerable.
• IS Management has evolved from support in the 1990's to their current role and are not willing to consider the change to the model.
• Strategically waiting to see which cloud provider wins out.

I'm sure you can think of others but the thing to consider is whether or not the adoption of cloud now will give you an advantage in the future.  I personally believe it will and I further believe that non adoption could put the company at a significant disadvantage.