By Phil Bridge, Managing Director at Kroll Ontrack Data Recovery

The impact of the Snowden scandal has only added to the need for the EU to increase the minimum security expectations for companies’ networks and personal data protection. The big problem is finding a way to manage information in a responsible and effective way. From high-level government information to the credit card details of an Amazon shopper, the digital network is crammed full of confidential data, and its volume and size is growing rapidly each day.

Without question, we’ve entered an epoch defined by Big Data (the term is used to sum up very large, complex, rapidly-changing datasets) and there’s no sign of a slowdown. So much Big Data is produced every second that it’s now tricky to store, manage and harness it for commercial purposes – and it’s not just the size that’s a problem but the type of data that’s being generated.

The challenge of unstructured data

In the past, most traditional data was structured, or stored neatly in databases. This was possible because there wasn’t a worldwide, interconnected network, and information was stored physically in filing cabinets or digitally on computer discs. When the digital age arrived, that arrangement disappeared and an explosion of unstructured data was produced as a result of growing digital interactions.

In addition, the world has seen a proliferation of gadgets, from smartphones to iPads to voice-activated televisions and fridges that can all record and transmit data. Industrial sensors and CCTV cameras also help to produce data so large and complex that a new approach must be taken to store, secure, and – in the case of individual rights – erase the data when a person wants to eliminate it.

How much data is out there?

Nobody can provide an exact figure on the current quantity of global data but some research claims that 90% of all the data in the world today has been created in only the past two years.

Other experts suggest a figure that adds up to billions of information units each day.  According to IBM, over 2.5 billion gigabytes (GB) of data were generated every day in 2012 and the number is even greater now with the addition of more mobile and computer users around the world.

Without a doubt, the birth of portable devices has been the biggest generator of data. IBM believes over 75% of the information we produce each day is unstructured and mostly coming from mobile phones. The sheer complexity of managing this large volume of data will only increase, as the number of mobile users is expected to grow to nearly 70% of the world’s population by 2017.

By then, the world would have also downloaded over 268 billion apps, generating revenue of more than £60 billion and making apps one of the most popular computing tools for global users. Research firm Gartner concludes that mobile users will provide personalised data streams to more than 100 apps and services every day.

The accumulation of data and the rise of malware attacks and information leaks have put the spotlight on the importance of good information handling and the need for data protection.

Right to erasure

In response to the challenges of managing Big Data, the EU is introducing new legislation to combat future security threats.  Among them is the EU General Data Protection Regulation (GDPR) that will strengthen individuals’ right to erasure and the right to be forgotten. The new legislation will arrive in 2015 and organisations will need to comply with the rules.

The GDPR is an important policy that will unify different regulations, like the EU Data Protection Directive 95/46/EC , thereby making it easier for companies to understand their data administration responsibilities.  Furthermore, the current EU regulations do not fully cover important aspects like globalisation or popular technological developments, such as Facebook, Twitter, Google+ and other social media circles. The new legislation will encompass all of the new ways of communicating in the digital age – and the subsequent information that’s generated from our interaction with it.

When this legislation comes into force, companies in both the private and public sectors will need to prove that data is securely erased in line with the new guidelines and show that they are fully accountable for monitoring, reviewing and assessing relevant processing procedures. They will need to show a willingness to minimise data processing and   unnecessary retention as well as incorporate safeguards for all data-related activities.

Companies are gradually becoming aware of this new responsibility – especially given the high cost for non-compliance.  If companies are caught out, they could face a severe penalty of up to 5% of their worldwide turnover. However, many are ill equipped to deal with the data erasure process. Additionally, they don’t fully grasp the risk or effort involved in collecting so much information and the consequences of security breaches.

Challenges of erasure

Indeed, we see that many organisations – especially SMEs – don’t know where to begin when it comes to erasing data, or they may only have partial or limited methods to erase.

One of the challenges is that a single standard for managing data erasure doesn’t exist.  Each organisation, including NATO, the Communications Electronics Security Group (CESG) and the British Standards Institute (BSI), has their own recommendations and algorithms, and it is not clear whether this will be addressed by the new directive.

For the most part, particularly with SMEs, data erasure will be performed using free erasure software with no certification (there is currently no obligation to certify the process so SMEs can save money this way) or by smashing up their discs using a drill or hammer.  Medium sized companies tend to entrust data erasure to IT administrators.

Bigger organisations are more likely to utilise third-party leasing companies, under which data erasure is part of a service agreement.  However, it is still important to check that such agreements will comply with the new regulation. There have been several high profile cases where disks or tapes left an organisation’s premises to be shredded, only to be later found dumped or discarded in non-secure locations. An erasure certification should be provided by the third-party to ensure that the process has been completed. Alternatively, an external verification provider can confirm whether the third party is complying with the promised erasure service.

Unintended data breaches

Kroll Ontrack’s experience suggests that these aforementioned approaches don’t always work. The press is full of stories revealing how companies have been caught out for failing to destroy sensitive data effectively and they have paid the price, both legally and financially.  For example, the National Health Service (NHS) has been slapped with hefty fines on numerous occasions for serious data breaches without intending to commit them.

In one case, Brighton and Sussex University hospital was fined £325,000 after hard drives, with highly sensitive patient data, were sold on eBay. In this case, a third party was commissioned to destroy the disks, but this action was not performed.

We also note that nearly 60 per cent of computers sent to data-removal specialists still contain data from the previous owner when they were recycled or resold. Many disks that are broken also hold recoverable data. These are risks that can no longer be taken by individuals or companies.

Increased malware attacks

Hospitals are not the only institutions caught out when it comes to unintended data breaches. The global telecoms sector has also had its fair share of embarrassments. For example, UK telecoms giant Vodafone fell victim to a malware attack. An unknown cybercriminal stole the names, addresses and bank account numbers of two million German customers.

The cost of data breaches

Analysts believe the global cost of data theft to companies and individuals is so great that it can’t be ignored, but trying to prevent it is difficult and costly.

The NHS, telecoms and Snowden scandals also show how vulnerable people become when they interact with digital technology. The more time they spend online, sharing personal information, the more exposed they can be to fraud, because the amount of unstructured data that is produced is particularly difficult to contain.

No room for complacency

Thanks to the upcoming GDPR regulation, everyone from third-party erasure companies to IT staff will be legally obliged to securely erase data.

Before they can follow guidelines, however, organisations must review their policies and be fully educated on the new law coming into effect. There are currently many independent events and congresses which are being used to educate IT and company leaders about the regulations, so there’s no excuse for ignorance. After education comes the important task of comparing current processes with new regulations and requirements.

Companies must adjust existing policies, processes and tools to meet new requirements, and to work with third-party companies that also know what’s expected from them by the new EU legislation.

How to permanently erase data

HDD

Crucially, organisations need to know how to properly erase data. There are so many different data storage types and they require different methods for wiping out data. For example, with Hard Disk Drives (HDDs), a degausser can be used to permanently erase data. It works by demagnetising HDDs, tapes, or any other magnetic media, thereby wiping data completely.

Companies can also choose eraser software that removes all data from HDDs, including both server and single drives. If eraser software is used, the hard disks can be reused.

SSD

Erasure on Solid State Drives (SSDs) is a trickier process. Unlike magnetic discs, SSDs store data electrically and apply complex data management schemes to disburse data across the memory. Furthermore, an SSD flash controller contains software modules that are hidden from the view of the operating system and the user.  There is also no standard SSD format, which means that erasure procedures for SSDs vary by brand and model.

Traditional erasure methods present different risks to SSDs.  For example, degaussing might work for HDDs, but SSDs use integrated circuits to store data and are electrically programmed and erased. A magnetic field will be ineffective at wiping out data. Physically destroying SSDs to wipe out data is also not advisable because skilled IT professionals can still recover data from flash chip fragments.

Unfortunately there is no publicly available single software tool that can securely erase data from every type of SSD or flash media; nor is there any way of knowing if the data has been successfully erased without verification from a third-party expert.

If companies wish to remove sensitive data on an SSD and don’t want to use data erasure services, they should at least use software encryption from the first day of deploying the disk. By doing this, companies can then wipe out any remaining data with a cryptographic erase option by simply deleting the encryption key.

Once this procedure is completed, companies can use physical destruction, such as shredding, to permanently destroy the disk. If there are still concerns that the data may remain on the disk, the best option is to contact a data erasure company for a final verification of the erasure procedure.  A good data expert will provide unbiased testing to ensure the effectiveness of the data erasure and the required validation to prove it.

Big Data / In-Memory System i.e. SAP HANA

In-memory systems are built using a specialised architecture, which combines traditional hard drives and flash memory as mass storage. This means that erasing a system large enough to withhold Big Data (i.e. after a proof of concept with a SAP HANA installation) is complex due to the system architecture. The individual storage devices cannot be deleted within the server, as data is consistently exchanged between the mass storage and the system cache.

Therefore, both the HDDs and the flash cards must be removed from the in-memory system and deleted externally, so that a secure, standardised environment is guaranteed to delete all data. Subsequently, the individual drives can be completely erased by repeated overwriting.

Tapes

The most commonly used approach to securely erase tapes is by shredding them or using a degausser where the extremely strong electromagnetic field causes all magnetic structures to be destroyed on the tape. When companies are trying to destroy data, it’s important that they remember their legacy tapes as well.  Indeed, legacy tapes contain a lot of data that are difficult to permanently remove unless they are also degaussed or shredded.

There’s no reason why any company should be caught out when the EU’s legislation is introduced in 2015. The knowledge and expertise to erase data is available and can prevent companies from future data breaches and serious legal penalties – arguably more difficult problems to overcome than the challenge of wiping out data.