What everyone is doing wrong about Big Data


I saw so many Big Data “initiatives” in the last month in companies. And guess what? Most of them failed either completely or simply didn’t deliver the results expected. A recent Gartner study even mentioned that only 20% of Hadoop projects are put “live”. But why do these projects fail? What is everyone doing wrong?

Whenever customers are coming to me, they “heard” of what Big Data can help them with. So they looked at 1-3 use cases and now want to have them put into production. However, this is where the problem starts: they are not aware of the fact that also Big Data needs a strategic approach. To get this right, it is necessary to understand the industry (e.g. TelCo, Banking, …) and associated opportunities. To achieve that, a Big Data roadmap has to be built. This is normally done in a couple of workshops with the business. This roadmap will then outline what projects are done in what priority and how to measure results. Therefore, we have a Business Value Framework for different industries, where possible projects are defined.

The other thing I often see is that customers come and say: so now we built a data lake. What should we do with it? We simply can’t find value in our data. This is a totally wrong approach. We often talk about the data lake, but it is not as easy as IT marketing tells us; whenever you build a data lake, you first have to think about what you want to do with it. Why should you know what you might find if you don’t really know what you are looking for? Ever tried searching “something”? If you have no strategy, it is worth nothing and you will find nothing. Therefore, a data lake makes sense, but you need to know what you want to build on top of it. Building a data lake for Big Data is like buying bricks for a house – without knowing where you gonna construct that house and without knowing what the house should finally look like. However, a data lake is necessary to provide great analytics and to run projects on top of that.

Big Data and IT Business alignment
Big Data and IT Business alignment

 

Summing it up, what is necessary for Big Data is to have a clear strategy and vision in place. If you fail to do so, you will end up like many others – being desperate about the promises that didn’t turn out to be true.

 

Advertisements

Big Data in Logistics


In the last weeks, I outlined several Big Data benefits by industries. The next posts, I want to outline use-cases where Big Data are relevant in any company, as I will focus on the business functions.

This post’s focus: Logistics.

Big Data is a key driver for logistics. By logistics, companies that provide logistics solutions and companies that take advantage of logistics are meant. On the one hand, Big Data can significantly improve the supply chain of a company. For years – or even decades – companies rely on the “just in time” delivery. However, “just in time” wasn’t always “just in time”. In many cases, the time an item spent on stock was simply reduced but it still needed to be stored somewhere – either in a temporary warehouse on-site or in the delivery trucks themselves. The first approach is capital intensive, since these warehouses need to be built (and extended in case of growth). The second approach is to keep the delivery vehicles waiting – which creates expenses on the operational side – each minute a driver has to wait, costs money. With analytics, the just in time delivery can be further improved and optimized to lower costs and increase productivity.

Another key driver for Big Data and logistics is the route optimization. Routes can be improved by algorithms and make them faster. This lowers costs and on the other hand significantly saves the environment. But this is not the end of possibilities: routes can also be optimized in real-time. This includes traffic prediction and jam avoidance. Real-time algorithms will not only calculate the fastest route but also the environmental friendliest route and cheapest route. This again lowers costs and time for the company.

Header Image by  Nick Saltmarsh / CC BY

Big Data in IT


In the last weeks, I outlined several Big Data benefits by industries. The next posts, I want to outline use-cases where Big Data are relevant in any company, as I will focus on the business functions.

This post’s focus: IT.

Big Data is a hot IT topic. Not just because it comes from the IT, but also because it gives great benefits to the overall IT operations. In a recent project, I’ve been working with a large european corporation in the manufacturing/production sector. Their IT had some 400 IT employees, serving more than 50,000 corporate employees and operating a large number of servers that run specific services. A key challenge for them was reliability of their services. To find out how a service is utilised, large amounts of log data were analysed in order to find out how they can prioritise different services. This gave them detailed insights on where they want to move their services too since different services had different utilisation patterns. The company could improve their utilisation of servers. New services get integrated in that approach as well, which means that they are capable of delivering these new services without the need to invest in new hardware.

Another great approach – and another hot topic – is Big Data for IT security. With Big Data analytics, companies can find security issues before they become serious threads. Patterns on web site access can provide insights on DoS attacks and similar issues. These analytics are often provided in real-time and provide fast ways to react in case problems occur.

As described in today’s article, Big Data is not just a topic coming from the IT, it is a topic MADE for the IT.

Big Data for Customer Services


In the last weeks, I outlined several Big Data benefits by industries. The next posts, I want to outline use-cases where Big Data are relevant in any company, as I will focus on the business functions.

This post’s focus: Customer Services.

Big Data is great for customer services. In customer services, there are several benefits for it. A key benefit can be seen in the IT help desk. IT help desk applications can greatly be improved by Big Data. Analysing past incidents and calls, their occurrence and impact can give great benefits for future calls. On the one hand, a knowledge base can be built to give employees or customers an initial start. For challenging cases, trainings can be developed to reduce the number of tickets opened. This reduces costs on the one side and improves customer acceptance on the other side.

Big Data can have a large impact here. When a customer feels treated well, the customer is very likely to come back and buy more at the company. Big Data can serve as an enabler here.

Big Data for Sales


In the last weeks, I outlined several Big Data benefits by industries. The next posts, I want to outline use-cases where Big Data are relevant in any company, as I will focus on the business functions.

This post’s focus: Sales.

Las week I outlined Marketing possibilities (and downsides) with Big Data. Very similar to Marketing is Sales. Often,  those two things come together. However, I would say it needs to be stated separately. In this post, I won’t discuss the Sales opportunities in Big Data from Webshops and alike. Today, I want to focus on Big Data opportunities that respect privacy but still have an impact.

Last year, I attended a conference where a company outlined their big data case. It was about analysing bills issued in their chain stores. The data from the bills included no personal details like credit card number, bonus card number and alike. It was only about what was in the basket. With the help of that, they could figure out what products get more attention at a specific store and how it differs from other stores. This data was joined with open data from public sources and other data about demographics. They could also find out that specific products get bought with another products – which means that if customer X buys product C, the customer is very likely to buy product D. An example of that for instance is that if you buy a skirt, you are also likely to buy a top.

The later example focused on analysing data for fashion stores. However, most stores can benefit from Big Data. I recently had the chance to talk to the CIO of a large supermarket chain. They also have some Big Data algorithms that improve their chain stores. The company’s policy is to accept their customer’s privacy and they don’t work on their personal data. They figured out when the neighbourhood changes – e.g. because a university was built. They could see that other products are demanded and changed the assortment of goods accordingly.

There are many opportunities where Big Data can improve Sales, and as shown in these two examples, they don’t necessarily need to violate someone’s privacy.

Big Data for Marketing


In the last weeks, I outlined several Big Data benefits by industries. The next posts, I want to outline use-cases where Big Data are relevant in any company, as I will focus on the business functions.

This post’s focus: Marketing.

Marketing is one of the use-cases for Big Data, which are discussed controversial. One the one hand, it gives opportunities to companies to adjust offers to their customers and make the offers more “individual”. I will describe the themes here before I will discuss the downsides of this.

With customer loyalty programs, companies can better “target” their customers. When the company understands the behaviour of the customer, special offers and promotions can be sent to the customer. We all know this from large online shops, where you get regular offers by e-mail. But this also applies to retail stores around you: with programs from the retailers, they also collect data about their customers and can improve the portfolio. Furthermore, they can make their advertisement more individual – and increase the revenue. Marketing gets valuable insights for all industries. Retail is the most common, but also other industries that are not in retail can gain benefits from it. Companies that work in B2B can create value from Big Data by adjusting their sales processes adjusted by data – and react to new trends before competitors find out.

On the other side, this is somewhat frightening. I am basically in favour of Big Data. However, there must be some kind of assurance that personal privacy is respected. At present, it is hard to opt-out of such programs.

Big Data is everywhere! In all major industries


The last weeks I outlined several industries that can benefit from Big Data. However, this was just a short overview on what is possible. Let me use this post to sum up the industries that benefit from Big Data. You can get an overview by this tag.

In the first post I started with manufacturing. This traditional industry sees major benefits from Big Data, especially with Industry 4.0. You can read the full post here. Big Data is already used heavily by another industry – the finance sector. Major banks, insurances and financial service providers use Big Data. I outlined the possibilities in this post.

Big Data is also a Big Deal for the public sector. Not just that the Obama administration announced to make more data available – it also gives major benefits to smart cities and alike. You can read the full post here. Often included in public sector is healthcare. Healthcare sees great benefits from using Big Data as well. I’ve summed up the benefits here.

The oil and gas industry can also benefit from Big Data by applying them to sensors while drilling. A sector where you might not expect benefits from IT or Big Data is agriculture. But Big Data can give major benefits to this industry as well – as described here.

Next week I will start to look at the functions within a company – to see where Big Data is within a company – independent from the industry.