Author: Scottie Todd

Digital Marketing Lead

How Artificial Intelligence Will Disrupt the Financial Sector

Post author By Scottie Todd
Post date May 24, 2019

Artificial intelligence thrives with data. The more data you have, the better your algorithms will be. However , just having a lot of data is not sufficient anymore. You also need high-quality data, or in the words of Peter Norvig, you need better data:

“We don’t possess better algorithms, we just have a lot more data. More information beats clever algorithm, but better data beats more information. ” – Peter Norvig – Director of Research, Google

Nowadays, most organisations collect vast troves of data, but especially the financial sector is well-suited for also collecting high-quality data. Simply because of regulations and because a lot of data in the financial sector is structured data. There is also an abundance of data within public markets or even other external sources that can become linked for additional insights. As it seems, banks and insurance companies can benefit a lot from AI, if implemented correctly, of course.

Financial Institutions Have to Innovate

Besides, more and more consumers require financial institutions to innovate. They have become fatigued with overbearing fees to their manage capital and provide products such as credit. The below graph by State of AI clearly shows the difference in costs between traditionally managed wealth and automated management of wealth. As a…

How AI is Transforming the Aviation Industry

Post author By Scottie Todd
Post date May 24, 2019

The aviation industry, particularly commercial aviation, is continually aiming to improve both the manner in which it works and its consumer loyalty. Keeping that in mind, it has started utilizing artificial intelligence. In spite of the fact that AI in the aviation business is still in the beginning stage, some advancement has been made as of now as certain leading carriers put resources into AI. To begin with, certain use cases are being achieved, for example, facial recognition, baggage check-in, client inquiries and replies, plane fuel enhancement and factory assignments improvement. Be that as it may, AI can conceivably go a long way past the present use cases.

Commercial airline travel is a financial engine which effected an expected $168.2 billion in operating income in 2016. Ticket fees spiked to 74. 5% of operating income or $125.2 billion dollars, and airline traveler traffic is anticipated to double throughout the following two decades.

Today, leading airlines are investigating how AI can enable them to keep pace with client demand and improve operational adequacy, speed plus consumer loyalty. The following are a couple of changes we have seen, and what’s in store sooner rather than later.

Baggage Screening

Baggage screening is a dull yet significant task done at the airplane terminal. In any case, AI has disentangled the procedure associated with baggage screening. Osaka Airport in Japan is intending to introduce the Syntech ONE 200, which is an AI innovation created to screen baggage for numerous passenger lanes. Such devices won’t just automate typically the procedure of baggage screening, in addition they help authorities identify unlawful activities. Syntech ONE 200 uses an X-beam security system and it increases the likelihood of identifying potential dangers.

In 2017, American Airlines led an application development competition with the objective of having an application created for making baggage screening simpler for travelers. The particular competition, named HackWars, was themed on AI, drones in addition to augmented reality and VR. The winner, known as “ Team Avatar, ” built up an application that would not just permit travelers to decide their baggage size before arriving at the airline terminal, but in addition, prepay any potential related costs.

Virtual Assistants

Artificial intelligence based virtual assistants help aircraft organizations improve the productivity and even effectiveness of their pilots by decreasing repetitive assignments, for example, changing radio channels, perusing wind forecasts, and giving position data on request, among others. These repetitive jobs can be taken care of by AI-empowered virtual assistants. Organizations, for instance, Garmin (US) offer AI-empowered audio boards, which are invaluable tools for pilots.

Virtual assistants are likewise utilized by aircraft organizations to improve client services. Artificial intelligence empowered virtual assistance can give instant answers to basic inquiries. Normal inquiries incorporate points like flight status or services/contributions (sound, video, Wi-Fi) on flights. This allows the human customer service ambassadors to take care of more significant issues requiring a human.

Alongside that, virtual assistants are helping travelers book and plan their trips. A wide range of organizations are making their very own applications to enable clients to automate various tasks related to travel. Gone are the days when you needed to book your flights and hotels, rent a vehicle, check in, and plan your itenerary alone. Artificial intelligence and the virtual assistants inside these applications gather information from you through simple prompts, at that point automate the tasks for you.

Customer Assistance

United Airlines is utilizing Amazon’s Alexa to reply to routine traveler questions. In September 2017, United reported a collaboration with Amazon’s Alexa. The feature is known as the United skill. To begin, travelers should simply add the United ability to their Alexa application and after that begin posing questions. Alexa answers regular questions effectively, for example, the status regarding a trip simply by number, check-in requests, and accessibility of Wi-Fi on a new flight. The reviews so far have been mixed, which suggests there is still more learning and adaptation for this technology. It may be a few more years before AI can completely take over client assistance.

AI Maintenance Prediction

Airline companies are wanting to use AI innovation to predict potential failures of and plan maintenance on aircraft. Leading aircraft producer Airbus is taking measures to be able to improve the dependability of aircraft through enhanced maintenance. They are utilizing Skywise, a cloud-based data storing framework. It oversees its fleet in gathering and recording a massive quantity of real-time information.

AI in predictive maintenance analytics is also establishing patterns and best practice methodologies for how and when the airplane maintenance should be completed. Enhanced, more predictable maintenance means fewer unscheduled delays and a better traveler experience.

In the meantime, organizations are making changes to screen the “health and status” of their aircraft in real-time. Air Canada CEO Calin Rovinescu says advanced analytics are required to keep planes flying over 16 hours every day. AI frameworks could anticipate when maintenance is required even before a part fails, incorporating quicker fixes and avoiding downtime for the aircraft. So-called “wellbeing monitoring” of aircraft enables data to be examined more rapidly and precisely, enabling preventive activities to be quickly performed.

Data Management

Enormous data volumes are being produced and used. As the aviation industry embraces AI, this volume will inevitably lead to some data confidentiality risks. The need to appropriately govern and secure information goes hand in hand with this increased adoption of AI benefits. Several breaches and events, such as one where Emirates, a leading airline, leaked client data to third parties without approval. It was discovered that key customer details: name, email, schedule, telephone number and even passport number were exposed to third-party service providers. Even though Emirates policy specifically expresses that there will be no information sharing, travelers need to be cautious.

Preventing future disasters

Possibly one of the most important applications of AI-based analytics, however, might be in identifying risks to the security of aircraft in front of a tragedy –such as the crash of Lion Air Flight 610, even when a failure of the automated control system onto a prior flight might have indicated a major security issue. NASA Ames Research Center at Silicon Valley is significantly engaged in aviation-related AI, and a few of NASA’s jobs there is focused on distinguishing”anomalous operations” within data from commercial aviation–events which could be precursors to possibly larger issues. Since commercial aviation’s safety record is so good–much better than driving, for example–it is much more difficult to recognize those few cases where there is an anomaly that might represent a safety problem.NASA has performed some first development of algorithms related to anomaly detection and episode precursor diagnosis, and it’s started the process for gathering feedback from experts in the area. The airlines upload some subsets of their flight-recorded information to Mitre, which performs analysis and provides feedback on possible problems. (The data is shared supplied by the airlines.)The hope for those analytics being developed at Ames is that the AI can discover patterns of anomalies in flight info that may be indicative of a systematic issue with aircraft. Analysts would love to discover as soon as possible and produce some kind of a reduction to stop it happening again. Up to now, instead of AI replacing humans in air, AI and human specialists have proven to be complementary–a venture that can save human lives.

Big Data Business Intelligence Data Analytics Data Monetization

7 Ways To Grow Your Business with Data Monetization

Post author By Scottie Todd
Post date May 21, 2019

It’s estimated that by the year 2020 revenues around the world for big data and business analytics are going to exceed $203 billion. With all this earning potential, it makes sense to want to get your business “in on it.”

One of the best ways to do this is with data monetization. After all, data is the new currency.

In the past, businesses in the information technology sector have always been deriving value from data. However, the ability to effectively use and monetize data is now impacting virtually all types of business.

This means that driving value from data is something you can implement in your own business strategy. What many people may not realize is that this process can be extremely challenging.

As a result, you need to learn some helpful tools and actionable steps you can take to monetize data for your business.

If you are interested in learning more, then keep reading.

1. Decision Architecture

When thinking about analytics, the majority of organizations want to know how their business is performing, along with what information is needed to answer various performance questions. While this can help to inform and to describe what is taking place in the organization, it doesn’t enable any type of action.

Instead, the goal needs to be to capture the decision architecture of specific business problems. Once this is done, you can build analytics capabilities to create a diagnosis that enables decisions and actions. Leaders need to focus on making decisions that are based on data, rather than just answering questions about what already happened.

2. Stop Revenue Leaks

Busy healthcare providers, clinics, and hospitals can easily lose track of the services being rendered. Every procedure has an assigned code and description. Each of these often includes errors.

By using analytics, the organizations can identify patterns associated with procedures and codes, flagging patient invoices for possible errors or even missing charges. Intelligent data use can also help the organizations improve the ROI of their collections process.

3. Data Aggregation

The method that is at the very bottom of the pyramid, but that represents the biggest opportunity to earn, is data aggregation.

This means taking data from various sources, including your business, and merging it together to create a larger, integrated picture. While the data sources on their own may be interesting, when they are combined, they become valuable.

An example of this would be your credit report. The information credit bureaus aggregate, such as the credit cards you have, if you have a mortgage, and if you pay your bills on time, can be sold for a profit.

By aggregating this information into a single report, the information can be sold to interested parties. While there isn’t a lot of money in this, it’s still money.

4. Infer Customer Satisfaction

Many organizations use social media and survey sentiment to understand the levels of customer satisfaction. By combining data from several sources, airlines can now infer how satisfied a customer is based on factors, like where they are sitting.

This process requires information to be aggregated from several sources. However, in the airline example, you can use the information to determine if a customer is going to fly with you again, and if not, offer a free upgrade or other incentives.

5. Embrace a New Revenue Model

Today, data is actively changing relationships companies have with customers. Manufacturers of tangible goods are now supplementing the products they sell with flexible software options and services to offer customers new choices and new revenue streams.

Additionally, these companies are providing much higher levels of personalization. Across several industries, new economic models are starting to be explored – like replacing an auto fleet with self-driving cars.

In this example, rather than selling data, people are going to pay you to solve a problem or to provide answers. This is a unique revenue model.

The value lies in the fact that you have married your data to the mission of a business and solving a problem that businesses have. This is what is going to generate revenue.

6. Detect Piracy and Fraud

Most online retailers sell products on several different websites. Supplemental sales channels typically include eBay, Amazon.com and other online marketplaces maintained by larger retailers, like Best Buy and Walmart.

Selling through these channels is extremely data-intensive, since the customer types, products, and pricing can vary greatly across the channels. In some case, the price discrepancies are so large that they signal possible piracy or fraud.

If you sell across dozens of e-commerce websites, then consider building databases of your own products and your unique pricing. You can then compare this to existing expected pricing data, allowing you to detect stolen goods or suppliers who are mispricing their goods.

With this information, it’s possible to go to the marketplace and make a report stating that they believe someone is selling stolen items.

How Can You Use Data Monetization Methods for Your Business?

Data monetization is an ever-evolving concept that offers opportunities to earn profits by providing information to others. Your business can take advantage of this by utilizing the tips and information here.

The fact is, there are already countless businesses, in all industries, that are currently using data monetization. Now is the time to begin doing so, too, as it offers huge revenue stream potential.

If you are convinced that data monetization is something you want to use for your company, then contact us. We can provide you with help and information about how this process works.

Big Data Data Analytics

How Social Media Data Can Boost Your Sales

Post author By Scottie Todd
Post date May 21, 2019

Social media data is one of the richest sources of information available to modern marketers, influencers, website operators and data scientists. One of the challenges, though, is finding the right way for your operation to harness that power. Let’s take a look at how social media data can boost your sales.

The Raw Data in Social Media

There are plenty of ways to deploy data analysis tools to both mine data and derive insights from it. These include looking at data points like:

Shares and likes
Mentions
Hashtags
Click-thrus to URLs
Addition and loss of followers
Demographic groups
Influencer networks

It’s important to not obsess about the vanity metrics, though. All the followers in the world don’t mean much if they’re not translating into sales. For example, tracking codes need to be embedded with URLs to verify that social media followers are moving into the marketing funnel. By using embedded referral codes specifically designed for your social media presence, you can keep tabs on whether followers are converting.

Finding useful sources of data is also important. There are plenty of free options, such as pulling marketing data from:

Facebook Insights
Google Analytics
Twitter Analytics
LinkedIn Analytics

Some social media companies, such as Instagram, also offer paid access to their data. In many cases, however, it’s possible to pull data using other solutions, such as web scrapers.

If your setup is properly configured, you should be able to track engagement as it moves through your marketing funnel. For example, your Twitter-specific referral code will show up in both Twitter Analytics and Google Analytics, making it easier to tie user behavior to particular campaigns.

Developing Insights from Social Media Data

The best pool of information means nothing if you can’t use data analysis tools to derive insights from it. Foremost, you need to know what goals your business is shooting for. You can make a checklist that covers things like:

Acquiring new customers
Developing a more widely recognized brand
Making decisions based on social media data
Responding better to customer concerns
Fostering a superior customer experience

Let’s say your business wants to focus on social media as a way to quickly identify customer complaints. One great thing about social media is that folks quitting your brand might not call your customer support hotline to express their discontent, but you can bet they’ll complain to their friends online about your company’s products and services.

One way companies take advantage of this is sentiment analysis. This is a data-driven decision-making tool that focuses on gathering data regarding positive, negative and neutral statements that people make about companies online. By regularly scanning social media, these firms are able to “read the room” at a global scale. Instead of letting customer anger fester out of sight, sentiment analysis allows companies to get out in front of problems.

There is also plenty of information hiding in the networks that folks form on social media. Marketing data can be developed by creating network maps of their social associations. For example, a retailer that wants to build an influencer campaign on Instagram wants to know which users are going to spread ideas the fastest. They can then supply those Instagram influencers with:

Early access to product details
Marketing and brand materials
Product demos and samples
Immediate access to top-tier customer and technical support
Opportunities to meet with key players
Invitations to company-sponsored events

Driving Business Decisions

Using marketing data should not be seen as a one-way street. There’s a lot that can be learned by monitoring the social media sphere. Trend analysis, for example, can allow companies to get ahead of what people are excited about. A clothing company might focus on analyzing trends coming into each of the fashion seasons, allowing them to handle ordering issues like:

Choosing quantities
Conveying customer demands to overseas buyers
Establishing transport times to put products in stores in time for trends to peak
Re-ordering items that are expected to sell out

It’s important to develop a data-driven culture at a company in order to make the most of social media data. Stakeholders and decision-makers shouldn’t be stuck wondering what the social media budget is actually doing. By deploying dashboards, data scientists at companies can provide real-time, engaging insights to those parties. In no time at all, folks who once questioned data and social media expenditures will be checking the dashboards on their cellphones to see how campaigns are unfolding.

Building this sort of data-centric business culture requires an investment. Infrastructure has to be put in place to ensure data scientists on your team have the servers they need to pull data, clean it up, analyze it and generate insights. Done the right way, though, building out this sort of infrastructure can help you get a better grasp on how customers interact with your brands, products and services.

6 Information And Analytics Trends To Prepare for in 2020

Post author By Scottie Todd
Post date May 20, 2019

We are well past the point of realization that large data and advanced analytics solutions are valuable — nearly everyone knows that by now. Actually, there’s no escaping that the increasing dependence on technology. Substantial data has become a modern staple of nearly every sector in retail to manufacturing, and for good reason.

IDC forecasts that if our digital universe or total information content were represented by tablets, then by 2020 they would extend all of the way to the moon over six times. That is equal to 44 trillion gigabytes, or 44 zettabytes of information.

There are lots of reasons why information has been created so quickly — doubling in size every 2 years. Although the demand for reliable information is another the birth of IoT and connected devices is one source. What’s more interesting are the tendencies formed as a result of the options that are digitally-reliant that are more recent. They specifically help form the business, altering business analysts work with information.

Will our future look like? How will we handle all this information? What abilities need to company analysts be focused on developing?

1. Specialization of Job Roles

For quite a while, the information scientist and analyst roles have been universal in character. It is not that specializations did not exist, they have but firms are starting to search for professionals who have industry-specific experience. They want someone well versed specifically in the sort of data they are dealing with.

Everything from financial services to manufacturing and logistics has been upgraded to rely on more digital services and as a result a influx of real time information. There are plenty of opportunities, so livelihood decisions won’t be hurt by picking a specialty, but doing exactly the can. It is important to construct a good CV by working together with businesses and teams that fit a specialty, so choose one.

2. Machine Learning Experience is a Must

From 2020, over 40 percent of data science tasks will be automated. Its capacity and machine learning technologies is a massive driver of that automation. It is for good reason because effective machine learning tools and automation will help extract insights that would otherwise be tricky to find even by skilled analysts.

The whole process is achieved much faster, boosting not just general efficiency but the response time of an organization to certain events.

Data scaling analysis, quantitative analysis, automation resources and, of course, overall machine learning are skills that modern data analysts must try to hone. Talent and the more experience that an analyst has with automation technologies, the more desirable they will be.

3. The Growth of Legislation

GDPR helped to spur the requirement for qualitative information governance, and frankly, it happened so quickly it left many firms scrambling to comply — even still some are fumbling with the idea. But it is not. More lately, that the California Consumer Privacy Act reared its head, that will go into effect in 2020. It won’t be the last either, not by a longshot.

These regulations have a monumental effect on information processing and managing , customer profiling and information security. Businesses are under extreme pressure not only to comprehend the effect on operations but also to comply with the requirements.

Data scientists and analysts who understand the ramifications will help organizations browse the guidelines and are skilled in security and data privacy are in large demand. As regulations come to be, that need will continue to rise making it a viable specialty for current and future professionals.

4. Stay close to the Bleeding Edge

It’s no small accomplishment to stay up-to-date with anything that relates to modern technology. Tools and solutions are growing at absurd prices, new opportunities are being introduced, and many different tendencies take form year after year. But regardless of how difficult it is, information analysts have to continue to stay in the forefront of that growth.

A good analyst may focus but never puts their whole stock in toolset, platform or one technology. Using databases, as an instance, choices may include HBase NoSQL and MongoDB but its most priorities may change over time. Information processing is another skill key to remaining relevant in the analytics area. Professionals will probably be desired by companies, individuals and government offices .

For frameworks and languages, there’s Apache Hadoop, Python, R, SAS and others. However, more importantly, these would be the cases now — right now — and they could shift or alter over time.

It is up to analysts to stay present with all and any options readily available, and that means embracing a state of growth and constant improvement so far as knowledge and abilities are involved.

5. Cloud Computing and Related Mechanics

Software engineers and data scientists are two fields, but that does not necessarily imply overlap does not happen. It certainly does and professionals need to understand that achieving it is a very important part of staying relevant in the market of today.

As the requirement for more liquid and elastic infrastructure develops, scientists and analysts need to comprehend how this relates to current operations and gear. When dealing with possible troubles and performance demands Having the ability to evaluate the load on a machine, for instance, can go a very long way.

It is this concept of knowing infrastructure and the hardware in the helm that will elevate professionals to fresh heights. Substantial data analytics, machine learning, none of these technologies would exist with no cloud computing along with the infrastructure.

Until recently, the focus has been about processes and the instruments that can help attain a better understanding of information stores.

The technology gets more capable and is adopted more publicly, as, the requirement to comprehend the hardware has also become more important. The overlap between software and hardware related jobs and the demand for professionals to comprehend the range of the systems.

6. Basic Business Intelligence Expertise is Key

The data analyst of today kept separate and then is not secured in a tower. In fact, it’s nearly always the opposite that is complete, as scientist interact with decision makers and groups. This means that information professionals have to be able to efficiently communicate complicated issues to professionals.

Communication occurs to be a critical soft skill of company intelligence. But it’s not the only skill required to thrive. SQL programming abilities Tableau such as — and problem-solving are just a couple of examples.

The past and tomorrow’s analysts should have a good foundation in business intelligence.

Growth is Always a Must, however the Right Growth is Key

Obviously, build experience in the industry and information analysts may continue to rise as they take on more projects. But boosting the growth that is perfect, into particular areas and abilities, can assist professionals achieve victory, but also secure opportunities in the area.

An increasing number of organizations deploy information analytics tools to affect their operations and to understand consumer behaviour.

These tools have become more advanced alongside the technology, as time progresses. It is around analysts to comprehend tools and the core systems but also the underpinning hardware

Business Intelligence Data Quality

Chapter 3 – Cleaning Data with Excel and Google Sheets

Post author By Scottie Todd
Post date May 16, 2019

How Spreadsheets Became The #1 BI Tool in the World

(Excerpt from The Ultimate Guide to Cleaning Data in Excel and Google Sheets)

Microsoft Excel and Google Sheets are the first choice of many users when it comes to handling large amounts of data. They’re readily available, easy to learn and support universal file formats. When it comes to using a spreadsheet application like Excel or Google Sheets, the point is to present data in a neat, organized manner which is easy to comprehend. They’re also on nearly everyone’s desktop, and were probably the first data-centric software tool any of us learned.

In this eBook, we are going to tell you some of the tips as to how to clean and prep up your data using Excel and Google Sheets, and make it accurate and consistent, and make it look elegant, precise, and user-friendly.

Risks of Cleaning Data In Spreadsheets

While spreadsheet tools are quite adequate for many small to mid-level data chores, there are some important risks to be aware of. Spreadsheets are desktop-class, file-oriented tools which means their entire data contents are stored in volatile RAM while in use and on disk while you’re not using them. That means that between saves, the data is stored in RAM, and can be lost.

Spreadsheet tools also lack any auditing, change control, and meta-data features that would be available in a more sophisticated data cleaning tool. These features act as backstops for any unintended user error. Caution must be exercised when using them as multiple hours of work can be erased in a microsecond.

Unnoticed sorting and paste errors can also tarnish your hard work. If the data saves to disk while in this state, it can be very hard, if not impossible, to undo the damage and revert to an earlier version.

Spreadsheets also lack repeatable processes and automation. If you spend 8 hours cleaning a data file one month, you’ll have to repeat nearly all of those steps the next time another refreshed data file comes along. More purpose-designed tools like Inzata Analytics allow you to record and script your cleaning activities via automation. Data is also staged throughout the cleaning process, and rollbacks are instantaneous. You can set up data flows that automatically perform cleaning steps on new, incoming data. Basically, this lets you get out of the data cleaning business almost permanently.

Performance and Size Limits in Spreadsheet Tools

Most folks don’t bother to check the performance limits in Spreadsheet tools before they start working with them. That’s because the majority won’t run up against them. However, if you start to experience slow performance, it might be a good idea to refer to the limits below to measure where you are and make sure you don’t start stepping beyond them. Like I said above, spreadsheet tools are fine for most small data, which will suit the majority of users.

Excel Limits

Excel is limited to 1,048,576 rows by 16,384 columns in a single worksheet.

A 32-bit Excel environment is subject to 2 gigabytes (GB) of virtual address space, shared by Excel, the workbook, and add-ins that run in the same process.
64-bit Excel is not subject to these limits and can consume as much memory as you can give it. A data model’s share of the address space might run up to 500 – 700 megabytes (MB), but could be less if other data models and add-ins are loaded.

Google Sheets Limits

Google Spreadsheets are limited to 5,000,000 cells, with a maximum of 256 columns per sheet. (Which means the rows limit can be as low as 19,231, if your file has a lot of columns!)
Uploaded files that are converted to the Google spreadsheets format can’t be larger than 20 MB and need to be under 400,000 cells and 256 columns per sheet.

In real-world experience, running on midrange hardware, Excel can begin to slow to an unusable state on data files as small as 50mb-100mb. Even if you have the patience to operate in this slow state, remember you are running at redline. Crashes and data loss are much more likely!

If you believe you will be working with larger data, why not check out a tool like Inzata, designed to handle profiling and cleaning of larger datasets?

(Excerpt from The Ultimate Guide to Cleaning Data in Excel and Google Sheets)

Data Quality

Ebook Excerpt: Chapter 2 – Why Clean Data?

Post author By Scottie Todd
Post date May 16, 2019

Chapter 2: Why Clean Data?

(Excerpt from The Ultimate Guide to Cleaning Data in Excel and Google Sheets)

Data cleansing is the process of spotting and correcting inaccurate data. Organizations rely on data for many things, but few actively address data quality. Whether it’s the integrity of customer addresses or ensuring invoice accuracy. Ensuring effective and reliable use of data can increase the intrinsic value of the brand. Business enterprises must assign importance to data quality.

A data-driven marketing survey conducted by Tetra data found that 40% of marketers do not use data to its full effect. Managing and ensuring that the data is clean can provide significant business value.

Improving data quality can eliminate problems like expensive processing errors, manual troubleshooting, and incorrect invoices. Data quality is also a way of life because important data like customer information is always changing and evolving.

Business enterprises can achieve a wide range of benefits by cleansing data and managing quality which can lead to lowering operational costs and maximizing profits.

Who are the heroes who allow the organization to seize and enjoy all these benefits? I affectionately refer to these poor souls as PWCD’s, or People Who Clean Data[1].

These brave people, and hopefully you are reading this because you hope to be one of them, are the noblest. They often get little recognition even though they clean up the messes of hundreds, if not thousands of other people every day. They are the noble janitors of the data world. And I salute them.

Top 5 Benefits of Data Cleaning

1. Improve the Efficiency of Customer Acquisition Activities

Business enterprises can significantly boost their customer acquisition and retention efforts by cleansing their data regularly. With the high throughput of the prospecting and lead process, filtering, cleansing, enriching having accurate data is essential to its effectiveness. Throughout the marketing process, enterprises must ensure that the data is clean, up-to-date and accurate by regularly following data quality routines. Clean data can also ensure the highest returns on email or postal campaigns as chances of encountering outdated addresses or missed deliveries are very low. Multi-channel customer data can also be managed seamlessly which provides the enterprise with an opportunity to carry out successful marketing campaigns in the future as they would be aware of the methods to effectively reach out to their target audience.

2. Improve Decision-Making Processes

The cornerstone of effective decision making in a business enterprise is data. According to Sirius Decisions, data in an average B2B organization doubles every 12-18 months and though the data might be clean initially, errors can creep in at any time. In fact, in nearly all businesses where data quality is not managed, data quality decay is constantly at work. Each time new records are added; duplicates may be created. Things happening outside your organization, like customers moving and changing emails and telephone numbers will, over time, degrade data quality.

Yet the majority of enterprises fail to prioritize data quality management, or even acknowledge they have a problem! In fact, many of them don’t even have a record of the last time quality control was performed on their customer’s data. More often than not they merely discard or ignore data they believe to be of poor quality, and make decisions through other means. Here you can see that data quality is a massive barrier toward digital transformation and business intelligence, much less every company’s desire to become more Data-Driven.

Accurate information and quality data are essential to decision making. Clean data can support better analytics as well as all-round business intelligence which can facilitate better decision making and execution. In the end, having accurate data can help business enterprises make better decisions which will contribute to the success of the business in the long run.

3. Streamline Business Practices

Eradicating duplicate and erroneous data can help business enterprises to streamline business practices and avoid wasteful spending. Data cleansing can also help in determining if particular job descriptions within the enterprise can be changed or if those positions can be integrated somewhere else. If reliable and accurate sales information is available, the performance of a product or a service in the market can be easily assessed.

Data cleansing along with the right analytics can also help the enterprise to identify an opportunity to launch new products or services into the market at the right time. It can highlight various marketing avenues that the enterprises can try. In practically any other business process you can name, decisions are made every day, some large, but many small. It is this systematic pushing of high-quality information down the chain of command, into the hands of individual contributors that helps them improve decisions made at all levels of the organization. Called Operational Intelligence, it is used more commonly for quick lookups and to inform the thousands of decisions that are made every day inside the organization.

4. Increase Productivity

Having a clean and properly maintained enterprise dataset can help organizations ensure that the employees are making the best use of their time and resources. It can also prevent the staff of the enterprise from contacting customers with out-of-date information or create invalid vendor files in the system by conveniently helping them to work with clean records thereby maximizing the staff’s efficiency and productivity. High-quality data helps reduce the risk of fraud, ensuring the staff has access to accurate vendor or customer data when payments or refunds are initiated.

5. Increase Revenue

Business enterprises that work on improving the consistency and increasing the accuracy of their data can drastically improve their response rates which results in increased revenue. Clean data can help business enterprises to significantly reduce the number of returned mails. If there are any time-sensitive information or promotions that the enterprise wants to convey to their customers directly, accurate information can help in reaching the customers conveniently and quickly.

Duplicate data is another aspect which can be effectively eradicated by data cleansing. According to Sirius Decisions, the financial impact of duplicate data is directly proportional to the time that it remains in the database.

Duplicate data can significantly drain the enterprise’s resources as they will have to spend twice as much on a single customer. For example, if multiple mails are sent to the same customer, they might get annoyed and might completely lose interest in the enterprise’s products and services.

[1] “People with Crappy Data” is an alternate interpretation coined by some of my clients.

Big Data Data Analytics Data Quality

What You Need to Know About Monetizing Healthcare Data

Post author By Scottie Todd
Post date May 13, 2019

Healthcare services providers generate huge amounts of data in the course of any given year. Many organizations, though, see this work as a source of financial losses. In a more modern view of the situation, however, all this healthcare data maintenance can be seen as a potential way to decrease losses and to create profit centers. Let’s explore some of the ways data monetization can benefit a business in the healthcare industry.

Ethical and Legal Concerns with Data Monetization

HIPAA is, rightly, the dominant issue when dealing with the legality of any monetization effort, but not as much as one might think. Bear in mind that anonymization, when performed competently, does cover the confidentiality issues related to HIPAA.

The more concerning problem is on the ethical side of the equation. In particular, efforts to anonymize data need to focus on ensuring identifying factors, such as addresses, Social Security numbers and even uniquely assigned identifiers aren’t traceable to any one patient. This can be surprisingly challenging, as evidenced by work from MIT researchers that found anonymized datasets could be mapped to individuals based on location data and networks.

When setting up data sets, you definitely want to discuss these worries in detail with the parties handling them. Other stakeholders, including doctors, patients and your organization’s lawyers should be included in the process.

One solution worth considering is asking patients to opt in to information sharing. This requires creating a framework that guarantees the confidentiality of the data, and there also needs to be legal language that explains patients’ rights in detail. Such documents should always include an opt-in process that requires a patient to clearly indicate their interest and to provide their consent. This is absolutely essential if you’re going to be monetizing data by selling to third parties.

Reducing Losses

Much of the low-hanging fruit in the industry comes from situations where data analysis can provide insights regarding losses. In the simplest form, this comes from streamlining processes, such as:

Scheduling appointments between doctors and patients more efficiently
Avoiding duplication of medical efforts
Preventing potential slip-ups
Maintaining contact with patients about screenings and check-ups

There’s also another level at which healthcare data can be utilized to spur the well being of patients. For example, insurance carriers and hospitals mining patient data have discovered trends among their customers where preventive measures can be taken. By providing those customers with medical devices, preventative screenings and other forms of care, they’ve reduced costs by avoiding more expensive, radical and reactionary solutions.

Healthcare data can also be utilized to establish the efficacy of medical options. Rather than relying on old habits and tried-and-true solutions, professionals can utilize large-scale to develop insights about which drugs produce the best outcomes for the dollar. Data is also employed in researching:

Genomic and epigenetic concerns
Pharmacology studies
New drug discoveries and treatments

Developing Healthcare Data Profit Centers

While HIPAA rules limit the amount of specific data that can be sold in relation to patient care, anonymized data is still a strong potential profit center. Researchers, insurance companies, government agencies and marketers are all looking for information that can fuel their own data analysis. This sort of data can benefit groups that are trying to develop:

Economic models
Government policies
Metastudies
Information regarding rare disease
Trend analysis

Packaging data for third parties carries with it several concerns that need to be taken seriously. Foremost, it’s important that all patient data be scrubbed of any identifying features. Secondly, large banks of data become targets for hackers, and it’s important to secure all your systems. Thirdly, aggregation of anonymous data will likely demand some investment in bringing in qualified data scientists, establishing in-house standards and building out computing resources.

There is also the cultural component that comes with all efforts to become more data-centric. Stakeholders need to be brought in on monetization efforts, and it’s critical to confirm they are on board with the technical, cultural, legal and ethical requirements of the process. While you don’t want to clear out folks who have honest objections, there usually are situations where stakeholders have to be bought out of contracts or given severance packages. Your goal during a monetization push should be to develop a larger organizational commitment to doing it well.

A commitment to data and monetization takes time. Resources and talent have to be accumulated, and data often has to be prepped for outside consumption. This means taking into account data consumers’ concerns about data lineage, unique identifiers and other information that allows them to do their job well. Being able to present both internal stakeholders and third parties with finished products can make offerings significantly more appealing.

Plenty of thought goes into monetizing data from a healthcare organization. In time, though, a portion of your business that seems like it costs you money can end up curtailing losses and generating new sources of revenue.

Read more similar content here

Big Data Data Analytics Data Quality

What is Data Lineage & Why is it Important?

Post author By Scottie Todd
Post date May 7, 2019

In the world of data analytics in 2019, keeping tabs on where bits of information came from, how they were processed and where they ended up at is more important than ever. This concept is boiled down to two words: data lineage. Just as a dog breeder would want to the lineage of a pooch they’re paying for, folks in the business intelligence sector want to know the lineage of the data that shows up in a final work product. Let’s look at the what, the why and the how of this process.

What is Data Lineage?

The simplest form of lineage for data is indexing items with unique keys that follow them everywhere. From the moment a piece of data is entered into a system, it should be tagged with a unique identifier that will follow it through every process it’s subjected to. This will ensure that all data points can be tracked across departments, systems and even data centers.

The concept can be extended significantly. Meta-data about entries can include information regarding:

Original publication dates
Names of authors
Copyright attributions
The date of the original entry
Any subsequently dates when it was accessed or modified
Parties that accessed or modified the data
Analytics methods that were used to process the data

In other words, the lineage functions as a pedigree that allows anyone looking at it to evaluate where it came from and how it got where it is today.

Why Does This Matter?

Within the context of business intelligence, there will always be questions about the inputs that went into a final product. Individual data points can be reviewed to discover problems with processes or to show how transformations occurred. This allows folks to:

Perform quality control on both the data and analytics techniques
Explain how particular insights were arrived at
Consider alternative approaches
Refine techniques
Mine older sources of data using new technologies

When someone wants to pull a specific anecdote from the data, the lineage allows them to get very granular, too. In the NBA of 2019, for example, shot location data is used to study players, set defenses and even choose when and where to shoot. If a coach wants to cite an example, they can look through the lineage for a shot in order to find film to pull up.

The same logic applies in many business use cases. An insurance company may be trying to find ways to deal with specific kinds of claims. No amount of data in the world is going to have the narrative power of a particular anecdote. In presenting insights, data scientists can enhance their presentations by honing in on a handful of data points that really highlight the ideas they’re trying to convey. This might include:

Providing quotes from adjuster’s reports
Comparing specifics of an incident to more generalized insights
Showing how the numbers align
Talking about what still needs to be studied

Data governance is also becoming a bigger deal with each passing year. Questions about privacy and anonymization can be answered based on the lineage of a company’s data. Knowing what the entire life cycle of a piece of information is ultimately enhances trust both within an organization and with the larger public.

Cost savings may be discovered along the way, too. Verification can be sped up by having a good lineage already available. Errors like duplication are more likely to be discovered and to be found sooner, ultimately improving both the quality and speed of a process. If a data set is outdated, it will be more evident based on its lineage.

The How

Talking about data lineage in the abstract is one thing. Implementing sensible and practical policies is another.

Just as data analytics demands a number of particular cultural changes within an organization, caring about lineage takes that one step further. It entails being able to:

Document where all the company’s data came from
Account for who has used it and how
Explain why certain use cases were explored
Vouch for the governance of the data with a high level of confidence

At a technical level, databases have to be configured to make tracking lineage possible. Data architecture takes on new meaning under these circumstances, and systems have to be designed from the start with lineage in mind. This can often be a major undertaking when confronting banks of older data. If it’s implemented in the acquisition and use of new data, though, it can save a ton of headaches.

Conclusion

Tracking the lineage of a company’s data allows it to handle a wide array of tasks more professionally and competently. This is especially the case when pulling data from outsides sources, particularly when paying for third-party data. Not only is caring about lineage the right thing to do, but it also has a strong business case to back it up.

Your Company Needs to Be Data Driven, Here’s Why

Post author By Scottie Todd
Post date April 15, 2019

Data driven decision making is an increasingly important part of the modern business landscape. Amazingly enough, 58% of business leaders who responded to one survey said that the majority of their decisions are still based on gut feelings or experience rather than data. While the human element will never be eliminated from the process of making decisions, there’s a strong argument for an organization focusing on developing a data driven attitude.

Data Driven Decision Making Frequently Fixes Biases

For most industries, making money is a question of discovering what hasn’t yet been exploited by other companies. Spotting and exploiting these sorts of inefficiencies allows firms to gain first-mover advantages.

The folks who run call centers at Xerox Services turned to big data to reassess how they pick job candidates for interviews. The initial proposed solution based on the data left some managers downright shocked. In some cases, the system was actually sending in people with no relevant prior experience. It also singled out individuals who were on four or more social networks to not be sent in. As the program moved forward, though, attrition rates for new hires dropped 20%.

How did this happen? Data driven decision making often moves companies past human biases. Human hiring managers frequently look for signals that feel relevant but aren’t. The machines cut out all the noise of human interaction, focusing on results rather than imputing biases.

The Data Analytics Arms Race

In some industries, building out data analytics capabilities is well on its way to being an arms race between companies. The NBA has been revolutionized by analytics, with the league utilizing technologies derived from missile-tracking systems to keep tabs on every footstep and dribble made in each game. A league that was once dominated by the slam dunk rapidly switched to 3-pt shooting, and the Golden State Warriors are widely considered the first champion built on hard data. Other teams have since been racing to catch up.

On Wall Street, companies that use programmatic trading and algorithms are considered dinosaurs stuck in the 1980s. Private equity has long since moved beyond learning from the past and is now focused on predictive data analytics. One high-frequency trading firm posted a profit in 1237 out of 1238 trading days. It’s easy to see why “data scientist” is the hottest job trend in finance.

Data Driven Marketing

Some sectors have found the concurrent rise of social media and big data to be the confluence of events they required to get out in front of the competition. For large corporations, this has allowed them to target niches that were often unreachable. If you’ve walked through the grocery store and read the packages, there’s a good chance you’ve seen data driven marketing in action. Brands like Betty Crocker and General Mills now frequently emphasize niche selling points such as “non-GMO” and “gluten-free.” These selling propositions were designed by sifting through social media data to find what concerns drove consumer decisions. The brands then adjusted their marketing to have appeal to both the general public and niche markets, allowing them to maximize their exposure without making massive investments in advertising. Instead, they changed a few things on their packages.

Cutting Costs

The difference between a profitable year and a bad one often boils down to nothing more than costs. Nearly 50% of Fortune 1000 firms say they’ve started data driven initiatives to cut expenses and seen a return on the investment.

In the fashion world, using big data to track trends has become a key part of the purchasing process. No one wants to be sitting on inventory because they made the wrong buy or bought at the wrong moment. Timing this out can be challenging, too, as most retailers depend on global supply chains to bring purchased inventory from overseas to target markets. By monitoring social media trends, for example, a fashion retailer can send real-time data to a buyer in Bangladesh informing them of what styles are trending and how strongly. That can be distilled to data that enables a buyer on the other side of the planet to determine everything from purchasing volume to shipping method.

Becoming a Data Driven Operation

It’s not enough to want your company become a data driven organization. You need to lay out a plan that gets you there. This includes:

Fostering a culture that values data
Putting standards in place
Hiring professionals with big data skills
Educating stakeholders about the advantage of driving decisions with data
Building out the necessary infrastructure, particularly computer servers
Adjusting hiring practices to incorporate big data skills
Opening up the discussion to all parties from top to bottom

The move to a data-centric worldview also means getting tough about things. Companies often end up using severance packages to ship out folks who refuse to get on board with the changes. This requires a hard look at why certain people are employed and whether they can adjust to the new reality.

Ultimately, a data driven approach is about competitiveness. Other companies are already doing it and succeeding. The sooner your operation becomes one that values data, the sooner it can attract the right candidates for jobs and become more competitive.