Blog Archives

The data security paradox

Data gains value when it’s available to the right audience. The greater the audience, the greater the potential for unlocking value in the data. However, it is often the most sensitive data that is also the most valuable. Customer records, transaction histories, and medical information are a rich source for data mining. These datasets are also closely guarded, and their confidentiality might be legally protected or their access restricted by corporate policy. The reasons are clear: a data break can be disastrous for a company and its customers. This is the data security paradox: the more valuable the data, the harder it is to obtain the data.

WHY IT MATTERS

First and foremost, the data security paradox affects the owners of the data. A bank department might want to analyze transaction data in a public cloud, like Azure or AWS, but are restricted from doing so. Or the bank department might want an outside analyst to look for money laundering activities, but are afraid the data might be leaked to the Internet.
Second, the data have value to an audience outside the data owner’s organization and its trusted partners. Prevailing wisdom is that data can not be safely shared with the public. “Bad actors” could use the data maliciously or inappropriately. However, this audience also contains researchers, non-profits, NGOs, think tanks, and altruistically minded individuals can use the data to discover new insights, advance everyone’s understanding of a subject, and create new value. A McKinsey Global Institute reports that “open data can help unlock $3 trillion to $5 trillion in economic value annually…”

SAFELY SHARING DATA

Organizations share data, even sensitive data. Smart organizations lower their risk by anonymizing or obfuscating sensitive data, especially personally identifiable information (PII). PII includes identifying information like name, phone number, or date and place of birth. PII extends to anything that can identify who the person is, including medical health history or IP address. Getting data in a state where it is ready to share means removing PII and anything that might be damaging in the context of the data. This is sometimes called “sanitizing” the data.
A common approach to sanitizing is to remove all sensitive attributes of the data before sharing it. This approach has two potential problems. First, the sensitive fields are often the primary keys of the data. Once these keys are gone, the ability to connect sanitized records with the original data is lost. This can be important. For example, if a third party sanitized bank transactions to find money laundering, the data owners will ultimately want to know which customers are implicated. If all PII is simply excised, this could be difficult or impossible to do.
The second problem is that sensitive data can be intrinsically valuable. Sometimes this loss is unavoidable. For example, removing a customer’s name from anti-money laundering data before sharing it would preclude an outside analyst researching that customer’s legal filings. However, sometimes the value of the information can be retained even if the data is sanitized. To continue with the banking example, the transfer of money between customers can reveal important patterns, even if the names of the customers are not known.

LIFTING THE BURDEN OF SAFELY SHARING DATA

The average analyst does not have the tools or experience to sanitize data. We have seen clients mitigate the problem by either never sharing data or sharing data after signing NDAs. Sometimes, the data are shared with nothing more than the hope that they are not compromised.
We set out to lift the burden of sanitizing data so they can be safely shared. We wanted to create an application that data analysts could use to remove sensitive information so that data could be shared with someone in another group at the same organization, an outside business partner, or even industry analysts and researchers. Important requirements for the solution included:
Spreadsheets: Most of the data our clients want to share is in Excel spreadsheets. Even if the data originates in a RDBMS, the data is typically exported as a flat file before being shared.
Accessible: Many data analysts work in secure environments where they cannot install applications on their computers. None of our potential users wanted to install an application and many would not trust an executable file even if they could install applications.
Simple: Complexity is a common source of security weaknesses. Simple systems are easy to analyze and their behavior is more predictable and less likely to contain bugs.
Isolated: The solution had to run entirely on the user’s computer with no need to access the network. Security becomes much more difficult once data is transmitted over a network.
Irreversible: Data must be sanitized in such a way that it was impossible to recover sensitive information after it was anonymized or obfuscated.
Retain relationships: Relationship mining is an important part of many analyses. The solution had to preserve relationships in the original data.
Two-way: In our use cases, sanitizing data to make it ready to be shared is only half the equation. Once the data is shared, another party typically analyzes it and enhances it by adding additional data attributes. For example, an outside analyst might work with shared customer data to create a risk score for each customer. The analyst then provides that information back to the data owner. It should be easy for the data owner to join the enhanced data with the original, sensitive data.
The Alacer Data Cloak solution To meet our requirements, we created a browser-based application that transforms a file with sensitive data into a public file and a private file. The shareable public file contains sanitized information. It is safe to share with outside parties. The private file contains the sensitive information and stays with the data owner. The private file can be used to re-integrate (uncloak) analysis provided by an outside party.
Brower-based application: The Data Cloak runs entirely in a browser. This makes the application very accessible; there is no need to install or start an executable. Browsers are installed on almost every workstation. Browsers also allow the application to run transparently on different operating systems (e.g. Windows, Mac, Linux).
Comma-separated value files: The Data Cloak operates on comma separated value (CSV) files. All spreadsheet applications can save their output as CSV files. Spreadsheet programs can also import CSV files easily. CSV files are not specific to an operating system, browser or other technology.
User interaction: The idea behind Data Cloak is simple. The Data Cloak scans an input file and presents the user with a simple list of choices for each attribute in the data. The user easily selects one of three actions for each attribute:

• Declare the attribute is safe to remain in the shared data.
• Declare the attribute as sensitive and should be cloaked.
• Declare the attribute as sensitive and remove from the shared data.

Local processing: No data, not even sanitized data, are transmitted to another computer. The output is generated and saved to the user’s local storage files. The sanitized public file can then be safely emailed to the intended recipient.
Random generation: Sanitized values are randomly generated. The sensitive data is not used as part of the sanitizing process. There is no way to divine the original data from their sanitized replacement values.
Global substitution: Sensitive values are always replaced by the same cloaked value to preserve relationships between entities.
Uncloak application: A second browser-based application, Alacer Data Uncloak, recombines the public and private files to uncloak or restore the data.

Start cloaking your sensitive data now

Alacer’s Data Cloak and Uncloak applications are simple, security tested and affordable. They streamline managing data files for any organization and provide a new standard of protection for sharing sensitive data.
The new Alacer Data Cloak

• Run off-line using existing browser software
• Secure sensitive data with reversible cloaking
• Powerful controls in easy-to-use interface
• Share data freely without risk

EXECUTIVE SUMMARY

In the increasingly competitive mobile telecommunications industry, retaining customers is a challenge that Tier 1 carriers are aggressively rising to meet. While many marketing efforts to combat churn are still reactive, the Alacer Group recently developed a model for an American telecommunications company that outlined proactive approaches for contacting customers before they decide to change service providers.
Looking at more than 70 different data points on over 70,000 customers, we created a binary logistic regression that could predict with 60% accuracy whether or not a customer will churn. We then considered the Probability Factor, and calculated each customer’s value to the carrier to create individual Profit-Churn scores. The results determined where each customer fit into a four-quadrant matrix, which gave the service provider a road map for determining the best actions for each audience.

A PERVASIVE PROBLEM: CUSTOMER CHURN

The American mobile telecommunications industry has matured from its early days of explosive growth to a saturated and fiercely competitive market where carriers are battling to simply retain existing customers. The current buzzword within the industry is customer churn, a term used to describe customers who voluntarily unsubscribe to join a competitive carrier. It’s become so pervasive that the industry now has one of the highest customer churn rates in business; as an example, in August 2012, T-Mobile’s churn rate was 2.10 percent, which may seem minuscule until you factor in its millions of subscribers. As a result, many companies are redirecting vast sums of their marketing dollars from developing customer acquisition programs to devising churn management strategies. In fact, the analysts at Gartner Research recently reported that the marketing budget allocated to retaining customers and increasing loyalty through customer service will more than double by 2015.
Most often, service providers are reactionary in their efforts to address voluntary churn. But what if unstructured data could be used to develop a model that could identify the most lucrative customers and help develop a proactive program for ensuring their loyalty? Big data offers just this opportunity.
With this in mind, the Alacer Group recently worked with a Tier 1 service provider to leverage its customer data, which could then be used to create a proactive churn management program. Using 75 relevant data points for each of approximately 70,000 customers, 28.8% of which had churned, we could design a useful churn prediction model for restructuring marketing operations.

DEVELOPING A CHURN SCORING MODEL

In order to predict the likelihood of a customer churning, it is important to understand what variables may cause a customer to leave in the first place. To do this, we created a Churn Scoring Model using binary logistic regression.
We began by dividing our data set into estimation and validation samples;
the first was to be used to create a predictive model, with the validation sample as a test of the model’s performance. Next, a logistic regression was created using almost all of the independent variables as inputs and the binary churn variable as the output. If done manually, it would take a long time to test and eliminate the many independent variables that are either not statistically significant or that have high correlations with other independent variables used in the model. Fortunately, IBM’s SPSS statistical software has a feature that automates much of the variable selection process.
With most of the 75 independent variables inputted into a logistic regression, we ran an analysis using the Backward: Conditional method. This analyzes all variables and then eliminates them one-by-one until a desired model is achieved.
With this regression model, we identified 36 of the independent variables that are statistically significant for predicting customer churn; however, at this point the model is not yet complete. In its current format, it only gives us a t score.
The t score is not very helpful in itself, but it can be converted into a Bernoulli Probability by applying this formula giving us our scoring model.
With the scoring model complete, we can apply it to our validation sample and evaluate its predictive performance. We exported the t score model using the Propensity to Purchase option in the Direct Marketing module in SPSS, and applied this to our validation sample using SPSS’s Apply Scores from Model feature. As a result, each customer received a Predicted Value and a Selected Probability (that y=1). We then grouped the Selected Probability values into seven different bins.
We noted that more than 1,000 customers were not given a churn score. Initially, we used the age of the head of household as a variable factor; however, more than 20,000 customers were shown to either not have a listed age (hence no score), or an age of 0 (impossible), so age was not a useful variable.
After removing age as a variable factor, we ran the Backwards: Conditional binary logistic regression again. We also computed a new “accuracy test” variable, which was the sum of the value of the actual churn result (0 or 1) and two times the churn value predicted by our model (0 or 2).
Values of 0 = the model correctly predicted someone would not churn (42.7%)
Values of 1 = the model incorrectly predicted someone would not churn (12.4%)
Values of 2 = the model incorrectly predicted someone would churn (28.4%)
Values of 3 = the model correctly predicted that someone would churn (16.4%)
Overall, the model was accurate 59.1% of the time in predicting whether or not a customer would churn (42.7 + 16.4 = 59.1). We then decided that the profitability of each customer was important to consider as well.

THE PROFITABILITY FACTOR

Sprint CEO Dan Hesse recently confirmed the importance of the Profitability Factor. In his April 2013 conference call with shareholders, he explained that mobile operators can’t rely on adding new high-value contract subscribers to boost their bottom lines. Instead, companies will need to focus on retaining the customers they have, while encouraging individual subscribers to spend more.
That said, the goal for any voluntary churn reduction program is revenue stabilization and augmentation. Looking at a subscriber’s profitability is arguably a better model for developing a targeted marketing program than utilizing overall subscription numbers as a benchmark. Not all subscribers are equal in value; in fact, based on cost vs. profitability, there are some customers that it might be wiser for a service provider to proactively lose from a revenue standpoint.
By considering the profitability of each of customer in addition to his or her likeliness to churn, we can determine if some churn is desirable, identify high value customers who should be proactively targeted to remain loyal, and develop an overall action plan.
In order to analyze the data in a way that provides insight into customer profitability, we used the customer lifetime value (CLV) approach to perform an annual profit & loss analysis for each subscriber. Because the data is missing actual costs, we made the following assumptions:

• Monthly COGs = 40% of monthly revenue
• Monthly Directas Cost/Call = $0.50
  Director assisted call representative wage = $10/hr
  Director assisted call representative productivity = 20 calls/hr
• Monthly Custcare Cost/Call = $1
  Customer care service representative wage = $10/hr
  Customer care service representative productivity = 10 calls/hr
• Monthly Retcalls Cost = $2.50 * (# of Calls / # of Months in Service)
  Retention team call representative wage = $15/hr
  Retention team call representative productivity = 6 calls/hr
• Monthly Phone Cost = $12.50 * # of Phones
  Average phone cost to company = $300
  Average phone life = 24 months

We then took the profitability for each customer and discounted it using a hurdle rate of 10% (we assumed an annual cost of capital of 10%). Using the calculated annual value for each customer and each customer’s credit rating, we calculated each customer’s ongoing monthly expected profit by using the formula: Monthly Expected Profit = (Monthly Revenue – Monthly Cost) – (Monthly Cost * Default Rate)
With each customer’s Monthly Expected Profit calculation, we could now compare an individual’s profitability. We were astounded to learn that while 35.37% of the company’s customers are more profitable than the average customer, these customers generate more than half (61.52%) of the cumulative Monthly Expected Profit.
To better visualize the distribution of profitability, we grouped customers into seven bins according to each customer’s expected profit. We observed that 10% of customers are significantly more profitable than the average customer (>1 standard deviation, bins 5-7) and 737 customers appear to be unprofitable (bin 0).

RECOMMENDATIONS

By melding the Churn Model with the Profitability Factor, the service provider gets a useful framework for making more informed marketing decisions, and can concentrate retention efforts on those highly profitable customers who are most likely to churn. These are the customers the carrier cannot afford to lose. Unprofitable subscribers who are likely to churn can either be let go as customers or can be enticed to move to a profitable status.
Using the seven bins from the Churn and Profitability scores, each customer’s combined score is used to assign him or her to a quadrant. For example, a customer with a Profitability score of 7 and a Churn score of 5 gets a combined score of 75. Each two-digit score can be read as a set of X,Y coordinates.

The quadrant boundaries were created from the company data set at break points driven by business needs. The Churn axis breaks between 2 and 3 where there is a 50% or more probability of churn. The Profitability axis breaks between 3 and 4 where the mean profitability was calculated. Figure 7 illustrates the breakdown of the quadrants and the customer distribution.
Within each quadrant, marketing efforts would be prioritized based on Expected Churn Loss:
Expected Churn Loss = Monthly Expected Profit * Probability of Churn
As an example, this calculation could help determine if a Profit7-Churn6 individual should be addressed before a Profit6-Churn7 customer.
The actionable predictors of churn can now be used to develop specific marketing actions appropriate to each customer. Here are a couple of ways churn predictors could help determine how the company could retain a valued customer:

• Subscriber has called customer service. The customer service team would know (via integrated CRM) that this customer is a priority; likewise, during workload management for callbacks, profitable customers who are likely to churn would be at the top of the work queues.
• Higher unique subscribers: This would indicate that the customer does a lot of business and will regularly shop for volume discounts. The service provider could discount additional lines and offer to consolidate any lines the subscriber has with competitive carriers.
• Children in household: The carrier could offer discounted family plans to encourage children to join the plan instead of choosing a competing offer.
• High credit rating: Customers with high credit ratings can easily leave and are not locked into a plan. The service provider can offer early discounts to extend a 24-month commitment, a few months prior to plan expiration.

In conclusion, while Churn and Profitability/CLV models are not new, this unique quantitative approach of combining churn with the Profitability Factor is a new twist. Through a simple scoring model similar to RFM, service provider marketers can now prioritize their retention actions for the best customers.