If you regularly read technology news, you’re likely well aware of big data. In fact, everyone from The Wall Street Journal to ITworld is hailing 2012 as the year of big data. While there’s no denying the trend, there are several unsolved challenges facing most enterprises. It’s not about how much data you have; it’s how well you manage that data. That’s why this year is the year of master data management. The goal behind [...]

The volume of big data is such a change, that it’s a change in the kind of how data is managed fundamentally. It’s like running a startup where every day the number of employees grow exponentially. Until a day-before you had 5 developers, Today you have 500. Big volume isn’t even the big story. The big story is the variety of data and its speed of change. This is like running a company where the [...]

Over the past few years, an increasing number of organizations are making BI and analytics functionality more broadly available to all decision makers inside and outside the organization. Internally, more widely available BI solutions lead to greater accountability by all employees and greater consistency in performance management. Externally, relationships with vendors and partners can be strengthened through effective sharing of key performance indicators (KPIs). However, having widely available BI means more than having the appropriate [...]

Factless fact tables can simplify the overall design. Think of a property and casualty insurance company providing automobile coverages. It’s reasonable to create a transaction fact table that captures the written premiums resulting from a new sale or change to an existing policy. Likewise, it makes perfect sense to implement a monthly snapshot fact table to capture the earned premium associated with every policy by customer, by named insured, by household, by vehicle, by driver [...]

Although the Type 1 and 2 SCDs are the primary workhorse techniques for responding to changes in a dimension, we need a third technique for handling alternate realities. Unlike physical attributes that can only have one value at a point in time, some user-assigned attributes can legitimately have more than one assigned value depending on the observer’s point of view. For example, a product category can have more than one interpretation. In a stationery store, [...]

The Type 2 SCD requires that we issue a new employee record for Ram effective July 12, 2011. This has many interesting side effects: Type 2 requires that we generalize the primary key of the Employee dimension. If Ram’s employee natural key is RM12345, then that natural key will be the “glue” that holds Ram’s multiple records together. It is not recommend creating a smart primary key for Type 2 SCDs that contains the literal [...]

Suppose we are notified that the Home City field dimension has changed from Jersey City to New York as of today. Furthermore, we are advised that this is an error correction, not an actual change of location. In this case, we may decide to overwrite the Home City field with the new value. This is a classic Type 1 change. Type 1 changes are appropriate for correcting errors and for situations where a conscious choice [...]