Data Quality Services(DQS)

  • 10:24 PM
  • 0 comments
What is Data Quality Services(DQS)


DQS is a new feature in SQL Server "Denali" that provides us with a knowledge-driven data cleansing solution.

We deal with data and there is a high possibility that the data will be incorrect;may be the user has enter wrong data, the data transmission is corrupted etc. Also integrating those inconsistent data from various sources is also a problem and time consuming that will affect the business and the worst victims in such situations are the customers which on the other hand will affect the company from loss of credibility and revenue, customer dissatisfaction etc. Manual processing is not again a good choice as it is always inclined to have erroneous report however meticulously it has been measure. Automation will not give 100% result. Ultimately it will hamper the historical data and as a consequence will affect data analysis and reporting.

In order to rescue the business from such situations, DQS is the need of the day. It ensures high quality data,improves accuracy,data consistency and resolve problems cause by bad data entry in BI or data warehouse or OLTP systems.

It helps business user or a non database professional to create, maintain and execute their organization’s data quality operations with minimal setup or preparation time and with excellent quality.

It improves the data quality by creating a Knowledge Base(KB) about the data and then clean the data based on the knowledge in the knowledge base.

A Knowledge Base(KB) is a repository of three types of knowledge:

1. Out of the box knowledge

2. Knowledge generated by the DQS server

3. Knowledge generated by the user

This knowledge base once build , can server as a reusable stuff which can be continuously improve and then apply in multiple data-quality improvement processes.

KB identifies the incorrect data,proposes changes to the data,find data matches, mismatches and enables to perform data deduplication.It compares source data with the reference data(cloud-based) provided by data quality providers.

The data steward or IT professional verifies both the knowledge in the knowledge base and the changes to be made to the data, and executes the cleansing, deduplication, and reference data services.

The knowledge in the database stores all kind of knowledges applicable to a specific data source.They are stored in data domains which can even span and each of which is a semantic representation of a type of data in a data field.

A domain contains all kind of values like valid values, invalid values, and erroneous data.

Domain Knowledge(DK) consists of synonym associations, term relationships, validation and business rules,and matching policies. Eqipped with this KB, the IT professional/data steward/non-database user can makes a concrete decision about the data analysis and provides support as to whether to correct those data or not.

The DQS knowledge-driven solution uses two fundamental steps to cleanse data:

1. Builds a Knowledge base through the knowledge management process

2. Changes to be done (if needed for the Knowledge symantics to satisfy) in teh source data based on the knowledge in the KB.This is done through a data quality project.

What are the DQS Components

DOS consists of the

1. DQS Server.

2. DQS Client.

In addition to these, there is a transformation component call as DQS Cleansing Component in SSIS transformation toolbox that helps us in cleansing the data based on the knowledge base created.

The client and the server component are install as part of SQL Server setup program.(Denali CTP3)

DQS Server

The DQS server is implemented as two SQL Server catalogs that we can manage and monitor in the SQL Server Management Studio.They are

1. DQS_MAIN

2. DQS_PROJECT

DQS_MAIN includes DQS stored procedures, the DQS engine, and published knowledge bases.

DQS_PROJECT includes data that is required for knowledge base management and DQS project activities.

The source database that contains the data to be analyzed must also be in the same SQL Server instance as the DQS server.

DQS Client

It is a standalone application, designed for data stewards and DQS administrators that helps to perform knowledge management,domain management, matching policy creation, data cleansing, matching, monitoring, data quality projects, and administration in one user interface.

The client application can installed and run on the same computer as the DQS Server or remotely on a separate computer.



0 comments:

Post a Comment

 

Copyright © 2010 SQL Cached, All Rights Reserved. Design by DZignine