Organizations own huge amounts of data (big data) that must be stored somewhere. All that data contains valuable information that provides a lot of insight for business operations. How and where that Big Data is stored is very important. BI architecture not only helps with the storage, recording and processing of this data, but also with the analysis. And is therefore very valuable.
In this blog I will go into more detail about this. I describe the IT architecture, what BI architecture is and I explain why it is structured like this.
IT Architecture
IT Architecture describes the substantive cohesion between the business strategy, IT strategy, roles, information provision and the IT infrastructure. The models that are drawn up within the architecture describe the information involved and visualize the information flows. Mutual communication and its relationship, the cohesion, are central to this. (see Figure 1).
Figure 1: IT Architecture
Information architecture
Information architecture describes the communication of systems and data models. The architecture forms the blueprint of all (technical) information within an organization. And it is the basis with which organizations can successfully realize BI. The Information Model provides a complete picture (model) of the components mentioned above.
Figure 1 shows an example of a website information model. A well described/modelled information architecture prevents:
- insufficient data quality;
- limited integration of different systems;
- limited data deliveries;
- infrastructure not aligned with the solution;
- systems do not sufficiently support business processes;
- shadow IT.
For more info visit the following websites:
Figure 2: Example of Information Architecture
BI Architecture
Figure 2: Example of Information Architecture
BI Architecture
The BI architecture is the framework for the various technologies that an organization deploys to run BI and analytics applications. It is only a small part of the information architecture, but it is indispensable in most organizations.
It includes the IT systems and software tools used to collect, integrate, store and analyze BI data. The architecture represents all processes, roles and information flows that are necessary to realize BI within an organization. It provides information about business activities and trends that you can then present to management.
Unlike the IT and Information Architecture, which is 100% custom, you can adopt and implement existing BI Architectures.
There are several gurus who have described the BI architecture based on their vision. Those are:
- Ralph Kimball (Kimball Bus)
- Bill Immon (Corporate Information Factory )
- Dan Linstedt (Data Vault)
Below I go into more detail about each of these.
Kimball Bus
The Kimball bus is the simplest architecture for starting a Data Warehouse in a static environment. This architecture was originally developed to improve the performance of reports and to relieve the burden on source systems. It consists of two layers (see figure 3):
- Staging Area: one on one copy of the source on a separate database
- Data Mart Area: Historical, demand-oriented layer
Figure 3: Kimball Information Architecture
It is important to note that the Kimball (bus) has been fully adopted by Microsoft in their perception of a DWH. The star models used by Kimball in the Data Marts are still used by the other gurus.
Corporate Information Factory
Bill Immon has come up with a variant of the Kimball Architecture. That is, a layer between the staging and the Data Marts. This layer is called the “Operational Data Store” or historical integration layer. This layer retains the logic of the source and in this way tries to link equivalent information. The result is a data model that is suitable for deriving Data Marts. See figure 4.
Figure 4: Bill Immon Information Architecture
Data Vault
Dan Linstedt’s architecture is very similar to Bill Immon’s. Except that the data model is aligned with the terminology of the organization. The advantage of this is that the model can be tested in the organization to see if it meets the requirements.
This architecture also takes Big Data (unstructured/ semi-structured data) into account. These are stored (when it is valuable) in the “Raw Vault”. The Business Vault has been added to this to show the information from the point of view of the “Real life world”.
Figure 5: Data Vault Information Architecture
Comparison of BI/DWH architectures
The flexibility and predictability of the different architectures distinguish themselves from straightforward to flexible. The table below describes which architecture is most suitable for an Agile environment.
Table 1: Advantages and disadvantages of the different BI Architectures
Is it a match for BI?
Before you start with Big Data in your organization, it is useful to ensure that your architecture matches existing (BI) architecture principles. For BI, certain gurus have already figured out how you can set up BI generically in the different environments. Which one you choose depends on the complexity, size and variation of the delivery systems. The more the organization wants to be Agile and flexible, the more likely it is to opt for a Data Vault implementation.