Data Warehousing Implementations from Readers

"The latest system I put together gathers information from some 60 locations around the world from a variety of source database systems. My customer is in the mining business and owns mines on each continent.

The source operating systems are also a variety of operating systems, including Windows, Linux and flavors of Unix. The source database systems vary from flat files to Oracle and SQL Server.

I wrote a JAVA application to gather the information using JDBC so that it would be easier to deploy.

The destination system is a SQL Server and the data arrives by using the Service Broker using XML. A series of transform TSQL routines take the XML and store it on a local database.

We then have designed a series of interlocking cubes to render the information to reports. Those are then delivered to the users through ProClarity, because of the need for sophisticated AdHoc reports.

Its quite a system." – Indeed.

… and this from LC:

"[H]ere is our BI Team’s approach to handling BI data.

Archiving is important since the amount of data we deal with goes beyond 10 TB in one year. We retain current and previous year’s data on one server (Production server for current data is beefier than the server where data is archived). The data from two to four years prior is archived to another server (may be archived to a section of the SAN), and the data prior to 4 years – possibly to 7 years (dependent on the type of data and based on what our Records Retention Dept. recommends) are archived to DVD (burned, labeled, and stored off-site).

We are in the process of migrating our Production. Development, and Test servers to Blade Servers – 64-bit environment, and creating a ‘Virtual’ environment for Development. This would give us more resources for conducting proof of concepts (POC) for future applications needed and accommodate our expanding BI environment.

Basically, here’s what we are considering to accommodate our BI environment (this configuration does not include the separate SAN storage we are planning to implement as well as these servers):

c-Class Chassis BL680 Configuration
7 * BL680 Blade Servers
7 * 4 Quad Core Processor Intel E7340
4 * 64 GB RAM
3 * 32 GB RAM
7 * 2 146GB Disk
14 * QLogic Mezzanine Cards
2 * Virtual Connect Ethernet Modules
4 * Virtual Connect FC Modules
7 * 512MB SA BBWC Kit
6 * Fans
10 * Power Supplies
Single Phase Power
3 Yr 6 hr CTR + Software support,
7 MSFT W2K3 Svr R2 Enterpris

I hope this gives you an idea of one approach to supporting a BI environment."