In recent years, there has been tremendous growth in the data
warehousing market. Despite the sophistication and maturity of
conventional database technologies, the ever-increasing size of
corporate databases, coupled with the emergence of the new global
Internet "database", suggests that new computing models may soon
be required to fully support many crucial data management tasks.
In particular, the exploitation of parallel algorithms and
architectures holds considerable promise, given their inherent
capacity for both concurrent computation and data access.

We are interested in the design parallel data mining and OLAP algorithms and their implementation on coarse grained parallel multicomputers and PC-based clusters. To date we have been focusing on parallelization of Data cube methods. Data cube queries represent an important class of On-Line Analytical Processing (OLAP) queries in decision support systems. The precomputation of the different group-bys of a data cube (i.e., the forming of aggregates for every combination of GROUP BY attributes) is critical to improving the response time of the queries [Gray et. al. 1997]. The resulting data structures can then be used to dramatically accelerate visualization and query tasks associated with large information sets.

We have developed a parallel Data Cube system using C++ and MPI for communications.  This software can generate both complete and partial data cubes and we are working on a parallel query engine.  Our generation methods are based on a parallel version of Pipesort and have been extensively tested on both PC-based clusters and on a SUN Sunfire parallel multicomputer.

This project represents ongoing joint work with a number of researchers including Frank Dehne (Carleton, Canada) and Susanne Hambrusch (Purdue, USA). Todd Eavis, a Ph.D. student at Dalhousie has also played an major role.

The following Undergraduate and Masters  students in Computer Science have been involved in the implementation effort: Zimin Chen, Steven Blimkie, Khoi Manh Nguyen, Thomas Pehle, and Suganthan Sivagnanasundaram.

Publications