GBase Insights: The Extended Analytics Database

I. Multi-Model Databases

As big data applications evolve, enterprises face explosive growth in data volumes alongside increasingly flexible and diverse data structures. Traditional database management systems built on relational theory are encountering significant challenges. To meet these evolving demands, different business IT systems require different types of databases for support. The following illustrates various database types and their applicable scenarios.

While using different databases for different applications can deliver targeted data support, it also increases database maintenance complexity and data exchange difficulties. Consequently, multi-model databases have garnered significant attention. A multi-model database manages data across different models—such as relational, tree, graph, and object models. Equipped with multiple database engines, it can simultaneously satisfy an application's unified management needs for structured, semi-structured, and unstructured data. Multi-model data management enables unified, cross-departmental, and cross-business data governance, facilitating multi-business data fusion to support diverse application services.

Several multi-model database products exist today. One of the more well-known is ArangoDB. In ArangoDB, data can be stored as documents, key/value pairs, or graphs. A single declarative query language can access any or all of this data, and different models can even be combined within a single query. Furthermore, its multi-model style allows developers to build lean applications that can scale horizontally using any or all of the supported data models.

Although a multi-model database can support diverse business applications in a unified manner, it faces significant challenges. Because different database engines vary greatly in response latency, computational storage, and the semantics and syntax of operational languages, integration often prevents each engine from reaching its full potential, imposing constraints on overall performance. As the saying goes, let professionals do professional work; similarly, it makes sense to let specialized databases support specialized business applications. A general-purpose database may be capable of supporting a wide range of operations, but in specific, niche business areas, its capabilities often fall short of what a specialized database can deliver, leading to suboptimal overall performance.

II. HTAP (Hybrid Transactional/Analytical Processing) Databases

Compared to multi-model databases, HTAP databases focus on blending relational transactional (OLTP) and relational analytical (OLAP) processing. HTAP enables large-scale, real-time applications to simultaneously support big data analytics and mining. Typically, HTAP can be achieved through two systems—one for OLTP and one for OLAP—or through a single, unified system. The former requires the application to coordinate the use of AP and TP systems itself, with data synchronized via ETL. The latter, based on a unified architecture, handles both transactional and analytical requests simultaneously. This not only eliminates the need to extract, transform, and load data from the relational transactional database to the analytical one, but also supports real-time analysis of the latest transaction data. A well-designed HTAP database can support both transaction processing and data analytics concurrently, avoiding the heavy data interaction between online and offline databases inherent in traditional architectures.

The following illustrates a typical HTAP system architecture:

While HTAP integrates key features of both OLTP and OLAP, it still faces several challenges that require further development.

Its main challenges include:

1. Most HTAP solutions can handle AP and TP requests separately but lack support for executing AP scenarios within a TP context.

2. Most systems require combining various solutions to meet the demands of HTAP scenarios.

3. To accelerate TP updates and point queries, HTAP systems often place all indexes in memory. However, for larger-scale data scenarios, having all indexes in memory can slow down the TP system.

4. Storage engines designed for AP scenarios typically use object storage or shared file systems. These storage formats are primarily optimized for scan operations and cannot provide efficient point-query and update capabilities.

Although HTAP cannot fully match the individual performance of dedicated OLTP or OLAP systems, the two typically share many commonalities—both are relational databases supporting standard SQL, with no fundamental difference in table structures. Therefore, overall database performance can be effectively optimized. To a significant extent, HTAP can leverage the strengths of both OLTP and OLAP while simplifying system maintenance costs.

III. The Extended Analytics Database

The OLTP and OLAP databases within an HTAP system share many commonalities, meaning HTAP not only meets specific application needs but also incurs less performance loss compared to multi-model databases. HTAP integrates two database types from the dimension of relational databases. Similarly, from an analytics dimension, analytical databases (OLAP) and graph databases (RDF) form a complementary pair that can meet higher-level data analysis requirements. OLAP and RDF databases also share commonalities: both are analysis-oriented, handle large data volumes, and their analytical capabilities complement each other. Combined, they can provide richer and more efficient data mining. We refer to the integration of OLAP relational analysis and RDF graph analysis as the Extended Analytics Database.

Big data analysis and mining have long relied on OLAP relational databases. Data warehousing technology based on OLAP databases has been crucial for data analysis across industries for many years. However, OLAP relational databases excel at row-and-column data computation and storage, but their ability to traverse relationship networks and extract information is comparatively weak. When data scales are massive, relational databases struggle with multi-layer association analysis; their join operations often fail due to excessive time consumption. Graph databases are perfectly suited to compensate for this weakness. They naturally express real-world entities and their relationships, requiring no time- and memory-consuming join operations and maintaining constant time complexity. Graph databases hold a significant advantage over relational databases in multi-level associative queries.

To achieve more powerful data analysis and processing, two architectural approaches can be adopted:

A separated architecture, where services are routed to one of two different database types based on their respective strengths, with synchronization required between the two systems. A unified architecture employs an engine adapted for both relational table analysis and association analysis, providing unified interpretation, scheduling, and optimization for applications to deliver integrated data analysis services. The specific structure of a unified Extended Analytics Database is illustrated below:

Since both OLAP and RDF engines are capable of large-scale, massive data processing, such an integrated analytical database leverages the analytical strengths of both OLAP and RDF, further enhancing data analysis capabilities.

IV. GBase Extended Analytics Database

GBase's graph database offering leverages its R&D integration platform UP, the analytical database GBase 8a technology, and China-developed graph database techniques to build an extended analytics data platform. By harnessing the strengths of these three technologies, it not only meets ultra-large-scale data analysis needs but also achieves breakthroughs in association analysis performance, further enabling big data mining and analysis for complex, multi-level association knowledge graphs. The Extended Analytics Database will provide enhanced application in the following scenarios:

Financial Risk Control and Auditing

Financial risk control analysis involves numerous indicators, massive data volumes, and complex, multi-layered entity associations. For instance, during audit analysis, customer relationships involve corporate, personal, and employee management across a large number and variety of dimensions. Traditional databases struggle to support this. An Extended Analytics Database can efficiently analyze high-value activity, such as frequent transfers between key customers or employees and their relatives or deep financial transactions.

Cognitive Computing Platforms for Petroleum Exploration

The "well logging" phase of petroleum exploration involves massive datasets. Simultaneously, geophysicists need to identify oil and gas layers by studying comprehensive data like resistivity, spontaneous potential, and acoustic waves. In oil and gas production, IoT technology and machine learning methods are used for quantitative diagnosis of oil well conditions and remote, real-time online management.

Through extended data analysis, massive data and machine learning can be mined to discover patterns in oil and gas layers, while knowledge graphs enable deeper exploration of complex associations, providing a basis for scientific exploration.

Power Grid Flow Calculation

Power grid flow calculations describe data for generators, loads, lines, and transformers. Using an Extended Analytics Database, not only can the massive data involved in power flow be computed and stored, but knowledge graphs can also rapidly update complex power grid topologies, identify fault paths, and provide technical support for rapid fault recovery in smart grids.

V. Conclusion

The evolution of big data technologies and applications places higher demands on analytical databases. Traditional analytical databases have proven their significant capabilities in data warehousing, but they lack the ability to handle multi-level data association. Graph databases, born for data association analysis, excel precisely at this multi-level associative analysis. The Extended Analytics Database therefore integrates the strengths of traditional analytical databases and graph databases, greatly enhancing analytical capabilities to better meet the growing needs of big data business.