Inside the Kernel | Eliminating Data Latency: How GBase 8c HTAP Enables Real-Time Reporting

In municipal digitalization systems, reporting is often the hardest nut to crack. Underlying transactions must be written in real time, upper-layer reports must produce data on demand, and business users must sift through dozens of tables to find what they need. In a municipal and public service integration project, GBase 8c used a single technology stack to support heterogeneous workloads including real-time reporting (HTAP), a data middle platform, IoT sensing, CIM, smart integrated poles, and AI large models — spanning transactional processing, batch analytics, HTAP, and spatial data queries. The system migrated smoothly from PostgreSQL, with database form precisely selected to match each business characteristic, ultimately achieving "one database, multiple capabilities, unified management." How was this architecture implemented? This article breaks it down in detail.

Three Barriers in Municipal Data

Anyone who has worked on government digitalization knows that the complexity of municipal systems often lies not in any single workload's size, but in multiplicity — many systems, many data sources, and rapidly changing requirements.

This project covered multiple business systems: real-time reporting, IoT sensing, CIM, smart integrated poles, a data middle platform, and AI large models. Business users faced three barriers when retrieving data:

First, too many tables, too heterogeneous. 

Different business systems have different underlying table structures, and field meanings are buried in technical documentation. Finding a single table can take half a day, making data retrieval extremely difficult.

Second, stringent timeliness requirements. 

IoT device data streams in continuously, and municipal decision-making needs real-time data. The traditional T+1 synchronization model simply cannot keep up with the business cadence.

Third, widely varying workloads. 

Some systems are continuous high-write, others are primarily business read/write. Forcing all of them onto a single architecture either wastes resources or falls short on performance.

In the past, completing a single report retrieval required business users to first identify the target table, then verify fields, and finally write a query statement — with enormous time spent on preparation and data value slow to materialize.

Real-Time Reporting: HTAP Lets Transactions and Analytics Work in the Same Database

Real-time reporting was the most complex workload in this project: it had to handle transactional writes while running report analytics at any time.

What is the conventional industry approach? Two systems: one dedicated to transactional writes, another dedicated to report analytics, with a synchronization pipeline moving data between them. The benefit is isolation; the drawback is equally obvious — reports always lag half a beat behind the business, and every additional system adds operational cost.

GBase 8c's answer is an HTAP architecture — transactions and analytics are not separated. A single data foundation serves both. Transactional writes and query analytics share the same data, eliminating the need to move data between databases. Within the database, an incremental merge mechanism ensures the analytical side sees the latest data. Under the traditional T+1 synchronization model, reports can never catch up with the business cadence; HTAP brings reporting in step with real-time business.

What is more interesting is the intelligent data retrieval layer built on top of the reporting platform: the platform catalogs the "instruction manuals" for all tables into the database and combines them with semantic matching technology. Business users describe their needs in plain language, and the system automatically identifies the corresponding tables, generates the query, and returns the data. What used to be "find the table, dig through documentation, write SQL" becomes "say what you need and get the data."

Behind this is GBase 8c's work at the storage level: within a single copy of data, there is both a row-store structure suited to high-concurrency transactions and a column-store structure suited to large-scale statistical aggregation — resolving the inherent conflict between transactions and analytics within one database.

No One-Size-Fits-All: Precise Pairing of Distributed and Primary-Standby

The most noteworthy aspect of this project is not that distributed deployment was used, but that distributed deployment was not used everywhere.

Across nine systems, data characteristics differed enormously:

• Two IoT systems — characterized by high device access volume, continuous data writes, and large data volumes — used distributed deployment (two shards).

• CIM platform, CIM population map, smart integrated poles, AI large models, and the data middle platform — primarily business data read/write — used primary-standby deployment.

Some may ask: if distributed is so powerful, why not use it everywhere?

Because distributed is not free — more shards mean more machines, higher cost, and more complex operations.

For read/write-dominant business systems, a primary-standby architecture is sufficiently stable and efficient; forcing distributed deployment onto them is pure resource waste. For scenarios like IoT systems with continuous high writes and large data volumes, distributed horizontal scaling is genuinely valuable.

On one side, storage mode is selected on demand (OLTP, OLAP, or HTAP); on the other, deployment form is configured on demand (distributed or centralized). There is no universal architecture — only the architecture that best fits the business.

From PostgreSQL to a Domestic Database: The Hard Work Behind a Smooth Migration

The project's source database was PostgreSQL. For domestic replacement, syntax compatibility and data consistency are two red lines.

Leveraging GBase 8c's high compatibility with PostgreSQL, the entire migration completed syntax compatibility, object migration, and data validation, transitioning smoothly. The system also provides high availability with failover when the primary node fails, safeguarding continuous operation of municipal public services.

The value of this migration extends beyond this single project: the successful migration of large-scale external-network PostgreSQL workloads also validates the domestic database's ability to carry the PostgreSQL technology stack, charting a replicable path for future domestic upgrades of more government systems.

One Foundation for Everything: The Long-Term Value of Unified Management

The project ultimately centralized management of all database instances through a unified database management platform. Monitoring, backup, and inspection are all performed on a single platform, dramatically reducing operational complexity.

More importantly, all systems now live on the same data foundation. Data from IoT sensing, CIM, smart integrated poles, and AI applications is no longer isolated in separate silos — it is now a cross-system, interoperable, collaborative data asset.

From HTAP real-time data retrieval in reporting, to distributed high-write capability in IoT systems, to smooth PostgreSQL migration, every step of the project's implementation is backed by GBase 8c's capabilities — one technology stack, with storage mode and deployment form fully adapted to the business.

The highest level of database selection is not "it can run everything," but "every workload runs just right." That is the underlying logic we wanted to share in this installment of Inside the Kernel.

FAQ

Why not keep the conventional two-system split of a transaction database plus an analytics database?

The conventional industry approach is two systems: one dedicated to transactional writes, another dedicated to report analytics, with a synchronization pipeline moving data between them. The benefit is isolation; the drawback is equally obvious — reports always lag half a beat behind the business, and every additional system adds operational cost. GBase 8c's answer is an HTAP architecture: transactions and analytics are not separated, a single data foundation serves both, and an incremental merge mechanism ensures the analytical side sees the latest data.

How does HTAP handle high-concurrency transactions and large-scale aggregation at the same time?

Behind this is GBase 8c's work at the storage level: within a single copy of data, there is both a row-store structure suited to high-concurrency transactions and a column-store structure suited to large-scale statistical aggregation — resolving the inherent conflict between transactions and analytics within one database.

How do business users retrieve data without writing SQL?

The reporting platform catalogs the "instruction manuals" for all tables into the database and combines them with semantic matching technology. Business users describe their needs in plain language, and the system automatically identifies the corresponding tables, generates the query, and returns the data. What used to be "find the table, dig through documentation, write SQL" becomes "say what you need and get the data."

If distributed scales so well, why not use it for all nine systems?

Because distributed is not free — more shards mean more machines, higher cost, and more complex operations. For read/write-dominant business systems, a primary-standby architecture is sufficiently stable and efficient; forcing distributed deployment onto them is pure resource waste. For scenarios like IoT systems with continuous high writes and large data volumes, distributed horizontal scaling is genuinely valuable. Of the nine systems, the two IoT systems used distributed deployment (two shards), while the CIM platform, CIM population map, smart integrated poles, AI large models, and data middle platform used primary-standby deployment.

How are syntax and data consistency guaranteed when migrating from PostgreSQL?

For domestic replacement, syntax compatibility and data consistency are two red lines. Leveraging GBase 8c's high compatibility with PostgreSQL, the entire migration completed syntax compatibility, object migration, and data validation, transitioning smoothly. The system also provides high availability with failover when the primary node fails, safeguarding continuous operation of municipal public services.