GBase: Metaverse Database Technology Outlook

Key Features of Metaverse Databases

Whether in the real or virtual world, databases are essential IT infrastructure. To support the daily operations of the metaverse, databases provide data platform support for digital twin mirroring and virtual extensions in mixed reality. Every virtual concept in the metaverse, regardless of whether it describes a concrete or abstract object, as well as all real objects associated with metaverse entities, can potentially be represented in a database. Databases in the metaverse will face new technological challenges.

In the metaverse’s virtual world, the degree of digitization is far higher than in the physical world. Spatial structures, scenes, and entities delineated by digital technologies essentially exist as data, requiring the datafication of entities, scenes, and behaviors/interactions. Therefore, the metaverse’s virtual world demands more data support than the real one. While not all of this data is stored in databases—other systems will also handle metaverse data storage—the fundamental data, relational data, activity and operational data, and evolutionary data of virtual world spatial structures, scenes, and entities must be stored in databases. Hence, metaverse databases should at minimum support ultra-large data scale, data diversity, and multiple data processing modes.

In the metaverse virtual world, data types are varied. Economic systems need to record transaction information, which is mostly structured and suitable for transactional relational databases. Various direct and indirect relationships exist among entities and their economic activities; these intricate relationships are well-suited for graph databases. Streaming data from sensory devices and digital twins is appropriate for streaming or real-time databases. Activity trajectories and locations of virtual entities require GIS spatial databases. Miscellaneous documents, images, sounds, and videos may also fit document databases and other types.

Under the metaverse, data types are complex and processing methods are diverse. Since each type of database excels in different scenarios, it is difficult to rely on a single database to handle all use cases. However, using a unified database language—or one unified language per database type—is feasible. A unified language can be built on a unified virtual logical layer, under which specific database engines execute operations.

For database vendors, the metaverse presents not only new challenges but also rare opportunities. To better support the metaverse, on one hand, database product technology must further break through traditional requirements; on the other, database products need stronger ecosystem integration capabilities. We see great potential in Chinese database vendors in these aspects. Take GBase as an example: in recent years, GBase has continuously improved and surpassed itself in product R&D technology. Its distributed database product has passed the large-scale capability test of the CAICT Big Data Product, with a cluster reaching 4,096 nodes and setting an international record. GBase’s virtualized data technology enables cross-engine heterogeneous database integration, including fusion with graph databases, streaming databases, and broader databases. On the other hand, GBase has also been working on ecosystem integration and is now fully integrated into China-developed hardware and software system ecosystems.

Figure 1   An example of a China-developed database system ecosystem

These capabilities of Chinese database vendors demonstrate that Chinese databases are already well-prepared for metaverse development.

Key Features of Metaverse Databases

(1) Supporting Blockchain

To build an economic system, the metaverse relies on blockchain technology. Blockchain is essentially a decentralized ledger database, with its underlying layer typically being Key-Value NOSQL data. Blockchain distributed node data maintains ledger consistency across nodes through consensus algorithms, uses cryptographic algorithms to link blocks, implement data signing and rights confirmation, adopts account models to express who generated the data, who initiated it, and to whom it was transferred, and uses script systems to drive the validation and execution of such script data.

Blockchain databases can preserve current relevant information and all previous information, generating a historical database that grows like an ever-expanding archive of its own history. Generally, as metaverse transactions depending on blockchain increase and historical data accumulates, the synchronization delay between blockchain ledgers gradually rises and overall system performance degrades. This is because blockchain databases differ from distributed databases: each node in a distributed database stores only a portion of the data, whereas in a blockchain each node stores the entire database. Thus, under the metaverse, high-frequency and continuously expanding blockchain databases will face performance tests.

Below is an example of a blockchain ledger database:

Figure 2 Blockchain database records

(2) Enabling Digital Twins

The metaverse encompasses the mapping and interaction between the physical world and the virtual world, which is also digital twins—primarily oriented towards things. Digital twins need to simulate complete and extensive virtual objects and can be based on VR/AR devices, 5G/6G, edge-cloud, smart sensors, machine vision, etc., to achieve virtual world activity control, equipment operation and maintenance, and more. Digital twins can also provide excellent equipment development simulation, energy monitoring, and predictive maintenance. Manufacturers could establish virtual factories on a unified industrial metaverse platform, connecting various plants and cross-enterprise cooperation, benefiting large industry players with vast amounts of data.

From this we can see that under the metaverse, a substantial amount of data originates from smart wearables, sensory devices, and IoT systems, containing physical and virtual equipment operation and status data. Processing such data demands high real-time capabilities. Thus, metaverse databases will face massive, high-frequency real-time data. The time window size for storing real-time data directly affects the storage space required for real-time data.

An important characteristic of real-time databases is real-time processing, including data real-timeliness and transaction real-timeliness. Data real-timeliness refers to the update cycle of field I/O data; as a real-time database, data real-timeliness must be considered. Transaction real-timeliness refers to the speed at which the database processes transactions. This can be either event-triggered or time-triggered. Event-triggered means that once an event occurs it can be scheduled immediately, allowing immediate processing but consuming more system resources. Time-triggered means obtaining scheduling rights within a certain time frame. For a complete real-time database, considering system stability and real-time performance, both scheduling methods must be provided simultaneously.

Figure 3 Real-time database interaction for digital twin

(3) 3D Visualization

A picture is worth a thousand words. Traditional databases have already laid a solid foundation for visualization, especially BI visualization, which provides very intuitive effects for management decision-making. Since the metaverse is a virtual world parallel to the real world, all real-world entities—people, objects, mountains, rivers—may need to appear in the virtual space.

Virtual spaces need to display a large number of 3D dynamic entities. To simulate the real world, traditional 2D graphics and images are insufficient. Rendered 3D images will be extensively used in the metaverse. The cities, deserts, mountains, hills, forests, and oceans we often see in games can all serve as 3D models. Using model libraries, highly realistic characters, buildings, trees, and roads can be displayed.

Images are generally classified into two types—vector graphics and raster (bitmap) graphics. Taking photographs or pictures as an example, images consist of dot matrices of different colors. This is called a bitmap, which is what we typically see; common formats include *.bmp, *.jpg, etc. The other category, vector graphics, is like some engineering equipment diagrams and cartoons, mainly composed of lines and color blocks, expressed by algebraic formulas for each element. These algebraic formulas and their attributes are stored, producing vector graphics. In simple terms, such graphical effects are obtained through formula calculations and can be infinitely scaled without distortion. Current technology can already use basic image materials with machine learning or vector computation to assist in generating 3D dynamic images. Among these, highly realistic synthetic images used for deep learning algorithm training have achieved results, but they consume significant resources and impact I/O. Meanwhile, 3D images based on vector technology are relatively mature and resource-light.

Vector graphics are image or drawing-oriented, can be drawn based on geometric features, and are generated by algorithmic calculations. The main elements of vector graphics are points, lines, rectangles, polygons, circles, and arcs, computed through mathematical formulas. Vector graphics maintain very smooth, clear edges regardless of scaling up or down. Advantages of vector graphics include small storage footprint, editable object image elements, unaffected image resolution when scaling, and output device-independent resolution. Vector technology has been successfully applied in spatial geographic databases. Currently, 3D model database output formats include 3D MAX, FLT, OSG, etc., and support multiple terrain data format imports. Model databases can render highly realistic entities, requiring considerable computing power. I believe the future metaverse will also widely use 3D databases based on vector technology.

Figure 4 3D spatial database structure diagram

Key Technologies for Metaverse Databases

(1) Distributed Technology

From its inception, the concept of the metaverse determines that it must operate in a decentralized manner. Decentralization means this virtual space is not governed by the subjective will of a single service entity but is a space collectively operated and maintained by all independent individuals, requiring a decentralized, egalitarian sharing and co-governance mechanism.

Under a decentralized operating mechanism, distributed databases will be a crucial requirement for the metaverse. The scalability of distributed databases can meet ever-growing storage demands. However, because transactional business requires strong consistency, distributed databases in the real world must restrict nodes for distributed writes to maintain consistency among nodes—i.e., concurrent writes need to be queued, thus limiting database scalability and performance. As the fully digitized metaverse environment evolves, flexible handling of strong consistency will allow distributed database scale to meet development needs.

The core technology for distributed transactional databases to guarantee ACID properties under abnormal conditions is primarily two-phase/three-phase commit technology, also known as 2PC/3PC.

The principle of two-phase commit introduces a coordinating role in a distributed system, distinct from ordinary transaction participants. The coordinator is the node directly responding to users, splitting a complete transaction into distributed transactions assigned to participants. Thus, during transaction execution, participants perform a pre-execution or execution preparation instead of completing the commit in one step. This way, if an exception occurs during commit, the participant can report it to the coordinator, enabling the coordinator to instruct other participants to fail this operation as well.

Three-phase commit inserts a preparation phase between Phase 1 and Phase 2 of two-phase commit. This resolves the potential lengthy delay caused when, during two-phase commit’s pre-execution stage, the coordinator crashes or fails, leaving participants unsure whether to commit or abort (an “uncertain state”).

Because relational databases have high consistency requirements, distributed databases represent both the focus and difficulty of distributed database development. To achieve distribution, database sharding is needed; typically, horizontal sharding and vertical sharding are used. When data is horizontally sharded by business logic, performance depends mainly on how well data placement aligns with business. When data is sliced and stored by capacity, performance largely depends on the efficiency of data chunk (shard) scheduling.

Figure 5 Sharding methods for distributed databases

(2) Cloud Technology

Cloud computing is essentially an architecture and methodology that achieves large-scale computing through virtualization technology. In cloud computing, resources and functions are provided to users as services. Cloud computing represents resources in an abstract or logical manner; through virtualization, the resources of different servers can be offered as a unified whole.

The metaverse is a highly virtualized world that requires corresponding virtual technology support. Cloud databases need to be built upon cloud computing technology. As cloud resources expand, the computing and storage of a cloud database scale accordingly. Therefore, cloud databases first need a storage-computing separation architecture; traditional integrated storage-computing and shared-storage databases cannot meet the requirements of cloud databases.

The cloud database architecture is illustrated as follows:

Figure 6 Cloud database architecture diagram

Where:

Database Interface Proxy: Implements a unified data access interface proxy; business components or modules access underlying database services through this proxy. During implementation, database connection pool management, database load balancing, and related aspects must be considered.

SQL Parsing: Responsible for parsing the SQL syntax of client requests, identifying the read/write characteristics of the statement, and further parsing the schema, tables, fields, conditions, etc. based on the statement characteristics. For example, for INSERT statements, it parses the field names and values to be inserted; for SELECT, UPDATE, and DELETE statements, it parses the conditional expressions in the WHERE clause.

Data Routing: Responsible for searching the rule pool for relevant rules based on the parsing results. Once found, the parsing results are substituted into the rules for computation, obtaining the specific physical database node to which the statement should be forwarded. The rule pool primarily includes read/write rules, horizontal sharding partitioning rules, and data object access rules.

Multi-Tenant Management: Enables multi-tenant sharing and management at both the database instance and database schema levels. Database-level sharing uses databases as the basic partitioning unit—i.e., creating/assigning a database instance for each tenant and sharing storage and servers. Schema-level sharing uses User/Schema as the basic partitioning unit—i.e., after a database instance is created, creating a Schema for each tenant, with shared storage, servers, OS services, and database instance across tenants.

Management Agent: To achieve centralized management and performance monitoring of the entire database resource pool, a database management agent must be placed on each physical database node. On one hand, the agent provides a unified operational entry point to physical database nodes; on the other, it enables real-time resource monitoring and performance data collection.

Management Functions: Provide functionalities such as adding, removing, starting, and stopping different database service nodes within a service cluster, enabling scaling. This includes collecting node information, logging manual operations on nodes, interacting with the agent service, interacting with the monitoring system, and jointly accomplishing service management and monitoring.

(3) Security and Privacy

In the metaverse, every organization and individual will own their own data. Data security and privacy will inevitably be a key consideration for databases. Beyond blockchain, database technology itself must adopt necessary measures to ensure data security and privacy.

Key technologies for metaverse database security and privacy include:

Secure Communication: Using secure socket layer protocols, encrypting the link layer with cryptographic algorithms, and performing mutual authentication between server and client to guarantee the confidentiality and integrity of communications between client and server. Secure communication also protects communication between connection manager and client/server, as well as server-to-server communication in high-availability environments.

Identity Authentication: Supports conventional database user password authentication, as well as integration with Pluggable Authentication Modules (PAM), Lightweight Directory Access Protocol (LDAP), and Single Sign-On for user access verification to the database.

Role Management: Predefines default roles such as Database Security Administrator, Database Audit Administrator, and Database Administrator. System administrators can also create roles and assign users based on actual business needs.

Discretionary Access Control (DAC): The granularity of DAC reaches the database, table, and field levels. Data owners and their designated users can set different permission management on objects at different granularities.

Mandatory Access Control (MAC): Uses a label-based access control system to implement MAC requirements. Security labels consist of one or more ordered numeric components, unordered set components, and tree components representing hierarchical relationships. MAC enforcement reaches the row level.

Secure Storage: The confidentiality of stored data is one of the most important functions of a secure database. Data encryption in data management adopts intra-database encryption, performing encryption/decryption processing at the kernel storage engine level of the database management system—i.e., data is encrypted/decrypted during physical I/O. Since data pages are only encrypted/decrypted during actual I/O, it is completely transparent to legitimate users, thus also called transparent storage encryption. Encryption algorithms and keys are provided by hardware encryption cards compliant with Chinese national cryptographic standards.

Security Auditing: Security operators set audit masks, which can be configured per user or globally, including required and excluded audit masks. Audit masks can combine 160 audit events. Audit operators start/stop auditing, configure management, and analyze audit records.

Figure 7 Database security and privacy technology diagram

(4) Hardware Acceleration for Computing

  Compared to the physical world, the metaverse demands even greater computing power improvements. For example, massive virtual entity dynamic generation, extensive AI analysis, and real-time responses all require substantial computing power. The big data storage of the metaverse can adopt distributed storage technology by adding storage nodes. However, computing power cannot be adequately improved merely by adding nodes. For instance, distributed parallel processing in databases, constrained by consistency, sometimes prevents transaction records from all being accelerated by adding nodes. Similarly, some real-time tasks that cannot be sharded cannot rely on adding nodes to resolve real-time response issues. Therefore, such challenges need to be addressed jointly with hardware technology.

In addition to software and algorithm optimization, intensive computing in databases can also be improved through hardware technology to boost computing power. Using in-memory databases is one approach, but it has limitations, such as insufficient storage capacity and limited computing power gains. Using parallel heterogeneous computing technology to improve computing power is a proven and viable method. Currently, one effective hardware acceleration approach is to add GPUs and FPGAs alongside CPUs to achieve parallel heterogeneous computing.

GPUs are particularly adept at parallel computations similar to image processing, where computations are high-density but the processed data has little correlation. GPUs provide a large number of computing units (up to thousands) and substantial high-speed memory, enabling parallel processing of many pixels simultaneously.

GPUs are designed for compute-intensive, highly parallel computations. Their logic control units are simpler compared to CPUs. Thus, to enable instruction pipelining and improve instruction execution efficiency, the algorithms must be inherently low in complexity and the data being processed must be independent. Serial processing within the algorithm would lead to a significant reduction in GPU floating-point computing capability.

FPGAs, as high-performance, low-power programmable chips, can be customized for targeted algorithm design. Because their algorithms are customized, FPGAs bypass the instruction fetch and decode stages of CPUs and GPUs. The data flow directly performs fixed operations based on the customized algorithm, and the computing units can operate every clock cycle, fully leveraging floating-point computing capability and achieving higher computational efficiency than CPUs and GPUs.

The hardware acceleration technology architecture based on GPU and FPGA is illustrated below:

Figure 8 GPU/FPGA heterogeneous computing diagram