Terminology v5

Suggest edits

There are many terms you will come across in EDB Postgres Distributed that you may be unfamiliar with. This page is a list of many of those terms with quick definitions.

Asynchronous replication

A type of replication that copies data to other PGD cluster members after the transaction completes on the origin node. Asynchronous replication can provide higher performance and lower latency than synchronous replication. However, asynchronous replication can see a lag in how long changes take to appear in the various cluster members. While the cluster will be eventually consistent, there will be potential for nodes to be apparently out of sync with each other.

Commit scopes

Rules for managing how transactions are committed between the nodes and groups of a PGD cluster. Used to configure synchronous replication, Group Commit, CAMO, Eager, lag control and other PGD features.

CAMO or commit-at-most-once

High value transactions in some applications require that the application successfully commited exactly once, and in the event of failover and retrying, only once. To ensure this happens in PGD, CAMO can be enabled allowing the application to actively participate in the transaction.

Conflicts

As data is replicated across the nodes of a PGD cluster, there may be occasions when changes from one source clash with changes from another source. This is a conflict and can be handled with conflict resolution (rules which decide which source is correct or preferred), or avoided with conflict-free data types.

Consensus

How Raft makes group-wide decisions. Given a number of nodes in a group, Raft looks for a consensus of the majority (number of nodes divided by 2 plus 1) voting for a decision. For example, when a write leader is being selected, a Raft consensus is sought over which node in the group will be the write leader. Consensus can only be reached if there is a quorum of voting members.

Cluster

Generically, a cluster is a group of multiple redundant systems arranged to appear to end users as one system. See also PGD cluster and Postgres cluster.

DDL

Data Definition Language - The subset of SQL commands that deal with the defining and managing the structure of a database. DDL statements can create, modify and delete objects - schemas, tables and indexes - within the database. Common DDL commands are CREATE, ALTER and DROP.

DML

Data Manipulation Language - The subset of SQL commands that deal with manipulating the data held within a database. DML statements can create, modify and delete rows within tables in the database. Common DML commands are INSERT, UPDATE and DELETE.

Eager

Eager is a synchronous commit mode that avoids conflicts by detecting incoming potentially conflicting transactions and “eagerly” aborts one of them to maintain consistency.

Eventual consistency

A distributed computing consistency model stating changes to the same item in different cluster members will eventually converge to the same value. Asynchronous logical replication with conflict resolution and conflict-free replicated data types exhibit eventual consistency in PGD.

Failover

The automated process that recognizes a failure in a highly available database cluster and takes action to maintain consistency and availability. The goal is to minimize downtime and data loss.

Group commit

A synchronous commit mode which requires more than one PGD node to successfully receive and confirm a transaction at commit time.

Immediate consistency

A distributed computing model where all replicas are updated synchronously and simultaneously. This ensures that all reads after a write has completed will see the same value on all nodes. The downside of this approach is its negative impact on performance.

Logical replication

A more efficient method of replicating changes in the database. While physical streaming replication duplicate the originating database’s disk blocks, logical replication instead takes the changes made, independent of the underlying physical storage format, and publishes them to all systems that have subscribed to see the changes. Each subscriber then applies the changes locally. Logical replication is not able to support most DDL

Node

A general term for an element of a distributed system. A node can play host to any service. In PGD, there are PGD Nodes which run a Postgres database and the BDR extension and optionally a PGD Proxy service.

Typically, for high availability, each node runs on separate physical hardware, but not necessarily. For example, a proxy might share a hardware node with a database.

Node groups

PGD Nodes in PGD clusters can be organized into groups to reflect the logical operation of the cluster. For example, the data nodes in a particular physical location may be part of a dedicated node group for the location.

PGD cluster

A group of multiple redundant database systems and proxies arranged to avoid single points of failure while appearing to end users as one system. PGD clusters may be run on Docker instances, cloud instances or “bare” Linux hosts, or a combination of those platforms. A PGD cluster may also include backup and proxy nodes. The data nodes in a cluster are grouped together in a top level group and into various local node groups.

PGD node

In a PGD cluster, there are nodes which run databases and participate in the PGD Cluster. A typical PGD node will run a Postgres database and the BDR extension and optionally a PGD Proxy service. PGD Nodes may also be referred to as data nodes which suggests they store data, though some PGD Nodes, specifically witness nodes do not do that.

Physical replication

By making an exact copy of database disk blocks as they are modified to one or more standby cluster members, physical replication provides an easily implemented method to replicate servers. But there are restrictions on how it can be used. For example, only one master node can run write transactions. Also, the method requires that all cluster members are on the same major version of the database software with the same operating system and CPU architecture.

Postgres cluster

Traditionally, in PostgreSQL, a number of databases running on a single server is referred to as a cluster (of databases). This kind of Postgres cluster is not highly available. To get high availability and redundancy, you need a PGD Cluster.

Quorum

When a Raft consensus is needed by a PGD cluster, there needs to be a minimum number of voting nodes participating in the vote. This number is called a quorum. For example, with a 5 node cluster, the quorum would be 3 nodes in the cluster voting. A consensus would be 5/2+1 nodes, 3 nodes voting the same way. If there were only 2 voting nodes, then a consensus would never be established.

Raft

Replicated, Available, Fault Tolerance. A consensus algorithm which uses votes from a quorum of machines in a distributed cluster to establish a consensus. PGD uses RAFT within groups (top level or local) to establish which node is the write leader.

Read scalability

The ability of a system to handle increasing read workloads. For example, PGD is able to introduce one or more read replica nodes to a cluster and have the application direct writes to the primary node and reads to the replica nodes. As the read workload grows, you can increase the number of read replica nodes to maintain performance.

Switchover

A planned change in connection between the application or proxies and the active database node in a cluster, typically done for maintenance.

Synchronous replication

When changes are updated at all participating nodes at the same time, typically leveraging a two-phase commit. While this approach replicates changes and resolves conflicts before committing, a performance cost in latency occurs due to the coordination required across nodes.

Subscriber-Only nodes

A PGD cluster is based around bidirectional replication, but in some use cases such as needing a read-only server, bidirectional replication is not needed. A Subscriber-Only Node is used in this case; it only subscribes to changes in the database to keep itself up to date and provide correct results to any run directly on the node. This can be used to enable horizontal read scalability in a PGD cluster.

Two-phase commit (2PC)

A multi-step process for achieving consistency across multiple database nodes. The first phase sees a transaction prepared on an originating node and sent to all participating nodes. Each participating node validates that it can apply the transaction and signals its readiness to the originating node. This is the pre-commit or prepare phase. In the second phase, if all the participating nodes signal they are ready, the originating node proceeds to commit the transaction and signals the participating nodes to commit, too. This is the commit phase. If, in the pre-commit phase, any node signals it is not ready, the entire transaction is aborted. It's this process that ensures all nodes get the same changes.

Vertical scaling or scale up

A traditional computing approach of increasing a resource (CPU, memory, storage, network) to support a given workload until the physical limits of that architecture are reached, e.g., Oracle Exadata.

Witness nodes

Witness nodes primarily serve to help the cluster establish a consensus. An odd number of data nodes are needed to establish a consensus and, where resources are limited, a witness node can be used to participate in cluster decisions but not replicate the data. Not holding the data means it cannot operate as a standby server or provide majorities in synchronous commits.

Write leader

In an Always-On Architecture, a node is selected as the correct connection endpoint for applications. This node is called the write leader, and once selected, proxy nodes route queries and updates to it. With only one node receiving writes, unintended multi-node writes can be avoided. The write leader is selected by consensus of a quorum of data nodes. If the write leader becomes unavailable, the data nodes select another node to become write leader. Nodes that aren't the write leader are referred to as shadow nodes.