11.3.3: NewSQL Databases

NewSQL databases arose in the 2010s as a response to the need for horizontal scalability (like NoSQL) while retaining SQL and ACID semantics of relational databases. The term “NewSQL” was coined to describe modern distributed SQL databases that can handle high throughput and big data but still support SQL queries and strong transactional consistency. Essentially, NewSQL systems attempt to achieve the best of both worlds: the reliability and familiarity of SQL/relational models with the distributed, scalable architecture of NoSQL.

Characteristics of NewSQL databases include: distributed architecture (sharded data, often with a consensus algorithm to coordinate transactions), SQL support (so existing applications/tools can use standard SQL), and ACID transactions across a distributed system. To do this, NewSQL databases implement innovations under the hood – for example, using deterministic ordering or synchronization mechanisms like Google’s TrueTime (clock synchronization) in Spanner to ensure a global ordering of transactions. Many NewSQL systems use a shared-nothing cluster of nodes and partition data among them, but employ protocols (like two-phase commit or Paxos/Raft consensus) to ensure that transactions affecting multiple nodes remain consistent.

Notable NewSQL databases and technologies include:

• Google Spanner: Perhaps the poster child of NewSQL, Spanner is a globally distributed SQL database that provides external consistency (serializable transactions) at scale. It uses Google’s TrueTime API (atomic clocks + GPS) to timestamp transactions, achieving a form of synchronous replication across data centers with consistency. Spanner (available via Google Cloud Spanner) can scale horizontally across regions while giving the developer a familiar relational schema and SQL interface. Use case: multi-continent financial or inventory systems needing strong consistency.
• CockroachDB: An open-source NewSQL database inspired by Spanner, which also aims for global distribution with serializable isolation. It automatically shards and replicates data, and uses consensus for consistency.
• Amazon Aurora (with Aurora Serverless or Global Database): While Aurora is built on MySQL/Postgres, it introduced NewSQL-like concepts by separating storage and compute. It keeps six copies of data across three AZs and can automatically failover or scale read nodes. Aurora’s Global Database feature allows read replicas across regions, and although writes still funnel to one region’s primary, it pushes towards global scale relational usage.
• VoltDB / Volt Active Data: A distributed in-memory SQL database designed for high throughput on scalable clusters, maintaining ACID by partitioning data and running transactions in a single partition where possible (to avoid cross-node coordination).
• YugabyteDB: A newer entrant that offers a distributed SQL mode (YSQL) on top of a sharded, fault-tolerant design (built on a modified Cassandra storage engine with Raft for consistency).

Advantages: NewSQL databases can handle workloads that traditional single-node SQL would choke on, without forcing developers to abandon SQL or consistency. They reduce the complexity of manual sharding – instead of an application splitting data among multiple DBs and managing that, the NewSQL system handles distribution transparently. They also often maintain high performance for reads and writes by optimizing networking and using advanced algorithms. For instance, Google Spanner can maintain strong consistency across globally distributed data with minimal latency overhead by careful time synchronization, which is a remarkable feat. These systems are well-suited for cloud environments where elasticity and geo-distribution are needed but data correctness can’t be compromised.

Challenges/Disadvantages: The technology is complex and sometimes not as battle-tested as classic databases. NewSQL systems can be operationally complex (though managed services like Cloud Spanner mitigate this for users). Also, some NewSQL solutions impose certain constraints – e.g., Spanner requires a certain schema design (no joins without primary key, etc., to ensure performance). The ecosystem and tooling might be less mature, and finding expertise can be harder than for, say, MySQL. Additionally, NewSQL doesn’t necessarily mean magic infinite writes scaling – many still have trade-offs (for example, some might still serialize a global transaction log or have limits on cross-shard transactions for performance).

Use cases: NewSQL shines in scenarios requiring both high scalability and strict consistency. A classic example is financial systems or mission-critical SaaS that need to scale to many users globally but cannot lose transactions or allow dirty reads. For instance, a global multiplayer game could use a NewSQL database to keep user profiles and game state consistent across regions so that a player’s progress is reliably stored no matter where they connect. Spanner is used internally at Google for services like AdWords, which demands both massive scale and consistency in billing and ad serving data. Another use case is retail inventory across many stores: a NewSQL DB can ensure no two warehouses sell the last item simultaneously (consistency), even under high load, without resorting to partitioning the data manually. Essentially, if the domain requires transactions and scaling out, NewSQL is a candidate.

In the cloud, Google Cloud Spanner is a prominent NewSQL service (with multi-region and even multi-continent replicas, offering a 99.999% availability SLA and strong consistency). Azure Cosmos DB is not exactly NewSQL (since it is multi-model and often used as NoSQL), but it offers an SQL API and supports transactional consistency on single partition keys with options for global consistency, blurring the line – Cosmos can be configured to provide strong consistency at the cost of performance, approaching NewSQL territory for certain workloads. AWS does not have a direct Spanner equivalent at the time of writing, though Aurora with its high throughput and Global Database approach is a partial step, and third-party NewSQL (like CockroachCloud) can be run on AWS.

Comparison Summary: In choosing SQL vs NoSQL vs NewSQL, consider the nature of your data and workload:

• If you require strict data integrity, complex queries, and transactions, and the workload can be handled by vertical scaling or read replicas, a SQL relational database is often best. Cloud-managed SQL databases can handle a wide range of app needs with minimal hassle.
• If you need to scale to massive write/read throughput, handle very flexible data schemas or very large data volumes, and can tolerate eventual consistency or don’t need SQL joins, a NoSQL database might be appropriate. It excels for big web-scale applications, real-time analytics, caching user sessions, etc.
• If you need both scalability and consistency/SQL, and are building a system where partitioning data manually is not desirable, a NewSQL database could be the solution – giving you a single logical SQL database that scales out. This is especially relevant for next-generation enterprise systems and global applications.
  

Often, a single application will use a combination: for example, an online game might use NoSQL (Redis or DynamoDB) for caching player sessions and leaderboards (to handle high reads/writes with low latency), but use a SQL/NewSQL database for financial transactions or account data where consistency is critical. Cloud architecture allows mixing these components, and many cloud services integrate (e.g., using AWS DynamoDB for some parts and Amazon Aurora for others, connected through application logic).

As technology evolves, the lines blur – many NoSQL databases now offer some SQL-like querying or multi-record ACID transactions (e.g., Cosmos DB, MongoDB with multi-document transactions), and some SQL platforms scale better than before (sharded MySQL with Vitess, etc.). Cloud project managers should evaluate data requirements (volume, speed, structure, consistency needs) and choose the appropriate database or combination thereof.

---