At re:Invent in Las Vegas, Amazon previewed Aurora DSQL - a distributed SQL database manager with fast reads and writes, and declared that it was PostgreSQL compatible. Yet there are missing features including foreign keys, views, triggers, sequences, extensions ... and much more.
"Aurora DSQL doesn't support all PostgreSQL features," the docs state. And when it comes to SQL commands like , "PostgreSQL has a large number of clauses and parameters that Aurora DSQL doesn't support."
Amazon CTO Werner Vogels described the architecture of Aurora DSQL in his keynote at re:Invent, and explained that it is "a hierarchy of independent components, that are each distributed and resilient." These components include the front end, the query processor, the journal, and an adjudicator. Each component scales independently and in different ways. If a monolithic design had been used, as in traditional database engines, it would involve "massively overscaling and becoming extremely inefficient," he claimed. Another key feature of DSQL is that transactions complete when they are written to the journal (a log of all updates) - rather than when the data is updated in storage, as in most databases.
DevClass spoke to G2 Krishnamoorthy, VP AWS database services, about DQSL and this issue of the missing features. "All of the query processing happens in the query processor process and that is running the PostgreSQL engine," he told us. "This enables us to provide the same protocol, the same connectivity. So then why do we have not full compatibility?"
One reason, Krishnamoorthy explained, is that the storage system is different. This impacts "extensions that take advantage of how PostgreSQL stores bytes on disk" - such as the extension built for geospatial data. "We would need to change the APIs ... we will work with the community to do that," he told us.
Second, "there is functionality that assumes that all state is in a single process. So auto increment counters are incompatible with distributed SQL." This is the case in many distributed SQL systems, he noted.
Third, there is "functionality that we will actually implement and enable. An example is foreign keys, or support for pgSQL stored procedures. We have not enabled them because we are ... making sure that we can validate them with high quality."
The requirement for asynchronous code is another factor. Regarding the data definition language (DDL), Krishnamoorthy told us that it must all run asynchronously "because in a distributed system you cannot have DDL as a locking operation. We are entirely optimistic concurrency, which means that every DDL that we need to support, we need to come up with an approach that enables that DDL to run asynchronously." This is why DSQL requires , for example, rather than just , which only works on tables that have no data.
Compatibility is therefore partly a work in progress, and partly a consequence of a different architecture and breaking up the database engine into multiple components. It is worth it, Krishnamoorthy argues, because of the benefits. The intention is to "make databases effortless," he told us. DSQL is "entirely serverless, and it does have a single writer or a single reader, so it is very elastic. As you're adding load, we can keep spinning up more of everything - more query processor, more journals for transaction throughput, more storage replicas."
He added that another benefit of microservices is that "we are able to make software updates completely seamlessly."
Despite the lack of compatibility, Krishnamoorthy said, "we have a set of applications that we are tracking that are PostgreSQL-based - like there is an open source ERP application that is built on Postgres. We were able to port that on top of DSQL with just a day of work."