Creating copies of your multi-TB databases doesn’t have to take forever

Back in the day when I worked as a production DBA I vividly remember all the coordination meetings that we held to ensure that the planned clone of our production database (2 TB at the time, which was huge for us) didn’t cause unwanted side effects. Our database ran as a two node Real Application Cluster (RAC) on block volumes provided to the operating system via a couple of 4 Gbit/s Fibre Channel adapters. This was fairly standard at that time.

The storage backend, a midrange Storage Area Network (SAN) provided a bunch of Logical Unit Numbers (LUNs) to the database servers, which had to be cloned on the array using the vendor’s tool. The initial clone operation completed quickly, but the time it took for the array to copy everything from the source to the target LUNs was like an eternity. Access to the cloned LUNs was possible almost immediately. However, until the background copy completed, reads of data that had not yet been copied required additional I/O, increasing latency.

Fortunately, creating a usable clone no longer has to involve waiting for a complete physical copy.

Background & Motivation

Implementing DevOps Principles with Oracle Database dedicates a chapter to provisioning test databases, including creating copies of production datasets. It also covers Exascale, but space constraints limited the detail we could include. This short series builds on that discussion with practical, hands-on examples. In this first post, I demonstrate how to create a thin clone of a Pluggable Database (PDB) within the same Container Database (CDB). Future posts will explore other cloning strategies and approaches.

In a CI/CD workflow, a thin clone can provide a fresh database for testing a proposed change. The pipeline can create the clone from a maintained baseline, apply the schema migrations, run the tests, and remove the clone afterwards. Reducing provisioning time makes it more practical to repeat this process with realistic data volumes and give developers faster feedback.

Introducing Exadata Exascale

Nowadays cloning of a database can be a much simpler and quicker process, but it’s still problematic with VLDBs (very large databases). There is no fixed definition of a VLDB, except this one coined by Tim Gorman: “a VLDB is any database that creates operational headaches when you think about maintenance operations”.

With the introduction of Exascale storage thin cloning became much easier.

Oracle Exadata Exascale is Exadata’s new-ish storage architecture and software layer. It pools storage across Exadata storage servers and separates storage management from database servers, so compute and storage can be allocated and scaled more independently. For databases using native Exascale file storage, Exascale vaults take the place of ASM (Automatic Storage Management) disk groups for storing database files. Furthermore you don’t need sparse ASM disks for cloning.

Let’s look at cloning a PDB within the same CDB. All of these examples in the article were run on Exadata 26.2.0.0.0/Database & Grid Infrastructure 23.26.2.0.0.

SELECT con_id,
name,
open_mode,
ROUND(total_size / POWER(1024,3), 2) AS logical_gib,
ROUND(total_size / POWER(1024,4), 2) AS logical_tib
FROM v$pdbs
WHERE name = 'SWINGBENCH';
CON_ID NAME OPEN_MODE LOGICAL_GIB LOGICAL_TIB
_________ _____________ _____________ ______________ ______________
6 SWINGBENCH READ WRITE 4493.58 4.39

As you can see, this is a pretty substantial Swingbench schema. Let’s look at the PDB’s data files, note the query uses v$exa_file (🔗 to docs):

SELECT c.name AS pdb_name,
ROUND(SUM(f.size_in_bytes) /
POWER(1024,3), 2) AS logical_file_gib,
ROUND(SUM(f.space_used) /
POWER(1024,3), 2) AS exascale_space_used_gib,
ROUND(SUM(f.space_used) / 3 /
POWER(1024,3), 2) AS approximate_one_copy_gib,
f.redundancy
FROM v$exa_file f
JOIN v$containers c
ON c.con_id = f.con_id
WHERE f.file_type = 'DATAFILE'
AND c.name = 'SWINGBENCH'
GROUP BY all;

SIZE_IN_BYTES is the logical file size; SPACE_USED is the Exascale storage consumed by the file. Exascale datafiles use high redundancy, meaning three mirrored copies, hence the optional /3 column for an easier comparison with logical size.

PDB_NAME LOGICAL_FILE_GIB EXASCALE_SPACE_USED_GIB APPROXIMATE_ONE_COPY_GIB REDUNDANCY
_____________ ___________________ __________________________ ___________________________ _____________
SWINGBENCH 4461.58 13384.73 4461.58 high

So let’s see how long it takes to clone the database on Exascale.

SQL> SET TIMING ON
SQL> create pluggable database swingbench2 from swingbench snapshot copy;
Pluggable database SWINGBENCH2 created.
Elapsed: 00:00:08.888
SQL> alter pluggable database swingbench2 open instances=all;
Pluggable database SWINGBENCH2 altered.
Elapsed: 00:00:06.936

That’s far faster than anything I have seen before. In less than 16 seconds the clone was available and ready for use.

SELECT c.name AS pdb_name,
c.snapshot_parent_con_id,
ROUND(SUM(f.size_in_bytes) /
POWER(1024,3), 2) AS logical_file_gib,
ROUND(SUM(f.space_used) /
POWER(1024,3), 2) AS exascale_space_used_gib,
ROUND(SUM(f.space_used) / 3 /
POWER(1024,3), 2) AS approximate_one_copy_gib
FROM v$exa_file f
JOIN v$containers c
ON c.con_id = f.con_id
WHERE f.file_type = 'DATAFILE'
AND c.name like 'SWINGBENCH%'
GROUP BY c.name, c.snapshot_parent_con_id
ORDER BY c.name;
PDB_NAME SNAPSHOT_PARENT_CON_ID LOGICAL_FILE_GIB EXASCALE_SPACE_USED_GIB APPROXIMATE_ONE_COPY_GIB
______________ _________________________ ___________________ __________________________ ___________________________
SWINGBENCH 4461.58 13384.73 4461.58
SWINGBENCH2 4461.58 1338.51 446.17

This is pretty neat in my opinion: both PDBs report the same logical datafile size. At the time of measurement, V$EXA_FILE reported approximately one-tenth as much storage usage for the clone as for the source.

Exascale implements this using redirect-on-write: unchanged extents are shared, while changed data is written to new extents, as discussed in this tech brief.

Summary

In this test, Exadata Exascale created and opened a snapshot clone of a PDB with approximately 4.36 TiB of logical datafile capacity in less than 16 seconds. Both PDBs reported the same logical datafile size, while V$EXA_FILE reported approximately one-tenth as much storage usage for the clone as for the source.

The key is sharing unchanged extents instead of copying every data block. Redirect-on-write allows the source and clone to diverge as data changes, with additional storage allocated along the way. For development and testing, this makes working with a multi-terabyte database much more practical.

PS: Special thanks to Alex Blyth for helping me write this post 🙏