Time and technology keep accelerating, and the data we save keep on growing exponentially. Thus, the exact definition of a Very Large Database (VLDB) continues to change with new advances in hardware and software. But fitting a VLDB on your new tablet, or even a laptop, is typically not going to happen, at least not yet.

Just how large is a Very Large Database?
Many tech authorities debate the actual starting point of what constitutes a VLDB. Some say databases of 100 gigabytes or more. Others argue that a VLDB is at least 1 terabyte (1000GB) in size. It can also be measured by the number and type of complex queries and transactions it can support. But regardless of the exact quantity, it’s safe to say that a VLDB contains an enormous amount of information, disseminated in an extremely high number of tuples (database rows), and occupies massive storage space in a physical file system.

You know you are dealing with a VLDB when the very size of it mandates more work. Not being able to backup the database in a reasonable amount of time, and the financial constraints in duplicating the DB for test purposes, are just a couple of the issues which must be dealt with.

Who uses VLDBs?
Many companies and organizations utilize VLDBs, including municipalities, government defense contractors, research firms, and big businesses in the financial industry. A prime example of one who regularly exploits VLDBs to its strategic advantage is TAC, a fully independent European research group providing advisory services on international economic and financial issues for financial investors, industrial companies and multilateral organizations. TAC constructs VLDBs, mostly for their own internal use, but some have been made available to the public for the analytical purposes of monitoring foreign investments and development goals.

Other businesses use VLDBs for data comparison, data mining, data warehousing, replications, queries, demonstrations, and a multitude of additional functions. Technology trend watchers monitor with them. IT managers design and develop with them. R&D experts and innovation units track statistics with them. And VLDBs play an ever increasingly important role in the demand for personalized information on the World Wide Web.

So how do you manage a Very Large Database?
The story of David vs. Goliath often comes to mind when folks find their databases approaching a terabyte. It becomes more and more difficult to allocate space and speed up queries. So that seemingly insurmountable task of reclaiming space and improving performance must then be assigned to an expert, one with the know-how and technological tools to conquer that giant.

VLDBs can be effectively controlled by the right type of Database Management System (DBMS); i.e., specifically designed computer software (or suite of software) that provides an interface between the database and the end user, enabling the creation, querying, updating, and administration of the data. A DBMS also maintains the security and integrity of stored data, and facilitates the recovery of information if a system happens to fail. And there are different types of DBMS’s, but we’ll save that dissertation for another blog at another time.

The development of most Database Management Systems is extremely complex and time-intensive, requiring continual upgrades with advancements in computer memory, storage, processors, and networks.  Databases continue to increase in size, scope, and performance, as software developers strive to keep pace with the demands of modern data volume increases. Like most of life in the 21st century, we’ve come a long way very fast, and we’ve got the data to prove it!