Instantly "Copy" Huge Files Using Hard Links in Windows

Mike Wolfe -

In Windows, a hard link is a file system feature that allows multiple file names to be associated with the same underlying data on disk.

Hard links provide an alternative name or path for accessing a file without duplicating its contents. They are different from symbolic links, which are references to the file or directory by name rather than by the file's data.  Hard links and symbolic links are also different than shell links, which are more commonly known as shortcut files.

Conceptually, hard links are very similar to object variables and object instances in VBA.

Here are some key points about hard links in Windows:

  1. Identical file content: A hard link points to the same file content as the original file. Any changes made to one file will be reflected in all other hard links associated with it.
  2. Same file attributes: Hard links have the same file attributes (such as permissions, timestamps, and ownership) as the original file.
  3. Independent file names: Each hard link has its own file name, but they refer to the same underlying data. This means that deleting a hard link doesn't affect the other hard links or the original file until all hard links are deleted.
  4. Same file size: All hard links to a file have the same file size since they share the same data.
  5. No distinction between the original and hard links: There is no concept of a "parent" or "child" file when it comes to hard links. All hard links are treated equally, and there is no inherent distinction between the original file and its hard links.
  6. Same file permissions: When it comes to permissions, the hard link and the original file are indistinguishable. Modifying the permissions on one will affect the other.
  7. Cannot cross disk volumes: Hard links can only be created within the same volume.  This makes sense, since the point of a hard link is to point to an already existing set of data.  

File System Equivalent of VBA Object Variables

Hard links in Windows and object references in VBA share some similarities in terms of referencing a single set of data by multiple variables. Here's a comparison between the two:

  1. Shared data: Both hard links and object references allow multiple entities to access and operate on the same underlying data. In the case of hard links, multiple file names can point to the same file content on disk. In VBA, multiple object variables can refer to the same object in memory.
  2. Modifications affect all references: Any changes made to the data through one hard link or object reference will be reflected in all other hard links or object variables pointing to the same data. If you modify the data through one reference, the changes will be visible when accessing the data through other references.
  3. No duplication of data: Hard links and object references do not duplicate the data they refer to. They simply provide alternative names or references to the same data. This helps conserve storage space and memory.
  4. Independent identifiers: In both cases, each hard link and object reference has its own identifier or name. Deleting a hard link or removing an object reference doesn't immediately affect the underlying data until all references to it are removed.
  5. No inherent hierarchy: Hard links and object references don't establish a hierarchy between the original data and the references. They treat all references equally. There is no inherent distinction between the original file or object and its hard links or references.
  6. Same data attributes: Both hard links and object references typically share the same attributes and properties as the original data. For example, in Windows, hard links inherit the file attributes of the original file. In VBA, object references inherit the properties and methods of the original object.
  7. Reference counting: In VBA, objects live for as long as they have at least one reference.  In Windows, a file persists on the hard drive as long as it has at least one hard link.  Once all hard links have been removed, the data location is marked as available and other data may overwrite it at any time.

Conceptually, hard links and object variables have a lot of overlap.  I found this analogy helped me better understand how hard links and VBA object variables both work.

There are two basic ways to create hard links in Windows:

  1. The Hard Way: via the command line
  2. The Easy Way: via the Link Shell Extension utility

It's worth noting that not all file systems support hard links. NTFS is the most common file system in Windows that supports hard links, while FAT and exFAT do not.  

NTFS is by far the most common file system for hard disks in Windows 10 and 11.  FAT is a simpler file system that was more common in older versions of Windows.  These days, you are most likely to run into a FAT or exFAT file system on removable media, such as flash drives.

The Hard Way (pun intended)

To create a hard link in Windows, you can use the mklink command in the Command Prompt or PowerShell. The syntax is as follows:

mklink /H <linkname> <targetfile>

Where <linkname> is the name and path of the new hard link, and <targetfile> is the existing file to which the hard link should point.

The Easy Way

The command line approach is not particularly difficult to use.  

However, I find that creating hard links is something I do so rarely that I can never remember the right commands.  As I've written about in the past, processes that happen infrequently benefit from a wizard-like experience.  That's what you get with the Link Shell Extension.

You can download the software from here: Download Link Shell Extension.

Personally, I prefer to install it using chocolatey:

choco install linkshellextension

If you need to install chocolatey–the Windows package manager–you can do that here.

Acknowledgements