The NTFS file system supports many volume and file-level
features that may lead to what appears to be lost or misreported free disk
space. This behavior may be apparent when an NTFS volume suddenly becomes very
full for no reason, and yet an administrator cannot find the cause or locate
the offending folders and files. This sometimes occurs because of malicious or
unauthorized access to an NTFS volume on which very large files or a high
quantity of small files are secretly copied, which then have their NTFS
permissions removed or restricted. This behavior may also occur after a system
malfunction or power outage occurs, causing volume corruption.
article discusses how to check an NTFS file systems disk space allocation to
discover offending files and directories or check for volume corruption. This
article is intended for Windows 2000-and-later operating systems that support
some advanced storage features and troubleshooting methods; however Windows NT
users may also find this article useful. The following is a short list of
reasons why the disk space allocation of an NTFS volume may appear to be
- The NTFS volume's cluster size is too large for the
average-sized files being stored.
- File attributes or NTFS permissions prevent files or
folders from being displayed or accessed by Windows Explorer or from a Windows
- The folder path exceeds 255 characters.
- Directories or files contain invalid or reserved file
- NTFS metafiles (such as the Master File Table) have grown
and cannot be de-allocated.
- Files or folders contain alternate data streams.
- NTFS File system corruption causes free space to be
reported as being in use.
- Other NTFS features may cause file-allocation
Please use the following information to help optimize,
repair, or gain a better understanding of how disk space is used on your NTFS
volume or volumes.
Cluster size is too large
Disk space can be consumed only by files and directories that
include internal NTFS metafiles like the Master File Table (MFT), directory
indexes, and so forth. All file space allocation is consumed by using multiples
of a cluster. A cluster is a collection of contiguous sectors. The cluster size
is determined at the time the volume is formatted, and is further determined by
the partition size. For more information about clusters, click the following
article number to view the article in the Microsoft Knowledge Base:
Default cluster size for FAT and NTFS
When a file is first created, it consumes a minimum of a single cluster
of disk space, depending on the initial file size. When data is later added to
a file, NTFS increases the file's allocation in multiples of the cluster
To determine the current cluster size and volume statistics,
run a read-only chkdsk
command from a command prompt, as follows
IE: Chkdsk D:
and then view the resulting output. For example:
4096543 KB total disk space. <--- Total formatted disk capacity.
2906360 KB in 19901 files. <--- Space used by user file data.
6344 KB in 1301 indexes. <--- Space used by NTFS indexes.
0 KB in bad sectors. <--- Space lost to bad sectors.
49379 KB in use by the system. <--- Includes MFT and other NTFS metafiles.
22544 KB occupied by the log file. <--- NTFS Log file - (Can be adjusted using chkdsk /L:size)
1134460 KB available on disk. <--- Available FREE disk space
4096 bytes in each allocation unit. <--- Cluster Size. (4K)
1024135 total allocation units on disk. <--- Total Clusters on disk.
283615 allocation units available on disk. <--- Available free clusters.
: Multiply each value that is reported in kilobytes (KB) by 1024
to determine accurate byte counts. For example: 2906360 x 1024 = 2,976,112,640
By using this output, you can determine how your disk space is
being used, along with the default cluster size. To see if this is the optimal
cluster size, determine the amount of wasted space by following these steps:
- Double-click My Computer on the desktop, and then double-click the drive letter (for
example, D) of the volume in question.
This opens the volume and
displays folders and files contained in the root.
- Click any file or folder, and then click the Select All option on the Edit menu.
- With all files and folders selected, right-click any file
or folder, and then click the Properties option.
This opens a Properties dialog box that contains a General tab. It also tabulates the total number of files and directories
on the entire volume and provides two file size statistics: SIZE and SIZE ON
If you are not
using NTFS compression for any files or folders contained on the
volume, the difference between SIZE and SIZE ON DISK is wasted space because of
a larger-than-necessary cluster size. You should attempt to use an optimal
cluster size so that the SIZE ON DISK value is as close to the SIZE value as
possible. An excessive discrepancy between the SIZE ON DISK and the SIZE value
is an indication that the default cluster size is too large for the average
file size that you are storing on the volume, and that it should be decreased.
This can be done only by backing up the volume and then reformatting the volume
by using the format command and the /a
switch to specify the appropriate allocation size:
IE: format D: /a:2048
(This example uses a 2-KB cluster size).NOTE
: Alternately, you can enable NTFS compression to regain space
lost because of an incorrect cluster size; however, this may result in a slight
decrease in performance.
File attributes or NTFS permissions
Both Windows Explorer and the following directory list command
DIR /A /S
will total file and directory statistics for only those files and
directories that you have permissions to access. Files always excluded by
default include all hidden files and protected operating system files. This
behavior may result in inaccurate file and folder totals and size statistics to
be displayed by Windows Explorer or DIR command outputs. To include these types
of files in the overall statistics, change Folder Options by following these
- Double-click My Computer on the desktop, and then double-click the drive letter (D) of the
volume in question. This opens the volume and displays folders and files
contained in the root.
- On the Tools menu, click Folder Options, and then click the View tab.
- Select the Show Hidden Files and Folders
option, and then click to clear the Hide protected operating system
files check box.
- Acknowledge the warning message, and then click the Apply button.
This allows Windows Explorer and the DIR /A /S command to total
all files and directories that are contained on the volume that the user has
To determine which directories and files cannot be
accessed, follow these steps:
- From a command prompt, pipe the output of a DIR /A /S to a
For example: DIR D: /A /S
- Run ntbackup.exe, and then select the Detailed option on the Backup Log tab under Options on the Tools menu.
This enables detailed backup logs.
- On the Backup tab, back up the entire volume that is affected (D), and then
start the backup.
- After backup is complete, open the backup report and
compare directory for directory the NTBackup log output with the D-DIR.TXT
output that you saved in step 1.
Because backup can access all files, its report may contain
folders and files not seen or counted by Windows Explorer and the DIR command.
You may find it easier to use the NTBackup GUI to navigate the volume when you
are looking for large files or directories that otherwise are inaccessible
through Windows Explorer without actually backing up the volume.
After you locate files that you don't have access to, you should be able to add
or change permissions by using the Security
tab while you view the properties of the file or directory in
Windows Explorer. By default, you will not have access to the System Volume
Information directory. You will need to add the appropriate permissions for the
directory to be included in the DIR /A /S command.NOTE
: You may run across directories or files that have no security
tab, or find that you cannot re-assign permissions to the affected folders and
files. You may receive the following error message while you try to access
D:\directory_name\ is not accessible
Access is denied
If you have any such directories, contact
Product Support Services for additional assistance at the following Microsoft
Invalid file names
Directories or files that contain invalid or reserved file names
may also be excluded from file and directory statistics. Directories or files
that contain leading or trailing spaces are perfectly legal from an NTFS file
system perspective, but are not legal from a Win32 subsystem point of view, and
therefore, neither Windows Explorer nor a command prompt can reliably handle
them. For more
information, click the following article number to view the article in the
Microsoft Knowledge Base:
How to Remove Files with Reserved Names in Windows
Often times it is not possible to rename or delete these files or
folders. When you attempt to rename or delete them, you may receive one of the
following error messages:
Error renaming file or
Cannot rename file: Cannot read from the source file or disk.
Error deleting file or folder
Cannot delete file: Cannot read from the source file or disk.
have directories or files that cannot be deleted or renamed, contact Product
Support Services at the following Microsoft Web site:
NTFS Master File Table (MFT) expansion
When an NTFS volume is first created and formatted, NTFS
metafiles are created. One of these metafiles is called the Master File Table
(MFT). It is very small when first created (approximately 16 KB), but it grows
as files and directories are created on the volume. When a file is first
created, it is entered into the MFT as a File Record Segment (FRS), which is
always 1024 bytes (1 KB) in size. As files are added to the volume, the MFT
grows as required. However, when files are deleted, the associated FRSs are
marked as free to be reused, but the total FRSs and associated MFT allocation
remains. This explains why, after deleting a large number of files, you don't
regain the space used by the MFT.
To see exactly how large the MFT
is, you can use the built-in defrag utility to analyze the volume. The
resulting defrag report provides detailed information about the size and number
of fragments in the MFT.EXAMPLE
Master File Table (MFT) fragmentation
Total MFT size = 26,203 KB
MFT record count = 21,444
Percent MFT in use = 81 %
Total MFT fragments = 4
However, for a more complete picture of how much space (overhead)
is being used by the entire NTFS file system, perform a chkdsk
, and then look at the resulting output for the following line:
In use by system.
Currently, only third-party defrag utilities consolidate unused
MFT FRS records and reclaim unused MFT allocated space.
Alternate data streams
NTFS allows files and directories to contain alternate data
streams. This feature allow multiple data allocations to be associated with a
single file or directory. Please be aware of the following limitations when you
use alternate data streams on files and directories.
When a file or
directory contains alternate data streams:
- Windows Explorer and the DIR command do not report the data in alternate data streams as part of the file
size or volume statistics. Rather, they show only the total bytes for the
primary data stream.
- The output from chkdsk accurately reports space used by a user's data files, including
alternate data streams.
- Disk quotas accurately track and report all data stream
allocations that are part of a user's data files.
- NTBackup records the number of bytes backed up in the
backup log report. However it does not show which files contain alternate data
streams, nor does it show accurate file sizes for files that include data in
NTFS File system corruption
In very rare circumstances, the NTFS Metafiles $MFT or $BITMAP
may become corrupted and result in lost disk space. This issue can be
identified and fixed by running a chkdsk /F
against the volume in question. Toward the end of chkdsk
, you receive the following message if the $BITMAP needs to be
Correcting errors in the master file table's (MFT) BITMAP attribute.
CHKDSK discovered free space marked as allocated in the volume bitmap.
Windows has made corrections to the file system.
Other NTFS features that may cause file allocation confusion
NTFS also supports hard links and reparse points that allow
volume mount points and directory junctions to be created. These additional
NTFS features may cause confusion when you try to determine how much space is
being consumed on a physical volume.
A hard link is a directory entry
for a file regardless of where the actual file data exists on that volume.
Every file can be considered to have at least one hard link. On NTFS volumes,
each file can have multiple hard links, and therefore a single file can appear
in many directories (or even in the same directory with different names).
Because all of the links reference the same file, programs can open any of the
links and modify the file. A file is deleted from the file system only after
all links to it have been deleted. After you create a hard link, programs can
use it like any other file name. Keep in mind that Windows Explorer and a
command prompt will show all linked files as being the same size, even though
they all share the same data and don't actually use that amount of disk
Volume mount points and directory junctions allow an empty
folder on an NTFS volume to point to the root or subfolder on another volume.
Windows Explorer and a DIR /s command follow the reparse point, count any files
and directories on the destination volume, and include them in the host
volume's statistics. This may lead you to believe that more space is being used
on the host volume than what is actually being used. For more information about junction points, click
the following article number to view the article in the Microsoft Knowledge
How to create and manipulate NTFS junction points
In summary, chkdsk
output, NTBackup GUI or backup logs, and the viewing of disk
quotas are all good methods of determining how disk space is being used on a
volume. On the other hand, Windows Explorer and the DIR command have some
limitations and drawbacks when used for this purpose.