Consider the following scenario.
- In Microsoft SQL Server 2005, you have a SQL Server Integration Services (SSIS) package that uses a data flow task.
- The data flow task contains references to columns of binary large object (BLOB) data. A BLOB column is a column of the DT_TEXT data type, the DT_NTEXT data type, or the DT_IMAGE data type.
- You set the BLOBTempStoragePath property for the data flow task. When SSIS processes the BLOB column, it will temporarily write data from the BLOB column to the disk.
- There are more than 65,536 active rows that contain BLOB data in the data flow task.
In this scenario, you run the SSIS package. When SSIS tries to write the BLOB data to the temporary storage, you receive the following error messages:Error message 1
Error message 2
Error: 0x80070050 at Data Flow Task, DTS.Pipeline: The file exists.
Error message 3
Error: 0xC0048019 at Data Flow Task, DTS.Pipeline: The buffer manager could not get a temporary file name. The call to GetTempFileName failed.
Error message 4
Error: 0xC0048013 at Data Flow Task, DTS.Pipeline: The buffer manager could not create a temporary file on the path "Path". The path will not be considered for temporary storage again.
Error message 5
Error: 0xC0047070 at Data Flow Task, DTS.Pipeline: The buffer manager cannot create a file to spool a long object on the directories named in the BLOBTempStoragePath property. Either an incorrect file name was provided, or there are no permissions.
Error message 6
Error: 0xC02020E5 at Data Flow Task, Flat File Source : Writing long object data of type DT_TEXT, DT_NTEXT, or DT_IMAGE to Data Flow task buffer failed for column "ColumnName".
Error: 0xC0208266 at Data Flow Task, DTS.Pipeline: Long data was retrieved for a column but cannot be added to the Data Flow task buffer.
It is theoretically possible that a similar problem may also occur in the following scenario:
- You run an SSIS package that contains a data flow task.
- You set the BufferTempStoragePath property for the data flow task. When a low-memory-resource notification is sent to the data flow engine, the data flow engine will write data buffers to the directories that are designated in the BufferTempStoragePath property.
- More than 65,536 temporary files of data buffers are written to the directories.
In this rare scenario, you receive different error messages.
If you specify additional folders in the BLOBTempStoragePath
property or in the BufferTempStoragePath
property, and then run the SSIS package again, you find the additional paths are ignored and the SSIS package still does not run successfully.
The folders that are designated in the BufferTempStoragePath
properties contain temporary files. These temporary files are named in a range from DTS.tmp through DTSFFFF.tmp. The number range is from 0x0 through 0xFFFF. For example, possible temporary file names include DTS.tmp, DTS1.tmp…, and DTSFFFF.tmp. There are 65,536 possible file names in total. When there are more than 65,536 files, SSIS may run out of file names and throw the errors that are mentioned in the Symptom section.
The fix for this issue was first released in Cumulative Update 5 for SQL Server 2005 Service Pack 3. For more information about this cumulative update package, click the following article number to view the article in the Microsoft Knowledge Base:
Cumulative update package 5 for SQL Server 2005 Service Pack 3
Because the builds are cumulative, each new fix release contains all the hotfixes and all the security fixes that were included with the previous SQL Server 2005 fix release. Microsoft recommends that you consider applying the most recent fix release that contains this hotfix. For more information, click the following article number to view the article in the Microsoft Knowledge Base:
The SQL Server 2005 builds that were released after SQL Server 2005 Service Pack 3 was released
Microsoft SQL Server 2005 hotfixes are created for specific SQL Server service packs. You must apply a SQL Server 2005 Service Pack 3 hotfix to an installation of SQL Server 2005 Service Pack 3. By default, any hotfix that is provided in a SQL Server service pack is included in the next SQL Server service pack.
To work around this issue, use one of the following methods:
Increase the buffer size by setting the DefaultBufferSize
property to the maximum value of 104,857,600. Note
If you use this method, there may be more memory consumption when you run the SSIS package.
In the design stage, try to reduce blocking transforms or downstream bottlenecks after the point where the BLOB columns are introduced to the data flow task. This method may reduce the possibility that the limit will be reached.
Microsoft has confirmed that this is a problem in the Microsoft products that are listed in the "Applies to" section.
After you apply this hotfix, the following two problems are resolved:
- Multiple paths delimited by semicolons will enable the buffers to spill into the additional directories that are specified in the BLOBTempStoragePath or BufferTempStoragePath properties.
If you specify multiple folders in the BLOBTempStoragePath or BufferTempStoragePath properties such as “C:\temp1; D:\temp2”and if there is problem when SSIS tries to access the first folder, the second folder will be used. The temporary files are not distributed across the additional folder paths unless a problem is encountered with the first folder. If it is not a fatal error, SSIS will use the next folder in the list successfully.
However, if the first folder runs out of disk space, severe errors will be posted. Additionally, SSIS cannot gracefully ignore the error and use the next folder.
- SSIS will not run out of names. The new file naming pattern dynamically generates a random GUID for each temporary file name. New file names start with the following three prefixes:
The following are examples of new temporary file names:
- DTT: DTT temporary files are test files. These files are used to make sure that the temporary directory can be accessed and written before data is saved into the directory.
- DTB: DTB temporary files are BLOB temporary files. These files are used when the DT_TEXT, DT_NTEXT, or DT_IMAGE data types are processed in a data flow task.
- DTS: DTS temporary files are buffer swap temporary files.
For more information about the Incremental Servicing Model for SQL Server, click the following article number to view the article in the Microsoft Knowledge Base:
An Incremental Servicing Model is available from the SQL Server team to deliver hotfixes for reported problems
For more information about how to obtain SQL Server 2005 Service Pack 3, click the following article number to view the article in the Microsoft Knowledge Base:
How to obtain the latest service pack for SQL Server 2005
For more information about the new features in SQL Server 2005 Service Pack 3 (SP3) and about the improvements in SQL Server 2005 SP3, visit the following Microsoft Web site:
For more information about the naming schema for SQL Server updates, click the following article number to view the article in the Microsoft Knowledge Base:
New naming schema for Microsoft SQL Server software update packages
For more information about software update terminology, click the following article number to view the article in the Microsoft Knowledge Base:
Description of the standard terminology that is used to describe Microsoft software updates