LonnieWest.com

Mind. Music. Muse.

Archiving Floppy Disk Media

FloppyCHALLENGE

Determine the best software to create complete bit-for-bit images of the 3.5” floppy disks that contain all data, boot information, and any “hidden” data on the disk. The software must work on a modern operating system (Windows 7 / Windows 8) and use a readily available off-the-shelf USB floppy drive. These image files are to be used for archival purposes, and can be written to blank media to create an exact copy of the original disk.

 

BACKGROUND INFORMATION

For many clients I’ve worked with over the years, a commonly overlooked issue is the proper archival and storage of floppy disk and optical media. This is particularly true for manufacturing sites that employ automated equipment.

There are several reasons that archival projects are passed off to other departments, forgotten about, or never bubbling up to the top of the priority list:

  1. Automation software is typically not managed by a corporate IT department.
  2. Automation software is usually not considered part of “traditional” software development, so source code, parameter files, and firmware are not included in source code management systems.
  3. Most people are not aware of the practical lifespan of physical media.
  4. Source code is rarely needed once the project has been deployed.
  5. There is a false sense of security in the concept of physical storage.

Too often the solution is the engineering or QA department has a file cabinet filled with labelled floppy disks, hopefully with an index of what is stored there.

The problem with physical storage is media degradation. The lifespan of physical media is often not considered, or incorrectly estimated, resulting of loss of intellectual property. Here is a compilation of statistics drawn from NIST, the National Archives, and the American Library Association:

Floppy Disk 3-5 years
Flash Media (USB, CD, CFlash, SSD) 1-10 years
Hard Drive 2-8 years
Optical Disk (commercially pressed) 2-10 years
Optical Disk (burned to blank media) 2-5 years
Magnetic Tape (archival quality) 10-30 years

These are averages culled from decades of practical use, and do not represent an absolute limit. For example: while a floppy disk has an average lifespan of 3-5 years it can last 10 years or more, or it can fail after only 6 months. It all depends on the quality of the media, the quality of the equipment used to write the disk, the storage conditions, and equipment used to read the media.

Optical media was once considered permanent storage when it was first introduced, but the reality is that the dyes used in writable CD-R disks fade fairly quickly and the plastics used in commercial CD’s degrade so the underlying information cannot be read reliably over time.

NIST, the National Archives, and the American Library Association have all adopted processes for creating complete image files of the original physical media that are then saved to a network archive in addition to physical storage.

For optical media, the ISO file format is a standard that wraps all information on a disk into a single file. This file can be read directly via the operating system, mounted as a “virtual” drive that acts just like a real CD or DVD, and can be burned to blank media to create an exact clone of the original.

For floppy disks, the IMG (IMA, VFD, or RAW extensions are also used interchangeably) format is a raw data, bit-for-bit copy of the information on a disk. Like the ISO file, this IMG file can be mounted as a “virtual” floppy disk and can be written to a blank disk to create an exact clone of the original.

You can think of an ISO or IMG file as a ZIP file that contains the entire contents of a disk, including boot sector information and other hidden data. It is a single file that can be safely stored on a corporate network because it is not directly executable and does not usually trigger corporate security exclusions.

While creating and using ISO files for optical media are pretty straightforward, creating and using IMG files requires a little more effort.

 

WHY IS CREATING FLOPPY DISK IMAGES DIFFICULT?

Unlike optical media, the floppy disk was developed era where there was far more diversity in hardware and operating systems. Even through there were only a few different physical sizes (the 3.5” disk being the most common for PC and Mac users) there were a many different ways to format a disk depending on the hardware and operating system. For example, a “standard” 3.5” floppy could be formatted as a 360k, 400k, 720k, 800k, 1.44M or 2.88M disk. It could use a single side, or double sides. Depending on the operating system, the speed at which it was written was between 300rpm and 590rpm.

Prior to Windows XP, application software had direct access to the floppy drive. While the operating system defined the basic structure of the drive (boot sectors, index locations, and how bad sectors are marked), the software was free to write directly to the disk in any manner needed.

This led many manufacturers to write data in unconventional ways (eg: formatting a 1.44M disk as 720k and writing to the “empty” area, or writing data in sectors marked as bad) so they could be used to contain product license information that were impossible to copy via standard operating system commands.

Allen-Bradley (Rockwell Automation) created “Activation Disks” using these techniques and they served as their primary method of copy protection for over a decade until it was retired a few years ago in favor of the now-common online activation.

From Windows XP and forward, application software no longer had direct access to disks and must go through a standard OS device driver interface to read and write data. This effectively ended the era of hiding license information on a disk and now makes it more difficult to access that information using a modern OS such as Windows 7 or Windows 8.

It is still possible to read this information, but it requires a 3rd party driver to be installed to allow the software to circumnavigate the standard operating system drivers to allow access to data that is not normally accessible.

 

THIRD PARTY IMAGING SOFTWARE

There are several software packages out there that create and manage floppy disk images, however I settled on the following 3 packages:

  • MagicISO
  • UltraISO
  • WinImage

These 3 packages were selected as they all support Windows 7 / Windows 8, support creating and writing image files, and support both floppy and optical media. (If I’m going to invest in imaging software for archiving, I want to be able to create ISO files from CD and DVD media as well.)

During testing I relied on a forensic imaging tool named FTK Imager to provide a reference image to compare the others against. FTK Imager is intended to be used with a data analysis suite designed for legal and law enforcement investigative work, so the imaging portion itself is strictly designed to do one thing: collect an exact copy of all the available data as it exists on the disk without corrupting the original data.

Each software package was tested against 3 unique disks that represent challenging conditions that most disk copy application have problems with:

Disk Type Unique Characteristics
RsLogix Activation This disk uses advanced data obfuscation techniques to hide license information in areas that cannot be copied using standard OS commands.
720k Hidden Data This is a 1.44M floppy formatted to appear as a 720k floppy, with data written to the area outside the 720k zone. It was created by an OS/2 system.
MS-DOS 6.22 Boot This is standard OS boot disk.

 

The goal is to determine how effective each application is when creating image files of a non-standard disk, and writing those files successfully back to a blank disk. The resulting disk should be a perfect copy of the original with all data intact.

Methodology:

  • A reference image file of each disk was created with FTK Imager,
  • An image file was created from the disk by MagicISO, UltraISO, and WinImage
  • The image files were then written to a blank disk by MagicISO, UltraISO, and WinImage
  • The resulting copied disks were then tested to verify that:
    • All data is present,
    • The disk can boot the computer (if bootable image)
    • The license information is intact (if Activation disk)
    • Hidden data is present (if disk uses deliberate incorrect format)
  • The image files created by MagicISO, UltraISO, and WinImage were all compared against the reference FTK Imager file to confirm that they contain identical data.

Criteria:

Imaging software must be able to read and write all formats without error, and not require outside tools to format blank disks.

 

 

The table below summarizes the Read and Write testing:

Application Disk Read
IMG Size
Read Speed Read Errors Write Speed Write Errors Requires External Format? Exact Clone? Notes
FTK Imager RsLogix Activation 1440k 1 min None N/A N/A N/A N/A FTK is read-only tool
FTK Imager 720k Hidden 1440k 1 min None N/A N/A N/A N/A FTK is read-only tool
FTK Imager MS-DOS Boot 1440k 1 min None N/A N/A N/A N/A FTK is read-only tool
MagicISO RsLogix Activation 1440k 5 mins None 5 mins No No Yes
MagicISO 720k Hidden 720k 1 5 mins None 7 mins No No Yes 2 1 Does not read data from remainder of 1440k disk formatted as 720k.
2 If FTK image (full 1440k) is used, a perfect clone is created.
MagicISO MS-DOS Boot 1440k 5 mins None 2 mins No No Yes
UltraISO RsLogix Activation 1440k 1 min Yes 1 1 min Yes 2 Yes 3 Yes 1 Reported incorrect format after read complete.
2 Reported incorrect format after write.
3 Will only write to disk if it is already formatted in same format as image file.
UltraISO 720k Hidden 720k 1 min None 1 min Yes 1 Yes 2 Yes 3 1 Refused to write file until disk was re-formatted.
2 Will only write to disk if it is already formatted in same format as image file.
3 If FTK image (full 1440k) is used, a perfect clone is created.
UltraISO MS-DOS Boot 1440k 1 min None 1 min No Yes 1 Yes 1 Will only write to disk if it is already formatted in same format as image file.
WinImage RsLogix Activation 1458k 1 min Yes 1 2 mins Yes 2 Yes Yes 1 Disk Sector on Track 80, Head 0. Format error. Ignored error to complete.
2 Will only write to disk if it is already formatted in same format as image file.
WinImage 720k Hidden N/A N/A Yes 1 N/A Yes 2 N/A No 1 Threw error when image reading started. Not able to complete.
2 Unable to write 720k image to disk regardless of image file used.
WinImage MS-DOS Boot 1440k 1 min None 2 mins No No Yes

 

While WinImage is fast and is the only tool that offers the ability to format disks to non-standard formats, it also failed the most tests. WinImage completely failed to handle the 720k disk, and threw an error message on the RsLogix disk (even though it created a working copy) so as an archival tool, I would not be comfortable this application could handle everything that may be encountered in an archival situation.

UltraISO fared better, although it completely refused to write to a disk unless it was already formatted to match the IMG file. Unfortunately, it offers no formatting tools, and the only option in Windows was to format the disk to its full size. If this were the only tool for archival purposes, it would not be able to deliver a working disk when needed.

MagicISO is very slow, but it had no problem reading and writing whatever was thrown at it. The only exception is that it failed to read the complete contents of the 1440k (1.44M) disk formatted as a 720k disk, so any data hidden on this disk would be lost. Even so, when given a full image (1440k of data read by FTK Imager), it successfully wrote the disk complete with hidden data.

Interestingly, only FTK Imager was able to read the full contents of the 1440k disk formatted to appear as a 720k disk. This is a credit to its use as a forensic tool: it ignored the Logical Disk size that Windows reported and read every track available.

 

SUMMARY

Testing was conducted with an off-the-shelf Sabrent USB floppy disk drive, using a Windows 8.1 64bit OS. There is a strong possibility that some of the problems exhibited by WinImage and UltraISO may not appear in other OS and hardware combinations. However, as the intent of this study was to find an effective archive solution that would run on a modern OS with currently available hardware, neither WinImage nor UltraISO would be suitable to be the sole archival solution.

My current recommendation is to use FTK Imager to read disk images, and MagicISO to write them.

While the workflow for FTK Imager is designed for evidence handling (it prompts for case information metadata), reads a complete image of any floppy in about one minute and returns data for all the physical information on the disk regardless of how the logical format makes it appear to the operating system.

MagicISO is the only product that appears to forcibly write data to a disk. Instead of requiring a disk to be formatted, it simply writes all the data presented in the IMG file (which also includes the format information) to the disk without question.

This isn’t a perfect solution to the challenge, but it is one I am confident will be able to create perfect images of any disk that is encountered in an archival process.