using System;
using System.IO;

public class Writer
{
    static void Main(string[] args)
    {
        Random rng = new Random();
        
        byte[] buffer = new byte[1024*1024];
        
        DateTime start = DateTime.Now;
        using (FileStream stream = new FileStream (args[0], FileMode.Create))
        {
            for (int i=0; i < int.Parse(args[1]); i++)
            {
                rng.NextBytes(buffer);
                Console.Write(".");
                stream.Write(buffer, 0, buffer.Length);
            }
        }
        DateTime end = DateTime.Now;
        Console.WriteLine();
        Console.WriteLine (end-start);
    }
}

using System;
using System.IO;

public class Reader
{
    static void Main(string[] args)
    {
        byte[] buffer = new byte[1024*1024];
        
        DateTime start = DateTime.Now;
        int total=0;
        using (FileStream stream = new FileStream (args[0], FileMode.Open))
        {
            int read;
            while ( (read=stream.Read (buffer, 0, buffer.Length)) > 0)
            {
                total += read;
                Console.Write(".");
            }
        }
        DateTime end = DateTime.Now;
        Console.WriteLine();
        Console.WriteLine (end-start);
        Console.WriteLine (total);
    }
}

using System;
using System.IO;

public class RandomReader
{
    static void Main(string[] args)
    {
        byte[] buffer = new byte[1024*1024];
        
        Random rng = new Random();
        DateTime start = DateTime.Now;
        int total=0;
        using (FileStream stream = new FileStream (args[0], FileMode.Open))
        {
            int length = (int) stream.Length;
            for (int i=0; i < int.Parse(args[1]); i++)
            {
                stream.Position = rng.Next(length-buffer.Length);                
                total += stream.Read (buffer, 0, buffer.Length);
                Console.Write(".");
            }
        }
        DateTime end = DateTime.Now;
        Console.WriteLine();
        Console.WriteLine (end-start);
        Console.WriteLine (total);
    }
}

8 thoughts on “Vista and External Memory Devices”

1. I am not an expert on this by any means, but I heard that the difference was for random seek time, where flash is much better than HD (while they are about the same for sequencial read).

   I would also guess that you are testing stuff too big.
   Try checking the same amount of data, but in 4Kb, 16Kb, 32Kb, 256Kb chunks, which is what I believe the paging system is using. That may be where the sweet spot is.

   LikeLike

   Reply

2. Ah, that was it. I realised that seeking might have been important, but stupidly didn’t spot that when reading a large chunk of data, the seek could easily be the cheapest part, making it less significant as a differentiator.

   Changing the RandomReader to read 4K blocks made it about 15 times faster on the USB flash device than with a hard disk.

   However, the data still needs to get onto the flash device in the first place – and that’s going to take a while, by the looks of it. I guess that can be done in the background where the transfer time isn’t important, however.

   Good to see my blog ending up as a way of broadcasting my stupidity ;)

   LikeLike

   Reply

3. Well, I also ran the programs on my RAID 0 SCSI disks in my Dell box, and on a kingston usb2 512MB USB stick. I can tell you, nothing beat the scsi rig. if I execute the tests 10 times (specify 10 as second parameter) the reads/randoms aren’t even noticable (00:00:00 :P) on the scsi harddisks though are on the usb stick seeks. Also with 4KB blocks.

   So it might work on laptops, it’s not going to be as simple on desktop boxes with fast harddisks.

   LikeLike

   Reply

4. So just to check Frans – the test I did which showed flash being fast was changing RandomReader to have only a 4K buffer instead of 1MB, then running it with 200 reads. Note that you’ve got to do this when the data isn’t cached, otherwise it really will be instantaneous.

   I’ll try it on my desktop at work…

   LikeLike

   Reply

5. Aha, I think that’s it. I didn’t disable any disk cache (I don’t know if that’s even possible on this monster :)).

   LikeLike

   Reply

6. The easiest way to flush the disk cache is to just use Writer again to write another 1GB file – then use RandomReader on the first file you wrote.

   I tried the test on my desktop at work, and the seeking was much faster than at home – I think it was about the same speed as the flash drives. My guess is that this will be a feature for laptops much more than desktops. That makes sense in another way too – desktops tend to be relatively easy to upgrade in terms of memory, as there are usually more slots available.

   LikeLike

   Reply

7. Also to perform writes directly to the disk you can use FileOption.WriteThrough.

   About seeking, IMO in flash drives there’s no mechanical part, so seeking there must be relatively fast.

   Also memory devices such as flash drives are slow on writing, but can be fast, when reading

   LikeLike

   Reply

8. I am not an expert so I often find that this topic is complicated to get your solution has provided good knowledge about the subject.

   LikeLike

   Reply