Copy Files to Multiple USB Drives

The other day I had to copy a couple of HTML and PDF files to 30 USB thumb drives. To make this as efficient as possible, I found a way to do this using (Perl and) Python. This project was also a opportunity to learn a new language.

I feel that the difference between being a magician/wizard and a programmer is being blurred by the advances of technology. What was impossible and considered magical just a few years ago is reality today. For me, coding something that is more physical is actually quite rewarding. I can’t wait to buy a 3D printer and see what I can do with that.

The (actually) 29 USB thumb drives. One of them was faulty, so I put it aside for the photoshoot.

On the road

First, I needed to extract the name of the titles from the PDF files to put them into a HTML file. So, I tried Perl I have never programmed in Perl before, so I was excited. I followed some example codes on the internet and found out how to install modules and get metadata from PDF files. According to cspan/install we need to install the modules we import in our scripts. Simply type and run cpanm PDF::API2 in the command line. Unfortunately, the solution I wrote did not provide me with the data I needed, because the PDF metadata didn’t contain titles. However, if it had, we could have written the following in a loop and saved the titles. I got this solution from lwebzem.com.

use CAM::PDF; 

my $file = "test.pdf"; 
my $pdf = CAM::PDF->new($file); 
my $doc="";

for ($i=1; $i < = $pdf->numPages(); $i++) 
{
	$doc = $doc.$pdf->getPageText($i);
	print $doc;   
}

The copying

The second step was to copy the files to multiple USB thumbdrives. I actually had a USB hub which can connect 10 USB:s at the same time.

My first thought was to attach 10 USBs in one go and write a script which copies the data to them, so I could do something else in the meantime. This proved to be somewhat tricky. First, I wanted to copy all the files from a folder to the root of the USB:s. I found shutil, which can do this using rootcopy, but it does not allow you to copy to root, which I needed. Otherwise it would defeat the purpose of placing the files in the drives. I found the code for a function called copytree that does this for me at Stackoverflow. The user Cyrille merged the solutions of atzz and Mital Vora and “Will not fail if dst already exists”, which is good.

USB hub with 10 drives. Notice the one on the right hand side.

def copytree(src, dst, symlinks = False, ignore = None):

  if not os.path.exists(dst):
    os.makedirs(dst)
    shutil.copystat(src, dst)
  lst = os.listdir(src)
  if ignore:
    excl = ignore(src, lst)
    lst = [x for x in lst if x not in excl]
  for item in lst:
    s = os.path.join(src, item)
    d = os.path.join(dst, item)
    if symlinks and os.path.islink(s):
      if os.path.lexists(d):
        os.remove(d)
      os.symlink(os.readlink(s), d)
      try:
        st = os.lstat(s)
        mode = stat.S_IMODE(st.st_mode)
        os.lchmod(d, mode)
      except:
        pass # lchmod not available
    elif os.path.isdir(s):
      copytree(s, d, symlinks, ignore)
    else:
      shutil.copy2(s, d)

Additionally, I wanted to copy to all USB drives, but without having to know the drive names. I used a program called devcon to get the USB information. Unfortunately, it was not reliable, so I scrapped that idea. Instead I write down the letters of the drives in a text file and let Python read from that file, simple.

Each USB thumbdrive contains a default name, so I had to change the name to something nice, like “Event_2014” or something. To do this I call os.system(“label ” + tmp + ” ” + labelname) where tmp is the drive name, labelname is “Event_2014”.

So, the copying was solved, but I also needed a way to unmount the thumbdrives because of write buffers. I found RemoveDrive which allows this. I call it using call with the loop flag -L, meaning it will attempt to unmount until it is unmounted.

import sys
import os #for command line

filename = "destinations.txt"
fd  = open(filename, "rb")

#Unmount the drives, keep trying until they are unmounted...
print "Unmounting drives..."

for dest in fd:
    tmp = dest.rstrip('\r\n')
    print "Unmounting...", tmp,
    sys.stdout.flush()
    try:
        from subprocess import call
        call(["RemoveDrive.exe", (tmp+":"), "-L"])
    except:
        pass  

print "Finished unmounting all drives"
sys.stdout.flush()
    
fd.close()

I wanted the unmounting step to be separate from the copying and naming, because I wanted to visually check that the files had been copied.

So, I kicked my heels three times and all 30 USBs were suddenly containing the files. There is no place like home.

Screenshot from “The Wizard of Oz” (1939) IMDB.