Once bitten, twice backed.

Have you ever seriously thought about backing up your data? If not, chances are that you haven't been the victim of a system/hardware/disk failure. Most of the people who seriously think backing up their data and maintaining snapshots is an important part of their routine are the ones who have spent countless hours of their time trying to recover/rebuild their lost data after a system crash. Why does it require someone to suffer before they start becoming cautious with their precious data? Maybe the reason for it is; to quote from Shakespeare's The Merchant of Venice: "For sufferance is the badge of all our tribe".

Whatever, it may be, I would like to say that it's important to always back up your important data and protect it against system failure(s). In fact, I have set up an RAID-1(mirrored) array at home where I store my important data and my home directory. This way, even if one of my Hard Disks were to crash, I'd have the data safely backed up on the other one in the mirrored array. However, this solution does not prevent against:

1. Accidental file deletion,


2. Accidental Disk formatting.


3. File System corruption.

To prevent against such problems, I'd want to have a separate backup(or preferably snapshotting) utility which saves versions of files as you save them. Ideally, the solution would be to introduce a surrogate file system such as a one written using the fuse file system, and have that file system save versions of the files as you save them. So, changes between an open() and close() on a file should be logged and saved as a file version. This way, the user will be able to recover files by how old it is and also get a fairly fine granularity control over the changes that happened. An optimization that could be done is to store only the changes to a file if only edits were made to it, whereas save a complete copy if the file size was changed.

I'm looking for more ideas and people interested in helping me develop such a solution, so if anyone is interested in helping out, please let me know and we'll try to work something out.

Hacking after a very long time!!!!

After very many months did I finally get down to doing some actual hacking on a program that I wanted to fool into thinking it something else. Please note that I'm use the word work hacking in the true sense of the word, and not to illegally gain access to a system. Please read this page to get a low-down on that controversy.

Anyways, so there was this utility very similar to the UNIX cat utility that read in a file, and wrote it to some other location. However, this utility also performs a check to determine if:

1. There is sufficient space on the destination disk.


2. The destination file is on a special file system

Only if both the conditions are met does the utility go ahead with the task of creating the destination file. A special feature of this utility is that it will create sparse files if the source file contains many holes in it. So, it is not strictly required that the destination file system contain as much space as the source file size. Also, the second check is made to make sure that the destination file system supports sparse files. However, if we know that there is another file system which supports sparse files, we can do away with that check as well.

All said and done, I did not have the source code to the utility that I am talking about, so I decided to do something else. I used the LD_PRELOAD environment variable available on all Linux machines to load in a custom compiled shared object file which overrides system calls such as statfs(), statvfs() and ioctl() that are the typical calls used to get this sort of information from the underlying OS, and viola! there it was. My hooks worked, and I was able to see what was being returned for each request. The only thing I had to do was fool the program into thinking it was getting the information it needed to go ahead with the file conversion. That again was just a matter of inspecting and inserting the correct values for each over-ridden system call. the whole process took about 4hrs, but it was time well spent. It felt good to get back to doing this sort of hacking-programming after a while....