Tuesday, July 14, 2009

NextGen Storage??

I've been thinking about what storage systems have in store for us in the future. I mean I've been trying to follow the progress of these beasts, and the next big thing I think should be storage devices that have in-built support for some form of transactional memory. Lots of file-system drivers are building consistency mechanisms into their implementation by using logging in some form or the other. This logging is happening on the same medium as the one where data is being stored. A consistent view of the data is formed by looking at (data+log) as a unit of storage.

I think that logging implementations would benefit a great deal if they had some sort of hardware support. For example, NetApp's WAFL is utilizing Flash memory to store it's writes. This I think is a nice way to look at the problem at hand, and if device manufacturers could integrate some of this on to their device using some nice interface, I think that file-systems of the future could be much faster than they presently are.

The only problem that I do however see with this approach is that the flash memory may wear out and cease to function much faster than the magnetic stable storage. A log-based file-system(or storage structure) would be needed to store the writes on the flash storage(something like JFFS maybe).

No comments: