Smart Web Storage System

* Overview

A successful web storage system could have thousands of users with millions of files. With such enormous number of files, duplicity is common with ordinary storage system. A smart web storage system will have to be able to overcome the problem.

* Objective

When a user uploads a file which is quite big such as mp3, a unique hash for the file will be calculated. The hash will be compared and stored in database if it is not exist yet. In database, a hash corresponds to a file and its directory structure (where it’s placed). Thus, if a user upload a file with a hash which is already in database, a link to the file which is already exist in the system will be place to his/her storage box instead of the file he/she is uploading. The process will be very fast because the user doesn’t need to upload actually. Besides, the storage need for that particular file is reduced, all because other user have uploaded the similar file before.

* System requirement

- Apache or IIS webserver

- PHP/Perl or ASP.net

- Javascript or VBscript

- MySQL database

- SHA1 or MD5 hash calculation algorithm

- Icons or image editing software.

* Benefits

- Much faster upload for certain files of users

- Less storage for certain files of users.

* Future improvement if it’s feasible

- File type detection.

- File authenticity verification.

* Conclusion

Ever use yahoo briefcase? How long does it take to upload a normal mp3 file? Can it be improved?