Friday, 23 January 2009

EPrints gets Cloud Storage Support

As some people may know as part of the move towards EPrints 3.2 we have rewritten the way file storage works within EPrints entirely (after the recode we had less lines of source code!). Basically we have implemented an abstracted storage layer called the EPrints Storage Controller that is able to utilize new "storage plug-ins" to store files in different places.

As a starting point myself and Tim Brody have just successfully tested an EPrints install which is storing it's content in Amazon S3/Cloudfront! Details on the power and capabilities of providing these 2 services are outlined briefly here.

Storage Layer features:

  • Write your own plug-ins to marry EPrints with any chosen storage platform.

  • Use multiple plug-ins simultaneously.

  • Impose rules such that all volatile files (thumbnails etc) are stored locally, with the main file being stored in the cloud.

  • Impose rules which state that any document submitted by a Physicist on a Thursday are stored both locally and by two cloud storage providers. All rules are implemented in the existing epc xml scripting language.


What does Cloud Storage mean to your repository?


  • Cloud Storage means less bandwidth usage for provisioning of resources to users.

  • Cloud Storage enables those who don't have money or space to host 100's of Terabytes, to build and control a repository of this size.

  • Services such as Amazons Cloudfront can handle the replication and worldwide distribution of your objects.

  • Your users download documents from their local continental mirror, an advanced feature enabled by the new Storage Controller.

  • Using Amazon services also provides a means by which the users pay for their download bandwidth.


For the Amazon S3 plug-in we used a Storage Controller ruleset which imposed rules meaning only full files were stored in Amazon S3, with all thumbnails and previews remaining local. This reduces the cost of storing and retrieving you files.

Of course the Amazon S3 plug-in is one example of a storage plug-in, you could just as easily write your own.

10 comments:

andy said...
This comment has been removed by the author.
andy said...

I always enjoy learning how other people employ Amazon S3 and CloudFront. I am wondering if you can check out my very own tool CloudBerry Explorer that helps to manage S3 and CloudFront. It is a freeware. http://cloudberrylab.com/

Calvin Brock said...

open and common code base for businesses to build public and private cloud infrastructure – while sharing common approaches to management services and APIs. www.acnc.com

Curt Linking said...

I would like to thnkx for the efforts you've put in writing this website. I'm hoping the same high-grade site post from you in the future too. Actually your creative writing abilities has inspired me to get my own blog going now. Actually blogging is spreading its wings and growing rapidly. Your write up is a great example.
best mattress

Josh Anderson said...

Individual can this last without the credit check process for the individual suffering from bad credit tags like bankruptcy, arrears, late, late payments, foreclosures, bankruptcy, CCJ, IVA, etc.
http://www.3monthpaydayloans6uk.co.uk/

Jake Watson said...

Usually lenders approve people with low credit profile and have no problem, should be supported. On the other side of the coin, you will find no shortage of people running less than perfect loans for bad credit credit history. For these people, online lenders have come forward, and support as needed. So if you have arrears, defaults, CCJ, bankruptcy, late payments and so on, you need to relax. You will be approved online.
It is the style of living is high and the salary that you earn is not enough to assume all costs? If so, then use the bad credit loans no fax individual without cutting any of your expenses on your budget you can easily lifestyle of yours.

Jony walker said...

Moreover, it is unsecured and therefore not have to put any collateral at the front. Therefore he would not go through any risk in relation to your valuable assets.You can avoid any delay in the process of borrowing. Borrower, 6 month loans no guarantor for a pleasant experience, not because they need their credit records must be presented for examination. So you would be able, this outstanding unsecured loans, even if he gets a bad credit rating.
http://uk6monthloans.co.uk/
http://6monthtextloansnoguarantor.co.uk/
http://www.elongtermpaydayloansuk.co.uk/6-month-payday-loans.html

Josh Anderson said...

Finance, operations, financing of up to 100 by this means the repayment period of seven days may get, and to pay a bit high rate of same day loans interest, because it requires no collateral against the loan. By, for example, you can pay the medical bill, electricity bill, grocery bill, water bill, gas bill, phone bill, buy new clothes, business gifts and employ many more aid for 3 month payday loans emergencies. Bad credit factors like defaults, arrears, late payments, arrears, bankruptcy, CCJs, foreclosures, IVA, late payments, missed payments and so on to get taken advantage of loans text without having to go through any credit check formality.
http://www.donglepaydayloans.co.uk/

Josh Anderson said...

This post is really pleasant. I have a great opportunity for you. I am a mediator of 12 month loans it gives you 12 month loans in low interest rate. You are applying this loan in 24 hours. If you are interested please apply here. Every one of us realizes that benefiting an advance obliges bunches of customs that require some serious energy along these lines; individuals attempt to organize cash from different ways when they require it. All things considered, in the event that you are searching for some little money backing and you are having no real way to mastermind it by one means or another, you can seek little fleeting credits that don't request that you experience any long paperwork process or whatever other troublesome strategy to acquire it.
http://www.uk12monthpaydayloans.co/

Bret Smith said...

The form is prohibited without charge, as it is completely free. Everything is done immediately with mouse clicks. The time is stored largely as the need, somewhere is not a part of these loans. The loan is available for 2-3 weeks and can take the 12 month payday loans amount in the range of 500 to 1200 euros. On your next payday the amount back. Ensure that the amount of penalty time otherwise you could pay for free. These loans have a very nice feature, easy payday loans. People who are embarrassed by your bad credit loan can easily without disclosing your credit score. Arrears, late payments, defaults in the past are not taken into account.
http://www.easyloanssinuk.co.uk/