Uploading to Multiple S3 Buckets with Paperclip and Rails

When your application uses many static files (photos for example), you should consider placing these files on different hosts to improve the speed at which they are downloaded by web browsers.

Most web browsers have a limit on how many simultaneous connections can be made to a single named host. And this limit is usually two. It means that if you let's say display many photos on a single page, user's browser can only download two at a time. The broadband internet connection will be no help here. The browser will not fully use it, because it will keep opening and closing connections.

With little server and Rails configuration you pretend to be serving your files from different hosts and trick the browser. This is easy if you keep all the files on your own server, but when you use Paperclip to upload files to Amazon's S3 servers it gets more trickier. In fact Paperclip doesn't support uploading to different hosts (buckets).

First, notice that these addresses means the same for Amazon's S3:

• http://s3.amazonaws.com/bucket_name/filename.ext

• http://bucket_name.s3.amazonaws.com/filename.ext

So all we need to do is to make Paperclip upload to different buckets and return attachment's url of the latter type. All you need to do is download both Paperclip and PaperclipExtended and change your model definition.

class User < ActiveRecord::Base
    has_attached_file :avatar,
                      :storage => :s3,
                      :s3_credentials => "#{RAILS_ROOT}/config/s3.yml",
                      :path => "avatars/:id/:style_:extension",
                      :bucket => lambda do |attachment|
                        i = attachment.instance.id % 4
                        "bucket_#{i}"
                      end
end

This will place each avatar in one of four buckets: bucket_0, bucket_1, bucket_2 or bucket_3. The exact bucket is chosen at runtime and in this case it’s based on models id.

Getting attachment’s url:

puts User.find(1).avatar.url(:original)
# => http://bucket_1.s3.amazonaws.com/avatars/1/original.jpg

If you have a page on your website that displays many uploaded photos, placing them in different buckets should make noticeable difference to user - they will load faster.