Shrine is a toolkit for file attachments in Ruby applications. Some highlights:
- Modular design ā the plugin system allows you to load only the functionality you need
- Memory friendly ā streaming uploads and downloads make it work great with large files
- Cloud storage ā store files on disk, AWS S3, Google Cloud, Cloudinary and others
- ORM integrations ā works with Sequel, ActiveRecord, Hanami::Model and Mongoid
- Flexible processing ā generate thumbnails on upload or on-the-fly using ImageMagick or libvips
- Metadata validation ā validate files based on extracted metadata
- Direct uploads ā upload asynchronously to your app or to the cloud using Uppy
- Resumable uploads ā make large file uploads resumable on S3 or tus
- Background jobs ā built-in support for background processing that supports any backgrounding library
If you're curious how it compares to other file attachment libraries, see the Advantages of Shrine.
Resource | URL |
---|---|
Website | shrinerb.com |
Demo code | Roda / Rails |
Source | github.com/shrinerb/shrine |
Wiki | github.com/shrinerb/shrine/wiki |
Bugs | github.com/shrinerb/shrine/issues |
Help & Discussion | groups.google.com/group/ruby-shrine |
- Quick start
- Storage
- Uploader
- Uploaded file
- Attachment
- Attacher
- Plugin system
- Metadata
- Processing
- Validation
- Location
- Direct uploads
- Backgrounding
- Clearing cache
Add Shrine to the Gemfile and write an initializer which sets up the storage and loads the ORM plugin:
# Gemfile
gem "shrine", "~> 2.0"
require "shrine"
require "shrine/storage/file_system"
Shrine.storages = {
cache: Shrine::Storage::FileSystem.new("public", prefix: "uploads/cache"), # temporary
store: Shrine::Storage::FileSystem.new("public", prefix: "uploads"), # permanent
}
Shrine.plugin :sequel # or :activerecord
Shrine.plugin :cached_attachment_data # for retaining the cached file across form redisplays
Shrine.plugin :restore_cached_data # re-extract metadata when attaching a cached file
Shrine.plugin :rack_file # for non-Rails apps
Next decide how you will name the attachment attribute on your model, and run a
migration that adds an <attachment>_data
text or JSON column, which Shrine
will use to store all information about the attachment:
Sequel.migration do
change do
add_column :photos, :image_data, :text
end
end
In Rails with Active Record the migration would look similar:
$ rails generate migration add_image_data_to_photos image_data:text
class AddImageDataToPhotos < ActiveRecord::Migration
def change
add_column :photos, :image_data, :text
end
end
Now you can create an uploader class for the type of files you want to upload,
and add a virtual attribute for handling attachments using this uploader to
your model. If you do not care about adding plugins or additional processing,
you can use Shrine::Attachment
.
class ImageUploader < Shrine
# plugins and uploading logic
end
class Photo < Sequel::Model # ActiveRecord::Base
include ImageUploader::Attachment.new(:image) # adds an `image` virtual attribute
end
Let's now add the form fields which will use this virtual attribute. We need (1) a file field for choosing files, and (2) a hidden field for retaining the uploaded file in case of validation errors and for potential direct uploads.
# with Rails form builder:
form_for @photo do |f|
f.hidden_field :image, value: @photo.cached_image_data
f.file_field :image
f.submit
end
# with Simple Form:
simple_form_for @photo do |f|
f.input :image, as: :hidden, input_html: { value: @photo.cached_image_data }
f.input :image, as: :file
f.button :submit
end
# with Forme:
form @photo, action: "/photos", enctype: "multipart/form-data" do |f|
f.input :image, type: :hidden, value: @photo.cached_image_data
f.input :image, type: :file
f.button "Create"
end
Note that the file field needs to go after the hidden field, so that
selecting a new file can always override the cached file in the hidden field.
Also notice the enctype="multipart/form-data"
HTML attribute, which is
required for submitting files through the form (the Rails form builder
will automatically generate this for you).
When the form is submitted, in your router/controller you can assign the file from request params to the attachment attribute on the model.
# In Rails:
class PhotosController < ApplicationController
def create
Photo.create(photo_params)
# ...
end
private
def photo_params
params.require(:photo).permit(:image)
end
end
# In Sinatra:
post "/photos" do
Photo.create(params[:photo])
# ...
end
Once a file is uploaded and attached to the record, you can retrieve a URL to
the uploaded file with #<attachment>_url
and display it on the page:
<!-- In Rails: -->
<%= image_tag @photo.image_url %>
<!-- In HTML: -->
<img src="<%= @photo.image_url %>" />
A "storage" in Shrine is an object that encapsulates communication with a
specific storage service, by implementing a common public interface. Storage
instances are registered under an identifier in Shrine.storages
, so that they
can later be used by uploaders.
Previously we've shown the FileSystem storage which saves files to disk, but Shrine also ships with S3 storage which stores files on AWS S3 (or any S3-compatible service such as DigitalOcean Spaces or MinIO).
# Gemfile
gem "aws-sdk-s3", "~> 1.14" # for AWS S3 storage
require "shrine/storage/s3"
s3_options = {
bucket: "<YOUR BUCKET>", # required
access_key_id: "<YOUR ACCESS KEY ID>",
secret_access_key: "<YOUR SECRET ACCESS KEY>",
region: "<YOUR REGION>",
}
Shrine.storages = {
cache: Shrine::Storage::S3.new(prefix: "cache", **s3_options),
store: Shrine::Storage::S3.new(**s3_options),
}
The above example sets up S3 for both temporary and permanent storage, which is
suitable for direct uploads. The :cache
and
:store
names are special only in terms that the attacher will automatically
pick them up, you can also register more storage objects under different names.
See the FileSystem and S3 storage docs for more details. There are many more Shrine storages provided by external gems, and you can also create your own storage.
Uploaders are subclasses of Shrine
, and they wrap the actual upload to the
storage. They perform common tasks around upload that aren't related to a
particular storage.
class MyUploader < Shrine
# image attachent logic
end
It's common to create an uploader for each type of file that you want to handle
(ImageUploader
, VideoUploader
, AudioUploader
etc), but really you can
organize them in any way you like.
The main method of the uploader is #upload
, which takes an IO-like
object and a storage identifier on the input, and returns a
representation of the uploaded file on the output.
MyUploader.upload(file, :store) #=> #<Shrine::UploadedFile>
Internally this instantiates the uploader with the storage and calls #upload
on it:
uploader = MyUploader.new(:store)
uploader.upload(file) #=> #<Shrine::UploadedFile>
Some of the tasks performed by #upload
include:
- any defined file processing
- extracting metadata
- generating location
- uploading (this is where the storage is called)
- closing the uploaded file
The second argument is a context
hash which is forwarded to places like
metadata extraction and location generation, but it has a few special options:
uploader.upload(io, metadata: { "foo" => "bar" }) # add metadata
uploader.upload(io, location: "path/to/file") # specify custom location
uploader.upload(io, upload_options: { acl: "public-read" }) # add options to Storage#upload
Shrine is able to upload any IO-like object that implement methods #read
,
#rewind
, #eof?
and #close
whose behaviour matches the IO
class.
This includes built-in IO and IO-like objects like File, Tempfile and StringIO.
When a file is uploaded to a Rails app, in request params it will be
represented by an ActionDispatch::Http::UploadedFile
object, which is also an
IO-like object accepted by Shrine. In other Rack applications the uploaded file
will be represented as a Hash, but it can be converted into an IO-like object
with the rack_file
plugin.
Here are some examples of various IO-like objects that can be uploaded:
uploader.upload File.open("/path/to/file", binmode: true) # upload from disk
uploader.upload StringIO.new("file content") # upload from memory
uploader.upload ActionDispatch::Http::UploadedFile.new(...) # upload from Rails controller
uploader.upload Shrine.rack_file({ tempfile: tempfile }) # upload from Rack controller
uploader.upload Rack::Test::UploadedFile.new(...) # upload from rack-test
uploader.upload Down.open("https://example.org/file") # upload from internet
uploader.upload Shrine::UploadedFile.new(...) # upload from Shrine storage
The Shrine::UploadedFile
object represents the file that was uploaded to a
storage, and it's what's returned from Shrine#upload
or when retrieving a
record attachment. It contains the following information:
Key | Description |
---|---|
id |
location of the file on the storage |
storage |
identifier of the storage the file was uploaded to |
metadata |
file metadata that was extracted before upload |
uploaded_file = uploader.upload(file)
uploaded_file.data #=> {"id"=>"949sdjg834.jpg","storage"=>"store","metadata"=>{...}}
uploaded_file.id #=> "949sdjg834.jpg"
uploaded_file.storage #=> #<Shrine::Storage::S3>
uploaded_file.metadata #=> {...}
It comes with many convenient methods that delegate to the storage:
uploaded_file.url #=> "https://my-bucket.s3.amazonaws.com/949sdjg834.jpg"
uploaded_file.open { |io| ... } # opens the uploaded file stream
uploaded_file.download { |file| ... } # downloads the uploaded file to disk
uploaded_file.stream(destination) # streams uploaded content into a writable destination
uploaded_file.exists? #=> true
uploaded_file.delete # deletes the uploaded file from the storage
It also implements the IO-like interface that conforms to Shrine's IO abstraction, which allows it to be uploaded again to other storages.
uploaded_file.read # returns content of the uploaded file
uploaded_file.eof? # returns true if the whole IO was read
uploaded_file.rewind # rewinds the IO
uploaded_file.close # closes the IO
For more details, see the Retrieving Uploads guide and
Shrine::UploadedFile
API docs.
Storage objects, uploaders, and uploaded file objects are Shrine's foundational components. To help you actually attach uploaded files to database records in your application, Shrine comes with a high-level attachment interface built on top of these components.
There are plugins for hooking into most database libraries, and in case of ActiveRecord and Sequel the plugin will automatically tie the attached files to records' lifecycles. But you can also use Shrine just with plain old Ruby objects.
Shrine.plugin :sequel # :activerecord
class Photo < Sequel::Model # ActiveRecord::Base
include ImageUploader::Attachment.new(:image) #
include ImageUploader::Attachment(:image) # these are all equivalent
include ImageUploader[:image] #
end
You can choose whichever of these syntaxes you prefer. Either of these
will create a Shrine::Attachment
module with attachment methods for the
specified attribute, which then get added to your model when you include it:
Method | Description |
---|---|
#image= |
uploads the file to temporary storage and serializes the result into image_data |
#image |
returns Shrine::UploadedFile instantiated from image_data |
#image_url |
calls url on the attachment if it's present, otherwise returns nil |
#image_attacher |
returns instance of Shrine::Attacher which handles the attaching |
The ORM plugin that we loaded adds appropriate callbacks. For example, saving the record uploads the attachment to permanent storage, while deleting the record deletes the attachment.
# no file is attached
photo.image #=> nil
# the assigned file is cached to temporary storage and written to `image_data` column
photo.image = File.open("waterfall.jpg")
photo.image #=> #<Shrine::UploadedFile @data={...}>
photo.image_url #=> "/uploads/cache/0sdfllasfi842.jpg"
photo.image_data #=> '{"id":"0sdfllasfi842.jpg","storage":"cache","metadata":{...}}'
# the cached file is promoted to permanent storage and saved to `image_data` column
photo.save
photo.image #=> #<Shrine::UploadedFile @data={...}>
photo.image_url #=> "/uploads/store/l02kladf8jlda.jpg"
photo.image_data #=> '{"id":"l02kladf8jlda.jpg","storage":"store","metadata":{...}}'
# the attached file is deleted with the record
photo.destroy
photo.image.exists? #=> false
If there is already a file attached and a new file is attached, the previous attachment will get deleted when the record gets saved.
photo.update(image: new_file) # changes the attachment and deletes previous
photo.update(image: nil) # removes the attachment and deletes previous
The model attachment attributes and callbacks added by Shrine::Attachment
just delegate the behaviour to their underlying Shrine::Attacher
object.
photo.image_attacher #=> #<Shrine::Attacher>
The Shrine::Attacher
object can be instantiated and used directly:
attacher = ImageUploader::Attacher.new(photo, :image)
attacher.assign(file) # equivalent to `photo.image = file`
attacher.get # equivalent to `photo.image`
attacher.url # equivalent to `photo.image_url`
The attacher is what drives attaching files to model instances; you can use it as a more explicit alternative to models' attachment interface, or simply when you need something that's not available through the attachment methods.
You can do things such as change the temporary and permanent storage the attacher uses, or upload files directly to permanent storage. See the Using Attacher guide for more details.
By default Shrine comes with a small core which provides only the essential functionality. All additional features are available via plugins, which also ship with Shrine. This way you can choose exactly what and how much Shrine does for you, and you load the code only for features that you use.
Shrine.plugin :logging # adds logging
Plugins add behaviour by extending Shrine core classes via module inclusion, and many of them also accept configuration options. The plugin system respects inheritance, so you can choose to load a plugin globally or per uploader.
class ImageUploader < Shrine
plugin :store_dimensions # extract image dimensions only for this uploader and its descendants
end
If you want to extend Shrine functionality with custom behaviour, you can also create your own plugin.
Shrine automatically extracts some basic file metadata and saves them to the
Shrine::UploadedFile
. You can access them through the #metadata
hash or via
metadata methods:
uploaded_file.metadata #=>
# {
# "filename" => "matrix.mp4",
# "mime_type" => "video/mp4",
# "size" => 345993,
# }
uploaded_file.original_filename #=> "matrix.mp4"
uploaded_file.extension #=> "mp4"
uploaded_file.mime_type #=> "video/mp4"
uploaded_file.size #=> 345993
By default, mime_type
metadata will be inherited from the #content_type
attribute of the uploaded file, which is generally not secure and will trigger
a warning. You can load the determine_mime_type
plugin to have MIME type extracted from file content instead.
Shrine.plugin :determine_mime_type
photo = Photo.create(image: StringIO.new("<?php ... ?>"))
photo.image.mime_type #=> "text/x-php"
By the default the UNIX file
utility is used to determine the MIME type,
but you can also choose a different analyzer, see the
determine_mime_type
docs for more details.
In addition to basic metadata, you can also extract image dimensions, calculate signatures, and in general extract any custom metadata. Check out the Extracting Metadata guide for more details.
Shrine allows you to process attached files either "on upload" or "on-the-fly". For example, if your app is accepting image uploads, you can generate a pre-defined set of of thumbnails as soon as the image is attached to a record ("on upload"), or you can generate necessary thumbnails dynamically as they're needed ("on-the-fly").
For image processing it's recommended to use the ImageProcessing gem, which is a high-level wrapper for processing with ImageMagick (via MiniMagick) or libvips (via ruby-vips).
For processing "on upload", you can intercept when a cached file is being
uploaded to permanent storage, and perform any file processing you might want.
The processing
plugin provides the promotion hook, while
the versions
plugin enables handling a hash of versions.
$ brew install imagemagick
# Gemfile
gem "image_processing", "~> 1.2"
require "image_processing/mini_magick"
class ImageUploader < Shrine
plugin :processing # allows hooking into promoting
plugin :versions # enable Shrine to handle a hash of files
plugin :delete_raw # delete processed files after uploading
process(:store) do |io, context|
versions = { original: io } # retain original
io.download do |original|
pipeline = ImageProcessing::MiniMagick.source(original)
versions[:large] = pipeline.resize_to_limit!(800, 800)
versions[:medium] = pipeline.resize_to_limit!(500, 500)
versions[:small] = pipeline.resize_to_limit!(300, 300)
end
versions # return the hash of processed files
end
end
After the files are uploaded, their data is saved into the <attachment>_data
column, and the attachment getter will read them as a Hash of
Shrine::UploadedFile
objects.
photo = Photo.create(image: file) # processing is triggered
photo.image #=>
# {
# :original => #<Shrine::UploadedFile @data={"id"=>"9sd84.jpg", ...}>,
# :large => #<Shrine::UploadedFile @data={"id"=>"lg043.jpg", ...}>,
# :medium => #<Shrine::UploadedFile @data={"id"=>"kd9fk.jpg", ...}>,
# :small => #<Shrine::UploadedFile @data={"id"=>"932fl.jpg", ...}>,
# }
photo.image[:medium] #=> #<Shrine::UploadedFile>
photo.image[:medium].url #=> "/uploads/store/lg043.jpg"
photo.image[:medium].size #=> 5825949
photo.image[:medium].mime_type #=> "image/jpeg"
photo.image_url(:large) # returns version URL with fallbacks in case version is missing
By default processing is executed synchronously, but you can choose to delay it into a background job. You can also do any other type of file processing you want, see the File Processing guide for more details.
On-the-fly processing is provided by the
derivation_endpoint
plugin. It comes with a
mountable Rack app which applies processing on request
and returns processed files.
To set it up, we mount the Rack app in our router on a chosen path prefix, configure the plugin with a secret key and that path prefix, and define processing we want to perform:
$ brew install imagemagick
# Gemfile
gem "image_processing", "~> 1.2"
# config/routes.rb (Rails)
Rails.application.routes.draw do
# ...
mount ImageUploader.derivation_endpoint => "/derivations/image"
end
require "image_processing/mini_magick"
class ImageUploader < Shrine
plugin :derivation_endpoint,
secret_key: "<YOUR SECRET KEY>",
prefix: "derivations/image" # needs to match the mount point in routes
derivation :thumbnail do |file, width, height|
ImageProcessing::MiniMagick
.source(file)
.resize_to_limit!(width.to_i, height.to_i)
end
end
Now we can generate URLs from attached files that will perform the desired processing:
photo.image.derivation_url(:thumbnail, "600", "400")
#=> "/derivations/image/thumbnail/600/400/eyJpZCI6ImZvbyIsInN0b3JhZ2UiOiJzdG9yZSJ9?signature=..."
The on-the-fly processing feature is highly customizable, see the
derivation_endpoint
plugin documentation for
more details.
Shrine can perform file validations for files assigned to the model, with
validation_helpers
plugin providing some common
validation methods:
class DocumentUploader < Shrine
plugin :validation_helpers
Attacher.validate do
validate_max_size 5*1024*1024, message: "is too large (max is 5 MB)"
validate_mime_type_inclusion %w[application/pdf]
end
end
user = User.new
user.cv = File.open("cv.pdf", "rb")
user.valid? #=> false
user.errors.to_hash #=> {:cv=>["is too large (max is 5 MB)"]}
For more details, see the File Validation guide and
validation_helpers
plugin docs.
Shrine automatically generated random locations before uploading files. By
default the hierarchy is flat, meaning all files are stored in the root
directory of the storage. The pretty_location
plugin provides a good default hierarchy, but you can also override
#generate_location
with a custom implementation:
class ImageUploader < Shrine
def generate_location(io, context)
type = context[:record].class.name.downcase if context[:record]
style = context[:version] == :original ? "originals" : "thumbs" if context[:version]
name = super # the default unique identifier
[type, style, name].compact.join("/")
end
end
uploads/
photos/
originals/
la98lda74j3g.jpg
thumbs/
95kd8kafg80a.jpg
ka8agiaf9gk4.jpg
Note that there should always be a random component in the location, so that
the ORM dirty tracking is detected properly. Inside #generate_location
you
can also access the extracted metadata through context[:metadata]
.
To improve the user experience, it's recommended to upload files asynchronously as soon as the user selects them. The direct uploads would go to temporary storage, just like in the synchronous flow. Then, instead of attaching a raw file to your model, you assign the cached file JSON data.
# in the regular synchronous flow
photo.image = file
# in the direct upload flow
photo.image = '{"id":"...","storage":"cache","metadata":{...}}'
On the client side it's highly recommended to use Uppy š¶, a very flexible modern JavaScript file upload library that happens to integrate nicely with Shrine.
The simplest approach is to upload directly to an endpoint in your app, which
forwards uploads to the specified storage. The
upload_endpoint
Shrine plugin provides a
mountable Rack app that implements this endpoint:
Shrine.plugin :upload_endpoint
# config/routes.rb (Rails)
Rails.application.routes.draw do
# ...
mount ImageUploader.upload_endpoint(:cache) => "/images/upload" # POST /images/upload
end
Then you can configure Uppy's XHR Upload plugin to upload to this endpoint. See this walkthrough for adding simple direct uploads from scratch, it includes a complete JavaScript example (there is also the Roda / Rails demo app).
For better performance, you can also upload files directly to your cloud storage service (AWS S3, Google Cloud Storage etc). For this, your temporary storage needs to be your cloud service:
require "shrine/storage/s3"
Shrine.storages = {
cache: Shrine::Storage::S3.new(prefix: "cache", **s3_options),
store: Shrine::Storage::S3.new(**s3_options)
}
In this flow, the client needs to first fetch upload parameters from the
server, and then use these parameters for the upload to the cloud service.
The presign_endpoint
Shrine plugin provides a
mountable Rack app that generates upload parameters:
Shrine.plugin :presign_endpoint
# config/routes.rb (Rails)
Rails.application.routes.draw do
# ...
mount Shrine.presign_endpoint(:cache) => "/s3/params" # GET /s3/params
end
Then you can configure Uppy's AWS S3 plugin to fetch params from your endpoint before uploading to S3. See this walkthrough for adding direct uploads to S3 from scratch, it includes a complete JavaScript example (there is also the Roda / Rails demo). See also the Direct Uploads to S3 guide for more details.
If your app is accepting large uploads, you can improve resilience by making the uploads resumable. This can significantly improve experience for users on slow and flaky internet connections.
You can achieve resumable uploads directly to S3 with the AWS S3
Multipart Uppy plugin, accompanied with
uppy_s3_multipart
Shrine plugin provided by the uppy-s3_multipart gem.
# Gemfile
gem "uppy-s3_multipart", "~> 0.3"
Shrine.plugin :uppy_s3_multipart
# config/routes.rb (Rails)
Rails.application.routes.draw do
# ...
mount Shrine.uppy_s3_multipart(:cache) => "/s3/multipart"
end
See the uppy-s3_multipart docs for more details.
If you want a more generic approach, you can build your resumable uploads on tus ā an open resumable upload protocol. On the server side you can use the tus-ruby-server gem, on the client side Uppy's Tus plugin, and the shrine-tus gem for the glue.
# Gemfile
gem "tus-server", "~> 2.0"
gem "shrine-tus", "~> 1.2"
require "shrine/storage/tus"
Shrine.storages = {
cache: Shrine::Storage::Tus.new, # tus server acts as temporary storage
store: ..., # your permanent storage
}
# config/routes.rb (Rails)
Rails.application.routes.draw do
# ...
mount Tus::Server => "/files"
end
See this walkthrough for adding tus-powered resumable uploads from scratch, it includes a complete JavaScript example (there is also a demo app). See also shrine-tus and tus-ruby-server docs for more details.
Shrine allows you to put file deletion and promotion of cached files to
permanent storage into a background job. The backgrounding
plugin provides hooks for plugging in your favourite backgrounding
library.
Shrine.plugin :backgrounding
Shrine::Attacher.promote { |data| PromoteJob.perform_async(data) }
Shrine::Attacher.delete { |data| DeleteJob.perform_async(data) }
class PromoteJob
include Sidekiq::Worker
def perform(data)
Shrine::Attacher.promote(data)
end
end
class DeleteJob
include Sidekiq::Worker
def perform(data)
Shrine::Attacher.delete(data)
end
end
Shrine doesn't automatically delete files uploaded to temporary storage, instead you should set up a separate recurring task that will automatically delete old cached files.
Most Shrine storage classes come with a #clear!
method, which you can call in
a recurring script. For FileSystem and S3 storage it would look like this:
# FileSystem storage
file_system = Shrine.storages[:cache]
file_system.clear! { |path| path.mtime < Time.now - 7*24*60*60 } # delete files older than 1 week
# S3 storage
s3 = Shrine.storages[:cache]
s3.clear! { |object| object.last_modified < Time.now - 7*24*60*60 } # delete files older than 1 week
Shrine was heavily inspired by Refile and Roda. From Refile it borrows the idea of "backends" (here named "storages"), attachment interface, and direct uploads. From Roda it borrows the implementation of an extensible plugin system.
- Paperclip
- CarrierWave
- Dragonfly
- Refile
- Active Storage
Everyone interacting in the Shrine projectās codebases, issue trackers, and mailing lists is expected to follow the Shrine code of conduct.
The gem is available as open source under the terms of the MIT License.