The MogileFS::MogileFS#new_file method is enhanced in v3.1.0+ to support the :largefile parameter. While we have always supported large files via the “store_file” method, streaming large amounts of content of an unknown length required the use of awkward APIs.
It is possible to stream large content of known length any WebDAV server. One example of this is for mirroring a large file from an existing HTTP server to MogileFS without letting it hit the local filesystem.
uri = URI('http://example.com/large_file') Net::HTTP.start(uri.host, uri.port) do |http| req = Net::HTTP::Get.new(uri.request_uri) http.request(req) do |response| if len = response.content_length io = mg.new_file('key', :largefile => true, :content_length => len) else warn "trying to upload with Transfer-Encoding: chunked" warn "this is not supported by all WebDAV servers" io = mg.new_file('key', :largefile => :stream) end response.read_body { |buf| io.write(buf) } io.close end end
If your WebDAV servers have chunked PUT support (e.g. Perlbal), you can stream a file of unknown length using “Transfer-Encoding: chunked”.
nf = mg.new_file("key", :largefile => :stream) nf.write "hello" nf.write ... nf.close
If your WebDAV server has partial PUT support (e.g Apache), you can you can use multiple PUT requests with “Content-Range” support. This method is slower than Transfer-Encoding: chunked.
nf = mg.new_file("key", :largefile => :content_range) nf.write "hello" nf.write ... nf.close
Finally, if your WebDAV servers does not support either partial nor nor chunked PUTs, you must buffer a large file of unknown length using a Tempfile:
nf = mg.new_file("key", :largefile => :tempfile) nf.write "hello" nf.write ... nf.close
Generated with the Darkfish Rdoc Generator 2.