Struct ObjectStore

pub struct ObjectStore {
    accessor: Arc<dyn AccessDyn>,
    default_executor: Option<Executor>,
}

Expand description

The Operator serves as the entry point for all public asynchronous APIs.

For more details about the Operator, refer to the [concepts][crate::docs::concepts] section.

§Build

Users can initialize an Operator through the following methods:

Operator::new: Creates an operator using a services builder, such as services::S3.
Operator::from_config: Creates an operator using a services configuration, such as services::S3Config.
Operator::from_iter: Creates an operator from an iterator of configuration key-value pairs.

use opendal::services::Memory;
use opendal::Operator;
async fn test() -> Result<()> {
    // Build an `Operator` to start operating the storage.
    let _: Operator = Operator::new(Memory::default())?.finish();

    Ok(())
}

§Layer

After the operator is built, users can add the layers they need on top of it.

OpenDAL offers various layers for users to choose from, such as RetryLayer, LoggingLayer, and more. Visit [layers] for further details.

use opendal::layers::RetryLayer;
use opendal::services::Memory;
use opendal::Operator;
async fn test() -> Result<()> {
    let op: Operator = Operator::new(Memory::default())?.finish();

    // OpenDAL will retry failed operations now.
    let op = op.layer(RetryLayer::default());

    Ok(())
}

§Operate

After the operator is built and the layers are added, users can start operating the storage.

The operator is Send, Sync, and Clone. It has no internal state, and all APIs only take a &self reference, making it safe to share the operator across threads.

Operator provides a consistent API pattern for data operations. For reading operations, it exposes:

Operator::read: Basic operation that reads entire content into memory
Operator::read_with: Enhanced read operation with additional options (range, if_match, if_none_match)
Operator::reader: Creates a lazy reader for on-demand data streaming
Operator::reader_with: Creates a configurable reader with conditional options (if_match, if_none_match)

The Reader created by Operator supports custom read control methods and can be converted into futures::AsyncRead for broader ecosystem compatibility.

use opendal::layers::RetryLayer;
use opendal::services::Memory;
use opendal::Operator;
async fn test() -> Result<()> {
    let op: Operator = Operator::new(Memory::default())?.finish();

    // OpenDAL will retry failed operations now.
    let op = op.layer(RetryLayer::default());

    // Read all data into memory.
    let data = op.read("path/to/file").await?;

    Ok(())
}

Fields§

§accessor: Arc<dyn AccessDyn>§default_executor: Option<Executor>

Implementations§

§

impl Operator

§Operator basic API.

pub fn inner(&self) -> &Arc<dyn AccessDyn>

Fetch the internal accessor.

pub fn from_inner(accessor: Arc<dyn AccessDyn>) -> Operator

Convert inner accessor into operator.

pub fn into_inner(self) -> Arc<dyn AccessDyn>

Convert operator into inner accessor.

pub fn limit(&self) -> usize

👎Deprecated since 0.52.0: limit is no-op for now

Get current operator’s limit. Limit is usually the maximum size of data that operator will handle in one operation.

pub fn with_limit(&self, _: usize) -> Operator

👎Deprecated since 0.52.0: limit is no-op for now

Specify the batch limit.

Default: 1000

pub fn default_executor(&self) -> Option<Executor>

Get the default executor.

pub fn with_default_executor(&self, executor: Executor) -> Operator

Specify the default executor.

pub fn info(&self) -> OperatorInfo

Get information of underlying accessor.

§Examples

use opendal::Operator;

let info = op.info();

pub fn blocking(&self) -> BlockingOperator

Create a new blocking operator.

This operation is nearly no cost.

§

impl Operator

§Operator async API.

pub async fn check(&self) -> Result<(), Error>

Check if this operator can work correctly.

We will send a list request to path and return any errors we met.

use opendal::Operator;

op.check().await?;

pub async fn stat(&self, path: &str) -> Result<Metadata, Error>

Get given path’s metadata.

§Notes

§Extra Options

Operator::stat is a wrapper of Operator::stat_with without any options. To use extra options like if_match and if_none_match, please use Operator::stat_with instead.

§Examples

§Check if file exists

use opendal::ErrorKind;
if let Err(e) = op.stat("test").await {
    if e.kind() == ErrorKind::NotFound {
        println!("file not exist")
    }
}

pub fn stat_with( &self, path: &str, ) -> OperatorFuture<OpStat, Metadata, impl Future<Output = Result<Metadata, Error>>>

Get given path’s metadata with extra options.

§Options

§`if_match`

Set if_match for this stat request.

This feature can be used to check if the file’s ETag matches the given ETag.

If file exists, and it’s etag doesn’t match, an error with kind ErrorKind::ConditionNotMatch will be returned.

use opendal::Operator;

let mut metadata = op.stat_with("path/to/file").if_match(etag).await?;

§`if_none_match`

Set if_none_match for this stat request.

This feature can be used to check if the file’s ETag doesn’t match the given ETag.

If file exists, and it’s etag match, an error with kind ErrorKind::ConditionNotMatch will be returned.

use opendal::Operator;

let mut metadata = op.stat_with("path/to/file").if_none_match(etag).await?;

§`if_modified_since`

set if_modified_since for this stat request.

This feature can be used to check if the file has been modified since the given time.

If file exists, and it’s not modified after the given time, an error with kind ErrorKind::ConditionNotMatch will be returned.

use opendal::Operator;
use chrono::Utc;

let mut metadata = op.stat_with("path/to/file").if_modified_since(Utc::now()).await?;

§`if_unmodified_since`

set if_unmodified_since for this stat request.

This feature can be used to check if the file has NOT been modified since the given time.

If file exists, and it’s modified after the given time, an error with kind ErrorKind::ConditionNotMatch will be returned.

use opendal::Operator;
use chrono::Utc;

let mut metadata = op.stat_with("path/to/file").if_unmodified_since(Utc::now()).await?;

§`version`

Set version for this stat request.

This feature can be used to retrieve the metadata of a specific version of the given path

If the version doesn’t exist, an error with kind ErrorKind::NotFound will be returned.


let mut metadata = op.stat_with("path/to/file").version(version).await?;

§Examples

§Get metadata while `ETag` matches

stat_with will

return Ok(metadata) if ETag matches
return Err(error) and error.kind() == ErrorKind::ConditionNotMatch if file exists but ETag mismatch
return Err(err) if other errors occur, for example, NotFound.

use opendal::ErrorKind;
if let Err(e) = op.stat_with("test").if_match("<etag>").await {
    if e.kind() == ErrorKind::ConditionNotMatch {
        println!("file exists, but etag mismatch")
    }
    if e.kind() == ErrorKind::NotFound {
        println!("file not exist")
    }
}

§Behavior

§Services that support `create_dir`

test and test/ may vary in some services such as S3. However, on a local file system, they’re identical. Therefore, the behavior of stat("test") and stat("test/") might differ in certain edge cases. Always use stat("test/") when you need to access a directory if possible.

Here are the behavior list:

Case	Path	Result
stat existing dir	`abc/`	Metadata with dir mode
stat existing file	`abc/def_file`	Metadata with file mode
stat dir without `/`	`abc/def_dir`	Error `NotFound` or metadata with dir mode
stat file with `/`	`abc/def_file/`	Error `NotFound`
stat not existing path	`xyz`	Error `NotFound`

Refer to [RFC: List Prefix][crate::docs::rfcs::rfc_3243_list_prefix] for more details.

§Services that not support `create_dir`

For services that not support create_dir, stat("test/") will return NotFound even when test/abc exists since the service won’t have the concept of dir. There is nothing we can do about this.

pub async fn exists(&self, path: &str) -> Result<bool, Error>

Check if this path exists or not.

§Example

use anyhow::Result;
use futures::io;
use opendal::Operator;

async fn test(op: Operator) -> Result<()> {
    let _ = op.exists("test").await?;

    Ok(())
}

pub async fn is_exist(&self, path: &str) -> Result<bool, Error>

👎Deprecated: rename to exists for consistence with std::fs::exists

Check if this path exists or not.

§Example

use anyhow::Result;
use futures::io;
use opendal::Operator;

async fn test(op: Operator) -> Result<()> {
    let _ = op.is_exist("test").await?;

    Ok(())
}

pub async fn create_dir(&self, path: &str) -> Result<(), Error>

Create a dir at given path.

§Notes

To indicate that a path is a directory, it is compulsory to include a trailing / in the path. Failure to do so may result in NotADirectory error being returned by OpenDAL.

§Behavior

Create on existing dir will succeed.
Create dir is always recursive, works like mkdir -p

§Examples

op.create_dir("path/to/dir/").await?;

pub async fn read(&self, path: &str) -> Result<Buffer, Error>

Read the whole path into a bytes.

§Notes

§Extra Options

Operator::read is a wrapper of Operator::read_with without any options. To use extra options like range and if_match, please use Operator::read_with instead.

§Streaming Read

This function will allocate a new bytes internally. For more precise memory control or reading data lazily, please use Operator::reader

§Examples

let bs = op.read("path/to/file").await?;

pub fn read_with( &self, path: &str, ) -> OperatorFuture<(OpRead, OpReader), Buffer, impl Future<Output = Result<Buffer, Error>>>

Read the whole path into a bytes with extra options.

This function will allocate a new bytes internally. For more precise memory control or reading data lazily, please use Operator::reader

§Notes

§Streaming Read

This function will allocate a new bytes internally. For more precise memory control or reading data lazily, please use Operator::reader

§Options

Visit [FutureRead] for all available options.

range: Set range for the read.
concurrent: Set concurrent for the read.
chunk: Set chunk for the read.
version: Set version for the read.
if_match: Set if-match for the read.
if_none_match: Set if-none-match for the read.
if_modified_since: Set if-modified-since for the read.
if_unmodified_since: Set if-unmodified-since for the read.

§Examples

Read the whole path into a bytes.

let bs = op.read_with("path/to/file").await?;
let bs = op.read_with("path/to/file").range(0..10).await?;

pub async fn reader(&self, path: &str) -> Result<Reader, Error>

Create a new reader which can read the whole path.

§Notes

§Extra Options

Operator::reader is a wrapper of Operator::reader_with without any options. To use extra options like concurrent, please use Operator::reader_with instead.

§Examples

let r = op.reader("path/to/file").await?;

pub fn reader_with( &self, path: &str, ) -> OperatorFuture<(OpRead, OpReader), Reader, impl Future<Output = Result<Reader, Error>>>

Create a new reader with extra options

§Notes

§Extra Options

Operator::reader is a wrapper of Operator::reader_with without any options. To use extra options like version, please use Operator::reader_with instead.

§Options

Visit [FutureReader] for all available options.

version: Set version for the reader.
concurrent: Set concurrent for the reader.
chunk: Set chunk for the reader.
gap: Set gap for the reader.
if_match: Set if-match for the reader.
if_none_match: Set if-none-match for the reader.
if_modified_since: Set if-modified-since for the reader.
if_unmodified_since: Set if-unmodified-since for the reader.

§Examples

let r = op.reader_with("path/to/file").version("version_id").await?;

pub async fn write( &self, path: &str, bs: impl Into<Buffer>, ) -> Result<Metadata, Error>

Write bytes into path.

§Notes

§Extra Options

Operator::write is a simplified version of Operator::write_with without additional options. For advanced features like content_type and cache_control, use Operator::write_with instead.

§Streaming Write

This method performs a single bulk write operation. For finer-grained memory control or streaming data writes, use Operator::writer instead.

§Multipart Uploads

OpenDAL provides multipart upload functionality through the Writer abstraction, handling all upload details automatically. You can customize the upload behavior by configuring chunk size and concurrent operations via Operator::writer_with.

§Examples

use bytes::Bytes;

op.write("path/to/file", vec![0; 4096]).await?;

pub async fn copy(&self, from: &str, to: &str) -> Result<(), Error>

Copy a file from from to to.

§Notes

from and to must be a file.
to will be overwritten if it exists.
If from and to are the same, an IsSameFile error will occur.
copy is idempotent. For same from and to input, the result will be the same.

§Examples


op.copy("path/to/file", "path/to/file2").await?;

pub async fn rename(&self, from: &str, to: &str) -> Result<(), Error>

Rename a file from from to to.

§Notes

from and to must be a file.
to will be overwritten if it exists.
If from and to are the same, an IsSameFile error will occur.

§Examples


op.rename("path/to/file", "path/to/file2").await?;

pub async fn writer(&self, path: &str) -> Result<Writer, Error>

Create a writer for streaming data to the given path.

§Notes

§Writer Features

The writer provides several powerful capabilities:

Streaming writes for continuous data transfer
Automatic multipart upload handling
Memory-efficient chunk-based writing

§Extra Options

Operator::writer is a simplified version of Operator::writer_with without additional options. For advanced features like content_type and cache_control, use Operator::writer_with instead.

§Chunk Size Handling

Storage services often have specific requirements for chunk sizes:

Services like s3 may return EntityTooSmall errors for undersized chunks
Using small chunks in cloud storage services can lead to increased costs

OpenDAL automatically determines optimal chunk sizes based on the service’s Capability. However, you can override this by explicitly setting the chunk parameter.

For improved performance, consider setting an appropriate chunk size using Operator::writer_with.

§Examples

use bytes::Bytes;

let mut w = op.writer("path/to/file").await?;
w.write(vec![0; 4096]).await?;
w.write(vec![1; 4096]).await?;
w.close().await?;

pub fn writer_with( &self, path: &str, ) -> OperatorFuture<(OpWrite, OpWriter), Writer, impl Future<Output = Result<Writer, Error>>>

Create a writer for streaming data to the given path with more options.

§Options

Visit [FutureWriter] for all available options.

append: Sets append mode for this write request.
chunk: Sets chunk size for buffered writes.
concurrent: Sets concurrent write operations for this writer.
cache_control: Sets cache control for this write request.
content_type: Sets content type for this write request.
content_disposition: Sets content disposition for this write request.
content_encoding: Sets content encoding for this write request.
if_match: Sets if-match for this write request.
if_none_match: Sets if-none-match for this write request.
if_not_exist: Sets if-not-exist for this write request.
user_metadata: Sets user metadata for this write request.

§Examples

use bytes::Bytes;

let mut w = op.writer_with("path/to/file")
    .chunk(4*1024*1024)
    .concurrent(8)
    .await?;
w.write(vec![0; 4096]).await?;
w.write(vec![1; 4096]).await?;
w.close().await?;

pub fn write_with( &self, path: &str, bs: impl Into<Buffer>, ) -> OperatorFuture<(OpWrite, OpWriter, Buffer), Metadata, impl Future<Output = Result<Metadata, Error>>>

Write data with extra options.

§Notes

§Streaming Write

This method performs a single bulk write operation for all bytes. For finer-grained memory control or lazy writing, consider using Operator::writer_with instead.

§Multipart Uploads

OpenDAL handles multipart uploads through the Writer abstraction, managing all the upload details automatically. You can customize the upload behavior by configuring chunk size and concurrent operations via Operator::writer_with.

§Options

Visit [FutureWrite] for all available options.

append: Sets append mode for this write request.
chunk: Sets chunk size for buffered writes.
concurrent: Sets concurrent write operations for this writer.
cache_control: Sets cache control for this write request.
content_type: Sets content type for this write request.
content_disposition: Sets content disposition for this write request.
content_encoding: Sets content encoding for this write request.
if_match: Sets if-match for this write request.
if_none_match: Sets if-none-match for this write request.
if_not_exist: Sets if-not-exist for this write request.
user_metadata: Sets user metadata for this write request.

§Examples

use bytes::Bytes;

let _ = op.write_with("path/to/file", vec![0; 4096])
    .if_not_exists(true)
    .await?;

pub async fn delete(&self, path: &str) -> Result<(), Error>

Delete the given path.

§Notes

Deleting a file that does not exist won’t return errors.

§Examples

op.delete("test").await?;

pub fn delete_with( &self, path: &str, ) -> OperatorFuture<OpDelete, (), impl Future<Output = Result<(), Error>>>

Delete the given path with extra options.

§Notes

Deleting a file that does not exist won’t return errors.

§Options

§`version`

Set version for this delete request.

remove a specific version of the given path.

If the version doesn’t exist, OpenDAL will not return errors.


op.delete_with("path/to/file").version(version).await?;

§Examples


op.delete_with("test").await?;

pub async fn delete_iter<I, D>(&self, iter: I) -> Result<(), Error>
where I: IntoIterator<Item = D>, D: IntoDeleteInput,

Delete an infallible iterator of paths.

Also see:

Operator::delete_try_iter: delete an fallible iterator of paths.
Operator::delete_stream: delete an infallible stream of paths.
Operator::delete_try_stream: delete an fallible stream of paths.

pub async fn delete_try_iter<I, D>(&self, try_iter: I) -> Result<(), Error>
where I: IntoIterator<Item = Result<D, Error>>, D: IntoDeleteInput,

Delete a fallible iterator of paths.

Also see:

Operator::delete_iter: delete an infallible iterator of paths.
Operator::delete_stream: delete an infallible stream of paths.
Operator::delete_try_stream: delete an fallible stream of paths.

pub async fn delete_stream<S, D>(&self, stream: S) -> Result<(), Error>
where S: Stream<Item = D>, D: IntoDeleteInput,

Delete an infallible stream of paths.

Also see:

Operator::delete_iter: delete an infallible iterator of paths.
Operator::delete_try_iter: delete an fallible iterator of paths.
Operator::delete_try_stream: delete an fallible stream of paths.

pub async fn delete_try_stream<S, D>(&self, try_stream: S) -> Result<(), Error>
where S: Stream<Item = Result<D, Error>>, D: IntoDeleteInput,

Delete an fallible stream of paths.

Also see:

Operator::delete_iter: delete an infallible iterator of paths.
Operator::delete_try_iter: delete an fallible iterator of paths.
Operator::delete_stream: delete an infallible stream of paths.

pub async fn deleter(&self) -> Result<Deleter, Error>

Create a [Deleter] to continuously remove content from storage.

It leverages batch deletion capabilities provided by storage services for efficient removal.

Users can have more control over the deletion process by using [Deleter] directly.

pub async fn remove(&self, paths: Vec<String>) -> Result<(), Error>

👎Deprecated since 0.52.0: use Operator::delete_iter instead

§Notes

If underlying services support delete in batch, we will use batch delete instead.

§Examples

op.remove(vec!["abc".to_string(), "def".to_string()])
    .await?;

pub async fn remove_via( &self, input: impl Stream<Item = String> + Unpin, ) -> Result<(), Error>

👎Deprecated since 0.52.0: use Operator::delete_stream instead

remove will remove files via the given paths.

remove_via will remove files via the given stream.

We will delete by chunks with given batch limit on the stream.

§Notes

If underlying services support delete in batch, we will use batch delete instead.

§Examples

use futures::stream;
let stream = stream::iter(vec!["abc".to_string(), "def".to_string()]);
op.remove_via(stream).await?;

pub async fn remove_all(&self, path: &str) -> Result<(), Error>

Remove the path and all nested dirs and files recursively.

§Notes

If underlying services support delete in batch, we will use batch delete instead.

§Examples

op.remove_all("path/to/dir").await?;

pub async fn list(&self, path: &str) -> Result<Vec<Entry>, Error>

List entries that starts with given path in parent dir.

§Notes

§Recursively List

This function only read the children of the given directory. To read all entries recursively, use Operator::list_with("path").recursive(true) instead.

§Streaming List

This function will read all entries in the given directory. It could take very long time and consume a lot of memory if the directory contains a lot of entries.

In order to avoid this, you can use Operator::lister to list entries in a streaming way.

§Examples

§List entries under a dir

This example will list all entries under the dir path/to/dir/.

use opendal::EntryMode;
use opendal::Operator;
let mut entries = op.list("path/to/dir/").await?;
for entry in entries {
    match entry.metadata().mode() {
        EntryMode::FILE => {
            println!("Handling file")
        }
        EntryMode::DIR => {
            println!("Handling dir {}", entry.path())
        }
        EntryMode::Unknown => continue,
    }
}

§List entries with prefix

This example will list all entries under the prefix path/to/prefix.

NOTE: it’s possible that the prefix itself is also a dir. In this case, you could get path/to/prefix/, path/to/prefix_1 and so on. If you do want to list a dir, please make sure the path is end with /.

use opendal::EntryMode;
use opendal::Operator;
let mut entries = op.list("path/to/prefix").await?;
for entry in entries {
    match entry.metadata().mode() {
        EntryMode::FILE => {
            println!("Handling file")
        }
        EntryMode::DIR => {
            println!("Handling dir {}", entry.path())
        }
        EntryMode::Unknown => continue,
    }
}

pub fn list_with( &self, path: &str, ) -> OperatorFuture<OpList, Vec<Entry>, impl Future<Output = Result<Vec<Entry>, Error>>>

List entries that starts with given path in parent dir with more options.

§Notes

§Streaming list

This function will read all entries in the given directory. It could take very long time and consume a lot of memory if the directory contains a lot of entries.

In order to avoid this, you can use Operator::lister to list entries in a streaming way.

§Options

§`start_after`

Specify the specified key to start listing from.

This feature can be used to resume a listing from a previous point.

The following example will resume the list operation from the breakpoint.

use opendal::Operator;
let mut entries = op
    .list_with("path/to/dir/")
    .start_after("breakpoint")
    .await?;

§`recursive`

Specify whether to list recursively or not.

If recursive is set to true, we will list all entries recursively. If not, we’ll only list the entries in the specified dir.

use opendal::Operator;
let mut entries = op.list_with("path/to/dir/").recursive(true).await?;

§`version`

Specify whether to list files along with all their versions

if version is enabled, all file versions will be returned; otherwise, only the current files will be returned.

let mut entries = op.list_with("path/to/dir/").version(true).await?;

§Examples

§List all entries recursively

This example will list all entries under the dir path/to/dir/

use opendal::EntryMode;
use opendal::Operator;
let mut entries = op.list_with("path/to/dir/").recursive(true).await?;
for entry in entries {
    match entry.metadata().mode() {
        EntryMode::FILE => {
            println!("Handling file")
        }
        EntryMode::DIR => {
            println!("Handling dir like start a new list via meta.path()")
        }
        EntryMode::Unknown => continue,
    }
}

§List all entries start with prefix

This example will list all entries starts with prefix path/to/prefix

use opendal::EntryMode;
use opendal::Operator;
let mut entries = op.list_with("path/to/prefix").recursive(true).await?;
for entry in entries {
    match entry.metadata().mode() {
        EntryMode::FILE => {
            println!("Handling file")
        }
        EntryMode::DIR => {
            println!("Handling dir like start a new list via meta.path()")
        }
        EntryMode::Unknown => continue,
    }
}

pub async fn lister(&self, path: &str) -> Result<Lister, Error>

List entries that starts with given path in parent dir.

This function will create a new Lister to list entries. Users can stop listing via dropping this Lister.

§Notes

§Recursively list

This function only read the children of the given directory. To read all entries recursively, use Operator::lister_with and recursive(true) instead.

§Examples

use futures::TryStreamExt;
use opendal::EntryMode;
use opendal::Operator;
let mut ds = op.lister("path/to/dir/").await?;
while let Some(mut de) = ds.try_next().await? {
    match de.metadata().mode() {
        EntryMode::FILE => {
            println!("Handling file")
        }
        EntryMode::DIR => {
            println!("Handling dir like start a new list via meta.path()")
        }
        EntryMode::Unknown => continue,
    }
}

pub fn lister_with( &self, path: &str, ) -> OperatorFuture<OpList, Lister, impl Future<Output = Result<Lister, Error>>>

List entries that starts with given path in parent dir with options.

This function will create a new Lister to list entries. Users can stop listing via dropping this Lister.

§Options

§`start_after`

Specify the specified key to start listing from.

This feature can be used to resume a listing from a previous point.

The following example will resume the list operation from the breakpoint.

use opendal::Operator;
let mut lister = op
    .lister_with("path/to/dir/")
    .start_after("breakpoint")
    .await?;

§`recursive`

Specify whether to list recursively or not.

If recursive is set to true, we will list all entries recursively. If not, we’ll only list the entries in the specified dir.

use opendal::Operator;
let mut lister = op.lister_with("path/to/dir/").recursive(true).await?;

§`version`

Specify whether to list files along with all their versions

if version is enabled, all file versions will be returned; otherwise, only the current files will be returned.

let mut entries = op.lister_with("path/to/dir/").version(true).await?;

§Examples

§List all files recursively

use futures::TryStreamExt;
use opendal::EntryMode;
use opendal::Operator;
let mut lister = op.lister_with("path/to/dir/").recursive(true).await?;
while let Some(mut entry) = lister.try_next().await? {
    match entry.metadata().mode() {
        EntryMode::FILE => {
            println!("Handling file {}", entry.path())
        }
        EntryMode::DIR => {
            println!("Handling dir {}", entry.path())
        }
        EntryMode::Unknown => continue,
    }
}

§

impl Operator

Operator presign API.

pub async fn presign_stat( &self, path: &str, expire: Duration, ) -> Result<PresignedRequest, Error>

Presign an operation for stat(head).

§Example

use anyhow::Result;
use futures::io;
use opendal::Operator;
use std::time::Duration;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op.presign_stat("test",Duration::from_secs(3600)).await?;
    let req = http::Request::builder()
        .method(signed_req.method())
        .uri(signed_req.uri())
        .body(())?;

pub fn presign_stat_with( &self, path: &str, expire: Duration, ) -> OperatorFuture<(OpStat, Duration), PresignedRequest, impl Future<Output = Result<PresignedRequest, Error>>>

Presign an operation for stat(head).

§Example

use anyhow::Result;
use futures::io;
use opendal::Operator;
use std::time::Duration;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op.presign_stat_with("test",Duration::from_secs(3600)).override_content_disposition("attachment; filename=\"othertext.txt\"").await?;

pub async fn presign_read( &self, path: &str, expire: Duration, ) -> Result<PresignedRequest, Error>

Presign an operation for read.

§Notes

§Extra Options

presign_read is a wrapper of Self::presign_read_with without any options. To use extra options like override_content_disposition, please use Self::presign_read_with instead.

§Example

use anyhow::Result;
use futures::io;
use opendal::Operator;
use std::time::Duration;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op.presign_read("test.txt", Duration::from_secs(3600)).await?;

signed_req.method(): GET
signed_req.uri(): https://s3.amazonaws.com/examplebucket/test.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=access_key_id/20130721/us-east-1/s3/aws4_request&X-Amz-Date=20130721T201207Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=<signature-value>
signed_req.headers(): { "host": "s3.amazonaws.com" }

We can download this file via curl or other tools without credentials:

curl "https://s3.amazonaws.com/examplebucket/test.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=access_key_id/20130721/us-east-1/s3/aws4_request&X-Amz-Date=20130721T201207Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=<signature-value>" -O /tmp/test.txt

pub fn presign_read_with( &self, path: &str, expire: Duration, ) -> OperatorFuture<(OpRead, Duration), PresignedRequest, impl Future<Output = Result<PresignedRequest, Error>>>

Presign an operation for read with extra options.

§Options

§`override_content_disposition`

Override the content-disposition header returned by storage services.

use std::time::Duration;

use anyhow::Result;
use opendal::Operator;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op
        .presign_read_with("test.txt", Duration::from_secs(3600))
        .override_content_disposition("attachment; filename=\"othertext.txt\"")
        .await?;
    Ok(())
}

§`override_cache_control`

Override the cache-control header returned by storage services.

use std::time::Duration;

use anyhow::Result;
use opendal::Operator;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op
        .presign_read_with("test.txt", Duration::from_secs(3600))
        .override_cache_control("no-store")
        .await?;
    Ok(())
}

§`override_content_type`

Override the content-type header returned by storage services.

use std::time::Duration;

use anyhow::Result;
use futures::io;
use opendal::Operator;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op
        .presign_read_with("test.txt", Duration::from_secs(3600))
        .override_content_type("text/plain")
        .await?;
    Ok(())
}

pub async fn presign_write( &self, path: &str, expire: Duration, ) -> Result<PresignedRequest, Error>

Presign an operation for write.

§Notes

§Extra Options

presign_write is a wrapper of Self::presign_write_with without any options. To use extra options like content_type, please use Self::presign_write_with instead.

§Example

use std::time::Duration;

use anyhow::Result;
use opendal::Operator;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op
        .presign_write("test.txt", Duration::from_secs(3600))
        .await?;
    Ok(())
}

signed_req.method(): PUT
signed_req.uri(): https://s3.amazonaws.com/examplebucket/test.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=access_key_id/20130721/us-east-1/s3/aws4_request&X-Amz-Date=20130721T201207Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=<signature-value>
signed_req.headers(): { "host": "s3.amazonaws.com" }

We can upload file as this file via curl or other tools without credential:

curl -X PUT "https://s3.amazonaws.com/examplebucket/test.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=access_key_id/20130721/us-east-1/s3/aws4_request&X-Amz-Date=20130721T201207Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=<signature-value>" -d "Hello, World!"

pub fn presign_write_with( &self, path: &str, expire: Duration, ) -> OperatorFuture<(OpWrite, Duration), PresignedRequest, impl Future<Output = Result<PresignedRequest, Error>>>

Presign an operation for write with extra options.

§Options

§`content_type`

Set the content-type header returned by storage services.

use std::time::Duration;

use anyhow::Result;
use opendal::Operator;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op
        .presign_write_with("test", Duration::from_secs(3600))
        .content_type("text/csv")
        .await?;
    let req = http::Request::builder()
        .method(signed_req.method())
        .uri(signed_req.uri())
        .body(())?;

    Ok(())
}

§`content_disposition`

Set the content-disposition header returned by storage services.

use std::time::Duration;

use anyhow::Result;
use opendal::Operator;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op
        .presign_write_with("test", Duration::from_secs(3600))
        .content_disposition("attachment; filename=\"cool.html\"")
        .await?;
    let req = http::Request::builder()
        .method(signed_req.method())
        .uri(signed_req.uri())
        .body(())?;

    Ok(())
}

§`cache_control`

Set the cache-control header returned by storage services.

use std::time::Duration;

use anyhow::Result;
use opendal::Operator;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op
        .presign_write_with("test", Duration::from_secs(3600))
        .cache_control("no-store")
        .await?;
    let req = http::Request::builder()
        .method(signed_req.method())
        .uri(signed_req.uri())
        .body(())?;

    Ok(())
}

§

impl Operator

§Operator build API

Operator should be built via [OperatorBuilder]. We recommend to use Operator::new to get started:

use opendal::services::Fs;
use opendal::Operator;
async fn test() -> Result<()> {
    // Create fs backend builder.
    let builder = Fs::default().root("/tmp");

    // Build an `Operator` to start operating the storage.
    let op: Operator = Operator::new(builder)?.finish();

    Ok(())
}

pub fn new(ab: B) -> Result<OperatorBuilder<impl Access>, Error>
where B: Builder,

Create a new operator with input builder.

OpenDAL will call builder.build() internally, so we don’t need to import opendal::Builder trait.

§Examples

Read more backend init examples in examples.

use opendal::services::Fs;
use opendal::Operator;
async fn test() -> Result<()> {
    // Create fs backend builder.
    let builder = Fs::default().root("/tmp");

    // Build an `Operator` to start operating the storage.
    let op: Operator = Operator::new(builder)?.finish();

    Ok(())
}

pub fn from_config<C>(cfg: C) -> Result<OperatorBuilder<impl Access>, Error>
where C: Configurator,

Create a new operator from given config.

§Examples

use std::collections::HashMap;

use opendal::services::MemoryConfig;
use opendal::Operator;
async fn test() -> Result<()> {
    let cfg = MemoryConfig::default();

    // Build an `Operator` to start operating the storage.
    let op: Operator = Operator::from_config(cfg)?.finish();

    Ok(())
}

pub fn from_iter( iter: impl IntoIterator<Item = (String, String)>, ) -> Result<OperatorBuilder<impl Access>, Error>
where B: Builder,

Create a new operator from given iterator in static dispatch.

§Notes

from_iter generates a OperatorBuilder which allows adding layer in zero-cost way.

§Examples

use std::collections::HashMap;

use opendal::services::Fs;
use opendal::Operator;
async fn test() -> Result<()> {
    let map = HashMap::from([
        // Set the root for fs, all operations will happen under this root.
        //
        // NOTE: the root must be absolute path.
        ("root".to_string(), "/tmp".to_string()),
    ]);

    // Build an `Operator` to start operating the storage.
    let op: Operator = Operator::from_iter::<Fs>(map)?.finish();

    Ok(())
}

pub fn via_iter( scheme: Scheme, iter: impl IntoIterator<Item = (String, String)>, ) -> Result<Operator, Error>

Create a new operator via given scheme and iterator of config value in dynamic dispatch.

§Notes

via_iter generates a Operator which allows building operator without generic type.

§Examples

use std::collections::HashMap;

use opendal::Operator;
use opendal::Scheme;
async fn test() -> Result<()> {
    let map = [
        // Set the root for fs, all operations will happen under this root.
        //
        // NOTE: the root must be absolute path.
        ("root".to_string(), "/tmp".to_string()),
    ];

    // Build an `Operator` to start operating the storage.
    let op: Operator = Operator::via_iter(Scheme::Fs, map)?;

    Ok(())
}

pub fn from_map( map: HashMap<String, String>, ) -> Result<OperatorBuilder<impl Access>, Error>
where B: Builder,

👎Deprecated: use from_iter instead

Create a new operator from given map.

§Notes

from_map is using static dispatch layers which is zero cost. via_map is using dynamic dispatch layers which has a bit runtime overhead with an extra vtable lookup and unable to inline. But from_map requires generic type parameter which is not always easy to be used.

§Examples

use std::collections::HashMap;

use opendal::services::Fs;
use opendal::Operator;
async fn test() -> Result<()> {
    let map = HashMap::from([
        // Set the root for fs, all operations will happen under this root.
        //
        // NOTE: the root must be absolute path.
        ("root".to_string(), "/tmp".to_string()),
    ]);

    // Build an `Operator` to start operating the storage.
    let op: Operator = Operator::from_map::<Fs>(map)?.finish();

    Ok(())
}

pub fn via_map( scheme: Scheme, map: HashMap<String, String>, ) -> Result<Operator, Error>

👎Deprecated: use via_iter instead

Create a new operator from given scheme and map.

§Notes

from_map is using static dispatch layers which is zero cost. via_map is using dynamic dispatch layers which has a bit runtime overhead with an extra vtable lookup and unable to inline. But from_map requires generic type parameter which is not always easy to be used.

§Examples

use std::collections::HashMap;

use opendal::Operator;
use opendal::Scheme;
async fn test() -> Result<()> {
    let map = HashMap::from([
        // Set the root for fs, all operations will happen under this root.
        //
        // NOTE: the root must be absolute path.
        ("root".to_string(), "/tmp".to_string()),
    ]);

    // Build an `Operator` to start operating the storage.
    let op: Operator = Operator::via_map(Scheme::Fs, map)?;

    Ok(())
}

pub fn layer<L>(self, layer: L) -> Operator
where L: Layer<Arc<dyn AccessDyn>>,

Create a new layer with dynamic dispatch.

§Notes

OperatorBuilder::layer() is using static dispatch which is zero cost. Operator::layer() is using dynamic dispatch which has a bit runtime overhead with an extra vtable lookup and unable to inline.

It’s always recommended to use OperatorBuilder::layer() instead.

§Examples

use opendal::layers::LoggingLayer;
use opendal::services::Fs;
use opendal::Operator;

let op = Operator::new(Fs::default())?.finish();
let op = op.layer(LoggingLayer::default());
// All operations will go through the new_layer
let _ = op.read("test_file").await?;