Struct ObjectStore 

pub struct ObjectStore {
    accessor: Arc<dyn AccessDyn>,
}

The Operator serves as the entry point for all public asynchronous APIs.

For more details about the Operator, refer to the [concepts][crate::docs::concepts] section.

All cloned Operator instances share the same internal state, such as HttpClient and Runtime. Some layers may modify the internal state of the Operator too like inject logging and metrics for HttpClient.

Build

Users can initialize an Operator through the following methods:

• Operator::new: Creates an operator using a [services] builder, such as services::Memory.
• Operator::from_config: Creates an operator using a [services] configuration, such as services::MemoryConfig.
• Operator::from_iter: Creates an operator from an iterator of configuration key-value pairs.

use opendal_core::services::Memory;
use opendal_core::Operator;
async fn test() -> Result<()> {
    // Build an `Operator` to start operating the storage.
    let _: Operator = Operator::new(Memory::default())?.finish();

    Ok(())
}

Layer

After the operator is built, users can add the layers they need on top of it.

OpenDAL offers various layers for users to choose from. Visit [layers] for further details.

Please note that Layer can modify internal contexts such as HttpClient and Runtime for all clones of given operator. Therefore, it is recommended to add layers before interacting with the storage. Adding or duplicating layers after accessing the storage may result in unexpected behavior.

use opendal_core::layers::HttpClientLayer;
use opendal_core::raw::HttpClient;
use opendal_core::services::Memory;
use opendal_core::Operator;
use opendal_core::Result;

async fn test() -> Result<()> {
    let op: Operator = Operator::new(Memory::default())?.finish();

    // OpenDAL will replace the default HTTP client now.
    let client = HttpClient::new()?;
    let op = op.layer(HttpClientLayer::new(client));

    Ok(())
}

Operate

After the operator is built and the layers are added, users can start operating the storage.

The operator is Send, Sync, and Clone. It has no internal state, and all APIs only take a &self reference, making it safe to share the operator across threads.

Operator provides a consistent API pattern for data operations. For reading operations, it exposes:

• Operator::read: Executes a read operation.
• Operator::read_with: Executes a read operation with additional options using the builder pattern.
• Operator::read_options: Executes a read operation with extra options provided via a [options::ReadOptions] struct.
• Operator::reader: Creates a reader for streaming data, allowing for flexible access.
• Operator::reader_with: Creates a reader with advanced options using the builder pattern.
• Operator::reader_options: Creates a reader with extra options provided via a [options::ReadOptions] struct.

The Reader created by Operator supports custom read control methods and can be converted into [futures::AsyncRead] or [futures::Stream] for broader ecosystem compatibility.

use opendal_core::options;
use opendal_core::services;
use opendal_core::Operator;
use opendal_core::Result;

#[tokio::main]
async fn main() -> Result<()> {
    // Pick a builder and configure it.
    let builder = services::Memory::default();

    // Init an operator
    let op = Operator::new(builder)?.finish();

    // Fetch this file's metadata
    let meta = op.stat("hello.txt").await?;
    let length = meta.content_length();

    // Read data from `hello.txt` with options.
    let bs = op
        .read_with("hello.txt")
        .range(0..8 * 1024 * 1024)
        .chunk(1024 * 1024)
        .concurrent(4)
        .await?;

    // The same to:
    let bs = op
        .read_options("hello.txt", options::ReadOptions {
            range: (0..8 * 1024 * 1024).into(),
            chunk: Some(1024 * 1024),
            concurrent: 4,
            ..Default::default()
        })
        .await?;

    Ok(())
}

Fields

accessor: Arc<dyn AccessDyn>

Implementations

impl Operator

Operator basic API.

pub fn inner(&self) -> &Arc<dyn AccessDyn>

Fetch the internal accessor.

pub fn from_inner(accessor: Arc<dyn AccessDyn>) -> Operator

Convert inner accessor into operator.

pub fn into_inner(self) -> Arc<dyn AccessDyn>

Convert operator into inner accessor.

pub fn info(&self) -> OperatorInfo

Get information of underlying accessor.

Examples

use opendal_core::Operator;

let info = op.info();

pub fn executor(&self) -> Executor

Get the executor used by current operator.

pub fn update_executor(&self, f: impl FnOnce(Executor) -> Executor)

Update executor for the context.

All cloned Operator instances share the same internal state, such as HttpClient and Runtime. Some layers may modify the internal state of the Operator too like inject logging and metrics for HttpClient.

Note

Tasks must be forwarded to the old executor after the update. Otherwise, features such as retry, timeout, and metrics may not function properly.

impl Operator

Operator async API.

pub async fn check(&self) -> Result<(), Error>

Check if this operator can work correctly.

We will send a list request to path and return any errors we met.

use opendal_core::Operator;

op.check().await?;

pub async fn stat(&self, path: &str) -> Result<Metadata, Error>

Retrieve the metadata for the specified path.

Notes

Extra Options

Operator::stat is a wrapper around Operator::stat_with that uses no additional options. To specify extra options such as if_match and if_none_match, please use Operator::stat_with instead.

Examples

Check if file exists

use opendal_core::ErrorKind;
if let Err(e) = op.stat("test").await {
    if e.kind() == ErrorKind::NotFound {
        println!("file not exist")
    }
}

pub fn stat_with( &self, path: &str, ) -> OperatorFuture<StatOptions, Metadata, impl Future<Output = Result<Metadata, Error>>>

Retrieve the metadata of the specified path with additional options.

Options

Check [options::StatOptions] for all available options.

Examples

Get metadata while ETag matches

stat_with will

• return Ok(metadata) if ETag matches
• return Err(error) and error.kind() == ErrorKind::ConditionNotMatch if file exists but ETag mismatch
• return Err(err) if other errors occur, for example, NotFound.

use opendal_core::ErrorKind;
if let Err(e) = op.stat_with("test").if_match("<etag>").await {
    if e.kind() == ErrorKind::ConditionNotMatch {
        println!("file exists, but etag mismatch")
    }
    if e.kind() == ErrorKind::NotFound {
        println!("file not exist")
    }
}

pub async fn stat_options( &self, path: &str, opts: StatOptions, ) -> Result<Metadata, Error>

Retrieve the metadata of the specified path with additional options.

Examples

Get metadata while ETag matches

stat_with will

• return Ok(metadata) if ETag matches
• return Err(error) and error.kind() == ErrorKind::ConditionNotMatch if file exists but ETag mismatch
• return Err(err) if other errors occur, for example, NotFound.

use opendal_core::options;
use opendal_core::ErrorKind;
let res = op
    .stat_options("test", options::StatOptions {
        if_match: Some("<etag>".to_string()),
        ..Default::default()
    })
    .await;
if let Err(e) = res {
    if e.kind() == ErrorKind::ConditionNotMatch {
        println!("file exists, but etag mismatch")
    }
    if e.kind() == ErrorKind::NotFound {
        println!("file not exist")
    }
}

pub async fn exists(&self, path: &str) -> Result<bool, Error>

Check whether this path exists.

Example

use anyhow::Result;
use futures::io;
use opendal_core::Operator;

async fn test(op: Operator) -> Result<()> {
    let _ = op.exists("test").await?;

    Ok(())
}

pub async fn create_dir(&self, path: &str) -> Result<(), Error>

Create a directory at the specified path.

Notes

To specify that a path is a directory, you must include a trailing slash (/). Omitting the trailing slash may cause OpenDAL to return a NotADirectory error.

Behavior

• Creating a directory that already exists will succeed.
• Directory creation is always recursive, functioning like mkdir -p.

Examples

op.create_dir("path/to/dir/").await?;

pub async fn read(&self, path: &str) -> Result<Buffer, Error>

Read the entire file into bytes from given path.

Notes

Additional Options

Operator::read is a simplified method that does not support additional options. To access features like range and if_match, please use Operator::read_with or Operator::read_options instead.

Streaming Read

This function reads all content into memory at once. For more precise memory management or to read big file lazily, please use Operator::reader.

Examples

let bs = op.read("path/to/file").await?;

pub fn read_with( &self, path: &str, ) -> OperatorFuture<ReadOptions, Buffer, impl Future<Output = Result<Buffer, Error>>>

Read the entire file into bytes from given path with additional options.

Notes

Streaming Read

This function reads all content into memory at once. For more precise memory management or to read big file lazily, please use Operator::reader.

Options

Visit [options::ReadOptions] for all available options.

Examples

Read the first 10 bytes of a file:

let bs = op.read_with("path/to/file").range(0..10).await?;

pub async fn read_options( &self, path: &str, opts: ReadOptions, ) -> Result<Buffer, Error>

Read the entire file into bytes from given path with additional options.

Notes

Streaming Read

This function reads all content into memory at once. For more precise memory management or to read big file lazily, please use Operator::reader.

Examples

Read the first 10 bytes of a file:

use opendal_core::options;
let bs = op
    .read_options("path/to/file", options::ReadOptions {
        range: (0..10).into(),
        ..Default::default()
    })
    .await?;

pub async fn reader(&self, path: &str) -> Result<Reader, Error>

Create a new reader of given path.

Notes

Extra Options

Operator::reader is a simplified method without any options. To use additional options such as concurrent or if_match, please use Operator::reader_with or Operator::reader_options instead.

Examples

let r = op.reader("path/to/file").await?;
// Read the first 10 bytes of the file
let data = r.read(0..10).await?;

pub fn reader_with( &self, path: &str, ) -> OperatorFuture<ReaderOptions, Reader, impl Future<Output = Result<Reader, Error>>>

Create a new reader of given path with additional options.

Options

Visit [options::ReaderOptions] for all available options.

Examples

Create a reader with a specific version ID:

let r = op.reader_with("path/to/file").version("version_id").await?;
// Read the first 10 bytes of the file
let data = r.read(0..10).await?;

pub async fn reader_options( &self, path: &str, opts: ReaderOptions, ) -> Result<Reader, Error>

Create a new reader of given path with additional options.

Examples

Create a reader with a specific version ID:

use opendal_core::options;
let r = op
    .reader_options("path/to/file", options::ReaderOptions {
        version: Some("version_id".to_string()),
        ..Default::default()
    })
    .await?;
// Read the first 10 bytes of the file
let data = r.read(0..10).await?;

pub async fn write( &self, path: &str, bs: impl Into<Buffer>, ) -> Result<Metadata, Error>

Write all data to the specified path at once.

Notes

Visit [performance::concurrent_write][crate::docs::performance::concurrent_write] for more details on concurrent writes.

Extra Options

Operator::write is a simplified method that does not include additional options. For advanced features such as chunk and concurrent, use Operator::write_with or Operator::write_options instead.

Streaming Write

This method executes a single bulk write operation. For more precise memory management or to write data in a streaming fashion, use Operator::writer instead.

Multipart Uploads

OpenDAL offers multipart upload capabilities through the Writer abstraction, automatically managing all upload details for you. You can fine-tune the upload process by adjusting the chunk size and the number of concurrent operations using Operator::writer_with.

Examples

use bytes::Bytes;

op.write("path/to/file", vec![0; 4096]).await?;

pub fn write_with( &self, path: &str, bs: impl Into<Buffer>, ) -> OperatorFuture<(WriteOptions, Buffer), Metadata, impl Future<Output = Result<Metadata, Error>>>

Write all data to the specified path at once with additional options.

Notes

Visit [performance::concurrent_write][crate::docs::performance::concurrent_write] for more details on concurrent writes.

Streaming Write

This method executes a single bulk write operation. For more precise memory management or to write data in a streaming fashion, use Operator::writer instead.

Multipart Uploads

OpenDAL offers multipart upload capabilities through the Writer abstraction, automatically managing all upload details for you. You can fine-tune the upload process by adjusting the chunk size and the number of concurrent operations using Operator::writer_with.

Options

Visit [options::WriteOptions] for all available options.

Examples

Write data to a file only when it does not already exist:

use bytes::Bytes;

let _ = op
    .write_with("path/to/file", vec![0; 4096])
    .if_not_exists(true)
    .await?;

pub async fn write_options( &self, path: &str, bs: impl Into<Buffer>, opts: WriteOptions, ) -> Result<Metadata, Error>

Write all data to the specified path at once with additional options.

Notes

Visit [performance::concurrent_write][crate::docs::performance::concurrent_write] for more details on concurrent writes.

Streaming Write

This method executes a single bulk write operation. For more precise memory management or to write data in a streaming fashion, use Operator::writer instead.

Multipart Uploads

OpenDAL offers multipart upload capabilities through the Writer abstraction, automatically managing all upload details for you. You can fine-tune the upload process by adjusting the chunk size and the number of concurrent operations using Operator::writer_with.

Examples

Write data to a file only when it does not already exist:

use opendal_core::options;

let _ = op
    .write_options("path/to/file", vec![0; 4096], options::WriteOptions {
        if_not_exists: true,
        ..Default::default()
    })
    .await?;

pub async fn writer(&self, path: &str) -> Result<Writer, Error>

Create a new writer of given path.

Notes

Writer Features

The writer provides several powerful capabilities:

• Streaming writes for continuous data transfer
• Automatic multipart upload handling
• Memory-efficient chunk-based writing

Extra Options

Operator::writer is a simplified version that does not include additional options. For advanced features such as chunk and concurrent, use Operator::writer_with or Operator::writer_options instead.

Examples

use bytes::Bytes;

let mut w = op.writer("path/to/file").await?;
w.write(vec![0; 4096]).await?;
w.write(vec![1; 4096]).await?;
w.close().await?;

pub fn writer_with( &self, path: &str, ) -> OperatorFuture<WriteOptions, Writer, impl Future<Output = Result<Writer, Error>>>

Create a new writer of given path with additional options.

Notes

Writer Features

The writer provides several powerful capabilities:

• Streaming writes for continuous data transfer
• Automatic multipart upload handling
• Memory-efficient chunk-based writing

Chunk Size Handling

Storage services often have specific requirements for chunk sizes:

• Services like s3 may return EntityTooSmall errors for undersized chunks
• Using small chunks in cloud storage services can lead to increased costs

OpenDAL automatically determines optimal chunk sizes based on the service’s Capability. However, you can override this by explicitly setting the chunk parameter.

Visit [performance::concurrent_write][crate::docs::performance::concurrent_write] for more details on concurrent writes.

Examples

use bytes::Bytes;

let mut w = op
    .writer_with("path/to/file")
    .chunk(4 * 1024 * 1024)
    .concurrent(8)
    .await?;
w.write(vec![0; 4096]).await?;
w.write(vec![1; 4096]).await?;
w.close().await?;

pub async fn writer_options( &self, path: &str, opts: WriteOptions, ) -> Result<Writer, Error>

Create a new writer of given path with additional options.

Notes

Writer Features

The writer provides several powerful capabilities:

• Streaming writes for continuous data transfer
• Automatic multipart upload handling
• Memory-efficient chunk-based writing

Chunk Size Handling

Storage services often have specific requirements for chunk sizes:

• Services like s3 may return EntityTooSmall errors for undersized chunks
• Using small chunks in cloud storage services can lead to increased costs

OpenDAL automatically determines optimal chunk sizes based on the service’s Capability. However, you can override this by explicitly setting the chunk parameter.

Visit [performance::concurrent_write][crate::docs::performance::concurrent_write] for more details on concurrent writes.

Examples

Write data to a file in 4MiB chunk size and at 8 concurrency:

use bytes::Bytes;

let mut w = op
    .writer_options(
        "path/to/file",
        options::WriteOptions {
            chunk: Some(4 * 1024 * 1024),
            concurrent: 8,
          ..Default::default()
        },
    )
    .await?;
w.write(vec![0; 4096]).await?;
w.write(vec![1; 4096]).await?;
w.close().await?;

pub async fn copy(&self, from: &str, to: &str) -> Result<(), Error>

Copy a file from from to to.

Notes

• from and to must be a file.
• to will be overwritten if it exists.
• If from and to are the same, an IsSameFile error will occur.
• copy is idempotent. For same from and to input, the result will be the same.

Examples

op.copy("path/to/file", "path/to/file2").await?;

pub fn copy_with( &self, from: &str, to: &str, ) -> OperatorFuture<(CopyOptions, String), (), impl Future<Output = Result<(), Error>>>

Copy a file from from to to with additional options.

Notes

• from and to must be a file.
• If from and to are the same, an IsSameFile error will occur.
• copy is idempotent. For same from and to input, the result will be the same.

Options

Visit [options::CopyOptions] for all available options.

Examples

Copy a file only if the destination doesn’t exist:

op.copy_with("path/to/file", "path/to/file2")
    .if_not_exists(true)
    .await?;

pub async fn copy_options( &self, from: &str, to: &str, opts: impl Into<CopyOptions>, ) -> Result<(), Error>

Copy a file from from to to with additional options.

Notes

• from and to must be a file.
• If from and to are the same, an IsSameFile error will occur.
• copy is idempotent. For same from and to input, the result will be the same.

Options

Check [options::CopyOptions] for all available options.

Examples

Copy a file only if the destination doesn’t exist:

let mut opts = CopyOptions::default();
opts.if_not_exists = true;
op.copy_options("path/to/file", "path/to/file2", opts).await?;

pub async fn rename(&self, from: &str, to: &str) -> Result<(), Error>

Rename a file from from to to.

Notes

• from and to must be a file.
• to will be overwritten if it exists.
• If from and to are the same, an IsSameFile error will occur.

Examples

op.rename("path/to/file", "path/to/file2").await?;

pub async fn delete(&self, path: &str) -> Result<(), Error>

Delete the given path.

Notes

• Deleting a file that does not exist won’t return errors.

Examples

op.delete("test").await?;

pub fn delete_with( &self, path: &str, ) -> OperatorFuture<DeleteOptions, (), impl Future<Output = Result<(), Error>>>

Delete the given path with additional options.

Notes

• Deleting a file that does not exist won’t return errors.

Options

Visit [options::DeleteOptions] for all available options.

Examples

Delete a specific version of a file:

op.delete_with("path/to/file").version(version).await?;

pub async fn delete_options( &self, path: &str, opts: DeleteOptions, ) -> Result<(), Error>

Delete the given path with additional options.

Notes

• Deleting a file that does not exist won’t return errors.

Examples

Delete a specific version of a file:

use opendal_core::options;

op.delete_options("path/to/file", options::DeleteOptions {
    version: Some(version.to_string()),
    ..Default::default()
})
.await?;

pub async fn delete_iter<I, D>(&self, iter: I) -> Result<(), Error>

where I: IntoIterator<Item = D>, D: IntoDeleteInput,

Delete an infallible iterator of paths.

Also see:

• Operator::delete_try_iter: delete a fallible iterator of paths.
• Operator::delete_stream: delete an infallible stream of paths.
• Operator::delete_try_stream: delete a fallible stream of paths.

pub async fn delete_try_iter<I, D>(&self, try_iter: I) -> Result<(), Error>

where I: IntoIterator<Item = Result<D, Error>>, D: IntoDeleteInput,

Delete a fallible iterator of paths.

Also see:

• Operator::delete_iter: delete an infallible iterator of paths.
• Operator::delete_stream: delete an infallible stream of paths.
• Operator::delete_try_stream: delete a fallible stream of paths.

pub async fn delete_stream<S, D>(&self, stream: S) -> Result<(), Error>

where S: Stream<Item = D>, D: IntoDeleteInput,

Delete an infallible stream of paths.

Also see:

• Operator::delete_iter: delete an infallible iterator of paths.
• Operator::delete_try_iter: delete a fallible iterator of paths.
• Operator::delete_try_stream: delete a fallible stream of paths.

pub async fn delete_try_stream<S, D>(&self, try_stream: S) -> Result<(), Error>

where S: Stream<Item = Result<D, Error>>, D: IntoDeleteInput,

Delete a fallible stream of paths.

Also see:

• Operator::delete_iter: delete an infallible iterator of paths.
• Operator::delete_try_iter: delete a fallible iterator of paths.
• Operator::delete_stream: delete an infallible stream of paths.

pub async fn deleter(&self) -> Result<Deleter, Error>

Create a [Deleter] to continuously remove content from storage.

It leverages batch deletion capabilities provided by storage services for efficient removal.

Users can have more control over the deletion process by using [Deleter] directly.

pub async fn remove_all(&self, path: &str) -> Result<(), Error>

👎Deprecated since 0.55.0:

Use delete_with with recursive(true) instead

Remove the path and all nested dirs and files recursively.

Deprecated

This method is deprecated since v0.55.0. Use Operator::delete_with with recursive(true) instead.

Migration Example

Instead of:

op.remove_all("path/to/dir").await?;

Use:

op.delete_with("path/to/dir").recursive(true).await?;

Notes

If underlying services support delete in batch, we will use batch delete instead.

Examples

op.remove_all("path/to/dir").await?;

pub async fn list(&self, path: &str) -> Result<Vec<Entry>, Error>

List entries whose paths start with the given prefix path.

Semantics

• Listing is prefix-based. It does not require the parent directory to exist.
• If path itself exists (file or dir), it will be returned as an entry in addition to any prefixed children.
• If path is absent but deeper objects exist (e.g. path/child), the list succeeds and returns those prefixed entries instead of an error.

Streaming List

This function materializes the entire list into memory. For large listings, prefer Operator::lister to stream entries.

Examples

This example will list all entries under the dir path/to/dir/.

use opendal_core::EntryMode;
use opendal_core::Operator;
let mut entries = op.list("path/to/dir/").await?;
for entry in entries {
    match entry.metadata().mode() {
        EntryMode::FILE => {
            println!("Handling file")
        }
        EntryMode::DIR => {
            println!("Handling dir {}", entry.path())
        }
        EntryMode::Unknown => continue,
    }
}

pub fn list_with( &self, path: &str, ) -> OperatorFuture<ListOptions, Vec<Entry>, impl Future<Output = Result<Vec<Entry>, Error>>>

List entries whose paths start with the given prefix path with additional options.

Semantics

Inherits the prefix semantics described in Operator::list: returns path itself if it exists and tolerates missing parents when prefixed objects exist.

Notes

Streaming List

This function materializes the entire list into memory. For large listings, prefer Operator::lister to stream entries.

Options

See [options::ListOptions] for the full set. Common knobs:

• Traversal: recursive (default false) toggles depth-first listing under the prefix.
• Pagination: limit and start_after tune page size and resume positions (backend dependent).
• Versioning: versions / deleted ask versioned backends to return extra entries.

Examples

This example will list all entries recursively under the prefix path/to/prefix.

use opendal_core::EntryMode;
use opendal_core::Operator;
let mut entries = op.list_with("path/to/prefix").recursive(true).await?;
for entry in entries {
    match entry.metadata().mode() {
        EntryMode::FILE => {
            println!("Handling file")
        }
        EntryMode::DIR => {
            println!("Handling dir like start a new list via meta.path()")
        }
        EntryMode::Unknown => continue,
    }
}

pub async fn list_options( &self, path: &str, opts: ListOptions, ) -> Result<Vec<Entry>, Error>

List entries whose paths start with the given prefix path using explicit options.

Semantics

Same prefix behavior as Operator::list: returns path itself if present and tolerates missing parents when prefixed objects exist.

Options

Accepts [options::ListOptions] (see field docs for meaning).

Streaming List

Materializes the entire list; use Operator::lister to stream large result sets.

Examples

This example will list all entries recursively under the prefix path/to/prefix.

use opendal_core::options;
use opendal_core::EntryMode;
use opendal_core::Operator;
let mut entries = op
    .list_options("path/to/prefix", options::ListOptions {
        recursive: true,
        ..Default::default()
    })
    .await?;
for entry in entries {
    match entry.metadata().mode() {
        EntryMode::FILE => {
            println!("Handling file")
        }
        EntryMode::DIR => {
            println!("Handling dir like start a new list via meta.path()")
        }
        EntryMode::Unknown => continue,
    }
}

pub async fn lister(&self, path: &str) -> Result<Lister, Error>

Create a streaming lister for entries whose paths start with the given prefix path.

Semantics

Shares the same prefix semantics as Operator::list: the parent directory is not required to exist; the entry for path is yielded if present; missing parents with deeper objects are accepted.

Options

Takes the same [options::ListOptions] as list_with: traversal (recursive), pagination (limit, start_after), and versioning (versions, deleted).

Examples

use futures::TryStreamExt;
use opendal_core::EntryMode;
use opendal_core::Operator;
let mut ds = op.lister("path/to/dir/").await?;
while let Some(mut de) = ds.try_next().await? {
    match de.metadata().mode() {
        EntryMode::FILE => {
            println!("Handling file")
        }
        EntryMode::DIR => {
            println!("Handling dir like start a new list via meta.path()")
        }
        EntryMode::Unknown => continue,
    }
}

pub fn lister_with( &self, path: &str, ) -> OperatorFuture<ListOptions, Lister, impl Future<Output = Result<Lister, Error>>>

Create a new lister to list entries that start with the given prefix path using additional options.

Options

Same as lister_with; see [options::ListOptions] for traversal, pagination, and versioning knobs.

Examples

List all files recursively

use futures::TryStreamExt;
use opendal_core::EntryMode;
use opendal_core::Operator;
let mut lister = op.lister_with("path/to/dir/").recursive(true).await?;
while let Some(mut entry) = lister.try_next().await? {
    match entry.metadata().mode() {
        EntryMode::FILE => {
            println!("Handling file {}", entry.path())
        }
        EntryMode::DIR => {
            println!("Handling dir {}", entry.path())
        }
        EntryMode::Unknown => continue,
    }
}

pub async fn lister_options( &self, path: &str, opts: ListOptions, ) -> Result<Lister, Error>

Create a new lister to list entries that start with the given prefix path using additional options.

Semantics

Inherits the prefix behavior of Operator::lister_with.

Options

Uses [options::ListOptions] to control traversal, pagination, and versioning.

Examples

List all files recursively

use futures::TryStreamExt;
use opendal_core::options;
use opendal_core::EntryMode;
use opendal_core::Operator;
let mut lister = op
    .lister_options("path/to/dir/", options::ListOptions {
        recursive: true,
        ..Default::default()
    })
    .await?;
while let Some(mut entry) = lister.try_next().await? {
    match entry.metadata().mode() {
        EntryMode::FILE => {
            println!("Handling file {}", entry.path())
        }
        EntryMode::DIR => {
            println!("Handling dir {}", entry.path())
        }
        EntryMode::Unknown => continue,
    }
}

impl Operator

Operator presign API.

pub async fn presign_stat( &self, path: &str, expire: Duration, ) -> Result<PresignedRequest, Error>

Presign an operation for stat(head).

Example

use anyhow::Result;
use futures::io;
use opendal_core::Operator;
use std::time::Duration;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op.presign_stat("test",Duration::from_secs(3600)).await?;
    let req = http::Request::builder()
        .method(signed_req.method())
        .uri(signed_req.uri())
        .body(())?;

pub fn presign_stat_with( &self, path: &str, expire: Duration, ) -> OperatorFuture<(StatOptions, Duration), PresignedRequest, impl Future<Output = Result<PresignedRequest, Error>>>

Presign an operation for stat(head).

Example

use anyhow::Result;
use futures::io;
use opendal_core::Operator;
use std::time::Duration;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op.presign_stat_with("test",Duration::from_secs(3600)).override_content_disposition("attachment; filename=\"othertext.txt\"").await?;

pub async fn presign_stat_options( &self, path: &str, expire: Duration, opts: StatOptions, ) -> Result<PresignedRequest, Error>

Presign an operation for stat(head) with additional options.

Options

Visit [options::StatOptions] for all available options.

Example

use anyhow::Result;
use opendal_core::Operator;
use opendal_core::options;
use std::time::Duration;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op.presign_stat_options(
        "test",
        Duration::from_secs(3600),
        options::StatOptions {
            if_match: Some("<etag>".to_string()),
            ..Default::default()
        }
    ).await?;
    let req = http::Request::builder()
        .method(signed_req.method())
        .uri(signed_req.uri())
        .body(())?;

pub async fn presign_read( &self, path: &str, expire: Duration, ) -> Result<PresignedRequest, Error>

Presign an operation for read.

Notes

Extra Options

presign_read is a wrapper of Self::presign_read_with without any options. To use extra options like override_content_disposition, please use Self::presign_read_with or `Self::presign_read_options instead.

Example

use anyhow::Result;
use futures::io;
use opendal_core::Operator;
use std::time::Duration;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op.presign_read("test.txt", Duration::from_secs(3600)).await?;

• signed_req.method(): GET
• signed_req.uri(): https://s3.amazonaws.com/examplebucket/test.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=access_key_id/20130721/us-east-1/s3/aws4_request&X-Amz-Date=20130721T201207Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=<signature-value>
• signed_req.headers(): { "host": "s3.amazonaws.com" }

We can download this file via curl or other tools without credentials:

curl "https://s3.amazonaws.com/examplebucket/test.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=access_key_id/20130721/us-east-1/s3/aws4_request&X-Amz-Date=20130721T201207Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=<signature-value>" -O /tmp/test.txt

pub fn presign_read_with( &self, path: &str, expire: Duration, ) -> OperatorFuture<(ReadOptions, Duration), PresignedRequest, impl Future<Output = Result<PresignedRequest, Error>>>

Presign an operation for read with extra options.

Options

Visit [options::ReadOptions] for all available options.

Example

use std::time::Duration;

use anyhow::Result;
use futures::io;
use opendal_core::Operator;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op
        .presign_read_with("test.txt", Duration::from_secs(3600))
        .override_content_type("text/plain")
        .await?;
    Ok(())
}

pub async fn presign_read_options( &self, path: &str, expire: Duration, opts: ReadOptions, ) -> Result<PresignedRequest, Error>

Presign an operation for read with additional options.

Options

Visit [options::ReadOptions] for all available options.

Example

use anyhow::Result;
use opendal_core::Operator;
use opendal_core::options;
use std::time::Duration;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op.presign_read_options(
        "file",
        Duration::from_secs(3600),
        options::ReadOptions {
            override_content_disposition: Some("attachment; filename=\"othertext.txt\"".to_string()),
            ..Default::default()
        }
    ).await?;
    let req = http::Request::builder()
        .method(signed_req.method())
        .uri(signed_req.uri())
        .body(())?;

pub async fn presign_write( &self, path: &str, expire: Duration, ) -> Result<PresignedRequest, Error>

Presign an operation for write.

Notes

Extra Options

presign_write is a wrapper of Self::presign_write_with without any options. To use extra options like content_type, please use Self::presign_write_with or Self::presign_write_options instead.

Example

use std::time::Duration;

use anyhow::Result;
use opendal_core::Operator;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op
        .presign_write("test.txt", Duration::from_secs(3600))
        .await?;
    Ok(())
}

• signed_req.method(): PUT
• signed_req.uri(): https://s3.amazonaws.com/examplebucket/test.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=access_key_id/20130721/us-east-1/s3/aws4_request&X-Amz-Date=20130721T201207Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=<signature-value>
• signed_req.headers(): { "host": "s3.amazonaws.com" }

We can upload file as this file via curl or other tools without credential:

curl -X PUT "https://s3.amazonaws.com/examplebucket/test.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=access_key_id/20130721/us-east-1/s3/aws4_request&X-Amz-Date=20130721T201207Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=<signature-value>" -d "Hello, World!"

pub fn presign_write_with( &self, path: &str, expire: Duration, ) -> OperatorFuture<(WriteOptions, Duration), PresignedRequest, impl Future<Output = Result<PresignedRequest, Error>>>

Presign an operation for write with extra options.

Options

Visit [options::WriteOptions] for all available options.

Example

use std::time::Duration;

use anyhow::Result;
use opendal_core::Operator;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op
        .presign_write_with("test", Duration::from_secs(3600))
        .cache_control("no-store")
        .await?;
    let req = http::Request::builder()
        .method(signed_req.method())
        .uri(signed_req.uri())
        .body(())?;

    Ok(())
}

pub async fn presign_write_options( &self, path: &str, expire: Duration, opts: WriteOptions, ) -> Result<PresignedRequest, Error>

Presign an operation for write with additional options.

Options

Check [options::WriteOptions] for all available options.

Example

use anyhow::Result;
use opendal_core::Operator;
use opendal_core::options;
use std::time::Duration;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op.presign_write_options(
        "file",
        Duration::from_secs(3600),
        options::WriteOptions {
            content_type: Some("application/json".to_string()),
            cache_control: Some("max-age=3600".to_string()),
            if_not_exists: true,
            ..Default::default()
        }
    ).await?;
    let req = http::Request::builder()
        .method(signed_req.method())
        .uri(signed_req.uri())
        .body(())?;

pub async fn presign_delete( &self, path: &str, expire: Duration, ) -> Result<PresignedRequest, Error>

Presign an operation for delete.

Notes

Extra Options

presign_delete is a wrapper of Self::presign_delete_with without any options.

Example

use std::time::Duration;

use anyhow::Result;
use opendal_core::Operator;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op
        .presign_delete("test.txt", Duration::from_secs(3600))
        .await?;
    Ok(())
}

• signed_req.method(): DELETE
• signed_req.uri(): https://s3.amazonaws.com/examplebucket/test.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=access_key_id/20130721/us-east-1/s3/aws4_request&X-Amz-Date=20130721T201207Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=<signature-value>
• signed_req.headers(): { "host": "s3.amazonaws.com" }

We can delete file as this file via curl or other tools without credential:

curl -X DELETE "https://s3.amazonaws.com/examplebucket/test.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=access_key_id/20130721/us-east-1/s3/aws4_request&X-Amz-Date=20130721T201207Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=<signature-value>"

pub fn presign_delete_with( &self, path: &str, expire: Duration, ) -> OperatorFuture<(DeleteOptions, Duration), PresignedRequest, impl Future<Output = Result<PresignedRequest, Error>>>

Presign an operation for delete without extra options.

pub async fn presign_delete_options( &self, path: &str, expire: Duration, opts: DeleteOptions, ) -> Result<PresignedRequest, Error>

Presign an operation for delete with additional options.

Options

Visit [options::DeleteOptions] for all available options.

Example

use anyhow::Result;
use opendal_core::Operator;
use opendal_core::options;
use std::time::Duration;

async fn test(op: Operator) -> Result<()> {
    let signed_req = op.presign_delete_options(
        "path/to/file",
        Duration::from_secs(3600),
        options::DeleteOptions {
            ..Default::default()
        }
    ).await?;
    let req = http::Request::builder()
        .method(signed_req.method())
        .uri(signed_req.uri())
        .body(())?;

impl Operator

Operator build API

Operator should be built via [OperatorBuilder]. We recommend to use Operator::new to get started:

use opendal_core::services::Memory;
use opendal_core::Operator;
async fn test() -> Result<()> {
    // Create memory backend builder.
    let builder = Memory::default();

    // Build an `Operator` to start operating the storage.
    let op: Operator = Operator::new(builder)?.finish();

    Ok(())
}

pub fn new<B>(ab: B) -> Result<OperatorBuilder<impl Access>, Error>

where B: Builder,

Create a new operator with input builder.

OpenDAL will call builder.build() internally, so we don’t need to import opendal::Builder trait.

Examples

Read more backend init examples in examples.

use opendal_core::services::Memory;
use opendal_core::Operator;
async fn test() -> Result<()> {
    // Create memory backend builder.
    let builder = Memory::default();

    // Build an `Operator` to start operating the storage.
    let op: Operator = Operator::new(builder)?.finish();

    Ok(())
}

pub fn from_config<C>(cfg: C) -> Result<OperatorBuilder<impl Access>, Error>

where C: Configurator,

Create a new operator from given config.

Examples

use std::collections::HashMap;

use opendal_core::services::MemoryConfig;
use opendal_core::Operator;
async fn test() -> Result<()> {
    let cfg = MemoryConfig::default();

    // Build an `Operator` to start operating the storage.
    let op: Operator = Operator::from_config(cfg)?.finish();

    Ok(())
}

pub fn from_iter<B>( iter: impl IntoIterator<Item = (String, String)>, ) -> Result<OperatorBuilder<impl Access>, Error>

where B: Builder,

Create a new operator from given iterator in static dispatch.

Notes

from_iter generates a OperatorBuilder which allows adding layer in zero-cost way.

Examples

use std::collections::HashMap;

use opendal_core::services::Memory;
use opendal_core::Operator;
async fn test() -> Result<()> {
    let map = HashMap::new();

    // Build an `Operator` to start operating the storage.
    let op: Operator = Operator::from_iter::<Memory>(map)?.finish();

    Ok(())
}

pub fn from_uri(uri: impl IntoOperatorUri) -> Result<Operator, Error>

Create a new operator by parsing configuration from a URI.

Examples

use opendal_core::Operator;

let op = Operator::from_uri("memory://localhost/")?;

pub fn via_iter( scheme: impl AsRef<str>, iter: impl IntoIterator<Item = (String, String)>, ) -> Result<Operator, Error>

Create a new operator via given scheme and iterator of config value in dynamic dispatch.

Notes

via_iter generates a Operator which allows building operator without generic type.

Examples

use std::collections::HashMap;

use opendal_core::Operator;
use opendal_core::services;

async fn test() -> Result<()> {
    let map: Vec<(String, String)> = vec![];

    // Build an `Operator` to start operating the storage.
    let op: Operator = Operator::via_iter(services::MEMORY_SCHEME, map)?;

    Ok(())
}

pub fn layer<L>(self, layer: L) -> Operator

where L: Layer<Arc<dyn AccessDyn>>,

Create a new layer with dynamic dispatch.

Please note that Layer can modify internal contexts such as HttpClient and Runtime for the operator. Therefore, it is recommended to add layers before interacting with the storage. Adding or duplicating layers after accessing the storage may result in unexpected behavior.

Notes

OperatorBuilder::layer() is using static dispatch which is zero cost. Operator::layer() is using dynamic dispatch which has a bit runtime overhead with an extra vtable lookup and unable to inline.

It’s always recommended to use OperatorBuilder::layer() instead.

Examples

let client = HttpClient::new()?;
let op = Operator::new(Memory::default())?.finish();
let op = op.layer(HttpClientLayer::new(client));
// All operations will go through the new_layer
let _ = op.read("test_file").await?;

Trait Implementations

impl Clone for Operator

fn clone(&self) -> Operator

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Operator

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl From<Operator> for OpendalStore

fn from(value: Operator) -> Self

Converts to this type from the input type.