diff --git a/README.md b/README.md new file mode 100644 index 0000000..7682f75 --- /dev/null +++ b/README.md @@ -0,0 +1,80 @@ +# VirtualFS + +VirtualFS is a .NET library for composing multiple file-like data sources into one virtual hierarchy. + +The core model is: + +- every backend implements `IFileSystem` +- a `RootFileSystem` mounts many backends under virtual paths +- callers interact with virtual paths like `/assets/logo.png` +- reads and writes are dispatched to the mounted backend that owns that path + +It behaves more like a small VFS layer than a thin `System.IO` wrapper. + +## Status + +The project currently targets: + +- `netstandard2.0` +- `net472` +- `net6.0` +- `net8.0` +- `net10.0` + +The test project targets `net10.0`. + +## Key Rules + +- paths are always rooted +- `/foo/` is a directory +- `/foo` is a file +- trailing `/` defines directory vs file +- path separator is always `/` +- mounted filesystems can overlap +- mount priority controls lookup order + +## Minimal Example + +```csharp +using System.IO; +using VirtualFS; +using VirtualFS.Implementation; +using VirtualFS.Memory; +using VirtualFS.Physical; + +var root = new RootFileSystem(); + +root.Mount(new PhysicalFileSystem(new DirectoryInfo("./content")), "/"); +root.Mount(new MemoryFileSystem { Priority = FileSystemMountPriority.High }, "/"); + +root.Root.FileCreate("runtime.txt").WriteAllText("generated at runtime"); + +foreach (var entry in root.Root.GetEntries()) +{ + Console.WriteLine($"{entry.FullPath} readonly={entry.IsReadOnly}"); +} +``` + +## Documentation + +Detailed documentation is split into the `docs/` folder: + +- [Documentation Index](./docs/README.md) +- [Architecture](./docs/architecture.md) +- [Usage](./docs/usage.md) +- [Backends](./docs/backends.md) +- [Development](./docs/development.md) + +## Build and Test + +The repository includes a Docker-based build/test flow using the official .NET 10 SDK image. + +```bash +docker build --target test -t virtualfs:test . +``` + +The repository CI runs the same Docker target on pushes and pull requests. + +## License + +The project metadata declares the MIT license. diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..8a57d5b --- /dev/null +++ b/docs/README.md @@ -0,0 +1,17 @@ +# Documentation + +This folder contains the detailed project documentation for VirtualFS. + +Sections: + +- [Architecture](./architecture.md) +- [Usage](./usage.md) +- [Backends](./backends.md) +- [Development](./development.md) + +Recommended reading order: + +1. [Architecture](./architecture.md) +2. [Usage](./usage.md) +3. [Backends](./backends.md) +4. [Development](./development.md) diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 0000000..4338ab8 --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,204 @@ +# Architecture + +## Overview + +VirtualFS is a virtual filesystem layer for .NET applications. It combines multiple storage backends into one unified virtual tree. + +At a high level: + +- every backend implements `IFileSystem` +- a `RootFileSystem` mounts many backends under virtual paths +- callers operate on virtual paths like `/assets/logo.png` +- the root dispatches operations to the mounted backend that owns the target path + +This gives the library an overlay-style design similar to a mount table. + +## Core Types + +### `Path` + +VirtualFS uses its own `VirtualFS.Path` type instead of platform-native path handling. + +Rules: + +- all paths are rooted +- `/` is the root +- a trailing `/` means directory +- no trailing `/` means file +- separator is always `/` +- `.` is ignored +- `..` is normalized but cannot escape root + +Examples: + +```csharp +using VirtualFS; + +Path root = new Path(); // "/" +Path dir = "/content/"; // directory +Path file = "/content/app.js"; // file +Path combined = dir + "app.js"; // "/content/app.js" +``` + +Important: + +- `/content` and `/content/` are different paths + +### `Entry`, `Directory`, `File` + +The public object model is based on three types: + +- `Entry` +- `Directory` +- `File` + +`Entry` contains: + +- path +- owning filesystem +- read-only state +- timestamps + +`Directory` adds: + +- enumeration helpers +- child lookup +- directory creation +- file creation +- delete operations + +`File` adds: + +- open for read/write +- convenience read helpers +- convenience write helpers +- delete operations + +### Local Path vs Full Path + +For entries returned from mounted filesystems: + +- `Path` is local to the owning backend +- `FullPath` is the path as seen from the root + +This distinction matters when the same backend is mounted under a non-root path. + +## Filesystem Model + +### `IFileSystem` + +Represents a backend filesystem. + +Main responsibilities: + +- existence checks +- metadata lookup +- enumeration +- create, delete, move, copy, rename +- file stream access +- optional mapping back to a physical path or URI-like location + +### `IRootFileSystem` + +Extends `IFileSystem` with mount management: + +- `Mount` +- `Umount` +- `GetMounted` +- `GetMountPath` +- `Root` + +### `BaseFileSystem` + +`BaseFileSystem` provides shared behavior for backend implementations. + +Important responsibilities: + +- tracks read-only state +- tracks mount priority +- tracks open stream count through `VirtualStream` +- provides generic copy/move behavior when a backend does not override it + +## Root Composition + +`RootFileSystem` is the composition layer. It is itself read-only, but mounted filesystems may be writable. + +Typical setup: + +```csharp +using System.IO; +using VirtualFS.Compressed; +using VirtualFS.Implementation; +using VirtualFS.Memory; +using VirtualFS.Physical; + +var root = new RootFileSystem(); +root.Mount(new PhysicalFileSystem(new DirectoryInfo("/srv/site")), "/"); +root.Mount(new ZipReadFileSystem(zipStream), "/assets/"); +root.Mount(new MemoryFileSystem(), "/generated/"); +``` + +After mounting: + +- `/index.html` can resolve from the physical filesystem +- `/assets/...` can resolve from the ZIP +- `/generated/...` can resolve from memory + +## Mount Priority + +When multiple filesystems are mounted under the same virtual directory, resolution order is controlled by `FileSystemMountPriority`. + +Values: + +- `High` +- `Normal` +- `Low` + +Current behavior: + +- `High` mounts are checked first +- `Low` mounts are checked last +- `Normal` mounts are placed between them + +This supports overlay patterns such as: + +- immutable base content from a ZIP +- writable overrides from disk +- generated files from memory + +## Local vs Rooted Operations + +Many `Directory` and `File` instance methods accept `forceLocal`. + +Behavior: + +- `forceLocal: false` uses the owning root filesystem if one exists +- `forceLocal: true` keeps the operation inside the mounted backend + +This matters when several filesystems overlap. + +## Copy and Move Behavior + +The library can copy and move across filesystem boundaries. + +Behavior to understand: + +- same-backend operations may use optimized backend-specific implementations +- cross-backend operations may fall back to stream-based copy logic +- moves across backends may become copy-then-delete +- deep remote copies can be slower than native host operations + +## Streams and Busy State + +Open streams are wrapped in `VirtualStream`. + +Purpose: + +- increment and decrement backend open counts +- keep backend-specific transport state alive while a stream is open +- support `IsBusy` + +Practical implication: + +- a busy filesystem cannot be unmounted +- callers should dispose streams promptly diff --git a/docs/backends.md b/docs/backends.md new file mode 100644 index 0000000..ed89d73 --- /dev/null +++ b/docs/backends.md @@ -0,0 +1,154 @@ +# Backends + +This page summarizes the built-in filesystem implementations in VirtualFS. + +## `PhysicalFileSystem` + +Backs a virtual subtree with a real local directory. + +Characteristics: + +- writable +- creates the root directory if it does not exist +- maps virtual `/` separators to host-native separators internally +- supports native file and directory metadata + +Good fit for: + +- local content directories +- writable application storage +- test fixtures backed by temporary directories + +## `MemoryFileSystem` + +Stores entries entirely in memory. + +Characteristics: + +- writable +- no physical path +- lightweight and fast for temporary content + +Good fit for: + +- generated content +- overlays +- temporary runtime state +- tests + +## `ZipReadFileSystem` + +Exposes a ZIP archive as a read-only virtual filesystem. + +Characteristics: + +- read-only +- backed by `SharpZipLib` +- requires a seekable stream +- optional path scoping to a ZIP subdirectory +- optional regex filtering + +Good fit for: + +- packaged assets +- static content bundles +- immutable deployment artifacts + +## `EmbeddedResourceFileSystem` + +Exposes assembly embedded resources as a read-only filesystem. + +Characteristics: + +- read-only +- backed by manifest resources +- optional root namespace trimming +- optional regex filtering + +Good fit for: + +- embedded templates +- application assets bundled in assemblies +- shipping immutable defaults inside an executable or library + +## `FtpFileSystem` + +FTP-backed implementation using `FluentFTP`. + +Characteristics: + +- writable +- supports file, directory, and symlink listings +- supports recursive directory deletion +- opens a fresh FTP connection per operation + +Good fit for: + +- remote FTP stores where a fuller client implementation is needed + +Notes: + +- network operations are naturally slower than local backends +- deep copy/move operations may become expensive + +## `SimpleFtpFileSystem` + +Minimal FTP implementation based on `FtpWebRequest`. + +Characteristics: + +- writable +- simpler than `FtpFileSystem` +- recursive delete is implemented in library code + +Good fit for: + +- compatibility scenarios +- cases where the simpler implementation is sufficient + +## `SimpleFtpCachedFileSystem` + +Caching layer on top of `SimpleFtpFileSystem`. + +Characteristics: + +- caches directory listings by path +- updates cache on some mutations +- reduces repeated directory enumeration cost + +Good fit for: + +- FTP trees with repeated read-heavy directory access + +## `SFtpFileSystem` + +SFTP-backed implementation using `SSH.NET`. + +Characteristics: + +- writable +- supports password authentication +- supports private key authentication +- can validate host fingerprint +- can forward exceptions to a callback + +Good fit for: + +- secure remote filesystem access over SFTP + +## Backend Selection Guidance + +Choose a backend based on the shape of the content: + +- use `PhysicalFileSystem` for local writable data +- use `MemoryFileSystem` for generated or ephemeral data +- use `ZipReadFileSystem` for immutable packaged assets +- use `EmbeddedResourceFileSystem` for assembly-bundled assets +- use `FtpFileSystem` or `SFtpFileSystem` for remote stores +- use `SimpleFtpCachedFileSystem` when FTP directory reads are repetitive + +Common combinations: + +- ZIP as a low-priority base layer + physical overrides as high priority +- physical content + memory-generated overlay +- embedded resources + memory patch layer diff --git a/docs/development.md b/docs/development.md new file mode 100644 index 0000000..fe6bd3d --- /dev/null +++ b/docs/development.md @@ -0,0 +1,112 @@ +# Development + +## Repository Layout + +Main folders: + +- `VirtualFS/` +- `VirtualFS/Implementation/` +- `VirtualFS/Physical/` +- `VirtualFS/Memory/` +- `VirtualFS/Compressed/` +- `VirtualFS/EmbeddedResource/` +- `VirtualFS.Tests/` + +Important files: + +- `VirtualFS.sln` +- `VirtualFS/VirtualFS.csproj` +- `VirtualFS.Tests/VirtualFS.Tests.csproj` +- `Dockerfile` +- `.github/workflows/docker.yml` + +## Target Frameworks + +Library targets: + +- `netstandard2.0` +- `net472` +- `net6.0` +- `net8.0` +- `net10.0` + +Tests target: + +- `net10.0` + +## Build and Test + +The repository uses a Docker-based build/test flow with the official .NET 10 SDK image. + +Run the full containerized verification: + +```bash +docker build --target test -t virtualfs:test . +``` + +This performs: + +- restore +- build +- test + +## Test Stack + +The test project currently uses: + +- xUnit v3 +- `Microsoft.NET.Test.Sdk` +- `coverlet.collector` + +## CI + +The repository workflow is: + +- `.github/workflows/docker.yml` + +It runs the same Docker build/test path on: + +- pushes +- pull requests + +## Adding a New Backend + +The expected implementation approach is: + +1. derive from `BaseFileSystem` +2. implement the abstract operations required by `IFileSystem` +3. return `VirtualStream` for open file handles +4. populate entry metadata consistently +5. respect read-only semantics +6. add focused tests for backend-specific behavior + +Backend responsibilities generally include: + +- existence checks +- metadata lookup +- enumeration +- create/delete +- read/write stream opening +- rename/move/copy where backend-specific optimization is useful + +## Design Characteristics to Keep in Mind + +These are important when changing the library: + +- path semantics are custom and intentionally `/`-based +- directory vs file is determined by trailing slash +- `RootFileSystem` composes mounted backends but is not itself writable +- `Entry.Path` is backend-local, `Entry.FullPath` is root-visible +- `EntryRealPath` may be unavailable for some backends +- unmounting depends on `IsBusy` +- generic copy/move logic may be correct but not optimal for remote backends + +## Documentation Structure + +Current documentation: + +- top-level overview: `README.md` +- architecture details: `docs/architecture.md` +- usage examples: `docs/usage.md` +- backend summary: `docs/backends.md` +- development notes: `docs/development.md` diff --git a/docs/usage.md b/docs/usage.md new file mode 100644 index 0000000..8fb9adb --- /dev/null +++ b/docs/usage.md @@ -0,0 +1,205 @@ +# Usage + +## Quick Start + +### Create a Root Filesystem + +```csharp +using System.IO; +using VirtualFS; +using VirtualFS.Implementation; +using VirtualFS.Memory; +using VirtualFS.Physical; + +var root = new RootFileSystem(); + +var physical = new PhysicalFileSystem(new DirectoryInfo("/srv/site")); +var generated = new MemoryFileSystem +{ + Priority = FileSystemMountPriority.High +}; + +root.Mount(physical, "/"); +root.Mount(generated, "/"); +``` + +In this setup: + +- files present in `generated` can override files from `physical` +- reads are resolved through the root +- writes go to the writable backend chosen for the path + +## Creating and Reading Files + +```csharp +var file = root.Root.FileCreate("hello.txt"); +file.WriteAllText("Hello from VirtualFS"); + +string text = root.GetFile("/hello.txt").ReadAllText(); +``` + +## Working Inside a Mounted Subtree + +```csharp +root.Mount(new MemoryFileSystem(), "/cache/"); + +var cacheDir = root.GetDirectory("/cache/"); +cacheDir.Create("images"); +cacheDir.GetDirectory("images/").FileCreate("thumb.txt").WriteAllText("ok"); +``` + +## Overlay Example + +```csharp +using System.IO; +using VirtualFS; +using VirtualFS.Compressed; +using VirtualFS.Implementation; +using VirtualFS.Physical; + +var root = new RootFileSystem(); + +var baseAssets = new ZipReadFileSystem(System.IO.File.OpenRead("base-assets.zip")) +{ + Priority = FileSystemMountPriority.Low +}; + +var overrides = new PhysicalFileSystem(new DirectoryInfo("./overrides")) +{ + Priority = FileSystemMountPriority.High +}; + +root.Mount(baseAssets, "/assets/"); +root.Mount(overrides, "/assets/"); +``` + +Reads under `/assets/` prefer files from `./overrides`, then fall back to the ZIP. + +## Enumerating Entries + +```csharp +foreach (var entry in root.Root.GetEntries()) +{ + Console.WriteLine($"{entry.FullPath} readonly={entry.IsReadOnly}"); +} +``` + +You can also enumerate only files or only directories: + +- `GetFiles()` +- `GetDirectories()` + +All enumeration methods support an optional regular-expression filter. + +## Using `Directory` + +Common operations: + +- `GetEntries()` +- `GetFiles()` +- `GetDirectories()` +- `Exists(name)` +- `GetEntry(name)` +- `GetFile(name)` +- `GetDirectory(name)` +- `Create(name)` +- `FileCreate(name)` +- `Delete(recursive)` + +Example: + +```csharp +var dir = root.GetDirectory("/content/"); + +if (!dir.Exists("images/")) + dir.Create("images"); + +var imageDir = dir.GetDirectory("images/"); +``` + +## Using `File` + +Common operations: + +- `OpenRead()` +- `OpenWrite()` +- `ReadAllBytes()` +- `ReadAllLines()` +- `ReadAllText()` +- `WriteAllBytes()` +- `WriteAllLines()` +- `WriteAllText()` +- `AppendAllBytes()` +- `AppendAllLines()` +- `AppendAllText()` +- `Delete()` + +Example: + +```csharp +var file = root.GetFile("/config/appsettings.json"); +var text = file.ReadAllText(); +``` + +## `forceLocal` + +Many `Directory` and `File` instance methods accept `forceLocal`. + +Use it when: + +- you explicitly want to bypass the root overlay behavior +- you want to operate only inside the backend that produced the entry + +Example: + +```csharp +var entry = root.GetFile("/assets/logo.png"); +using var localStream = entry.OpenRead(forceLocal: true); +``` + +## Path Examples + +```csharp +Path dir = "/docs/"; +Path file = dir + "readme.txt"; // "/docs/readme.txt" +Path parent = ((Path)"/docs/api/").Parent; // "/docs/" +``` + +Normalization: + +- `"/a/./b.txt"` becomes `"/a/b.txt"` +- `"/a/b/../c.txt"` becomes `"/a/c.txt"` + +## Copy, Move, Rename + +Available on `IFileSystem`: + +- `Copy` +- `Move` +- `ReName` + +Notes: + +- copy and move may work across different mounted backends +- cross-backend operations may be slower +- directory destinations are expected to be directory paths + +## Unmounting + +`Umount` succeeds only if the filesystem is not busy. + +That means: + +- open file streams must be closed first +- callers should use `using` or explicit disposal + +Example: + +```csharp +using (var stream = root.GetFile("/data/report.txt").OpenRead()) +{ + // use stream +} + +root.Umount(someFs); +```