One folder, streamed from several servers at once
You give the SpliceFS client one host name. It logs in there, gets the list of every server behind that name, and from then on asks all of them. It fetches the list again every minute, so servers can join and leave.
- Finding files. When you open a folder, the client asks every server what it has there and shows you the combined answer. A server that does not have the folder says so, which costs almost nothing. The client asks it again the next time, because the folder may turn up there later.
- Reading files. A file is read only from the servers that said they have it. Each of them sends different blocks at the same time, so their speeds add up. The client puts the blocks in order and keeps them in a cache on local disk.
- Slow and failing servers. The client measures every server as it reads. A slow server gets less work, and only blocks that are not needed yet. A server that fails is rested, for one second at first and for at most thirty, and then tried again, so it carries its share again as soon as it recovers.
Status: alpha, step one. Read-only is the first step toward a complete distributed filesystem. The first 0.x versions are published as packages. It has not run in production. See how it compares with mature systems today.
Native packages
- Debian12, 13deb, apt repository
- Ubuntu22.04, 24.04, 26.04deb, apt repository
- Red Hat Enterprise LinuxRocky Linux AlmaLinux8, 9, 10rpm repository, tested on Rocky Linux
- Amazon Linux2023rpm repository
- openSUSE Leap16.0rpm repository
- Alpine Linux3.22, 3.23, 3.24apk repository
- FreeBSD14, 15pkg, server and client. Built and tested, not published yet.
- macOS (Apple silicon)Homebrew tapToday the formulas build from source.
Linux packages are built for x86-64 and arm64, signed, install-tested on every version listed, and published.
- CompareAn honest look next to NFS, SMB, CephFS, GlusterFS, BitTorrent, Resilio Sync, Syncthing and Rclone, and where it is headed
- How it worksHow the client, the servers and the blocks of a file fit together, with diagrams
- InstallPackages for Debian, Ubuntu, RHEL, Rocky Linux, AlmaLinux and Alpine
- Serversplicefsd, source and documentation
- Clientsplicefs, source and documentation
Project status
Live results of the builds, tests and security scans of every repository. Each badge links to its report.
serversplicefsd, the server
clientsplicefs, the client
commonshared Go packages
packagesinstall script and package site
homebrew-tapHomebrew formulas for macOS