What is LucidLink?

LucidLink is a new distributed file system technology that streams data directly from any object store.

This is a distributed streaming file system, purpose built for high latency environments like the Internet, and specifically as a POSIX type of front end for cloud object storage.

We have not seen any other product like this out there. Instead, companies rely on old protocols which were not meant for this kind of environment (NFS, CIFS, SMB), which break down due to latency, and which require re-writing applications or unnatural workarounds.

What’s wrong with these “old” protocols? (NFS, CIFS, SMB)

There is nothing wrong with them – as long as they are used as intended in local low latency networks! However, as soon as you introduce distance (and latency), they break down.

If you have ever had the frustrating experience of connecting to your corporate file server through a VPN while traveling or at home, you know what we mean!

How is LucidLink different?

LucidLink is a service which work in conjunction with the LucidLink app.

The service coordinates and distributes metadata from your files stored in cloud object storage (Amazon S3, Microsoft Azure, Google storage, Wasabi etc.). The app inserts this metadata as a local file system within the OS.

Whatever data is contained within the cloud object storage in the global namespace, shows up “virtually” here as a locally stored file. When the application opens the file, the data from that file is streamed directly from the object bucket to the device.

Do I need to learn a new app and integrate it into my production workflow?


LucidLink natively integrates with the host OS at the file system level. Whatever application you are using can simply be pointed at this folder or mount point, and the application will use the files there as if they are stored locally.

The only change you need to make to your workflow is where your application gets and stores its files.

What Cloud Object Storage providers are supported?

LucidLink supports all object storage providers. Our service is designed to capitalize on a provider’s REST API libraries. Amazon S3, Microsoft Azure Blob or S3 compatible object stores are supported.

What language is the application written in?

It is written in C++.

From the outset we decided to build this service for scale and performance. We are old enterprise storage guys who know the importance of these attributes! We also took great care to ensure all OS versions are supported by the same code base.

Although much more difficult from the beginning to implement, this pays off tremendously in our ability to innovate and support all of our varied customers in the future.

How is LucidLink different from sync solutions?

Sync is the big hammer approach, and was the first technology to address the issue over 10 years ago by Dropbox and Box. The idea is simple, put a copy of the file wherever you may need to access it, and keep changes synchronized across all devices.

The approach is OK for individual users with smaller sized files and data sets with casual sharing requirements, and both companies have achieved great success with their services.However as data sets and file sizes grow, this approach does not easily scale to production use for primary data. As anyone who has a large data set can attest, it is just too noisy for production.

While there are solutions for those seeking to use data stored in these services – generally connectors which pull the data from the service for specific applications – they are fraught with compromise. Users still need to inefficiently download and sync files across multiple locations, consuming precious time and resources.

Besides the noisy overhead, one of the biggest downsides we hear from users is the lack of customer control over where and how their data is stored. Supreme trust in your provider is a prerequisite.

Accessing object storage via file sync products is possible, they just weren’t designed for direct application transactions relying on primary data.

LucidLink is a distributed file system optimized for object storage. It is a true cloud approach to the architecture, dealing with both the challenges and the advantages that cloud object storage present. LucidLink distributes metadata to the devices, removing the bulk of the chatter over the wire between applications and the file system. This allows us to transfer the data much more efficiently to the client. We further enhance performance through caching and pre-fetching.