The Ultimate Global File Server: LucidLink and Object Storage

How would you describe the ultimate global file server? Scalable? Secure? Easy-to Access? Affordable? What about all that and more?

LucidLink Filespaces is a distributed global file system for object storage that makes object storage  perform as if its local disk. There’s your easy-to-access, affordable scalability. And with security that’s second to none, you can’t beat the data protection. But what about the “all that and more”? Well, the use cases for LucidLink are numerous. Frankly, anywhere where one needs storage that doesn’t require local flash performance is a potential use case for LucidLink—and that’s no joke.

Object storage changed the storage landscape with its unparalleled level of redundancy, scalability, and unmatched economics when using it in the cloud. However, the ease of accessibility in conjunction with performance narrowed the use cases of object storage. With cloud object storage, the issues of distance and latency come into play, especially with large data sets, and thus have limited the potential use cases.

We’ve all heard the disclaimers that come with object storage … “Well, it’s great for archival purposes…” or “Performance is not what this is about…”

Whoa! … Hold on …

All this implies the unspoken obvious … putting my data on this local object storage or in this cloud object storage platform may work fine if I don’t need to access it. But when I do need to access it, well, it ain’t gonna be that great of an experience. Sometimes, many of us learned, the performance of accessing that data can be downright dismal.

Let’s be honest here, when we want our data, we want it now … no excuses!

Unfortunately over the years, there are many mothballed object storage appliances that currently have more use as a table for the IT guy’s tools and empty coffee cups rather than for their intended storage use. At best, many of these object storage implementations are in use but have not lived up to their expected or anticipated potential. The same goes for the cloud where use cases become limited due to the added issues of distance and latency.

There has been an abundance of solutions out on the market in recent years to address these limitations of object storage, especially with the cloud: sync-and-share solutions, storage gateways, etc. Still, these solutions fail to solve all the problems, especially with larger data sets.

LucidLink has solved these problems. LucidLink has made object storage and its inherent benefits available for many applications where object storage alone would not even be considered as a viable option, or at best, be limited in its implementation.

How does LucidLink solve these problems?

First and foremost, LucidLink places a cloud-native distributed file system on object storage where many users using any of the major operating systems can access it concurrently—all via a folder on the local client. It’s important to note that LucidLink honors local file system permissions as well.

Secondly, LucidLink enables unparalleled performance with object storage where only the data bits required at the current time are streamed to and from the client and storage. Data is divided up into blocks that equate to objects in the storage bucket. The default size is 256KB, and this is configurable. Typically with object storage, one file equals one object. With larger data sets, this becomes impractical. With LucidLink, one file equals many objects. Multiple parallel streams to and from the storage are used. The default is 64 parallel streams to and from the storage. This is also configurable. Intelligent read prefetching is utilized along with local disk caching. Local disk caching is set to only 5GB by default (adequate for most cases) and can be enlarged up to 1TB.

Finally, LucidLink provides a “Zero-Knowledge” security model that encrypts all the data starting in the local cache on the client, in-flight, and in the object storage itself—and only the customer has the encryption key. Essentially, this antiquates the encryption-at-rest security model. Neither LucidLink nor the cloud storage provider have access to the data. Additionally, users can be configured where RO and RW permissions can be granted to folders while still adhering to LucidLink’s “Zero-Knowledge” security model.

With this all said, we can get to the point of this blog post … creating the ultimate file server using LucidLink and object storage.

With a local object storage implementation using MinIO, Ceph, Scality, Cloudian, and others, one can now merge the benefits of object storage with that of a distributed file system, data security, and best of all—performance.

Extending LucidLink to the cloud make it even more attractive. Cloud object storage offerings such as Amazon S3, Microsoft Azure, Google Cloud, IBM Cloud, Wasabi, and others can be leveraged with LucidLink where Windows, Linux, and macOS clients located geographically anywhere can have concurrent access to files. In the case of a cloud storage architecture, LucidLink is the last storage upgrade one will ever need.

Building a file server in the cloud with LucidLink is quite simple. Referencing our support site, we have a plethora of step-by-step information for just about anything one would like to do.

• Managing Filespace Users
• Administering Filespace Shares

One could even take the route of setting up a local NFS or SMB file server but allocate the disk space for it in the cloud via LucidLink.

• LucidLink and SMB file server
• LucidLink and NFS file server

Think about it … isn’t a secure, distributed global file server a dream come true?

Feel free to contact us with any additional questions and check out our free trial!