We’ve covered a lot already – so it’s time to talk about something that we’ve only touched on before. Metadata. Welcome to episode six of our short series explaining SymplyPERIFERY – our S3 native scalable storage solution.
Let’s get meta. Whatever that means. We’ve mentioned metadata several times in the course of this series. The fact that it’s encapsulated with objects and that it can be used to drive Perifery behaviour such as the object life cycle. But what is it? Metadata is… well it’s data about data. In our case it’s data that describes an object – a video or audio file, an image, an x-ray, a legal document, or any other kind of object stored on SymplyPERIFERY. Let’s use my apple from episode two. Although I ate that, so here’s a new one.
Previously, we talked about how when data is written to disks in SymplyPERFIERY it’s always stored as two parts: object data and metadata. These are written strictly once and always remain physically encapsulated together. Unlike other object storage solutions that keep the data and metadata separately – typically in a database – SymplyPERIFERY is uniquely metadata aware as the metadata itself is stored directly alongside the object it’s describing. Storage that uses file systems struggle to do this. Historically, with file systems, there was no good way to easily get additional metadata associated with a file – you need a database to do that. My apple was grown in my friend Tamsin’s garden. It was picked by my friend Hazel. It’s small, tasty, organic, green and red, and it’s being held by me. That’s metadata that describes this apple. In SymplyPERIFERY that data sticks to the apple. In other systems that data is stored far, far away so it takes longer to get, search, and retrieve the relevant object.
This is what makes SymplyPERIFERY incredibly powerful and is game changing for data at scale.
Sounds interesting, but what does it mean?
In practical terms it means speed. If you’ve been watching our channel you’ll know that SymplyPEFIFERY is fast and one of the reasons for that is we don’t use databases. No database has to be stored on expensive flash media, and no databases to consume valuable CPU resources. And no need for the system to update both the object and a database containing the metadata about the object. That’s a lot of faff.
With SymplyPERIFERY there’s also no practical limit on the amount of metadata that can be associated with any given object, essentially the list could be endless. With many S3 systems there’s a surprisingly small number of tags that can be added. Which is a shame really, because the more metadata you add the easier it is to find the content you want at a later date, in a day, a year or maybe in the next decade.
In addition, this metadata can be used by applications and be updated in real time. For example applications could search this metadata for documents, images, videos containing that metadata and these applications could also update the metadata back on the SymplyPERIFERY storage itself. Think of it as a realt-time two way door: an application creates new metadata fields, populates them and stores that back in PERIFERY so that a second application two will also have immediate visibility of the new and updated metadata stored with the object. In fact any and all applications that connect to SymplyPERFIERY can access that metadata. Back to my apple: if I start working with it and decide that I want to describe its surface as smooth, I can create a new metadata field to add that detail. As soon as I do, anyone else with access to the storage can see it and search for it.
Your content, whatever it may be, is nothing without metadata. It’s the key to finding what you’ve stored, Once metadata is written or updated with an object in SymplyPEFIERY, it’s encapsulated with it and this stays with that object forever, no matter where it’s moved to – from editor to editor, application to application, between clusters, to the cloud or into file systems.
Next time we’ll talk about what makes SymplyPERIFERY more environmentally friendly than other systems along with how your PERIFERY cluster can scale almost infinitely.
