Keep your META where your DATA is. In the filesystem?

Peter Bubestinger-Steindl
(p.bubestinger@ArkThis.com)

November 2023

What’s this all about?

Metadata- and File-Wrangling.
Professional and Private.

My “AHA” Effect…

  • Simply making use of “name=value” tags;

  • Moving metadata to the same level as a file or foldername;

  • Keeping the META where the DATA is.

  • Allow using the filesystem directly as database.

  • All FOSS and standard and awesome, of course.

What’s with the “Holodeck”?

Yes, I’m trying to be funny.
Yes, I’m very serious about this idea.

My Perception:

Current Project Desires:

“Make things all digital and awesome! And easy. In no-time.”

My Perception:

Disclaimer

What I’m proposing is not a new technology.

I’m suggesting a different usage of what already exists.

And I apologize: I may be an ad for FOSS.

Existing Components

  • “Object Storages” already support payload with metadata.
    (Jonáš Svatoš has practical insights on object storages)

  • Local “search-tools” already deal with indexing and UIs for existing meta+data.
    (Marion Jaks has experience with in-house DAM)

  • Other base components already exist too.

  • Things need to be orchestrated.

  • And used/tested/tuned.

https://min.io/resources/img/subnet/subnet-instantaneous.gif

Where to begin?

Terms

  • Files/Folders?
  • Metadata?
  • Data “payload”?
  • Data Objects?
  • Filesystem?
  • All clear?

Status quo: Assigning “a plain title”?

Angea Albertini: PNG File Format Specs

See: “Keywords and Text Strings” (PNG Specification, W3.org)
And: “What software can I use to read png metadata?”

The Unix Philosophy

“Everything is a file.” See: https://en.wikipedia.org/wiki/Unix_philosophy

If we translate files to Objects, it becomes…?

The AHA-Holodeck Philosophy

“Everything is an Object.”

  • Anything can have its metadata with its payload content.

  • As simple a Lego and to-be taken for granted like filenames.

  • The filesystem serves as (semantic-graph-) database.

What if you could simply…?

  • store (and use) any metadata reliably/persistent with its payload content?

  • drag-n-drop, copy/paste, convert, view, edit any metadata:
    In any file manager or tool?

  • import/export catalog (database) entries, like copy/pasting files?

https://arstechnica.com/gaming/2016/09/how-time-travel-works-in-star-trek/3/

What if…?

  • that was as easy as using file/foldernames?

  • but without classical restrictions?

  • over its whole lifecycle?

  • you can store hashcodes with any file format?
    Even for .txt or .bin?
    By default?

https://en.wikipedia.org/wiki/Back_to_the_Future_Part_II

What if…?

  • you don’t have to worry/think about “naming a file” anymore? (even keep multiple filenames in parallel (if you like))

  • same was true for folders?

  • you could keep as many (nested) “name=value” information as you’d like?

  • this would work from local/small (USB-Stick) to network/large scale storage?

https://min.io/docs/minio/linux/reference/minio-mc/mc-sql.html

Metadata-only Objects = Catalog entries

  • Right-click “New Object”:
    Annotate as desired.
    No payload required.

  • Relate two Data Objects:
    Drag-n-drop, then describe Relationship(-Object).

  • Any “Catalog Object” could also have a payload. (eg preview image/icon)

Interoperability of Features

  • Any music/photo/collection application becomes (even (more)) compatible.

  • Metadata is transformed on access/demand.

  • Metadata exchange and usage between systems is facilitated.

Clementine Music Library/Player

Embedded metadata?

What is the use-case for embedding any metadata?

So the meta stays with the data!

Media container formats?

What is the use-case for (media) container formats?

So related data stays together!

Performance?!

  • Size?
  • Speed?
  • Interoperability?
  • MacGyver-able?
https://imgflip.com/i/84lqea

Where to begin (implementing this)?

What to feed it with?

  • Real-world collection (web-)access copies.
  • Corresponding data (catalog) entries. (XML, JSON, CSV, etc)
  • …More ideas?

The rabbit hole goes deeper.

But that’s a story for another time…

If such an implementation/prototype is not awesome…

…then it’s not what I suggest here 😎️
(Or simply not finished yet)

Oh, btw:

  • IMO, we all will sooner or later use Object storages, because hierarchical filesystems don’t scale well enough anymore (with today’s use cases and sizes)

  • So you’ll have the underbelly to support metadata-with-payload Objects anyways 🤩️

Min.IO: Website Screenshot

Ideas? Input? Questions?

Please! Go ahead. Now and later :)

Peter Bubestinger-Steindl
Peter@ArkThis.com

https://github.com/ArkThis/AHA_ObjectWorld/
https://diode.av-rd.com/nextcloud/index.php/s/z2M4JZY8RFt8Nnd

CC-BY-SA