Introduction to Data and Encoding
What do you think
happens when you open a file?
How do you think
a program/machine identifies
a file?
How do you usually
identify a file?
What is there to identify?
What is a digital file?
Wikipedia:
Filename extension
Most people identify what a file is, according to its filename - and
the filetype according to its suffix after the “.” dot.
If all is well, usually a quick and sane choice, but there’s more…
What kind of files are there?
Understanding digital objects
Bit :A single binary digit (0/1) Byte :A unit: 8 bits (half = Nibble ) File :Stored segment or block of information available to a computer
program File
system :A mechanism for controlling and organizing bytes into structure
(files/folders) for storage and retrieval File
Format :A standard way that information is encoded in a computer
file.
Identifying files
Directory listing example
What can you say about these files?
These file properties (filename, date/time, size, ownership, access
rights, flags) can often be used to say something/more about a digital
object, therefore it’s good to consider preserving this layer of
information too.
For example, when documenting the original state of externally acquired
collections/objects. More about this in the metadata session…
The Filesystem
Filename
Date/time
Filesize
File extension
Path
Access rights
Without a filesystem, data on a storage device is just a long string
of numbers… No beginning, no end, no structure, no folders, no files.
Just bytes!
If your filesystem is broken, you can’t access your data - although the
“data” is actually exactly where it was. Untouched. But there’s no “map”
to find where to go, and where a file starts or ends.
The 2 major types of Data
Everything’s a number
ASCII Table
Each byte in a file is a number. Depending on the encoding, each
number maps to a certain character. This table shows a common character
encoding: “ASCII”
(American Standard Code for Information Interchange)
This view also shows the hexadecimal (short “hex”) value which
is more common and better to view data as, than decimal.
Character encoding
See: Character
sets, encodings, and Unicode (By Nick Gammon)
Classic “code pages” work fine for the language/region they are
designed for. Mixing characters from different languages is a problem
with this approach though!
Mis-interpreting a character by applying the wrong codepage is the
reason for encoding errors. For practical and history reasons, the ASCII
set is usually mapped compatible across all codepages.
Encoding Interoperability
“Sch�ner Tag. Recht hei�. (□ )”
Schöner Tag. Recht heiß. (🙃)
□ (WHITE SQUARE, U+25A1): Replaces a missing or unsupported Unicode
character. � (REPLACEMENT CHARACTER, U+FFFD): Replaces an invalid or
unrecognizable character. Indicates a Unicode error.
Unicode
“Unicode is a computing industry standard for the consistent
encoding, representation, and handling of text expressed in most of the
world’s writing systems.”
– Wikipedia:
Unicode
Mixing languages
Лорем ипсум долор сит амет
See: UTF-8 encoding
table
Comments?
Questions?