The Marriage of Structure and Semantics
December 31, 2018
Prologue
I have a bad habit of tackling unnecessarily big topics at inopportune times. After a year of near-complete silence, I will make a gesture in that direction once more. As this is a particularly bodacious topic, I suspect that I will be coming back to it several times both with revisions to this post and with follow-on posts. But in just the same way as applies whenever you are confronted with seemingly overwhelming challenges, the best way to start is to start.
One of my less welcome aphorisms goes something like this:
Think about something long enough and you will be guaranteed to be...wrong
Content is a fundamentally hard problem and we have been thinking about it for a very long time. My nagging concern, and one that I cannot seem to shake (in part because it rears its head so frequently within the practical confines of projects), is that we may have thought ourselves into a corner when it comes to content. And I suspect that the area where we have collectively lost the handle on the problem of content is in how we think about and balance its two fundamental dimensions: structure and semantics. Now it is true that, in this, I may be demonstrating the validity of my own sardonic aphorism. My hope in returning to this aspect of the problem, and exploring how structure and semantics might be usefully separated and then individually optimized, we may find find a way to build a new and more durable marriage between the two domains. That is the hope anyway and as we well know, hope springs eternal.
Meeting the Demands of Today
Within my various project experiences, of which I have been blessed with so many good ones over the years including this year, it is increasingly clear that we have reached a tipping point of sorts. With the introduction of chatbots and voice assistants, not to mention Artificial Intelligence (AI) agents and an escalating demand for unfettered content interoperability between applications living anywhere within the enterprise or accessed anywhere in the cloud, we have been unceremoniously pushed off the edge of any illusions of predictability and control that we might have been unconsciously clinging to when we thought about our content assets.
The Meaning of Structure
It turns out that as soon as you try to move your content, the materials you prepare to inform and engage people, from one location to another and to continue working with it there, you run into interesting challenges. Among the stories that have descended to us from the early days of markup languages was the experience some encountered when they tried to move document files from one virtual machine within a mainframe to another virtual machine and the result was a catastrophic failure of the entire system. These challenges force us to ask some pretty hard questions. Why should the ways we structure our content matter so much to different software applications? What do we set out to do when we structure our content? And do we introduce supplemental meaning to our content when we structure it and if so what meanings do we convey?
In this context, by "structure" we mean the "physical organization" of content. And it does carry a meaning with it. If we try, as I have been doing repeatedly over the years, to be clear and focused when we are structuring content that will need to move between many applications, then we should isolate the meaning of structure around what I call the "publishing semantic". In effect, this is a reference to the language we use as publishing specialists when we want to organize information content in a way that is optimal for its management and for its delivery to, and use by, people. This publishing semantic has evolved over millennia and it features the ever-evolving practices around print publishing as well as the more emergent practices around what Ted Nelson termed "hypertext". Rather than downplaying this domain, I have spoken of it as the fundamental mechanism that sustains the advancement of knowledge in our world.
The Structure of Semantics
So what of semantics? Semantics refers to the study of meaning. If that were not bad enough, I generally continue with the observation that "meaning emerges within systems where applications lead to outcomes". To indulge in a little recursion, every word in this observation is "pregnant with meaning". One word I will pick on though is "emerges" as this tells us that meaning always emerges from working systems and in particular within language systems. It is not a far stretch to draw a more practical observation that meaning emerges from our content, from how we articulate, exchange, and interpret what we believe and what we want other people to believe. Somewhat uncomfortably for some of my good friends who prefer to see semantics as exclusively limited to what can be formalized for consumption by computers, this means that semantics comes after and is ultimately dependent upon content. But yes, among the systems and applications within which meaning emerges, finds utility, and yields outcomes, are automated computer systems. As this probably trumpets loudly enough, semantics is a challenging problem area and as such it is a worthy companion to structure in our understanding of content. I am, again unhelpfully, reminded of what Maurice Merleau-Ponty had to say: "Because we are in the world, we are condemned to meaning" (Phenomenology of Perception).
In slightly more practical terms, semantics refers to how we define, manage, and apply meaning to things - and this includes to content or shall we say to structures within our content. One very good thing that has resulted from our hitherto mediocre attempts to instruct computers on how to do something useful with semantics is that we have been forced to apply more and more formality to how we articulate semantics. Interestingly, at least to me, is the fact that in formalizing how we articulate semantics we find that what we are doing is structuring those articulations in ever more precise ways. We use structured content to declare our semantics and this observation can actually be used to help improve how we tackle the problem of defining, sharing, managing, and leveraging the semantics of an enterprise.
The Successful Marriage of Structure and Semantics
So from the above ruminations we come away with two things (in addition, perhaps, to a headache):
- Structure refers to the physical organization of content and we structure content so as to optimize how we manage content assets so we can deliver informative and compelling messages to people (what I refer to as the "publishing semantic").
- Semantics refers to the definition, management, application, and use of meaning that emerges from and in turn informs the systems (including automated system but in no way exclusively) at work within an enterprise that seek to achieve goals and that generate outcomes as a result.
So what is it that we are doing when we set out to combine structure and semantics in our content?
Beyond the unavoidable basics of applying the publishing semantic to our content, what we are doing when we apply semantics is relating components within our content structure to concepts that make sense within one or more systems where applications endeavour to achieve specific goals and where they will generate outcomes that range between stellar success and abject failure. It is for this reason that we talk sometimes of "annotating" content with reference to a given taxonomy of concepts - we are associating content components with nodes in a semantic model that has been formalized in some way. Why would we want to do that? Well perhaps the systems in question can leverage parts of the content to achieve their goals. If fact this is exactly why.
If our application is to help users to find specific content based on a unique context then the annotations we apply will help to filter what would otherwise be a mountain of content to the subset that will be relevant to that context. Or as another example, from a past project, perhaps a manufacturing application needs to extract very specific details from a legally-mandated document source and the annotations applied ensure that this extraction and application of content details works at a level of precision that will be compliant and will avoid catastrophic failure in the manufactured system. In this example, the semantics emerged from, and vitally lived within, a variety of systems including an external third party who controlled the legally-mandated document sources. Those semantics needed to be precisely articulated and then exactly applied to structural components within the document content.
While there is a great deal to be discussed about how exactly we should facilitate the integration of structure and semantics in content assets, such that the resulting assets can move continuously between the many systems and applications operating within an enterprise, the fundamentals are clear. The semantics we may wish to apply to content structures will exist outside of the content even if, as I highlighted earlier, those semantics are articulated with reference to discourse (the dynamic interchange of information content) and articulated using structured content framed for that purpose. What we must do then is find the simplest, most intuitive, and most easily managed way to make the necessary associations between our content structures and the semantics that may apply to them.
Perhaps uniquely, I have come to cast metadata in this role - as the overlap between, or intersection of, the structure and the semantic domains. I have further described it as a subset of content structure that formalizes and facilitates the connection of content components to semantic contexts. How much of the semantic domain needs to be reflected in the metadata may alter from case to case so long as the connection being established will be sufficient for its intended uses. It is this relation to "uses" that I think is critical. The "meta" in metadata actually means "after" and while this is almost universally ignored or downplayed I believe it is fundamental. If we stress this idea of "after", what we are saying with "metadata", I think, is that we are establishing the context of content structures for use by one or more system applications. And so long as we keep our notion of systems and applications sufficiently broad, we can see that this conception of metadata covers the full range of considerations including security, administration, discovery, and automated processing. It is also fully compatible with any and all possible ways of establishing the conceptual connections whether that uses "out-of-line" metadata (where the metadata exists totally separately from the content and linked to specific structural locations) or "inline" metadata (where the metadata exists within, and as part of, the content) or some fusion of the two.
It is down this path, I would contend, that we indeed find a practical way to separate the concerns of structure and semantics, to optimize each in turn, and then to facilitate the efficient and effective integration of the two domains in a happy, and resilient, marriage.
Mathematical Content
I actually do have some preferences on how metadata is physically integrated into content structures although even in these preferences there is flexibility about when in the content lifecycle it needs to happen. I myself, and perhaps this is a confession on several levels, like it when metadata structures are relatively complete and informative and then are incorporated by reference into the content structures as a physically simple way to establish the metadata association. This can be an attractive and intuitive approach from the perspective of the authoring experience (how the people responsible for preparing the content assets come to fully understand the contexts within which the content will be used). And this can be highly applicable late in the content lifecycle, when content assets are staged for use, when the metadata (which may be further resolved and expanded with reference to the associated semantic models) is leveraged when indexing the content for high-performance retrieval and high-precision application consumption.
As a strange association, I find myself thinking, of all things, of Alan Turing and his seminal "On Computable Numbers" (1937). What I specifically find myself thinking about is how the approach to handling metadata that I have only glanced upon above might be used to facilitate a more efficient and scalable method for processing content assets. What I have been exploring is how this approach to metadata, as the physical intersection of the domains of structure and semantics, might be used to optimize content processing scenarios as the standardized handling of sets, and this, again following Turing's lead, will also help us to see and understand the limits of content computability. So it looks like I am thinking of a potentially terrifying blog post called "On Computable Content".
All this is to say that there are some very interesting and very attractive reasons for looking at the structure and semantics of content with a different lens and then consciously forging a working solution that leverages the unique strengths of each domain. When all is said and done, what we should have is a marriage of structure and semantics that can stand the test of time and even withstand the many challenges that will invariably arise.
But perhaps I have been thinking about all this for too long...
By sea-girls wreathed with seaweed red and brown
Till human voices wake us, and we drown.
T.S. Eliot
That's a very interesting read, Joe. I'm wondering if you can point to any current examples of content-creation tools/methodologies that are addressing the problem, even in a rudimentary way, of ensuring that content is consumable by the applications that need it. Or is that still in the future?
Posted by: Larry_kunz | January 01, 2019 at 12:25 PM
Hi Larry. Sorry it has taken me so long to get back to you. An excellent question. I am thinking that an answer indeed lies in the future. There have been success stories where the vision has been realized within the specific context of a given organization - typically ones creating or sustaining large-scale equipment systems - but those techniques still await distillation into a fully repeatable method and system. No surprise, this is one of my skunk works projects - and it is a project being advanced in collaboration with others so I am routinely pulled back down to earth. The good news is that from all those stubbed toes and bumped foreheads we have accumulated the necessary lessons so this final step doesn't seem that big anymore.
Joe
Posted by: Joe Gollner | March 18, 2019 at 07:25 AM