Identifying the problem spaces in story sifting for Twine

The authoring tool Twine has increasingly seen greater steps toward more dynamic narrative design approaches across story formats and associated libraries. This can be first seen with TinyQBN (2019), a library for use with the story format SugarCube to create and manipulate “storylets.” Originally introduced as part of what would become the game Fallen London (2009), the design team behind the game coined the term “storylets” to describe the “basic building block[s]” of their narrative work where a selection of content would become available over others based on the current values of “qualities” (variables) within the story (Failbetter Games, 2010). Building on the work of Kreminski and Wardrip-Fruin (2018), Emily Short has also written extensively about historical and more current examples of the use of storylets in games and other interactive projects (Short 2019, 2020, 2023). Harlowe, the default story format for Twine 2, also adopted its own implementation of storylets in 2021. While direct support is planned for the story format SugarCube and Snowman in coming versions, the flurry of activity between 2019 to 2021 has mostly slowed as many developers and researchers confront the multiple problem spaces captured within the original description of “story recognition” described by Ryan et al. (2015) as part of a paper on the “open design challenges” for emergent narrative design and authoring tools.

To narrow the problem of “story recognition,” Kreminski et al. (2019) and Leong et al. (2022) have adopted the term “story sifting,” a process through which “specific criteria that are appropriate for the domain of interest” are applied to find “event sequences and patterns” within some larger collection (Leong et al., 2022). This moves the focus from “recognition,” a potentially very complicated problem dealing with story arcs, narrative completion, and player interpretation of events, into the organization and searching of story content based on specific criteria, helping to better define potential solutions based on domain-specific languages (DSLs) and other approaches. This shift can be seen in the continued work of Kreminski et al. (2022) on “heuristics” for story sifting and even Leong et al.’s (2022) presentation of a tool to “sift” story content based on search queries into potential timelines of event sequences. More general work has focused on identifying and sifting larger collections of content.

This shift toward story sifting as a focus for emergent and dynamic narrative design helps illustrate two ongoing problems faced by potential applications of this design:

Need for Domain-Specific Languages

In order to find the next event or story content, it must be identified in some manner from other, similar content. This can be seen in the hyphenate Grams Notation used in TinyQBN, the Winnow language developed by Kreminski et al. (2021), Harlowe’s when expressions, the view-if conditional approach of Dendry (2014), and even my own more more recent work on Quis (2023). Leong et al. (2022) explain the problem as the need for “manual specification of search queries [. . .] in a technical language.” While some work has been done to allow tools to write their own searches, most implementations, including those using storylets, author the conditions under which microstories or storylets become available and use variables, global collection, or state to dynamically filter results during run-time.

Within Twine, the use of a DSL is complicated by the parsing of passage content. While each passage can be thought of as a discrete “unit” of a story, searching through a collection requires, as with TinyQBN and Harlowe, the use of a language to describe when the content is available for use or otherwise can be selected. As with the articulation by Leong et al. (2022), this requires the “manual specification” of the conditions as part of authoring passages in Twine.

As the conversation around the recent adoption of the Twine 2 JSON specification also outlines, using external tools or processes to generate passages and then importing them during story compilation with Twine-compatible tools is often frustrated by the story uniqueness issue of names and the Interactive Fiction IDentifier (IFID) of each work across input and output formats. Twine will allow multiple stories to use the same IFID in a library but not the same name. Tools like Extwee often only translate between formats or perform compilation without regard to uniqueness. Story compilation processes may eventually allow the re-use or selective inclusion of large collections of passages in the same projects or stories in the future, but current tools struggle with such an approach.

Need for Module Expression and Representation

Noted as part of the “open challenges” for emergency narrative design by Ryan et al. (2015), a central challenge still faced by authoring tools is in its use of “modular content” and ability to “express the underlying system state to the player.” As the amount of story content grows, so too does the need for faster “sifting” to quickly find, understand, and potentially present story content to players. Often, large collections of content need to parsed or translated into a secondary or searchable format. The Winnow language, for example, uses DataScript, a JavaScript implementation of the language Clojure. This requires story data be present in a database through which a parsing program for the Winnow query language can access. TinyQBN, Harlowe, and Quis all provide their own DSLs to translate conditional statements. Twine 2 HTML elements are parsed into JavaScript structures and searches performed on these internal collections based on secondary representations of the author’s use of the DSL involved.

As Twine and other compatible tools slowly add support for JSON in coming months to years, this opens the possibility to support string-based representations where additional DSLs could be encoded alongside other passage metadata within a specific format. It could be possible to generate large collections of story content and then selectively include, either during story compilation or run-time searches, selections of them based on stored metadata separate from passage content. However, this might also mean changes to the existing format specifications. The ability to edit passage metadata directly in Twine would also speed this progress, allowing the use of additional key-value pairs when editing passages, but such functionality seems unlikely to be added in the short term. While the Twee 3 format supports passage metadata, Twine 2 HTML does not, for example. Those using external tools to edit Twine-compatible formats can encode extra values those using Twine itself cannot.

What’s next?

The arrival of storylet support in upcoming versions of SugarCube and Snowman will bring the use of the approach up to three of the four built-in story formats in Twine. Given this, chances are high Chapbook would also add such functionality soon after most of the other story formats included it. At the same time, active research into story sifting domain-specific languages show a strong interest in trying to better understand and propose solutions to the problem of needing a DSL and module content problem spaces. A more recent approach to creating storylets in ink (mirroring my own approach in Chapter 9 of the Dynamic Story Scripting with the ink Scripting Language book) also shows ongoing interest in storylets as an approach across multiple languages and tools.