Orchestration Scripts: A System for Encoding an Organization’s Ways of Working to Support Situated Work

2.1 Ill-Structured Work and How Socio-Technical Workplaces Support It

Modern work and learning communities require their members to tackle complex, ill-structured problems, such as in research, engineering, design, and entrepreneurship work. These problems are challenging due to the uncertainty involved when working on them, where work needs change as progress is made and resources from around the organization must be coordinated to resolve them [37, 58]. Working effectively on these problems requires developing situated work practices where workers determine a course of action by assessing their work needs and what forms of support are available, adapting their plans as their needs and the availability of resources change over time [62].

Emerging needs often require support from others when workers do not have the resources or skills to address them on their own [30, 36]. In response, workplaces have focused on providing increased access to support opportunities, including peers, collaboration venues, and resources [35, 41]. This includes shifting away from strict hierarchies and siloed teams towards more networked models of organization [52, 59]; adopting more flexible work processes (e.g., Agile practices that encourage rapid iteration versus waterfall ones [19, 63, 65]); and work and collaboration tools (e.g., Asana [1]; Jira [5]; Trello [8]) that support identifying work needs and collaborating across social structures in the organization [27].

Despite increased access to support, enacting effective situated work practices remains challenging [31]. Workers must continually monitor for emerging work situations; identify and plan relevant work strategies; and enact those strategies across interactions along the work process, re-planning and re-strategizing as progress is made. However, they struggle with monitoring for their needs [30]; determining relevant strategies in those situations [17, 42]; and knowing when and where support across the organization can be found [11, 12]. Personalized coaching from mentors can help [20, 70], but it becomes infeasible since people’s work practices are distributed across situated interactions throughout the ecosystem. Given the challenges with human-only solutions, we must consider how technology can support situated work by understanding the work situations people encounter and surface situated strategies they can attempt at relevant times along the work process.

2.2 Tools to Support Common Work Activities

Unfortunately, many existing workplace tools leave the difficult task of actually orchestrating situated work practices across an ecosystem entirely to the user. While these tools support workers across different work activities (e.g., planning [1, 5, 8], help-seeking [13, 14, 34, 44, 45], and reflection [53]) and in particular venues (e.g., collaboration activities in the classroom [25, 28, 38, 47, 72]) by helping them strategize on how to resolve known work needs (e.g., guides for assessing project risks when struggling with planning [42]), they provide little support for monitoring emerging needs in the first place. Further, they do not help enact relevant strategies in the appropriate support opportunities across the work ecosystem (e.g., at a venue dedicated to planning). Instead, our work aims to develop systems that support these facets of orchestrating work and provide relevant strategies to practice.

2.3 Automated Approaches That Detect Work Needs and Facilitate Work Processes

Closer to this goal, significant work has studied workflow automation approaches for business processes in organizations [32, 46, 68, 73], and trigger-action programming systems where actions are contextually triggered based on a monitored condition (e.g., for online governance [74], context-aware computing [56, 66, 69], workplaces [7, 10, 55], and more [4, 67]). These approaches encode data-driven workflows where a sequence of actions is initiated across disparate systems when data is added, modified, or matches a set condition. For example, when a worker has too many tasks assigned to them on a Trello board during a work sprint, a workflow can message the team over Slack to discuss how to triage tasks at the start of their planning meeting stored in Google Calendar.

However, providing programming primitives at this level limits our ability to encode situated work practices. A key challenge is a difference in abstraction between what workers want to express about effective work practices and the programming model workflow tools use. For example, we may want to say, “when a worker is overcommitted based on the number of tasks, inform their team at the next planning meeting.” Instead, we must express the specifics of how to enact the script to the system (e.g., the worker’s relevant board and tasks, their planning meeting time, and their team’s communication channel) for each individual worker rather than the general strategy that applies to all of them. Not only is this level of encoding removed from how we think about situated work practices, but it also makes the workflows brittle to changes (e.g., a different team manager or planning venue time) since a script author must manually revise all workflows that reference these data. Without technology that understands where and with whom work activities are happening (e.g., a planning or collaboration venue; teams and their leads), we will continue to struggle with developing software that lets us express the kinds of engagement with others that we want workers to have along their work process.

Finally, a general challenge with any trigger-action-based system is its philosophy of execution. When the trigger condition is realized, the desired system behavior is to execute any follow-up actions assigned to the trigger. However, work processes in practice–especially for the ill-structured problems found in the workplace–are not easy to codify due to the day-to-day complexities and uncertainties of the work [36, 50, 60, 61]. As a result, these approaches tend to be overly prescriptive in how to resolve work needs and can fail if systems lack essential tacit knowledge workers have about their needs [33].

In our work, we consider how to develop workplace systems that use computational abstractions of an organization’s ways of working to support situated work. We develop programming constructs that provide access to a workplace’s work processes, social relationships, collaboration structures, and data from its productivity tools. Through these constructs, we can encode models of situated work that inform us of when work situations arise and potential strategies to resolve them along our work processes but leave the locus of control on how to what strategies to enact to the workers (rather than automated systems).

3.1 Design Goals

For systems to orchestrate situated activities to resolve work needs, they must model the work situations and relevant strategies to practice, and where to attempt them [31]. These activities occur along an organization’s work processes, across social structures and collaboration venues, and are supported by various productivity tools. For example, a team may recognize a planning-related issue during a work session and bring that into a weekly planning meeting with their mentor later to address it. This suggests that modeling these situated activities requires that systems understand these ways of working. Therefore, our first design goal is to enable the encoding of situated strategies using computational constructs of how work is done in an organization.

Moreover, organizations often have shared practices and tools across people and teams, but the times and places where practices are enacted may differ. For example, two project teams follow the same planning practices where planning-related work situations and strategies should be raised at planning venues, but these venues occur with different team leads on different days. This suggests that scripts modeling general work situations and strategies relevant to the organization as a whole are useful but must be enacted specifically for each worker’s individual context (i.e., present relevant planning strategies at their planning session to their team and lead). Therefore, our second design goal is to allow for the expression of general situated work practices that become tailored to different people’s specific situations.

Finally, enacting situated activities requires systems to continually monitor for work situations to occur and present situated strategies at appropriate places and times (i.e., at specific venues along the work process). However, the moments when work situations arise and when situated strategies should be presented are often different. For instance, a team may want to discuss planning-related issues during their weekly planning session rather than at other times during the week. Moreover, these venues occur at different times for each worker or team in the organization. This suggests that an engine for supporting situated work needs to track work situations and enact support strategies at relevant points of the work process separately for each worker or team.

3.2 Encoding Situated Strategies in Terms of an Organization’s Ways of Working

Figure 1:

View Figure

To encode programs that support situated work, we introduce Organizational Objects that provide computational abstractions of an organization’s ways of working. To create this object, we included base-level constructs for work processes, social structures, venues, and productivity tools. These constructs can be used to generally model how work is done in an organization and customized to each specific organization. For example, an Agile workplace’s Organizational Objects may include sprint cycles for their work processes, teams and their leads for social structures, planning-specific and general office hours venues, and tools for Agile planning (e.g., Trello) and asynchronous communication (e.g., Slack) [19, 63]. In addition, we provide Helper Functions for common computations on the Organizational Objects, such as presenting a strategy the morning before a venue. Through these constructs, general situated work practices tied to an organization’s ways of working can be expressed to software and surfaced at relevant times.

To operationalize Organizational Objects, we created a Studio API that provides a queryable instrumentation of the organization’s ways of working; see Figure 1. It does this by collecting information about the organization’s work processes, social relationships between members, collaboration venues, and traces of data from the organization’s existing tools (e.g., Trello for planning purposes; Slack for asynchronous communication) into a single space. Moreover, these data can be mapped to each other (e.g., Trello board and Slack channel for a team), allowing us to query for different aspects of the relationships between people, their venues, and tools, such as when a project lead has their next office hours. This allows the general practices described using the Organizational Objects from earlier to become executable in organizations, allowing systems to coordinate situated work practices.

3.3 Expressing General Strategies That Become Tailored for Specific People

Figure 2:

View Figure

Using the Organizational Objects, Situated Scripts can be composed to externalize general work situations when support is needed and suggested situated strategies to resolve them. A Situated Script is defined by an object containing:

Unlike automated systems that enact follow-up interactions once the work_situation_detector is met, a key idea behind Situated Scripts is to surface relevant strategies to workers but ultimately let them decide whether to enact those strategies. This allows for situated support to be provided at relevant venues along the work process while still being flexible to the uncertainty in work needs and constraints present in ill-structured work.

As an illustrative example, consider a script that helps teams form and enact feasible work plans; see Figure 2. For all teams (1), this script encodes a work_situation_detector that checks if significantly more time was spent on the prior sprint than allocated, indicating that the time needed for the original plan was underestimated (2). When detected, the team’s mentor is informed of this situation at the start of their Planning Meeting, along with some strategies they can discuss with their team (3). When executed using data from the Studio API, this script triggers differently for the team of Jim and Bob, who have their planning meeting on Mondays (4a), versus for Kayla and Paul, who have their planning meeting on Thursday (4b). This shows how we can express general work strategies using the Organizational Objects (e.g., connect with your project mentor at the next planning session before the sprint begins) but enacted in specific situations for different people using the Studio API (e.g., Jim and Bob meet with Luna Monday morning).

3.4 Enacting Situated Strategies at Relevant Points of the Work Process

Figure 3:

View Figure

To execute Situated Scripts, we designed the Orchestration Execution Engine that allows us to instantiate situated work practices for an arbitrary number of people and teams in an organization at different times; see Figure 3. It does this by maintaining two separate threads that simultaneously monitor for work situations to arise and for opportunities to present situated strategies. Monitoring for work situations begins by computing the applicable_set from a situated script (1), and then checking the work_situation_detector for each of the targets of the applicable_set using data from the Studio API (2). If any evaluates to true, a new ActiveSituation document is created (3), and monitoring for the rest of the targets and scripts continues. Separately, monitoring for opportunities to present situated_strategies begins by checking for the encoded strategies for each ActiveSituation using data from the Studio API (4). If the opportunity for any of these has arrived (5), the engine sends the encoded strategy to the relevant social structure (6). By monitoring these scripts and their executions, we can let the strategies unfold and be enacted in different venues where they are appropriate, rather than immediately when a work situation is detected, across people in the organization.

3.5 Technical Implementation

Orchestration Scripts is implemented using Node.js, Express.js, and MongoDB. Situated Scripts are written by defining an object with an applicable_set, a work_situation_detector, and a list of situated_strategies using the Organizational Objects and Helper Functions. The Orchestration Engine stores these scripts and any ActiveSituations generated from the monitoring process in a MongoDB database. Every 15 minutes, it checks to see if new work situations have emerged or if it is time to present a situated strategy for an ActiveSituation. Evaluation and presentation of the strategies are done using the Studio API, which has access to data about the organization and ways to send messages through communication tools (e.g., Slack). Our current implementation of the Studio API integrates with APIs provided by Google Drive [3] and Slack [6]; other workplace tools and their APIs can be similarly integrated (e.g., Trello [9]; Github [2]). Strategies from triggered scripts are currently sent to people using Slack; in the discussion, we expand on alternative methods for surfacing situated strategies that future work may explore.

4.1 Research Setting

We situate our study within the Design, Technology, and Research (DTR)¹ academic research community at Northwestern University, which uses the Agile Research Studios (ARS) model to support students learning to independently lead research projects [75]. Work in DTR follows an Agile process [19, 63] where students plan research sprints that progress their research understanding each week. During this process, students are supported by venues where they can access peers and mentors from across the organization for help and coaching on their work needs (e.g., related to planning, feedback on research activities, prototyping or testing, and more), and work and collaboration tools that facilitate those interactions. Working effectively in this organization involves integrating these situated support opportunities into one’s working process to progress work. In these ways, the DTR community and the ARS model resemble the kinds of socio-technical work ecosystems we want to develop systems for since they offer multiple support opportunities along established work processes in an organization.

In what follows, we show Orchestration Scripts can encode a strategy for helping students form scoped research plans and enacting those places across DTR’s work ecosystem.

4.2 Encoding General Programs to Support Situated Work

Figure 4:

View Figure

Orchestration Scripts allow for encoding programs to support situated work in a more natural and less brittle manner than workflow tools due to the Organizational Object constructs our system provides. In Figure 4, we show a strategy where mentors want to know if any of their undergraduate students are overcommitted on their research plans, suggesting students strategies for re-scoping plans during the planning meeting if so. On the left, we show how Organizational Objects allow for a general representation of this strategy to be encoded while letting the author stay at a level of abstraction closer to how they think about situated work practices. In contrast, the right shows how individual workflows in a tool like Zapier must be authored for each team, along with hard-coded data about their tools, venues, and social relationships. Not only does this require a worker to focus on the details of the script implementation rather than the strategy (e.g, specific time for a venue versus saying the desired venue), any changes to these values (e.g., a planning venue gets moved) means editing all scripts that use those values, not just the one example shown here. Moreover, changes to the script, like adding new strategies or resources, means that all script instances would need to be revised. With our system, a worker only needs to update data in the Studio API, which automatically propagates to all active scripts since the Organizational Objects are evaluated using that API. Similarly, a revision to a situated script means that everyone the script is monitored for now receives the revised script. In this way, our use of abstractions of an organization’s ways of working allows Situated Scripts to be more expressive, maintainable, and extensible than existing workflow systems.

4.3 Tailoring General Strategies to Workers’ Specific Social Relationships and Venues

Figure 5:

View Figure

Even further, these constructs allow us to encode general strategies that apply to different people and teams, but become tailored based on their relationships with others and their specific collaboration venues. In Figure 5, we extend the example from the previous section to have conditional strategies that are delivered based on which team the script was triggered for. The mentors are provided with general strategies for all teams, but recommended supporting teams with only one person to a greater extent since they do not have a partner to re-plan with. Furthermore, when an individual has an onboarding mentor, this strategy is also suggested to the mentor since their student could get support on replanning from them. Despite being specific to each project team when enacted, the conditional strategies can be expressed in a general manner with ease using our system, with the engine taking responsibility for presenting the relevant information. In this way, Orchestration Scripts can allow for even richer, general models of working and strategies to be encoded that are applicable across the organization but may execute in very different ways for each team.

4.4 Flexibly Supporting Work Activities Across a Work Ecosystem

Figure 6:

View Figure

Finally, Orchestration Scripts allows us to create spaces to discuss emerging work situations and some relevant strategies to attempt, but leaves the decision of how to address issues to the workers. Even though the higher-level work process might be well-structured, the day-to-day work remains ill-structured, often causing unexpected needs to arise as progress is made. In Figure 6, we show how a student’s mentor was notified that they were overcommitted for the week since they were finishing up a prototype for an upcoming study. While the mentor and system suggested that the student come to Office Hours to pair program (based on their prior interaction and the student’s planned deliverables), the student raised a new issue about not being able to recruit users for their study that he and the mentor worked on instead. By leaving the locus of control on how to use support opportunities surface by the system to the people, Orchestration Scripts can flexibly provide situated support that fully automated approaches would prohibit.

5.1 Participants

We situated our study in the DTR academic research community detailed in Section 4.1, from which we recruited 17 people during their Spring 2022 academic term. 12 people were undergraduate or masters students, 4 were Ph.D. students, and 1 was a faculty member. Undergraduate and masters students received situated strategies to support them as students when work situations were detected, while Ph.D. students received these as well as ones to discuss with the students they mentored; the faculty member only received strategies for the students they mentored. For privacy, we gave all participants pseudonyms. Two of the paper’s authors are a faculty member and a Ph.D. student in the community.

5.2 Procedure

We deployed a variety of Situated Scripts to the DTR community modeled on existing effective work practices that students are encouraged to practice, such as forming feasible research plans and using support opportunities across the ecosystem to progress work; Table 1 includes summaries of these scripts. We instructed users to work as normal, interacting with the strategies surfaced by Orchestration Scripts over Slack when they thought it was helpful. Following the deployment, users participated in a 15-45 minutes long semi-structured interview, with the length depending on how many strategies were presented. During interviews, users were shown the strategies Orchestration Scripts presented and asked follow-up questions based on a semi-structured interview guide. For students, we asked if and how the presented strategies influenced their work practices, such as how they may have replanned, shared artifacts, or discussed strategies with their mentor. For mentors, we asked similar questions but focused on how the presented strategies influenced their mentoring with their students. Finally, we closed the interview with general questions about what users liked and disliked about the strategy prompts they had received.

Before the study, all users consented to participate and allowed their data to be used anonymously for research purposes. Before interviews, participants consented to audio, video, and screen recordings; audio recordings were transcribed for analysis. The lead author conducted all interviews over Zoom video calls.

5.3 Measures and Analysis

We used a thematic analysis approach [16] to analyze our interview transcripts. Our goal when coding the data was to identify how the Orchestration Scripts informed users about the work situations they were in and the strategies they could attempt. We did this by coding for moments where users said they became aware of these situations and how the surfaced strategies influenced their work practices. Once all data was coded, we reviewed the sub-themes generated from the analysis by identifying instances where our data supported or contradicted them. These sub-themes were then iteratively refined by splitting or combining them until they were distinct. We then reviewed all the themes and finalized the set we present below. The lead author conducted the coding of the interview transcripts and the generation and review of the initial themes; all authors collectively refined and finalized the themes.

5.4 Results

Orchestration Scripts presented 36 situated scripts to students and mentors during the study; see Table 1. Most strategies (22 out of 36) were delivered to project channels where students and mentors could see them; the rest were direct messages to mentors (12 out of 36) or to a channel for Ph.D. students and their faculty mentor (2 of 36). Strategies were delivered at venues for planning, getting help during office hours, and community-wide meetings, along with 1 presented mid-week. Most strategies were presented before the start of these venues so students could prepare for them–such as by sharing artifacts or progress–or to make mentors aware of potential strategies to discuss for a work situation.

In the rest of this section, we discuss how Orchestration Scripts supported situated work by: (1) monitoring for work situations that often get overlooked; (2) helping people strategize how to resolve work situations via suggested strategies; and (3) enacting check-ins between students and mentors across venues throughout the week.

5.4.1 Promoting Awareness of Emergent Work Situations.

Figure 7:

View Figure

Orchestration Scripts informed mentors about situations in which their students were working ineffectively and raised those issues for discussion at appropriate venues. In one example, mentors were informed that their student’s research plans were infeasible for the available time; see Figure 7. As Brady explained, it became an agenda item to talk with his students about: “[Going into planning meeting], I was almost ready as a talking point like, ‘hey it looks like you’re over points,’ and that was a nice thing to point to already have.” His student Melissa recalled how this led to them discussing strategies for rescoping their paper deliverable so that it and their user testing goals could both be accomplished: “we were planning out how we can get the end-of-term deliverable in faster, looking at different parts of the paper that we need to update [based on a prior draft] instead of writing a new thing.”

Before using our tool, mentors primarily monitored for ineffective practices–like being overcommitted on research plans or how widely students seek help from peers–by manually polling for any relevant data. In reality, mentors often forgot or chose not to check these things, focusing instead on issues they were already aware of and thought were important to discuss. However, when brought into focus by Orchestration Scripts, mentors, like Irene, made an effort to do so: “I would [talk about planning] with new-ish students, so I didn’t have the intention to do it with Amanda (her senior student), but when I saw that she was overcommitted, then I was like, ‘Okay, we should actually talk about this, and like how many hours it’s going to take to do each thing.”’ This discussion helped Amanda shift her goal from having a complete draft of findings and discussion for her latest study towards a more feasible outline that showed her new research understanding but involved substantially less work. In other words, these scripts provided an expanded awareness of work situations, particularly those that mentors may overlook due to heuristics they apply and that students may not bring up on their own (e.g., my experienced student knows how to plan within time constraints and will ask if they need help).

Orchestration Scripts also promoted timely discussion of “seasonal” situations that people have peripheral awareness of but only attend to when they get closer. For example, Lawrence had an upcoming Community-Wide presentation the following week, where he would get feedback from the entire community on his research. Lawrence shared that the prompt was, “a good reminder to get the ball rolling on planning his [Community-Wide presentation],” and to, “potentially bring it up to Clark (his mentor), whether it be in [the planning meeting] or offline.” Clark also found prompt useful since the presentation is, “one of the few times that [students] can get a ton of feedback very quickly.” Another example was for students preparing end-of-term deliverables during the final week of the academic term. Crystal found it helpful to be prompted the day before her last planning meeting of the term since she, “usually preps for [planning] meetings like the day before. So it was nice to have [the script] there to think about while I was planning what I was going to say and talk about.” She also liked seeing what the deliverables involved, saying: “[it was] nice to just make sure if I had any questions about anything, especially with things like the video or the write-up, that I could ask about that in [planning meeting] as well.” In this way, we see how Orchestration Scripts can have an awareness of situations–whether it be emergent ones based on people’s working patterns or upcoming events–bubble up for discussion, but only for the things that are currently relevant to discuss and at the times where discussion is helpful (i.e., at a venue).

Figure 8:

View Figure

Finally, Orchestration Scripts surfaced opportune moments at venues that could help progress one’s work; see Figure 8. While routine venues are generally known to people, having the system suggest them as support opportunities for this week’s work needs prompted helpful discussions that would have otherwise been missed. For example, Amanda shared how the strategy for an upcoming Office Hour helped her reflect on what to work on with her mentor: “I liked hearing more specifically about what Irene (her mentor) wanted me to practice in [office hours] before I went in, so I had more of a focus.” Similarly, Lawrence found it helpful to think about how he could work on his research with coaching from mentors and seek help from peers during a Community-Wide meeting: “I thought about what I wanted to do for [the peer help session]...I definitely tried to look at the [research activity scaffolds] spreadsheet [before the Community-Wide meeting], working out which one would fit and what I wanted to do.” This shows how people benefit from being aware of even routine venues so they can strategize how those venues can help progress their work.

5.4.2 Suggesting Relevant Strategies for Work Needs at Appropriate Times.

Figure 9:

View Figure

Orchestration Scripts surfaced situated strategies for detected work situations at opportune times to discuss them. Mentors found that the suggested strategies provided a useful scaffold when discussing the situation with their student. For example with earlier the sprint re-scoping example (see Figure 7), mentor Brady shared how he, “really liked [how the system showed] the different ways [his students] can re-scope their sprint because they were all good strategies [for re-scoping research plans].” As another example for deliverable planning (see Figure 9), Brady shared how the suggested strategies in the script became agenda items to discuss during the meeting, stating that “I’ve been a mentor for a while, but it is quite helpful to have the explicit intentions for [discussing end-of-term deliverables].” Robbie, a novice mentor, found the strategies targeted to the current situation, like the final planning meeting with students, particularly useful, stating: “If this advice had been in some generic document, it wouldn’t be as helpful as getting exactly the relevant thing right before that particular [planning] meeting.” Students similarly found the suggested strategies useful. Ellen shared that seeing the strategies helped her plan how she would accomplish her research work and deliverables for the end of the term: “I kind of structured my Planning Log for the last week around what I need to get done for the end-of-term checklist.” In this way, Orchestration Scripts supported mentors and students in recognizing what strategies they could discuss or attempt at the relevant times and places.

5.4.3 Orchestrating Interactions Throughout an Ecosystem.

Figure 10:

View Figure

Orchestration Scripts helped students adopt effective strategies to work towards their goals by providing a space for them and mentors to discuss work needs across ecosystem interactions during the week. For example, recall Amanda and her mentor Irene’s interaction before their Office Hours; see Figure 8. Irene shared how this interaction helped her push Amanda to practice independent thinking rather than leaning on Irene to provide direction: “I had a hunch that she wanted to check in because she was short on time, and this is the first time she can think about the structure [of the draft]. But rather than think about it with me, I’d rather she start thinking about it herself. So, I was glad we had that conversation because I think it reinforces this tone of trying [to get her] to do things independently.”

As another example, Orchestration Scripts enabled “course correcting” interactions between Steven, the faculty member, and his Ph.D. students when they felt stuck; see Figure 10. For example, Brady reflected on the feedback he got from his mid-week update, sharing how it helped him rethink the scope of his goals and progress his deliverables: “...it gave me a way to think about the user study [that I was unsure about]. I was confused about whether that was a case study versus a user study, and this showed me that I could have a way to write a study that’s pretty clear [regardless of the type].” From Steven’s perspective, this interaction was helpful because it created a check-in opportunity with Brady before the next planning meeting, allowing him to progress more effectively towards his goals: “[Brady] was able to share how they didn’t think there was a path forward and they were starting to like taking steps backward. And then I was basically like, couldn’t you just do this and this? So it was me kind of holding them to a path or showing them a little bit more of a path which they probably had, but in their fear they couldn’t see.” By facilitating check-ins across support opportunities in the ecosystem, practices discussed at earlier meetings can be re-emphasized at future working sessions people have, allowing for better practices to build up over time. In this way, we see how orchestration scripts can provide a sort of “continuity” of discussion that goes beyond the venues they originally happened in and potentially much further into the future.

6.1 Providing Programming Abstractions for an Organization’s Ways of Working

Programming abstractions for an organization’s ways of working allows us to tie effective practices to work situations along an established work process. Through our case study, we showed how Organizational Objects could model situated work practices that generalize across people and projects in an organization. Moreover, these general scripts can easily embed conditional strategies using our programming constructs, providing people and teams with tailored support at appropriate times. Not only are these scripts less brittle than existing workflow automation approaches to changes in people’s relationships with each other or when venues occur, but also more closely resemble how we think about and discuss situated work practices in the workplace.

While the relationships captured by Organizational Objects are already useful for modeling situated work practices, they could be extended with additional relational ideas on where and with whom strategies should be attempted. When work needs arise, multiple venues can sometimes be appropriate to resolve them, and the best strategy is to attend the next venue that can provide support. For example, a programmer needing debugging support for ReactJS could go to a team lead’s 1-1 venue or a pair programming session with a peer, whichever comes first and has expertise in ReactJS. Another extension could be adding constructs that sit at even higher levels of abstraction than what we provide. For example, instead of having to check data for how many people are on a project to determine if the project has teammates that can support each other (like in Figure 5), we could include a hasTeammates construct that is more natural and expressive for script writers to use to encode the same idea [49]. In this way, Organizational Objects can be expanded in the future to capture even more of the relational knowledge that workers already apply when working (e.g., attending the “next” relevant support opportunity; whether a person has a teammate or onboarding mentor; expertise of others [13, 45]), allowing for richer strategies based on these constructs to be encoded.

Organizational Objects are also helpful for work activities beyond supporting situated work. Because they capture people’s relationships with each other (e.g., a person’s mentor) and venues in an organization (e.g., their planning meeting), any tasks that are also relational in this way may benefit from using Organizational Objects. For example, this could be valuable for a human resources department doing quarterly performance reviews where each worker meets with their managers to discuss their growth and contributions; or to an administrator in an academic department for creating template emails for general tasks that would be routed to the appropriate people, such as confirming a student’s funding source with their advisor each term [39, 51]. In short, having our constructs in future software may allow for various systems that benefit from relational knowledge in the workplace to be created.

6.2 Authoring Situated Scripts to Support Situated Work Practices

Situated Scripts enacted by Orchestration Scripts helped people practice effective situated work practices. Users in our field study found that having these strategies delivered at venues along their work process helped them realize effective ways of working and coaching at opportune moments. Because these strategies were suggested to users (rather than an automatically enacted workflow), students and mentors could choose to use the strategies they felt were relevant. In short, our approach provides the necessary flexibility for ill-structured work, while still surfacing the necessary context (i.e., work situation and strategies) to workers at the opportune times to support situated work practices.

More broadly, the usefulness of Situated Scripts likely depends on the kinds of situations raised and peoples’ familiarity with the suggested strategies. While we would expect scripts that surface emergent situations that are easy to miss but straightforward to detect to be the most valuable (e.g., from our study, when students were overcommitted on research plans), scripts surfacing routine collaboration opportunities were well-liked for the discussion space they created. Similarly, while those less familiar with effective strategies benefited the most from our system’s prompts, more experienced mentors still found value in being reminded of strategies to discuss with their students. Put another way, supporting situated work ultimately means that everyone–mentors and students (or managers and employees)–know what work situations they are in and the strategies they can use. As long as Situated Scripts continue to provide this awareness, we argue that they will be helpful for many use cases and experience levels.

When encoding Situated Scripts, we expect most scripts to be written based on authors’ prior experience with situated work practices. While not a prerequisite, it is easier to encode programs for common challenges people encounter and the general approaches to resolve them (e.g., re-planning strategies when someone is overworking), or for activities along the existing work process that script authors are familiar with (e.g., planning weekly deliverables; onboarding newcomers; how to use collaboration opportunities). We can also consider a community-based approach to script authoring where people from across the organization can create customized forks of scripts others have written that can be applied to themselves or teams they manage [15, 40]. This would be possible and easy to do since the scripts are all written using the Organizational Object constructs, meaning that strategies can apply to people other than those whom the script author originally intended.

Related is how we can support the end-user authoring process of Situated Scripts. Our work focused on developing the necessary computational tools for supporting situated work, with the authors handling writing scripts based on practices used by the community. For systems to be usable by programmers and non-programmers alike, future work should consider developing authoring tools built on prior work for end-user programming tools for trigger-action programming [21, 22, 23, 24, 43, 71, 76]. Moreover, future work can consider studying how to support potential challenges in the authoring process, including: debugging work detectors triggering incorrectly, when it is unclear what strategies to suggest for a work situation, or when the situation authors want to model is several steps removed from the available programming constructs (e.g., when someone is struggling to make progress) [43]. Through this work, we can help people express the kinds of situated support they want a system like Orchestration Scripts to facilitate in their workplaces.

6.3 Towards Systems That Integrate With Our Ways of Working

Our work shows that supporting situated work requires our systems to be embedded with ecosystem-level intelligence about where and how people work. As workplaces grow in complexity, it is increasingly crucial for tools to support us in taking effective situated action [62], which will be challenging for tools to do if they do not understand our situated ways of working. Orchestration Scripts moves towards realizing this vision by explicitly modeling the ways of working that underpin a workplace and allowing people to write scripts that facilitate situated work using those constructs.

To fully realize this vision, future work will need to consider how organizations develop and maintain the Studio APIs and Organizational Objects that underpin systems like Orchestration Scripts. These components are necessary for any system that supports situated work since these constructs are what provide the necessary ecosystem-level intelligence to orchestrate situated work practices. As a first step, we can consider how Studio APIs–which are simplified knowledge bases of how people work in an organization–can be developed and maintained using inspiration from prior work that studied organizational knowledge bases in workplaces [12, 44].

With the newly available ecosystem-level intelligence our work provides, we can consider how to imbue our existing tools with this understanding. For example, we could extend Slack’s standard workflow and reminder system to use our constructs, allowing the team lead to ask things like, “remind my project team to discuss progress on their study design deliverable during Office Hours,” rather than specifically saying who and what time a reminder should be sent. This would allow for us to continue using the tools we find effective for our work needs, but add in an additional layer of intelligence for how we want them to support us in doing situated work.

Crucially, we can start moving workplace systems towards ones that support us in communicating and strategizing about work activities in a way that considers interactions across an ecosystem as part of their designs. For example, consider routine meetings between teams and their leads. Prior to these meetings teams may generate meeting agendas to surface work needs they want to address, and then generate follow-up tasks during the meeting based on their discussions [18, 57]. However, these tools lack the ability to pull in relevant context about how work is progressing across the broader work ecosystems (e.g., whether people are getting help from each other), relying solely on workers’ self-report for what to discuss. Moreover, the action items generated from the meetings cannot be contextualized to the venues or situations in which they should be enacted (e.g., introduce potential helpers to a person at the next inter-team meeting). To that end, we envision future ecosystem-wide meeting support tools built upon Orchestration Scripts that can provide awareness of tracked work situations and any relevant context, a collaborative space to discuss them along with any other work needs a team identifies, and encode situated follow-ups at other venues that Orchestration Scripts would enact. In short, for future workplace tools to effectively support situated work, we need them to understand and surface the relevant context for the situations that the tools are designed for and help us enact any follow-up actions throughout our work process.

6.4 Limitations

Our work has two limitations that future work should consider. First, our user study focused on understanding how Orchestration Scripts can support situated work practices by using a Slack bot interface to surface strategies since the studied community used Slack across venues. Given the study’s short length, we do not know how behaviors may change over an extended period or what potential issues are with this kind of intervention. For example, people may suffer from alert fatigue from being over-notified as time goes on. As a result, they may ignore helpful prompts about routine situations like collaboration venues or–more critically–ignore important prompts for emergent work situations that they are less aware of. As such, future work should conduct longer-term studies to understand how usage patterns with Orchestration Scripts change when using a bot-based interface [26, 29, 64].

Second, Orchestration Scripts was studied in the context of the ARS academic organization where goals and motivations for using the system may differ from a workplace. While ARS provides students and mentors with work processes and socio-technical support similar to those in a workplace, its primary goal is helping students learn effective work practices for conducting research. Learning effective work practices is also of emphasis in the workplace, but so is the increased importance of meeting work outcomes. While these motivations might differ, we expect that the kind of situated support our approach provides will still be valuable in the workplace. Given these differences, future work should consider studying our approach in workplaces to understand what aspects still work well and potential changes to make due to domain differences between workplaces and academic institutions.