Creating a Haptic 3D Model of Wenceslas Hill in Olomouc

Abstract

Interactivity in today’s society finds its way into many facets of life and can be used in various ways, including 3D printing. For example, various 3D models can be incorporated into museum exhibitions and serve as interactive media for visitors, deepening their experience. One of the advantages of haptic 3D models is the immediate haptic feedback. Such models can have various uses, from being a part of an interactive exhibition to providing assistance to people with visual impairment. This article describes the process of creating a haptic 3D model depicting Wenceslas Hill in Olomouc in the eighteenth century. The model has several surface elements printed from conductive material that react to touch. The interactive model itself is unchanged from its original modelled 3D version, meaning the shape of the object stays the exact same throughout modifications. The resulting model conveys additional information about the object or its parts by means of a web interface via a connected tablet device. To implement the desired functionality, TouchIt3D technology was used. This technology uses a combination of conductive and non-conductive materials for 3D printing. The conductive material serves to propagate an electrical signal caused by touching a chosen part of the model. A 3D printer with two extruders was used for printing the model, allowing simultaneous printing of two different materials. The model’s scalability is advantageous for potential use by people with visual impairment. The model shall serve as a tool for enriching historical knowledge about the object by using interactivity.

1. Introduction

Interactivity can be perceived as a process of executing an action based on a previous action. In today’s world, it is used in many ways as people expect a specific form of interactivity in everyday life. However, physical interaction with displayed historical objects is forbidden in most conventional museums. This restriction is imposed because various objects of natural or cultural character have to be preserved for future generations [1].

There are many approaches to presenting models in museums, especially in the context of education. One approach is tailoring the visitor’s experience to provide choice. Each visitor should be free to choose which objects they want to interact with. This choice can then support their curiosity [2], and most modern science museums that present natural phenomena try to present them in an interactive way. This approach encourages the visitor to explore and experiment [3]. Interactivity can be achieved by 3D printing to create an interactive 3D model. By using 3D printing, users can use haptic feedback to interact with objects. The physical objects can be recreated in a 3D environment and manufactured using 3D printing. Various items in cultural heritage are 3D printed to make fairly precise replicas of existing originals [4]. Physical interaction with the objects is then possible. Combining interactivity and 3D printing is one of the ways of creating a historical object containing parts that react to a user’s touch and presenting additional data about the object or its parts, bringing forward information that is not usually obtained in museum exhibitions.

Another way to convey information interactively is by using an application with 3D touch controls implemented via a multi-touch table while focusing on UX (user experience) aspects and guided inquiry [5]. Nakamura and Yamamoto [6] have also researched haptic feedback implemented via a multi-touch display. Additional related studies were presented by Ciolfi and Bannon [7], Gammon [8] and Csikszentmihalyi and Hermanson [9]. 3D printing can be used in various subject areas, with one of the more prominent being cultural heritage (CH). In recent years, the accuracy of 3D printing has increased considerably so it is feasible to present more visually pleasing and accurate results in the field of CH [10]. There are many ways of preserving historical artefacts for future generations, with 3D printing being one of them, and especially significant for creating tangible models [11]. Digitalisation and subsequent 3D printing of objects can be used to reconstruct and preserve objects of historical importance throughout various disciplines. Further studies on this topic include Laycock et al. [12], Balletti and Ballarin [13], and Al-Baghdadi [14]. Virtual Reality (VR) technology is also a significant contributor to CH, as it can enhance interactivity with printed 3D models [15] or provide 3D visualisation of objects generated via photogrammetry [16].

A multitude of ways to provide interactivity using 3D printing have been employed. For example, a 3D model can be combined with electronic parts such as a position sensor and computer for visualisation [17]. Sarik et al. [18] have also studied a similar topic. Another solution for providing interactivity is subtracting material from 3D models so that they can, thereafter, be filled with pipes distributing various substances. Depending on the substance used, there are different interactive effects, such as reacting to proximity or touch [19]. It is also possible to use 3D printing from conductive materials to create tangible models for information sharing and information selection using, respectively, BYOI (Bring Your Own Information) and BYOT (Bring Your Own Tool) principles [20]. Notably, 3D printing can also serve to transfer user input from a model to a touch device, using conductive PLA [21]. Other interactive 3D methods include using a web-based environment making use of 3D scanning data [22] or virtual reality using CAD (computer aided design) data [23]. Other ways of combining interactivity and 3D models have been researched by Willis et al. [24], Li et al. [25] and Liu et al. [26].

3D printing can also be used to assist people with visual impairment. Examples include using 3D tangible maps made from conductive material, connected to a mobile device to help with route planning when going shopping etc. [27]. The use of conductive material became a topic of interest for other researchers, such as Taylor et al. [28], Schmitz et al. [29] and Götzelmann [30]. Götzelmann and Althaus [31] provide an approach for enabling touch sensitive surfaces on existing 3D objects using a set of conductive parts. Three-dimensional models can also be used to help with education of the visually impaired, allowing the relay of information about abstract concepts, which are otherwise not easily conveyable [32]. Additional studies on the subject include Thevin and Brock [33] or Shi et al. [34]. Another interesting approach to creating interactive 3D models for people with visual impairment is to use a 2.5D display capable of adjusting a 3D model in real time via programmable scripts, and thus providing instant haptic feedback [35]. Similar programmable 3D models have also been a research interest for Leo et al. [36]. Other notable examples of providing haptic feedback include a so-called haptic edge display that provides tactile interactions for mobile devices by the use of tactile pixels, which are implemented using piezoelectric actuators [37].

The 3D model presented in this paper provides audio-visual feedback based on haptic interaction. It is also capable of providing information about the historical state of the object of interest by means of observation, as it captures the area of Wenceslas Hill in the eighteenth century. Combining this with the aforementioned haptic feedback can point out the differences between seeing and touching an object. The model is easily replicable by any printer with multi-material capabilities or dual extrusion, as it does not rely on any additional electronics, except for the tablet computer. When factoring in filament pricing and the amount of filament spent, printing out the model costs around 30 USD. The shape of the model does not deviate from the original and, as such, is a representation of the historical object, which is valuable for cultural heritage and object preservation. The model and its functionality show the possibilities of TouchIt3D technology and could serve as a template for similar solutions, as it is a unique approach to creating haptic 3D models.

This paper is structured as follows: Section 2 presents the methods used to create the haptic 3D model, including the TouchIt3D technology, workflow to implement the technology, designs for connecting multi-part 3D models and web interface design. Section 3 describes the achieved results, notably the haptic 3D model of Wenceslas Hill and the web interface that serves to present additional information. Section 4 describes various issues of the resulting 3D model and compares it and its functionality to similar studies. Section 5 then concludes the paper.

2. Materials and Methods

This article aims to describe the process of creating an interactive haptic 3D model of Wenceslas Hill in Olomouc, which depicts its historical state and is accompanied by a tablet computer that serves to present additional information. To fulfil the said goal, TouchIt3D technology was used [38]. The 3D model contains several surfaces that react to touch, providing interactivity. For a user-friendly way of displaying desired information about the object or its parts, the tablet computer serves as a part of the resulting model. In addition, several solutions for conducting an electrical signal in the case of the models, which are composed of multiple parts, were designed. The created haptic 3D model holds an ambition to serve as a tool for enriching knowledge about the given area.

Wenceslas Hill is located in the city of Olomouc, Czech Republic. On the top of Wenceslas Hill, Olomouc Castle is situated. There are many significant places of interest in the area of the castle, with the most distinctive one being Saint Wenceslas Cathedral [39]. Other places of interest include a partly preserved Romanesque palace, and the Chapel of St. Anne and Chapel of St. Barbara. The area has seen a multitude of architectural makeovers throughout history, especially the cathedral. The model described in this article depicts the area of Wenceslas Hill in the eighteenth century. The current appearance of some of the buildings in the area is shown in Figure 1.

The basis for creating the printed interactive model was a 3D model of Wenceslas Hill in Olomouc, depicting the area in the eighteenth century. The model was originally designed as part of a time series of 3D models presenting Wenceslas Hill in several historical periods. For the process of 3D modelling, SketchUp (sketchup.com) ver. 18 from Trimble was used. The model contains 180,108 polygons. Most of the polygons (121,078) form a generated digital terrain model (DTM), which serves as the model base. Without the DTM, the rest of the model has 59,102 polygons. The model was modelled in LOD 3 [40]. It was created by referencing available historical material from the target time period (e.g., sketches, historical drawings) for the Olomouc Museum of Arts. The eighteenth century model was chosen mainly due to a suitable level of detail for modifying and subsequently printing the model.

TouchIt3D technology served to design the conductive parts of the model and create designs for conducting electrical signals for 3D models composed of multiple parts. The idea behind TouchIt3D is to provide interactivity by combining a device with a touchscreen with a model printed on a 3D printer. The technology uses a combination of conductive and non-conductive materials for 3D printing. The material used for printing was polylactic acid (PLA), both for the conductive and non-conductive parts. The basic principle of TouchIt3D technology is the use of any 3D model with parts modelled for the purpose of being printed using conductive material. The conductive part, containing carbon, reacts to touch and can then conduct an electrical signal. To provide interactivity, the technology is accompanied by a touchscreen device with interactive virtual buttons. Upon touching a chosen conductive part, additional information about the conductive part is shown on the display of the device. The technology was implemented using the process of 3D modelling that served to design and place parts of the model, which are to be conductive in the resulting printed model. Figure 2 shows the steps needed to implement the TouchIt3D technology. When implementing the technology, a workflow was followed to ensure proper functionality. The workflow is presented in detail in Section 2.3. The model was printed on Craftbot FLOW IDEX XL. This 3D printer has two extruders and, therefore, provides an option of simultaneously printing two materials (conductive and non-conductive filament). Proto-pasta PLA was chosen for the conductive parts and Polymaker PolyMax PLA for the non-conductive parts.

SketchUp was used for creating the conductive parts of the model and modifying the base part of the model, enabling accommodation of the tablet computer. It was also used for minor modifications of both the conductive and non-conductive parts, and modelling the designs for conducting electrical signals in multi-part 3D models. Each presented design consists of two standalone cubes of fixed dimensions. One of the cubes contains a conductive surface and a tube, also conductive, which serves to connect the parts. The second cube contains the rest of the conductive tube and an extended surface designed to be in direct contact with a touchscreen display. The designs aimed to find a suitable way of connecting two standalone parts that would allow for an electrical signal to be conducted. Another important part of creating the haptic model was subtracting the conductive and non-conductive parts. For this purpose, Netfabb Premium ver. 2021.1 was used. The aforementioned software was utilised to execute Boolean subtraction of the model parts. Netfabb was also used to repair the 3D models’ geometry errors, making them manifold and correcting inverted or intersecting faces, using mainly automatic repairs. Some repairs, however, needed to be done manually, which was achieved in SketchUp.

CMS (content management system) WordPress was used for creating a web interface that would display additional information. The interface needed to be very specific. Therefore, a custom WordPress theme was created using the PHP programming language, HTML5 markup language and CSS3. Before printing the model, a so-called G-code containing a series of instructions for the 3D printer had to be generated from the 3D model. For this purpose, Ultimaker Cura ver. 4.8.0 was used. Before generating the G-code, layer height and infill density settings were adjusted. Layer height was set to 0.1 mm and infill density to 15% for both extruders. Afterwards, the model was sliced, generating the G-code.

The functionality of the 3D model and web interface has been tested and validated by the authors. The principles, functionality and detailed setup of the TouchIt3D technology have been tested in various application scenarios as the technology plays a significant role in past and currently ongoing projects at Palacký University Olomouc, mostly focused on working with people with visual impairment and partially sighted people [41].

2.1. Conducting Electrical Signal for Multi-Part 3D Models

In the process of creating the haptic 3D model, several designs for conducting electrical signal connections in the case of multi-part 3D models were made. Five designs were created in total, each composed of conductive and non-conductive parts, created using Boolean subtraction. Each design was modelled, printed and tested. The testing showed that only two out of five designs provided the expected functionality. One of the models was printed out in a non-intended way, effectively making it functional despite the imprecise printing process. The mentioned model was originally designed to contain an element of cylindrical shape that would serve to connect two standalone cubes. However, in the final print, the cylindrical element was entirely absent. The model, therefore, only contained two flat conductive surfaces that were in contact with each other, providing the desired effect. The only other valid design was one with a convex and a concave side that precisely joined to each other to conduct an electrical signal. The other designs failed in the ability to conduct an electrical signal, which was mainly caused by imprecisions in the 3D printing process. The resulting designs are shown in Figure 3.

2.2. Designing the Web Interface

The web interface, extending the model functionality by displaying additional information, was designed in such a way that it could be used and modified even by people without any knowledge of either HTML or CSS. To achieve this, CMS WordPress was used, as it allows for modifying the content, from rewriting text to adding an image or a video, all without the need to edit the code itself, making it an effective tool for content management. Since the web interface needed to be very specific, it was decided to create a custom WordPress theme. The interface consists of an intro page with a list of the selectable conductive parts and quick guide on interacting with the model. Since the model has eight conductive elements in total, the interface navigation menu has eight links, each providing additional information about the chosen part of the model in the form of written text and images. It is possible to modify the content shown based on preferences or feedback from users.

2.3. Workflow for Creating a Haptic 3D Model

The presented workflow could, with minor modifications, be used for essentially any 3D model. The workflow results in a 3D model containing surfaces printed from conductive material. Such a model can be combined with a device equipped with a capacitive display, and therefore react to electrical impulses caused by touch. The key step of the workflow is designing and modelling parts that are conductive while following several requirements for the final product to function properly. In particular, it is crucial for the surfaces in contact with a display to have a large enough size. If the size is too small, the propagation of the electrical signal is not sufficient and functionality is limited. Another requirement is sufficient distance between conductive elements, assuring they do not cross each other inside the model. The next step of the workflow is doing Boolean subtraction of conductive and non-conductive parts. The subtraction takes two 3D models as input—the original model and the separate conductive part, which was previously designed and modelled. After executing the subtraction, a model with gaps representing the non-conductive part is created. Before executing the subtraction, it may optionally be necessary to repair small geometry errors of the 3D models. The amount of errors depends on the quality of the original 3D models. If the repairs are not made, it could result in the subtraction not executing properly. This operation can be performed either manually or by using automatic repair tools, e.g., those implemented in Netfabb and 3D Builder software. When having two standalone models for the conductive and non-conductive parts, different printing parameters can be set for each part, using a so-called slicer (e.g., Ultimaker Cura, PrusaSlicer). In the slicer, a preferred printing material can be set for each part, after which the parts should be joined together again, forming one object. The next step is the 3D printing itself. Before proceeding with the printing process, it is possible to set various properties of the printing process, such as layer height, or enabling the so-called Wipe Tower, which is used when printing from two different filaments. The tower serves to clean the nozzles of the extruders when changing filament. Even if a 3D printer capable of dual extrusion is used, it is still recommended to use the Wipe Tower to prevent the materials from blending. After preferred printing properties have been set, the model can be sliced, generating G-code, and finally printed (Figure 4).

3. Results

3.1. Haptic 3D Model of Wenceslas Hill

The resulting haptic 3D model of Wenceslas Hill is based on a 3D model capturing the state of the object in the eighteenth century (Figure 5). The haptic model is composed of three parts: a conductive part, non-conductive part and model base. The conductive part contains eight conductive elements in total, printed using conductive PLA Proto-pasta filament to provide interactivity. Each conductive element consists of one or more conductive surfaces, a 3 mm diameter conductive tube and a block with a rectangular surface that is designed to be in contact with a touchscreen device (Figure 6). The conductive elements are designed not to cross each other and also to allow sufficient distance for the electrical signal to be insulated.

The non-conductive part was obtained as a result of Boolean subtraction between the original 3D model and created conductive part. The subtraction resulted in a model with gaps located in areas where the original model and conductive part would intersect. In other words, after putting the conductive and non-conductive parts together, the missing material is filled out with conductive material. After combining the two parts and finally printing them, the main segment of the model was established, as shown in Figure 7.

Furthermore, the model base was designed to create enough room for the computer tablet to fit in and prevent any sort of manipulation with the device, which could result in the model not functioning properly. When printing the model, the main segment was printed first, followed by the base part. Neodymium magnets, one in each corner of the model base, were used to join the whole setup together (Figure 8), enabling the desired functionality.

3.2. Web Interface

The implemented web interface serves to display additional information about the model. It contains eight virtual buttons in the website header, each one linked to one or more conductive surfaces. Touching a conductive surface provides the same functionality as interacting with a touchscreen. Upon touching a conductive surface, an electrical signal is sent and the corresponding link is pressed. The web is then redirected to a page containing additional details about the chosen part. In addition, the interface contains a main page with basic information as well as a page describing how to interact with the models and also listing the available interactive parts of the model. The instructions are available from the main page via a separate virtual button available on the upper right of the web. Users can also return to the main page at any time via the same button (when not already on the main page). Figure 9 shows the web interface without the tablet attached, so the virtual buttons in the header can be seen. As the web is made in WordPress, the content can be easily extended and customised.

4. Discussion

The resulting 3D haptic model was created in the scale of 1:1000, which was influenced by several factors. The dimensions of the print bed of the Craftbot FLOW IDEX XL are 425 × 250 × 500 mm, which resulted in the need to limit the size of the printed model so it would fit on the bed. It would potentially be possible to print a larger model composed of multiple parts which could then be joined together. However, this would lead to an increase of total print time. Even in the scale of 1:1000, the total print time was roughly four days. If the model were enlarged by a mere 10%, the time would significantly increase. The dimensions of the model were also influenced by the number of interactive elements. Theoretically speaking, the model could contain more than eight elements, but to provide enough insulation distance between each element, their number was limited. The size of the tablet computer was also taken into account when deciding the scale of the final model.

The resulting haptic 3D model has several issues, unfortunately. The conductive Protopasta PLA, used to print the conductive part, suffered from an unwanted trait of leaving marks on the non-conductive white part. Since black and white are contrast colours, the mixing of the materials was visually obvious, fortunately without any functionality limitations. If the non-conductive part were printed using a different colour filament, these visual issues could have been alleviated. Another issue of the model is its technical shortcomings. The tablet computer needs to be placed in a very specific way without being manipulated later. The conductive elements also have to be in direct contact with the display of the tablet. If the model is then moved in any way, the intended functionality could be prevented. These shortcomings can be solved by a more precise modelling process and by pressure using foam material.

On the other hand, the presented solution has several advantages when compared with similar products, such as the model of Kroměříž Castle, created at the Department of Geoinformatics, Palacký University several years ago [42] (p. 110). The model also uses a tablet computer to present additional information upon touching a conductive part. However, about half of the display was covered by the model limiting the display potential, which is not the case for the haptic model of Wenceslas Hill. Here, only a small part of the display is covered by the model, providing more space for text, images and other forms of media. If some modifications were made, e.g., making the model larger or providing a different solution for connecting the tablet to the model, it would be possible to use a larger device to display the information. Similar to [20], the contents of the web could even be transferred to a much larger screen (e.g., monitor, TV) to display more information. Currently, the web is set specifically for the resolution of the device used. However, as the web uses CSS, multiple breakpoints could be set to control the layout of the content on larger displays. The styling and overall presentation of the web could be improved as well.

Contrary to the solutions presented in [17,31,37], the haptic 3D model of Wenceslas Hill does not rely on internal electronic components for its functionality and as such is easier to produce. In other words, no additional parts are needed for the intended functionality, as an electrical signal transfers through the conductive filament without any additional components. However, a device is then needed to interpret the signal, thus the presence of the tablet computer, which is in direct contact with the conductive parts. The need for a 3D printer with multi-material capabilities or dual extrusion could be seen as a drawback. On the other hand, the model is otherwise relatively cheap to produce. As the model itself does not rely on any additional components, it is easily replicable and its principles applicable to any other 3D model.

While the use of conductive filament lends itself quite well to the purposes of making 3D models for people with visual impairment, as seen in [27,28,30], the model presented in this paper serves primarily for sighted people and as such, it focuses on providing information via the web interface. However, it is scalable, so when modified, it could be suitable for people with visual impairment. The model even incorporates choice and experimentation as described in [2,3], as users are free to choose the part of the model they want to interact with and, in turn, which content they want to display. The content of the web can be changed if the need to present different information arises. However, the number of conductive surfaces and, therefore, the number of interactive links in the header of the web stays the same.

5. Conclusions

The result of the presented article is the haptic model of Wenceslas Hill in Olomouc, capturing the object’s appearance in the eighteenth century. Using TouchIt3D technology, the model was made interactive, providing additional information to users. Each interactive element was represented by a corresponding feature in the developed web interface. The information is presented upon touching a chosen conductive surface. The content of the web interface changes according to the surface selected by a user, making it possible for the user to choose only the desired parts, obtaining only the desired information. The additional information is shown on the majority of the display surface.

It is also possible to perceive the historical look of the object from just looking at the 3D model itself. However, the experience is enhanced by conveying additional information that can be tailored to provide relevant historical facts or historical context. It is not difficult to imagine there could be multiple models, each showing the object in a different time, together serving as a showcase of the historical development of a building or any other historical site. The usage of 3D printing provides multiple advantages, mainly the ability to touch objects without the fear of damaging them. 3D printing is also a cheap solution for small-series production. Therefore, in the case of a model being damaged, a spare one can easily be printed, making it much less expensive than using castings or other fabrication methods. Conversely, the disadvantage of 3D printing is the time necessary to print out the final product. Various 3D models, like the one presented, can be used in museum exhibitions, not limited to showing a historical state of an object. Similar principles can be applied for a wide range of geovisualisations, even ones of a technical nature, explaining their functionality and other aspects.

The haptic model can also be used to provide people with visual impairment with valuable information, lowering their information deficit. For this purpose, the model would have to be modified to suit the needs of this target user group, which could, for example, be achieved by adding the text-to-speech function. The basic functionality would be preserved though.