Talking to
Industrial robotS
Talking to
Industrial robotS
Why a Talking Industrial robot?
Industrial robots are used in multiple areas, such as automotive, constructions, manufacturing, mining, etc. In all these applications the human-robot interaction plays a crucial role during task programming, operation and validation.
We collected and analyzed data concerning a robotic application meant for welding jack-up rigs on shipyards. We found that programming and interacting with such robots requires highly skilled trained human operators. However, welders usually don’t have such knowledge, nor can they be extensively trained.
Therefore, in this project we aim at understanding the welding context and designing a friendly and efficient multimodal interface for operating the robot welder.
FIeld Study
To better understand our design context, we carried out an ethnographic study at the shipyard where the welding robot will be deployed.
A semi-structured interview was prepared in advanced by our team. All members prepared questions for the welders. Each question was written on a separate note. The notes were grouped in 10 topics with subtopics. Finally, 14 individual questions (see Fig.1) emerged for the interview. This technique is very helpful in getting a precise interview focus reflecting the perspectives of the entire team. Additional questions to demographics were also added.
A total of 1.5 hour of raw video material and 262 photos were collected. Observations were written down and questions were clarified on the spot. 6 interviews were recorded and transcribed.
Observations
The shipyard is spread over a 68 hectares area operating four graving docks. Within the main area several large halls are located were welders work. Metal plates, pipes and other welding materials are deposited near the entrance or along the hall’s sides. To transport them inside the hall, load rail flat wagons and cranes are used. The floor is covered with transversal flat metal bars (see fig. 2a).
Due to ongoing working processes the environment is dusty, hot and loud. However, the open entrance sides of the halls ensure moderate ventilation and natural light. On the top, several neon lamps provide additional light during the day and at night.
Metal surfaces are first cut using a special cutting machine and then transported to the welding location. Before (and after) welding, the surface is cleaned with brooms to remove metal parts and dust (see fig. 2b).
Most of the welding jobs are done manually. Usually, welders work in groups of 3-4 people. Each group is accompanied by a senior supervisor who assigns tasks and ensure the welding is performed up to the standards. There are also a few automatic welding jobs performed with a machine (see fig. 2c). In average, the automatic welding machine is three times faster than a human welder. However, the machine can be used only for horizontal welding processes and only when the welding path is straight. For any path deviations, edges, minor repairs or critical spits manual welding is required (see fig. 2d).
The pipe welding for jack-up rigs is also done only manually. The complicated structure of the rig and the circular form of the pipes make difficult to automatize the process: the place is narrow and the pipe requires the usage of a welding arm with 7 degrees of freedom for a full pass.
To access the rig structure, welders use scaffolds or elevators. It takes about 2 weeks to align a complete structure. First, the pipes need to be profile cut. Then, they are brought by crane to the joint location. Once the alignment is proof checked, the welders attach metal holders to keep the pipes together and start filling the gaps.
In general, welding seem to be easy to learn, but hard to master: a welder with 5 years of experience is considered still a beginner. Welding is also an intense physical activity: welders are often standing, bending over, getting on their knees, stretching their arms to reach a remote welding spot or squatting under the pipe structure or climbing on the jack-up rig.
INTERVIEWS
The vast majority of welders on the shipyard are foreigners. They are normally recruited in their native country. Some have welding experience while others are complete beginners. On the shipyard, they receive training courses and become certified welders.
We interviewed 5 welders and one engineer in charge of welding supervision and training. The engineer was local Singaporean while the welders were from Bangladesh (4) and India (1). The welders spoke Bengali, and respectively Telugu and were aged between19 to 45. Their level of English was relatively low often requiring translation help from senior fellows. One welder was a beginner (1.5 years welding experience) while others were more advanced welders: 3 of them had 6-7 years welding experience, while the 4th was a senior welder who had worked as welder for more than 15 years. All, except one got their working experience on the shipyard.
Welders usually work 8-11 hours daily. When welding manually, the productivity of an intermediary/advance welder is around 50 meters per day as compared with 40-45 meters for a beginner. If using an automatic machine 100 meters of welding can be achieved in one day.
A usual day on the shipyard involves three main activities: preparation (cleaning, preheating material and adjusting parameters), welding and coordinating tasks with co-workers.
To determine whether they are doing the “right thing” welders often rely on supervisor’s advices and on their own “feelings” as one welder said: “You can feel when you do it right”.
design recommendations
In table 1, we summarizes how the information obtained from our observation and interviews was further translated into design recommendations.
User interface
Following our design recommendations, we defined the following functionalities:
- Save History: the system will store internally the history of past configurations. As such, previously validated settings can be re-used increasing interaction efficiency and easiness.
- Embedded camera: a camera streaming will enable users to inspect the welding spot.
- Multimodal interaction: apart from a graphical interface for typing or choosing menu parameters the interaction will allow speech input for a hands-free interaction. In this way, the users could inspect the welding area while talking with the robot. The speech modality implies a close-talk microphone and robustness to noise.
- Dialogue capabilities: the robot should understand questions in context beyond a simple Q&A session.
Our first interface sketch is shown in figure 4. The sketch presents the transition steps from the moment the user opens the application up until he finishes setting up the parameters. The interface has a menu bar on the top, a camera stream on the central left, a Q&A window on the central right displaying the dialogue history. At the bottom, the interface shows the tasks to be completed with the help of the interface: preparation, monitor and inspection.
Based on our sketch we further developed and implemented a high-fidelity prototype. The The UI follows the welding steps observed on the shipyard (see figure 2).
- Preparation (renamed ‘Setup’): in this tab, welders can load pre-defined parameter values for a particular welding typ. Users can adjust parameter values within a certain range
- Monitor: this section enables users to supervise the welding process; the process is automatic and once started cannot be modified; in case of failure, the process can be re-started from the beginning.
- Inspect: enables users to choose an inspection method and check the weld quality.
Fig 5 series: RoMI UI final design
Video 1: RoMi explainer video of our first prototype
