Assignment 5 Report

Introduction/Overview

Since Assignment 4, the main change is that we replaced the 2D image display panel where the instructor gives hand signs to instruct where to place the object with a 3D model that moves and shows the same hand signs. In addition, the main interactions such as the left and right crane joysticks to operate the crane have been improved. Since most parts of the application were finalized in the 4th assignment, our team just made a few minor changes by cleaning the scripts and fixing bugs to run a smooth VR Crane Simulation.

“The new era of VR Crane Training. With no risk, consequences or experience. Learn with an immersive experience that replicates a real Tower Crane operator in a virtual construction environment.”

Technical Development

In this application, users can use two VR controllers to control the crane, with the left VR controller for rotating the crane and moving the trolley, and the right VR controller for lowering and raising the crane hook, along with two buttons for hooking and unhooking the load onto the cranes hook.  In addition,  there are also some objects which could be grabbed with the crane to test the control of the crane. For the guide to use this application, a gif of various hand signals is also created to introduce how to operate this application. Besides that, a system for a crane operator tutor to activate various dangerous scenarios will also be added, which would allow the tutor to start scenarios such as wind, which the user would then have to react to. We have also added vibration feedback during the operation process of users.  

We put all functions that we have below. 

Scripts:

HandSignalDirector: Used to move the hand signaller on the ground around to different positions and to change the hand signal being displayed depending on where the crane's hook is or the hooked object.

Hook: Used to detect when objects are close enough to be hooked and when the user presses the hook button, also attaches the hook to the objects using a fixed joint.

HookableObject: Used on objects that are hookable, detects whether that object is hooked or not and passes information to the HandSignalDirector script.

HookableObjectLocation: Used for the position that the hooked objects should be taken, detects whether an object is correctly placed in the location and passes information to the HandSignalDirector script.

JoystickController: Used to take the positions of the grabbable joysticks and compare them to the default position to work out how a joystick is being moved by the user grabbing them, and then passing that information to each of the scripts that control the crane's movement.

LowerCrane: Extends or retracts the 2 spring joints connected to the hook to lower or raise it, using information passed from the JoystickController.

MoveCrane: Moves the crane's trolley backwards or forwards along the crane arm, based on information passed from the JoystickController script.

RotateTower: Rotates the upper section of the crane, based on information passed from the JoystickController script.

VRButton: Detects collisions with a VR hand controller's index finger, and invokes a unity event action when it is pressed, used for the buttons that hook or unhook objects with the crane's hook.

3D Content

Crane

The crane is what the user controls, sitting within the crane's cabin. It is made up of the crane tower, crane arm, trolley, cabin, and hook. The crane arm can move to the left and right, then the hook can be raised or lowered. Users can operate the crane to move objects shown in the scenario everywhere (no excess boundary) they want to enjoy the training process. 

Construction Site

The construction site is a place where the user can drop items onto using the crane, which looks like a building that is being constructed. Users can follow the guide to move the objects to this construction site.

Fences

The fences surround the construction zone and make up part of the virtual environment. This is also to reach a sense of reality in a virtual environment.

Buildings

The buildings surround the outside of the construction zone and make up part of the virtual environment and block the users view of the infinite skybox void.  It is like a boundary or a limitation for users during their operation. Users can only work in the boundary.

Pipes

The pipes are one of the objects the user can pick up with the crane. Pipes are one of the objects which could be moved by users.

Girder

The girder is one of the objects the user can pick up with the crane. Pipes and girders both have a grabble area, users can move the hook to the area and grab them.

Construction Worker

The construction worker model displays the hand signals that the user needs to follow to move the crane into the correct position for hooking or unhooking objects

He has different animations for each direction the user might need to move the crane.

Usability Testing

Application prototype testing is an essential part of application development (Kuhnel et al. 2018). Usability testing refers to letting a group of representative users perform typical operations on the product while observers and developers observe and record. (Sasmito & Nishom  2019). Also, the real-time interview and voice recording will be used as auxiliary methods for testing.

Design and Plan

The target of this application:

This application aims to train employees to operate the crane and follow the instructions in a virtual environment without any risks and consequences. While increasing efficiency and reducing costs, we believe this application can provide a great training experience to beginners or even experienced employees. 

To realize the target, we are looking for some participants to test the application, and give them a questionnaire to get some feedback to improve the application. 

The main target of this application is to train the operation of cranes, so the types of participants are divided into two parts. One is the pure starters who do not have any experience in crane construction, the other is the experienced crane constructors. We hope to get feedback from both inexperienced and experienced users to improve our application in different directions. 

Recruitment of testers

The testers were invited from variable platforms including online platforms and offline channels. The online platforms for this testing are Facebook and Wechat. Offline channels include UTAS staff, students, and family members. We have 5 testers in total, and three of them are inexperienced in crane construction, the other two testers are experienced. 

Protocol of the testing

To ensure the fluency and integrity of the test, we have designed a complete testing process including pre-test, during the test, and post-test.

The following shows what our questionnaire and record table look like.

Questionnaire: 

The questionnaire was divided into two parts, one is the ranking questions,  another one is short answer questions.

As shown in the above ranking questionnaire, we prepared several questions for testers to rate, some of the questions are related to the feeling in the testing process, and some other questions are related to our main interaction functions. The first part of the questions can help us to check our scenario and overall application, and the second part is to check our interaction functions.  

We set two short answer questions, one is to find out which operation is not clear, another one is to get recommendations from the testers.

Record table: 

The record table is to record the time consumption and the comments from testers. The comments shows the combinations of what the testers said during the testing and the answer in the short answer questions of the questionnaire.

Report of the findings

The testing results are shown in the following table, the overall rating below is the average rating of the questionnaire, and the comments are the main conclusion of the  feedback from these five testers including positive feedback and negative feedback.

Average in time used and rating:

Time used: 11.2 mins

rating: 4.56

Analyses of the findings

Based on the findings above, some points are concluded for things we can improve:

1. Make a guide of controllers.
2. Increase the colliders to grab the pipes.
3. To add some feedback when the hook is connected or disconnected from the objects, like vibration or reminder light.
4. Give some functions for the camera.
5. Adding some more functions to get close to the real crane.
6. Adjust the position of the instruction table.
7. The sight of view is limited.

Addressing the Results of the Usability Testing 

Detail a plan to address the user feedback

According to the analyses of the findings, we hope to make some changes in the following part.

1. Clarify the guide table and animated person.

1. Expand the sight of view. 
2. Make feedback when the hook collides with objects
3. Add more functions to make the virtual crane get close to the real one.

Changes made to the interface application

The crane cabin's model has been modified with larger side windows to allow for greater visibility around the crane without needing to rotate it. This was done to increase the sight of view, as that was found to be a problem from testing.

The 3D model of the construction worker has been added into the cabin with each hand signal animation being played and with a guide for the user to learn what each signal means. This was done as the old image of the hand signals did not correctly represent the different hand signals that the  3D model uses, and accurate hand signals would help to achieve the goal of the application which is to train people in these hand signals.

The mass and drag of the hookable objects have been increased to give them more realistic physics when they are picked up by the crane. This was done to increase the realism of the application, which is important because we want to train crane operators how to use real cranes, and the training wouldn't  be effective if the application didn't accurately simulate the real world.

References

Kuhnel, M, Seiler, L, Honal, A & Ifenthaler, D 2018, ‘Mobile learning analytics in higher education: usability testing and evaluation of an app prototype’, Interactive Technology and Smart Education, vol. 15, no. 4, pp. 332–347

Sasmito, G.W. and Nishom, M., 2019, December. Usability testing based on system usability scale and net promoter score. In 2019 International Seminar on Research of Information Technology and Intelligent Systems (ISRITI) (pp. 540-545). IEEE.