Create a Gesture Using Visapi on B&R HMI

Hi guys, welcome back to my blog. I want to write about how to create a simple trick on VC4 Visualization HMI on B&R Automation Studio using Visapi Library. You can use version 3.0 until the latest version of  Automation Studio, as long you using VC4 as the visualization. How if your HMI has a gesture like a smartphone nowadays have, will be cool right?

To create a gesture, it's mean that you will play with your touch activities on the HMI display, so that's why I'm using Visapi. Visapi allows us to control the visualization application, include touch activities on the display. Visapi offers a function called VA_GetTouchAction to retrieve the location of the current touch position on the display, as well as the touch status, is it pressed or released. But, in the gesture we need a direction, not only pressed or released status. So, using Visapi, I record the position of our touch and calculate it when I release my finger from the display to get the direction.

To using Visapi, you need to setup the Visapi function first and connect it to your VC4 Visualization using VA_Setup. To see that your setup is success, you need to call VA_Saccess function, make sure that VA_Saccess returns 0 before you continue to execute the next Visapi function.

VisapiHandle := VA_Setup(1,VC4_Visualization_Name);
VA_Saccess(1,VisapiHandle)=0

Visapi already offers a structure to monitor the touch action called TouchAction, it retrieves the position of our touch in 2 dimensions, X and Y (the value depends on your HMI resolution). Also our touch status, 1 for pressed and 2 for released. But you must note that Visapi handle needs to be release every time it called using another Visapi function called VA_Srelease. SO, after we get the touch action using VA_GetTouchAction, we need to release the VIsapi on the next line.

Structure of Gesture Monitor and Status

VA_GetTouchAction(1,VisapiHandle,0,ADR(TouchAct));
VA_Srelease(1,VisapiHandle);

Done, now touch information will be stored on the TouchAct variable. Next, we will calculate it to get the direction status. On the picture above you will see 4 status that retrieved the direction information on the Boolean data type that will be generated once your finger is released from the HMI display. Is it going up, down, left or right, you can also customize it to have upright direction or downleft direction just by calculate the range between start and end touch.

Now is the calculation, first I record the touch position on the first time my finger touch the display. So it's mean that the first time the status of the TouchAction is 1, meaning pressed, I will store the value of the touch position (X and Y) on the TouchAction on other variables called StartX and StartY.

Next, when the finger is released, I store the position of my touch in EndX and EndY. While your finger is touching the display, the value of your touch position will update in real time. Note that the value is beginning with 0 on the top left of your display and ended with the number of your resolution in down right of your display.

So, this is my code to get the direction, I compare each axis (X and Y), which one is greater. From that, I will know which direction is the touch is going. I compare X separately with Y, so I may have Up status going true and right status going true and I can call it Up Right.

IF (TouchAct.status=1) AND EdgeDetection=0 THEN
RangeX := 0;
RangeY := 0;
StartX := TouchAct.x;
StartY := TouchAct.y;
ELSIF NOT(TouchAct.status=1) AND EdgeDetection THEN
EndX := TouchAct.x;
EndY := TouchAct.y;

IF StartY
RangeY := EndY-StartY;
SlideDown := 1;
END_IF;
IF StartY>EndY THEN
RangeY := StartY-EndY;
SlideUp := 1;
END_IF;
IF StartX
RangeX := EndX-StartX;
SlideRight := 1;
END_IF;
IF StartX>EndX THEN
RangeX := StartX-EndX;
SlideLeft := 1;
END_IF;
END_IF;

EdgeDetection := (TouchAct.status=1);

You may have a better solution, write it in the comment.