International Journal of Science, Engineering and Technology Research (IJSETR), Volume 3, Issue 10, October 2014

3D Mouse Controlling with Wrist Watch
Jayaprakash Nagaruru
Abstract: Human Computer Interaction plays a key role in daily life like controlling Mouse operations, Keyboard operations and Gaming modes like Xbox. Human Computer interaction occurs in three different ways 1.Physical 2.Cognitive and 3. Affective. In our work, we are concentrated on Cognitive Mouse Pointer Controlling and Virtual Mouse Pointer Control Using RGB Technique .Cognitive Mouse Control is based on RF Communication with the help of MSP430 Transmitter inbuilt in watch and Watch as a Transmitter and USB RF CC1111 as a receiver. User can control the mouse movements and it operations by tilting the watch in 3-Axis/3-D and Fitt‟s law play‟s the key role in mouse controlling. In Cognitive Mouse control user controls the mouse action by establishing the RF Communication between user and system. It works in >1GHz frequency range and in virtual mouse is based on Image Subtraction Algorithm and user has to wear different colour sticker to control the operations. Keywords: Human Computer Interaction, Image Processing, RF Communication, Texas MSP430, Fitt‟s law.

V.Ravi Kumar

Manasa Nagaruru

Much of our technology for creating and displaying colour is based on empirical observation that a wide variety of colours can be obtained by mixing green, red and blue light in different proportions. For this reason red(R), green (G) and blue (B) are described as primary colours of the additive colour systems. In a colour image conforming o the RGB model, the value of f (x, y) is a vector with three components, corresponding to R, G and B. In a normalized model, these components each vary between 0.0 and 1.0 R, G and B can be regarded as orthogonal axes defining a three-dimensional colour space. Every possible value of f (x, y) is a point in this colour cube. Other colour models are CMY model, HSI model, CMYK model.

1. Introduction: The User interface is a component of a computer or its software which can be visualized, heard, touched, interacted with, run and understood by the common people or users of the computer. The two main elements of the interfaces are 1.Input and 2.Output. The Importance of the User Interfaces The user interface design belongs to the field called human computer interaction. Human Computer interaction is a technique that deals with learning, thinking and designing as how users and computer communicates with each other. It gives an idea about how person‟s requirements are fulfilled by computer in an efficient manner. The development of new communication and display techniques leads to package known as Graphical User Interface (GUI). The interaction design means designing interactive products to support people in their everyday and working lives. A good-design interface is very useful to its users to work easily. Image Processing Image processing is a general term for the wide range of techniques that exist for manipulating and modifying of images in various ways. The RGB model:

Figure 1: RGB color mixing Fitt’s Law: The cursor on the screen depends on the movement of the user‟s hand over a pointing device. Fitt‟s law is a predictive model of Human Computer Interaction which predicts the time required to rapidly move the target area and as is defined a function of the distance „D‟ to the moved to reach the target and the width „W‟ of the target. Fitt‟s law is used to model the act of pointing with a hand or finger using a mouse or any other pointing device. Fitt‟s law for a movement along a single-Dimension is given by the following formula. T=K1+K2(ID) ID=log2(2D/W) (1) (2)

ISSN: 2278 – 7798

All Rights Reserved © 2014 IJSETR

2733

International Journal of Science, Engineering and Technology Research (IJSETR), Volume 3, Issue 10, October 2014

Where, T= Average time taken to complete the movement K1,k2 = Constants K1= Start/stop point of the Device K2= Inherent speed of the device D= Distance from starting point to the centre of the target W= Width of the target measured along the axis of motion. Literature Survey Many methods have been developed by several researchers for controlling the mouse movement using a real time camera. Most of them are not robust enough for real time implementation and all of them use ambiguous methods for making a click event of a mouse [5]. Pandit et al. developed hardware related approach for hand gesture recognition. This requires the user to wear data gloves with markers from which hand posture could be extracted. An approach developed by Chu-Feng Lien [6] used finger tips for mouse movement and actions. Another approach from Erdem used finger tracking for mouse control and the click was performed when the hand passed over a specified region [7]. A simpler method was developed by Park. The action of clicking of mouse was done by keeping a track of the finger tips [5]. Paul et al, used still another method to click. They used the motion of the thumb (from a „thumbs-up‟ position to a fist) to mark a clicking event thumb. Movement of the hand while making a special hand sign moved the mouse pointer [5] [7] Outline: Section 1 consists of Introduction and description of Project. Section 2 Work‟s on implementation of virtual mouse and its Procedure. Section 3 Work‟s on 3D Mouse Controlling and its Procedure. Section 4 Work‟s on Results and Discussion, Advantages of both methods implemented. 2. IMPLEMENTATION OF VIRTUAL MOUSE IN MATLAB Implementation of Virtual Mouse is worked out with the help of Matlab and Colour Camera. The working technique in virtual mouse is RGB colour detection technique .Because RGB had sensing power where Blue colour is used for right click and left click, Green colour is used for Scrolling and Red colour is used to detect the mouse pointer and to control it.

Steps in our Approach: 1. Capturing real time video using colour camera. 2. Processing the individual image frame. 3. Flipping of each image Frame. 4. Conversion of each frame to a grey scale image. 5. Color detection and extraction of the different colours from flipped grey scale image. 6. Tracking the mouse pointer using the coordinates. Cursor control has been the easiest mouse function to be achieved through visual techniques. Using visual techniques, cursor control has been achieved by tracking different features and maps the changes in location to the cursor’s x, y coordinates on the screen. Argyros et al [2] proposed a vision-based remote, non-contact mouse control interface that employed web-cameras and use hand gestures for interaction. The tracking of hands was done by detecting the color blobs. In addition to hand gestures, finger detection was integrated. The twodimensional (2D) and three-dimensional (3D) trackers for hands and fingers provided the perceptual input for defining a set of gestures to control a computer’s mouse. Interestingly, in their project 2D hands gestures could involve users‟ both hands; one hand responsible for moving cursor (pointer hand) and the second hand assuming the role of a commanding hand.

Real-Time Video from colour camera

Converting Video into Images Extraction of different colours from detected image Performing different mouse operations assigned by user
Figure 2: Flow of working System.

ISSN: 2278 – 7798

All Rights Reserved © 2014 IJSETR

2734

International Journal of Science, Engineering and Technology Research (IJSETR), Volume 3, Issue 10, October 2014

Working: Working Procedure of Virtual Mouse is explained with the help of block diagram shown in figure 3.It Consists of Image Acquisition Device, Image Processor, Image Analysis Tool and Machine Control.
Image Acquis ition Device Image Process or Image Analysi s Tool Machi ne Contro l

The transmitter part is based on direct synthesis of the RF frequency. The frequency synthesizer includes a complete on-chip LC VCO and a 90◦ phase shift for generating the I and Q LO signals to the down-conversion mixers in receive mode.
Antenna

Display

Transmitter

Accelerator Sensor

Figure 4: Transmitter Block Diagram Receiver Section:

Figure 3: Block Diagram Image Acquisition Device: It consists of a Colour Camera connected to Computer. Processor: It consists Computer Image Analysis: It Consists of tools used to analyze the images captured from camera. Example: Matlab Machine Control: It is a device to which we are performing our work. Example: Mouse, Keyboard. 3. Implementation of Cognitive Mouse controller in Wrist Watch Implementation of Mouse controller in Wrist Watch module consists of Transmitter and Receiver section. Three-dimensional (3D) trackers/rotation of hand to control the mouse movements Transmitter section: In transmitter section it consists of Texas MSP430 Microcontroller. Where MSP stands for Multi-Signal Processing unit it mainly used for multifunctioning of controller. It mainly based on RF Communication to the fixed area with high security communication; we can call it as point-to-point communication which works up to 20 -30 feet distance depends on the locations. IAR is the software used to implement the Code.

Receiver section consists of USB/RS-232 RF receiver CC1111 which is connected to the computer. The receiver works