Mouse and keyboard automation using Python?

In this section, we’ll try to automate the movements of mouse and keyboard using pyautogui module in python.

Pyautogui is a library that allows you to control the mouse and keyboard to do various things.

It  is a cross-platform GUI automation Python module for human beings.

As it is a third party library, we need to install it.

pip install pyautogui

Mouse

Below is program to automate the movement of your mouse. On running your programme, you can see the mouse movement with every command. I run below command on CLI so as to capture the mouse movement. You can try with others different values too.

Example

>>> import pyautogui
>>> pyautogui.size()
Size(width=1366, height=768)
>>> width, height = pyautogui.size()
>>> width
1366
>>> height
768
>>> pyautogui.position()
Point(x=750, y=293)
>>> pyautogui.position()
Point(x=750, y=293)
>>> pyautogui.position()
Point(x=750, y=293)
>>> pyautogui.moveTo(10, 10)
>>> pyautogui.move(10, 10, duration=1.5)
>>> pyautogui.move(10, 10, duration=1.5)
>>>
>>> pyautogui.move(10, 10, duration=1.5)
>>> pyautogui.move(10, 10, duration=1.5)
>>> pyautogui.move(10, 10, duration=3.0)
>>> pyautogui.moveRel(20, 0)
>>> pyautogui.moveRel(250,0)
>>> pyautogui.moveRel(250, 0, duration=2.5)
>>> pyautogui.moveRel(0, -100)
>>> pyautogui.moveRel(0, -150, duration=1.5)
>>> pyautogui.position()
Point(x=210, y=526)
>>> pyautogui.click(339, 38)
>>> pyautogui.click()

On running above program, you may have notice, when we set the duration, mouse movement will continue for that period. Press escape to get the control back for the last command- pyautogui.click()

Keyboard Control Functions

The typewrite() function

The primary keyboard function is typewrite(). This function will type the characters in the string is passed. To add a delay interval in between pressing each character key, pass an int or float for the interval keyword argument.

>>> pyautogui.typewrite('Hello, TutorialsPoint!') #Prints out 'Hello, Tutorialspoint!' instantly
>>> Hello, TutorialsPoint!
>>> pyautogui.typewrite('Hello, TutorialsPoint!', interval=0.25) #Prints out 'Hello, Tutorialspoint!' with a quarter second delay after each character
>>> Hello, TutorialsPoint!

The press(), keyDown() and keyUp() functions

To press these keys, call the press() function and pass it a string from the pyautogui.KEYBOARD_KEYS for e.g.: enter, esc, f1.

>>> pyautogui.press('enter') #press the Enter key
>>>
>>> pyautogui.press('f1') #press the F1 key
>>> p
>>> pyautogui.press('left') #press the left arrow key

To press multiple keys similar to what typewrite() does, pass a list of strings to press(), like−

>>> pyautogui.press(['left','left', 'left'])

The hotkey() function

To make pressing hotkeys or keyboard shortcuts convenient, the hotkey() can be passed several key string which will be pressed down in order, and then released in reverse order.

>>> pyautogui.hotkey('ctrl', 'shift', 'esc')

Running above opens ‘Task Manager’ window in my machine (windows).

Above is same as running below program−

>>> pyautogui.keyDown('ctrl')
>>> pyautogui.keyDown('shift')
>>> pyautogui.keyDown('esc')
>>> pyautogui.keyUp('esc')
>>> pyautogui.keyUp('shift')
>>> pyautogui.keyUp('ctrl')

Below is program to demonstrate couple of pyautogui keyboard capabilities−

>>> import pyautogui
>>> pyautogui.click(90, 90)
>>> pyautogui.click(90, 90);pyautogui.typewrite('Hello, World!')
>>> pyautogui.click(150, 150)
>>> pyautogui.click(150, 150);pyautogui.typewrite('Hello, World!')
>>> pyautogui.click(150, 150);pyautogui.typewrite('Hello, World!', interval=0.2)
>>> pyautogui.click(150, 150);pyautogui.typewrite(['A', 'B', 'left', 'left', 'x', 'y'])
>>> pyautogui.click(150, 150);pyautogui.typewrite(['A', 'B', 'left', 'left', 'x', 'y'], interval=1)
>>> pyautogui.KEYBOARD_KEYS
['\t', '\n', '\r', ' ', '!', '"', '#', '$', '%', '&', "'", '(', ')', '*', '+', ',', '-', '.', '/', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', ':', ';', '<', '=', '>', '?', '@', '[', '\', ']', '^', '_', '`', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z', '{', '|', '}', '~', 'accept', 'add', 'alt', 'altleft', 'altright', 'apps', 'backspace', 'browserback', 'browserfavorites', 'browserforward', 'browserhome', 'browserrefresh', 'browsersearch', 'browserstop', 'capslock', 'clear', 'convert', 'ctrl', 'ctrlleft', 'ctrlright', 'decimal', 'del', 'delete', 'divide', 'down', 'end', 'enter', 'esc', 'escape', 'execute', 'f1', 'f10', 'f11', 'f12', 'f13', 'f14', 'f15', 'f16', 'f17', 'f18', 'f19', 'f2', 'f20', 'f21', 'f22', 'f23', 'f24', 'f3', 'f4', 'f5', 'f6', 'f7', 'f8', 'f9', 'final', 'fn', 'hanguel', 'hangul', 'hanja', 'help', 'home', 'insert', 'junja', 'kana', 'kanji', 'launchapp1', 'launchapp2', 'launchmail', 'launchmediaselect', 'left', 'modechange', 'multiply', 'nexttrack', 'nonconvert', 'num0', 'num1', 'num2', 'num3', 'num4', 'num5', 'num6', 'num7', 'num8', 'num9', 'numlock', 'pagedown', 'pageup', 'pause', 'pgdn', 'pgup', 'playpause', 'prevtrack', 'print', 'printscreen', 'prntscrn', 'prtsc', 'prtscr', 'return', 'right', 'scrolllock', 'select', 'separator', 'shift', 'shiftleft', 'shiftright', 'sleep', 'space', 'stop', 'subtract', 'tab', 'up', 'volumedown', 'volumemute', 'volumeup', 'win', 'winleft', 'winright', 'yen', 'command', 'option', 'optionleft', 'optionright']
>>>
>>> pyautogui.click(100,100);
>>> pyautogui.click(100,100); pyautogui.typewrite('Hello world!')
>>> Hello world!
>>> pyautogui.click(100,100); pyautogui.typewrite('Hello world!', interval=0.5)
>>> Hello world!
>>> pyautogui.click(100,100); pyautogui.typewrite(['a', 'b', 'left)', 'left', 'X', 'Y'])
>>> aXYb
>>> pyautogui.click(100,100); pyautogui.typewrite(['a', 'b', 'left', 'left', 'X', 'Y'])
>>> XYab
>>> pyautogui.click(100,100); pyautogui.typewrite(['a', 'b', 'left', 'left', 'X', 'Y'], interval=1)
>>> XYab
>>> pyautogui.press('f1')