Mouse and keyboard automation using Python?

Python automation of mouse and keyboard controls allows you to create scripts that can perform repetitive tasks, control applications, and simulate user interactions. The PyAutoGUI module provides a simple way to automate GUI interactions programmatically.

Installing PyAutoGUI

PyAutoGUI is a cross-platform GUI automation library that needs to be installed separately ?

pip install pyautogui

Mouse Automation

PyAutoGUI provides several functions to control mouse movement and clicking. Let's explore the basic mouse operations ?

Getting Screen Information

import pyautogui

# Get screen dimensions
screen_size = pyautogui.size()
print(f"Screen size: {screen_size}")

width, height = pyautogui.size()
print(f"Width: {width}, Height: {height}")

# Get current mouse position
position = pyautogui.position()
print(f"Current position: {position}")

Screen size: Size(width=1366, height=768)
Width: 1366, Height: 768
Current position: Point(x=750, y=293)

Mouse Movement

import pyautogui
import time

# Move to absolute position
pyautogui.moveTo(100, 100)
time.sleep(1)

# Move relative to current position
pyautogui.moveRel(50, 0, duration=1.5)  # Move right 50 pixels
pyautogui.moveRel(0, 50, duration=1.5)  # Move down 50 pixels

# Click at specific coordinates
pyautogui.click(200, 200)

Mouse Click Operations

import pyautogui

# Different click types
pyautogui.click()                    # Single left click at current position
pyautogui.click(100, 100)           # Click at coordinates (100, 100)
pyautogui.doubleClick(150, 150)     # Double click
pyautogui.rightClick(200, 200)      # Right click
pyautogui.drag(100, 100, 200, 200)  # Drag from (100,100) to (200,200)

Keyboard Automation

PyAutoGUI offers comprehensive keyboard control through various functions for typing text and pressing keys ?

Text Typing with typewrite()

import pyautogui

# Type text instantly
pyautogui.typewrite('Hello, TutorialsPoint!')

# Type with delay between characters
pyautogui.typewrite('Hello, World!', interval=0.25)

Key Pressing Operations

import pyautogui

# Press individual keys
pyautogui.press('enter')
pyautogui.press('f1')
pyautogui.press('left')

# Press multiple keys in sequence
pyautogui.press(['left', 'left', 'left'])

# Type using key names and characters
pyautogui.typewrite(['A', 'B', 'left', 'left', 'X', 'Y'])
# Result: XYAB (cursor moves left after typing A,B then types X,Y)

Hotkey Combinations

Use hotkey() to press key combinations like Ctrl+C or Alt+Tab ?

import pyautogui

# Press Ctrl+Shift+Esc (opens Task Manager on Windows)
pyautogui.hotkey('ctrl', 'shift', 'esc')

# Copy and paste operations
pyautogui.hotkey('ctrl', 'c')  # Copy
pyautogui.hotkey('ctrl', 'v')  # Paste

# This is equivalent to:
pyautogui.keyDown('ctrl')
pyautogui.keyDown('c')
pyautogui.keyUp('c')
pyautogui.keyUp('ctrl')

Available Key Names

import pyautogui

# View all available key names
print("Available keys:", len(pyautogui.KEYBOARD_KEYS))
print("Some examples:", pyautogui.KEYBOARD_KEYS[:20])

Available keys: 255
Some examples: ['\t', '\n', '\r', ' ', '!', '"', '#', '$', '%', '&', "'", '(', ')', '*', '+', ',', '-', '.', '/', '0']

Practical Example

Here's a complete example combining mouse and keyboard automation ?

import pyautogui
import time

def automate_text_input():
    # Click at a text field location
    pyautogui.click(300, 300)
    time.sleep(0.5)
    
    # Type some text with interval
    pyautogui.typewrite('Python Automation', interval=0.1)
    
    # Press Enter
    pyautogui.press('enter')
    
    # Select all text using Ctrl+A
    pyautogui.hotkey('ctrl', 'a')
    time.sleep(0.5)
    
    # Copy the text
    pyautogui.hotkey('ctrl', 'c')
    
    print("Automation completed!")

# Uncomment to run (be careful with real automation)
# automate_text_input()

Safety Considerations

When using PyAutoGUI for automation, consider these safety measures ?

import pyautogui

# Enable fail-safe (move mouse to top-left corner to stop)
pyautogui.FAILSAFE = True

# Add pauses between actions
pyautogui.PAUSE = 1

# Get screen size before positioning
width, height = pyautogui.size()
# Only click within screen bounds
if 0 < 500 < width and 0 < 300 < height:
    pyautogui.click(500, 300)

Conclusion

PyAutoGUI provides powerful mouse and keyboard automation capabilities for Python applications. Use moveTo() and click() for mouse control, typewrite() and hotkey() for keyboard input. Always implement safety measures like fail-safe and pauses when creating automation scripts.