Skip to main content

Command System

The command system is at the heart of Whisper2Linux, allowing users to interact with their Linux desktop using voice commands. This document provides a comprehensive overview of the command system, including built-in commands, how to use them, and how to extend the system with custom commands.

Command Structure​

Whisper2Linux commands generally follow this structure:

[Trigger Word]: [Command] [Arguments]
  • Trigger Word: By default, this is "Olga," but it can be customized.
  • Command: The specific action or function you want to execute.
  • Arguments: Additional information needed for the command (optional).

Built-in Commands​

Whisper2Linux comes with a variety of built-in commands to handle common tasks:

Text Manipulation Commands​

  1. Copy:

    • Usage: "Olga: Copy [text]"
    • Function: Copies the specified text to the in-memory clipboard.

  2. Paste:

    • Usage: "Olga: Paste"
    • Function: Pastes the content of the in-memory clipboard at the current cursor position.

  3. Select All:

    • Usage: "Olga: Select all"
    • Function: Simulates pressing Ctrl+A to select all text in the current application.

  1. Enter:

    • Usage: "Olga: Press Enter"
    • Function: Simulates pressing the Enter key.

  2. Space:

    • Usage: "Olga: Space"
    • Function: Simulates pressing the Space key.

  3. Tab:

    • Usage: "Olga: Tab"
    • Function: Simulates pressing the Tab key.

  4. Arrow Keys:

    • Usage: "Olga: Left", "Olga: Right", "Olga: Up", "Olga: Down"
    • Function: Simulates pressing the corresponding arrow key.

  5. Home:

    • Usage: "Olga: Home"
    • Function: Simulates pressing the Home key.

  6. End:

    • Usage: "Olga: End"
    • Function: Simulates pressing the End key.

  7. Page Up/Down:

    • Usage: "Olga: Page Up", "Olga: Page Down"
    • Function: Simulates pressing the Page Up or Page Down key.

Editing Commands​

  1. Backspace:

    • Usage: "Olga: Backspace"
    • Function: Simulates pressing the Backspace key.

  2. Delete:

    • Usage: "Olga: Delete"
    • Function: Simulates pressing the Delete key.

Function Key Commands​

  1. F1 to F12:
    • Usage: "Olga: F1", "Olga: F2", ..., "Olga: F12"
    • Function: Simulates pressing the corresponding function key.

Special Commands​

  1. Respond:

    • Usage: "[Question or statement] Olga respond"
    • Function: Triggers an AI response to the given input.

  2. Search:

    • Usage: "Olga: Search for [query]"
    • Function: Performs a web search for the specified query and provides a summary.

  3. Escape:

    • Usage: "Olga: Escape"
    • Function: Simulates pressing the Escape key.

Command Processing​

Whisper2Linux uses a sophisticated command processing system:

  1. Transcription: The voice input is transcribed using the Whisper API.

  2. Trigger Word Detection: The system looks for the trigger word (default "Olga") in the transcription.

  3. Command Extraction: The text following the trigger word is analyzed to identify the command.

  4. Fuzzy Matching: Commands are matched using fuzzy string matching, allowing for natural variations in speech.

  5. Execution: Once a command is identified, the corresponding function is executed.

Fuzzy Matching System​

The fuzzy matching system allows for flexibility in command recognition:

  • It uses the rapidfuzz library to calculate similarity scores between spoken commands and known commands.
  • A threshold is applied to determine if a spoken phrase matches a known command.
  • The system considers factors like word order and additional words, making it more robust to natural speech patterns.

Extending the Command System​

You can extend Whisper2Linux's capabilities by adding custom commands:

  1. Open the whisper2linux.py file.

  2. Locate the commands dictionary.

  3. Add a new entry to the dictionary:

    "your_command_name": your_command_function

  4. Define your command function:

    def your_command_function():
        # Your command logic here
        pass

  5. Ensure your command function is properly integrated with the existing system, considering aspects like error handling and logging.

Example: Adding a Custom Command​

Let's add a command to open the terminal:

def cmd_open_terminal():
    subprocess.Popen(['gnome-terminal'])  # Adjust for your desktop environment

commands.update({
    "open terminal": cmd_open_terminal
})

Now you can say "Olga: Open terminal" to launch a new terminal window.

Best Practices for Custom Commands​

  1. Naming: Choose clear, concise names for your commands.
  2. Function Design: Keep command functions focused on a single task.
  3. Error Handling: Implement proper error handling to manage potential issues.
  4. Logging: Add appropriate logging statements for debugging and monitoring.
  5. Performance: Be mindful of the command's execution time to maintain responsiveness.

Command Priorities​

When multiple commands match a spoken phrase:

  1. Exact matches are prioritized.
  2. Longer commands are given slightly higher priority to differentiate similar commands.
  3. Commands that match the beginning of the spoken phrase are favored.

Limitations and Considerations​

  • Some commands may not work in all applications or desktop environments.
  • Complex commands or those requiring additional context may not always be interpreted correctly.
  • Be cautious when adding commands that could potentially disrupt system operations.

Conclusion​

The Whisper2Linux command system provides a flexible and powerful way to control your Linux desktop using voice commands. By understanding its structure and capabilities, you can effectively use existing commands and extend the system to suit your specific needs. Remember to test thoroughly when adding custom commands to ensure they integrate smoothly with the existing functionality.