User Create-able Plugins update (v2.7)

M.I.L.E.S Version 2.7: Introducing Plugin Creation

M.I.L.E.S introduces a transformative feature in Version 2.7: the ability for users and developers to create and integrate custom plugins. This significant update opens up endless possibilities for customization and functionality.

Plugin Creation System

At the heart of Version 2.7 is the brand new plugin creation capability. Key features of this update include:

• Open-ended Plugin Development: For the first time, M.I.L.E.S allows the integration of custom plugins. Developers can create functionalities using any Python library, helper functions, and can design plugins to perform virtually any task, no matter the complexity.

• Plugin.py Format Requirement: To ensure seamless integration into M.I.L.E.S, all plugins must adhere to a specific format detailed in their docstrings. This requirement is outlined more in the wiki for plugin creation.

• Comprehensive Flexibility: The introduction of plugin creation brings unlimited flexibility to M.I.L.E.S. Whether it's extending the system's capabilities with new features or integrating with external APIs and services, if you can code it, or if ChatGPT can code it, M.I.L.E.S can run it.

• Enhanced Documentation and Support for Developers: With this update, developers receive extensive documentation and support tools, including guidelines on the plugin.py format, to aid in creating compatible and functional plugins.

Getting Started with Your First Plugin

Interested in building your own plugin? Check out our updated plugin wiki guide for all the information you need to get started with your first plugin creation.

How to Write Plugins for MILES

💿 Install M.I.L.E.S:

Install for Windows
Install for macOS

⚠️ Known Issues

• Graphical Display Glitches: Persisting from earlier versions, the graphical display glitches are under continuous investigation.
• Speech Output for Complex Inputs: Despite improvements, speech output quality for complex inputs may still experience issues.

Full Changelog: Visit our GitHub

Smart Home Support (v2.6)

M.I.L.E.S Version 2.6

M.I.L.E.S takes a leap into smart home integration with Version 2.6, building upon the foundational enhancements introduced in Version 2.5. This update brings smart home capabilities, improved responsiveness, and refined interaction mechanisms to the forefront, solidifying M.I.L.E.S's position as an advanced personal assistant system.

Smart Home Integration

With Version 2.6, M.I.L.E.S introduces support for smart home devices, marking the beginning of a seamless integration with your home environment. Key features include:

• Home Assistant Compatibility: Connect M.I.L.E.S to your Home Assistant setup effortlessly. Simply enter your long-lived ID and the Home Assistant URL during the setup process, and M.I.L.E.S will discover available lights and switches.
• Device Control: Handpick which devices you'd like M.I.L.E.S to manage. Initial support covers the basics—turning lights and switches on and off. We're committed to expanding these capabilities to include more devices and controls.

This is the screen that shows after you agree to setting up Home Assistant:

From here, you can check off which lights or switches you want Miles to control, once that's done, click Confirm and Save.

Performance Enhancements

Refined Model Responses

Efficiency and accuracy are paramount. In Version 2.6, we've optimized the response mechanism of M.I.L.E.S to deliver faster and more accurate answers to your queries.

Improved Follow-up Questions

To streamline conversations, M.I.L.E.S now handles follow-up questions without requiring the wake word to be repeated. This makes interactions feel more natural and fluid.

First-Time Welcome Message

New users will be greeted with a warm welcome message, making your first interaction with M.I.L.E.S more engaging and informative.

Overhauled Configuration Script

The setup process is now smoother than ever, thanks to a comprehensive overhaul of the configuration script. Setting up M.I.L.E.S is straightforward, ensuring you can get started without a hitch.

Advanced Recognition and Understanding

M.I.L.E.S now boasts an enhanced understanding of its surroundings and the applications it's interacting with, alongside slight improvements to its image recognition capabilities. These upgrades contribute to a more intuitive and versatile user experience.

Bug Fixes and Improvements

• Fixed a bug preventing M.I.L.E.S from correctly interpreting tool parameters.
• Addressed issues with LaTeX formatting, ensuring mathematical expressions are displayed correctly.
• Resolved problems with persisting conversation history across sessions, enhancing the continuity of interactions.

💿 Install M.I.L.E.S:

install for Windows
Install for macOS

⚠️ Known Issues

• Graphical Display Glitches: Persisting from earlier versions, the graphical display glitches are under continuous investigation.
• Speech Output for Complex Inputs: While improvements have been made, speech output quality for complex inputs may still experience issues.

Full Changelog: v2.5...v2.6

Image Recognition Update (v2.5)

M.I.L.E.S Version 2.5

Building on the progress made in Version 2.4, M.I.L.E.S evolves further with Version 2.5, introducing groundbreaking advancements in personalized interaction and system responsiveness. This version marks a significant enhancement in M.I.L.E.S's capabilities, particularly in the realm of image recognition and wake word detection, ensuring an unparalleled user experience.

Enhanced Wake Word Detection

In Version 2.5, we've significantly improved wake word detection accuracy and responsiveness. Our custom wake word model, initially introduced in Version 2.4, has been fine-tuned to virtually eliminate false activations—a common issue in previous versions. The sensitivity thresholds are now set to either 0.1 or 0.01 by default, making M.I.L.E.S extremely responsive to the "Miles" wake word while minimizing false positives for unmatched user interaction.

New Features

Image Recognition with GPT-4-Vision

Expanding upon our efforts to create a more intuitive and interactive experience, Version 2.5 introduces an innovative image recognition feature powered by GPT-4-Vision. This allows M.I.L.E.S to provide highly detailed descriptions of visuals captured through your webcam, enhancing how you interact with the world around you. Key highlights include:

• User-Driven Activation: M.I.L.E.S will only activate the webcam and perform image recognition when explicitly requested, adhering to our strict privacy guidelines. Your permission is paramount, and M.I.L.E.S is programmed never to use the webcam without it.
• Dual Analysis Modes: Catering to diverse needs, M.I.L.E.S offers two modes of image analysis:
  • Normal Mode: Provides concise, clear responses to your queries, perfect for quick insights.
  • Extreme Mode: Delivers exhaustive details about every aspect of the image, leaving no stone unturned for those who seek thorough information.

M.I.L.E.S Version 2.5 is a testament to our commitment to delivering advanced, user-centric features while maintaining the highest standards of privacy and interaction quality. We're excited for you to experience the enhanced capabilities and look forward to your feedback.

💿 Install M.I.L.E.S:

install for Windows
Install for macOS

⚠️ Known Issues

• Graphical Display Glitches: The graphical display issues from version 2.2.1 persist. The root cause is still under investigation.
• Speech Output for Complex Inputs: There remains an issue with speech output quality for complex mathematical inputs.
• Delay in speech recognition: After saying "Miles" if you speak too quickly after the beep, Miles will not hear the first word you say, please just wait a little before talking, about 0.5 seconds after the beep should be fine.

Full Changelog: v2.3...v2.4

Replaced picovoice for custom wake word (v2.4)

M.I.L.E.S Version 2.4

With Version 2.4, M.I.L.E.S takes a significant leap forward in personalized interaction and efficiency. In response to changes in the landscape of voice recognition services, we've transitioned away from Picovoice wake word detection. This version introduces a custom-trained wake word model, enhancing the reliability and responsiveness of M.I.L.E.S to your commands. Here's what's new:

New Features

Custom Wake Word Model
Enhanced Wake Word Detection: Moving away from Picovoice due to their recent policy changes on account management, we now utilize a custom wake word model trained with over 50,000 samples and refined through 10,000 training steps on an Nvidia a100. This shift not only increases the system's sensitivity to the "Miles" wake word but also reduces false positives, ensuring a more seamless interaction. And no, it doesn't need a supercomputer to run, it can run on basically anything. (I love the people at @openWakeWord)

Internet Browsing

Browse the Web: Continuing from version 2.3, M.I.L.E.S supports internet browsing, allowing direct web content access through the system.

• You can now search through main.py, and edit # webbrowser.open(url) and remove the # making it webbrowser.open(url). This will allow Miles to open every web page he views in real time on your desktop. To turn it off, just add the # back. It's off by default.

💿 Install M.I.L.E.S:

install for Windows
Install for macOS

⚠️ Known Issues

• Graphical Display Glitches: The graphical display issues from version 2.2.1 persist. The root cause is still under investigation.
• Speech Output for Complex Inputs: There remains an issue with speech output quality for complex mathematical inputs.
• Delay in speech recognition: After saying "Miles" if you speak too quickly after the beep, Miles will not hear the first word you say, please just wait a little before talking, about 0.5 seconds after the beep should be fine.

Full Changelog: v2.3...v2.4

Added Internet browsing for real time info (v2.3)

M.I.L.E.S Version 2.3

Building upon the improvements in user interaction and flexibility introduced in Version 2.2.2, M.I.L.E.S Version 2.3 further enhances the system with internet browsing capabilities and advanced input device handling. Here's what's been added:

New Features

Internet Browsing

• Browse the Web: M.I.L.E.S now supports internet browsing, you can now ask for web content directly through the Miles.

Advanced Device Input Logging

• Device Input Logging: For scenarios where M.I.L.E.S does not respond to the default audio input source, the system now logs device inputs, they will show up in the terminal after every command.

💿 Install M.I.L.E.S:

install for Windows
Install for macOS

⚠️ Known Issues

• Graphical Display Glitches: The graphical display issues from version 2.2.1 persist. The root cause is still under investigation.
• Speech Output for Complex Inputs: There remains an issue with speech output quality for complex mathematical inputs.
• Delay in speech recognition: After saying "Miles" if you speak too quickly after the beep, Miles will not hear the first word you say, please just wait a little before talking, about 0.5 seconds after the beep should be fine.

Full Changelog: v2.2.2...v2.3

Improved Audio Switching and Error Handling (v2.2.2)

M.I.L.E.S Version 2.2.2

M.I.L.E.S Version 2.2.2 focuses on enhancing user interaction through improved error handling and audio input flexibility. Here's what's new:

Enhanced Error Handling

• UI Error Notifications: Errors are now more transparently communicated to users directly within the UI, so you're informed of what went wrong during your interaction.
• OpenAI Error Identification: Specific attention has been given to identifying and handling OpenAI-related errors. The system will update error messages as new ones are encountered.

Improved Audio Input Handling

Audio input has been significantly improved to offer a more seamless experience. Users can now:

• Seamless Input Switching: Open M.I.L.E.S with AirPods, ask a question, switch to a different audio input source, and continue interacting without encountering errors. This flexibility applies vice versa.
• Robust Input Management: The system has been optimized to handle changes in audio input sources without requiring a restart of the app, this should reduce interruptions during use.

⚠️ Known Issues

• Graphical Display Glitches: Issues from the previous version 2.2.1 regarding long outputs and graphical display glitches persist and are under investigation - I do not know what causes it.
• Speech Output for Complex Inputs: Math outputs will still be skewed when Miles is speaking them.

This update aims to streamline the user experience by addressing common frustrations with audio bitrates and bluetooth device behavior on Mac and Windows, as well as clear up misconception when an error would happen, as previously it would state that Miles is processing the request, indefinitely.

Full Changelog: v2.2.1...v2.2.2

Dramatically improved math abilities (v2.2.1)

M.I.L.E.S Version 2.2.1

M.I.L.E.S Version 2.2.1 brings enhancements to math functionalities and introduces LaTeX formatting for a clearer representation of mathematical expressions. Here's what has been updated:

Improved Math Capabilities

The math abilities of M.I.L.E.S have been enhanced to handle complex equations, support multi-variable calculations, and display results in LaTeX formatting. This update allows for a clearer and more accurate representation of mathematical solutions.

LaTeX Formatting Support

Integration of LaTeX support ensures that mathematical expressions are displayed in high-quality formatting directly within M.I.L.E.S's responses.

Examples of Enhanced Math Formatting

• Simple Arithmetic: "2 + 2" is now displayed as (2 + 2 = 4).
• Complex Equations: For "Solve 3x + 5 = 11", M.I.L.E.S displays (x = 2).
• Multi-Variable Calculations: Asking "Solve 2x + 3y = 5 and x - y = 1" results in M.I.L.E.S solving the system, displaying (x = 2, y = 1).

⚠️ Known Issues (affects previous v2.2 version too)

• Graphical Display Glitches: Long outputs over 5 sentences, with more than 10 chats loaded into context, may result in misplaced and wrongly displayed text that could clip over other display elements. This is a graphical issue and does not affect performance.
• Speech Output for LaTeX Text: Speech outputs for newly LaTeX formatted text may be slightly skewed, resulting in M.I.L.E.S vocalizing math results inaccurately. A fix is planned for the next release.
• Code Output Restrictions: M.I.L.E.S has been temporarily instructed not to provide code outputs until improvements for speech disregard in code outputs and correct code highlighting are implemented. As a temporary workaround, saying "Miles, change your prompt so you can code" may allow M.I.L.E.S to generate code for you. Fixes for these issues are anticipated in the next update.

This update is part of the ongoing effort to improve M.I.L.E.S's functionality and user experience. Feedback on these changes is welcome as it helps inform future enhancements.

Full Changelog: v2.2...v2.2.1

M.I.L.E.S is now cross-platform! 🎉 (V2.2)

M.I.L.E.S Now Supports Windows and Mac!

In the latest update, M.I.L.E.S Version 2.1, I've made significant strides in cross-platform compatibility, now supporting both Windows and Mac users:

• Unified Application File: I've implemented custom logic to combine the macOS and Windows versions into a single, streamlined file, simplifying the installation process.

• Custom Windows Setup Script: Windows users will benefit from a custom setup script, tailored to ensure a (not so smooth) installation experience on Windows platforms.

• Comprehensive Installation Wiki: For detailed installation instructions, please visit the wikis:

  • For Mac users: Install M.I.L.E.S on MacOS (Easy Install method)
  • For Windows users: Install M.I.L.E.S On Windows

This update focuses on enhancing accessibility and ease of use across different operating systems, ensuring a seamless experience for all users.

v2.1

M.I.L.E.S just got better 🎉 (V2.1)

M.I.L.E.S Version 2.1 is here, and it brings a blend of aesthetic enhancements and crucial fixes. Check out what's new and improved:

🌟 Enhanced Visuals During Setup:
Experience a dramatically improved visual interface during the setup process. I've refined the UI to make setting up M.I.L.E.S a breeze, with clearer instructions and a more intuitive layout.

🛠️ Bug Fixes and Stability Improvements:
I've squashed several bugs and made numerous stability improvements. These changes ensure a smoother, more reliable experience while using M.I.L.E.S.

👁️ Refined User Interface:
The overall look and feel of M.I.L.E.S have been polished. Enjoy a cleaner, more modern interface with enhanced readability and interaction.

💡 Streamlined Setup Process:
Setting up M.I.L.E.S is now more straightforward thanks to the revamped instructions and guided steps. Get started with M.I.L.E.S quicker and easier than ever.

🔗 Enhanced Link Integration:
Directly access relevant information with improved link integration throughout the application, providing you quick access to what you need, when you need it.

🐞 Resolved Minor Issues:
From UI glitches to small performance hiccups, we've addressed various minor issues, ensuring a more polished and seamless interaction.

🚀 Optimized Performance:
M.I.L.E.S now runs smoother and faster, thanks to under-the-hood performance optimizations.

⚠️ Important Note:

M.I.L.E.S is continuously evolving. While currently supported on MacOS, I'm working diligently to bring you a unified version that runs seamlessly on both Windows 11/10 and MacOS. Stay tuned for upcoming updates, and remember, M.I.L.E.S is still in its beta phase, growing and improving every day.

M.I.L.E.S V2

M.I.L.E.S just got better 🎉 (V2)

Version 2 of Miles is now available! This release introduces several exciting changes and enhancements. Here are the key ones:

🖥️ M.I.L.E.S Got a Redesign: Jump into the conversation screen in the app, or look at the screenshots on the GitHub page, I worked on making contrast higher and making the difference between you and Miles stand out more, as well as some small border radius changes and box shadows.

🎵 Full Spotify Control:
Now, M.I.L.E.S offers complete control over Spotify. You can search and play songs, pause, toggle playback, and adjust the volume – all through simple voice commands.

🧮 More Accurate Calculator:
We've improved the calculator for enhanced precision and reliability, making your calculations more accurate.

🛠️ Triple Tool Functionality:
Multi-task like never before with the ability to use up to three tools at once.

🔧 Troubleshooting Section:
Run into an issue? Our new troubleshooting section provides quick solutions to common problems.

🔄 Self-Modifying LLM Model:
M.I.L.E.S can now autonomously switch between GPT-4-Turbo and GPT-3.5-Turbo models. Simply tell M.I.L.E.S to change his model, and he'll adapt to offer optimal performance.

📋 Flexible System Prompts:
You can choose from three system prompt options for M.I.L.E.S: Default (most reliable), Cheap and Short (less expensive, less reliable), or Custom (where M.I.L.E.S chooses based on the query). Note: His name will always be 'Miles', so there's no need to address him differently.

🔔 Dynamic Action Notifications:
Stay updated with specific actions. Instead of generic messages, M.I.L.E.S will now display detailed notifications like "Searching for 'One Dance' by Drake on Spotify."

🔊 System Volume Control:
Adjust the system volume directly through M.I.L.E.S. This also affects his speaking volume, allowing for a cohesive user experience.

💰 Cheaper Modes: I understand that M.I.L.E.S is like super expensive to run with OpenAI's API, so I gave Miles some tools to help with that, just ask Miles to use all the tools he can to be as cheap as possible, and he should change his LLM to the cheaper one, and he should reduce the token length of his system prompt by 650 tokens.

⚠️ Note: M.I.L.E.S is only supported on MacOS right now, I have a fully functional Windows 11/10 version, but I'm trying to find a way to make it work for both versions instead of 2 separate files, idek if that's possible, but that's why there's no Windows version YET, remember Miles is still VERY beta.

Also I don't know how releases work so I hope this works.

Full Changelog: https://github.com/small-cactus/M.I.L.E.S/commits/v2.0