Back to Blog

    Agentic AI

    Misty, our AI employee, would like to meet you

    Thanks to Microsoft Azure OpenAI, our "Misty" can now see, hear and speak, and is impressively intelligent. Misty is an open robotics platform equipped with a wealth of programming tools such as Blockly, Python and C#, an HTTP API and advanced sensors.

    April 6, 20243 min read
    Marcel Haas

    Marcel Haas

    Solution Architect, CEO

    marcel.haas@cnext.ch
    CNEXT AI Agent
    In this article

    This post was first published on 6 April 2024 and is kept as it appeared then. Details about products, dates and people reflect that time.

    Who is Misty?

    Misty is a robot developed by Misty Robotics. Misty Robotics is a company that specialises in creating robots for personal and professional use. Misty Robotics' vision is to give robots a personality and make them part of everyday life.

    The Misty Robotics platform is an open and extensible platform that lets developers and hobbyists build their own applications for Misty. The platform offers a range of tools and resources to simplify and speed up programming Misty. Among other things, the platform includes:

    • A web-based development environment for writing and running code for Misty in various languages such as JavaScript, Python or C#.
    • A mobile app for controlling and configuring Misty via Bluetooth or Wi-Fi.
    • A REST API for communicating with Misty via HTTP requests and accessing her sensors, cameras, microphones, speakers and other hardware.
    • An SDK for building native applications for Misty in various languages such as Java, Swift or Kotlin.
    • A community for connecting with other Misty developers and learning from their experiences and projects.

    What do we do with Misty?

    Misty is a playground and a fun project for CNEXT employees to explore robotics and AI. We have made Misty much more intelligent by connecting her seamlessly to Azure Speech and integrating Azure OpenAI for reasoning and video/speech analysis. Misty has also been connected to Power Automate and Logic Apps to process live data from our ERP system and draw conclusions from it.

    What features does Misty offer?

    Misty offers a range of features that make her a versatile and intelligent robot. The most important features include:

    • Face recognition: Misty can recognise faces and assign names to them. She can also recognise emotions such as joy, sadness or anger and react accordingly.
    • Speech recognition: Misty can understand voice commands and respond to them. She can also hold natural conversations and tell jokes.
    • Object recognition: Misty can recognise and name objects. She can also track objects and avoid them.
    • Obstacle detection: Misty can detect obstacles and avoid them. She can also climb up and down stairs.
    • Mapping: Misty can map her surroundings and find her way around them. She can also set waypoints and follow them.
    • Self-expression: Misty can express her mood through her eyes, her head, her arms and her tail. She can also make sounds and play music.

    What does Azure OpenAI offer?

    Azure OpenAI is a service based on OpenAI Codex technology. OpenAI Codex is a neural network that can translate natural language into code. Azure OpenAI lets developers build applications with natural language without having to write code. Among other things, Azure OpenAI offers the following features:

    • Natural language generation: Azure OpenAI can generate texts in various languages that are coherent, relevant and creative.
    • Natural language understanding: Azure OpenAI can understand and analyse texts in various languages. It can also answer questions, create summaries or rewrite texts.
    • Natural language interaction: Azure OpenAI can hold natural conversations and respond to different intentions. It can also recognise emotions and respond to them appropriately.

    Image material from mistyrobotics

    Misty is a robot with a lot of potential to enrich our lives. With the Misty Robotics platform and Azure OpenAI, we can build our own applications for Misty and turn her into a personal assistant or an entertaining companion. Misty is more than just a robot, she is a friend.

    You can find more information about Misty at: www.mistyrobotics.com

    Do you have questions about this article? I look forward to hearing from you.

    Power PlatformAgentic AI
    Teilen:

    AI tools may have supported the creation of this article. CNEXT is responsible for the published content. Product features and prices can change; check the linked primary sources for current details.

    Marcel Haas

    Marcel Haas

    Solution Architect, CEO

    6x Microsoft Applied Skills

    Have questions about this topic?

    Our experts are happy to advise you – free and without obligation.