In today’s fast-paced technological landscape, the fusion of artificial intelligence (AI) with real-world applications presents a frontier full of potential. A recent development in this space has culminated in a groundbreaking proof of concept that allows AI to physically interact with the real world. Here’s how it was achieved.
Unlocking New Possibilities with Function Calling Models
OpenAI has introduced an intriguing way to engage with the world of AI through their gpt-4-0613 and gpt-3.5-turbo-0613 function calling models. As per the official documentation, developers can now describe functions to these models and have them intelligently choose to output a JSON object containing arguments to call those functions. This has created an innovative way to more reliably connect GPT’s capabilities with external tools and APIs.
Drawing Parallels with Swagger
For those familiar with API development, this process is reminiscent of accessing Swagger for an API. With GPT-4’s higher intelligence, the responses are so accurate that you could copy and paste them directly into Postman. In other words, GPT is now capable of making API calls.
From Chat to Function Object
Ordinarily, OpenAI Chat Completions respond with a Chat object. This new development has added a new dimension, enabling the response to be a Function object. Once you obtain this Function object, the possibilities are endless.
In my personal AI system, I have implemented TypeScript Cloud Functions that I call from my client Angular app when I receive the OpenAI response. The flexibility of this approach means it can be implemented in numerous ways, tailoring to various needs and applications.
A Connection to the Physical World
The most exciting part of this journey was to bring AI closer to physical reality. Using 110v AC Wifi electric switches, specifically Sonoff switches available readily on Amazon, I embarked on a mission to bridge this gap. By cutting an extension cord, or wall wiring in half, and adding this device in between, I was able to connect AI to the real world.
Light Bulb: The First Step
My first proof of concept was a humble light bulb. By asking my personal AI system to control it, I successfully turned it on and off. This opens doors to practical applications like setting up webhooks to detect weather changes and automatically turning on lights when the sun sets.
Beyond the Basics: A Complex Scenario
The possibilities don’t end there. Imagine a motion detector alarm triggered in a remote area, automatically dispatching a drone to a specific GPS location to capture pictures of the event. The potential applications are limitless.
Conclusion
What started as a simple text input-output system in ChatGPT has evolved into something far more powerful and interactive. By harnessing APIs to control physical devices, a whole new approach to thinking and problem-solving has emerged.
This advancement is not merely a technological curiosity; it is a significant step towards the seamless integration of digital intelligence with our everyday lives. The future of AI is not confined to screens and keyboards; it’s spreading its wings to touch and transform the world around us. And this is just the beginning.