The Background
Our conversations at Learning Agent started over the question, "What roles can Generative AI play in learning that no one seems to be talking about?" We've seen the paradigm of a text chat with an all-knowing tutor bot, but what else is possible?
One angle is to challenge the "text chat" part---is it possible to make a tutor bot that could see and meaningfully interpret a user's hand-drawn sketches, and incorporate these for a richer tutoring experience?
As a proof of concept, we've developed Hummingbird---a chatbot tutor that still does text chat, but also affords the user a scratchpad to submit. It's built with TLDraw and GPTVision, and we've seen it do some impressive things!
Beware of Dragons
As educators we feel compelled to share this with a warning label: Hummingbird is sitting on GPT4, so especially for math topics, it's not hard to get it to say something wrong or get it confused. Tread carefully with introducing this iteration of Hummingbird to novice learners---it might be fun with supervision but be wary of needlessly confusing someone. (Though we are curious about the soon-to-be-released models with reasoning capabilities.)
How to Try It
Try Hummingbird with your own prompt (also requires your own API key).
See It In Action
Hummingbird helps you understand area and perimeter of a rectangle:
Hummingbird helps you derive side length relationships for a special right triangle:
Hummingbird helps you sketch parabolas:
Comments