• ThePrompt
  • Posts
  • GPT3 is powerful but blind 🥸

GPT3 is powerful but blind 🥸

MojoDojo gives GPT3 the ability to see

Hi folks! 👋🏻 After a long weekend of Thanksgiving feasting, it's time to get back to work. Here's some AI news to help ease you back into the work week.

MojoDojo - Giving sight to GPT3

GPT3 has changed the way we use AI in our daily lives — many tasks like question answering, code generation, and article writing are based on this model.

But that model is not action-driven, and it doesn't self-improve.

MineDojo's vision (pun intended) is to build an “embodied GPT3” that takes the right actions given any language prompts.

To do that, they collected massive Minecraft player data from Youtube video streams, Reddit forum threads, and Wiki pages that teach AI to do things. And they have 3,000+ tasks, making it one of the largest agent benchmarks for embodied agent research*.

GPT3 is very important for innovators and makers today, but the results from MineDojo could be even more significant in the future — and their current NeurIPS 2022 Outstanding Paper Award is a testament to that.

*What is embodied agent research?

Embodied agent research is a field of Artificial Intelligence that studies how artificial agents can be made to interact with the environment through a physical body within that environment, without giving any instructions.

3D analysis of The 2022 World Cup - by NY Times⚽️

The research team at NY Times utilized a novel model called ICON to create a 3D scene from real photos that explain how the U.S. goal against Wales unfolded.

A very interesting application for this model, and you can try the model here.

📚 Educational corner

🛼 Makers corner

  • Yohei created an app that analyzes pitch decks and creates questions a VC might ask - This is a great way for founders to test their decks and see where they may have blind spots

🤓 Tech corner

  • SinDiffusion - a new model that generates realistic and diverse images by learning from a single image.

  • EDICT - a new method that can go back in the diffusion process of an image, find the starting point of an element, and replace it with another.

  • MagicPony - a new model that learns a function that can estimate the 3D shape of an articulated animal, like a horse, given a single test image.

🎨 Creativity corner

🧱 What are you building?

If you're using AI in your work or projects, I'd love to hear about it! Please reply to this email and let us know what you're up to. We may feature your work in our newsletter.

❤️ If you like The Prompt, and want to support my work:

  • Share The Prompt with a friend, and invite them to subscribe here.

  • Book an ad in The Prompt ( reply to this email if you’re interested)

What'd you think of today's edition?

Login or Subscribe to participate in polls.

Thank you for reading! ✌🏼