I asked the question

How would a TikTok clone built on the AT protocol work?

Cool demo from Burke Holland using GitHub Copilot Edits and a preview Vision plugin for image recognition and UI code generation. Burke was using Claude 35 Sonnet with the plug in for the demo.

I gave this a try with OpenAI gpt-4o but couldn’t reproduce the results. While it’s a cool concept to generate boilerplate in another language (Flutter), but I think the more powerful feature is the ability to convert design images into code.

Not something new, especially with a well-built design system, but once mature this technology will certainly make things easier.

Two honorable mentions were Naming & Branding and UI Design. None of the LLM or generative models are great at this. Figma did attempt to create flow charts so it’s possible a different tool set may need to be field tested.

Meta AI Lama 3

Design Report Card

Technical Report Card

CoPilot

Design Report Card

Technical Report Card

Gemini

Design Report Card

Technical Report Card

ChatGPT 3.5

Design Report Card

Technical Report Card

Claude 3 Sonnet

Design Report Card

Technical Report Card

I ran through a human powered excercise in both product and technical design. Here are my brief notes.

Findings

Much like my prior experiment with first principles reasoning, LLMs aren’t sophisticated enough to provide granular enough product and architectural design to be immediately actionable by a development team.

While I anticipate that eventually LLM tools could be useful enough to assist in product and engineering tasks, it still requires a human to design something that is ubiquitous, intuitive, scalable, and secure.

Bottom line: don’t spend a lot of time bouncing ideas off the major models, and certainly don’t believe the hype that these tools can magically produce GOOD software with a few AI prompts.

Grok

CoPilot

Gemini

ChatGPT 3.5

Claude 3 Sonnet

Prompt

What is it?

The idea of first principals is a powerful problem-solving approach that involves breaking down complex issues into its most basic and fundamental parts.

I’ll use this thought exercise as a first step to challenge assumptions and to unlock a creative solution to our problem. This experiment should demonstrate how capable current LLM is at human style reasoning and problem solving.

Findings

While I think LLMs can be used to generate ideas or compile a list of potential features or technologies, the models aren’t sophisticated enough to understand the context of the problem and how to maximize user value.

This leaves a lot of gaps that humans inevitably need to fill. That’s to be expected. Anyone who has built an app from scratch will tell you that gathering feedback early and often is the best way to create the most useful app.

Novel concepts might be inspired by humans interacting with LLMs, but it’s doubtful it could come up with anything compelling on its own. Could you use LLM as a brainstorming tool, absolutely!

My overall experience generating both a newsletter logo and a title / wordmark was satisfactory. While AI accelerated the ideation phase, the tools struggled to produce simplified graphics.

Results

• Ideation and Design
• Image Generation

I needed a title or banner image which Substack calls a “Wordmark”. The models struggled with this concept on a few levels. I used the following request prompt:

Give me ideas for a wordmark for a Substack newsletter called 
"Project Maestro" about using AI to design and develop a 
consumer application.

NOTE: I had to abandon the prompt above and used the following:

Describe the design aesthetics of a stylized image version of 
newsletter title "Project Maestro". The topic of the newsletter 
is using AI to design and develop a consumer application.

Strategy

Stack multiple AI technologies in three phases:

1. Theme Creation - Brainstorm themes using large language models
2. Prompt Optimization - Use LLM to build efficient text-to-image prompts
3. Image Generation - Create images using generative models

I wanted to generate a square logo that I could use to identify the project and I needed something that both captured the intention of the project but would also be iconic when scaled down.

I started the following request prompt:

Give me ideas for a brand logo for a Substack newsletter about using 
AI to design and develop a consumer application.

Strategy

Stack multiple AI technologies in three phases:

1. Theme Creation - Brainstorm themes using large language models
2. Prompt Optimization - Use LLM to build efficient text-to-image prompts
3. Image Generation - Create images using generative models

I settled on Substack for a couple of reasons, popularity, ease of use and discoverability. I’m already active on the platform and I figured it was a great place to start this journey.

For Maestro’s prologue I decided to put the prospective AI tools to the test by creating the branding for a Substack newsletter.

Overview

• The Tech
• Base Prompts