Turning Hand-Drawn Architecture Diagrams into Digital Diagrams with Generative AI

Disclaimer: This content reflects my personal opinions, not those of any organizations I am or have been affiliated with. Code samples are provided for illustration purposes only, use with caution and test thoroughly before deployment.

Background

As an architect, whiteboarding is an essential part of the design and problem-solving process. However, once the whiteboard session is over, it can be challenging to translate those hand-drawn diagrams into a digital format that can be easily shared and collaborated on. Traditionally, this process involved manually recreating the diagrams using diagramming tools, which can be time-consuming and prone to errors.

My Approach with Generative AI

To streamline this process, I’ve adopted a new approach that leverages the power of generative AI. By combining a multi-model large language model (LLM) from Amazon Bedrock with the versatile diagramming tool draw.io, I can effortlessly convert hand-drawn architecture diagrams into digital diagrams.

Step-by-Step Guide

Take a photo of the whiteboard diagram: Start by capturing a clear image of your hand-drawn architecture diagram.

Use Bedrock chat to convert the image to Mermaid syntax: Upload the image to the Bedrock chat interface and provide a prompt asking it to convert the hand-drawn diagram into Mermaid syntax. Mermaid is a popular markdown-based diagramming language that can be easily imported into various diagramming tools, including draw.io.

Here is my prompt:

Please turn this architecture diagram into mermaid diagram syntax. The mermaid diagram should be left to right and use human-readable labels. Box that wraps other nodes should us e mermaid subgraph. First list all the nodes and their connections, then convert it into mermaid syntax

I use the Claude 3 Sonnet model so I can attach the photo together with the prompt. Also notice that I use chain of thought prompting, asking it to list all the nodes and connections before writing the mermaid syntax. This helps to make sure the LLM will not miss some nodes and connections

This is the response from the LLM:

Here is a list of all the nodes and their connections:

Nodes:

1. User
2. Web GUI
3. API Server
4. DB
5. S3
6. SageMaker Endpoint
7. Model
8. CI/CD pipeline in CodePipeline
9. Machine Log
10. Storage Gateway

Connections:

• User -> Web GUI
• Web GUI -> API Server
• API Server -> DB
• API Server -> SageMaker Endpoint
• Machine Log -> Storage Gateway
• Storage Gateway -> S3
• S3 -> SageMaker
• SageMaker -> Model
• Model -> CI/CD pipeline in CodePipeline
• CI/CD pipeline in CodePipeline -> SageMaker Endpoint

Here’s the mermaid syntax for the diagram, with a left-to-right layout and human-readable labels:

graph LR
    subgraph "On-Prem"
        MachineLogs[Machine Log] --> StorageGateway[Storage Gateway]
    end

    User --> WebGUI[Web GUI]
    WebGUI --> APIServer[API Server]
    APIServer --> DB[DB]
    APIServer --> SageMkerEndpoint[SageMaker Endpoint]

    StorageGateway --> S3
    S3 --> SageMaker[SageMaker]
    SageMaker --> Model
    Model --> CICDPipeline[CI/CD Pipeline in CodePipeline]
    CICDPipeline --> SageMkerEndpoint

This mermaid diagram represents the architecture with nodes arranged from left to right, and the “On-Prem” components (Machine Log and Storage Gateway) enclosed in a subgraph. The connections between the nodes are shown with arrows, and each node has a human-readable label.

Sometimes the LLM will generate invalid syntax or it will make obvious mistakes. You can continue to chat and ask it to fix the syntax error or revise the diagram.

Import the Mermaid syntax into draw.io: Once you have the Mermaid syntax generated by Bedrock, open draw.io and navigate to Arrange > Insert > Advanced > Mermaid.

A pop-up will appear and you can paste the Mermaid syntax into the provided text area.

Once you click Insert, draw.io will automatically interpret the Mermaid syntax and render the corresponding diagram.

Make manual adjustments: While the Mermaid syntax generated by Bedrock is often accurate, there may be instances where manual adjustments are required, especially when my handwriting is ineligible even to humans. Also, as an electrical engineer by training, I cannot tolerate curved lines. Because lines on circuit diagrams are usually diagonal. You can select the lines and then use the right hand side panel to turn it into diagonal.

By following these steps, you can easily transition from hand-drawn architecture diagrams on a whiteboard to digital diagrams that can be easily shared, edited, and collaborated on using draw.io. The combination of generative AI and powerful diagramming tools empowers you to work more efficiently and effectively, ultimately leading to better design outcomes and improved communication within your team.