Sketch2Code is part of Microsoft Azure Cognitive Services, the Custom Vision and Computer Vision APIs used for object detection, which uses trained models to detect HTML objects, then employs the text recognition capabilities in extracting handwritten text in the design.
Now, Microsoft has launched Sketch2Code as a web-based application to help in converting hand-drawn sketches into HTML codebase, by combining the object and text to generate HTML snippets of different web design elements.
While Microsoft AI research is dated back to over two decades, but only restricted to internal deployment; now the company is gradually bringing it to open web for everyone to use through simple, and yet powerful AI tools.
How Sketch2Code Dynamic designs work
The process of transforming handwritten image to HTML using Sketch2Code, first requires that the user uploads an image through the website. Then a custom vision model will try to predict what HTML elements are present in the image and their location.
Once the handwritten text recognition service reads the text within the predicted elements, the layout algorithm will use the spatial information from the bounding boxes of the predicted elements to generate a grid structure to accommodate them all.
The HTML generation engine, then uses all the pieces of information to generate HTML markup code reflecting the exact result.
Use cases for Sketch2Code Dynamic designs
Web designers can use Sketch2Code to collaborate and draw on whiteboard to see the resulting code to test their own new web page ideas immediately. As the interface design process involve lots of creativity that starts right from the whiteboard where designers share ideas.
So the design is captured within a photograph and manually translated into working HTML wireframe to play within the web browser with the help of Sketch2Code.
Sketch2Code is developed in conjunction with Kabel and Spike Techniques, and the source code is now available on GitHub for developers to tinker with, or anyone to get started with AI and become an AI developer.
No comments