We need a simpler approach to CAD design.
First time CAD users have to learn a complicated set of actions to even begin to be able to draw an object. The problem with CAD programs is that they all seem to start from what the CAD program wants to do (draw lines and shapes like cubes and cylinders) rather than starting from the picture in the user's mind.
If you do a lot of 3D printing, you will experience continuous challenges with this approach. First, many people do not draw well. Second, most are not fluent in CAD. Third, most people don’t want to spend months becoming fluent in CAD. They do nonetheless want to be able to create objects, express their ideas visually and 3D print some of them. I believe there is a big universe of people who would like to draw / make / create, but who can't because of these challenges.
Proposed Resolution
I suggest that we approach the problem by turning the system on its head, by starting with what the user has in mind. Translate the picture from their mind and their words into language that the computer and CAD program can understand and manipulate. Importantly, to speed up the process, start with a 3D object that serves as a template or jumping off point. I call that template a "digital analogue." I really like that term, which probably says a lot about me.
First time CAD users have to learn a complicated set of actions to even begin to be able to draw an object. The problem with CAD programs is that they all seem to start from what the CAD program wants to do (draw lines and shapes like cubes and cylinders) rather than starting from the picture in the user's mind.
If you do a lot of 3D printing, you will experience continuous challenges with this approach. First, many people do not draw well. Second, most are not fluent in CAD. Third, most people don’t want to spend months becoming fluent in CAD. They do nonetheless want to be able to create objects, express their ideas visually and 3D print some of them. I believe there is a big universe of people who would like to draw / make / create, but who can't because of these challenges.
Proposed Resolution
I suggest that we approach the problem by turning the system on its head, by starting with what the user has in mind. Translate the picture from their mind and their words into language that the computer and CAD program can understand and manipulate. Importantly, to speed up the process, start with a 3D object that serves as a template or jumping off point. I call that template a "digital analogue." I really like that term, which probably says a lot about me.
Next, tie together the work already done in the areas of computer/human language interface (such as Dragon Systems, Apple’s Siri, Google’s speech recognition), machine learning, natural language processing and artificial intelligence (IBM’s Watson), and advances in CAD software to set up a human/computer collaboration. While this iterative man/machine conversation may start out crudely with a first generation system taking an object from the program's library and doing something basic to that object (for example connecting it to another object, changing it in some way, etc.), I envision eventually making this a natural language conversation between user and computer to deliver a 3D drawing of the image the user has in his or her mind, analogous to the work done by a police sketch artist.
During this interactive description and query session I envision the computer offering up possible examples and asking which one is the closest to what the user has in mind. It could offer several examples and ask which of the following images is a) the closest, b) pretty much what the user is thinking, or c) none of these images are anything like what the user is thinking and go back to the drawing board.
I envision this system needing a large library of generic shapes including not only geometric shapes but also objects in the world. There are a number of 3D image repositories that might serve that purpose (e.g., GrabCAD, Turbosquid, etc.) Those objects could serve as the starting/reference points for the computer’s sketch. To the extent the object (the “digital analogue”) is close enough to what the user is envisioning, then the user saves time.
Once the initial reference model has been selected, then the user would work with the computer to change that model by adding or subtracting elements or changing parts of the shape of the model to match what the user has in mind. On this front, researchers at MIT have developed a method of taking CAD files and by using simple to manipulate sliders, allowing the user to manipulate the parameters of the CAD object. See MIT Makes it Much Easier for You to Tweak 3D Printing Designs September 2015 This could be an excellent interim step toward creating the longer term vision of a computerized 3D sketch artist.
The elevator pitch description of this project is as follows:
1. This project is about simplifying the design process for CAD novices.
2. We start from the user side, in the mind of user.
3. The user describes to the computer what she wants drawn.
4. The system translates those words into suggested models and CAD actions.
5. We envision an iterative conversation between user and computer.
6. Where the computer registers ambiguity as to what is expected, the computer will ask for clarification or offer alternatives.
7. Through this human/computer conversation, the computer will sketch and refine the described image.
Here are the basics of what I think of as a "Take and Tweak" methodology for 3D object design:
Phase 1 – Initial reference model selection
1. Understand the part of the model to be acted upon
Interpret through either gesture control, touch screen, machine learning and/or probabilistic inference the portion/area, face and applicable lines/vertices of the model the user is describing (i.e., what they want modified), [Query: I suggest that the Princeton Shape Retrieval and Analysis and Princeton Segmentation Benchmark work could help with this problem.]
2. Understand what the user wants done to that part of the model
Perhaps this is done by defining CAD commands by equating the general natural language words people use to describe the action prompted by the command – i.e., a series of “if user says X, then do Z CAD command” routines. Query whether this would best be accomplished via touch screen or gesture control input to identify the planar surface to be acted on and move model components. Again, MIT's approach might be an excellent start.
3. Offer rendered best guesses for approval.
Draw the modification to the model. Perhaps where there is any ambiguity, the computer could offer to the user the highest probability guesses the computer thinks answers the user’s description.
Then repeat this process until the sketch is completed. It seems apparent that the deeper the pool of digital analogues available, the better the initial reference model and the faster and likely better the overall process will go.
It might help to illustrate what I have in mind by offering the following first generation "dialogue" that describes the interactivity between user and computer. Something like:
User: "I’m thinking of drawing a trophy."
Watson: Shows 5-7 (or more) different types of trophies from its database of images. User picks closest template and then tweaks from there.
User: "Make the top part bigger and less round."
Watson gives 3-5 more choices incorporating its best guess as it tries to follow the user’s instruction. Note: In the early days of the system, make liberal use of the "undo" function! Watson learns from these interactions over time.
User: Here the user might alternatively move away from use of the natural language processing powers of the system and simply use the slider-based tweaking function developed by MIT researchers.
Watson may during the session suggest other alternatives from the database based on the input and changes from the user. Watson may also ask refining questions based on historical preferences by the user or other users.
User: When satisfied with the 3D model: Watson, please make that shape water tight for printing and provide me with an .stl file.
Watston does the proper check, and produces the file as requested.
Is using Watson necessary? Perhaps not initially. It may advance the ball to simply use a database dip to find an appropriate starting model and then to apply the take and tweak methodology to improve the starting model toward what the user has in mind. That said, incorporating a more nuanced "understanding" based on natural language could certainly improve the outcome and make it even more accessible to a broader range of users.
One final note on this concept. I should distinguish that what I am proposing is not a replacement for CAD which is an extraordinarily powerful and useful tool but rather a way for CAD novices to have some access to the power of communicating their ideas in three dimensions regardless of their CAD abilities. Starting to work toward a system that "listens" to the user and helps them to create the 3D vision of what they have in their mind can only help to speed up the process of creative innovation and open up to many more people the power of communicating their ideas visually. I like the power of a system where if you can describe it, you can create it and discuss it more easily with others in the visual language that makes it easier for everybody to be on the same page.
During this interactive description and query session I envision the computer offering up possible examples and asking which one is the closest to what the user has in mind. It could offer several examples and ask which of the following images is a) the closest, b) pretty much what the user is thinking, or c) none of these images are anything like what the user is thinking and go back to the drawing board.
I envision this system needing a large library of generic shapes including not only geometric shapes but also objects in the world. There are a number of 3D image repositories that might serve that purpose (e.g., GrabCAD, Turbosquid, etc.) Those objects could serve as the starting/reference points for the computer’s sketch. To the extent the object (the “digital analogue”) is close enough to what the user is envisioning, then the user saves time.
Once the initial reference model has been selected, then the user would work with the computer to change that model by adding or subtracting elements or changing parts of the shape of the model to match what the user has in mind. On this front, researchers at MIT have developed a method of taking CAD files and by using simple to manipulate sliders, allowing the user to manipulate the parameters of the CAD object. See MIT Makes it Much Easier for You to Tweak 3D Printing Designs September 2015 This could be an excellent interim step toward creating the longer term vision of a computerized 3D sketch artist.
The elevator pitch description of this project is as follows:
1. This project is about simplifying the design process for CAD novices.
2. We start from the user side, in the mind of user.
3. The user describes to the computer what she wants drawn.
4. The system translates those words into suggested models and CAD actions.
5. We envision an iterative conversation between user and computer.
6. Where the computer registers ambiguity as to what is expected, the computer will ask for clarification or offer alternatives.
7. Through this human/computer conversation, the computer will sketch and refine the described image.
Here are the basics of what I think of as a "Take and Tweak" methodology for 3D object design:
Phase 1 – Initial reference model selection
- User: I want you to draw me a ______ [noun].
- Computer: Searches for the 3D digital analogues that best fit the user’s description (near models) and offers to user X number of choices.
- User selects the closest analogue or rejects them all and tries with a new word/description.
- Once the user selects a “close enough to start” reference model, the user describes how they want to change the model to closer match what they have in mind.
- The user will either add something (shape, line or other image) to the reference model, subtract something from the model, or change an attribute of the model (i.e., size, proportion of part, orientation of a component part, etc.).
1. Understand the part of the model to be acted upon
Interpret through either gesture control, touch screen, machine learning and/or probabilistic inference the portion/area, face and applicable lines/vertices of the model the user is describing (i.e., what they want modified), [Query: I suggest that the Princeton Shape Retrieval and Analysis and Princeton Segmentation Benchmark work could help with this problem.]
2. Understand what the user wants done to that part of the model
Perhaps this is done by defining CAD commands by equating the general natural language words people use to describe the action prompted by the command – i.e., a series of “if user says X, then do Z CAD command” routines. Query whether this would best be accomplished via touch screen or gesture control input to identify the planar surface to be acted on and move model components. Again, MIT's approach might be an excellent start.
3. Offer rendered best guesses for approval.
Draw the modification to the model. Perhaps where there is any ambiguity, the computer could offer to the user the highest probability guesses the computer thinks answers the user’s description.
Then repeat this process until the sketch is completed. It seems apparent that the deeper the pool of digital analogues available, the better the initial reference model and the faster and likely better the overall process will go.
It might help to illustrate what I have in mind by offering the following first generation "dialogue" that describes the interactivity between user and computer. Something like:
User: "I’m thinking of drawing a trophy."
Watson: Shows 5-7 (or more) different types of trophies from its database of images. User picks closest template and then tweaks from there.
User: "Make the top part bigger and less round."
Watson gives 3-5 more choices incorporating its best guess as it tries to follow the user’s instruction. Note: In the early days of the system, make liberal use of the "undo" function! Watson learns from these interactions over time.
User: Here the user might alternatively move away from use of the natural language processing powers of the system and simply use the slider-based tweaking function developed by MIT researchers.
Watson may during the session suggest other alternatives from the database based on the input and changes from the user. Watson may also ask refining questions based on historical preferences by the user or other users.
User: When satisfied with the 3D model: Watson, please make that shape water tight for printing and provide me with an .stl file.
Watston does the proper check, and produces the file as requested.
Is using Watson necessary? Perhaps not initially. It may advance the ball to simply use a database dip to find an appropriate starting model and then to apply the take and tweak methodology to improve the starting model toward what the user has in mind. That said, incorporating a more nuanced "understanding" based on natural language could certainly improve the outcome and make it even more accessible to a broader range of users.
One final note on this concept. I should distinguish that what I am proposing is not a replacement for CAD which is an extraordinarily powerful and useful tool but rather a way for CAD novices to have some access to the power of communicating their ideas in three dimensions regardless of their CAD abilities. Starting to work toward a system that "listens" to the user and helps them to create the 3D vision of what they have in their mind can only help to speed up the process of creative innovation and open up to many more people the power of communicating their ideas visually. I like the power of a system where if you can describe it, you can create it and discuss it more easily with others in the visual language that makes it easier for everybody to be on the same page.