Welcome to the Onshape forum! Ask questions and join in the discussions about everything Onshape.

First time visiting? Here are some places to start:
  1. Looking for a certain topic? Check out the categories filter or use Search (upper right).
  2. Need support? Ask a question to our Community Support category.
  3. Please submit support tickets for bugs but you can request improvements in the Product Feedback category.
  4. Be respectful, on topic and if you see a problem, Flag it.

If you would like to contact our Community Manager personally, feel free to send a private message or an email.

ChatGPT/Copilot for FeatureScript?

S1monS1mon Member Posts: 2,321 PRO
edited December 2022 in FeatureScript
I've spent too much time over the past few days playing with ChatGPT. I've tried asking it about simple things in FeatureScript, and it just makes up code which may be more like Javascript - which is disappointing. ChatGPT supposedly doesn't know anything newer than 2021, which is also frustrating.

In any case, it seems like people are having success getting ChatGPT to write certain types of code. It's passed the AP Computer Science written exam. There are plenty of testimonials of it being able to write code that it hasn't seen before. Here's an example of some 3D graphics in three js/HTML. People are starting to get it to successfully do the advent of code exercises. ChatGPT can help debug code too.

I would love to see something like this working for FeatureScript. It's not clear how the large language model was trained on programming, but one would assume that it's scraping things like GitHub and StackOverflow.  Other than this forum, and a few public resources, I wonder what can easily be scraped for training? Are the public FeatureScripts set up in a way that they get indexed by search engines? What can be done to help with training AI models on FeatureScript?


Comments

  • Miguel_PiedrahitaMiguel_Piedrahita Member Posts: 2
    I'm having the same experience, it writes code that looks somewhat like FeatureScript at first glance, but it's way wrong. Bummer! (Interestingly, I tested it with OpenSCAD and it does somewhat better.)

    "Are the public FeatureScripts set up in a way that they get indexed by search engines?" This is a good question, perhaps somebody from OnShape can answer?

    When GPT4 comes out, everything will likely get somewhat better, so that may help. Further down the road, hopefully some startup (or OnShape themselves) will build a model on top of GPT with additional tuning and training, resulting in an AI that performs well for this purpose. That would be awesome!
  • emagdalenaC2iemagdalenaC2i Member, Developers, Channel partner Posts: 858 ✭✭✭✭✭
    Looks amazing!!
    I have seen other examples like this creating a module for Odoo (An Open source ERP system) 

    S1mon said:
    Are the public FeatureScripts set up in a way that they get indexed by search engines? What can be done to help with training AI models on FeatureScript?

    I think Onshape could add a FeatureScript repository in their GitHub account https://github.com/onshape-public and add there the "Community sporlight" FeatureScripts

    Even better, users could then add PR to that repository to add new features or improve the existing ones.


    Un saludo,

    Eduardo Magdalena                         C2i Change 2 improve                         ☑ ¿Por qué no organizamos una reunión online?  
                                                                         Partner de PTC - Onshape                                     Averigua a quién conocemos en común
  • MichaelPascoeMichaelPascoe Member Posts: 1,695 PRO
    edited December 2022

    This would be great. It doesn't know fs, but you can give it an example and it will auto fill like excel:




    Learn more about the Gospel of Christ  ( Here )

    CADSharp  -  We make custom features and integrated Onshape apps!   cadsharp.com/featurescripts 💎
  • S1monS1mon Member Posts: 2,321 PRO
    @MichaelPascoe
    Is there some reason you’re using GPT3 instead of ChatGPT?
  • MichaelPascoeMichaelPascoe Member Posts: 1,695 PRO
    edited December 2022

    I didn't know I wasn't using ChatGPT lol.
    Looks like ChatGPT gets overloaded by too many users, this one seems to work all the time.


    Learn more about the Gospel of Christ  ( Here )

    CADSharp  -  We make custom features and integrated Onshape apps!   cadsharp.com/featurescripts 💎
  • emagdalenaC2iemagdalenaC2i Member, Developers, Channel partner Posts: 858 ✭✭✭✭✭
    @sebastian_glanzner that could be a game changer in the CAD market
    Un saludo,

    Eduardo Magdalena                         C2i Change 2 improve                         ☑ ¿Por qué no organizamos una reunión online?  
                                                                         Partner de PTC - Onshape                                     Averigua a quién conocemos en común
  • Evan_ReeseEvan_Reese Member Posts: 2,060 PRO
    edited December 2022

    This would be great. It doesn't know fs, but you can give it an example and it will auto fill like excel:
    This is a cool use case in the meantime! I bet it could produce some intense tables from a set of references. If you wanted it to take a copy/pasted table for some hardware, and get it to format as a map. I just tested it a bit with this table and prompted it to match this code format:

    
    NPTChart = {
    "size" : {
    "diameter" : value,
    "tpi" : value,
    etc...
    }
    };
    
    
    It seemed to work, but the Playground one didn't write it all (maybe there's a word limit?) and ChatGPT had a network error due to high traffic. What I saw was promising though.
    Evan Reese / Principal and Industrial Designer with Ovyl
    Website: ovyl.io
  • jason_1jason_1 Member Posts: 24 PRO
    I had the same idea and tried to use it to generate regex to use for for find and replace to parse AoC inputs in FS. The examples it gave were well explained (and seemed plausible) but didn't actually work. I then asked about doing the same thing in Google Sheets instead and the same thing happened. It's very confident, but often wrong. 
  • MichaelPascoeMichaelPascoe Member Posts: 1,695 PRO
    edited December 2022

    To get it to write longer code, you need to adjust the Maximum length:

    Also, you can chain questions and responses if it didn't quite get it right. Below it's response, tell it the details that it missed and how to do them. Then hit submit again.


    Learn more about the Gospel of Christ  ( Here )

    CADSharp  -  We make custom features and integrated Onshape apps!   cadsharp.com/featurescripts 💎
  • hugo_gammonhugo_gammon Member Posts: 4 EDU
    If some kind of integration between ChatGPT and Onshape was made it would be a impressive distinguishing factor between this and other CAD programs. There's really interesting stuff to come from this, and I cannot wait for it to be implemented.

    Mainly just commenting to try and draw discussion to this topic.
  • steve_shubinsteve_shubin Member Posts: 1,066 ✭✭✭✭
    edited January 2023
    https://www.dailymail.co.uk/news/article-11679757/ChatGPT-averaged-C-law-professors-used-generate-answers-law-school-exams.html

    QUOTE — performed particularly poorly at multiple choice questions involving math

    For some reason, i supposed this might be one area it would have been half decent at

  • S1monS1mon Member Posts: 2,321 PRO
    Large language models are apparently not good at math, at least not the way that they have been trained so far. There are a lot of examples at this point of ChatGPT making very stupid mistakes on very simple math problems. It has passed parts of the Bar exam and the AP computer science test free response section.

    Wolfram has a post about combining their tools with ChatGPT which would be pretty amazing.


  • S1monS1mon Member Posts: 2,321 PRO
    GPT4 and plugins for ChatGPT (Wolfram!) have come out since this thread was last active. It's getting more and more amazing...

    I've added an improvement request to create a "Copilot for FeatureScript". Please vote!
  • emagdalenaC2iemagdalenaC2i Member, Developers, Channel partner Posts: 858 ✭✭✭✭✭
    edited March 2023
    Done!
    I have written a series of articles on LinkedIn about ChatGPT and I just published the last one a while ago "I have failed the Turing test again".

    Also some specific about ChatGPT and FeatureScript, like this https://www.linkedin.com/pulse/ejemplo-pr%C3%A1ctico-de-uso-chatgpt-con-featurescript/
    Un saludo,

    Eduardo Magdalena                         C2i Change 2 improve                         ☑ ¿Por qué no organizamos una reunión online?  
                                                                         Partner de PTC - Onshape                                     Averigua a quién conocemos en común
  • sebastian_glanznersebastian_glanzner Member, Developers Posts: 398 PRO
    I use ChatGPT 4 to review my FeatureScripts. It often gives me tipps to use better variable names and makes suggestions.

    Once I asked it to create a FeatureScript itself to build a simple cylinder. The user should be able to input radius and height. The Code had small errors but everything was there. It added inputs, created a sketch and added the extrude. It also added comments!
  • S1monS1mon Member Posts: 2,321 PRO
    I use ChatGPT 4 to review my FeatureScripts. It often gives me tipps to use better variable names and makes suggestions.

    Once I asked it to create a FeatureScript itself to build a simple cylinder. The user should be able to input radius and height. The Code had small errors but everything was there. It added inputs, created a sketch and added the extrude. It also added comments!

    Interesting. I just signed up for for ChatGPT Plus, and have just started to play with GPT4. Previously when I tried to use ChatGPT (3.5) and I asked it Featurescript questions, it seemed happy to completely make up syntax which didn't make sense. I don't know what language or API it thought it was talking to, but it was useless. 

    Can you share an example of the prompts and dialog you're having with GPT4? 
  • S1monS1mon Member Posts: 2,321 PRO
    I tried getting it to help me remember where the "show code" command is (it's in the part studio tab right-click menu). Here are the various almost intelligent but not quite right answers from GPT4 (the Github URL is BS or is no longer real, and it missed the part studio tab menu completely):


  • sebastian_glanznersebastian_glanzner Member, Developers Posts: 398 PRO
    edited March 2023
    @S1mon

    I used german when I talked to it so I translated my prompt:
    Can you help me to program something? I am supposed to create a "FeatureScript" for Onshape. The program should create a cylinder and the user has to enter the radius and the height.

    Of course I can help you with that! FeatureScript is a programming language developed specifically for Onshape. Here is a simple example of a FeatureScript that creates a cylinder where the user can enter radius and height:



    Copy the above code into a new FeatureScript document in Onshape. To use the feature, add it to your Part Studio and enter the desired values for Radius and Height. The cylinder will be created accordingly.


  • sebastian_glanznersebastian_glanzner Member, Developers Posts: 398 PRO
    The I told ChatGPT that I corrected the code, posted it and it replied:

    Great, I'm glad you adapted the code to your needs and it works now. Your version of the code uses a newer version of FeatureScript (1977 instead of 1.0), and you have adjusted the definition of the coordinate systems and extrusion direction accordingly.

    The improved code looks good, and it should now create a cylinder in your Onshape Part Studio based on the user entered values for radius and height. If you have any further questions or need help programming with FeatureScript, don't hesitate to ask.
  • NeilCookeNeilCooke Moderator, Onshape Employees Posts: 5,310
    When I ask the exact same question it gives me this:

    // Define cylinder parameters (radius and height)
    annotation { "Name" : "Cylinder Radius", "Filter" : EntityType.EDGE, "CustomControl" : "SPINNER_CONTROL" }
        radius is dimension( length );
    annotation { "Name" : "Cylinder Height", "Filter" : EntityType.FACE, "CustomControl" : "SLIDER_CONTROL" }
        height is dimension( length );
    
    // Define cylinder center point
    center_point = vector(0, 0, 0);
    
    // Create cylinder
    cylinder = cylinder(center_point, radius, height);
    
    // Add cylinder to feature list
    opSheetBody(sheetId, id + " Cylinder", cylinder)<span>;</span>

    Maybe I should ask in German :) 
    Senior Director, Technical Services, EMEAI
  • sebastian_glanznersebastian_glanzner Member, Developers Posts: 398 PRO
    Yeah, it is really weird. You can ask the same question and get different results each time. You can also click on "regenerate response" and you get a "new" answer  :neutral:

    I tried it again and provided a "Template" as a start:





    There are two errors: The plane for the sketch and the opExtrude command is wrong.

  • robert_stilesrobert_stiles Member Posts: 89 PRO
    ha, I just tried GPT (being not very good at feature script). Great I thought, and shared some trial code past with my co founder (who is good at feature script) and he just laughed. I clearly will not be able to stop burdening him with my requests just yet. 
    GPT is really interesting, but its obviously all about how much information it has to learn from. Last year I asked it to write a planning design and access statement - I was blown away at how "good" the results were, with relevant, local, and accurate planning policy references. These documents are mostly worthless page fill and box ticking exercises, and GPT is evidently very good at reproducing this. In the UK, all our planning documentation is out there in the public realm. 
    I then asked GPT to try and write an outline construction specification for a simple build. It was absolutely laughable. Very few architects share this information, its hard to find even half decent examples out there anywhere. Architects silo this information - its not about protecting IP per say, its just there is no reason to put it in the public domain, nor any repo to put it in. 
    I think that's what's so interesting about Onshape in the next 5 - 10 years. Because its all in the cloud, there are so many more examples for AI to learn from, both feature script and model building strategies. Power CAD users have been scripting custom utilities for years, but its always been kept in a silo until now. I appreciate that pro users data might not be shared publicly, but Onshape might still have access to it - if a privacy choice is made? Do we want AI to do get better at this? I do! but is that because I don't do it very well so its not my skills at risk!? To be honest I'm not really sure what my tangible skills actually are, so AI will have to identify these before it takes them from me. 
  • S1monS1mon Member Posts: 2,321 PRO
    @robert_stiles

    Availability of training data is huge. Unfortunately if it's not available, it's shocking how much these LLMs will just make up stuff, and still sound confident. 

    For all of the Onshape STD source and public Feature Scripts, it would make sense if they were available to some tool to learn from. Internally used stuff would have to only be opt-in. 
  • sebastian_glanznersebastian_glanzner Member, Developers Posts: 398 PRO
    edited July 2023
    It's beneficial to inform ChatGPT when it makes a mistake, especially with commands like opextrude. You can do this by simply copying the correct opextrude command syntax from the help documentation (https://cad.onshape.com/FsDoc/library.html) and pasting it into your conversation with ChatGPT. This allows the AI to learn from its error and make the necessary corrections.

    It would be great to have a custom AI that you can train with Onshape data!
  • MichaelPascoeMichaelPascoe Member Posts: 1,695 PRO

    @sebastian_glanzner does it actually add it to its global data or only the data for that conversation?


    Learn more about the Gospel of Christ  ( Here )

    CADSharp  -  We make custom features and integrated Onshape apps!   cadsharp.com/featurescripts 💎
  • sebastian_glanznersebastian_glanzner Member, Developers Posts: 398 PRO
    @MichaelPascoe
    This will only be relevant for the current conversation with ChatGPT 
Sign In or Register to comment.