ChatGPT/Copilot for FeatureScript?

S1mon

I've spent too much time over the past few days playing with ChatGPT. I've tried asking it about simple things in FeatureScript, and it just makes up code which may be more like Javascript - which is disappointing. ChatGPT supposedly doesn't know anything newer than 2021, which is also frustrating.

In any case, it seems like people are having success getting ChatGPT to write certain types of code. It's passed the AP Computer Science written exam. There are plenty of testimonials of it being able to write code that it hasn't seen before. Here's an example of some 3D graphics in three js/HTML. People are starting to get it to successfully do the advent of code exercises. ChatGPT can help debug code too.

I would love to see something like this working for FeatureScript. It's not clear how the large language model was trained on programming, but one would assume that it's scraping things like GitHub and StackOverflow.  Other than this forum, and a few public resources, I wonder what can easily be scraped for training? Are the public FeatureScripts set up in a way that they get indexed by search engines? What can be done to help with training AI models on FeatureScript?

Miguel_Piedrahita

I'm having the same experience, it writes code that looks somewhat like FeatureScript at first glance, but it's way wrong. Bummer! (Interestingly, I tested it with OpenSCAD and it does somewhat better.)

"Are the public FeatureScripts set up in a way that they get indexed by search engines?" This is a good question, perhaps somebody from OnShape can answer?

When GPT4 comes out, everything will likely get somewhat better, so that may help. Further down the road, hopefully some startup (or OnShape themselves) will build a model on top of GPT with additional tuning and training, resulting in an AI that performs well for this purpose. That would be awesome!

emagdalenaC2i

Looks amazing!!
I have seen other examples like this creating a module for Odoo (An Open source ERP system) 

S1mon said:

Are the public FeatureScripts set up in a way that they get indexed by search engines? What can be done to help with training AI models on FeatureScript?

I think Onshape could add a FeatureScript repository in their GitHub account https://github.com/onshape-public and add there the "Community sporlight" FeatureScripts

Even better, users could then add PR to that repository to add new features or improve the existing ones.

MichaelPascoe

This would be great. It doesn't know fs, but you can give it an example and it will auto fill like excel:

S1mon

@MichaelPascoe
Is there some reason you’re using GPT3 instead of ChatGPT?

MichaelPascoe

I didn't know I wasn't using ChatGPT lol.
Looks like ChatGPT gets overloaded by too many users, this one seems to work all the time.

sebastian_glanzner

Great ideas! Let's take it one step further:

What if we allowed ChatGPT to read the code of every public PartStudio on Onshape? This would enable it to understand the geometry and features of each part, and potentially even generate new PartStudios based on user input.

Imagine being able to discuss your design ideas with ChatGPT and have it create the necessary geometry for you in real-time. This would greatly improve the design process and make it even more efficient and user-friendly. Plus, with ChatGPT's natural language processing capabilities, you could simply describe the part you need and let it do the rest.

Overall, implementing this feature would be a major step forward for Onshape and could significantly enhance the user experience.

BTW: The text above was improved by ChatGPT!!! This was my original text:

Great ideas! 
Maybe we could go a step further:

We should let ChatGPT to read the code of every public PartStudio!

Onshape could implement the possibility for ChatGPT to create new PartStudios.

Then the user could talk with ChatGPT about the needed parts and its features while Onshape creates the geometry.

emagdalenaC2i

@sebastian_glanzner that could be a game changer in the CAD market

EvanReese

MichaelPascoe said:

This would be great. It doesn't know fs, but you can give it an example and it will auto fill like excel:

This is a cool use case in the meantime! I bet it could produce some intense tables from a set of references. If you wanted it to take a copy/pasted table for some hardware, and get it to format as a map. I just tested it a bit with this table and prompted it to match this code format:

NPTChart = {
"size" : {
"diameter" : value,
"tpi" : value,
etc...
}
};

It seemed to work, but the Playground one didn't write it all (maybe there's a word limit?) and ChatGPT had a network error due to high traffic. What I saw was promising though.

jason_1

I had the same idea and tried to use it to generate regex to use for for find and replace to parse AoC inputs in FS. The examples it gave were well explained (and seemed plausible) but didn't actually work. I then asked about doing the same thing in Google Sheets instead and the same thing happened. It's very confident, but often wrong. 

MichaelPascoe

To get it to write longer code, you need to adjust the Maximum length:

Also, you can chain questions and responses if it didn't quite get it right. Below it's response, tell it the details that it missed and how to do them. Then hit submit again.

hugo_gammon

If some kind of integration between ChatGPT and Onshape was made it would be a impressive distinguishing factor between this and other CAD programs. There's really interesting stuff to come from this, and I cannot wait for it to be implemented.

Mainly just commenting to try and draw discussion to this topic.

steve_shubin

https://www.dailymail.co.uk/news/article-11679757/ChatGPT-averaged-C-law-professors-used-generate-answers-law-school-exams.html

QUOTE — performed particularly poorly at multiple choice questions involving math

For some reason, i supposed this might be one area it would have been half decent at

S1mon

Large language models are apparently not good at math, at least not the way that they have been trained so far. There are a lot of examples at this point of ChatGPT making very stupid mistakes on very simple math problems. It has passed parts of the Bar exam and the AP computer science test free response section.

Wolfram has a post about combining their tools with ChatGPT which would be pretty amazing.

S1mon

GPT4 and plugins for ChatGPT (Wolfram!) have come out since this thread was last active. It's getting more and more amazing...

I've added an improvement request to create a "Copilot for FeatureScript". Please vote!

emagdalenaC2i

Done!
I have written a series of articles on LinkedIn about ChatGPT and I just published the last one a while ago "I have failed the Turing test again".

Also some specific about ChatGPT and FeatureScript, like this https://www.linkedin.com/pulse/ejemplo-pr%C3%A1ctico-de-uso-chatgpt-con-featurescript/

sebastian_glanzner

I use ChatGPT 4 to review my FeatureScripts. It often gives me tipps to use better variable names and makes suggestions.

Once I asked it to create a FeatureScript itself to build a simple cylinder. The user should be able to input radius and height. The Code had small errors but everything was there. It added inputs, created a sketch and added the extrude. It also added comments!

S1mon

sebastian_glanzner said:

I use ChatGPT 4 to review my FeatureScripts. It often gives me tipps to use better variable names and makes suggestions.

Once I asked it to create a FeatureScript itself to build a simple cylinder. The user should be able to input radius and height. The Code had small errors but everything was there. It added inputs, created a sketch and added the extrude. It also added comments!

Interesting. I just signed up for for ChatGPT Plus, and have just started to play with GPT4. Previously when I tried to use ChatGPT (3.5) and I asked it Featurescript questions, it seemed happy to completely make up syntax which didn't make sense. I don't know what language or API it thought it was talking to, but it was useless. 

Can you share an example of the prompts and dialog you're having with GPT4? 

S1mon

I tried getting it to help me remember where the "show code" command is (it's in the part studio tab right-click menu). Here are the various almost intelligent but not quite right answers from GPT4 (the Github URL is BS or is no longer real, and it missed the part studio tab menu completely):

sebastian_glanzner

@S1mon

I used german when I talked to it so I translated my prompt:

Can you help me to program something? I am supposed to create a "FeatureScript" for Onshape. The program should create a cylinder and the user has to enter the radius and the height.

Of course I can help you with that! FeatureScript is a programming language developed specifically for Onshape. Here is a simple example of a FeatureScript that creates a cylinder where the user can enter radius and height:

Copy the above code into a new FeatureScript document in Onshape. To use the feature, add it to your Part Studio and enter the desired values for Radius and Height. The cylinder will be created accordingly.

sebastian_glanzner

The I told ChatGPT that I corrected the code, posted it and it replied:

Great, I'm glad you adapted the code to your needs and it works now. Your version of the code uses a newer version of FeatureScript (1977 instead of 1.0), and you have adjusted the definition of the coordinate systems and extrusion direction accordingly.

The improved code looks good, and it should now create a cylinder in your Onshape Part Studio based on the user entered values for radius and height. If you have any further questions or need help programming with FeatureScript, don't hesitate to ask.

NeilCooke

When I ask the exact same question it gives me this:

// Define cylinder parameters (radius and height)
annotation { "Name" : "Cylinder Radius", "Filter" : EntityType.EDGE, "CustomControl" : "SPINNER_CONTROL" }
    radius is dimension( length );
annotation { "Name" : "Cylinder Height", "Filter" : EntityType.FACE, "CustomControl" : "SLIDER_CONTROL" }
    height is dimension( length );

// Define cylinder center point
center_point = vector(0, 0, 0);

// Create cylinder
cylinder = cylinder(center_point, radius, height);

// Add cylinder to feature list
opSheetBody(sheetId, id + " Cylinder", cylinder)<span>;</span>

Maybe I should ask in German  

sebastian_glanzner

Yeah, it is really weird. You can ask the same question and get different results each time. You can also click on "regenerate response" and you get a "new" answer 

I tried it again and provided a "Template" as a start:





There are two errors: The plane for the sketch and the opExtrude command is wrong.

robert_stiles

ha, I just tried GPT (being not very good at feature script). Great I thought, and shared some trial code past with my co founder (who is good at feature script) and he just laughed. I clearly will not be able to stop burdening him with my requests just yet. 
GPT is really interesting, but its obviously all about how much information it has to learn from. Last year I asked it to write a planning design and access statement - I was blown away at how "good" the results were, with relevant, local, and accurate planning policy references. These documents are mostly worthless page fill and box ticking exercises, and GPT is evidently very good at reproducing this. In the UK, all our planning documentation is out there in the public realm. 
I then asked GPT to try and write an outline construction specification for a simple build. It was absolutely laughable. Very few architects share this information, its hard to find even half decent examples out there anywhere. Architects silo this information - its not about protecting IP per say, its just there is no reason to put it in the public domain, nor any repo to put it in. 
I think that's what's so interesting about Onshape in the next 5 - 10 years. Because its all in the cloud, there are so many more examples for AI to learn from, both feature script and model building strategies. Power CAD users have been scripting custom utilities for years, but its always been kept in a silo until now. I appreciate that pro users data might not be shared publicly, but Onshape might still have access to it - if a privacy choice is made? Do we want AI to do get better at this? I do! but is that because I don't do it very well so its not my skills at risk!? To be honest I'm not really sure what my tangible skills actually are, so AI will have to identify these before it takes them from me. 

S1mon

@robert_stiles

Availability of training data is huge. Unfortunately if it's not available, it's shocking how much these LLMs will just make up stuff, and still sound confident. 

For all of the Onshape STD source and public Feature Scripts, it would make sense if they were available to some tool to learn from. Internally used stuff would have to only be opt-in. 

sebastian_glanzner

It's beneficial to inform ChatGPT when it makes a mistake, especially with commands like opextrude. You can do this by simply copying the correct opextrude command syntax from the help documentation (https://cad.onshape.com/FsDoc/library.html) and pasting it into your conversation with ChatGPT. This allows the AI to learn from its error and make the necessary corrections.

It would be great to have a custom AI that you can train with Onshape data!

MichaelPascoe

@sebastian_glanzner does it actually add it to its global data or only the data for that conversation?

sebastian_glanzner

@MichaelPascoe
This will only be relevant for the current conversation with ChatGPT