Best Of

I've spent ~15 minutes playing around with it, and here are my first impressions:

I tried asking for a few simple features, but as expected - its not able to produce anything working, even given the lowest hanging fruit. I've naysayed this enough times that the forums don't need another rant from me on the topic.

But I will be nice and evaluate it on the other more reasonable use cases described: documentation and question answering.
I asked the chat a number of easy enough questions, including a few real questions asked on the forum.

The good -

• Responses have citations.
• I tried to see if any data had been stolen to train the model - and nothing obvious comes up.
  
  The bad -
• It invents new featurescript functions that don't exist (or syntax, or doesn't use them correctly, etc)
• It gets answers to questions wrong more often than it is correct.
  • It gets even the simplest questions wrong, like "how do I join two lists"
• The citations don't match the answer. I'm seeing "opBooelan" as the source for joining two lists, and nearly every answer i get.
  

I think there is use and value in having an AI that can answer questions about featurescript. Many questions can be answered by reading the documentation, which can be a bit overwhelming if you don't know the correct words to search. But at this time, your AI would be more frustrating than useful to a new featurescripter.

I'm curious what your plans are for training data to improve your model?