Can I import .dat files using a feature script?

darren_13

so I have finished my NACA 4 digit aerofoil script and I now want to have the ability to use the vast amount of more comely aerofoils available for .dat download. Am I able to read a .dat file and store the co-ords in an array for aerofoil creation?

Kind regards,
Darren Lynch

ilya_baran

You cannot do this with a .dat file (not sure what the format is) but if you were to convert it to a csv file, you could upload it and access it from FeatureScript as an array.  See: https://cad.onshape.com/FsDoc/imports.html#importing-external-data

kevin_o_toole_1

FeatureScript doesn't access data directly through file names, but you can upload your profiles to a document (where they'll be saved and version controlled on our servers), and import data from those files into a custom FeatureScript feature within that document.

Here's a quick example of a feature that uses multiple CSV files, and allows the user to select among them:
https://cad.onshape.com/documents/d5df23667e8b892627d4b28d/w/cc3ec5cbfc63fa019e1a0874/e/70af96bc26b9d3730960e10d

FeatureScript 422;
import(path : "onshape/std/geometry.fs", version : "422.0");

// Imports data as top-level constants, which can be accessed through e.g. PROFILE_1::BLOB_DATA.csvData
PROFILE_1::import(path : "d6c489e34a7389ebc8c3364a", version : "d89c86b0c14dfd85bee00095");
PROFILE_2::import(path : "5622f19b69d1c079d95e51a0", version : "040bd0eb4f97a1a4fdca14fe");

export enum ProfileType
{
    annotation { "Name" : "Profile One" }
    ONE,
    annotation { "Name" : "Profile Two" }
    TWO
}


annotation { "Feature Type Name" : "Profile" }
export const profile = defineFeature(function(context is Context, id is Id, definition is map)
    precondition
    {
        annotation { "Name" : "Profile type" }
        definition.profileType is ProfileType;
        annotation { "Name" : "Length" }
        isLength(definition.length, LENGTH_BOUNDS);
        
    }
    {
        var csv_array;
        if (definition.profileType == ProfileType.ONE)
            csv_array = PROFILE_1::BLOB_DATA.csvData;
        if (definition.profileType == ProfileType.TWO)
            csv_array = PROFILE_2::BLOB_DATA.csvData;
        
        // Convert raw numbers to real lengths
        var points is array = mapArray(csv_array, function(values) {
            const point is Vector = vector(values) * definition.length;
            println(point);
            return point;
        });
        
        var sketch1 = newSketch(context, id + "sketch1", {
                "sketchPlane" : qCreatedBy(makeId("Front"), EntityType.FACE)
        });
        skFitSpline(sketch1, "spline1", {
                "points" : points
        });
        skSolve(sketch1);
    });

The imports at the top were selected from the "files" section of the import dialog:


As Ilya mentioned, some more documentation is at https://cad.onshape.com/FsDoc/imports.html#importing-external-data, and the forums are always a good place to ask more questions!

kevin_o_toole_1

Actually, now that I think about it... Seems to me like a better ad hoc solution (which doesn't involve the API at all) is to just put all your data into a single CSV file (with 3 rows dedicated to each profile), and then use FeatureScript select the correct rows and map them into vectors. This means you need to generate the combined CSV file in python and have some logic which maps the selected profile to the correct row indices. However, you won't have to generate all of the import statements, nor will you have to upload 1600 different tabs.

mbattistello

Were you planning to import 1600 individual files to your Onshape doc? Seems like a lot.

You might be able to take the individual files and put them in a single file and append a profileId to each one. Then in the featurescript use that profileid to group the outlines and draw the individual profiles. This makes it so you only have a single import you in featurescript.

If you do this then there is only a single import in your featurescript. On some of the uploads you can do an update, right click on the tab and see if update is available, and if this works then a user could just update that individual file an the import should still work.

I tried this and it seems like there is a bug in the .csv file type update. If I update a csv file and then try to use the update I get the error "Cannot update an element with file type "text/csv" with a different file type "application/vnd.ms-excel"." If the file is a text file it seems to work ok. 

ilya_baran

You cannot do this with a .dat file (not sure what the format is) but if you were to convert it to a csv file, you could upload it and access it from FeatureScript as an array.  See: https://cad.onshape.com/FsDoc/imports.html#importing-external-data

darren_13

Ok thank you, Ill give this a go.

darren_13

ok, so now my matlab file passes .csv instead of .dat (the standard for aerofoil profiles) and I want to import as you describe in the link and I come to two difficulties, how do I know the path? and if the user types the profiles name i.e. Aerofoil5.csv how do I make OnShape recognise this as its path? in the future I wish to run this in a loop importing a different profile (all exported from matlab) onto a new plane at say 1 meter intervals. before I go any further is this possible? or will I have to run the script for each profile defining the name?

This script is to be used in a parametric analysis of a wind turbine in SimScale.

Kind regards,
Darren Lynch

kevin_o_toole_1

FeatureScript doesn't access data directly through file names, but you can upload your profiles to a document (where they'll be saved and version controlled on our servers), and import data from those files into a custom FeatureScript feature within that document.

Here's a quick example of a feature that uses multiple CSV files, and allows the user to select among them:
https://cad.onshape.com/documents/d5df23667e8b892627d4b28d/w/cc3ec5cbfc63fa019e1a0874/e/70af96bc26b9d3730960e10d

FeatureScript 422;
import(path : "onshape/std/geometry.fs", version : "422.0");

// Imports data as top-level constants, which can be accessed through e.g. PROFILE_1::BLOB_DATA.csvData
PROFILE_1::import(path : "d6c489e34a7389ebc8c3364a", version : "d89c86b0c14dfd85bee00095");
PROFILE_2::import(path : "5622f19b69d1c079d95e51a0", version : "040bd0eb4f97a1a4fdca14fe");

export enum ProfileType
{
    annotation { "Name" : "Profile One" }
    ONE,
    annotation { "Name" : "Profile Two" }
    TWO
}


annotation { "Feature Type Name" : "Profile" }
export const profile = defineFeature(function(context is Context, id is Id, definition is map)
    precondition
    {
        annotation { "Name" : "Profile type" }
        definition.profileType is ProfileType;
        annotation { "Name" : "Length" }
        isLength(definition.length, LENGTH_BOUNDS);
        
    }
    {
        var csv_array;
        if (definition.profileType == ProfileType.ONE)
            csv_array = PROFILE_1::BLOB_DATA.csvData;
        if (definition.profileType == ProfileType.TWO)
            csv_array = PROFILE_2::BLOB_DATA.csvData;
        
        // Convert raw numbers to real lengths
        var points is array = mapArray(csv_array, function(values) {
            const point is Vector = vector(values) * definition.length;
            println(point);
            return point;
        });
        
        var sketch1 = newSketch(context, id + "sketch1", {
                "sketchPlane" : qCreatedBy(makeId("Front"), EntityType.FACE)
        });
        skFitSpline(sketch1, "spline1", {
                "points" : points
        });
        skSolve(sketch1);
    });

The imports at the top were selected from the "files" section of the import dialog:


As Ilya mentioned, some more documentation is at https://cad.onshape.com/FsDoc/imports.html#importing-external-data, and the forums are always a good place to ask more questions!

darren_13

@kevin_o_toole_1, I'm revisiting this. I am creating a script (python not FS) that pulls, converts and generates the code to past into feature script to have all 1600 aerofoils in a list. I'll be dumping the file with all the CSV but I will need to import them to my script, is there any way I can get the path and version:

PROFILE_1::import(path : "d6c489e34a7389ebc8c3364a", version : "d89c86b0c14dfd85bee00095");

without manually going through importing all 1600 files? Ideally, I would like to determine the path based upon its location within my document? 

The reason I want to do this is because I believe people would rather copy the file, add their own profiles without getting me to update the script. This isn't a problem, however, others then don't benefit from the addition of those profiles. I have contemplated linking this to an app created using your API, however, wouldnt know where to start and for now, feature script is comfortable to use, although I feel I'm gonna have to bit the bullet at some point.

Kind regards,
Darren

kevin_o_toole_1

The "Path" for intraworkspace imports (like the one you pasted) is just the unique id for that tab (AKA element). The preferred way to get these ids is through the API. However, if it's easier, when looking at that tab, you could also get it from end of the URL, which has the format cad.onshape.com/documents/<document id>/w/<workspace id>/e/<element id that you need>, however, can't think of a good automated way of getting these from every tab without the API. This also doesn't save you the step of uploading them all in the first place (which, without the API, would require many, many clicks).

The version for intraworkspace imports will automatically be updated to the latest version in the workspace of whatever you're pointing to. So, for your purposes, you can leave this blank (i.e. import(path: "elementid123123123", version: "") ). When you paste the code and commit it, the latest version will automatically be populated as the version string.

It likely won't surprise you to hear that we would one day like to have the ability for users to specify an new import right in the feature dialog (similar to what happens in derived part), instead of needing to modify the FeatureScript every time. In the meantime, it sounds like your python script could be a good solution.

kevin_o_toole_1

Actually, now that I think about it... Seems to me like a better ad hoc solution (which doesn't involve the API at all) is to just put all your data into a single CSV file (with 3 rows dedicated to each profile), and then use FeatureScript select the correct rows and map them into vectors. This means you need to generate the combined CSV file in python and have some logic which maps the selected profile to the correct row indices. However, you won't have to generate all of the import statements, nor will you have to upload 1600 different tabs.

mbattistello

Were you planning to import 1600 individual files to your Onshape doc? Seems like a lot.

You might be able to take the individual files and put them in a single file and append a profileId to each one. Then in the featurescript use that profileid to group the outlines and draw the individual profiles. This makes it so you only have a single import you in featurescript.

If you do this then there is only a single import in your featurescript. On some of the uploads you can do an update, right click on the tab and see if update is available, and if this works then a user could just update that individual file an the import should still work.

I tried this and it seems like there is a bug in the .csv file type update. If I update a csv file and then try to use the update I get the error "Cannot update an element with file type "text/csv" with a different file type "application/vnd.ms-excel"." If the file is a text file it seems to work ok. 

darren_13

Hi guys thank you for your responses. This seems the way to go, compiling into a single file then uploading as one. Then calling the coordinates that I know corresponds to each profile. Once again thanks for your out of the box thinking

Kind regards,
Darren

darren_13

Hi @kevin_o_toole_1, if im doing this it is going to be just as easy for me to auto write the lookup table with coordinates included. Is this more efficient that trying to pull a set data set from one massive file? Also could I make the lookup table a function in another script and import it? or can I not use functions for prerequisits? 

Thanks in advance,
Darren

darren_13

Hi again, This appears to be working for me thanks for the help. If any one in the future faces this problem I am posting the code I used.

import os

allowed = "0123456789-+. ()"

path = '/Users/darrenlynch/PycharmProjects/OnShape_Import/coord_seligFmt/'
file_names = os.listdir(path)
with open('lookUpTable.txt', 'w') as lookUpTable:
    print('export const aerofoilTable = {\n\t\t"name" : "DataBase",\n\t\t"displayName" : "Data Base",\n\t\t"default" : "UIUC",\n\t\t"entries" : {', file=lookUpTable)
    print('\t\t\t"UIUC": {', file=lookUpTable)
    print('\t\t\t\t"name" : "Aerofoil",\n\t\t\t\t"displayName" : "Aerofoil",\n\t\t\t\t"default" : "YS-930",\n\t\t\t\t"entries" : {', file=lookUpTable)


for file_name in file_names:
    line = 0;
    old_lines = ""

    try:

        with open(path + str(file_name), 'r') as data1:
            for lines in data1:
                if line > 0:
                    lines = ' '.join(lines.split())
                    for char in lines:
                        if char not in allowed:
                            lines = ''
                    if lines == '':
break
                    lines = lines.replace('.......', '0.00000')
                    lines = lines.replace('......', '0.00000')
                    lines = lines.replace('(', '')
                    lines = lines.replace(')', '')
                    lines = lines.replace(' ', ', ', 1)
                    if lines == old_lines:
continue
                    old_lines = lines
                    with open('lookUpTable.txt', 'a') as lookUpTable:
                        print("\t\t\t\t\t\t\tvector(" + lines + "),", file=lookUpTable)
                else:
                    lines = ' '.join(lines.split())
                    with open('lookUpTable.txt', 'a') as lookUpTable:
                        print('\t\t\t\t\t"' + lines + '": [', file=lookUpTable)
                line += 1
    except:
        with open('ErrorLog.txt', 'a') as errors:
            print("\n\nError: Unable to read file: {}\n\n".format(path + str(file_name)), file=errors)
continue

    with open('lookUpTable.txt', 'a') as lookUpTable:
        print('\t\t\t\t\t\t],', file=lookUpTable)

with open('lookUpTable.txt', 'a') as lookUpTable:
    print('\t\t\t}\n\t\t}\n\t}\n};', file=lookUpTable)

Once again many thanks,
Darren