Welcome to the Onshape forum! Ask questions and join in the discussions about everything Onshape.

First time visiting? Here are some places to start:
  1. Looking for a certain topic? Check out the categories filter or use Search (upper right).
  2. Need support? Ask a question to our Community Support category.
  3. Please submit support tickets for bugs but you can request improvements in the Product Feedback category.
  4. Be respectful, on topic and if you see a problem, Flag it.

If you would like to contact our Community Manager personally, feel free to send a private message or an email.

Unit Testing Framework?

dave_cowdendave_cowden Member, Developers Posts: 470 ✭✭✭
Alright, I'm a TDD guy, and now that i did my first dixie cup/learning FS file, i'm already bothered by not writing any tests.

Are there any unit testing or test harnesses out there for writing tests for features? I've not really seen any discussion along those lines. It bothers me to have more than 100 lines of production code without any test code....

Best Answer

Answers

  • kevin_o_toole_1kevin_o_toole_1 Onshape Employees, Developers, HDM Posts: 565
    edited January 2016
    We imagine that TDD workflows can use Part Studios as unit tests.

    An FS writer should use an Onshape document as container for everything associated with one feature (or a library of Features). So, a Widget feature type could be defined in one Feature Studio, and it might have a half dozen Part Studios which contain one or several Widgets, testing common cases and edge cases.

    Before FeatureScript is released publicly, we intend to have inter-document references, providing an interface for anyone shared on your Widget's document (or Public) to use your feature type. If you use the workflow above, the test cases have the added benefit of providing excellent word-free documentation for your features. A user can look at how your feature is used in your document and immediately grok its intended purpose.

    Now, this straightforward design pattern does rely on your eyeballs to make sure the edge cases all behave correctly. For more automated tests, you can make a feature type of the form "Create a Widget, make sure these properties are still valid, otherwise throw" (throwing inside a feature will turn that feature red in the Part Studio)

    These tests could be for invariants that should be true any time you make a widget (in which case, you might consider adding said test to the standard feature to alert users of strange behavior). If you want to get more specific, you can even add a specialized test, to be used once in a Part Studio, that asserts very specific things:
    annotation { "Feature Type Name" : "Widget (test 1)" }
    export const widgetWithTests1 = defineFeature(function(context is Context, id is Id, definition is map)
        precondition
        {
            // Precondition copied from widget, so the UI can be the same
        }
        {
            // Make a Widget
            widget(context, id + "create", {
                // parameters
            });
            
            // Assert properties
            const volume = evVolume(context, {
                   "entities" : qEverything(EntityType.BODY)
            });
            const tolerance = 0.000001 * meter^3;
            const expectedVolume = 0.05 * meter^3;
            if (abs(volume - expectedVolume) > tolerance)
                throw regenError(ErrorStringEnum.INVALID_INPUT);
    
            const numberOfEdges is number = size(evaluateQuery(context, qEverything(EntityType.EDGE)));
            if (numberOfEdges != 13)
                throw regenError(ErrorStringEnum.INVALID_INPUT);
            // etc.
        }, { });
    The workflow for this automatic style of testing are admittedly not great right now. It's not very DRY, it makes your document more cluttered, and it's hard to know what metrics are the "right" things to be testing (even if you've been doing this a while!). For the most part, we hope for the time being that you're happy with the workflow of creating canonical use cases and manually checking them. If there are regressions in your features, you can always use the version graph and history to bisect where the behavior changed.

    If your "units" for testing are smaller than the feature level, you can also make a feature wrapper that provides a test case for non-feature functions, which throw if, say, the return value of that function is incorrect. Then you can have a Part Studio with no geometry which simply includes the "Run tests" feature.

    Any errors thrown will show up in the FeatureScript notices panel with a full stack trace.
  • dave_cowdendave_cowden Member, Developers Posts: 470 ✭✭✭
    Hi, thanks for that answer.

    I think for this problem domain, visual verification, will be OK for what I would call 'integration tests'.
    I can't really see a faster way to test in many cases.

    Unit tests, on the other hand, are another thing. I definitely think it should be easier than you have proposed to unit test smaller functions. Those need automated tests with non manual assertions, and there should be a junit-like framework for those.

    For example, lets consider the dome feature I just made. I used the approach you proposed there:the enclosing document had a couple of uses that I visually looked at. The fact that they automatically update when the fs changes makes it pretty effective.

    But, inside my feature,  I have a function that computes the beginning and ending conditions based on the inputs. There is quite a bit if logic there, so I want to write a test that asserts that all of the input permutations result in the boundaries I expect.

    Because so many onshape functions operate using maps, this test is actually pretty easy. And for complex inputs , it is way, way better than doing those tests visually because I can assert all the possibilies without creating a part studio for each, which even in my simple case would be 8 parts.

    Tldr; part studios will work fine for integration testing, but not for unit tests...
  • dave_cowdendave_cowden Member, Developers Posts: 470 ✭✭✭
    I see. Yeah, i will try that out. Its a bit clunky, though.
Sign In or Register to comment.