Welcome to the Onshape forum! Ask questions and join in the discussions about everything Onshape.

First time visiting? Here are some places to start:
  1. Looking for a certain topic? Check out the categories filter or use Search (upper right).
  2. Need support? Ask a question to our Community Support category.
  3. Please submit support tickets for bugs but you can request improvements in the Product Feedback category.
  4. Be respectful, on topic and if you see a problem, Flag it.

If you would like to contact our Community Manager personally, feel free to send a private message or an email.

Onshape Benchmark (UNOFFICIAL) - Fill out to see your results!

MichaelPascoeMichaelPascoe Member Posts: 926 PRO
edited February 22 in General

Onshape Benchmark Sheet (UNOFFICIAL)

This is the Unofficial Onshape Benchmark. This was created so that we can compare our systems and see which ones are the very best for Onshape. There are several types of results; the "Open Division" is for everyone to participate in, the "Closed Division" will be strict tests where only one variable is changing at a time in order to obtain pure data. I would love to add more entries to the open division is anyone is willing to participate. 


Your benchmark sheet includes a live chart which is linked to the Open Division records.
If you send me your Benchmark sheet, I will add it to the records!




Results

Up to date results for the Open Division can be found here. This will be updated as more records are posted in this discussion. 
Open Division - Onshape Benchmark - Results
Closed Division - Onshape Benchmark GPU - Results


Special thanks to:

@alnis for providing the assembly studio
@Jason_S for walking me through Onshape technical info
@JD for letting me use those graphics cards for the GPU tests



How to use the Onshape Benchmark Sheet (UNOFFICIAL)


Comments

  • MichaelPascoeMichaelPascoe Member Posts: 926 PRO
    edited March 7
    Let me know if you can think of more user controlled criteria.
    Here are some closed division tests that would be good to do:
    • Graphics card
    • RAM
    • Internet download speed
    • Internet upload speed
    • Ping / latency
    • Processor
    • Screen resolution
  • MichaelPascoeMichaelPascoe Member Posts: 926 PRO
    edited March 1

    New results are available for download speeds. This was a closed division test, maintaining steady upload speed and latency for each test. See the link below for more details:
    Download Speed - Closed Division - Onshape Benchmark - Results

    Edit: The table in the results sheet, below the chart, also shows the AWS-3 server speeds at the time of the test.
  • eric_pestyeric_pesty Member Posts: 437 PRO
    I would think the server load at the time you run the test might have more of an impact than the the local resources (for a given available bandwidth) as all that is done locally is loading a webpage and displaying a bunch of triangles on screen, which should be a small part of the overall process (except for the slowest devices).

    It's interesting to see that there is almost no benefit to more bandwidth beyond 10Mbps

  • MichaelPascoeMichaelPascoe Member Posts: 926 PRO
    edited March 1
    Good point, there are so many variables; I have eliminated as many as I can think of so that only 1 variable is changing at a time. These test are really only for things that the user can control and potentially upgrade or fix.

    The table in the results sheet, below the chart, also shows the AWS-3 server speeds at the time of the test. In that same table, you can click on the link to the data sheet where the benchmark sheet of each of those recordings.

    Upload and latency test coming soon..
  • christian_pettychristian_petty Member Posts: 42 PRO
    Here is my benchmark:
    https://docs.google.com/spreadsheets/d/1R0fZ4pzEVtq6vMES8ey6lCFOfNff0uFTXzOCO1MzNnE/edit?usp=sharing

    I have a 4K screen and if that's of any interest for closed testing.
  • MichaelPascoeMichaelPascoe Member Posts: 926 PRO
    edited March 7
    Thanks for the info @christian_petty
    Your results have been added to the Open Division records.

    Congrats! Your setup is currently the fastest Onshape pc on record.  B)
  • christian_pettychristian_petty Member Posts: 42 PRO
    @MichaelPascoe no problem. I hope someone can topple the laptop at some point.

    Why do copy operations take so long in this test? Is information passed through the client and back to the servers or is that still a time intensive operation server to server?
  • MichaelPascoeMichaelPascoe Member Posts: 926 PRO
    When the document is copied, it forces the server to recalculate a portion of the document. I think.. Regardless, most of the time I am not copying a document, so I didn't emphasize that number too much.

    It did seem like it could be the most repeatable number, if it really does force the server to recalculate everything. I don't fully understand though, because copying a document still doesn't re-calculate some of the features. If it did, it would take much longer. I would like a way to hard refresh the entire tree and every feature. This isn't possible on our end unless we change a value in each feature.
  • nick_papageorge073nick_papageorge073 Member Posts: 200 PRO
    edited April 4
    Here are my results for the lowest end MacBook Air M1 chip. 8 CPU cores, 7 GPU cores, 8GB RAM (Shared).


    For the new M1 Macs, there should probably be a row that states how many cores are in the GPU. All the new M1 GPU's are called the same thing and none of the tests identify how many cores there are.


  • nick_papageorge073nick_papageorge073 Member Posts: 200 PRO
    edited April 4
    Here is the lowest spec M1 Ultra. 20 CPU cores, 48 GPU cores, 64GB RAM (Shared).


  • nick_papageorge073nick_papageorge073 Member Posts: 200 PRO
    edited April 4
    Here is a Win PC I homebuilt in 2020 to run Creo before I switched to OS. 9th gen i7, 5700xt GPU w/8GB RAM, 32GB RAM.



  • MichaelPascoeMichaelPascoe Member Posts: 926 PRO
    Legit! Thanks @nick_papageorge073

    Your results have been added to the Open Division records.
  • nick_papageorge073nick_papageorge073 Member Posts: 200 PRO
    edited April 6
    I have a new system. I tested my dayjob coworker's desktop computers that are used for machine learning. They are custom built Linux machines, 11th gen i7, and a 3080Ti GPU with 12GB GPU RAM. I believe this is the strongest GPU across all the prior entries. However, the results are nowhere near the strongest. I'm confused... Here is the result:


  • MichaelPascoeMichaelPascoe Member Posts: 926 PRO
    edited April 6
    Added. That's a sweet system.

    Keep in mind this test is only looking at document load times. I would imagine having a better card may help with dragging things around in an assembly, "dragability". I'm not sure how to test dragability in a reliable way. Any ideas?

    Perhaps I could record a mouse macro that drags at a consistent speed then monitor the fps. That might work for a closed division run.
  • nick_papageorge073nick_papageorge073 Member Posts: 200 PRO
    Yeah, I guess that's the main question. What is your test procedure actually measuring when we are using OS on a daily basis? I think on my work, I have two bottlenecks that are local. One is the GPU. I see its usage go to 100% and that is when my browser freezes. The second is system RAM. The more I have, the more OS browser windows I can have open, which I do often to see old versions of what I'm modifying, see multiple assemblies, etc. I'm almost never using only one browser window.
  • MichaelPascoeMichaelPascoe Member Posts: 926 PRO
    edited April 20

    Update!

    The Comparison Tool will help you compare any two data sheets from the Open Division records.


  • kevin_o_toole_1kevin_o_toole_1 Onshape Employees, Developers, HDM Posts: 552
    Nice tool!

    I figured I'd throw in a comparison from an Onshape developer's machine. Here's results from our standard issue Macbook Pro, tested from our Seaport office:
    https://docs.google.com/spreadsheets/d/1P12CM2Ih2fT6ya91LhlfPNUemnZllRojamcMVRYsISg/edit#gid=0

    All in all not far from the average! That's some validation in not showing hyper-fitting to our own setup. In theory, once basic needs are met, things like copy and refresh times in Onshape's architecture are going to be mostly independent of client hardware.
  • MichaelPascoeMichaelPascoe Member Posts: 926 PRO
    edited May 2
    Valuable info, thanks @kevin_o_toole_1! Added your sheet to the records.



  • MichaelPascoeMichaelPascoe Member Posts: 926 PRO
    In case anyone was unable to view the google sheet in the comment above, here is a snip of the internet download speed benchmark. Looks like results are all about the same until I throttled the connection to below 10 Mbps.

    Note that this is testing the generic download speed with speedtest.net. AWS speeds, upload, and ping are also listed in the table, but are not the focus of this specific closed division test.

    Download Speed - Closed Division - Onshape Benchmark - Results


Sign In or Register to comment.